Thomas Thrainer [Tue, 16 Jul 2013 13:50:39 +0000 (15:50 +0200)]
Revert "Disable failing QA test"
This reverts commit
b2525349f0da75c74432f2dd85abbcad56e56fed.
Issue 516 was fixed, so now the QA test can be re-enabled again.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Thomas Thrainer [Tue, 16 Jul 2013 13:47:44 +0000 (15:47 +0200)]
Fix parsing of drbdsetup show output for DRBD 8.4
DRBD 8.4 contains two `disk` entries under the `volume` section. In some
circumstances, the parsing code could wrongly use the empty one which
causes DRBD8Dev._SlowAssemble() to fail. This fixes issue 516.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Christos Stavrakakis [Tue, 16 Jul 2013 08:07:03 +0000 (11:07 +0300)]
Add timestamps to haskell network query fields
Add timestamp fields to the list of available network query fields in
the Haskell code.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>
Dimitris Aragiorgis [Mon, 15 Jul 2013 17:25:03 +0000 (20:25 +0300)]
Fix conflicts check in LUNetworkDisconnect
The function _NetworkConflictCheck expects a list of instances as
last argument, not a list of pairs of instance names and instances.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Mon, 15 Jul 2013 14:30:15 +0000 (16:30 +0200)]
Merge branch 'stable-2.8' into master
* stable-2.8
Change method dispatch in ClientOps to enforce luxi.REQ_ALL
Allow modify_etc_hosts to be changed
Add --modify-etc-hosts option for CLI tools
Add luxiReqQueryNetworks to LuxiOp
Log received message at debug level
Set the correct group for confd
Log RPC errors from inside executeRpcCall
Factor out the logRpcErrors function
Expose bulk parameter for GetJobs in RAPI client
Add git send-email to the chroot
Add tests for CanRead
Only generate node lists with nodes having different names
Cluster verify checks server.pem permissions
Add function for checking file access permissions
Prevent silent failure in case of connection problems
Fix apt-get invocation in chroot_builder
* merged from stable-2.7
Fix RAPI to include missing network fields
Add support for querying network timestamps
In the crontab example, look for the correct binary
Fix wrong numbering in UPGRADE documentation
Conflicts:
devel/build_chroot
lib/client/gnt_cluster.py
Resolved by taking both additions.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Mon, 15 Jul 2013 14:10:58 +0000 (16:10 +0200)]
Merge branch 'stable-2.7' into stable-2.8
* stable-2.7
Fix RAPI to include missing network fields
Add support for querying network timestamps
In the crontab example, look for the correct binary
Fix wrong numbering in UPGRADE documentation
Conflicts:
src/Ganeti/Objects.hs
test/hs/Test/Ganeti/Objects.hs
Resolved by taking both additions.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Klaus Aehlig [Mon, 15 Jul 2013 09:24:28 +0000 (11:24 +0200)]
Change method dispatch in ClientOps to enforce luxi.REQ_ALL
ClientOps' handle_request dispatches on the luxi request received. Change
this to first verify if the request is luxi.REQ_ALL. In this way, we catch
programming errors introducing "secret" luxi requests earlier. This is relevant,
as our other checks for internal consistency compare against luxi.REQ_ALL.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Thu, 11 Jul 2013 14:22:31 +0000 (16:22 +0200)]
Update documentation and NEWS wrt to file storage dir
This patch updates the 'gnt-cluster' man page and the NEWS
file regarding the changes related to the file storage
en/disabling and the configuration of the file storage
directory.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Thu, 11 Jul 2013 07:55:20 +0000 (09:55 +0200)]
Verify file storage path
This patch adds two verification steps to 'gnt-cluster
verify':
- The configured file storage directory is checked against
the allowed file storage directories file.
- We check whether the configured file storage directory
is existing and writable on each node.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Wed, 10 Jul 2013 14:24:25 +0000 (16:24 +0200)]
Prepare verification code for new file path verification
This patch prepares the verification code for adding
a new verification step for the file storage paths:
- It moves a couple of file storage helper functions from
bdev to filestorage (since they make more sense there
and bdev is too big anyway).
- They rename constants and functions related to the
verification step where the allowed file paths are
checked agains forbidden paths to a more expressive name,
because otherwise they would be confused to be related
to the verification of the file storage paths against
the allowed file storage paths.
- Use the cluster objects helper functions to check
if file storage is inabled instead of using the utils
function directly, because it simplifies the code.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Wed, 10 Jul 2013 12:29:19 +0000 (14:29 +0200)]
Remove file storage path from autoconf
This patch removes the option to configure the file
storage path at configure time, because it is now
replaced by a runtime option for gnt-cluster init.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Wed, 10 Jul 2013 12:14:26 +0000 (14:14 +0200)]
QA: Adjust QA for file storage
This improves the QA code wrt to file storage in the
following ways:
- It fixes a FIXME to make the file storage directory
customizable in the QA config. Fallback is the default
value.
- It makes QA derive the enabled storage types from
the enabled disk templates and runs the 'gnt-node
storage tests' only on the enabled ones.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Wed, 10 Jul 2013 12:06:04 +0000 (14:06 +0200)]
storage utils: read file storage dir from cluster config
This patch make the storage utils read the file storage
directory from the cluster config instead of the constants.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 16:41:10 +0000 (18:41 +0200)]
Remove ENABLE_FILE_STORAGE completely
This patch removes the ENABLE_FILE_STORAGE completely
from the remaining files, such as Makefile, configure,
constants, pathutils and objects.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 16:22:16 +0000 (18:22 +0200)]
cluster-merge: remove usage of 'ENABLE_FILE_STORAGE'
This patch adjusts the cluster-merge tool so that it does
no longer check the constant 'ENABLE_FILE_STORAGE' but
the clusters' configurations.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 16:20:04 +0000 (18:20 +0200)]
QA: Adjust test whether file storage is enabled
This patch fixes the node storage QA, which checked whether
file storage is enabled by looking at the constant
'ENABLE_FILE_STORAGE'. QA should rather retrieve this
information from the QA config.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 16:08:24 +0000 (18:08 +0200)]
Opcodes: move storage type checks to cmdlib
So far, there was a static check whether or not a given
storage type of an opcode was enabled. This check is
no longer possibe at this layer, because the information
whether or not file/shared file storage is enabled must
be retrieved from the cluster's configuration. This patch
removes the static check (which was incomplete anyway)
and adds it to the business logic of cmdlib.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 12:57:30 +0000 (14:57 +0200)]
cmdlib: remove usage of ENABLE_FILE_STORAGE
This patch removes all usages of the constant
'ENABLE_FILE_STORAGE' from the cmdlib code and
replaces them by checking the cluster's configuration.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 12:32:53 +0000 (14:32 +0200)]
backend: remove ENABLE_FILE_STORAGE
This patch removes the usage of the ENABLE_FILE_STORAGE
constant in the backend code. To avoid having to pass
it through various RPC calls, we instead move the check
to cmdlib.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 11:09:26 +0000 (13:09 +0200)]
cluster init: check for enabled disk templates
The purpose of this patch is to remove the usage of the
'ENABLE_FILE_STORAGE' constant. To get there, we do some
refactoring, add unit tests and add a FIXME for a forgotten
test regarding the file storage location.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 11:04:03 +0000 (13:04 +0200)]
cluster verify: adjust path verification
This patch adjusts the verification of file storage paths
by checking the list of enabled disk templates instead
of the 'ENABLE_FILE_STORAGE' constant.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 10:52:20 +0000 (12:52 +0200)]
Utility functions to check if a disk template is enabed
This patch adds some helper functions to the config and
storage utils which check whether a disk template is
enabled or not. The functions themselves are quite small
but they will be used quite often and therefore should
have a designated place.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 9 Jul 2013 10:40:59 +0000 (12:40 +0200)]
Add default file storage path to path utils
This patch adds the default storage path to the pathutils
for now. At the end of this patch series, it will no longer
be determined at configure time and therefore be removed
from the configure.ac file.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Fri, 12 Jul 2013 12:15:34 +0000 (12:15 +0000)]
Allow modify_etc_hosts to be changed
The modify_etc_hosts options, enabling the cluster to modify the /etc/hosts
files of nodes, and to keep them in sync, could only be set at cluster init
time.
With this commit it can now be changed through modify_etc_hosts as well.
Fixes Issue 206.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Fri, 12 Jul 2013 09:13:48 +0000 (09:13 +0000)]
Add --modify-etc-hosts option for CLI tools
The option will be used by gnt-cluster modify in the following commit.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Klaus Aehlig [Thu, 11 Jul 2013 08:33:50 +0000 (10:33 +0200)]
Add luxiReqQueryNetworks to LuxiOp
When the QueryNetwork was introduced as a method, apparently
it was forgotten in the Haskell world. Add it here as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Klaus Aehlig [Wed, 10 Jul 2013 14:58:10 +0000 (16:58 +0200)]
Log received message at debug level
At debug level, we can well afford to have a detailed entry
for each message received by a server.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Thu, 11 Jul 2013 12:59:50 +0000 (12:59 +0000)]
Set the correct group for confd
Starting confd as a member of the daemons group allows the RAPI daemon to access
the LUXI socket.
Fixes Issue 477.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Wed, 10 Jul 2013 14:48:05 +0000 (14:48 +0000)]
Log RPC errors from inside executeRpcCall
executeRpcCall is the function to be used for executing RPCs, so it makes sense
to use it as the single point for logging all thte RPC call errors.
Fixes Issue 293.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Wed, 10 Jul 2013 14:39:33 +0000 (14:39 +0000)]
Factor out the logRpcErrors function
This function can be useful to multiple RPC calls, therefore it is moved
to the file containing the common RPC functions.
Also, it is made more generic by changing its signature.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Leon Handreke [Wed, 10 Jul 2013 13:46:04 +0000 (14:46 +0100)]
Expose bulk parameter for GetJobs in RAPI client
This patch exposes the bulk argument of the jobs resource on
the RAPI python wrapper, making it possible to retrieve status
information about all jobs with a single call.
Signed-off-by: Leon Handreke <lhandreke@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Wed, 10 Jul 2013 14:07:41 +0000 (16:07 +0200)]
Add git send-email to the chroot
"git send-email" is used as part of the official workflow but it was not
installed in the chroot.
This patch adds it.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Wed, 10 Jul 2013 11:32:46 +0000 (11:32 +0000)]
Add tests for CanRead
Verify that the CanRead function is actually able to check whether a file has
the proper permissions.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Christos Stavrakakis [Wed, 10 Jul 2013 10:48:21 +0000 (13:48 +0300)]
Fix RAPI to include missing network fields
Fix RAPI interface to include missing network fields. Specifically, fix
instance queries to include the network name for instance NICs. Also fix
network queries to include missing common fields, like uuid and serial.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Christos Stavrakakis [Wed, 10 Jul 2013 10:48:20 +0000 (13:48 +0300)]
Add support for querying network timestamps
Add creation and modified timestamps when creating a new network, and
extend the available query fields for networks with these fields,
namely 'ctime' and 'mtime'.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Thu, 4 Jul 2013 15:32:25 +0000 (17:32 +0200)]
Unittesting for instance_storage._CheckNodesFreeDiskOnVG
This patch refactors the '_CheckNodesFreeDiskOnVG' of
cmdlib.instance_storage to make it at least a little
more testible and provides the unit tests for it.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Thu, 4 Jul 2013 15:30:17 +0000 (17:30 +0200)]
Unit tests for _GetVgInfo
This patch provides unit tests for the backend's
_GetVgInfo function. In order to mock the underlying
backend function, it was necessary to make it an
optional parameter of the function.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Thu, 4 Jul 2013 13:53:30 +0000 (15:53 +0200)]
Fix module functions overridden by mocks
This patch fixes remaining occurrences of module functions
that were overridden by mock functions. They need to be
set back to their original functions after the test case
ends.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Thu, 4 Jul 2013 13:48:32 +0000 (15:48 +0200)]
Unit tests for _GetVgSpindlesInfo
This patch provides unit tests for _GetVgSpindlesInfo.
In order to mock the used function for the 'bdev' module
I made the low-level storage function an optional parameter
of _GetVgSpindlesInfo. I also found out that mocked
module functions stay mocked even after the execution
of the text finished. Fixed this for one occurrence. More
in the next patch.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Thu, 4 Jul 2013 13:22:54 +0000 (15:22 +0200)]
More sanity checks for spindle space reporting
This patch adds a wrapper around the space reporting
for spindles which includes more sanity checks for
the input. Unit tests provided.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Thu, 4 Jul 2013 13:08:42 +0000 (15:08 +0200)]
Tighten sanity checks for '_GetLvmVgSpaceInfo'
This patch tightens the sanity checks for the input of
'_GetLvmVgSpaceInfo' and provides unit tests for it.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Thu, 4 Jul 2013 12:35:49 +0000 (14:35 +0200)]
Factor out check for storage params
This patch factors out the sanity checks for storage
parameters in its own function. Unit tests included.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Spyros Trigazis [Mon, 8 Jul 2013 15:45:02 +0000 (18:45 +0300)]
Add design for mond's CPU load collector
This commit extends monitoring agent's design document, with the
design of a new data collector that will provide CPU load statistics.
It also extends the monitoring agent's design to include:
* a new thread which triggers the collection of data
* the appropriate map to store the collected data
* a new collection function for the data collectors
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Jose A. Lopes [Wed, 10 Jul 2013 10:35:11 +0000 (12:35 +0200)]
Fix single-execution of Haskell tests and shelltests
Add Makefile.am target hs-test-% for single-execution of Haskell
tests.
Add Makefile.am target hs-shell-% for single-execution of Haskell
shelltests.
Update documentation on running individual tests.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Tue, 9 Jul 2013 15:05:48 +0000 (17:05 +0200)]
Only generate node lists with nodes having different names
genNodeList is used in testing to generate list of nodes
that could come up in cluster configurations. Since names
are used to assign indices to nodes, they have to be unique;
this is also the case in all real clusters.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Iustin Pop <iusty@k1024.org>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Wed, 10 Jul 2013 09:04:40 +0000 (11:04 +0200)]
Add missing newline to the NEWS file
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Tue, 9 Jul 2013 16:55:02 +0000 (16:55 +0000)]
ConfD-related NEWS file update
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Tue, 9 Jul 2013 16:49:19 +0000 (16:49 +0000)]
Add monitoring daemon updates to the NEWS file
The "inst-status-xen", "diskstats" and "lv" data collectors have been added.
The daemon is now running as root.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Tue, 9 Jul 2013 16:05:00 +0000 (18:05 +0200)]
Cluster verify checks server.pem permissions
Currently, ConfD must be able to access server.pem (though this is likely to
change in the future). If this is not true, all sorts of weird things happen,
such as "gnt-node list" printing lots of question marks instead of actual
data, with no meaningful error.
This patch adds a check for the proper access permission to
"gnt-cluster verify".
Fixes Issue 518.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Tue, 9 Jul 2013 16:02:07 +0000 (18:02 +0200)]
Add function for checking file access permissions
The CanRead function checks whether a user of the local machine (specified
by name) can access a given file.
IsUserInGroup is a helper function for CanRead, but might also be used
independently, so its name does not begin with an underscore.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Sebastian Gebhard [Tue, 9 Jul 2013 14:18:20 +0000 (16:18 +0200)]
More details on vlan support in openvswitch
This design doc give a more detailed design about the implementation
of openvswitch in regareds of VLANs.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Tue, 9 Jul 2013 13:06:36 +0000 (15:06 +0200)]
Prevent silent failure in case of connection problems
While running "gnt-node list", if a query to ConfD fails (especially
because of permission problems) it used to just fail silently, with gnt-node
showing question marks instead of data.
With this patch, ConfD records the error in its log file, together with a
message giving an indication of the reason.
Fixes Issue 517.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Tue, 9 Jul 2013 12:31:19 +0000 (14:31 +0200)]
Fix type error in the call of _NetworkConflictCheck
The function _NetworkConflictCheck expects a list of instances as
last argument, not a list of pairs of instance names and instances.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Tue, 9 Jul 2013 08:40:52 +0000 (10:40 +0200)]
In the crontab example, look for the correct binary
When ganeti-master-cleaner was merged back into ganeti-cleaner in
46118ed2,
the binary to test for should have also been updated. Do this now.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Mon, 8 Jul 2013 06:57:13 +0000 (06:57 +0000)]
Fix apt-get invocation in chroot_builder
Use the macro containing all the required parameters, instead of just using
"apt-get install".
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Mon, 8 Jul 2013 08:15:26 +0000 (08:15 +0000)]
Fix wrong numbering in UPGRADE documentation
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Fri, 5 Jul 2013 10:46:14 +0000 (12:46 +0200)]
Merge branch 'stable-2.8' into master
* stable-2.8:
Also remove prop_IterateAlloc_sane from test list
Fix documentation for prop_Alloc_sane
Remove IterateAllocSane test
Clean up work around for host name filtering
Do not handle host queries special
Fix error in installation instructions
Add documentation for harep
Fix typo in ganeti-2.8 design doc
Support big-step shrinking in tiered allocation
* merged from stable-2.7:
Release version 2.7.0
Fix "instance replace" typo in admin.rst
Fix typo in the install guide
Disable pylint E1101 on hypothetical import
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 5 Jul 2013 08:44:06 +0000 (10:44 +0200)]
Merge branch 'stable-2.7' into stable-2.8
* stable-2.7:
Release version 2.7.0
Fix "instance replace" typo in admin.rst
Fix typo in the install guide
Disable pylint E1101 on hypothetical import
Conflicts:
NEWS: trivial
configure.ac: trivial
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Wed, 3 Jul 2013 12:38:58 +0000 (12:38 +0000)]
Add documentation for the LV collector
Documentation added in the mon-collector man page.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Tue, 2 Jul 2013 08:24:14 +0000 (08:24 +0000)]
Include "instance" information in LV data collector
This commit enables the logical volume data collector to get information about
the instances and to link it to the information about logical volumes.
The list of parameters accepted by the collector is expanded to allow proper
testing and connections to Confd in non-standard locations.
The shelltest test is updated accordingly.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Wed, 3 Jul 2013 09:36:02 +0000 (09:36 +0000)]
Add "includeLogicalId" function for Disks
This function checks whether a disk contains a given Lvm logical ID,
directly or through its children.
Unit tests are added as well.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Tue, 2 Jul 2013 19:08:38 +0000 (19:08 +0000)]
Add option for loading serialized instances
Monitoring CLI tools might have to load serialized lists of instances
(mainly for testing reasons). This patch adds an option to allow that.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Tue, 2 Jul 2013 11:25:19 +0000 (11:25 +0000)]
Factor out lv info gathering function
The buildJsonReport function will soon have to perform the coupling of
instance data with LVInfo data. In preparation for that, in order to make
it more readable, the instructions for obtaining LVInfos are factored out
in a separate function.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Tue, 2 Jul 2013 09:24:26 +0000 (11:24 +0200)]
Add "instance" field to LVInfo
Extend the LVInfo data structure with the field for storing the name of
the instance it is paired with.
Update the tests accordingly.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Tue, 2 Jul 2013 08:41:32 +0000 (10:41 +0200)]
Better specify LVInfo "instance" field
Specify that it is serialized as "null" if empty.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Wed, 26 Jun 2013 08:02:13 +0000 (10:02 +0200)]
Factor out the getInstances function
The getInstances function can be useful in general, but is defined inside
the InstStatus data collector. This commit takes it out and adds it to
a proper (newly created) library.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Thu, 4 Jul 2013 16:43:49 +0000 (18:43 +0200)]
Also remove prop_IterateAlloc_sane from test list
In f4d1bb7 that test was removed, but forgotten to remove it
from the list of tests to be executed. Fix that.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Thu, 4 Jul 2013 15:39:54 +0000 (17:39 +0200)]
Disable failing QA test
See issue 516 for an explanation of the problem, and why it was required
to disable this test.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Thu, 4 Jul 2013 13:02:21 +0000 (15:02 +0200)]
Fix documentation for prop_Alloc_sane
As discussed in the last commit, placing a new instance on the cluster
can lead to a cluster that can be improved by moving previously added
instances. For an empty cluster, however, there are no previous
instances. So add this to the test description to make obvious why
this test tests for a valid property.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Thu, 4 Jul 2013 12:54:18 +0000 (14:54 +0200)]
Remove IterateAllocSane test
The test is testing for a property that just isn't true. Iterated
allocation greedily place one instance at a time taking the locally
most balanced solution. Then it is tested whether the resulting global
allocation can be improved.
To see that this assumption does not hold, consider placing 3
identical instances on 3 nodes. The most balanced allocation of all 3
instances would be that each node is primary and secondary for one
instance. Now let's see what iterative allocation does.
Up to symmetry, the placing of the first instance is unique. Any
placement of the second instance that keeps the way to the global
optimum open would be one node being primary and secondary for one
instance each, one node being only secondary for instance, and one
node being only primary for one instance. An alternative allocation
would be to place the instances on two different nodes as primaries
and using the third node as shared secondary.
For cpu (2 nodes with 1 cpu, 1 with none), free memory (on two nodes
all minus 1 unit, on one node all), and disk (2 nodes with 1 disk, 1
with 2) there is no difference between these allocations. For reserved
memory, there's a difference in the values. In the first case, on two nodes,
there's one unit reserved and nothing on the third. In the second
case, on two nodes, there is nothing reserved, while on the third
node, there still is only one unit reserved, as the two instances have
different primaries. Nevertheless, the standard deviations of
0,0,1 and 1,1,0 are both sqrt(5/9); so, in everything that contributes
to the balancedness metric these allocations are equal. Therefore, it is
locally correct to chose the wrong allocation.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Guido Trotter [Thu, 4 Jul 2013 11:35:02 +0000 (13:35 +0200)]
Release version 2.7.0
We don't have notice of anything blocking for 2.7, and it's been in
release candidate state long enough. Any future problems can be
addressed as bugfixes.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Wed, 3 Jul 2013 11:26:45 +0000 (13:26 +0200)]
Disable enabled-template-test for exclusive storage
This patch disables the tests regarding enabled disk
templates for exclusive storage, because the semantics
there are different and not covered in the test.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Wed, 3 Jul 2013 11:04:38 +0000 (13:04 +0200)]
Fix propagation of storage parameters to/from backend
This patch fixes two problems with the storage reporting
that showed up in the QA for exclusive storage:
- The processing of storage space information for instance
operations wrongly assumed that the volume group's
storage information is always the first in the list.
- The storage parameter 'exclusive storage' was not
correctly extracted from the list of storage parameters.
- There was a bug in the preparation of storage unit for
the node info call in the iallocator. The exclusive
storage flag was not set for spindles and the format
of the storage parameters for LVM vgs was a boolean
and not a list.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Wed, 3 Jul 2013 09:09:04 +0000 (11:09 +0200)]
Fix typo in iallocator
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Dimitris Aragiorgis [Wed, 3 Jul 2013 11:49:37 +0000 (14:49 +0300)]
Fix gnt-network info
QueryNetworks() returns the uuids of the instances that are
inside a network. Since QueryInstances() uses names and not uuids
for search, we create a uuid2name dict to get each instance name
and then retrieve its nics and networks.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 2 Jul 2013 16:36:27 +0000 (18:36 +0200)]
Extraction of storage info by type
There was a bug in the node queries. It was assumed that
the returned storage space information was in a particular
order. With the changes in the storage reporting, this
order is not that reliable anymore, in particular, the
assumption that the spindle's space reporting is always
the second entry is no longer valid.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 2 Jul 2013 16:20:12 +0000 (18:20 +0200)]
Improve handling of storage info
This patch improves the processing of storage information
as result of the 'node_info' call in the following way:
- It removes the override for LVM storage. This was needed
in MakeLegacyNode info to be compatible to the
iallocator, which no longer uses it.
- When no storage information is available (for example
when only disk templates are enabled, for which no
storage reporting exists), then this is handled more
gracefully.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Tue, 2 Jul 2013 09:05:36 +0000 (11:05 +0200)]
Mention default disk template in NEWS file
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Tue, 2 Jul 2013 08:49:39 +0000 (10:49 +0200)]
Add default disk template selection to man pages
This documents the newly implemented behavior of picking
the disk template from the list of enabled disk templates
on instance creation.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Tue, 2 Jul 2013 08:43:25 +0000 (10:43 +0200)]
Mention alternative template selection in design doc
This is a small adjustment of the design doc
'design-storagetypes.rst' with respect to the selection
of the default disk template.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Mon, 1 Jul 2013 15:21:53 +0000 (17:21 +0200)]
Let instance creation fall back to default disk template
So far, the '-t' option of 'gnt-instance add' is mandatory.
This patch makes it optional. In case, no disk template is
specified, the first of the list of enabled_disk_templates
is used.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Mon, 1 Jul 2013 15:09:33 +0000 (17:09 +0200)]
LUInstanceCreate: factor out disk checks
The 'CheckArguments' method of 'LUInstanceCreate' is way
too long and should be broken up in smaller pieces. This
patch starts with that and factors out the disk checks.
No functional changes whatsoever.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Tue, 2 Jul 2013 15:10:08 +0000 (17:10 +0200)]
Fix "instance replace" typo in admin.rst
gnt-instance `replace` should read `replace-disks`
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Mon, 1 Jul 2013 15:03:46 +0000 (17:03 +0200)]
Clean up work around for host name filtering
These functions simply served as a work around to express
host name matching by regular expressions, instead of using
correct equality filter on host names that provides
the correct matching already.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Mon, 1 Jul 2013 14:49:07 +0000 (16:49 +0200)]
Do not handle host queries special
As, since
91c1a265, the equality used for host names already
is based on matching, there is no need to use a special function
for this any more.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Thomas Thrainer [Tue, 2 Jul 2013 13:47:26 +0000 (15:47 +0200)]
Document the 'protocol' disk parameter
Document the new disk parameter 'protocol' which is used to set the DRBD
replication protocol.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Tue, 2 Jul 2013 11:56:03 +0000 (13:56 +0200)]
Make DRBD replication protocol configurable
Add a new disk parameter which allows to choose the DRBD replication
protocol. The protocol is fixed to "C" during dual primary mode though.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Tue, 2 Jul 2013 14:46:39 +0000 (16:46 +0200)]
Revert "QA: Make Query tests test for '?' values"
This reverts commit
bfbef99b9f056638411aa894e300aa095132e931. The problem is that
in some cases it makes sense for '?' to be displayed. This
is the case for storage reporting on clusters that have only
the 'diskless' disk template enabled. It would be nicer,
if a '-' would be displayed in those cases, but this is a
bigger change in the haskell query infrastructure. Thus,
I am reverting this for now and send a follow up patch in
the next days.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Tue, 2 Jul 2013 07:00:59 +0000 (09:00 +0200)]
Add QA test for verify-disks with broken DRBD
gnt-cluster verify-disks supports automatic activation of broken DRBD
disks. This new QA test verifies that it actually finds instances with
broken DRBD disks and activates the disks of those instances.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Mon, 1 Jul 2013 13:42:40 +0000 (15:42 +0200)]
Check DRBD status on verify-disks
Enhance `gnt-cluster verify-disks` such that it checks the DRBD status
of all disks. If the status of at least one disk of an instance is
either StandAlone or Diskless, the instance disks are activated
automatically.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Mon, 1 Jul 2013 06:08:19 +0000 (08:08 +0200)]
Make names more descriptive
LUGroupVerifyDisks had some not very descriptive names which made it
hard to read and understand the code. This patch renames some methods
and variables to make it more readable.
Also, the return type of MapInstanceLvsToNodes was changed to contain
Instance objects instead of instance names. This enables easier
processing of the result.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Tue, 2 Jul 2013 11:18:51 +0000 (11:18 +0000)]
Fix typo in the install guide
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Helga Velroyen [Mon, 1 Jul 2013 10:57:31 +0000 (12:57 +0200)]
QA: Make Query tests test for '?' values
When something goes wrong in the Haskell implementation of
queries, the 'gnt-* list' command does not actually fail,
but it just displays '?' as a sign of failure. We so far
do not test for that, which makes errors in the haskell
queries go undetected. This patch fixes that.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Thu, 27 Jun 2013 11:37:28 +0000 (13:37 +0200)]
Turn 'exclusive_storage' into storage parameter (hs)
This is the haskell implementation of my patch "Extend RPC call
'node_info' by storage parameters". It turns the 'exclusive
storage' flag into a storage parameter of the LVM storage types.
Besides that, this patch moves some types into the Types.hs.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Wed, 26 Jun 2013 10:43:06 +0000 (12:43 +0200)]
noded.py: remove fallback of 'node_info'
In order to change backend and frontend code of the RPC
call 'node_info' in separate patches, we included a
fallback to the old implementation. Since all RPC callers
are adjusted, this can be removed now.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Tue, 25 Jun 2013 14:51:09 +0000 (16:51 +0200)]
Extend RPC call 'node_info' by storage parameters
This patch extends the actual RPC call to accept storage
parameters with each storage units. It adjusts all code
which performs this code by integrating the 'exclusive
storage' flag into the lvm storage unit.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Tue, 25 Jun 2013 14:24:26 +0000 (16:24 +0200)]
Add general storage parameters to node info call
As described in the design doc about storage types,
we plan to generalize the RPC call "node info" wrt to
storage types. This patch extends the call by accepting
a list of storage units, that is not only identified
by storage type and identifier, but enhanced by additional
parameters. So far, only LVM storage has an additional
parameter, which is the 'exclusive storage' flag. This
patch enhances the backend code of the node info call
to be able to handle this change.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Tue, 25 Jun 2013 16:29:11 +0000 (18:29 +0200)]
backend unit tests: remove unused imports and variables
This patch removes some unused imports and an unused
variable of the backend unit test. No functional changes
otherwise.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Michele Tartara [Mon, 1 Jul 2013 16:25:54 +0000 (16:25 +0000)]
Fix error in installation instructions
Remove erroneous trailing "\"
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Mon, 1 Jul 2013 12:50:03 +0000 (14:50 +0200)]
Prevent LV parser compile error
The LV parser is not compiled correctly by more recent versions of GHC
because of more strict checks.
lvCommand is a surely non-empty list, but the compiler still refuses it
asking for explicitly management of the empty list case.
Instead of managing that case (that would be meaningless), the lvCommand
is split into lvCommand and lvParams, that can be used independently.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>