Fix tests regarding DISK_LD_DEFAULTS
This fixes two remaining issues regarding the renamingof DISK_LD_DEFAULTS:
- The keys of the map DISK_LD_DEFAULTS cannot be comparedto DISK_TEMPLATES right away, because there are nodefaults for the diskless template. This patch adjusts...
Fixing renaming of DISK_LD_DEFAULTS
This fixes the somewhat over-eager renaming ofDISK_LD_DEFAULTS to DISK_DT_DEFAULTS, which clasheswith another occurence of DISK_DT_DEFAULTS. Thispatch just undoes the renaming and additionally fixesa test regarding shared file storage....
Replace LD_* constants with DT_* constants
LD_* constants are basically like DT_* constants, exceptfor that both file and shared file were mapped to file.In order to not having to maintain three slightly differentsets of disk-related constants (DT, LD and ST), we merge...
Merge branch 'stable-2.8' into stable-2.9
Add unit test for GetLinuxNodeInfo
The function was not unit tested previously.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
ClusterSetParams: move vg-name checks from to CheckPrereq
This fixes a bug in the logic of 'gnt-cluster modify'.Some checks that should better be done in 'CheckPrereq'were actually done in 'Exec'. This lead to a strangebehavior in case the execution failed, because an...
Move Ipolicy utility function to cmdlib/common.py
Since the check of consistency between an ipolicy andthe list of enabled disk templates will not only beneeded on cluster modification, but also on groupmodification, we move the respective function and its...
bootstrap: restrict ipolicy to enabled disk templates
With this patch, on cluster creation, the initial instancepolicy's list of allowed disk templates will be modifiedin a way that it does not contain any disk templates whichare not enabled cluster-wise....
gnt-cluster modify: ipolicy vs enabled disk templates
This patch adds a check to LUClusterSetParams whichensures that the list of allowed disk templates is asubset of the list of cluster-wide enabled disktemplates. Unit tests are included.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Merge branch 'stable-2.7' into stable-2.8
Support DSA SSH keys in bootstrap
As outlined in issue 338, Ganeti failed to initialize a cluster if noRSA SSH key is present on the master node. This patch extends Ganetissupport to DSA keys, so clusters with only DSA keys are possible now.
This fixes issue 338....
ganeti.backend_unittest: chmod restricted cmd dir
TestVerifyRestrictedCmdDirectory.testNormal implicitly relies on the currentumask to check the behaviour of backend._VerifyRestrictedCmdDirectory. However,when run under a more relaxed umask (as in the case of the Debian buildd's...
Add cleanup parameter to instance failover
Most of the code is shared with instance migrate, so we actually only needto add the parameter and pass its value along the the common code.
Also, tests and harep are updated to support the right set of options to...
gnt-cluster modify --shared-file-storage-dir
This patch introduces to 'gnt-cluster modify' the option'--shared-file-storage-dir' to change the default directoryfor instances using shared file storage at cluster runtime.
Remove obsolete autoconf variable from remaining files
This removes the obsolete autoconf variable'ENABLE_SHARED_FILE_STORAGE' from all remaining files.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Fix another missing renaming
This fixes a (due to a merge) slipped occurence ofVALID_STORAGE_TYPES which was renamed to STORAGE_TYPES.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Rename VALID_STORAGE_TYPES to STORAGE_TYPES
For consistency, this patch renames 'VALID_STORAGE_TYPES'to 'STORAGE_TYPES', because the set of valid disktemplates is also just names 'DISK_TEMPLATES'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Unit tests for constants consistency
This patch adds some unit tests to ensure the consistencyof various constants regarding disk templates.
Move space reporting constant to constants.py
This patch moves the constant which is used to determinewhether a storage type provides storage space reportingfrom the utils package to the constants. This way, wecan also use it in haskell and it fits there semantically...
gnt-cluster {init, modify} --file-storage-dir
This patch implements consistent usage and behavior ofthe --file-storage-dir option in 'gnt-cluster init'and 'gnt-cluster modify'. It includes a bunch of unittests as well.
Additionally, it enables the previously written unit...
Conflicts: src/Ganeti/Utils.hs (trivial)trivial
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix permission problem related to Issue 477
Commit 91525dee856951ace940c78b6254a1c7344b4803 fixed Issue 477 but broke"gnt-cluster info".
This commit offers a solution to both problems, by changing the permissionof the socket instead of changing the permission the confd process is run...
Rename queryd to luxid
As queryd will, in the future, handle all LUXI request, queue jobs andmost likely perform various other tasks, it is renamed to luxid already.This will safe some headache when upgrading Ganeti installations, as wedon't have to deal with a daemon rename....
Move bdev unit tests to filestorage unit tests
Some code was moved from bdev to file storage inprevious patches, but the related unit tests wereaccidentally left in bdev. Moving and adjustingthem now.
Fix iallocator unit tests
This fixes the unit test for the iallocator. The fixis necessary, because a renamed functions was not properlyrenamed in the tests as well.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Document the rapi client not to have a QueryNetworks method
While there is a luxi query QueryNetworks, the rapi client does notprovide such a method. Document this by declaring it in theKNOWN_UNUSED_LUXI calls.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add queryd daemon (split from confd)
queryd is added as a new daemon which handles configuration queries overLUXI. This functionality was removed from confd, which now only queriesover the network.
The queryd user is added to the master group such that it can access...
Add unit test for drbdsetup show parsing
This unit test checks for proper parsing of the output from `drbdsetupshow` when re-attaching the local disk. In this case, disk parametersare not (yet) set, so the corresponding dict is empty.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Merge branch 'stable-2.8' into master
Conflicts:...
Verify file storage path
This patch adds two verification steps to 'gnt-clusterverify':- The configured file storage directory is checked against the allowed file storage directories file.- We check whether the configured file storage directory is existing and writable on each node....
Remove file storage path from autoconf
This patch removes the option to configure the filestorage path at configure time, because it is nowreplaced by a runtime option for gnt-cluster init.
storage utils: read file storage dir from cluster config
This patch make the storage utils read the file storagedirectory from the cluster config instead of the constants.
cmdlib: remove usage of ENABLE_FILE_STORAGE
This patch removes all usages of the constant'ENABLE_FILE_STORAGE' from the cmdlib code andreplaces them by checking the cluster's configuration.
cluster init: check for enabled disk templates
The purpose of this patch is to remove the usage of the'ENABLE_FILE_STORAGE' constant. To get there, we do somerefactoring, add unit tests and add a FIXME for a forgottentest regarding the file storage location....
Allow modify_etc_hosts to be changed
The modify_etc_hosts options, enabling the cluster to modify the /etc/hostsfiles of nodes, and to keep them in sync, could only be set at cluster inittime.
With this commit it can now be changed through modify_etc_hosts as well....
Add luxiReqQueryNetworks to LuxiOp
When the QueryNetwork was introduced as a method, apparentlyit was forgotten in the Haskell world. Add it here as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Expose bulk parameter for GetJobs in RAPI client
This patch exposes the bulk argument of the jobs resource onthe RAPI python wrapper, making it possible to retrieve statusinformation about all jobs with a single call.
Signed-off-by: Leon Handreke <lhandreke@google.com>...
Add tests for CanRead
Verify that the CanRead function is actually able to check whether a file hasthe proper permissions.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add support for querying network timestamps
Add creation and modified timestamps when creating a new network, andextend the available query fields for networks with these fields,namely 'ctime' and 'mtime'.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>...
Unittesting for instance_storage._CheckNodesFreeDiskOnVG
This patch refactors the '_CheckNodesFreeDiskOnVG' ofcmdlib.instance_storage to make it at least a littlemore testible and provides the unit tests for it.
Unit tests for _GetVgInfo
This patch provides unit tests for the backend's_GetVgInfo function. In order to mock the underlyingbackend function, it was necessary to make it anoptional parameter of the function.
Fix module functions overridden by mocks
This patch fixes remaining occurrences of module functionsthat were overridden by mock functions. They need to beset back to their original functions after the test caseends.
Unit tests for _GetVgSpindlesInfo
This patch provides unit tests for _GetVgSpindlesInfo.In order to mock the used function for the 'bdev' moduleI made the low-level storage function an optional parameterof _GetVgSpindlesInfo. I also found out that mocked...
More sanity checks for spindle space reporting
This patch adds a wrapper around the space reportingfor spindles which includes more sanity checks forthe input. Unit tests provided.
Tighten sanity checks for '_GetLvmVgSpaceInfo'
This patch tightens the sanity checks for the input of'_GetLvmVgSpaceInfo' and provides unit tests for it.
Factor out check for storage params
This patch factors out the sanity checks for storageparameters in its own function. Unit tests included.
Fix single-execution of Haskell tests and shelltests
Add Makefile.am target hs-test-% for single-execution of Haskelltests.
Add Makefile.am target hs-shell-% for single-execution of Haskellshelltests.
Update documentation on running individual tests....
Only generate node lists with nodes having different names
genNodeList is used in testing to generate list of nodesthat could come up in cluster configurations. Since namesare used to assign indices to nodes, they have to be unique;this is also the case in all real clusters....
Include "instance" information in LV data collector
This commit enables the logical volume data collector to get information aboutthe instances and to link it to the information about logical volumes.
The list of parameters accepted by the collector is expanded to allow proper...
Add "includeLogicalId" function for Disks
This function checks whether a disk contains a given Lvm logical ID,directly or through its children.
Unit tests are added as well.
Add "instance" field to LVInfo
Extend the LVInfo data structure with the field for storing the name ofthe instance it is paired with.
Update the tests accordingly.
Also remove prop_IterateAlloc_sane from test list
In f4d1bb7 that test was removed, but forgotten to remove itfrom the list of tests to be executed. Fix that.
Fix documentation for prop_Alloc_sane
As discussed in the last commit, placing a new instance on the clustercan lead to a cluster that can be improved by moving previously addedinstances. For an empty cluster, however, there are no previousinstances. So add this to the test description to make obvious why...
Remove IterateAllocSane test
The test is testing for a property that just isn't true. Iteratedallocation greedily place one instance at a time taking the locallymost balanced solution. Then it is tested whether the resulting globalallocation can be improved....
Improve handling of storage info
This patch improves the processing of storage informationas result of the 'node_info' call in the following way:- It removes the override for LVM storage. This was needed in MakeLegacyNode info to be compatible to the...
Turn 'exclusive_storage' into storage parameter (hs)
This is the haskell implementation of my patch "Extend RPC call'node_info' by storage parameters". It turns the 'exclusivestorage' flag into a storage parameter of the LVM storage types.Besides that, this patch moves some types into the Types.hs....
Add general storage parameters to node info call
As described in the design doc about storage types,we plan to generalize the RPC call "node info" wrt tostorage types. This patch extends the call by acceptinga list of storage units, that is not only identified...
backend unit tests: remove unused imports and variables
This patch removes some unused imports and an unusedvariable of the backend unit test. No functional changesotherwise.
Add htools shell test for VCPU handling
This tests that VCPUs used by both instances and node OS are correctlyaccounted in allocation.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Load CPUs used by the node OS in htools
A new field is added to the Node type, and it's used to initialize the usedCPUs field.
The signature of Node.create has been split among lines to match theparameter list.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Export CPUs used by the node OS
They are exported through the LUXI, RAPI, and IAllocator interfaces.
Hypervisors report CPUs used by the node OS
The XenHypervisor's "dom0_cpus" parameter (introduced in 1d60fec6 but neverused) is renamed to "cpu_dom0" for consistency with other parameternames. The analogous information is returned for the other hypervisors....
Add tests for the LV data collector
This commit adds a few shelltests for the logical volume data collector.
Also, it performs some cleanup of the comments already present in the file:given that the number of the tests should be changed manually, and this is...
Add tests for LVs parser
Add unit tests and Quickcheck tests for the LV parser.
Also, fix the alphabetic order of some imports in htest.hs.
Add function testing parsers with QuickCheck
Parsing a string automatically generated by QuickCheck can be useful in manycases. A function for doing this is added to the common test functions.
Add function for generating arbitrary UUID
Add a function for generating arbitrary UUIDs to be used for QuickCheck tests.
Add test for the --full-evacuation option
In this example cluster, there are four pairs of nodes, each hostingtwo instances. Each node has disk capacity for an additional instance.Therefore, for normal online rolling reboots, there are two node groups,...
iallocator: storage processing accepts space_info structure
This is a small refactoring to make the processing ofthe storage space information not accept a node inforesult, but the more specific space info structure instead.
iallocator: extract storage info directly from node info
So far, the iallocator made a node info RPC call, convertedthe result into a legacy node info format and then extractedthe storage information from that. This patch removes themiddle step of this procedure, skipping the legacy node info...
iallocator: computation of storage data
This patch extracts the processing of the storage-relateddata from the RPC call 'node_info' in a separate functionto increase testability. In addition, this patch containsunit tests for this function.
rpc.py: override default storage type in node info
This patch changes the semantics of dealing with the resultof the RPC call "node_info" when converting them into aLegacyNodeInfo. A legacy node info, can only hold storageinformation about one storage type, which so far was always...
Index instances by their UUID
No longer index instances by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the...
Fix testGetHvDefaults testcase
The introduction of the migration_mode parameter to the fake hypervisorbroke the testGetHvDefaults. This patch fixes this by taking the defaultparameters of the fake hypervisor into account.
Version bump to 2.9.0~alpha1
Bump the version number to 2.9.0~alpha1.
This also requires updating version numbers in cfgupgrade tool.
Furthermore, a data file for test representing a 2.8 configuration had theversion number set to 2.7 in order to prevent tests failing while the current...
Merge branch 'stable-2.8' into 'master'
Rename directory 'Block' to 'Storage'
This patch renames the 'Block' directory to 'Storage' inthe Haskell code base. The same rename was done in thepython code base earlier this quarter. We generalize thename, because we needed a place for general storage...
Add test for shrink heuristics over different resources
In this example, memory prevents adding more than one instance onthe first node. Yet, on the other nodes, 2 instances can be placedeach, if disks are shrunk appropriately. This, however, requiresto courageous decision to shrink disks next, even though more nodes...
Add tests for check-news
Unit tests verifying the behaviour of the check-news script are added.
This required adding a new make target, "autotools-check" directly called from"commit-check", because the usual tests are executed in the build directory,whereas these, being relative to the autotools, cannot be executed there because...
Refactor NodeInfo RPC regarding storage reporting
The NodeInfo RPC call is refactored to handle now more thanjust storage reporting for volume groups.
Since NodeInfo now returns storage space information notnecessarily for volume groups, but also for other storage...
Utility functions for storage types
Handling various storage types for the free space reportingrequires some utility functions. They will be invoked inlater patches. Unit tests are included.
Unit tests for ApplyStorageSpaceFunction
So far, the function 'ApplyStorageSpaceFunction' was notcovered by unit tests. Before the integration of spacereporting for file storage, we add those missing tests.
Renamining space reporting for file storage
Before being integrated, the function for space reportingfor file storage is renamed to a more expressive name.
Conflicts: (trival, take union of added files/tests) Makefile.am test/hs/shelltests/htools-hspace.test...
Add a test for hspace to respect instance policy
In this example, the instance policy for disks prevents allocationof more than one instance per node. The test verifies that tieredallocation still respects this and doesn't attempt to place policyviolating instances on the cluster....
Make shrinkByType aware of individual disks
When shrinking an instances, you can't just get smaller disk footprintwhile leaving the individual disks as they are. Make the shrinkheuristic aware of that fact, and decrease all individual disks aswell. Fixes issue 484....
Convert node indices to UUIDs in configuration
cfgupgrade replaces the occurences of node names (where they are used asindex) with the node UUID.It also supports the inverse replacement, so downgrades are supportedtoo.
Tests for cfgupgrade are adapted as well....
Index nodes by their UUID
No longer index nodes by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the ConfigWriter, including all methods....
Remove XEN_CMD from constants, adjust to PowercycleNode
This patch extend the RPC to powercycle a node by ahvparams parameter. This parameter is used by thexen hypervisor to determine which xen command to use.The respective LU is adjusted as well.
Since this was the last operation depending on the...
hv_xen.py: make hvparams mandatory, remove fallbacks
This patch removes the fallback to the xen command from theauto config and adds an exception instead. Also, thehvparams parameter of the functions GetCommand, _RunXen and_GetInstanceList are made mandatory....
Add hvparams to RPC call 'node_info'
This patch adds the hvparams parameter to the RPC call'node_info'. It also adjusts the related code in noded.pyand Query/Node.hs
backend.py: use hvparams in GetNodeInfo
This patch extends the GetNodeInfo function of the backendmodule by a hvparams parameter. For now, it contains afallback solution for when the parameter is None. Thisfallback will be removed in a later patch of this series....
hypervisors: add hvparams to GetNodeInfo
This patch extends the GetNodeInfo function of thehypervisors by a hvparams parameter. The parameteris currently only used by the xen hypervisor to determinewhich xen toolstack ('xm' or 'xl') to use.
hv_xen.py: renaming residual 'xm' occurrences
Some methods still refered to 'xm', which is inaccurate nowthat xen can also be run with 'xl'. No functional changesotherwise.
Include hvparams in ssconf files
The watcher needs access to the cluster's hypervisorparameters. This patch extends the ssconf infrastructureby adding a hvparams file for each hypervisor. The formatof the hvparams file is a simple key=value format.
Use hvparams in instance migration
This patch makes use of hvparams on instance migration todetermine the list of instance before the migration. This way, inxen, the xl/xm choice is respected.
Verify: xen toolstack, hypervisor and hvparams
This patch extends the node verification by:- Adding a check for the xen toolstack when the hypervisor is verified.- Factoring out the hypervisor verification in a subfunction to increase testability.- Factoring out the hvparams verification in a...