Split Luxi Client into a generic and a specific part
The generic part will be reused in WConfd.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add a Python directory for RPC code to keep it at one place
Move rpc.py to rpc/node.py and modify imports in existing code.
Gluster: add the Shared File storage type
The shared file and gluster disk templates should not report their diskspace information like file does, because they do not behave the same.
If a cluster pulls from the same, shared source of storage then it is...
Gluster: mount automatically
Add parameters to the Gluster disk template so Gluster can manage themount point point autonomously.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Gluster: use ssconf value for mountpoint directory
Gluster still does not mount anything autonomously, but this commitchanges where Gluster expects its mountpoint to be.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>...
ssconf: Add Gluster mount directory
This commit adds the gluster storage directory to ssconf (withoutactually using its value just yet).
Gluster: add GlusterVolume class
This commit teaches Gluster what a volume is and how to use it.
Gluster: minimal implementation
Add Gluster to Ganeti by essentially cloning the shared file behavioureverywhere in the code base.
netutils: Add ValidatePortNumber method
This method accepts a port number and checks that it is in fact valid.
FileStorage: extract file logic to a FileDeviceHelper object
This will allow code reuse for Gluster through composition, ratherthan inheritance.
Generalize the IO client handling in Luxi
... to be usable for WConfd as well. A daemon handler is encapsulatedinto `Handler` data type, which is then passed to a generic `listener`.
The changes are done in Luxi.hs so that the differences are visible and...
Add the Unix domain socket path to the Server data type
This simplifies code for closing such a socket.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Rename getClient/Server to getLuxiClient/Server
Later they will be split into LUXI-specific and general parts.
Add default_iallocator_params cluster parameter
Add a cluster parameter to hold the iallocator parameters usedby the default instance allocator. Implement the option tomodify config.data, query config.data and upgrade man pages,tests and cfgupgrade tool. The new default_iallocator_params is...
Export and network queries: remove opcodes and LUs
Removes the remains of the export (aka backup) and networkqueries.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Group queries: remove opcodes and LUs
Removes the remains of the group query code.
Remove opcodes from haskell tests
Clears the haskell tests from the query opcodes that are nolonger needed.
Remove instance query python code
This patch removes the python code for the instancequeries. So far, it replaces it by 'NotImplemented'exceptions. In a later patch of this series, theremaining part is remove completely.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Use query client when neccessary for group queries
This patch makes code use the query client for groupqueries where necessary in order to remove the pythonqueries soon.
Switch to haskell for export (aka backup) queries
This patch removes the python implementation of export(aka backup) queries. So far, it is replaced by'NotImplemented' exceptions, but later in this seriesit will be replaced completely.
masterd: implement query via luxi
The master daemon so far still did queries via the pythonimplementation. This patch implements that it uses thehaskell implementation and removes the node queries fromthe list of OP-queriable entities.
Use a query client for node queries where necessary
To remove the node query's python implementation soon, thispatch makes all users of the queries call the haskellimplementation instead of the python one.
Fix wrong expectation in hsqueeze shelltest
The shelltests expected the wrong position of the -f optionas well. Fix this.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add missing spindles paramter to idisk
When spindles where added to Ganeti, apparently it was forgottento add the parameter to the Haskell data type as well. Do this now.
Allow the NIC VLAN to be set to an empty string
The NIC VLAN has previously not been modified via Haskell, causing theINicParams class not to be used. With the recent job queuerefactorings, a modification definition is recorded, and for an emptystring (which is a legal default value) a crash happens. This patch...
Add tests for hsqueeze commands
In the examples, pick a node that certainly has to be powered on or off,respectively, and verify that the power command is there, as wellan appropriate migration command in the right position.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Merge branch 'stable-2.10' into master
Merge branch 'stable-2.9' into stable-2.10
Add NodeGroup to InstanceConsoleInfoParams
Before, calls to `gnt-instance list -o console` with an instance on anode with a custom SSH port failed because of missing groupconfiguration. This patch fixes the problem.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Update custom SSH ports config when downgrading to 2.10
If SSH ports other than 22 are configured, abort the config downgrade,as the operation would certainly fail. Otherwise just removendparams/ssh_port from the configuration.
Use configured SSH ports when connecting to a console
This is accomplished by passing the corresponding node group tohv_*.py. Tests for hv_*.py that call GetInstanceConsole updated.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Update unit tests for custom SSH ports
The unit tests now include the configurations of SSH ports for nodegroups/nodes.
Use custom SSH ports in node groups when working with nodes
Calling `gnt-instance console` with a custom SSH port doesn't work yet.
group queries: test niceSort and remove FIXME
In an effort to get rid of the python queries soon, thispatch fixes a FIXME of the group queries regarding themissing testing of niceSort in this context. Due to thelack of actually weirdly named hostnames, this patch...
Don't allow optional node parameters
Ganeti does not support optional fields in parameters(hypervisor-params, disk-params, etc.). OpenVSwitch related nodeparameters were the exception to this rule, which caused numerousproblems related to import/export and (de-)serialization....
Update tests
Update hypervisor unit tests.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add Haskell hypervisor instance state
Add 'InstanceState' datatype which is the Haskell counterpart of thePython type 'HvInstanceState'.
Add a constructor to BaseOpCode
In order to guarantee all mandatory slots to be present, adda custom constructor to BaseOpCode adding those fields withdefault value, instead of inheriting the constructor fromoutils.ValidatedSlots.
Remove hardcoded references to File, SharedFile templates
DTS_FILEBASED is a constant that exists and this commit makes surethat it is used whenever sensible, rather than resorting to hardcodingthe pair of templates in very many files.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Fix docstring for ganeti.storage.filestorage_unittest.py
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add Luxi Request to pick up a job in the queue
During the transition to the new daemon layout, from step 2onwards, luxid will write to the queue but masterd will triggerthe execution. Therefore, add a new luxi request to tell masterdto pick up a job that has already been written to the queue....
Add a test for hsqueeze expansion
In this example, there are 7 nodes, each with capacity for4 instances. Two nodes are online and host 4 instances each,thus forming a cluster with no free resources. Four additionalnodes a standby, and two additional node is offline for other...
hsqueeze: add a test for the -v option
This test also verifies that the master node isnot considered for being offlined.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Change default for target resource to 2.0
The target resources, as used by hsqueeze, are supposed tobe strictly higher than the minimal resources. However,keeping minimal resources of less than a single instanceis not a useful reserve.
Check if hotplug is supported in CheckPrereq
Introduce new RPC hotplug_supported that invokes the correspondinghypervisor's method which checks if hotplug is generallysupported. Call this RPC early in CheckPrereq() and abort ifhotplug is not supported. Currently only KVM hypervisor with...
Support import with compressed instance moves
Support compressing instance data while sending it to the target node oninstance imports.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add local compression to OpBackupExport
OpBackupExport is extended by a compress parameter. This parameter(either 'none' or 'gzip') controls if instance disks are compressedbefore being sent over the network to the destination node.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Add possibility to compress to OpInstanceMove
OpInstanceMove now supports the 'compress' option. It allows to enablecompression for intra-cluster instance moves.
Use import-export daemon for intra-cluster moves
This unifies the inter- and intra-cluster moves and the backup code, soless code needs to be maintained.
Also fix failing tests for LUInstanceMove by following the code changesin the tests.
Fix expectation of the htools wrapping
With the addition of hsqueeze, htools obtained a new personality.Additionally, the longer htool name also changes the line wrapping ofthe descriptions. Adapt the expectation accordingly.
Add test for hsqueeze compression
In this example cluster, there are 6 nodes withcapacity for 4 instances each, holding a totalof 8 instances. So even after turning off 2 nodesand rebalancing, every node still has capacity for2 instances.
Hs2Py constants: add 'daemonsExtraLogbase'
Add constant 'daemonsExtraLogbase' to the Haskell to Python constantgeneration and update Python reference.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add unittests for Hotplug support
Try base InstanceSetParams tests that modify instance'sdevices with hotplug=True.
Add some unittests for kvm hypervisor regarding runtime filemanipulation. Based on a sample runtime file search forspecific disk and nic entries based on their UUID....
Hotplug: client support
Add --hotplug option. Only used in OpInstanceSetParams.If this is omitted, modifications become effective after reboot.
Ask user confirmation in case NIC modify + hotplug because it willbe done via removing old NIC (and the corresponding tap) and adding...
Hotplug: cmdlib support
Hotplugging is done by functions invoked by ApplyContainerMods(). Inorder hotplugging to take place the --hotplug option must be passedotherwise the modifications will take place after reboot.
NIC hotplug supports add, remove and modify. The modify is done by removing...
Return link_name in blockdev_assemble rpc
Until now this RPC returned only dev_path. Since we use it inhotplug we have to know the simlink of the device so that wepass it to the corresponding hypervisor command and include itin block_devices entry in runtime files....
Merge branch 'origin/stable-2.10' into 'master'
Fix unit test regarding recent changes in node_info RPC
This fixes a unit test in instance_storage_unittest.py,which was affected by the recent change in the RCP'node_info'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrovje Ribicic <riba@google.com>
Tests for InstanceInfo retrieval added
As the retrieval of instance information is a tricky operation whichaffects the way the instance status is calculated, a few tests havebeen added to ensure its behaviour remains constant.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Add a console information RPC call
As the instance queries need console information, and the informationis retrieved through python classes that should not be ported yet, anRPC call supplying the information has been added. Some tests as well.
Add aliases of existing instance fields
For legacy reasons, some instance fields are aliased. The aliases havethe same definition but appear under different names. Support for suchfields has been added to Query/Common.hsAs aliases can cause somewhat harder to find bugs, a new test was...
Enable 'AllInstancesInfo' RPC call in Haskell
The AllInstancesInfo call appears to have been implemented but neverused. The commit makes it usable by modifying the parameter type andserialization.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Prepare master branch for 2.11 devel cycle
Bump versions in various places to 2.11 and remove the downgrade codefrom cfgupgrade. Also adapt cfgupgrade tests.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Move orphaned unit tests to cmdlib/cluster_unittest.py
Through some merge from 2.9 to master, some unit testsin the file ganeti.comdlib.common_unittest.py wereno longer referenced in the Makefile and as such notrun anymore. With the introduction of the cmdlib test...
Merge branch 'stable-2.9'
Add unit tests for LUBackupExport
This patch adds unit tests for LUBackupExport.
Add unit tests for LUBackupPrepare
This patch adds unit tests for LUBackupPrepare.
Add unit tests for LUBackupQuery
This patch adds unit tests for LUBackupQuery.
Make 'gnt-node list-storage' use default storage type
Currently, when no explicit storage type is given, the'gnt-node list-storage' command defaults to file storagewhether or not file storage is enabled on the clusteror not. This patch fixes it by defaulting to the default...
Streamlining handling of spindles and default templates
This rather lenghy patch comprises a couple of refactoringsto achieve the following goals:- 'gnt-node info' should only report space information about spindles, when exclusive storage is enabled on the node...
Rename entities of unit test for MakeLegacyNodeInfo
This is a small preparation step for the next patch.It just renames the key/value constants for the unittest of MakeLegacyNodeInfo, because they were ratherundescriptive and thus made changes to the test cumbersome....
Add test suite for some names in 'AutoConf'
Add test suite for some names in 'AutoConf' to test whether the valuesgenerated from 'configure' and the 'Makefile' are valid filepaths,user names, group names, among others. Fixes issue 601.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
cfg auto update: match ipolicy with enabled disk templates
This patch fixes issue 599. The problem was that with 2.9,a sanity check was introduced that made sure that theipolicies' set of disk templates is a subset of thecluster-wide enabled disk templates. In order to make...
Move unit tests for --file-storage-dir to cmdlib/*
There were some unit tests for the '--file-storage-dir'option of 'gnt-cluster modify' still left in the unittest file ganeti.cmdlib.cluster_unittest.py.With the introducion of the unit test framework for...
Correct properties of the cluster's file storage dir
This patch does:- Rename the haskell opcode parameter 'pGlobalFileStorageDir' and 'pGlobalSharedfileStorageDir' to 'pClusterFileStorageDir' and 'pClusterSharedfileStorageDir', respectively, because...
Move disk template unit tests to test/py/cmdlib/
This patch moves some unit tests related to dis/enablingdisk templates from ganeti.cmdlib.cluster_unittest.pyto cmdlib/cluster_unittest.py to utilized the new cmdlibunit test framework.
Make GetDiskTemplateSets output disabled disk templates
This patch extends the 'GetDiskTemplateSets' function toalso output the list of disk templates that were enabledbefore this operation, but will be disabled afterwards.This patch also includes code to check for instances using...
Rename GetEnabledDiskTemplates to include disabled ones
This patch solely renames the functions_GetEnabledDiskTemplates* to _GetDiskTemplateSets, becausein later patches, we will refactor it to not only outputdisk templates that are or get enabled, but also the ones...
Merge branch 'stable-2.9' into master
Use 'DTS_LVM' when possible
This patch replaces all usages of the utility function'GetLvmDiskTemplate' by the new 'DTS_LVM' constantto make it consistant with the usage of other DTS_*constants.
Additionally, it provides a unit tests to ensureconsistancy between DTS_LVM and the mapping of disk...
iallocator: use lookup by disk template
So far, the iallocator requested storage informationabout all enabled disk templates but discarded all butthe LVM information, even if it was allocating spacefor a non-LVM instance. With this patch, it now onlyasks for the one that is relevant for...
utils/storage.py: storage info lookup by disk template
This patch improves the handling of storage informationbefore and after a call to the RPC 'node_info'. Itadds a function to not only call for all storageinformation on the cluster (as it is used right now),...
Downgrade 'vif_script'
This patch adds removing the 'vif_script' attributefrom the cluster's config when downgrading from 2.9to 2.8. Adaption of the unit test is included.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Downgrade 'xen_cmd'
This downgrades the cluster's hypervisor parameters for thexen hypervisors with respect to the 'xen_cmd' attribute.A unit tests is provided.
Add test cluster config for version 2.9
This adds a cluster config example for version 2.9 tothe test data collection. It will be used in subsequenttests.
Remove superfluous import of 'errors'
Provide utility function to check a configuration version
In order for 'gnt-cluster upgrade --resume' to determine whether the configurationis already upgraded, it needs to compare whether a configuration version is compatiblewith a Ganeti target version. Provide a utility function for this....
Provide an inverse to UnescapeAndSplit
With utils.UnescapeAndSplit, we have a function to parsearbitrary non-empty string lists encoded as strings. Alsoprovide the appropriate encoding function.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Provide means of submitting jobs to a drained queue
During an upgrade, the job queue needs to be drained in order to avoidnew jobs coming to the cluster. Nevertheless, the upgrade processneeds to carry out some maintenance, like redistributing the new...
Add predicate whether to call cfgupgrade --downgrade
Provide a predicate that, given the current version and theversion to go to, tells whether it is appropriate to callcfgupgrade --downgrade.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Add utility function to recognize upgrade ranges
Upgrading is possible within the same major version to any equalor higher minor version. Downgrading is possible within the samemajor version to the previous minor version. Moreover, automaticupgrades are only supported from version 2.10 onwards. Add a utility...
Add utility function to parse version strings
The new 'gnt-cluster upgrade' command will get a Ganeti version asargument. So provide a function able to parse it.
Add tests specific to the user/kernelspace access feature
Check that the validation procedures don't accept invalid valuesand also don't choke on valid values.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Add Userspace RBD support in KVM
Define 'access' disk-parameter type
Add the "access" parameter to the RBD configuration. Fix test brokenby this change.
Merge 'EvacNode' and 'NodeEvacMode'
Merge 'EvacMode' and 'NodeEvacMode' which are associated with theduplicated constants 'IALLOCATOR_NEVAC_*' and 'NODE_EVAC_*'. Thismerge moves the type 'EvacMode' to 'Ganeti.Types' in order to avoid acircular dependency between 'Ganeti.HsConstants' and...
Replace 'IALLOCATOR_NEVAC_*' with 'NODE_EVAC_*'
Replace uses of 'IALLOCATOR_NEVAC_PRI', 'IALLOCATOR_NEVAC_SEC','IALLOCATOR_NEVAC_ALL', and 'IALLOCATOR_NEVAC_MODES', with'NODE_EVAC_PRI', 'NODE_EVAC_SEC', 'NODE_EVAC_ALL', and'NODE_EVAC_MODES', given that these constants are repeated....
Disable hlint camelcase on test module