Fix start/stop scripts to run luxid unconditionally
and update the unit check the list of daemons.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.10' into stable-2.11
Merge branch 'stable-2.9' into stable-2.10
Modify test to reflect RAPI operation changes
A rlib2 unittest tested for the wrong behaviour, and this patch changesthe inputs and expected values to account for this.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Assign unique filenames to filebased disks
With the new format for cmdline arguments, the user is able to add adisk to an instance at a specific index. But filebased disks' filenameshave the form "{0}/disk{1}" where '{0}' is the file_storage_dir and'{1}' is the index of the disk. So if an instance has 3 disks and we...
Disabling client certificate usage
This patch temporarily disables the usage of the clientSSL certificates. The handling of RPC connections had aconceptional flaw, because the certificates lack a propersignature. For this, Ganeti needs to implement a CA,...
query: fix detection of master in _GetNodeRole()
Commit 1c3231aa changed the invocation of _GetNodeRole() to pass themaster node by UUID and not by name, but didn't change theimplementation to compare the nodes by name. As a result, the masternode (which is also a master candidate) would always fall through to the...
Add utility to compare versions
This will be needed, e.g., for post-upgrade task, as theyhave to decide whether a feature was not yet present atthe version started from.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Verify client certificates
This patch adds a step to 'gnt-cluster verify' to verifythe existence and validity of the nodes' clientcertificates. Since this is a crucial point of thesecurity concept, the verification is very detailed withexpressive error messages and well tested by unit tests....
Verify incoming RPCs against candidate map
From this patch on, incoming RPC calls are checked againstthe map of valid master candidate certificates. If no mapis present, the cluster is assumed to be inbootstrap/upgrade mode and compares the incoming call...
Extend RPC call to create SSL certificates
So far the RPC call 'node_crypto_tokens' did only retrievethe certificate digest of an existing certificate. Thiscall is now enhanced to also create a new certificate andreturn the respective digest. This will be used in various...
Handle client certificates on node add/remove
This patch adds the certificate of a newly added orreadded master candidate node to the map of master candidatecertificates. It removes a master candidate node's certificatedigest from the candidate certificate map if the node is...
Add certificate for master node
On cluster initialization, the master node'sSSL certificate digest is added to the list of mastercandidate certificates.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add candiate certificate map to configuration
At the end of this patch series, incoming RPC calls arelegitimized against a map of master candidate nodes'SSL certificate digests. This patch adds the map itselfto the cluster's configuration.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Retrieve a node's certificate digest
In various cluster operations, the master node needs toretrieve the digest of a node's SSL certificate. For thispurpose, we add an RPC call to retrieve the digest. Thefunction is designed in a general way to make it possible...
Utility functions to manipulate the candidate map
This patch adds a couple of utility functions to manipulatethe map of master candidate SSL certificate digests.
Remove superfluous imports
This removes some superfluous imports from the X509 (SSL)unittests.
Ensure that all the hypervisors exist in the config file
All the hypervisors are supposed to exist in the config file, but it might notbe so after upgrades from old versions. This patch ensures that all the missinghypervisors are added with their default values to the config file....
Fix testEncodeInstance test input
The input of the testEncodeInstance test was not adherent to the actual formatof the Ganeti configuration file: kvm has no HV_BLOCKDEV_PREFIX, and "hvparams" inside an instance should only contain the values of the hypervisor parameters,...
Merge branch 'stable-2.10' into master
Replace errors re-export in luxi.py with proper imports
Instead of re-exporting errors in luxi.py, import rpc/errors.py in themodules that use them.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Rename CallLuxiMethod to CallRPCMethod
Also update error messages and testing code to refer to RPC instead ofLUXI.
Split Luxi Client into a generic and a specific part
The generic part will be reused in WConfd.
Add a Python directory for RPC code to keep it at one place
Move rpc.py to rpc/node.py and modify imports in existing code.
Gluster: add the Shared File storage type
The shared file and gluster disk templates should not report their diskspace information like file does, because they do not behave the same.
If a cluster pulls from the same, shared source of storage then it is...
Gluster: mount automatically
Add parameters to the Gluster disk template so Gluster can manage themount point point autonomously.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Gluster: use ssconf value for mountpoint directory
Gluster still does not mount anything autonomously, but this commitchanges where Gluster expects its mountpoint to be.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Gluster: add GlusterVolume class
This commit teaches Gluster what a volume is and how to use it.
Gluster: minimal implementation
Add Gluster to Ganeti by essentially cloning the shared file behavioureverywhere in the code base.
netutils: Add ValidatePortNumber method
This method accepts a port number and checks that it is in fact valid.
FileStorage: extract file logic to a FileDeviceHelper object
This will allow code reuse for Gluster through composition, ratherthan inheritance.
Merge branch 'stable-2.8' into stable-2.9
Add support for blktap2 file-driver
Newer Xen versions use blktap2 instead of blktap. This patch adds supportfor it in Ganeti.
Fixes Issue 638.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Update tests
Update hypervisor unit tests.
Partial cherry-pick from d2e4e099e4248832fef8ed7b0755d01bd4178e3a
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add default_iallocator_params cluster parameter
Add a cluster parameter to hold the iallocator parameters usedby the default instance allocator. Implement the option tomodify config.data, query config.data and upgrade man pages,tests and cfgupgrade tool. The new default_iallocator_params is...
Export and network queries: remove opcodes and LUs
Removes the remains of the export (aka backup) and networkqueries.
Group queries: remove opcodes and LUs
Removes the remains of the group query code.
Remove instance query python code
This patch removes the python code for the instancequeries. So far, it replaces it by 'NotImplemented'exceptions. In a later patch of this series, theremaining part is remove completely.
Use query client when neccessary for group queries
This patch makes code use the query client for groupqueries where necessary in order to remove the pythonqueries soon.
Switch to haskell for export (aka backup) queries
This patch removes the python implementation of export(aka backup) queries. So far, it is replaced by'NotImplemented' exceptions, but later in this seriesit will be replaced completely.
masterd: implement query via luxi
The master daemon so far still did queries via the pythonimplementation. This patch implements that it uses thehaskell implementation and removes the node queries fromthe list of OP-queriable entities.
Use a query client for node queries where necessary
To remove the node query's python implementation soon, thispatch makes all users of the queries call the haskellimplementation instead of the python one.
Introduce --hotplug-if-possible option
This will be useful for an external entity using RAPI thatwants to modify devices of instances.
The common use case for that is:"I want to add a NIC/disk to an instance. If it is runningthen try to hotplug the device. If not, then just add it to config."...
Allow instances to obtain externally reserved IPs
The administrator should be able to assign an externally reserved IPto a Ganeti instance manually, if desired. Currently this is notsupported. External reservations should act as holes in the pool andnot just as IPs already used by someone outside of Ganeti....
Update custom SSH ports config when downgrading to 2.10
If SSH ports other than 22 are configured, abort the config downgrade,as the operation would certainly fail. Otherwise just removendparams/ssh_port from the configuration.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Use configured SSH ports when connecting to a console
This is accomplished by passing the corresponding node group tohv_*.py. Tests for hv_*.py that call GetInstanceConsole updated.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Update unit tests for custom SSH ports
The unit tests now include the configurations of SSH ports for nodegroups/nodes.
Use custom SSH ports in node groups when working with nodes
Calling `gnt-instance console` with a custom SSH port doesn't work yet.
Don't allow optional node parameters
Ganeti does not support optional fields in parameters(hypervisor-params, disk-params, etc.). OpenVSwitch related nodeparameters were the exception to this rule, which caused numerousproblems related to import/export and (de-)serialization....
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add a constructor to BaseOpCode
In order to guarantee all mandatory slots to be present, adda custom constructor to BaseOpCode adding those fields withdefault value, instead of inheriting the constructor fromoutils.ValidatedSlots.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Remove hardcoded references to File, SharedFile templates
DTS_FILEBASED is a constant that exists and this commit makes surethat it is used whenever sensible, rather than resorting to hardcodingthe pair of templates in very many files.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Fix docstring for ganeti.storage.filestorage_unittest.py
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add Luxi Request to pick up a job in the queue
During the transition to the new daemon layout, from step 2onwards, luxid will write to the queue but masterd will triggerthe execution. Therefore, add a new luxi request to tell masterdto pick up a job that has already been written to the queue....
Check if hotplug is supported in CheckPrereq
Introduce new RPC hotplug_supported that invokes the correspondinghypervisor's method which checks if hotplug is generallysupported. Call this RPC early in CheckPrereq() and abort ifhotplug is not supported. Currently only KVM hypervisor with...
Use import-export daemon for intra-cluster moves
This unifies the inter- and intra-cluster moves and the backup code, soless code needs to be maintained.
Also fix failing tests for LUInstanceMove by following the code changesin the tests.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Hs2Py constants: add 'daemonsExtraLogbase'
Add constant 'daemonsExtraLogbase' to the Haskell to Python constantgeneration and update Python reference.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add unittests for Hotplug support
Try base InstanceSetParams tests that modify instance'sdevices with hotplug=True.
Add some unittests for kvm hypervisor regarding runtime filemanipulation. Based on a sample runtime file search forspecific disk and nic entries based on their UUID....
Hotplug: cmdlib support
Hotplugging is done by functions invoked by ApplyContainerMods(). Inorder hotplugging to take place the --hotplug option must be passedotherwise the modifications will take place after reboot.
NIC hotplug supports add, remove and modify. The modify is done by removing...
Return link_name in blockdev_assemble rpc
Until now this RPC returned only dev_path. Since we use it inhotplug we have to know the simlink of the device so that wepass it to the corresponding hypervisor command and include itin block_devices entry in runtime files....
Merge branch 'origin/stable-2.10' into 'master'
Fix unit test regarding recent changes in node_info RPC
This fixes a unit test in instance_storage_unittest.py,which was affected by the recent change in the RCP'node_info'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrovje Ribicic <riba@google.com>
Add a console information RPC call
As the instance queries need console information, and the informationis retrieved through python classes that should not be ported yet, anRPC call supplying the information has been added. Some tests as well.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Prepare master branch for 2.11 devel cycle
Bump versions in various places to 2.11 and remove the downgrade codefrom cfgupgrade. Also adapt cfgupgrade tests.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Move orphaned unit tests to cmdlib/cluster_unittest.py
Through some merge from 2.9 to master, some unit testsin the file ganeti.comdlib.common_unittest.py wereno longer referenced in the Makefile and as such notrun anymore. With the introduction of the cmdlib test...
Merge branch 'stable-2.9'
Add unit tests for LUBackupExport
This patch adds unit tests for LUBackupExport.
Add unit tests for LUBackupPrepare
This patch adds unit tests for LUBackupPrepare.
Add unit tests for LUBackupQuery
This patch adds unit tests for LUBackupQuery.
Make 'gnt-node list-storage' use default storage type
Currently, when no explicit storage type is given, the'gnt-node list-storage' command defaults to file storagewhether or not file storage is enabled on the clusteror not. This patch fixes it by defaulting to the default...
Streamlining handling of spindles and default templates
This rather lenghy patch comprises a couple of refactoringsto achieve the following goals:- 'gnt-node info' should only report space information about spindles, when exclusive storage is enabled on the node...
Rename entities of unit test for MakeLegacyNodeInfo
This is a small preparation step for the next patch.It just renames the key/value constants for the unittest of MakeLegacyNodeInfo, because they were ratherundescriptive and thus made changes to the test cumbersome....
cfg auto update: match ipolicy with enabled disk templates
This patch fixes issue 599. The problem was that with 2.9,a sanity check was introduced that made sure that theipolicies' set of disk templates is a subset of thecluster-wide enabled disk templates. In order to make...
Move unit tests for --file-storage-dir to cmdlib/*
There were some unit tests for the '--file-storage-dir'option of 'gnt-cluster modify' still left in the unittest file ganeti.cmdlib.cluster_unittest.py.With the introducion of the unit test framework for...
Move disk template unit tests to test/py/cmdlib/
This patch moves some unit tests related to dis/enablingdisk templates from ganeti.cmdlib.cluster_unittest.pyto cmdlib/cluster_unittest.py to utilized the new cmdlibunit test framework.
Make GetDiskTemplateSets output disabled disk templates
This patch extends the 'GetDiskTemplateSets' function toalso output the list of disk templates that were enabledbefore this operation, but will be disabled afterwards.This patch also includes code to check for instances using...
Rename GetEnabledDiskTemplates to include disabled ones
This patch solely renames the functions_GetEnabledDiskTemplates* to _GetDiskTemplateSets, becausein later patches, we will refactor it to not only outputdisk templates that are or get enabled, but also the ones...
Merge branch 'stable-2.9' into master
Use 'DTS_LVM' when possible
This patch replaces all usages of the utility function'GetLvmDiskTemplate' by the new 'DTS_LVM' constantto make it consistant with the usage of other DTS_*constants.
Additionally, it provides a unit tests to ensureconsistancy between DTS_LVM and the mapping of disk...
iallocator: use lookup by disk template
So far, the iallocator requested storage informationabout all enabled disk templates but discarded all butthe LVM information, even if it was allocating spacefor a non-LVM instance. With this patch, it now onlyasks for the one that is relevant for...
utils/storage.py: storage info lookup by disk template
This patch improves the handling of storage informationbefore and after a call to the RPC 'node_info'. Itadds a function to not only call for all storageinformation on the cluster (as it is used right now),...
Downgrade 'vif_script'
This patch adds removing the 'vif_script' attributefrom the cluster's config when downgrading from 2.9to 2.8. Adaption of the unit test is included.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Downgrade 'xen_cmd'
This downgrades the cluster's hypervisor parameters for thexen hypervisors with respect to the 'xen_cmd' attribute.A unit tests is provided.
Remove superfluous import of 'errors'
Provide utility function to check a configuration version
In order for 'gnt-cluster upgrade --resume' to determine whether the configurationis already upgraded, it needs to compare whether a configuration version is compatiblewith a Ganeti target version. Provide a utility function for this....
Provide an inverse to UnescapeAndSplit
With utils.UnescapeAndSplit, we have a function to parsearbitrary non-empty string lists encoded as strings. Alsoprovide the appropriate encoding function.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Provide means of submitting jobs to a drained queue
During an upgrade, the job queue needs to be drained in order to avoidnew jobs coming to the cluster. Nevertheless, the upgrade processneeds to carry out some maintenance, like redistributing the new...
Add predicate whether to call cfgupgrade --downgrade
Provide a predicate that, given the current version and theversion to go to, tells whether it is appropriate to callcfgupgrade --downgrade.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Add utility function to recognize upgrade ranges
Upgrading is possible within the same major version to any equalor higher minor version. Downgrading is possible within the samemajor version to the previous minor version. Moreover, automaticupgrades are only supported from version 2.10 onwards. Add a utility...
Add utility function to parse version strings
The new 'gnt-cluster upgrade' command will get a Ganeti version asargument. So provide a function able to parse it.
Add tests specific to the user/kernelspace access feature
Check that the validation procedures don't accept invalid valuesand also don't choke on valid values.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Add Userspace RBD support in KVM