Fix '_create_instance' in config_unittest
Previously '_create_instance' would initialize the config object everytime it would be called. This causes '_WriteConfig' to fail withConfigurationError: The configuration file has been modified since thelast write. Fix this by passing the config object to '_create_instance'...
Change TestInstance in objects unittest
Now that 'all_nodes' and 'secondary_nodes' of an instnace is computedfrom the 'config' module, move the corresponding tests from'objects_unittest' to 'config_unittest'.
Signed-off-by: Ilias Tsitsimpis <iliastsi@grnet.gr>
Merge branch 'stable-2.11' into master
Use node UUID as client certificate serial number
It turns out, that some implementations of OpenSSL are morepedantic in checking the certficates than others. In thisparticular case, the SSL connection could not beestablished when the serial number of the certificates...
Revert "Disabling client certificate usage"
This reverts commit 45f75526b848, which was introduced totemporarily disable the implementation of SSL clientcertificates. As this patch series fixes the reason forthe disabling, we are rolling back the patch....
Add the WConfD daemon itself
The daemon exposes the declared functions in Ganeti.WConfd.Core to RPCclients (currently just 'echo').
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add the WConfD daemon to build configuration files
Also list it in the Haskell datatype, constants, Python constants andtest configuration.
Add 'instance_communication_parameter' to 'Cluster'
Fix start/stop scripts to run luxid unconditionally
and update the unit check the list of daemons.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.10' into stable-2.11
Fix compatibility issues
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
SimpleFillOS: add unit tests for OS parameter merging
Adds tests to ensure OS parameters are dealt with consistently.
OpCodes: modify InstanceCreate for private, secret params
Modify InstanceCreate to accept process private and secret parameters.
OpCodes: modify ClusterSetParams for private parameters
Modify ClusterSetParams to accept and process private parameters.
Add private OS parameters to cluster and instance conf
This updates objects, constructors and mocks for Instance and Clusterobjects in Python and Haskell.
serializer: emit and encode Private values
For inbound data the simplest, safest thing to do is to traverse allJSON right after encoding and search for private parameters by key.
This ensures that all consumers of this data get Private valuestransparently and consistently; the serializing methods don't have to...
Merge branch 'stable-2.9' into stable-2.10
Modify test to reflect RAPI operation changes
A rlib2 unittest tested for the wrong behaviour, and this patch changesthe inputs and expected values to account for this.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Assign unique filenames to filebased disks
With the new format for cmdline arguments, the user is able to add adisk to an instance at a specific index. But filebased disks' filenameshave the form "{0}/disk{1}" where '{0}' is the file_storage_dir and'{1}' is the index of the disk. So if an instance has 3 disks and we...
Disabling client certificate usage
This patch temporarily disables the usage of the clientSSL certificates. The handling of RPC connections had aconceptional flaw, because the certificates lack a propersignature. For this, Ganeti needs to implement a CA,...
query: fix detection of master in _GetNodeRole()
Commit 1c3231aa changed the invocation of _GetNodeRole() to pass themaster node by UUID and not by name, but didn't change theimplementation to compare the nodes by name. As a result, the masternode (which is also a master candidate) would always fall through to the...
Prepare master as new 2.12 branch
This adapts various file to make the current masterbranch the new stable-2.12 branch.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add utility to compare versions
This will be needed, e.g., for post-upgrade task, as theyhave to decide whether a feature was not yet present atthe version started from.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Verify client certificates
This patch adds a step to 'gnt-cluster verify' to verifythe existence and validity of the nodes' clientcertificates. Since this is a crucial point of thesecurity concept, the verification is very detailed withexpressive error messages and well tested by unit tests....
Verify incoming RPCs against candidate map
From this patch on, incoming RPC calls are checked againstthe map of valid master candidate certificates. If no mapis present, the cluster is assumed to be inbootstrap/upgrade mode and compares the incoming call...
Extend RPC call to create SSL certificates
So far the RPC call 'node_crypto_tokens' did only retrievethe certificate digest of an existing certificate. Thiscall is now enhanced to also create a new certificate andreturn the respective digest. This will be used in various...
Handle client certificates on node add/remove
This patch adds the certificate of a newly added orreadded master candidate node to the map of master candidatecertificates. It removes a master candidate node's certificatedigest from the candidate certificate map if the node is...
Add certificate for master node
On cluster initialization, the master node'sSSL certificate digest is added to the list of mastercandidate certificates.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add candiate certificate map to configuration
At the end of this patch series, incoming RPC calls arelegitimized against a map of master candidate nodes'SSL certificate digests. This patch adds the map itselfto the cluster's configuration.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Retrieve a node's certificate digest
In various cluster operations, the master node needs toretrieve the digest of a node's SSL certificate. For thispurpose, we add an RPC call to retrieve the digest. Thefunction is designed in a general way to make it possible...
Utility functions to manipulate the candidate map
This patch adds a couple of utility functions to manipulatethe map of master candidate SSL certificate digests.
Remove superfluous imports
This removes some superfluous imports from the X509 (SSL)unittests.
Ensure that all the hypervisors exist in the config file
All the hypervisors are supposed to exist in the config file, but it might notbe so after upgrades from old versions. This patch ensures that all the missinghypervisors are added with their default values to the config file....
Fix testEncodeInstance test input
The input of the testEncodeInstance test was not adherent to the actual formatof the Ganeti configuration file: kvm has no HV_BLOCKDEV_PREFIX, and "hvparams" inside an instance should only contain the values of the hypervisor parameters,...
Merge branch 'stable-2.10' into master
Replace errors re-export in luxi.py with proper imports
Instead of re-exporting errors in luxi.py, import rpc/errors.py in themodules that use them.
Rename CallLuxiMethod to CallRPCMethod
Also update error messages and testing code to refer to RPC instead ofLUXI.
Split Luxi Client into a generic and a specific part
The generic part will be reused in WConfd.
Add a Python directory for RPC code to keep it at one place
Move rpc.py to rpc/node.py and modify imports in existing code.
Gluster: add the Shared File storage type
The shared file and gluster disk templates should not report their diskspace information like file does, because they do not behave the same.
If a cluster pulls from the same, shared source of storage then it is...
Gluster: mount automatically
Add parameters to the Gluster disk template so Gluster can manage themount point point autonomously.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Gluster: use ssconf value for mountpoint directory
Gluster still does not mount anything autonomously, but this commitchanges where Gluster expects its mountpoint to be.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Gluster: add GlusterVolume class
This commit teaches Gluster what a volume is and how to use it.
Gluster: minimal implementation
Add Gluster to Ganeti by essentially cloning the shared file behavioureverywhere in the code base.
netutils: Add ValidatePortNumber method
This method accepts a port number and checks that it is in fact valid.
FileStorage: extract file logic to a FileDeviceHelper object
This will allow code reuse for Gluster through composition, ratherthan inheritance.
Merge branch 'stable-2.8' into stable-2.9
Add support for blktap2 file-driver
Newer Xen versions use blktap2 instead of blktap. This patch adds supportfor it in Ganeti.
Fixes Issue 638.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Update tests
Update hypervisor unit tests.
Partial cherry-pick from d2e4e099e4248832fef8ed7b0755d01bd4178e3a
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add default_iallocator_params cluster parameter
Add a cluster parameter to hold the iallocator parameters usedby the default instance allocator. Implement the option tomodify config.data, query config.data and upgrade man pages,tests and cfgupgrade tool. The new default_iallocator_params is...
Export and network queries: remove opcodes and LUs
Removes the remains of the export (aka backup) and networkqueries.
Group queries: remove opcodes and LUs
Removes the remains of the group query code.
Remove instance query python code
This patch removes the python code for the instancequeries. So far, it replaces it by 'NotImplemented'exceptions. In a later patch of this series, theremaining part is remove completely.
Use query client when neccessary for group queries
This patch makes code use the query client for groupqueries where necessary in order to remove the pythonqueries soon.
Switch to haskell for export (aka backup) queries
This patch removes the python implementation of export(aka backup) queries. So far, it is replaced by'NotImplemented' exceptions, but later in this seriesit will be replaced completely.
masterd: implement query via luxi
The master daemon so far still did queries via the pythonimplementation. This patch implements that it uses thehaskell implementation and removes the node queries fromthe list of OP-queriable entities.
Use a query client for node queries where necessary
To remove the node query's python implementation soon, thispatch makes all users of the queries call the haskellimplementation instead of the python one.
Introduce --hotplug-if-possible option
This will be useful for an external entity using RAPI thatwants to modify devices of instances.
The common use case for that is:"I want to add a NIC/disk to an instance. If it is runningthen try to hotplug the device. If not, then just add it to config."...
Allow instances to obtain externally reserved IPs
The administrator should be able to assign an externally reserved IPto a Ganeti instance manually, if desired. Currently this is notsupported. External reservations should act as holes in the pool andnot just as IPs already used by someone outside of Ganeti....
Update custom SSH ports config when downgrading to 2.10
If SSH ports other than 22 are configured, abort the config downgrade,as the operation would certainly fail. Otherwise just removendparams/ssh_port from the configuration.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Use configured SSH ports when connecting to a console
This is accomplished by passing the corresponding node group tohv_*.py. Tests for hv_*.py that call GetInstanceConsole updated.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Update unit tests for custom SSH ports
The unit tests now include the configurations of SSH ports for nodegroups/nodes.
Use custom SSH ports in node groups when working with nodes
Calling `gnt-instance console` with a custom SSH port doesn't work yet.
Don't allow optional node parameters
Ganeti does not support optional fields in parameters(hypervisor-params, disk-params, etc.). OpenVSwitch related nodeparameters were the exception to this rule, which caused numerousproblems related to import/export and (de-)serialization....
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add a constructor to BaseOpCode
In order to guarantee all mandatory slots to be present, adda custom constructor to BaseOpCode adding those fields withdefault value, instead of inheriting the constructor fromoutils.ValidatedSlots.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Remove hardcoded references to File, SharedFile templates
DTS_FILEBASED is a constant that exists and this commit makes surethat it is used whenever sensible, rather than resorting to hardcodingthe pair of templates in very many files.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Fix docstring for ganeti.storage.filestorage_unittest.py
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add Luxi Request to pick up a job in the queue
During the transition to the new daemon layout, from step 2onwards, luxid will write to the queue but masterd will triggerthe execution. Therefore, add a new luxi request to tell masterdto pick up a job that has already been written to the queue....
Check if hotplug is supported in CheckPrereq
Introduce new RPC hotplug_supported that invokes the correspondinghypervisor's method which checks if hotplug is generallysupported. Call this RPC early in CheckPrereq() and abort ifhotplug is not supported. Currently only KVM hypervisor with...
Use import-export daemon for intra-cluster moves
This unifies the inter- and intra-cluster moves and the backup code, soless code needs to be maintained.
Also fix failing tests for LUInstanceMove by following the code changesin the tests.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Hs2Py constants: add 'daemonsExtraLogbase'
Add constant 'daemonsExtraLogbase' to the Haskell to Python constantgeneration and update Python reference.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add unittests for Hotplug support
Try base InstanceSetParams tests that modify instance'sdevices with hotplug=True.
Add some unittests for kvm hypervisor regarding runtime filemanipulation. Based on a sample runtime file search forspecific disk and nic entries based on their UUID....
Hotplug: cmdlib support
Hotplugging is done by functions invoked by ApplyContainerMods(). Inorder hotplugging to take place the --hotplug option must be passedotherwise the modifications will take place after reboot.
NIC hotplug supports add, remove and modify. The modify is done by removing...
Return link_name in blockdev_assemble rpc
Until now this RPC returned only dev_path. Since we use it inhotplug we have to know the simlink of the device so that wepass it to the corresponding hypervisor command and include itin block_devices entry in runtime files....
Merge branch 'origin/stable-2.10' into 'master'
Fix unit test regarding recent changes in node_info RPC
This fixes a unit test in instance_storage_unittest.py,which was affected by the recent change in the RCP'node_info'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrovje Ribicic <riba@google.com>
Add a console information RPC call
As the instance queries need console information, and the informationis retrieved through python classes that should not be ported yet, anRPC call supplying the information has been added. Some tests as well.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Prepare master branch for 2.11 devel cycle
Bump versions in various places to 2.11 and remove the downgrade codefrom cfgupgrade. Also adapt cfgupgrade tests.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Move orphaned unit tests to cmdlib/cluster_unittest.py
Through some merge from 2.9 to master, some unit testsin the file ganeti.comdlib.common_unittest.py wereno longer referenced in the Makefile and as such notrun anymore. With the introduction of the cmdlib test...
Merge branch 'stable-2.9'
Add unit tests for LUBackupExport
This patch adds unit tests for LUBackupExport.
Add unit tests for LUBackupPrepare
This patch adds unit tests for LUBackupPrepare.
Add unit tests for LUBackupQuery
This patch adds unit tests for LUBackupQuery.
Make 'gnt-node list-storage' use default storage type
Currently, when no explicit storage type is given, the'gnt-node list-storage' command defaults to file storagewhether or not file storage is enabled on the clusteror not. This patch fixes it by defaulting to the default...
Streamlining handling of spindles and default templates
This rather lenghy patch comprises a couple of refactoringsto achieve the following goals:- 'gnt-node info' should only report space information about spindles, when exclusive storage is enabled on the node...
Rename entities of unit test for MakeLegacyNodeInfo
This is a small preparation step for the next patch.It just renames the key/value constants for the unittest of MakeLegacyNodeInfo, because they were ratherundescriptive and thus made changes to the test cumbersome....
cfg auto update: match ipolicy with enabled disk templates
This patch fixes issue 599. The problem was that with 2.9,a sanity check was introduced that made sure that theipolicies' set of disk templates is a subset of thecluster-wide enabled disk templates. In order to make...
Move unit tests for --file-storage-dir to cmdlib/*
There were some unit tests for the '--file-storage-dir'option of 'gnt-cluster modify' still left in the unittest file ganeti.cmdlib.cluster_unittest.py.With the introducion of the unit test framework for...