Make watcher use query clients
So far, the watcher was still issuing node queriesvia the python implementation. This patche switches tothe haskell implementation.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Move GetClient() to runtime module
Soon both, the cli and the masterd, will use a luxiclient to run queries. The method to obtain and configurethe luxi client with the right address can be reused fromcli to masterd here. Therefore, this patch moves the...
Use a query client for node queries where necessary
To remove the node query's python implementation soon, thispatch makes all users of the queries call the haskellimplementation instead of the python one.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Remove --enable-split-query option
Switching from python to haskell queries, this patchremoves the option to dis/enable the haskell queriesat configure time.
Add Xen paused state as a Ganeti running state
The QA test 'TestClusterBurnin' pauses instances and the Xen statedomain 'paused' was not being handled properly by Ganeti because itwas not being classified as either a running or shutdown state. Thispatch adds the 'paused' state to the set of running states by...
Parse spindles as integer
The disk parameter "spindles", if present, has to be an integer.Enforce this already when parsing the command line, instead ofpassing around the string representation of that integer.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Merge branch 'stable-2.10' into master
Merge branch 'stable-2.9' into stable-2.10
Use custom SSH ports for other operations
In particular copyfile and renew-crypto.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Use configured SSH ports when connecting to a console
This is accomplished by passing the corresponding node group tohv_*.py. Tests for hv_*.py that call GetInstanceConsole updated.
Use custom SSH ports in node groups when working with nodes
Calling `gnt-instance console` with a custom SSH port doesn't work yet.
Add a method retrieving `NodeGroup` objects by group name
Also add a method for the retrieval of "ndparams" (node parameters)of groups, not just nodes.
Don't allow optional node parameters
Ganeti does not support optional fields in parameters(hypervisor-params, disk-params, etc.). OpenVSwitch related nodeparameters were the exception to this rule, which caused numerousproblems related to import/export and (de-)serialization....
Move OVS node parameters to the right place
OpenVSwitch node parameters were passed as separate command line flags.This was not only inconsistent with all other parameters(hypervisor-params, disk-params, ...), but also with the documentedbehavior in the man pages....
Make NIC VLAN queryable
Add an additional query field which makes it possible to query forVLANs of instance NICs.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Use constant instead of raw string
Use the appropriate constant instead of a raw string.
Fix instance info
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Fix class methods in KVM hypervisor
Several methods in KVM hypervisor behave like 'classmethods', however,they did not have the '@classmethod' annotation. This patch adds'@classmethod' annotations to several of these methods with the goalof making '_StopInstance' available to the QA. This step is necessary...
Watcher cleans user shutdown domains
Update watcher to destroy shutdown domains when it detects that adomain is in the state 'USER_down'. This requires adding logicsimilar to that of restart instance, that is tracking the number ofcleanup attempts in the watcher's state and give up if a given number...
Python instance queries report 'USER_down'
Update instance queries on the Python codebase to report 'USER_down'in the situation where the user has shutdown a Xen domain but Ganetistill marks this domain as 'ADMIN_up' in its configuration file.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Prevent instance start when user down
Prevent starting a given instance when that instance has been shutdownby the user, given that the instance must be first properly shutdown.
Xen handle domain shutdown
Update Xen backend to properly recognize when a domain has beenshutdown by the user and to properly cleanup a shutdown domain whenGaneti requests Xen to stop this domain.
Preserve Xen domain after shutdown
Change Xen domain configuration to preserve the domain state aftershutdown, as opposed to automatic reboot, which is the defaultbehaviour.
Add Python hypervisor instance state
Add 'HvInstanceState' which represents the state of an instance(either 'running' or 'shutdown') and fix the remaining hypervisorbackends to return a value of this type. Before this patch, eachbackend was returning their own value, some were returning strings,...
Fix typo in documentation
Fix retrieval of xen command in class method
This patch fixes issue 608. When introducing theconfigurability of the xen toolstack in commit8ef418bb92, the hypervisor api was accidentally changedin a way that let to this error in KVM.
Adapt TestJobSubmission to new semantics of SubmitManyJobs
Historically, Ganeti would partially handle syntactically incorrectluxi requests, if the outer structure was that of SubmitManyJobsrequest. Moreover, this behavior was verified by a test. We changed to...
Add a constructor to BaseOpCode
In order to guarantee all mandatory slots to be present, adda custom constructor to BaseOpCode adding those fields withdefault value, instead of inheriting the constructor fromoutils.ValidatedSlots.
FillDict: do not use exceptions for flow control
While it's generally better to beg for forgiveness than ask forpermission in Python, it's not a good idea to rely on exceptions tomodel control flow, especially if the resulting code is simpler.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Remove hardcoded references to File, SharedFile templates
DTS_FILEBASED is a constant that exists and this commit makes surethat it is used whenever sensible, rather than resorting to hardcodingthe pair of templates in very many files.
Make masterd call luxid to write jobs to queue
As the responsibility for writing the job queue changedto luxid, make masterd call to luxid when it is necessaryto write a job.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Set reason trail for derived opcodes
As the base parameters also includes the reason trail,set it, if not set already, together with the otherbase parameters in the case of an opcode expanding toa list of opcodes.
Add Luxi Request to pick up a job in the queue
During the transition to the new daemon layout, from step 2onwards, luxid will write to the queue but masterd will triggerthe execution. Therefore, add a new luxi request to tell masterdto pick up a job that has already been written to the queue....
Factor out functionality to pickup a job from the queue
The current restart procedure for masterd includes functionalityto pick up a job from the queue and restart it, if it hasn't beenstarted before. Move this functionality into a separate functionto be able to have the enqueuing be done by luxid....
Hs2Py constants: additional module jstore
Add constants from additional modules ('ganeti.jstore') to the Haskellto Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Hs2Py constants: additional module errors
Add constants from module 'ganeti.errors' to the Haskell to Pythonconstant generation.
Hs2Py constants: additional module qlang
Add constants from additional modules ('ganeti.qlang') to the Haskellto Python constant generation.
Hs2Py constants: additional module luxi
Add constants from additional modules ('ganeti.luxi') to the Haskellto Python constant generation.
Eliminate unused constants
Hs2Py constants: add 'UUID_REGEX'
Add constant 'UUID_REGEX' to the Haskell to Python constantgeneration.
Readd nodes as online
Patch d0d7d7cf accidentally removed the offline-flag resetwhen readding a node. Readd it.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
When verifying BRBD version, ignore missing values
When comparing for consistency of the DRBD versions, someversions might not be available via RPC, typically, if thenode is offline. In this case, leave these nodes out of thetest, instead of failing with an internal python error....
Merge branch 'stable-2.8' into stable-2.9
Conflicts: NEWS: trivial configure.ac: ignore version bump on stable-2.8 lib/bdev.py: manually apply the part of commit...
DRBD: ensure peers are UpToDate for dual-primary
DrbdAttachNet supports both, normal primary/secondary node operation, and(during live migration) dual-primary operation. When resources are newlyattached, we poll until we find all of them in connected or syncing operation....
Conflicts: NEWS: trivial configure.ac: drop suffix bump of stable-2.9
Conflicts: lib/cmdlib/instance_storage.pyResolved by manually applying the node name to uuidtransition on the version of stable-2.9.
Improve error message for replace-disks
In some conditions, replace-disks will fail if the disks are not properlyactivated. Improve the error message suggesting to run activate-disks beforeexecuting replace-disks.
Fixes Issue 606.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Replace all constant definitions with re-exports
Remove all constant definitions in 'lib/constants.py' and re-exportthe module 'lib/_constants.py', which contains the generatedconstants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Prepare constants for automatic reexport
Hs2Py constants: 'hvsParameterTypes' and 'hvsParameters'
Add constants 'hvsParameters' and 'hvsParameterTypes' to the Haskellto Python constant generation.
Fix indentation that triggers PEP8 error
Signed-off-by: Petr Pudlak <pudlak@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Check if hotplug is supported in CheckPrereq
Introduce new RPC hotplug_supported that invokes the correspondinghypervisor's method which checks if hotplug is generallysupported. Call this RPC early in CheckPrereq() and abort ifhotplug is not supported. Currently only KVM hypervisor with...
Conflicts: configure.ac: take both additions (and fix)...
Remove FIXME + update man pages wrt iallocator
This patch removes a FIXME regarding the retrievalof the disk template for computation of the clusterinformation for the iallocator. After examining thecode, I figured out that it is actually a quiterobust way to just check if the disk template is...
Don't check for master netdev when using OVS
This patch disables checking if the given master_netdev exists during clusterinit.When OVS is used, the needed Open vSwitch will be created later on (withOpClusterPostInit) and is thus not available at this point of time....
Fix TypeError in backend/ConfigureOVS
In case of running into a failure during Open vSwitch creation, the error isnot given, but a TypeError is thrown. This is because the number of argumentsfor the format string was not equal to the place holders.
This fixes the issue....
Create Open vSwitch on Master during Cluster init
Currently, Open vSwitch is only created on additional nodes, not on the masteritself.This fixes the issue and creates and configures an Open vSwitch on the masternode during cluster init.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>...
SimpleRetry on BlockDev.Remove()
Sometimes, upon disk removal, corresponding file descriptorsare kept briefly open by various processes (hypervisor, blkid, etc.).With this patch, we retry several times before raising the appropriateerror, thus making disk removal more robust against those corner cases....
Add --compress option to instance_move
instance_move now supports the --compress option with which it'spossible to select the compression mode to use during the instance move.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add possibility to compress to OpInstanceCreate
OpInstanceCreate now supports the 'compress' option. It allows to enablecompression during instance imports.
Honor compress parameter in remote backups
Use the specified compression mode not only for local exports, but alsofor remote ones.
Add --compress option to gnt-backup import
Add a new command line parameter to gnt-backup import and passits value to OpInstanceCreate. Also document the option in theman page.
Add --compress option to gnt-backup export
Add a new command line parameter to gnt-backup export and pass its valueto OpBackupExport. Also document the option in the man page.
Support import with compressed instance moves
Support compressing instance data while sending it to the target node oninstance imports.
Add local compression to OpBackupExport
OpBackupExport is extended by a compress parameter. This parameter(either 'none' or 'gzip') controls if instance disks are compressedbefore being sent over the network to the destination node.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Add --compress option to gnt-instance move
Add a new command line parameter to gnt-instance and pass its value toOpInstanceMove. Also document the option in the man page.
Add possibility to compress to OpInstanceMove
OpInstanceMove now supports the 'compress' option. It allows to enablecompression for intra-cluster instance moves.
Remove blockdev_export RPC
This RPC call is no longer used, as for intra-cluster instance moves theimport/export daemon is used too.
Optimize dd parameters for instances moves
Benchmarks have shown that increasing the buffer size and disablingsynchronous output increased the throughput by about 10MiB/s, so makethose parameters the default.
Note that not using sync for output doesn't increase the risk of data...
Use import-export daemon for intra-cluster moves
This unifies the inter- and intra-cluster moves and the backup code, soless code needs to be maintained.
Also fix failing tests for LUInstanceMove by following the code changesin the tests.
Use cluster variable consistently
The cluster object is already retrieved from the configuration, so useit consistently in LUInstanceMove.CheckPrereq().
Correct comments
Correct comments which confuse the target node with the secondary nodeof an instance in LUInstanceMove.
Show the key in "'None' is not a valid Maybe value"
Currently the error message doesn't say which key is affected, whichmakes it kind of useless.
Add RpcResult.__repr__ for debugging
This way it's possible to easily log the whole result in a humanreadable form.
Fix usage of xm/xl for gnt-instance modify --new-primary
This fixes issue 519. When calling 'gnt-instance modify--new-primary', an exception was raised, because aRPC call was not called with the right hypervisorparameters which caused the backend function to crash,...
Hs2Py constants: add 'hvcDefaults'
Add 'hvcDefaults' to the Haskell to Python constant generation.
Hs2Py constants: add 'diskDtDefaults'
Hs2Py constants: add 'value*' and 'niccDefaults'
Hs2Py constants: add 'ipolicyDefaults'
Add constant 'ipolicyDefaults' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'mond' latest version
Add 'mond' latest version constant to the Haskell to Python constantgeneration.
Hs2Py constants: add 'sshDaemonKeyfiles'
Add constant 'sshDaemonKeyfiles' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'diskLdDefaults'
Add constant 'diskLdDefaults' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'ndcDefaults'
Add constant 'ndcDefaults' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'becDefaults'
Add constant 'becDefaults' to the Haskell to Python constantgeneration. An instance of 'PyValue' is added to the type 'PyValueEx'in order to allow polymorphic types, namely, 'Map's.
Hs2Py constants: add 'drbdValidBarrierOpt'
Add constant 'drbdValidBarrierOpt' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'dtsLvm'
Add constant 'dtsLvm' to the Haskell to Python constant generation.
Hs2Py constants: remove useless constants
Remove unused constants from Python because they were only used byconstants which are currently generated, and move Python comment tothe Haskell side.
Hs2Py constants: add 'daemonsExtraLogbase'
Add constant 'daemonsExtraLogbase' to the Haskell to Python constantgeneration and update Python reference.
Hs2Py constants: remove unused constant
Remove unused Python constant whose use was replaced by otherconstants generated from Haskell.
Fix default value for reason trail
The default value for the reason trail field used to be ht.EmptyList in2.9 but was incorrectly changed to None in 2.10. This patch correctsthis error which fixes a bug in batch creation of instances.
drive_del after device_del in disk hot-remove
This is needed so that qemu process really releases thecorresponding file descriptor and thus we can dmsetup/lvremovethe device without errors.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Minor changes regarding hotplug support
Fix in RPC
Use _SingleDiskDictDP() instead of _ObjectDict() for serializinga disk.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
block_devices -> kvm_disks renaming
and a couple more styple fixes.block_devices are misleading so rename them to kvm_disk (just like kvm_nics)
Make hotplug related method raise HotplugError
Hotplug is not supported by default and thus we raiseHotplugError() for hotplug related methods.If a hypervisor wants to support it, we must override VerifyHotplugSupport()and implement the Hot* method.
Backend invokes VerifyHotplugSupport() and raises RPCFail if an...
Some fixes in _GenerateKVMBlockDevicesOptions()
Hotplug: client support
Add --hotplug option. Only used in OpInstanceSetParams.If this is omitted, modifications become effective after reboot.
Ask user confirmation in case NIC modify + hotplug because it willbe done via removing old NIC (and the corresponding tap) and adding...