Initialize self.needed_locks
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.11' into stable-2.12
Merge branch 'stable-2.10' into stable-2.11
Add --no-locks option to gnt-debug delay
Add the possibility to don't acquire locks during `gnt-debug delay`.This allows to run many delay jobs in parallel instead of havingthem run sequentially.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>...
Flush the configuration before cluster verify
For this expose the flushing operation in Python ConfigWriter.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Support group evacuation by failover
Support evacuating a node group not using migration.This can be useful if the group evacuated to has differenthardware.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Support sequential evacuation
Make gnt-group evacuate support the --sequential option,which causes all evacuation moves to be execuated sequentially.This can be used to avoid congestion on a possibly slow linkbetween the node groups.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Refactor instance create to accommodate unsafe OS scripts
This patch gives just the code structure. Implementation will follow.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Helper method to run OS scripts in a virtualized environment
This method temporarily changes the instance's disk to contain thehelper VM's boot disk and the target disk for the OS installation.
Helper method to handle the OS install package
This helper method either picks up the package passed through the OSparameters or creates a new package and updates the instance's OSparameters. This allows users to add their own OS install package.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Extract code that runs OS scripts to a separate method
... to make it simpler to introduced the install helper VM.
Generalize 'TemporaryDisk' to handle any number of disks
... including different disk templates, accesses, and sizes.
Move 'DetermineImageSize' to 'ganeti.cmdlib.common'
Move 'DetermineImageSize' from 'ganeti.cmdlib.backup' to'ganeti.cmdlib.common' so it can be used by disk zeroing and instanceinstallations. This patch also transforms this method into a functionand generalizes the error messages....
Add trusted/untrusted create scripts
Add timeouts for OS install helper VM
Add timeout parameters to client, opcodes, LU, etc. The startuptimeout is similar to that one used in zeroing, but the shutdowntimeout is different because it is a single timeout that includes thetime needed to install plus shutdown....
Add 'install_image' to bootstrap, CLI, opcodes, and queries
Fix 'IsInstanceRunning' to handle empty 'instance_info'
Instance info might return an empty dictionary in case the instancedoes not exist. Fix 'IsInstanceRunning' to handle that fact.
Fix var reference from 'instance' instead of 'lu.instance'
... given that 'lu.instance' does not exist in all LUs.
Fix exception message to be consistent with others
Filter disallowed compression tools
This patch makes import and export operations fail when anon-whitelisted compression type is used. The whitelisted tools have tobe specified manually via"gnt-cluster modify --custom-compression-tools=<tools>".
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Add the compression tools parameter
This patch makes the myriad of changes necessary for the compressiontool parameter to be added. The filtering of compression tools forsuspicious entries has been added for this exact purpose.
Clean up remove_locks
Locks are now identified by their name, and no longera resource in the master daemon. In particular, there isno more need for explicitly removing the locks; freeingthem is enough.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Fail in replace-disks if attaching disks fails
Previously, if attaching the new secondary during a replace-disksoperations failed, only a warning was emitted. The subsequent sync-disksoperation cannot finish in such a case, however.
Therefore, this patch changes the warning into an error. This way it's...
Lift the Disk objects from the Instances
This patch replaces 'instance.disks' with 'GetInstanceDisks' everywherein the codebase. From now on, the function 'GetInstanceDisks' from theconfig file has to be used in order to get the disks of an instance.Also the functions 'AddInstanceDisk'/'RemoveInstanceDisk' have to be...
Add 'instance_disks' parameter to 'CreateDisks'
Modify 'CreateDisks' function to accept the disks of an instance as anextra parameter. Since the instance may not have been saved to theconfig file yet, 'CreateDisks' can not query the config file for the...
Fix a 'map' application in ImageDisks
In the first branch of 'if' the 'disks' variable is of type'list of tuples' whereas in the second branch ('else') it isof type 'list of disk objects'. Fix the second one to agreewith the first.
Signed-off-by: Ilias Tsitsimpis <iliastsi@grnet.gr>...
Get MapLVsByNode from config
Use 'GetInstanceLVsByNode' from config file instead of Instance'sMapLVsByNode method.
Also remove all_lvs computation from config's 'AddInstance' method. Inorder to compute the lvs we have to use the 'GetInstanceLVsByNode'...
Get instance's nodes from config
Use 'GetInstanceNodes' to get the nodes of an instance, instead of the'all_nodes' Instance property.
Signed-off-by: Ilias Tsitsimpis <iliastsi@grnet.gr>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Get instance secondary nodes from config
Use 'GetInstanceSecondaryNodes' to get the secondary nodes of aninstance, instead of the 'secondary_nodes'.Annotate 'secondary_nodes' in rpc so that backend has access to it.
Override disks in BuildInstanceHookEnvByObject
The function BuildHooksEnv in LUInstanceRemove, calls the functionBuildInstanceHookEnvByObject which needs to query the config to getthe secondary_nodes/disks of an instance. The function BuildHooksEnvwill be called before and after the Exec function. The second time...
Fix the order of configuration calls
Since _SetDrbdHelper calls methods on self.cfg, it's necessary to updatethe configuration before it is called.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Merge branch 'stable-2.11' into master
Fix failed DRBD disk creation cleanup
When creating a DRBD disk, Ganeti reserves minor numbers on a per-nodebasis. In case of a failed disk creation, these reservations should bereleased. During the name/uuid refactoring, the invocation of thefunction that releases the minors was not updated, resulting in no...
Hooking up verification for shared file storage
As for the cluster modify, it was also forgotten tohook up the verification of the shared file storagepaths despite all infrastructure was done already.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix --shared-file-storage-dir option of gnt-cluster modify
While all infrastructure to make shared-file storageruntime-configurable was already submitted, the actualsetting of the path was forgotten. This patch fixes it.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Add missing save/re-read cluster configuration
.. in LUClusterSetParams. Since _SetFileStorageDir updates the 'cluster'variable directly, we need to re-read it and then save later (the othercalls use the configuration directly).
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Complete zeroing
This patch will do all that is necessary to allow the helper VM to dowhatever it will do with the zeroing image in place - which is,hopefully, to zero out the disks of the instance.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Add timeout parameters to gnt-backup export
This patch adds the two parameters to gnt-backup export, documentingtheir meanings in the manual file as well.
Add zeroing function shell
This patch moves towards the actual zeroing by providing a functionthat determines the necessary size and creates and destroys a temporarydisk.
Add image size estimation function
To create a temporary disk, it is necessary to know just how big thisdisk must be. This patch adds a function that retrieves thisinformation for both data sources - a URL and a file.
Modify failing assert
The assert stating that the disk did not have to be activated madesense once the instance's disks were never completely deactivated. Withzeroing, the assumption no longer holds true, and should be replaced bya check that the disks of the instance are indeed active....
Introduce class for temporary disk creation
The instance zeroing requires that a virtual appliance capable ofperforming it is used, and short of provisioning another VM, the bestGaneti can do is reuse the virtualization environment of the instance.To do this, a disk is needed to host the OS performing the zeroing, and...
Factor out and alter the instance running check
Examining whether an instance is running is useful outside the instancestate check, especially as the check relies on the admin_state recordedwithin the configuration, which the code treats as the desired rather...
Lock node if zeroing is used in LUBackupExport
This patch adds a node lock if zeroing is used, preventing anyoperations that might interfere with the amount of available space.
Add the zeroing-image option
This patch adds the zeroing-image option to gnt-cluster and theOpBackupExport params. The many changes are all minor, yet necessary.
Factor out image validity check
The image validity check has been implemented as a part of the OSinstalls, yet it could be useful for the zeroing image as well. Thispatch factors the utility out.
Add HV parameter checks
The first version of zeroing requires two things: that theself-initiated shutdown of the instance can be detected, and that theinstance boots from the first disk supplied to it. This patch checksfor these by examining the hypervisor parameters....
Add the zero-free-space option
Although unused for now, add the zero-free-space option to the backupexport opcode, and add a rudimentary check for whether it should beused.
Account for NODE_RES lock in opportunistic locking
This patch extends the opportunistic locking to take both theNODE_RES and NODE locks into account when selecting nodes to be usedby the iallocator.
Signed-off-by: Gerard Oskamp <gjo@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Fix request flooding of noded during disk sync
If a user manually pauses the DRBD sync, Ganeti started to poll the nodedaemon without any waiting between request. This caused heavy load onthe node. This patch fixes this by waiting at least 5 secondes between...
Update metadata on create, import, reinstall, modify
Note that create and import are the same LU, therefore, there are atotal of 3 calls, not 4.
Add helper function to modify metadata
This function is responsible for transforming the instance into a dictand optionally overriding the OS parameters (public, private, secret).Also, in case of failure it issues a warning.
Check for LVM-based verification results only when enabled
This patch fixes a little glitch in 'gnt-cluster verify'.If LVM-based storage was disabled, it would still checkLVM-related verification results and print a confusingerror message.
Fix output of gnt-instance info after migration
After migrating a DRBD based instance, the output of gnt-instance infowas wrong wrt. DRBD minors. This patch fixes the output in such cases.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Acquire config lock for RPC calls that check consistency
Since LUClusterVerifyGroup issues 3 RPC calls to get information fromnodes, and during every of these calls, the configuration can bemodified by somebody else, it's necessary to acquire a shared lock on...
Fix configuration calls for LUBackupExport
Re-read the instance in-memory objects from WConfd after calls thatchange its state.
Fix configuration calls for LUInstanceRename
Re-read the instance in-memory objects from WConfd after callsthat change its state.
Fix configuration calls for LUInstanceReboot
Fix configuration calls for LUInstanceReinstall
Fix configuration calls for LUInstanceShutdown
Fix configuration calls for LUInstanceStartup
Instance storage: Mark functions that modify instances
.. in the configuration with comments.
Also re-read an instance object where needed.
Fix calls to Update in TLMigrateInstance
Instead of modifying the primary node on an object and calling 'Update',a new method was added to ConfigWriter.
Fix calls to Update in LUNodeSetParams
Add calls to Update after modifying the node's configuration and movesome of the modifications so that they don't conflict with other callsto ConfigWriter.
Fix calls to Update in LUInstanceSetParams
In particular, move the call to Update so that it doesn't conflict withcalls to ConfigWriter methods that change the configuration state.
Fix calls to Update in LUClusterSetParams
Since this LU intersperses calls ConfigWriter methods with modifyingobjects directly, it's necessary to call 'Update' and/or re-read them atappropriate places.
Also a new ConfigWriter method is created for modifying the instance...
Make configuration per job/thread
Previously there was one shared configuration object for all jobs,threads and other tasks. This patch creates separate ConfigWriteinstances for distinct jobs/threads.
All exported methods of ConfigWriter are now wrapped in calls that...
Use explicit configuration for nodes in GanetiContext
This way, each call to AddNode or RemoveNode uses a passed configurationobject that belongs to the appropriate job.
This is required to subsequently get rid of the single sharedConfigWriter object....
Move utility functions for candidate certs. to ConfigWriter
In particular AddNodeToCandidateCerts and RemoveNodeFromCandidateCerts.
Calling 'cfg.Update(cluster)' causes problems in WConfd, asit doesn't operate on a shared configuration object any more....
Show OS variant information in gnt-os info
Currently, the non-standard/modified per-OS hypervisor parameters, orOS specific parameters can be listed only by the 'gnt-cluster info'command, which is a non-standard place to show them. Extend the'gnt-os info' command to display the available/supported OS variants...
Skip rename when OS scripts are absent
When an instance does not have OS scripts because, for example, ituses an OS image, do not rename the instance after an import.
Use raw disks in import/export when OS scripts are absent
When an instance does not have OS scripts because, for example, it hasan OS image, then the import/export should not try to run the OSscripts. Instead, it should use raw import/export.
Add a thread ID to the WConfd client id
This allows to distinguish threads that don't have a job id, which isneeded for answering queries.
Since Python thread IDs aren't guaranteed to be unique, in future it'dbe preferable to use a different, unique identifier....
Remove unused functions to check OS variants
... as this is now performed on the node.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Remove calls to 'CheckNodeHasOS'
... because 'CheckOSParams' already checks the OS variant.
Add 'force_variant' to RPC 'os_validate'
Move function 'CheckOSVariant' to the node and add parameter'force_variant' to RPC 'os_validate', thus making the node verify theOS variant together with the rest of the OS params.
Remove SSH copyfile from LU and assume the file exists
According to Ganeti design, files should not be copied from master tonodes and instead they are assumed to exist and it is the user'sresponsibility to assure that the file does exist.
Fix OS image detection on master
Make mcpu acquire WConfD locks
So far, the mcpu acquires locks that live in memoryof masterd. This design does not fit with our jobs-as-processesgoal. So make mcpu acquire the corresponding locks in WConfDinstead.
Note that this implies changes in various other files that call...
Instance reinstall with OS images
OS images in 'LUInstanceCreate' and OS scripts optional
Instance create with OS image
Extend 'LUInstanceCreate' to image the instance's first disk if an OSimage is specified via the OS params. If the OS image is a file, itwill be copied to the node via SSH. If ths OS image is a URL, it willbe passed directly to the node, which will then download the file....
Function to check if the OS image parameter is valid
Add helper function to check if the OS image parameter contained inthe OS parameters of an opcode is valid.
Function to image disks while ensuring that disks are paused
Function to remove instance if disks are degraded
Fix docstrings
Fix several docstrings.
Make gnt-debug delay interruptible
The gnt-debug delay command could be useful as a means of acquiringlocks for testing purposes. In practice, to be useful it should beinterruptible, otherwise we risk race conditions or long delays.
This patch follows the examples of the move-instance command and the...
Factor Unix domain socket creation into helper class
As the delay class will also have to start using domain sockets,extract the functionality into a helper class.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Handle incorrect duration more elegantly
The previous version of the LUTestDelay opcode relied on the utilityfunction complaining about the negative duration. As this function hasbeen removed for now, do the check ourselves, and issue a moreappropriate exception....
Make gnt-debug delay command run in parallel
The gnt-debug delay command executes the delay first on the master, andonly then on all the other nodes, causing a significant delay. Thispatch makes the command treat the master as it would all other nodes....
Remove duplicated '_CheckOSVariant'
It seems '_CheckOSVariant' was moved from 'ganeti.cmdlib.instance' to'ganeti.cmdlib.instance_utils' but the source was never deleted. Thispatch deletes the source copy if this function.