OS images in 'LUInstanceCreate' and OS scripts optional
Instance create with OS image
Extend 'LUInstanceCreate' to image the instance's first disk if an OSimage is specified via the OS params. If the OS image is a file, itwill be copied to the node via SSH. If ths OS image is a URL, it willbe passed directly to the node, which will then download the file....
Function to check if the OS image parameter is valid
Add helper function to check if the OS image parameter contained inthe OS parameters of an opcode is valid.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Function to image disks while ensuring that disks are paused
Function to remove instance if disks are degraded
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
RPC 'blockdev_image' to image devices
Add RPC 'blockdev_image' that uses 'ganeti.backend.BlockdevImage' todump an image to an instance's disk device, optionally downloadingthat image.
Helper function to image a device by downloading or dumping
Add 'BlockdevImage' which downloads a file and dumps it to aninstance's disk if the path is a URL, otherwise it dumps the filedirectly to the instance's disk.
Helper function that downloads an image and dumps it to disk
Generalize 'WipeDevice' to 'DumpDevice'
Helper functions to get and update OS image from OSParams
Add 'GetOSImage' and 'PutOSImage' which handle the OS image key in theOS parameters dict.
Fix export order according to definition order
Fix docstrings
Fix several docstrings.
Reuse method to parse name from OS 'name+variant' string
Merge branch 'stable-2.11' into master
Merge branch 'stable-2.10' into stable-2.11
Merge branch 'stable-2.9' into stable-2.10
Fix error introduced during merge
A parameter was lost while resolving a conflict in the signature of a function.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
gnt-cluster copyfile: accept relative paths
If, on the command line, the argument to gnt-cluster copyfile isa relative path, consider this a shorthand for the correspondingabsolute path.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Merge branch 'origin/stable-2.8' into stable-2.9
Let WConfd distribute SSConf to nodes
.. and remove the corresponding code from lib/config.py.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Let the SSConf RPC server side handle lists
Since on Haskell side we represent SSConf as list of lines, let thePython side understand it as well.
Let WConfd distribute the configuration to MCs
.. and remove the distribution from lib/config.py
Add a new RPC server call for uploading a single file
The server side processes the request exactly the same as for"upload_file".
Unlike "upload_file", the new call "upload_file_single" declares allrequired fields without requiring additional preprocessing....
Add more meaningful error messages to asserts in vcluster
.. to simplify debugging of RPC calls.
Improve RAPI detection of the watcher
If the watcher is not allowed to access RAPI, it doesn't mean that it is deadand needs to be restarted.
Fixes Issue 752.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Enable a timeout for instance shutdown
Add the timeout parameter to the StopInstance function of the hypervisor baseclass and to all its implementations.
Also, change the tests as required by this change.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Allow KVM commands to have a timeout
Modify the function that sends commands to the KVM monitor so that it ispossible to specify an optional timeout after which the command is killed.
Allow xen commands to have a timeout
Modify the function that runs Xen commands so that it is possible to specify anoptional timeout after which the command is killed.
Fix wrong docstring
Fields must be the final elements in an epytext string.
Use correct lockfile for gnt-debug wconfd
As jobs are currently running in masterd, use the masterd livelockfile.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add utility to guess livelock file for an owner
As livelock files are constructed in a systematic manner,we can guess what the livelock file for a given owner is.While this will not necessarily work perfectly, it will beuseful to simplify direct debugging of WConfD....
Make masterd create a livelock file
...so that it can request resources from WConfd.
Rename setup_queue to setup_context in masterd
...as this function sets up a much richer context than justthe job queue, including the current lock management.
Add utilities for liveliness lock files
To request resources from WConfD, requesters have to providethe name of a file they own an exclusive lock on. In this way,their death can be detected. Add utility functions to obtainsuch a file name.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Ensure the existence of LIVELOCK_DIR
Add a path to store the lock files presented to WConfD
When requesting resources from WConfD, a file has to bepresented where an exclusive lock is owned on, so thatWConfD can detect when the requester dies. Add a pathto a directory where these files are kept in....
Convert int to float when checking config. consistency
When reading the configuration file from RPC JSON, values without afloating point are parsed as 'int', not as 'float', and later theconsistency check fails.
This patch adds an automatic conversion from 'int' to 'float' during...
Align timestamps in gnt-job info
This patch aligns the timestamps output as a part of gnt-job info, andperforms minor refactorings in the process.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add alignment support to PrintGenericInfo
Aligning dictionary entries makes no difference to a YAML parser, butmakes the output much easier to read and compare. This patch adds thepossibility of specifying alignment groups to ordered dictionaryentries....
Make gnt-job info output valid YAML
This patch changes gnt-job info to use standard functions defined incli.py, and output valid YAML.
Make PrintGenericInfo handle tuples better
The PrintGenericInfo function in cli.py did not handle tuples ascontainers of items, making it impossible for these to be deserializedautomatically when a YAML parser is used. This patch adds separatehandling of tuples, including inlining them for readability when...
Make gnt-debug delay interruptible
The gnt-debug delay command could be useful as a means of acquiringlocks for testing purposes. In practice, to be useful it should beinterruptible, otherwise we risk race conditions or long delays.
This patch follows the examples of the move-instance command and the...
Add the interruptible option to gnt-debug delay
This patch allows the opcode option to be used through the gnt-debugclient.
Factor Unix domain socket creation into helper class
As the delay class will also have to start using domain sockets,extract the functionality into a helper class.
Fix minor accidental concatenation
Handle incorrect duration more elegantly
The previous version of the LUTestDelay opcode relied on the utilityfunction complaining about the negative duration. As this function hasbeen removed for now, do the check ourselves, and issue a moreappropriate exception....
Make gnt-debug delay command run in parallel
The gnt-debug delay command executes the delay first on the master, andonly then on all the other nodes, causing a significant delay. Thispatch makes the command treat the master as it would all other nodes....
Fix typo in RAPI client utility
Remove duplicated '_CheckOSVariant'
It seems '_CheckOSVariant' was moved from 'ganeti.cmdlib.instance' to'ganeti.cmdlib.instance_utils' but the source was never deleted. Thispatch deletes the source copy if this function.
Use node UUIDs for executing LU hooks
LUNodeAdd, the only LU using a node name still, is changed to overwritePreparePostHookNodes() and use node UUIDs only as well.This allows to remove the support for 3-tuples as results ofBuildHooksNodes() and removes the translation to node names....
Add PreparePostHookNodes to LUs
This method can be used to alter the list of node UUIDs on which posthooks are executed. PreparePostHookNodes is called after Exec, so LUscan use data only known after the execution of the LU.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Fix error propagation in post-commit hooks
An error in the post-commit hooks could not be propagated correctly and couldresult in e.g. the return code of gnt-cluster verify to be 0 even in presence ofan error in its output.
Fixes Issue 744.
Add listlocks to gnt-debug wconfd
So that wconfd's locking can be debugged directly.
Stop watcher from restarting down instances during an opcode
This patch changes the watcher to check whether an instance that isdown is also locked by some LU before attempting to restart theinstance. Without checking the lock status, the watcher could think...
Remove unused import in rpc/transport.py
.. which got there by mistake.
Retry luxi/wconfd RPC calls if the connection is closed
Since the daemon can decide to close a client connection after atimeout, the client needs to be able to automatically reconnect.
This patch introduces this functionality into the RPC client:If an attempt to send data fails on 'Broken pipe', it's retried one more...
Allow cluster mac prefix modification
Extend LUClusterSetParams to allow the modification of the clustermac-prefix setting in 'gnt-cluster modify' command.
This fixes part of issue 239.
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Show mac prefix setting in gnt-cluster info
Include mac-prefix setting in the output of 'gnt-cluster info'command.
Setting correct permissions of client cert (split-user)
This patch makes sure that the client certificate getsthe right permissions and owner when created. Additionallyit enhances the 'ensure_dirs' script to correct thepermissions in case they are broken for whatever reason....
Add a command to gnt-debug to test various aspects of wconfd
For debugging purpose, support direct communication to WConfD fromthe commandline for some of its commands. For the time being, supportthe echo command.
Add some whitespace to fix formatting
Some error messages were lacking some spaces between linesto make it more readable.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Consider old client cert only when available
This fixes a bug which occurred only after upgradingfrom 2.10 to 2.11. During the cluster renew-cryptooperation, Ganeti tries to include the old certificatein the candidate map while it is providing newcertificates. This failed when there was no certificate...
Fix return of 'Validate'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add reason for job pickup to the trail
Add a new entry in the reason trail when a job is picked up by MasterD from thehard drive, after LuxiD put it there.
Note that the signature of NameToReasonSrc is changed in an incompatible way,although it's a public method because in this commit we also change its only...
Make the AddReason method public
It will need to be accessed from outside the class too in one of the nextcommits.
Let config.py use WConfd for reading/writing the config
Currently it only relays the reads/writes to the file to WConfd,everything else yet remains in config.py.
Also if the 'ConfigWriter' is opened in "offline" mode (like inbootstrap.py), it doesn't use WConfd and resorts to the original...
Start WConfd temporarily during master failover
.. in order to update the configuration and distribute ssconf, beforestarting the daemons by the scripts.
Merge branch 'origin/stable-2.10' into stable-2.11
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Merge branch 'origin/stable-2.9' into stable-2.10
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Make gnt-debug locks display fake job locks properly
When a job is dependent on other jobs, a fake lock is created whosepending entry contains a list of job ids waiting on the job. gnt-debuglocks did not expect the job ids to be ints, crashing when encountering...
Make NiceSort treat integers well
NiceSort is invoked on arrays that may contain strings, but in othersituations can contain ints as well. As this surprisingly makes sense,add a tiny modification to make NiceSort work in these conditions.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Merge branch 'stable-2.8' into stable-2.9
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Fix expression describing optional parameters
The NIC's network and vlan are also newly added, hence need to beconsidered optional to remain backwards compatible.
Let the instance's tuple of nodes start with the primary
Before the tuple of nodes of an instance was created from a set, listingthe nodes in alphabetical order. This patch ensures that the primarynode is always the first one in the list.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Check the existence of system users and groups at bootstrap
Before, if any of these were missing, the creation of a cluster failedand the cluster remained in an inconsistent state, without thepossibility to destroy it or to re-create it (#603).
This patch calls 'GetEnts' during bootstrap, which tries to read all...
Conflicts: lib/cmdlib/instance.py: manually apply 0973f9ed on...
Improve job status assert affected by race condition
In the sliver of time between choosing a waiting job to be executed andtrying to acquire locks for its execution, the status of the job can bechanged to canceling. An assert checking the job status neglected to...
Export and import Disk/NIC name
Name of Disk/NIC were not exported during backup until now.Use the exported info during gnt-backup import.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix backup import in case NIC is inside a network
Network UUID is written in .ini file during backup exportbut is not used by _ReadExportParams(). This patch fixes it.
Please note that in case a network is given, link and mode shouldnot be included in NIC options....
Override get() method of ConfigParser
During backup import/export SafeConfigParser() is used tosave/restore instance's configuration. There is a possibility if anexport is done with a different Ganeti version, a specific value notto be saved during export (e.g. the NIC/Disk name) but still...
Smooth renewal of client certificates
This patch fixes another chicken-and-egg problem whichoccurred when the node certificates get renewed. Whenrenewing a node certificate, the previous certificatehas to be used to update the configuration. To address...
Constant for instance communication network mode
Create a new constant to hold the instance communication network modeas this constant will be necessary during the QA, and update thegeneral documentation about the constants related to the instancecommunication mechanism....
Add '-c | --instance-communication' flag to instance modify
Enable/disable instance comm via 'gnt-instance modify'
This patch adds the logic necessary to enable/disable the instancecommunication in a running instance via 'gnt-instance modify'. Withinstance communication enabled, the instance gets a new NIC that is...
Refactor instance comm NIC name creation
Refactor name creation for the NICs used in instance communication.These names are generated based on a prefix and the instance name.Also, these names must be unique within a single instance.
Fix copy of NIC objects to be consistent with the other call
... which can be found just right below in the same module.
Fix whitespace and typos in comments
Use node UUID as client certificate serial number
It turns out, that some implementations of OpenSSL are morepedantic in checking the certficates than others. In thisparticular case, the SSL connection could not beestablished when the serial number of the certificates...
Revert "Disabling client certificate usage"
This reverts commit 45f75526b848, which was introduced totemporarily disable the implementation of SSL clientcertificates. As this patch series fixes the reason forthe disabling, we are rolling back the patch....
Fix an ambiguity in the documentation for GetNodesSshPorts
This ambiguity was introduced by adding the WConfd client.
Add the Python client for WConfD
The client combines the abstract client class and the WConfDstub to provide a Python interface to WConfD.