(2.10) Workaround for monitor bug related to greeting msg
QMP may return multiple greeting messages upon connection.This is reported on qemu-devel. The fix is one-liner butuntil it get's released this is a quick and dirty workaroundthat flushes the client's buffer after getting the first...
(2.10) hotplug: Verify if a command succeeded or not
Just after issuing _CallHoplugCommands() we invoke_VerifyHotplugCommand() which parses `info pci` resultand searches for given PCI slot and device id.
If we previously had removed a device but it is still there...
(2.10) hotplug: Each hotplug qemu commmand with a separate socat
Previously we issued one socat command with two "\n" separatedactions (e.g. netdev_add ...\ndevice_add...)
After having observed a strange monitor behavior [1] splittingthose commands and introducing a sleep time in between, may reduce...
(2.9) Gracefully handle degraded instances in verification
The current code assumes that every instance either is of typediskless or has at least one disk. However, with the option toremove individual disk degraded 0-disk non-diskless instancescan occur. While such instances usually are not useful, Ganeti...
(2.9) Be aware of the degraded case when cleaning up an instance
In the case of a degraded file-based instance, the file storage directoryfor that instance cannot be obtained by looking at the first disk. Usethe standard location, computed from first principles, in this case....
(2.9) Preserve disk basename on instance rename
For file-based instances, upon rename, the directory containingthe instance disks is moved. Therefore, the basename needs tobe preserved in this case. Fix this. Note that so far, thisworked by accident as before 94e252a3 file names used to be...
(2.9) Assign unique filenames to filebased disks
With the new format for cmdline arguments, the user is able to add adisk to an instance at a specific index. But filebased disks' filenameshave the form "{0}/disk{1}" where '{0}' is the file_storage_dir and...
(2.9) Remove deprecated _ERROR_DATA_KEY in QMP
Commit de253f14 of QEMU repo "BREAKS QMP's compatibility forthe error response" as it removes "data" key from qmp errorresponse messages. To this end we only log "class" and "desc" values of the message....
(2.10) Include target node in hooks nodes for migration
In case of DRBD, hooks run on both primary (source) and secondary(target) nodes. To get the same behavior for DTS_EXT_MIRROR, where wedo not have secondary node, we should explicitly add target node to...
(ipfix) Allow NICs with mode=routed and ip=none
This allows us to define an IPv6 only network (currently with adummy IPv4 network) with connectivity mode 'routed' and letthe NICs inside this network have only IPv6 address (auto-generatedfrom their MAC and the network's prefix)....
(ext) Do not remove disks if --keep-disks and DT_EXT
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
(ext) Override disk_cache hvp with disk param in ext
This is a hack that brings disk_cache hypervisor paramat disk level. This is supported only for disks of exttemplate that allow arbitrary params. The correspondingparam that should be used is 'cache'....
(ext) Pass disk geometry in kvm command
Currently we allow this feature only for ext templates thatallow arbitrary params per disk. If both 'heads' and 'secs'params are given then 'cyls' is calculated from the disk size.
TODO: export these params in IDISK_PARAMS since the can be...
(ext) Export logical_id info to hooks env
DISK_ID as the second part of logical_id
(b64) Use base64 encoding in networks' bitarrays
This is needed for GRNET'S production environment.
(snap) Snapshot support for ExtStorage
Extend existing RPC params with the snapshot name andadd allow snapshot not only for LVM but also for EXT.
(grnet) Remove deprecated idx slot from NIC/Disk objects
This code will run when reading old kvm runtime files createdby hot-plugging related code back in Ganeti 2.6.2
(grnet) Enable -device option for old disks upon migration
Change _GenerateDeviceKVMId() to take device index as anextra argument. In case device has uuid and pci infowe return proper device id. In case idx is given we returna dummy id (e.g. hotdisk-0). Otherwise we raise an Exception....
(grnet) Move disk options before nic ones in kvm command
Older versions of Ganeti did ensure that during startupdisk devices will be inserted before nic devices in PCIconfiguration space. KVM inserts devices to PCI slotsdepending on the order of command line options....
(2.10) Introduce _UpgradeSerializedRuntime() method
This method is invoked during _AnalizeSerializedRuntime() and ismeant to modify runtime files in the way cfgupgrade does forconfig.data. This could remove deprecated fields, change theformat of the file, add/remove sections, etc....
(2.10) Export NIC's UUID and name to network scripts
In case of kvm None values are not allowed in env dictso we have to add name only if not None.
In case of Xen since we are writing on a file thatis going to be sourced we should not add INTERFACE_NAME=None....
(2.10) Use HooksDict() to export network options in Xen
Remove duplicate code that exports network options to environmentvariables.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>...
(2.10) Export tags via GetTags() to network scripts
Use GetTags() instance method in order to export instance tagsto NIC configuration scripts and files of kvm and xen hypervisors.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>...
(2.10) Introduce --hotplug-if-possible option
This will be useful for an external entity using RAPI thatwants to modify devices of instances.
The common use case for that is:"I want to add a NIC/disk to an instance. If it is runningthen try to hotplug the device. If not, then just add it to config."...
(2.10) RAPI: Make use of request_body in Reboot/Remove
Until now, the shutdown_timeout parameter could only be passed toShutdownInstance() inside kwargs, causing it to be included in therequest body. Based on that, extend RebootInstance() andRemoveInstance() so that they can take extra arguments (e.g....
(2.10) RAPI: Pass depends body arg (if any) to opcode
Most rlib2 classes override the default _GetDefaultData() method withcustom methods that parse a request's body and query args and return a(body, specific_static) tuple eventually passed to FillOpCode()....
(2.10) Allow instances to obtain externally reserved IPs
The administrator should be able to assign an externally reserved IPto a Ganeti instance manually, if desired. Currently this is notsupported. External reservations should act as holes in the pool and...
(2.10) Check if hotplug is supported in CheckPrereq
Introduce new RPC hotplug_supported that invokes the correspondinghypervisor's method which checks if hotplug is generallysupported. Call this RPC early in CheckPrereq() and abort ifhotplug is not supported. Currently only KVM hypervisor with...
(2.10) Mark cluster's IPs as externally reserved
Currently, upon network creation, nodes' and master's IPsare reserved in the pool.
This leads to pool reservations ('X' in map) that cannot be changedafterwards, although they may need to (e.g. in case of node...
(2.10) SimpleRetry on BlockDev.Remove()
Sometimes, upon disk removal, corresponding file descriptorsare kept briefly open by various processes (hypervisor, blkid, etc.).With this patch, we retry several times before raising the appropriateerror, thus making disk removal more robust against those corner cases....
(2.10) drive_del after device_del in disk hot-remove
This is needed so that qemu process really releases thecorresponding file descriptor and thus we can dmsetup/lvremovethe device without errors.
(2.10) Minor changes regarding hotplug support
(2.10) Fix in RPC
Use _SingleDiskDictDP() instead of _ObjectDict() for serializinga disk.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
In stable-2.10 rpc methods take node as first argument....
(2.10) block_devices -> kvm_disks renaming
and a couple more styple fixes.block_devices are misleading so rename them to kvm_disk (just like kvm_nics)
(2.10) Make hotplug related method raise HotplugError
Hotplug is not supported by default and thus we raiseHotplugError() for hotplug related methods.If a hypervisor wants to support it, we must override VerifyHotplugSupport()and implement the Hot* method....
(2.10) Some fixes in _GenerateKVMBlockDevicesOptions()
(2.10) Hotplug: client support
Add --hotplug option. Only used in OpInstanceSetParams.If this is omitted, modifications become effective after reboot.
Ask user confirmation in case NIC modify + hotplug because it willbe done via removing old NIC (and the corresponding tap) and adding...
(fix) SetDiskID() before blockdev_assemble()
Before blockdev_assemble we call SetDiskID() in order to fill upphysical_id of disk needed by bdev.Assemble and eventually byconstructor of each BlockDev.
This is not needed in 2.10 since physical_id has become deprecated....
(2.10) Hotplug: cmdlib support
Hotplugging is done by functions invoked by ApplyContainerMods(). Inorder hotplugging to take place the --hotplug option must be passedotherwise the modifications will take place after reboot.
NIC hotplug supports add, remove and modify. The modify is done by removing...
(2.10) Hotplug: rpc support
Introduce new RPC that eventually invoke hypervisor specifichotplug functions. In order to be generic it has the followingarguments: device type, action, device, extra info, seq.Device type can be NIC or DISK, action can be ADD, REMOVE,...
(2.10) Return link_name in blockdev_assemble rpc
Until now this RPC returned only dev_path. Since we use it inhotplug we have to know the simlink of the device so that wepass it to the corresponding hypervisor command and include itin block_devices entry in runtime files....
(2.10) Extract socket related code from QmpMonitor class
Separate unix socket related code from QmpMonitor class andmake the latter extend the newly introduced one: MonitorSocket
(2.10) Introduce hotplug methods (hypervisor inteface)
4 new methods: HotAddDevice, HotDelDevice, HotModDevice, HotplugSupportedAll these are the hypervisor interface with the backend.In general they read current runtime info, do the actual hotplugand update the runtime file....
(2.10) Helper methods that check for hotplug support
Hotplug is currently not supported for:
- KVM < 1.0 - existing devices in the cluster - python-fdsend module is not installed (NIC hotplug) - chroot (Disk hotplug) - security mode other than None (Disk hotplug)...
(2.10) Use -device with pci info if possible
Extend kvm_cmd with -device option for the case of paravirualDisks and NICs.
(2.10) Helper methods for PCI slots and device ids
Device naming:QEMU monitor expects devices to be uniquely named. Device ids derivefrom the following function:kvm_devid = <device_type>-<part of uuid>-pci-<pci_slot>Device ids must be reproduce-able when we want to remove them....
(fix) Remove uri slot from block_devices
In stable-2.10 we have userspace access mode for rbd. Backendcalculates uri along with link and dev_path. Thus we havethree items in block_devices and kvm_disks tuples.
We this patch we remove the third value....
(2.10) Prepare block_devices runtime entry
With this patch we add another entry in runtime files along withkvm_cmd, kvm_nics, and hvparams. block_devices that used to beencapsulated inside kvm_cmd, live now separately just like nics dobut in tupples of (L{objects.Disk}, dev_path)....
(2.10) Introduce _GenerateKVMBlockDevicesOptions()
Simply move logic that generates block device relatedoptions out of _GenerateKVMRuntime().
NOTE: Back-port corresponding commit from stable-2.10
- 2.8 does not have user-space access support for rbd
(2.10) Prepare kvm options to support -device for disks
Newer kvm version support -device driver,... option where availabledrivers can be found by running kvm -device ? command. Currentlyganeti uses `if` property of driver to define the disk type to use...
(2.10) Hotplug: objects and constants additions
Add pci slot in NIC/Disk objects. This slot will be used onlyby hypervisor code. Currently only KVM will use it and store ittemporarily in runtime files.Add HOTPLUG_* constants to define device types an hotplug actions....
(2.9) Make NIC setup more flexible in case of Xen
For every NIC write down a file that contains NIC'sinfo (MAC, mode, link, network details), that canbe sourced by any vif script. The file location is:/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>....
(2.8r) Workaround for Issue 621
Upon LUNetworkDisconnect() and LUNetworkConnect() try to acquireall cluster's instances.
By that _LS_ACQUIRE_ALL acquire mode is set and not_LS_ACQUIRE_EXACT and thus the deleted lock does cause any problem.
NOTE: This workaround is not merged upstream. They prefer to have...
Improve RAPI detection of the watcher
If the watcher is not allowed to access RAPI, it doesn't mean that it is deadand needs to be restarted.
Fixes Issue 752.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Enable a timeout for instance shutdown
Add the timeout parameter to the StopInstance function of the hypervisor baseclass and to all its implementations.
Also, change the tests as required by this change.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Allow KVM commands to have a timeout
Modify the function that sends commands to the KVM monitor so that it ispossible to specify an optional timeout after which the command is killed.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Allow xen commands to have a timeout
Modify the function that runs Xen commands so that it is possible to specify anoptional timeout after which the command is killed.
Fix wrong docstring
Fields must be the final elements in an epytext string.
Fix expression describing optional parameters
The NIC's network and vlan are also newly added, hence need to beconsidered optional to remain backwards compatible.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Improve job status assert affected by race condition
In the sliver of time between choosing a waiting job to be executed andtrying to acquire locks for its execution, the status of the job can bechanged to canceling. An assert checking the job status neglected to...
Export and import Disk/NIC name
Name of Disk/NIC were not exported during backup until now.Use the exported info during gnt-backup import.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix backup import in case NIC is inside a network
Network UUID is written in .ini file during backup exportbut is not used by _ReadExportParams(). This patch fixes it.
Please note that in case a network is given, link and mode shouldnot be included in NIC options....
Override get() method of ConfigParser
During backup import/export SafeConfigParser() is used tosave/restore instance's configuration. There is a possibility if anexport is done with a different Ganeti version, a specific value notto be saved during export (e.g. the NIC/Disk name) but still...
Add correct locking of master node to gnt-debug delay
The gnt-debug delay command required locks for all nodes except themaster - this patch fixes the issue by adding master to the lockswhenever needed.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add job id type assert to jqueue.py
While the changes introduced in previous patches should stop any jobid parameters reaching the queue as strings, add an assertion here tocatch any strings making it through.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Add job id transformation/check to Luxi Python client
This patch adds checks to the Luxi client, making sure that job idsare converted from strings to ints before being passed on, or that anerror is reported.
Improve backwards compatibility of Issue 649 fix
Commit e6e4ff4cf8d0100f331f94f7a27aa1e03a5d0e7d fixed Issue 649 by switching theseparator for usb_devices from comma to space. That solved the problem withthe command line, but RAPI was able to work with commas too, so, for backwards...
Change usb_devices separator to whitespace
The usb_devices parameter was using comma as a list separator, but this cannotwork because comma is already used as the hypervisor parameter separator.
Change it to use whitespace as a separator, in accordance to what already done...
Add support for blktap2 file-driver
Newer Xen versions use blktap2 instead of blktap. This patch adds supportfor it in Ganeti.
Fixes Issue 638.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Fix RAPI network tag handling
The network tags were absent from an if check used to actually listtags. The patch fixes the oversight, and adds a proper error message incase the issue occurs again for a new tag type.
Make network tags searchable
This patch adds the network tags to the tags searched by gnt-clustersearch-tags, and in the process cleans up the code slightly.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Support reseting arbitrary params of ext disks
If param=default and the param already exists then we removeit from params dict. This is stolen by GetUpdatedParams() whichis used for hvparams modification/inheritance.
This means that 'default' value is not accepted for an arbitrary...
Allow modification of arbitrary params for ext
Disks of ext template are allowed to have arbitrary parametersstored in the Disk object's params slot. Those parameters can bepassed during creation of a new disk, either in LUInstanceCreate()or in LUInsanceSetParams(). Still those parameters can not be...
Do not clear disk.params in UpgradeConfig()
Commits 5dbee5e and cce4616 fix disk upgrades concerning paramsslot. Since 2.7 params slot should be empty and gets filledany time needed.
Still ext template allows passing arbitrary params per disk.These params should be saved in config file for future use....
SetDiskID() before accepting an instance
SetDiskID() fills physical_id slot of a Disk object.
LUInstanceSetParams() does not invoke SetDiskID() upon creation of anew disk. As a result the physical_id slot of the Disk object inconfig data is missing.
In case of ext disk template, in AcceptInstance() we invoke...
Lock group(s) when creating instances
This is required to prevent race conditions such as removing a networkfrom a group and adding an instance at the same time. (See issue 621#2.)
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>...
Fix job error message after unclean master shutdown
According to commit 599ee321eb, any job-related error messages shouldbe encoded within a Ganeti-specific error and not passed on as astring, to allow for easier parsing.
For jobs suffering from an undesirable status after an unclean master...
Add default file_driver if missing
If the file driver of an instance with file based storage is not specified, thedefault one is automatically added by the UpgradeConfig function.
Fixes Issue 571.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Xen handle domain shutdown
Update Xen backend to properly recognize when a domain has beenshutdown by the user and to properly cleanup a shutdown domain whenGaneti requests Xen to stop this domain.
Partial cherry-pick from 9d22cc90609e3ee8f0f2b34b793a3daced3c0e61...
Fix evacuation out of drained node
Fix a bug in InstanceSetParams concerning names
In case no name is passed in disk modifications we shouldkeep the old one. If name=none then set disk name to None.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Fix error printing
Fixes issue 616.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Allow link local IPv6 gateways
Each host using IPv6 always has a link local address in fe80::/10. It iscommon to use fe80::1 as default gateway to ease client configuration.Ganeti prevented this usage, because it made sure that the IPv6 gatewayis in the IPv6 network the instance is connected to....
Fix NODE/NODE_RES locking in LUInstanceCreate
Both NODE and NODE_RES locks were acquired opportunistically if sorequested by the user. LUInstanceCreate requires, however, that theactually locked elements on NODE and NODE_RES level are the same.
This patch changes the locking of NODE_RES such that those locks are not...
Fix socket permissions after master-failover
When using gnt-cluster master-failover, on the soon-to-be-master theluxi daemon is started by the node daemon. This makes the luxidaemon inherit the node daemon's umask 077, making the communicationsocket unreadable to group members. When using Ganeti with non-root...
DRBD: ensure peers are UpToDate for dual-primary
DrbdAttachNet supports both, normal primary/secondary node operation, and(during live migration) dual-primary operation. When resources are newlyattached, we poll until we find all of them in connected or syncing operation....
Improve error message for replace-disks
In some conditions, replace-disks will fail if the disks are not properlyactivated. Improve the error message suggesting to run activate-disks beforeexecuting replace-disks.
Fixes Issue 606.
Placate warnings on ganeti.outils_unittest.py
Change ovf.py from using 'if foo:' to 'if foo is not None:' checks assuggested by the FutureWarnings the OVF library raises otherwise.This fixes issue 557.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Fix typo in storage.FileStorage docstring
Don't attemp ipv6 ssh in case of ipv4 cluster
In case of a cluster with primary-ip-version=4, there is no needfor cluster to try ipv6 ssh connections which may timeout.So append '-4' on ssh if cluster is ipv4-only
Signed-off-by: Costas Drogos <costas.drogos@gmail.com>...
Fix corner-case in handling of remaining retry time
Consider a remaining time of 0 as already timed out. Otherwise,there is no guarantee that calling utils.Retry with timeout 0will call the function precisely once; it might run in timeshorter than the resolution of timer....
Add a default to file-driver when unspecified over RAPI
The file-driver value, used by file-based instances, had a default value whenan instance was being created over the CLI, but not when the instance wascreated through the RAPI.
This patch introduces a default value for the remote API and, while doing so,...
Prepare GetLinuxNodeInfo for testing
The names of the input files are not hardcoded anymore, but they are now defaultparameters.
No changes for the callers, but it's now possible to test the function moreeasily.
Allow instance mods if only osparams are given
osparams are now recognized as changes, so changing only those withoutother changes works too now.
(cherry picked from commit 5eae613c2e1e65101c5d5f7d2e8ffd7cc6edc7d5)
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Merge branch 'stable-2.7' into stable-2.8
Fix batch creation of instances
During batch creation of instances the 'reason' field in opcodes is setto None (but the field is present). This caused problems when adding areasons to the reason trail.
Setting the default value for the 'reason' field to the empty list fixes...
Support multi instance allocs without iallocator
If all instances in the multi allocation request have already theirprimary and secondary node set, there is no need for an iallocator. Thusdon't require it in this case and omit the call to it all together....
Support DSA SSH keys in bootstrap
As outlined in issue 338, Ganeti failed to initialize a cluster if noRSA SSH key is present on the master node. This patch extends Ganetissupport to DSA keys, so clusters with only DSA keys are possible now.
This fixes issue 338....
Include VCS version in `gnt-cluster version`
Also print the VCS version in the output of `gnt-cluster version`. Thismakes the VCS version also available over RAPI, etc.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Support 'viridian' parameter in Xen HVM
This parameter is required to prevent bluescreens in Windows instances.
This fixes issue 233.
Signed-off-by: Heiko Baumann <heibau@googlemail.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>