(2.12) QMP: make QmpConnection act as a context manager
This will allow automatic connection and socket cleanup on commandcompletion.
We also repeat the Qmp tests using the context manager. For this to befeasible, we move the test scenario to class variables and modify...
(2.12) QMP: always return the command result
According to the QEMU Machine Protocol Specification, the messages sentby QMP as a response to a command can be of two types: either an errormessage (identified by the "error" key), or a success message(identified by the "return" key)....
(2.12) QMP: keep greeting message version info
QMP will always report the QEMU version and package-specific string inthe greeting message, as per the QEMU Machine Protocol Specification. Westore this information and make it available to users of the monitor....
(2.12) QMP: raise exception on unsupported QMP command
Raise a special exception, qmp.QmpCommandNotSupported, whenever anunsupported QMP command is about to be executed. This is intended toassist in falling back to the human monitor for hotplug operations. As...
(2.12) QMP: fetch supported commands on connect
Store the set of supported QMP commands inQmpConnection.supported_commands. This allows selective error handlingto give the caller a chance to downgrade to HMP when unsupportedcommands are encountered.
The `query-commands` QMP command used to implement this, appeared in...
(2.12) KVM: move tap control functions to a submodule
Move all tap-related functionality to the hv_kvm.netdev submodule.We rename _OpenTap to OpenTap, since it will now be used as a publicfunction.
Also, change the hv_kvm tests to import the new code....
(2.12) Convert QmpMessage to a new-style class
Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
(2.12) KVM: move monitor & QMP classes to a submodule
We move the HMP and QMP classes to the hv_kvm.monitor module.
Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
(2.12) Convert hv_kvm to a package
Weighing in at 3k LOC, it is probably time to split hv_kvm into anumber of modules before adding new code.
(2.12) kvm: Add migration capabilities as an hvparam
Latest QEMU versions support various migration capabilities. Eachcan be enabled/disabled with 'migrate_set_capability' monitorcommand.
Version 1.7.0 defines x-rdma-pin-all, auto-converge, zero-blocks,...
(2.12) Add TAP interface name as argument in '_OpenTap'
This is necessary in order to create TAP interfaces for thecommunication mechanism, which are named 'gnt.com.%d', where '%d' is aunique number within a given node.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
(2.12) Rename 'CdromOption' to '_CdromOption'
... because it is a protected definition.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
(2.12) Fix KVM cdrom image URL in the second cdrom drive
(2.12) kvm: check that the ISO image is there if it's a URL
Perform a simple urllib2 check on ISO images specified as URL beforeinstance start, so as to work around qemu bug #597575 [1].
[1] https://bugs.launchpad.net/qemu/+bug/597575
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>...
(2.12) Allow KVM to boot from HTTP
New versions of KVM support booting from HTTP-hosted ISO images, vialibcurl. This patch adds a proper check to allow defining either a sane,absolute path or an HTTP URL as an iso image path.
Remove "format=raw" from the cdrom device options when iso_image starts...
(2.11) Fix class methods in KVM hypervisor
Several methods in KVM hypervisor behave like 'classmethods', however,they did not have the '@classmethod' annotation. This patch adds'@classmethod' annotations to several of these methods with the goalof making '_StopInstance' available to the QA. This step is necessary...
Update the license statement
It was decided that Ganeti is relicensed under the 2-clauseBSD license. Update the license statements accordingly (issue #936).
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add non-cherry-pickable changes related to state "rb----"
Ganeti versions 2.11 and 2.12 have both had patches related to strangeXen domU states which are still legitimate running states, but Ganetidid not recognize them as such. The previous two cherry-picked patches...
Add Xen nothing state '------'
... which means that a domain is in the queue behind other domainswaiting to be scheduled. The domain is runnable but it is notcurrently running.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>...
Add Xen paused state as a Ganeti running state
The QA test 'TestClusterBurnin' pauses instances and the Xen statedomain 'paused' was not being handled properly by Ganeti because itwas not being classified as either a running or shutdown state. Thispatch adds the 'paused' state to the set of running states by...
KVM: fix NIC configuration with absent NIC VLAN
Commit 6b47261a exported the VLAN to the KVM NIC configuration scripts.However, in doing so it assumed that the VLAN is always present in theruntime, which is not true for instances being migrated but started on...
Allow disk hot-remove even with chroot or SM
Up until now, in case the KVM hypervisor used chroot or pool/usersecurity model, disk hot-remove was not allowed. This should not bethe case since only during hot-add there are permission constrains.
Fix that....
Export VLAN nicparam to NIC configuration scripts
The vlan nicparam is used in openvswitch mode. This patchexports it as VLAN environment variable so that NICconfiguration scripts can later use it.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Support disk hotplug with userspace access
Before any hotplug action we assemble the block device. Currentlycall_blockdev_assemble() returns the link_name as calculated by_SymlinkBlockDev().
With userspace support we have to return the drive_uri as calculated...
Merge branch 'stable-2.9' into stable-2.10
Conflicts: configure.ac # Taken both contributions
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
KVM: set IFF_ONE_QUEUE on created tap interfaces
The IFF_ONE_QUEUE flag directs the kernel to only queue tap packets once(as opposed to queueing them twice, once for the device, and once for theqdisc), possibly avoiding interface stalls when one of the queues overruns....
KVM: use running HVPs to calc blockdev options
Commit 0fe22ad2 moved the call to _GenerateKVMBlockDevicesOptions() from_GenerateKVMRuntime() to _ExecuteKvmRuntime. However, while in_GenerateKVMRuntime() there is only one set of HVPs, those of theinstance's configuration, in _ExecuteKVMRuntime there are two sets of...
KVM: reserve a PCI slot for the SCSI controller
Currently instances with disk_type=scsi are broken, because the SCSIcontroller uses a PCI slot not accounted for in the PCI assignmentlogic. We reserve a throw-away slot just for that.
This is a temporary workaround and will be reverted during the upcoming...
Fix conflict between virtio + spice or soundhw
With regard to PCI slot occupied by a KVM instance we haveobserved the following:
1) Slot 0 will always be Host bridge.2) Slot 1 will always be ISA bridge.3) Slot 2 will always be VGA controller (even with -display none)....
Fix bitarray ops wrt PCI slots
Introduce new method `_GetFreeSlot()` responsible only for bitarrayoperations. It fixes search in case of bitarray is either '0000..'or '1111..'.
Use it instead of `_UpdatePCISlots()` and in `_GetFreePCISlot()`.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Fix error introduced during merge
A parameter was lost while resolving a conflict in the signature of a function.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Merge branch 'origin/stable-2.8' into stable-2.9
Enable a timeout for instance shutdown
Add the timeout parameter to the StopInstance function of the hypervisor baseclass and to all its implementations.
Also, change the tests as required by this change.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Allow KVM commands to have a timeout
Modify the function that sends commands to the KVM monitor so that it ispossible to specify an optional timeout after which the command is killed.
Allow xen commands to have a timeout
Modify the function that runs Xen commands so that it is possible to specify anoptional timeout after which the command is killed.
Fix wrong docstring
Fields must be the final elements in an epytext string.
Workaround for monitor bug related to greeting msg
QMP may return multiple greeting messages upon connection.This is reported on qemu-devel. The fix is one-liner butuntil it get's released this is a quick and dirty workaroundthat flushes the client's buffer after getting the first...
hotplug: Verify if a command succeeded or not
Just after issuing _CallHoplugCommands() we invoke_VerifyHotplugCommand() which parses `info pci` resultand searches for given PCI slot and device id.
If we previously had removed a device but it is still there...
hotplug: Call each qemu commmand with an own socat
Previously we issued one socat command with two "\n" separatedactions (e.g. netdev_add ...\ndevice_add...)
After having observed a strange monitor behavior [1] splittingthose commands and introducing a sleep time in between, may reduce...
Fix 'hvparams' of '_InstanceStartupMemory' on hypervisors
Most hypervisors were calling '_InstanceStartupMemory' but not passingthe 'hvparams' keyword argument. Actually, it is not necessary topass this argument given that it is an attribute in the instance...
Remove deprecated _ERROR_DATA_KEY in QMP
Commit de253f14 of QEMU repo "BREAKS QMP's compatibility forthe error response" as it removes "data" key from qmp errorresponse messages. To this end we only log "class" and "desc" values of the message.
Improve backwards compatibility of Issue 649 fix
Commit e6e4ff4cf8d0100f331f94f7a27aa1e03a5d0e7d fixed Issue 649 by switching theseparator for usb_devices from comma to space. That solved the problem withthe command line, but RAPI was able to work with commas too, so, for backwards...
Change usb_devices separator to whitespace
The usb_devices parameter was using comma as a list separator, but this cannotwork because comma is already used as the hypervisor parameter separator.
Change it to use whitespace as a separator, in accordance to what already done...
Merge branch 'stable-2.8' into stable-2.9
Fix pylint 0.26.0/Python 2.7 warning
pylint 0.26.0 on Python 2.7 generates a warning on the string '\ ',recommending to use the r prefix. This patch adds the missing prefix.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add support for blktap2 file-driver
Newer Xen versions use blktap2 instead of blktap. This patch adds supportfor it in Ganeti.
Fixes Issue 638.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Pass hvparams to GetInstanceInfo
...so that the xen command to be called can be determined. Thisfixes another semantical conflict of the last merge.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Xen handle domain shutdown
Update Xen backend to properly recognize when a domain has beenshutdown by the user and to properly cleanup a shutdown domain whenGaneti requests Xen to stop this domain.
Partial cherry-pick from 9d22cc90609e3ee8f0f2b34b793a3daced3c0e61...
Introduce _UpgradeSerializedRuntime() method
This method is invoked during _AnalizeSerializedRuntime() and ismeant to modify runtime files in the way cfgupgrade does forconfig.data. This could remove deprecated fields, change theformat of the file, add/remove sections, etc....
Fix error printing
Fixes issue 616.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
KVM: use custom KVM path if set for version checking
This commit fixes two TODOs from 2008 about using the hardcoded"default" path for KVM where a custom one could've been set through`gnt-cluster modify`.
As a result, `gnt-cluster verify` will no longer fail if a custom...
Export NIC's UUID and name to network scripts
In case of kvm None values are not allowed in env dictso we have to add name only if not None.
In case of Xen since we are writing on a file thatis going to be sourced we should not add INTERFACE_NAME=None....
Use HooksDict() to export network options in Xen
Remove duplicate code that exports network options to environmentvariables.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Export tags via GetTags() to network scripts
Use GetTags() instance method in order to export instance tagsto NIC configuration scripts and files of kvm and xen hypervisors.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Introduce --hotplug-if-possible option
This will be useful for an external entity using RAPI thatwants to modify devices of instances.
The common use case for that is:"I want to add a NIC/disk to an instance. If it is runningthen try to hotplug the device. If not, then just add it to config."...
Fix retrieval of xen command in class method
This patch fixes issue 608. When introducing theconfigurability of the xen toolstack in commit8ef418bb92, the hypervisor api was accidentally changedin a way that let to this error in KVM.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Check if hotplug is supported in CheckPrereq
Introduce new RPC hotplug_supported that invokes the correspondinghypervisor's method which checks if hotplug is generallysupported. Call this RPC early in CheckPrereq() and abort ifhotplug is not supported. Currently only KVM hypervisor with...
drive_del after device_del in disk hot-remove
This is needed so that qemu process really releases thecorresponding file descriptor and thus we can dmsetup/lvremovethe device without errors.
block_devices -> kvm_disks renaming
and a couple more styple fixes.block_devices are misleading so rename them to kvm_disk (just like kvm_nics)
Make hotplug related method raise HotplugError
Hotplug is not supported by default and thus we raiseHotplugError() for hotplug related methods.If a hypervisor wants to support it, we must override VerifyHotplugSupport()and implement the Hot* method.
Backend invokes VerifyHotplugSupport() and raises RPCFail if an...
Some fixes in _GenerateKVMBlockDevicesOptions()
Hotplug: rpc support
Introduce new RPC that eventually invoke hypervisor specifichotplug functions. In order to be generic it has the followingarguments: device type, action, device, extra info, seq.Device type can be NIC or DISK, action can be ADD, REMOVE,...
Introduce hotplug methods (hypervisor inteface)
4 new methods: HotAddDevice, HotDelDevice, HotModDevice, HotplugSupportedAll these are the hypervisor interface with the backend.In general they read current runtime info, do the actual hotplugand update the runtime file....
Extract socket related code from QmpMonitor class
Separate unix socket related code from QmpMonitor class andmake the latter extend the newly introduced one: MonitorSocket
Helper methods that check for hotplug support
Hotplug is currently not supported for:
- KVM < 1.0 - existing devices in the cluster - python-fdsend module is not installed (NIC hotplug) - chroot (Disk hotplug) - security mode other than None (Disk hotplug)...
Use -device with pci info if possible
Extend kvm_cmd with -device option for the case of paravirualDisks and NICs.
Helper methods for PCI slots and device ids
Device naming:QEMU monitor expects devices to be uniquely named. Device ids derivefrom the following function:kvm_devid = <device_type>-<part of uuid>-pci-<pci_slot>Device ids must be reproduce-able when we want to remove them....
Prepare block_devices runtime entry
With this patch we add another entry in runtime files along withkvm_cmd, kvm_nics, and hvparams. block_devices that used to beencapsulated inside kvm_cmd, live now separately just like nics dobut in tupples of (L{objects.Disk}, dev_path)....
Prepare kvm options to support -device for disks
Newer kvm version support -device driver,... option where availabledrivers can be found by running kvm -device ? command. Currentlyganeti uses `if` property of driver to define the disk type to usewhich is also passed as hvparam. This patch prepares the use of...
block_devices = (disk, link, uri)
Do not pass BlockDev instance to hypervisor. Instead calculate thedrive_uri (if any) in backend level and pass only the string tohypervisor.
Hypervisor should not be aware of the entire block device but onlythe final path it can be reached....
Introduce _GenerateKVMBlockDevicesOptions()
Simply move logic that generates block device relatedoptions out of _GenerateKVMRuntime().
Hardcode default access mode to kernelspace
Hardcode constants.KERNELSPACE as the default access method forKVM disk templates, rather than allowing a change to the RBD defaultaccess mode to affect all disk templates for KVM.
At some point in the future it might make more sense to have this...
Handle scenarios where access is not set
It should not be possible to reach _GenerateKVMRuntime without havingthe access method set, but if it does happen this commit makes surethe default value is used anyway.
Signed-off-by: Raffa Santi <rsanti@google.com>...
Add Userspace RBD support in KVM
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Add soundhw support to XEN hypervisors
This allows XEN hypervisors to enable sound support, if the hostmachines has required support. This has been tested only up to thepoint of making sure the xen config file has the requiredconfiguration values as per:...
Add cpuid support to XEN hypervisors
Ease instance migration between heterogeneous nodes by letting systemadministrators hide CPU capabilities as needed.
Merge branch 'stable-2.9' into master
Make QmpMessage a proper container
Newer pylint versions complain that QmpMessage is not a proper container becauseit does not implement the len() and delitem() methods. This commitimplements them.
Also, tests are added.
Remove physical_id field from disk object
The 'physical_id' field of disk objects is no longer used, so remove it.Also, all references are removed together with the code which made surethat the physical_id is up to date when transmitted over RPC.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Replace LD_* constants with DT_* constants
LD_* constants are basically like DT_* constants, exceptfor that both file and shared file were mapped to file.In order to not having to maintain three slightly differentsets of disk-related constants (DT, LD and ST), we merge...
Lint improvements to regexps
Fix some regular expressions so that they pass lint checks with newer versionsof pylint.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Prepare GetLinuxNodeInfo for testing
The names of the input files are not hardcoded anymore, but they are now defaultparameters.
No changes for the callers, but it's now possible to test the function moreeasily.
Merge branch 'stable-2.9'
Support 'viridian' parameter in Xen HVM
This parameter is required to prevent bluescreens in Windows instances.
This fixes issue 233.
Signed-off-by: Heiko Baumann <heibau@googlemail.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add hvparam to disable VNET_HDR on tap devices
IFF_VNET_HDR allows tap devices to extract a VIRTIO_NET_HDR from pktsgoing through the tap iface, enabling segmentation offload for thevirtio nics.
Current (Debian) kernels (3.2 for Debian Squeeze backrports / Wheezy)...
Add VLAN-capabilities for Xen Hypervisor
This implementation depends on vif-openvswitch provided by Xen 4.3and provides Xen with switch name and vlan as "link".
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Hypervisors report CPUs used by the node OS
The XenHypervisor's "dom0_cpus" parameter (introduced in 1d60fec6 but neverused) is renamed to "cpu_dom0" for consistency with other parameternames. The analogous information is returned for the other hypervisors....
Fix docstrings for hypervisor functions returning node info
The documentation was out-of-sync with the code.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Make NIC setup more flexible in case of Xen
For every NIC write down a file that contains NIC'sinfo (MAC, mode, link, network details), that canbe sourced by any vif script. The file location is:/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>....
Merge branch 'stable-2.8' into master