Merge commit 'origin/branch-2.1' into feature/containers
Fix backend import errors from GetHypervisorClass
The merge of commit 360b0dc into branch-2.1 broke import of backend,since it uses hypervisor.GetHypervisor() which returns an instance ofthe hypervisor. Some of the hypervisors create directories at init time,...
Merge branch 'next' into branch-2.1
Fix pylint warnings
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Use ReadFile.splitlines() rather than readlines
A few places in the code open a file "manually" rather than using ourwrapper function, because they need an array with the lines. Combiningthe result of utils.ReadFile with splitlines() we get rid of theexceptions....
Fix various pylint warnings
There were multiple issues: - copy-paste resulted in wrong indentation - wrong function name - missing spaces around assignment - overriding built-in names (type, dir) or already defines ones (errors, hypervisor)
Signed-off-by: Iustin Pop <iustin@google.com>...
KVMHypervisor, fix a bug in MigrateInstance
This was introduced as a typo in commit7e66c35b5685525538dd65dd3dff75b5346d30d7 while removing duplicate codefrom the KVM hypervisor.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Xen: NIC parameters
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
KVM: NIC parameters
Add a node powercycle command
This (somewhat big) patch adds support for remotely rebooting the nodesvia whatever support the hypervisor has for such a concept.
For KVM/fake (and containers in the future) this just uses sysrq plus a‘reboot’ call if the sysrq method failed. For Xen, it first tries the...
IsNormAbsPath and users, use "normalized" term
We used to refer to normalized paths as "normal" which might beconfusing. This fixes the syntax in all current IsNormAbsPath users andin the docstring.
hv_chroot: update PARAMETERS value
In the new branch-2.1 PARAMETERS must be a dict rather than a list. Inthe hv_chroot case though we can just check that the path is absoluteand normal, and not its existence, because it's inside the targetchroot.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Merge branch 'branch-2.1' into feature/containers
Hypervisors: make absolute path checking strict
Use the new utils.IsAbsNormPath function, rather than just os.path.isabs
KVM: add the network script to the ancillary files
_RedistributeAncillaryFiles: add hypervisor files
Each hypervisor can declare additional files to be shipped to all nodes.
Xen: add ancillary files
KVM: replace hardcoded network script path
Currently the kvm automatic network scripts accepts to be overridden byan user supplied /etc/ganeti/kvm-vif-bridge script. We keep thisfunctionality but move the hardcoded path to a constant, dependent alsoon SYSCONFDIR....
Merge branch 'master' into branch-2.1
Merge branch 'master' into next
Move to data-based hvparam checks instead of code
Currently the hypervisor parameters are checked using hard-coded snippets ineach hypervisor. However, most parameter checks fall into three cases: - file check - directory check - string value in a set...
Merge commit 'origin/next' into branch-2.1
Move more hypervisor strings into constants
This patch adds constants for the mouse and boot order strings; whilethere are still some issues remaining, we're trying to cleanup hardcodedstrings from the hypervisors.
Since the formatting of frozensets is currently wrong, we also add an...
Move HVM's device_model to a hypervisor parameter
This moves yet another hardcoded value to a hypervisor parameter. Iremoved the 64/32 difference as it doesn't seem valid to me - it's moreof a local site config rather than arch config.
Implement the KERNEL_PATH parameter for xen-hvm
For the xen-hvm hypervisor, the KERNEL_PATH parameter is needed buttoday is hardcoded to a constants in the xen hypervisor library (argh!).
This patch moves this to a hypervisor constant with the default value...
KVM: Abstract runtime file removal in a function
This removes some code which was duplicated in shutdown and migrate.
New hypervisor implementation: chroot manager
This patch adds a new hypervisor implementation: a chroot manager. Thishypervisor type can be used to manage (in combination with special OSdefinitions) the start and stop of chroot areas, and if used with drbd...
KVMHypervisor: return memory and cpus as integers
Currently the KVM hypervisor returns strings for the memory and cpuvalues, while the xen hypervisor returns integers. Making this uniformconverting the values to integers in KVM as well.
Abstract Linux node information in hv_base
Currently both hv_fake and hv_kvm implement practically identical codeto get the node information. Since future container-like hypervisorswill also need this functionality, this patch moves it into the baseclass (as a separate function) which can then be called from classes...
Fix Xen soft reboot via polling
This patch fixes the Xen soft reboot ("xm reboot") via polling for a specifictime for either changed domain ID or decreased CPU run-time.
This sould prevent the race-conditions discussed on the mailing list forreboots....
kvm: use the correct vnc bind address
There is a bug in kvm, when binding vnc to a specific address theconstant 'vnc_bind_address' is passed in, instead of the actualrequested address. This patch fixes it.
Reviewed-by: iustinp
Xen: Remove one hardcoded constant
s/"vnc_bind_address"/constants.HV_VNC_BIND_ADDRESS/
Reviewed-by: imsnah
Use EnsureDirs in KVM as well.
The KVM hypervisor has also code to ensure a list of directories exist.Substitute it with our new utils function.
Fix xen-hvm and KERNEL_ARGS
xen-hvm doesn't have KERNEL_ARGS, and I just changed blindly all oldextra_args to HV_KERNEL_ARGS. This makes xen-hvm work again.
Remove the extra_args parameter in instance start
This patch removes the extra_args parameter and instead switches theinstance to the HV_KERNEL_ARGS hypervisor option.
This is a big change, but it's a needed cleanup, this extra parameter onall RPC calls is not generic and we also need to have a persistent value...
Simplify a little the hypervisor routines
Instead of “instance.hvparams”, we use a shorter “hvp” name to make readabilitybetter.
Show more details for failed xen commands
This patch also logs the output of the xm commands in case of failures;some corner cases were forgotten in the last redo.
Some small improvements to the fake hypervisor
This patch modifies the fake hypervisor to substract the memory “used”by “running” instances from the free memory, so the actual nodeinformation changes based on the running instances.
Also some style changes and fixes are added....
Some small fixes
This patch removes the admin_ram LUQueryInstances field (is brokenanyway) and fixes the VNC address checks in the Xen Hypervisor.
KVM: Correct CheckParameterSyntax docstring
The comment is not really true anymore, as we have a lot of parametersnowadays.
KVM: Fix _CallMonitorCommand error message
1) Only instance_name is available2) There was a missing string parameter
KVM: Add usb mouse type parameter
In some cases 'mouse' may work better than 'tablet', so we'll handleboth by allowing the user to specify a parameter. By default no mouse isused.
KVM: allow netboot
With this patch we allow KVM instances to be booted off the network.The only issue is that this is not compatible with virtio nics, sowe disallow them, when booting from the net.
KVM: actually support different nic types
When executing the KVM runtime we load the nic type from the runtimehvparams and use it to specify the nic model type. As for the disk wetranslate the DEV_PARAVIRTUAL type to 'virtio'.
KVM: export hvparams in the runtime
They'll be used to set the nic type when we execute the runtime, sincethe nics are processed later. We need to save the hvparams because wewant to use the same one as when we saved the runtime, rather than usethe current instance ones, to avoid applying only some changed...
KVM: actually support different disk types
By passing the relevant if= value to the disk we support different disktypes. The only change is that we'll translate "paravirtual" to"virtio" to keep only one "paravirtualized" value, around ganeti. Theif= value is calculated outside the disks loop, as it's the same for all...
Xen-HVM: Improve the invalid disk/nic type error
Copy the message from the KVM one, adding a missing 'the' and a list ofpossible values, to help the user in his decision.
KVM: parameters for different disk and nic types
- Add a bunch of NICs and DISKs types- Specify which one are valid disks and nics for KVM (the new ones toghether with some of the old ones)- Add the default values (paravirtual)- Allow the disk and nic types as parameters and check their validity...
Rename the device type constants
These are not HVM specific, so have been given an HT generic name.
s/HT_HVM_VNC_BASE_PORT/VNC_BASE_PORT/g
The VNC base port has nothing to do with HVM itself, and is general toVNC itself, so we're removing the HT_HVM prefix to the constant.
Export the cpu nodes and sockets from Xen
This is a hand-picked forward patch of commit 1755 on the 1.2 branch(hand-picked since the trees diverged too much since then):
The patch changed the xen hypervisor to compute the number of cpu sockets/nodes and enables the command line and the RAPI to show this...
KVM: don't boot from a virtio cdrom
Apparently it's not supported. Also add -boot command line parametersto kvm, since they seem to help booting from the right place. Everythingwill still only work when not using a kernel, but well... :)
KVM: don't boot from cdrom with no cdrom
Support cdrom image and boot order for KVM
The cdrom image has the same meaning than in Xen HVM, and so doesboot_order, even though it has a slightly different syntax, and uses thevalue 'disk' too boot from disk and 'cdrom' to boot from cdrom.
Get rid of constants.HT_HVM_DEFAULT_BOOT_ORDER
Confusingly, as a leftober from 1.2, there was aconstants.HT_HVM_DEFAULT_BOOT_ORDER constant, with a value opposite tothe default HV_BOOT_ORDER hv param that got enabled only ifHV_BOOT_ORDER was set to None. Since setting it to None is very...
KVM: add VNC TLS and X509 parameters
With this parameters VNC for KVM is able to be protected by tls,optionally with an x509 certificate, and optionally verifying theclient as well. Additionally in this patch we limit the bind address tobeing a directory, rather than a file or a directory, for simplicity, as...
KVM: allow binding vnc to a file
Before we forced the VNC_BIND_ADDRESS to be an ip. Now we also accept apath, and bind the instance to it, or to a file in it if it's adirectory.
KVM: Make GetAllInstancesInfo concurrency-safe
Or actually more so. If this function gets called while instances getshut down, it might try to report information on instances which don'texits. Try to fail gracefully if that happens, by just skipping an...
KVM: advise about VNC support on GetShellCommand
KVM: enable VNC if a VNC_BIND_ADDRESS is defined
We'll also enable a tablet usb device, as suggested by the kvm man page.
KVM: Allow the HV_VNC_BIND_ADDRESS parameter
KVM: make the kernel and initrd arguments optional
Under KVM we don't strictly need a kernel and initrd. If some are passedwe'll use them, otherwise the guest OS will need to behave as fullynative, and have its own boot loader and kernel.The root_path hypervisor parameter becomes mandatory only if a kernel is...
KVM: add the HV_SERIAL_CONSOLE parameter
Up until now a KVM instance was forced to have a serial port.With this change this is no longer mandatory, by default we'll use one,but if the HV_SERIAL_CONSOLE parameter is set to False we'll do without.
GetShellCommand: get hvparams and beparams
Sometimes the hypervisor will use the instance hv and/or be parametersto determine the best shell command. This is not possible, though,currently, as the instance hv/beparams are not filled, so we have topass the filled versions separately....
Actually mark drives as read-only if so configured
This patch correctly marks the drives as read-only for Xen, and raisesand exception for KVM since it doesn't support read-only drives.
Reviewed-by: ultrotter
Xen: use utils.WriteFile for the instance configs
Also raise HypervisorError rather than OpExecError.
Xen: use utils.Readfile to read the VNC password
KVM and Xen: add the HV_ROOT_PATH parameter
This parameter allows a different path to be passed to the instancekernel. The new parameter is mandatory, and by default has the value ofthe old hardcoded value for both kvm and xen.
Beta1 clusters will need to have this parameter added for their...
KVM: implement GetShellCommandForConsole
This is a class method, because it calls _InstanceSerial, which isanother class method. The patch changes it to classmethod for all thehypervisor classes.
KVM: classify _Instance{Monitor,Serial,KVMRuntime}
Those methods need nothing from the instantiated class, and justmanipulate strings, and fetch some class global variables, so they canbe classmethods.
Xen and KVM: correct a typo when checking args
A missing 'be' was present in the error string for both xen and kvm,when the kernel or initrd path was not absolute.
KVM live migration: handle failure
If the KVM live migration ends up in a 'failed' state it has beenaborted at the kvm level, and the machine is still running locally.We support also the 'cancelled' state even though there should be no wayof reaching it, without manual intervention....
KVM: change a few IOError with EnvironmentError
KVM: instance migration
The tcp port used for migrating KVM instances is selectable at./configure time. We use a single port as nodes are locked anyway duringa migration, so no two migrations can happen at the same time to thesame node.
KVM: add the _InstancePidAlive function
Throughout the kvm code we very often look for the instance pidfilename, read it, and check if the process is alive. Abstract this into aprivate function and use that one instead.
This patch also changes RebootInstance to check whether the instance is...
KVM: fix RebootInstance
RebootInstance was broken, because it just used to call StartInstancewith wrong parameters. With this patch we still stop the instance, butuse the saved kvm runtime to start it again.
KVM: retry the instance shutdown command
When we ask the instance to shutdown sometimes the command won't work,especially if the instance isn't fully booted up. We'll wait for a bit,and give it a few chances before giving up.
Xen: implement auxiliary migration functions
These are used, for the xen hypervisor, to copy the xen config file tothe remote node. This breaks migration for instances which have beenmigrated, but not restarted, with the old code, for which the configfile was just lost....
Fix some more pylint errors
Two are real errors (invalid names) and one is style error (overridingname from outer scope).
Implement the new live migration backend functions
MigrationInfo, AcceptInstance and AbortMigration are implemented ashypervisor specific functions, and by default they do nothing (asthey're not always necessary).
This patch also converts hv_base.MigrateInstance docstring to epydoc,...
KVM: save and remove the KVM runtime
At instance startup time we save the kvm runtime, and at stop time wedelete it. This patch also includes a function to load the kvm runtime,which is unused yet.
KVM: split KVM runtime generation and startup
Before we used to generate the kvm command line and then just run it.With this patch we split the generation from the time it is run,allowing us to save it and replay it at reboot.
We must take special care about instance nics:...
KVM: add a _CONF_DIR
Currently we keep pid files and control files. In the conf dir we'llalso keep the data to start the instance anew, and the networkinterface scripts. These will then be copied to a separate area (since_CONF_DIR could be mounted 'noexec') and used to start the instance....
KVM: Remove sockets after shutdown
Abstract the monitor and serial socket naming in two functions, andreuse them to cleanup the files after shutdown.
KVM: fix class docstring
Xen: use epydoc in MigrateInstance docstring
Forward port the live migration from 1.2 branch
This is forward port via copy (and not individual patches cherry-pick)of the latest code on the 1.2 branch related to the migration.
The changes compared to 1.2 are the fact that we don't need theIdentifyDisks step anymore (the drbd rpc calls are independent now), and...
Simplify hypervisor block_devices structure
This is a partial forward-port of commit 1136 on the 1.2 branch:
The hypervisor doesn't need to be passed the whole block device structure, so we'll just give it the block device name on the local node, and the name as seen by the instance. This will make it easier to...
Fix some pylint-detected issues
Two bad indentation cases and a missing variable.
KVM: improve socat interface
Call socat with a full path specified at configure time, rather thanjust by its name, and check for the binary to exist at hypervisorverify.
cleanup: xen hypervisor
Wrong indentation and uniformize one method signature.
Reviewed-by: amishchenko
cleanup: kvm code likes to redefine names
Fix epydoc format warnings
This patch should fix all outstanding epydoc parsing errors; as such, weswitch epydoc into verbose mode so that any new errors will be visible.
KVMHypervisor add two missing 'constants.'
Some calls to the HV parameters were missing them.
KVMHypervisor fix to case misspellings
Change Xen hypervisor to not use iv_name
Currently the iv_name is very linux-specific, and will break with themulti-disk changes.
The patch changes this to generate sdX names based on the disk index inthe disks structure, instead of relying on the iv_name....
Convert hv_xen.py to use the logging module
Convert the KVM hypervisor to hvparams
I'm not sure which parameters it should use, I took all I saw in thecode. Update: ultrotter says the list is OK for now.