KVM: Migration bandwidth and downtime control
Introduce 2 new hypervisor options, migration_bandwidth and migration_downtimeand implement KVM migration bandwidth and downtime control.
migration_bandwidth controls KVM's maximal bandwidth during migration, in...
Fix two race conditions in reboot instance
If the instance crashes between backend.InstanceReboot checks the listof running instances and the execution of hv_xen.RebootInstance,ini_info will be None. And if the instance doesn't reboot fast enough,new_info will be None. Both cases lead to “TypeError: unsubscriptable...
KVM: vhost net acceleration support
This will only work on patched or newer (>= 2.6.34) kernels and with apatched version of qemu-kvm.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Convert some ReadFile calls to ReadOneLineFile
For passwords we require strict oneliners, we're a bit more lax with pidand uid files.
Merge branch 'stable-2.1' into devel-2.1
KVM: only export instance tags if present
Currently non-tagged instances fail starting with a TypeError.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Hypervisors: use utils.EnsureDirs
Swap a few os.mkdir calls with utils.EnsureDirs
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
KVM: make tags available in KVM_NET_SCRIPT
Make instance tags available as a space-separated list during the execution ofthe network setup script. This allows tag-based control of things like firewallrules and/or networking setup.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix broken commit 9e302a8
Commit 9e302a8 split the StopInstance function in two without properlyduplicating the local variables.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
hv_chroot: move unmount to CleanupInstance()
This allows cleanup to be done properly if the "instance" wasn't runningat all (based on the CleanupInstance framework, instead of the retry inhypervisor).
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add a hypervisor constant for migration support
This variable can be used by other tools to determine in a generic waywhether a given hypervisor supports migration or not.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix indentation error
Commit 9cf4321fc39ec36359d9c90b22b36d33b6adc2f4 indented some lines by4 spaces rather than 2, and was git-amed without noticing. Fixing.
Add -usbdevice tablet to KVM when using vnc
When using VNC, it is recommended to use a tablet-styleinput device instead of a mouse. This allows most VNC viewersto send proper mouse coordinates to the virtual machine's desktopresulting in perfectly aligned guest and host mouse pointers....
Only use boot=on on non-ide disks only (KVM)
boot=on implies that KVM boots using extboot. This is only requiredto boot non-IDE disks and has the side-effect that there is at mostone bootable device. This behaviour breaks some operating systems,most notably the windows installer that tries to chainload the hard-disk...
KVM: fix a bug in _TryReadUidFile
If the uid pool is not in use _TryReadUidFile will try to return "uid" even if it hasn't tried to read it at all.
KVM: implement the HT_SM_POOL security model
In order not to complicate to much the _ExecuteKVMRuntime function withnested try/except/finally/else constructs we move the actualruncmd+check call in a separate _RunKVMCmd function.
KVM: add an instance uid file concept
If this file exists, for an instance, we release the uid containedinside back to the uid pool, at instance shutdown.
KVM: move post-shutdown cleanup operations
Currently putting the cleanup just after the actual shutdown ensuresthat it never gets called, due to how the retry/shutdown cycle works.So we move those cleanups to their new dedicated place.
Add CleanupInstance hypervisor call
Currently some hypervisors (namely kvm) need to do some cleanup aftermaking sure an instance is stopped. With the moving of the retry cyclein backend those cleanups were never done. In order to solve this we adda new optional hypervisor function, CleanupInstance, which gets called...
kvm_flag hypervisor parameter
Move the runas user at execution time
Everything still works the same way, but the user is calculated eachtime we start kvm, rather than stored in the config file. This makes iteasier to implement the "pool" security model.
Extend the hypervisor API with name-only shutdown
Currently the ShutdownInstance method of the hypervisors takes a fullinstance object. However, when doing instance shutdowns from the nodeonly, we don't have a full object, just the name.
To handle this use case, we add a new ‘name’ argument to the method,...
Some epydoc fixes
KVM: Check instances for actual liveness
Currently if we find a live process with the pid we saved we assume kvmis alive. What could happen, though, is that the pidfile has beenreused.
In order to avoid that we change the check to make sure, everywhere,...
KVM: improve GetInstanceInfo docstring
KVM: remove unused variable
We don't need the pwentry when checking if a username exists, just to besure the KeyError is not returned. Remove the variable, and thus shut uplint.
KVM: add security model and domain parameters
Initially we only support the "user" model (in which the user runningthe virtual machine can be specified as an additional parameter).
We use usernames rather than uids in this mode, because the kvm -runasflag doesn't support uids anyway, and we check the passed username for...
KVM: Remove boot restriction for paravirtual nics
Newer virtio can boot from the network perfectly well, so there's nopoint in keeping this restriction in place. This will still fail onolder kernels.
KVM: pass the instance name as the first kvm flag
This makes it the first argument show, for example under "ps".
KVM: Fix unintended qemu-level bridging of nics
Each nic should be connected to its own qemu vlan, to avoid them allbridging together.
Signed-off-by: Timothy Kuhlman <timkuhlman@gmail.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>...
hv_kvm: remove hard-coded path constructs
This switches hv_kvm to PathJoin. There are still a few cases of directpath construction, but those should be safe.
hv_fake: remove hard-coded path constructs
This changes to hv_fake to PathJoin.
hv_chroot: remove hard-coded path constructs
This patch abstract the computation of an instance's root directory intoa separate function (that uses PathJoin instead of "%s/%s").
Switch from os.path.join to utils.PathJoin
This passes a full burnin with lots of instances, and should be safe aswe mostly to join a known root (various constants) to a run-timevariable.
Avoid absolute path for privileged commands
Using absolute path for a privileged command is a bad idea as this path may vary.For example /usr/sbin/brctl in Debian and /sbin/brctl in ALTLinux. Using $PATH isa better idea.
Signed-off-by: Vitaly Kuznetsov <vitty@altlinux.ru>...
Merge remote branch 'origin/stable-2.1' into devel-2.1
Fix missing bridge for xen instances
Xen instances nic definitions miss the target bridge.
This bug was introduced in commit 503b97a9.
Signed-off-by: Alessandro Cincaglini <alessandro.ciancaglini@gmail.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...
Merge branch 'devel-2.0' into devel-2.1
Conflicts: lib/backend.py - trivial merge...
Ensure all int/float conversions are handled right
int()/float() can raise either ValueError (in case of int("a")), orTypeError (in case of int(None)). We had many bugs over time due tothis, and a recent one was just diagnosed, so we go over the codebase...
KVM: fix pylint warning
Specify string format arguments as logging function parameters
Signed-off-by: Guido Trotter <ultrotter@google.com>
KVM: be more resilient on broken migration answers
Before, when doing kvm live migrations we use to accept an "unknownstatus" but to reject anything that didn't match our regexp. Since we'veseen "info migrate" return a completely empty answer, we'll be more...
Fix unused imports or add silences where needed
In some cases pylint doesn't parse the import correctly, so we addsilences; but there are also many cases of unused imports, which wesimply remove.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
hv_xen/_GetConfigFileDiskData: remove unused arg
The disk template is not needed, all that's used is the disk data. Assuch, remove this parameter from the function.
Add targeted pylint disables
This patch should have only:
- pylint disables- docstring changes- whitespace changes
Implement all hv functions in hv_chroot/hv_fake
The chroot and fake hypervisors were missing:
- the powercycle node functionality- proper handling of migration requests
The powercycle was just used as in the other hypervisors (use thestandard linux powercycle). The migration for chroot was disabled...
KVM: Abstract/rework instance up checks
This patch abstract the check "is instance stopped" into a separatefunction, and thus simplifies a couple of higher-level functions. Italso moves from manual read of the pidfile to use the (correctabstraction of) _InstancePidAlive....
KVM: Split out the pidfile computation
In some cases we only need the pidfile, but not the pid or the alivestatus.
Remove many 'Unused variable' warnings
Note there are some cases left which need extra cleanup.
Fix use of the logging functions
The logging functions expand the arguments themselves, thus it's saferto let them do it rather than manual string formatting.
Also re-wraps one comment.
Fix two bugs in seldom-used codepaths
New version of pylint, new bugs found!
Fix indentation in hv_kvm
Per pylint warnings.
Add disk cache control parameter for KVM
This patch adds the 'cache' parameter for KVM; currently this is onlycustomisable at the hypervisor level, so it's the same for all drives(except any CDROM image, which gets the default).
Signed-off-by: Iustin Pop <iustin@google.com>...
KVM: fail when a routed nic has no ip
This shouldn't happen, but if it does it's better to fail at this level,rather than create a broken NIC script, which is hard to debug.
Revert "Get rid of utils.CommaJoin"
This reverts commit 6915bc28fe053e92aa16cf2d974d205f1140219c based on thread onganeti-devel.
Conflicts:
lib/cmdlib.py (due to the error code classification, trivial)
Fix epydoc error
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix and simplify socat escape detection
- Program paths should not be --with-… options (see Autoconf docs)- Simplify checks for escape functionality- Make SOCAT_USE_ESCAPE variable a bool
KVMHypervisor: fix broken error format string
Add use_localtime parameter for xen-hvm and kvm
Currently xen-hvm and kvm use different real time clock by default. Toreduce confusion, this patch adds an optional use_localtime parameter.
If the real time clock on the instance is set to local time, the...
kvm console: use socat raw mode with escape
If this is enabled at configure time, we pass in different parameters tothe socat console, making it a lot more manageable.
Migration: add check for listening target
This patch adds a check for listening on the remote port in Xen and KVMmigrations. This will be generating a single “load of migration failed”message for KVM, but otherwise not prevent the migration. For Xen (which...
hypervisors: switch to using HV_MIGRATION_PORT
This changes KVM to use HV_MIGRATION_PORT instead of KVM_MIGRATION_PORTand enables passing the port for Xen migrations.
Since KVM_MIGRATION_PORT is not used anymore, we stop exporting it fromconstants.py....
Introduce HV_MIGRATION_PORT hypervisor parameter
This parameter will replace the direct use of KVM_MIGRATION_PORT and theimplicit use of the Xen migration port.
While it doesn't make sense to change this at instance level, we don'thave any other infrastructure for cluster-wide hypervisor parameters, so...
hypervisors: change MigrateInstance API
Currently the $hypervisor.MigrateInstance takes the instance name. Thispatch changes it to take the instance object, such that other instanceproperties (especially hvparams) are available to it.
Revert "kvm console: use socat raw mode with escape"
This reverts commit ce0eb6694e3fb2510035501539c7acc92a0f174e, since it dependson 37fc2cf5ba8919cef407199ee540aad4b1a9a2b6 which will be reverted too.
hv_xen: Convert to utils.Retry
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
KVM netscript: add static routes, with no suffix
The /32 suffix is useless, since the kernel already assumes single-host,if no suffix is specified. Moreover we prefer these routes to be"static" so that routing daemons, if present, won't mess with them....
KVMHypervisor: implement instance policy routing
Until now we relied on traffic from instances being policy routed via arule based on the instance network. With this change we can enforce iton the instance interfaces. Since the ip rules survive interface...
KVMHypervisor: configure v6 parameters on nic
In routing mode we are tweaking a few parameters on the interface. Withthis patch we'll tweak both the v4 and v6 ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Xen: Ignore the retry argument in stop instance
Commit 4ad4511 changed the KVM hypervisor to send multiple shutdownrequests to the monitor, but it didn't change this for the Xenhypervisor. We simply remove the return on retry model, since we do wantto send multiple shutdown signals for both Xen and KVM (even if the...
Code and docstring style fixes
Found using pylint and epydoc.
ChrootManager: clean StopInstance
Currently it has lots for duplicated code, and internal retries.Clean it up with the following assumptions:
We'll probably be called more than once.It is ok to fail to stop, unless we're called with force=True.If we're called only once, and with force=True it's ok not to run the...
KVMHypervisor: use the StopInstance retry feature
Since we know StopInstance is going to be called more than once (atleast twice, once with force and once without, but normally quite a lotmore) we don't need our own sleep/loop, and we can just send one monitor...
Hypervisors: Add retry= to StopInstance
Currently some hypervisors need the stop operations to be retried morethan once, while other ones only do it in one pass. With this changewe'll handle retries outside the hypervisor code, but telling whetherthis is the first try or not....
Get rid of utils.CommaJoin
- We never remember to use it (5 uses vs 21 " ,".join())- It's longer to write than " ,".join()- The added value of the apostrophe in the string is not very much
VNC password: move to hv param and use in kvm
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Add support for using the bootloader in xen-pvm
This patch adds three optional parameters: - 'use_bootloader', whether use or not the bootloader - 'bootloader_path', absolute path to the bootloader - 'bootloader_args', extra arguments to the bootloader...
KVMHypervisor: wrap long line
KVM nic script: enable interface forwarding
If forwarding is enabled globally this is a no-op. If instead it'senabled only for some special interfaces where instance traffic has togo to/comes from (for example a gre tunnel) then it's useful toexplicitely enable it for the instances interfaces as well....
KVM nic script: use routed link as table
In order to be able to maintain the node network standard routinguntouched while routing instance traffic through a different dedicatedinterface (eg: a gre tunnel) we need to specify the instance routingpath inside a separate table, which will also contain different default...
Even more style fixes
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Luca Bigliardi <shammash@google.com>
KVM hypervisor: Use ReadFile
A few style fixes
Implement the KERNEL_PATH parameter for xen-hvm
For the xen-hvm hypervisor, the KERNEL_PATH parameter is needed buttoday is hardcoded to a constants in the xen hypervisor library (argh!).
This patch moves this to a hypervisor constant with the default value...
Move HVM's device_model to a hypervisor parameter
This moves yet another hardcoded value to a hypervisor parameter. Iremoved the 64/32 difference as it doesn't seem valid to me - it's moreof a local site config rather than arch config.
Merge commit 'origin/branch-2.1' into feature/containers
Fix backend import errors from GetHypervisorClass
The merge of commit 360b0dc into branch-2.1 broke import of backend,since it uses hypervisor.GetHypervisor() which returns an instance ofthe hypervisor. Some of the hypervisors create directories at init time,...
Merge branch 'next' into branch-2.1
Fix pylint warnings
Use ReadFile.splitlines() rather than readlines
A few places in the code open a file "manually" rather than using ourwrapper function, because they need an array with the lines. Combiningthe result of utils.ReadFile with splitlines() we get rid of theexceptions....
Fix various pylint warnings
There were multiple issues: - copy-paste resulted in wrong indentation - wrong function name - missing spaces around assignment - overriding built-in names (type, dir) or already defines ones (errors, hypervisor)
KVMHypervisor, fix a bug in MigrateInstance
This was introduced as a typo in commit7e66c35b5685525538dd65dd3dff75b5346d30d7 while removing duplicate codefrom the KVM hypervisor.
Xen: NIC parameters
KVM: NIC parameters
Add a node powercycle command
This (somewhat big) patch adds support for remotely rebooting the nodesvia whatever support the hypervisor has for such a concept.
For KVM/fake (and containers in the future) this just uses sysrq plus a‘reboot’ call if the sysrq method failed. For Xen, it first tries the...
IsNormAbsPath and users, use "normalized" term
We used to refer to normalized paths as "normal" which might beconfusing. This fixes the syntax in all current IsNormAbsPath users andin the docstring.
hv_chroot: update PARAMETERS value
In the new branch-2.1 PARAMETERS must be a dict rather than a list. Inthe hv_chroot case though we can just check that the path is absoluteand normal, and not its existence, because it's inside the targetchroot.
Merge branch 'branch-2.1' into feature/containers