Synnefo » snf-ganeti

KVM nic script: enable interface forwarding

If forwarding is enabled globally this is a no-op. If instead it's
enabled only for some special interfaces where instance traffic has to
go to/comes from (for example a gre tunnel) then it's useful to
explicitely enable it for the instances interfaces as well....

Even more style fixes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

A few style fixes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Fix backend import errors from GetHypervisorClass

The merge of commit 360b0dc into branch-2.1 broke import of backend,
since it uses hypervisor.GetHypervisor() which returns an instance of
the hypervisor. Some of the hypervisors create directories at init time,...

Merge branch 'next' into branch-2.1

• next:
  Create a new --no-voting option for masterfailover
  ganeti-masterd: allow non-interactive --no-voting
  Fix pylint warnings
  Add custom pylintrc
  bootstrap: Don't leak file descriptor when generating SSL certificate...

Use ReadFile.splitlines() rather than readlines

A few places in the code open a file "manually" rather than using our
wrapper function, because they need an array with the lines. Combining
the result of utils.ReadFile with splitlines() we get rid of the
exceptions....

KVMHypervisor, fix a bug in MigrateInstance

This was introduced as a typo in commit
7e66c35b5685525538dd65dd3dff75b5346d30d7 while removing duplicate code
from the KVM hypervisor.

Signed-off-by: Guido Trotter <ultrotter@google.com>

KVM: NIC parameters

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

IsNormAbsPath and users, use "normalized" term

We used to refer to normalized paths as "normal" which might be
confusing. This fixes the syntax in all current IsNormAbsPath users and
in the docstring.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Merge branch 'branch-2.1' into feature/containers

• branch-2.1:
  Hypervisors: make absolute path checking strict
  Add utils.IsNormAbsPath function
  Convert instance reinstall to multi instance model
  gnt-instance batch-create: use the job executor
  Modify cli.JobExecutor to use SubmitManyJobs...

KVM: add the network script to the ancillary files

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

_RedistributeAncillaryFiles: add hypervisor files

Each hypervisor can declare additional files to be shipped to all nodes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Merge branch 'branch-2.1' into feature/containers

• branch-2.1: (48 commits)
  Doc fixes for RAPI
  Release 2.0rc5
  Move to data-based hvparam checks instead of code
  Move more hypervisor strings into constants
  watcher: try to restart the master if down...

Merge branch 'master' into next

Signed-off-by: Guido Trotter <ultrotter@google.com>

Merge commit 'origin/next' into branch-2.1

• commit 'origin/next': (25 commits)
  Move more hypervisor strings into constants
  Add -H/-B startup parameters to gnt-instance
  call_instance_start: add optional hv/be parameters
  Fix gnt-job list argument handling...

Move HVM's device_model to a hypervisor parameter

This moves yet another hardcoded value to a hypervisor parameter. I
removed the 64/32 difference as it doesn't seem valid to me - it's more
of a local site config rather than arch config.

Signed-off-by: Iustin Pop <iustin@google.com>...

KVM: Abstract runtime file removal in a function

This removes some code which was duplicated in shutdown and migrate.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

KVMHypervisor: return memory and cpus as integers

Currently the KVM hypervisor returns strings for the memory and cpu
values, while the xen hypervisor returns integers. Making this uniform
converting the values to integers in KVM as well.

Signed-off-by: Guido Trotter <ultrotter@google.com>...

Fix Xen soft reboot via polling

This patch fixes the Xen soft reboot ("xm reboot") via polling for a specific
time for either changed domain ID or decreased CPU run-time.

This sould prevent the race-conditions discussed on the mailing list for
reboots.
...

Xen: Remove one hardcoded constant

s/"vnc_bind_address"/constants.HV_VNC_BIND_ADDRESS/

Reviewed-by: imsnah

Fix xen-hvm and KERNEL_ARGS

xen-hvm doesn't have KERNEL_ARGS, and I just changed blindly all old
extra_args to HV_KERNEL_ARGS. This makes xen-hvm work again.

Reviewed-by: imsnah

Simplify a little the hypervisor routines

Instead of “instance.hvparams”, we use a shorter “hvp” name to make readability
better.

Reviewed-by: imsnah

Some small improvements to the fake hypervisor

This patch modifies the fake hypervisor to substract the memory “used”
by “running” instances from the free memory, so the actual node
information changes based on the running instances.

Also some style changes and fixes are added....

KVM: Correct CheckParameterSyntax docstring

The comment is not really true anymore, as we have a lot of parameters
nowadays.

Reviewed-by: iustinp

KVM: Add usb mouse type parameter

In some cases 'mouse' may work better than 'tablet', so we'll handle
both by allowing the user to specify a parameter. By default no mouse is
used.

Reviewed-by: iustinp

KVM: actually support different nic types

When executing the KVM runtime we load the nic type from the runtime
hvparams and use it to specify the nic model type. As for the disk we
translate the DEV_PARAVIRTUAL type to 'virtio'.

Reviewed-by: iustinp

KVM: actually support different disk types

By passing the relevant if= value to the disk we support different disk
types. The only change is that we'll translate "paravirtual" to
"virtio" to keep only one "paravirtualized" value, around ganeti. The
if= value is calculated outside the disks loop, as it's the same for all...

KVM: parameters for different disk and nic types

- Add a bunch of NICs and DISKs types
- Specify which one are valid disks and nics for KVM (the new ones
toghether with some of the old ones)
- Add the default values (paravirtual)
- Allow the disk and nic types as parameters and check their validity...

s/HT_HVM_VNC_BASE_PORT/VNC_BASE_PORT/g

The VNC base port has nothing to do with HVM itself, and is general to
VNC itself, so we're removing the HT_HVM prefix to the constant.

Reviewed-by: iustinp

KVM: don't boot from a virtio cdrom

Apparently it's not supported. Also add -boot command line parameters
to kvm, since they seem to help booting from the right place. Everything
will still only work when not using a kernel, but well... :)

Reviewed-by: iustinp

Support cdrom image and boot order for KVM

The cdrom image has the same meaning than in Xen HVM, and so does
boot_order, even though it has a slightly different syntax, and uses the
value 'disk' too boot from disk and 'cdrom' to boot from cdrom.

Reviewed-by: iustinp

KVM: add VNC TLS and X509 parameters

With this parameters VNC for KVM is able to be protected by tls,
optionally with an x509 certificate, and optionally verifying the
client as well. Additionally in this patch we limit the bind address to
being a directory, rather than a file or a directory, for simplicity, as...

KVM: Make GetAllInstancesInfo concurrency-safe

Or actually more so. If this function gets called while instances get
shut down, it might try to report information on instances which don't
exits. Try to fail gracefully if that happens, by just skipping an...

KVM: enable VNC if a VNC_BIND_ADDRESS is defined

We'll also enable a tablet usb device, as suggested by the kvm man page.

Reviewed-by: iustinp

KVM: make the kernel and initrd arguments optional

Under KVM we don't strictly need a kernel and initrd. If some are passed
we'll use them, otherwise the guest OS will need to behave as fully
native, and have its own boot loader and kernel.
The root_path hypervisor parameter becomes mandatory only if a kernel is...

GetShellCommand: get hvparams and beparams

Sometimes the hypervisor will use the instance hv and/or be parameters
to determine the best shell command. This is not possible, though,
currently, as the instance hv/beparams are not filled, so we have to
pass the filled versions separately....

Xen: use utils.WriteFile for the instance configs

Also raise HypervisorError rather than OpExecError.

Reviewed-by: iustinp

KVM and Xen: add the HV_ROOT_PATH parameter

This parameter allows a different path to be passed to the instance
kernel. The new parameter is mandatory, and by default has the value of
the old hardcoded value for both kvm and xen.

Beta1 clusters will need to have this parameter added for their...

KVM: classify _Instance{Monitor,Serial,KVMRuntime}

Those methods need nothing from the instantiated class, and just
manipulate strings, and fetch some class global variables, so they can
be classmethods.

Reviewed-by: iustinp

KVM live migration: handle failure

If the KVM live migration ends up in a 'failed' state it has been
aborted at the kvm level, and the machine is still running locally.
We support also the 'cancelled' state even though there should be no way
of reaching it, without manual intervention....

KVM: instance migration

The tcp port used for migrating KVM instances is selectable at
./configure time. We use a single port as nodes are locked anyway during
a migration, so no two migrations can happen at the same time to the
same node.

Reviewed-by: iustinp

KVM: fix RebootInstance

RebootInstance was broken, because it just used to call StartInstance
with wrong parameters. With this patch we still stop the instance, but
use the saved kvm runtime to start it again.

Reviewed-by: iustinp

Xen: implement auxiliary migration functions

These are used, for the xen hypervisor, to copy the xen config file to
the remote node. This breaks migration for instances which have been
migrated, but not restarted, with the old code, for which the config
file was just lost....

Implement the new live migration backend functions

MigrationInfo, AcceptInstance and AbortMigration are implemented as
hypervisor specific functions, and by default they do nothing (as
they're not always necessary).

This patch also converts hv_base.MigrateInstance docstring to epydoc,...

KVM: split KVM runtime generation and startup

Before we used to generate the kvm command line and then just run it.
With this patch we split the generation from the time it is run,
allowing us to save it and replay it at reboot.

We must take special care about instance nics:...

KVM: Remove sockets after shutdown

Abstract the monitor and serial socket naming in two functions, and
reuse them to cleanup the files after shutdown.

Reviewed-by: iustinp

Xen: use epydoc in MigrateInstance docstring

Reviewed-by: iustinp

Simplify hypervisor block_devices structure

This is a partial forward-port of commit 1136 on the 1.2 branch:

The hypervisor doesn't need to be passed the whole block device
  structure, so we'll just give it the block device name on the local
  node, and the name as seen by the instance. This will make it easier to...

KVM: improve socat interface

Call socat with a full path specified at configure time, rather than
just by its name, and check for the binary to exist at hypervisor
verify.

Reviewed-by: iustinp

cleanup: kvm code likes to redefine names

Reviewed-by: amishchenko