Synnefo » snf-ganeti

Implement the new live migration backend functions

MigrationInfo, AcceptInstance and AbortMigration are implemented as
hypervisor specific functions, and by default they do nothing (as
they're not always necessary).

This patch also converts hv_base.MigrateInstance docstring to epydoc,...

KVM: split KVM runtime generation and startup

Before we used to generate the kvm command line and then just run it.
With this patch we split the generation from the time it is run,
allowing us to save it and replay it at reboot.

We must take special care about instance nics:...

Update the objects.Disk formatting method

With the addition of minors, this needs to show them too.

Reviewed-by: ultrotter

KVM: Remove sockets after shutdown

Abstract the monitor and serial socket naming in two functions, and
reuse them to cleanup the files after shutdown.

Reviewed-by: iustinp

Xen: use epydoc in MigrateInstance docstring

Reviewed-by: iustinp

ConfigObject docstring, close an open parenthesis

Reviewed-by: iustinp

Update the logging output of job processing

(this is related to the master daemon log)

Currently it's not possible to follow (in the non-debug runs) the
logical execution thread of jobs. This is due to the fact that we don't
log the thread name (so we lose the association of log messages to jobs)...

Convert RenameInstance to (status, data)

This allows the rename failures to show the ouput of OS scripts.

Reviewed-by: ultrotter

Make cluster-verify check the drbd minors space

This patch adds support for verification of drbd minors space in cluster
verify: minors which belong to running instances and should be online
but are not, and minors which do not belong to any instace but are in...

DRBD: check for in-use minor during Create

In order to prevent errors with old, in-use DRBD minors, we check and
abort at create time if our minor is already in use. For this we need to
also modify DRBD8Status to be able to parse cs:Unconfigured devices....

Some small fixes in cmdlib

Reviewed-by: ultrotter

Convert the start instance rpc to (status, data)

This will record the failure cause in starting up the instance in the
job log (and thus to the user).

Reviewed-by: ultrotter

Move the default MAC prefix to the constants file

Instead of having the default live in the gnt-cluster script, we move it
to the constants file. The patch also fixes a typo on constants.py.

Reviewed-by: ultrotter

Fix non-drbd instance creation

Commit 2294 introduced a new instance.all_nodes property, which
unfortunately is working incorrectly for non-drbd instances.

This patch fixes it by making sure the primary node is always added to
the set, even before recursing over (any potential) children....

Split the block device creation in two parts

Some callers of _CreateBlockDev need recursive behaviour, but not all.
The replace secondary first creates (manually) new LVs to ensure storage
is there, and then it creates the new DRBD. At this point, we need a...

Switch call_blockdev_create call to (status, data)

This allows errors to be visible at the user level instead of just node
daemon logs.

Reviewed-by: ultrotter

Block device creation cleanup

Currently when creation LVM-based instances, we always get the
extremely-confusing message "ERROR Can't find LV /dev/xenvg/..." which
is actually expected. This behaviour was introduced before we had
UUID-style LV names, since at that point it was not a unexpected to have...

Fix LUExportInstance

Due to deficiencies in our block device implementation, it is a must to
call SetDiskID on disks before passing them to remote nodes. Since in
export/import, we don't touch the disks themselves, this was not needed
before in this function....

Fix gnt-backup export with short names

We need to pass the fully-qualified node to _CheckNodeOnline, not the short
one.

Reviewed-by: imsnah

ganeti-noded: reduce log noise

The source port/addr is currently logged three times for each
connection, and this is unnecessary. We change two log entries to debug,
since they are useful for precise timing, and we keep only one at INFO
level.

Reviewed-by: imsnah

Port replace disk/change node to the new DRBD RPCs

In replace disks to new secondary, since Attach (and therefore
call_blockdev_find) is not modifying the devices anymore, we need to
switch this LU to the new call_drbd_disconnect_net and
call_drbd_attach_net functions....

backend: rename AttachOrAssemble to Assemble

Since now the Assemble function is different than Attach, we rename this
backend function to show that the intent is to fully assemble the device
(and it's always allowed to modify the device).

Reviewed-by: ultrotter

bdev: Do not call Assemble() on children

The caller of dev.Assemble() (backend._RecursiveAssembleBD) is doing an
explicit recursion over all the children of the device, with better
error reporting. As such, we don't need this repeated assembly inside
the base BlockDev class....

Increase resync speed to 60MB/s

This is a forward-port of commit 2219 on the 1.2 branch.

Reviewed-by: ultrotter

Fix some errors in instance modify --disk remove

The RpcResult introduction still left some bugs (after multiple patches):
- we don't correctly check the result type
- rename a variable to prevent a conflict

Reviewed-by: imsnah

Introduce a very simple LU to force config updates

This LU can be used to force a push of the config in case it's needed,
for example after an upgrade to update the ssconf_release_version file.

Reviewed-by: imsnah

Work around a DRBD sync speed race condition

This is modified forward-port of commit 1544 on the 1.2 branch:

When DRBD is doing its dance to establish a connection with its
  peer, it also sends the synchronization speed over the wire. In
  some cases setting the sync speed only after setting up both...

Fix gnt-os for offline nodes

We shouldn't query offline nodes in gnt-os. This patch adds an utility
function to ConfigWriter that returns the names of online nodes and uses
it in LUDiagnoseOS to query only the good nodes.

Reviewed-by: imsnah

Rework the daemonization sequence

The current fork+close fds sequence has deficiencies which are hard to
work around:
- logging can start logging before we fork (e.g. if we need to emit
messages related to master checking), and thus use FDs which we...

Fix cluster verify/node net test for offline nodes

For offline nodes, we shouldn't add them to the NV_NODELIST and
NV_NODENETTEST tests since they most likely won't succeed.

The patch makes gnt-cluster verify happy again in such cases.

Reviewed-by: imsnah

Add an instance_migratable rpc call

This is a forward-port of commit 1194 on the 1.2 branch:

This call will check whether an instance is up on its primary, and that
  it has been started with symlinks. We currently have no on-secondary
  checks, nor any hypervisor specific call....

backend: Remove symlinks by disk name

This is a modified forward-port of commit 1184 on the 1.2 branch:

backend: Remove symlinks by disk name, not using a wildcard

Reviewed-by: ultrotter

The changes to the original patch are related to the docstring style and...

Fix the _RemoveBlockDevLinks() function

This is a forward-port of commit 1163 on the 1.2 branch:
This fixes the removal of the instance symlinks (probably breakage from
the glob changes).

Reviewed-by: imsnah

Catch BlockDeviceError when starting instance

This is a forward-port of commit 1149 on the 1.2 branch:
_GatherAndLinkBlockDevs used to raise the errors.BlockDeviceError
exception when it failed to create a block device, and with this patch
set it does so also when it fails to create a symlink to it....

Simplify hypervisor block_devices structure

This is a partial forward-port of commit 1136 on the 1.2 branch:

The hypervisor doesn't need to be passed the whole block device
  structure, so we'll just give it the block device name on the local
  node, and the name as seen by the instance. This will make it easier to...

Fix some pylint-detected issues

Two bad indentation cases and a missing variable.

Reviewed-by: imsnah

ganeti.cmdlib: Check remote API certificate on "gnt-cluster verify"

Reviewed-by: amishchenko

ganeti.bootstrap: Prepare for remote API certificate

Reviewed-by: amishchenko

ganeti.bootstrap: Generate SSL certificate for remote API

Reviewed-by: amishchenko

ganeti-rapi: Implement HTTP authentication

Passwords are stored in "$localstatedir/lib/ganeti/rapi_users". User
options specify the access permissions of a user (see docstring for
ganeti.http.ReadPasswordFile), for which only "write" is supported
to grant write access. Every other user has read-only access....

ganeti.http: Add support for private data in HTTP requests

Reviewed-by: amishchenko

ganeti.http: Prepare authentication for HTTP server

The authentication class will override PreHandleRequest.

Reviewed-by: amishchenko

ganeti.jqueue: Group job archivals to reduce number of RPC calls

Reducing the actual number of RPC calls will come in another patch.

Reviewed-by: ultrotter

jqueue: When auto-archiving jobs, calculate job status only once

This is done by passing the job object to _ArchiveJobUnlocked instead
of only the job ID. Also return whether job was actually archived.

Reviewed-by: ultrotter

Add rename function automatically creating directories if needed

Unfortunately, os.makedirs in Python 2.4 is not safe against multiple
processes creating the same directory tree at the same time. This is
only fixed in Python 2.5 and up. Adding more checks in our code doesn't...

Rename http.HttpInternalError to HttpInternalServerError

All other exceptions are named after the error name in RFC2616 (HTTP/1.1).

Reviewed-by: amishchenko

ganeti.http: Ignore ENOTCONN when shutting down the connection

Reviewed-by: amishchenko

Add simple unittests for ganeti.http

More complex unittests will need some refactoring in the HTTP code.

Reviewed-by: amishchenko

Add job queue size limit

A job queue with too many jobs can increase memory usage and/or make
the master daemon slow. The current limit is just an arbitrary number.
A "soft" limit for automatic job archival is prepared.

Reviewed-by: iustinp

LUConnectConsole: fix primary_node online check

The primary node is part of the instance, not of the opcode.

Reviewed-by: iustinp