Synnefo » snf-ganeti » ganeti-local

Use lock timeout for queue updates in ganeti-noded

This helps to prevent complete deadlocks.

Reviewed-by: iustinp

Fix iallocator run

The rpc library returns a list, not a tuple, so we'll accept both.

Reviewed-by: iustinp

Parallelize LUExportInstance

Unfortunately for the first version we need to lock all nodes. The patch
discusses why this is and discuss ways to improve this in the future.

Reviewed-by: iustinp

LURebootInstance: lock only primary when possible

When rebooting an instance and we're not changing it's disks status (all
the cases except in a "full" reboot) we can lock just its primary node.

Reviewed-by: iustinp

utils.FileLock: Implement timeout

The timeout can be used in ganeti-noded to be more robust against
deadlocks.

Reviewed-by: iustinp

noded: Get job queue lock while purging queue content

Only one process should modify the queue at the same time.

Reviewed-by: iustinp

Add locking.ALL_SET constant and use it

Rather than specifying None in needed_locks every time, with a nice
comment saying to read what we mean rather than what we write, and that
None actually means All, in our magic world, we'll hide this secret
under the ALL_SET constant in the locking module, which has value, you...

Remove bom-byte

This is not nice, removing it :)

Please use 'set nobomb' in your vi init file.

Reviewed-by: ultrotter

Paralleliza LUQueryNodeVolumes and LUQueryExports

Reviewed-by: iustinp

LUQueryExports: make 'node' field mandatory

It turns out this fields was already mandatory. If it hadn't beed valid,
in fact, a value of None would have been passed to _GetWantedNodes which
would have thrown an exception.

Reviewed-by: iustinp

Fix wrong indentation in LUQueryNodes

Reviewed-by: iustinp

merge r1569 from branches/ganeti/ganeti-1.2

Implement more options for gnt-backup import

Reviewed-by: ultrotter

merge r1548 from branches/ganeti/ganeti-1.2

Fix wrong wording of instance rename error message.

Reviewed-by: imsnah

merge r1542, r1543, r1573 from branches/ganeti/ganeti-1.2

Implement interactive instance OS reinstall.

Reviewed-by: ultrotter

merge r1540 from branches/ganeti/ganeti-1.2

Allow access to HVM serial console

Reviewed-by: imsnah

merge r1538 from branches/ganeti/ganeti-1.2

Check HVM device type on instance modify as well.

Reviewed-by: imsnah

Pass the force param to SetInstanceParms

It was already allowed in gnt-instance modify, but ignored.
It will be used to force skipping parameter checks.

This is a forward-port from branches/ganeti-1.2

Original-Reviewed-by: imsnah
Reviewed-by: iustinp

Merge r1537 from branches/ganeti/ganeti-1.2

Add HVM device type flags 3/4

Reviewed-by: ultrotter

utils.SplitTime: Fix rounding of milliseconds

Reported by Iustin.

It used to return this:

utils.SplitTime(1234.999999999999)

(1234, 1000)

while it should've returned this:

utils.SplitTime(1234.999999999999)

(1235, 0)

Reviewed-by: ultrotter

Merge r1296 from branches/ganeti/ganeti-1.2

doc fix: Describe default values for HVM instance options & cleanup.

Reviewed-by: iustinp

Make WaitForJobChanges deal with long jobs

This patch alters the WaitForJobChanges luxi-RPC call to have a
configurable timeout, so that the call behaves nicely with long jobs
that have no update.

We do this by adding a timeout parameter in the RPC call, and returning...

Add doc/locking.txt, documenting locking order

Reviewed-by: imsnah

Fix issue when acquiring empty lock sets

By design if an empty list of locks is acquired from a set, no locks are
acquired, and thus release() cannot be called on the set. On the other
hand if None is passed instead of the list, the whole set is acquired,...

jqueue: Keep timestamp of opcode start and end

Reviewed-by: ultrotter

Another burnin fix

This is a result of the log timestamp changes.

Reviewed-by: imsnah

Make sure that client programs get all messages

This is a large patch, but I can't figure out how to split it without
breaking stuff. The old way of getting messages by always getting the
last one didn't bring all messages to the client if they were added...

Add simple lock debug output

Currently it can only be enabled by modifying utils.py, but we can
add a command line parameter later if needed.

Reviewed-by: schreiberal

Remove references to YAML

I forgot to remove these when converting the QA configuration from YAML
to JSON.

Reviewed-by: ultrotter

Parallelize LUQueryNodes

As for LUQueryInstances the first version just acquires a shared lock on all
nodes. In the future further optimizations are possible, as outlined by
comments in the code.

Reviewed-by: imsnah

A few more locking unit tests

A few more tests written while bug-hunting. One of them shows a real
issue, at last. :)

Reviewed-by: imsnah

LockSet: allow lists with duplicate values

If a list with a duplicate value is passed to a lockset what the code
now does is to try to acquire the lock twice, generating a
double-acquire exception in the SharedLock code. This is definitely an
issue. In order to solve it we can either forbit double values in a list...

LURebootInstance: move arg check in ExpandNames

The check for the reboot type can be done without any locks held, so
we'll move it to ExpandNames. Plus, we note in a FIXME that if the
reboot type is not full, we can probably just lock the primary node, and...

LUVerifyCluster: Return boolean indication success

Reviewed-by: schreiberal

QA: Try to run more scripts with --version

This patch also sorts the list.

Reviewed-by: schreiberal

QA: Do not upload known_hosts file anymore

The cluster no longer keeps individual host's SSH key, but rather
aliases all of them to the cluster name.

Reviewed-by: ultrotter

gnt-node: Add option to always accept peer's SSH key

This option will be used to add nodes to the cluster without
asking the user to confirm the key. Together with key based
authentication this can be used in the QA tests.

Reviewed-by: ultrotter

Move SSH option building into a function

I'm going to add another option and it would make maintaining
them in constants even more complicated.

Reviewed-by: ultrotter

Whitespace fixes for remote API QA checks

Reviewed-by: ultrotter

Pass hypervisor type to the OS scripts

It's handy to make the os scripts know which hypervisor the instance is
going to run under. In order not to change the os API we pass this
information in the environment, where the os scripts can access it if
they're hypervisor-aware....

KVM Hypervisor Cleanup

- Remove a few experiemental code lines left as comments
- Rework first disks' boot=on addition, which was calculated twice
- Remove an empty line
- Remove reference to hvm_pae which doesn't apply to kvm

Reviewed-by: imsnah

Add KVM hypervisor code

ht_kvm.py contains the code for ganeti to work under kvm.
This patch also modifies Makefile.am to ship that file, and
lib/hypervisor/__init__.py to import it, and add kvm to the
hypervisors map.

Reviewed-by: imsnah

Add --with-kvm-path configure option

This allows to configure a different path to the kvm binary. By default
/usr/bin/kvm is used, which is the one found in debian and ubuntu.

Reviewed-by: imsnah

Convert RunCmd to an epydoc docstring

Reviewed-by: imsnah

Fix race locking issue in noded

Noded didn't release the job queue lock after initialising it. This
patch makes sure to unlock once the work is done.

Reviewed-by: ultrotter

Add RPC call to wait for job changes

This way clients can react faster to status or message changes and
don't have to poll anymore.

Reviewed-by: ultrotter

Add functions to split time into tuple and merge it back

These will be used for job logs.

Reviewed-by: ultrotter

Add query function for exports

Reviewed-by: iustinp

backend: Add optional exclusion list to _CleanDirectory

The code cleaning the queue will make use of it.

Reviewed-by: iustinp

noded: Add RPC function to rename job queue files

This will be used to archive jobs.

Reviewed-by: iustinp

noded: Add decorator for job queue lock

The lock will also be needed by another function.

Reviewed-by: iustinp