Synnefo » snf-ganeti

Parallelize Tag operations

For now we lock the instance/node for adding/deleting tags from it, but
we could probably in the future do without, with more support from the
config for atomic operations.

Reviewed-by: iustinp

Fix a typo in gnt-instance.sgml

Reviewed-by: imsnah

Parallelize LURemoveInstance

Using the new add/remove infrastructure this becomes pretty easy! :)

Reviewed-by: imsnah

Implement adding/removal of locks by declaration

With this patch LUs can declare locks to be added when they start and/or
removed after they finish. For now locks can only be added in the
acquired state, and removed if owned, and added locks default to be...

Fix typo in a locking.py comment

Reviewed-by: imsnah

Add GanetiLockManager.is_owned function

This is a public version of the private function we already had.
We don't just change the previous version because it had lots of users
in the library itself and in the testing code.

Reviewed-by: imsnah

Add gnt-instance (start|stop) --submit

Finish the --submit changes with these two, which (because they are
multi-opcode commands) require special handling.

Reviewed-by: ultrotter

Remove locking of instances in certain queries

This patch is similar to the node patch (rev 1650). We disable locking
of instance (and nodes) if we only query static information.

Reviewed-by: ultrotter

Add ConfigWriter._UnlockedGetInstanceList/Info()

This patch splits the GetInstanceInfo and GetInstanceList methods into
two parts, one locked one _Unlocked similar to the way nodes are
queried.

Reviewed-by: ultrotter

Implement "--submit" on gnt-instance

This patch adds support for the “--submit” parameter in the gnt-instance
script, for the commands where it makes sense.

Reviewed-by: ultrotter

Optimize the OpQueryNodes for names only

Currently, OpQueryNodes is locking all nodes (in shared mode), which
will also block the special case of querying only for the node names
(this is needed for gnt-cluster command, for example). There is no
logical requirement to not give the administrator enough power if she/he...

ganeti-noded: Add constant for queue lock timeout

Reviewed-by: iustinp

Implement master startup safety check

This is an initial version of the master startup checks. It's a very
rudimentary change, however in normal usage (an old master was started,
the rest of the cluster is functioning normally) it will succeed in
preventing wrong startups....

Change backend._GetMasterInfo to return more data

The _GetMasterInfo() function needs to export the master name too to be
useful in master safety checks. This patch makes it a public (no _)
function and adds a third element in the return tuple. Its callers are...

Parallelize LUVerify{Cluster,Disks}

These are two easy querying LUs which require shared access to all
nodes/instances.

Reviewed-by: iustinp

_LockInstancesNodes: support append mode

This will be used to lock the instance's nodes in addition to some more.

Reviewed-by: iustinp

Parallelize LU{A,Dea}ctivateInstanceDisks

Now that they are not used in other opcodes by chaining,
this can easily be done.

Reviewed-by: iustinp

Create new _SafeShutdownInstanceDisks function

This new function checks whether an instance is running, before shutting
down its disks. This is what the Exec() of LUDeactivateInstanceDisks
did, so that is replaced by a call to this function.

Reviewed-by: iustinp

Use constants.LOCKS_REPLACE instead of hardcoding

This constant replaces what we used to write in recalculate_locks, and
represents the lock recalculation mode. It lives in constants.py because
it's used only in cmdlib, and thus doesn't deal with the locking library...

Use lock timeout for queue updates in ganeti-noded

This helps to prevent complete deadlocks.

Reviewed-by: iustinp

Fix iallocator run

The rpc library returns a list, not a tuple, so we'll accept both.

Reviewed-by: iustinp

Parallelize LUExportInstance

Unfortunately for the first version we need to lock all nodes. The patch
discusses why this is and discuss ways to improve this in the future.

Reviewed-by: iustinp

LURebootInstance: lock only primary when possible

When rebooting an instance and we're not changing it's disks status (all
the cases except in a "full" reboot) we can lock just its primary node.

Reviewed-by: iustinp

utils.FileLock: Implement timeout

The timeout can be used in ganeti-noded to be more robust against
deadlocks.

Reviewed-by: iustinp

noded: Get job queue lock while purging queue content

Only one process should modify the queue at the same time.

Reviewed-by: iustinp

Add locking.ALL_SET constant and use it

Rather than specifying None in needed_locks every time, with a nice
comment saying to read what we mean rather than what we write, and that
None actually means All, in our magic world, we'll hide this secret
under the ALL_SET constant in the locking module, which has value, you...

Remove bom-byte

This is not nice, removing it :)

Please use 'set nobomb' in your vi init file.

Reviewed-by: ultrotter

Paralleliza LUQueryNodeVolumes and LUQueryExports

Reviewed-by: iustinp

LUQueryExports: make 'node' field mandatory

It turns out this fields was already mandatory. If it hadn't beed valid,
in fact, a value of None would have been passed to _GetWantedNodes which
would have thrown an exception.

Reviewed-by: iustinp

Fix wrong indentation in LUQueryNodes

Reviewed-by: iustinp

merge r1569 from branches/ganeti/ganeti-1.2

Implement more options for gnt-backup import

Reviewed-by: ultrotter

merge r1548 from branches/ganeti/ganeti-1.2

Fix wrong wording of instance rename error message.

Reviewed-by: imsnah

merge r1542, r1543, r1573 from branches/ganeti/ganeti-1.2

Implement interactive instance OS reinstall.

Reviewed-by: ultrotter

merge r1540 from branches/ganeti/ganeti-1.2

Allow access to HVM serial console

Reviewed-by: imsnah

merge r1538 from branches/ganeti/ganeti-1.2

Check HVM device type on instance modify as well.

Reviewed-by: imsnah

Pass the force param to SetInstanceParms

It was already allowed in gnt-instance modify, but ignored.
It will be used to force skipping parameter checks.

This is a forward-port from branches/ganeti-1.2

Original-Reviewed-by: imsnah
Reviewed-by: iustinp

Merge r1537 from branches/ganeti/ganeti-1.2

Add HVM device type flags 3/4

Reviewed-by: ultrotter

utils.SplitTime: Fix rounding of milliseconds

Reported by Iustin.

It used to return this:

utils.SplitTime(1234.999999999999)

(1234, 1000)

while it should've returned this:

utils.SplitTime(1234.999999999999)

(1235, 0)

Reviewed-by: ultrotter

Merge r1296 from branches/ganeti/ganeti-1.2

doc fix: Describe default values for HVM instance options & cleanup.

Reviewed-by: iustinp

Make WaitForJobChanges deal with long jobs

This patch alters the WaitForJobChanges luxi-RPC call to have a
configurable timeout, so that the call behaves nicely with long jobs
that have no update.

We do this by adding a timeout parameter in the RPC call, and returning...

Add doc/locking.txt, documenting locking order

Reviewed-by: imsnah

Fix issue when acquiring empty lock sets

By design if an empty list of locks is acquired from a set, no locks are
acquired, and thus release() cannot be called on the set. On the other
hand if None is passed instead of the list, the whole set is acquired,...

jqueue: Keep timestamp of opcode start and end

Reviewed-by: ultrotter

Another burnin fix

This is a result of the log timestamp changes.

Reviewed-by: imsnah

Make sure that client programs get all messages

This is a large patch, but I can't figure out how to split it without
breaking stuff. The old way of getting messages by always getting the
last one didn't bring all messages to the client if they were added...

Add simple lock debug output

Currently it can only be enabled by modifying utils.py, but we can
add a command line parameter later if needed.

Reviewed-by: schreiberal

Remove references to YAML

I forgot to remove these when converting the QA configuration from YAML
to JSON.

Reviewed-by: ultrotter

Parallelize LUQueryNodes

As for LUQueryInstances the first version just acquires a shared lock on all
nodes. In the future further optimizations are possible, as outlined by
comments in the code.

Reviewed-by: imsnah

A few more locking unit tests

A few more tests written while bug-hunting. One of them shows a real
issue, at last. :)

Reviewed-by: imsnah

LockSet: allow lists with duplicate values

If a list with a duplicate value is passed to a lockset what the code
now does is to try to acquire the lock twice, generating a
double-acquire exception in the SharedLock code. This is definitely an
issue. In order to solve it we can either forbit double values in a list...