| Branch: | Tag: | Revision:

root / lib @ a1578d63

# Date Author Comment
a1578d63 09/23/2008 03:10 pm Iustin Pop

Switch to static minors for DRBD

With some todos remaining, this patch switches the DRBD devices to use
the passed minors, and the cmdlib code (add instance and replace disks)
to request and assign minors to the DRBD disks.

- look at the disk RPC calls to see which can be optimized away, since...

a81c53c9 09/23/2008 03:10 pm Iustin Pop

Implement config support for drbd static minors

This patch adds support for allocating static minors.

Like for the LVM uuids, we add a new cache for the temporarily allocated
requests, and the users of the new methods must manually clear the
cache. If this doesn't happen, at worst we lose some minors....

468b46f9 09/23/2008 03:10 pm Iustin Pop

Fix disk replace secondary with static minors

The code in 'updating instance configuration' section of the replace
disks with change secondary node was setting a wrong new logical_id for
the drbd devices (only set the new node, not the new minor). The patch...

ffa1c0dc 09/22/2008 02:32 pm Iustin Pop

Extend DRBD disks with minors attribute

This patch converts the DRBD disks to contain also a minor (per each
node) attribute. This minor is not yet used and is always initialized
with None, so the patch does not have any real-world impact - except for
automatically upgrading config files (it adds the minors as None, None)....

3fa93523 09/18/2008 02:13 pm Guido Trotter

Apply filter properly in LUQuery{Nodes, Instances}

Currently when not locking all nodes/instances are returned, regardless
if the user asked only for some of them. With this patch we return to
the previous behaviour:
- if no names are specified return info on all current ones...

c2c2a903 09/18/2008 02:12 pm Guido Trotter

Remove auto_balance from burnin/cmdlib

There is no such feature in trunk yet.

Reviewed-by: iustinp

ca0aa6d0 09/17/2008 07:07 pm Michael Hanselmann

Add utils.ReadFile function

It abstracts exception handling and is like a complement to

Reviewed-by: iustinp

64d3bd52 09/11/2008 08:45 pm Guido Trotter

GetAllInstancesInfo, change internal iterator name

GetAllInstancesInfo used "node" as an iterator name. Change it to
instance to make it less confusing.

Reviewed-by: iustinp

8646adce 09/11/2008 08:45 pm Guido Trotter

Parallelize Tag operations

For now we lock the instance/node for adding/deleting tags from it, but
we could probably in the future do without, with more support from the
config for atomic operations.

Reviewed-by: iustinp

c53279cf 09/11/2008 08:44 pm Guido Trotter

Parallelize LUSetClusterParams (and add a FIXME)

Reviewed-by: imsnah

3656b3af 09/11/2008 12:44 pm Guido Trotter

Parallelize LURemoveExport

Reviewed-by: imsnah

cf472233 09/11/2008 12:44 pm Guido Trotter

Parallelize LURemoveInstance

Using the new add/remove infrastructure this becomes pretty easy! :)

Reviewed-by: imsnah

7baf741d 09/11/2008 12:44 pm Guido Trotter

Parallelize LUCreateInstance

Finally, instance create on different node, without iallocator, can run
in parallel. Iallocator usage still needs all nodes to be locked,
unfortunately. As a bonus most checks which could have been moved to
ExpandNames, before any locking is done....

ca2a79e1 09/11/2008 12:44 pm Guido Trotter

Implement adding/removal of locks by declaration

With this patch LUs can declare locks to be added when they start and/or
removed after they finish. For now locks can only be added in the
acquired state, and removed if owned, and added locks default to be...

d2aff862 09/11/2008 12:44 pm Guido Trotter

LockSet: forbid add() on a partially owned set

This patch bans add() on a half-acquired set. This behavior was
previously possible, but created a deadlock if someone tried to acquire
the set-lock in the meantime, and thus is now forbidden. The
testAddRemove unit test is fixed for this new behavior, and includes a...

ab62526c 09/11/2008 12:43 pm Guido Trotter

Fix typo in a comment

Reviewed-by: imsnah

80ee04a4 09/11/2008 12:43 pm Guido Trotter

Use is_owned to determine whether to unlock

Now that is_owned is public we don't need to play games at the end of an
LU. If we're still owning anything we just release it.

Reviewed-by: imsnah

d4f4b3e7 09/11/2008 12:43 pm Guido Trotter

Add GanetiLockManager.is_owned function

This is a public version of the private function we already had.
We don't just change the previous version because it had lots of users
in the library itself and in the testing code.

Reviewed-by: imsnah

d4803c24 09/11/2008 12:43 pm Guido Trotter

Fix LockSet._names() to work with the set-lock

If the set-lock is acquired, currently, the _names function will fail on
a double acquire of a non-recursive lock. This patch fixes the behavior,
and some lines of code added to the testAcquireSetLock test check that...

e74798c1 09/10/2008 08:46 pm Michael Hanselmann

jqueue: Add common RPC error handling function

We didn't decide yet what exactly it should do with failed nodes.

Reviewed-by: ultrotter

57a2fb91 09/10/2008 08:07 pm Iustin Pop

Remove locking of instances in certain queries

This patch is similar to the node patch (rev 1650). We disable locking
of instance (and nodes) if we only query static information.

Reviewed-by: ultrotter

0b2de758 09/10/2008 08:07 pm Iustin Pop

Add an atomic ConfigWrite.GetAllInstanceInfo()

In order to be able to query instance without locking them, we need the
same atomic query of multiple instances as for nodes.

Reviewed-by: ultrotter

94bbfece 09/10/2008 08:06 pm Iustin Pop

Add ConfigWriter._UnlockedGetInstanceList/Info()

This patch splits the GetInstanceInfo and GetInstanceList methods into
two parts, one locked one _Unlocked similar to the way nodes are

Reviewed-by: ultrotter

e9d741b6 09/10/2008 06:43 pm Iustin Pop

Rewrite the 'only submit job' handling in scripts

The "sys.exit(0)" was not nice as you couldn't differentiate it from
other exit codes. We change this to a specially defined exception for
this, so that multi-opcode commands can handle this nicely.

Reviewed-by: imsnah

c8d8b4c8 09/10/2008 02:03 pm Iustin Pop

Optimize the OpQueryNodes for names only

Currently, OpQueryNodes is locking all nodes (in shared mode), which
will also block the special case of querying only for the node names
(this is needed for gnt-cluster command, for example). There is no
logical requirement to not give the administrator enough power if she/he...

d65e5776 09/10/2008 02:02 pm Iustin Pop

Add a way to export all node information at once

The patch adds a new function to export all node information at once
(i.e. atomically with respect to the configuration lock).

Reviewed-by: ultrotter

1bc59f76 09/09/2008 03:47 pm Michael Hanselmann

Never remove job queue lock in node daemon

Otherwise, corruption could occur in some corner cases. E.g. when
LeaveNode is running in a child and is in the process of removing
queue files, the main process gets killed, started again and gets
a request to update the queue. This is rather extreme corner case,...

4e071d3b 09/09/2008 03:24 pm Iustin Pop

Export backend.GetMasterInfo over the rpc layer

We create a multi-node call so that querying all nodes for agreement
will be fast.

Reviewed-by: imsnah

bd1e4562 09/09/2008 03:24 pm Iustin Pop

Change backend._GetMasterInfo to return more data

The _GetMasterInfo() function needs to export the master name too to be
useful in master safety checks. This patch makes it a public (no _)
function and adds a third element in the return tuple. Its callers are...

a987fa48 09/09/2008 01:42 pm Guido Trotter

Parallelize LUQueryInstanceData

Reviewed-by: iustinp

d4b9d97f 09/09/2008 01:42 pm Guido Trotter

Parallelize LUVerify{Cluster,Disks}

These are two easy querying LUs which require shared access to all

Reviewed-by: iustinp

efd990e4 09/09/2008 01:41 pm Guido Trotter

Parallelize LUReplaceDisks

This is the most complex parallelization so far. We have to lock one
instance (and its nodes) plus one more node if doing a remote replace,
or all nodes if doing a remote replace with iallocator.

Reviewed-by: iustinp

9513b6ab 09/09/2008 01:41 pm Guido Trotter

_LockInstancesNodes: support append mode

This will be used to lock the instance's nodes in addition to some more.

Reviewed-by: iustinp

b2751b57 09/09/2008 01:41 pm Guido Trotter

Processor: remove ChainOpCode

This function was incompatible with the new locking system, and its
usage has been removed from the code. For now LUs share code by calling
common module-private functions in, in the future they will
use tasklets (when those will be implemented)....

f22a8ba3 09/09/2008 01:41 pm Guido Trotter

Parallelize LU{A,Dea}ctivateInstanceDisks

Now that they are not used in other opcodes by chaining,
this can easily be done.

Reviewed-by: iustinp

023e3296 09/09/2008 01:40 pm Guido Trotter

LUReplaceDisks: remove use of ChainOpCode

The calls to OpActivateInstanceDisks and OpDeactivateInstanceDisks has
been replaced by _StartInstanceDisks and _SafeShutdownInstanceDisks
respectively. This is the last usage of ChainOpCode.

Reviewed-by: iustinp

155d6c75 09/09/2008 01:40 pm Guido Trotter

Create new _SafeShutdownInstanceDisks function

This new function checks whether an instance is running, before shutting
down its disks. This is what the Exec() of LUDeactivateInstanceDisks
did, so that is replaced by a call to this function.

Reviewed-by: iustinp

3a5d7305 09/09/2008 01:40 pm Guido Trotter

Fix a typo in LogicalUnit.ExpandNames docstring


Reviewed-by: iustinp

f6d9a522 09/09/2008 01:40 pm Guido Trotter

Use constants.LOCKS_REPLACE instead of hardcoding

This constant replaces what we used to write in recalculate_locks, and
represents the lock recalculation mode. It lives in because
it's used only in cmdlib, and thus doesn't deal with the locking library...

de8c7666 09/09/2008 12:39 pm Guido Trotter

Fix LUReplaceDisks with iallocator

self._RunAllocator() sets self.op.remote_node, but doesn't return the
new remote node. If we set it to the return value of the function we
basically reset it to None, and iallocator is never run.

Reviewed-by: imsnah

86de84dd 09/08/2008 06:54 pm Guido Trotter

Fix LUGrowDisk

The rpc library returns a list, not a tuple, so we'll accept both.

Reviewed-by: iustinp

43f5ea7a 09/08/2008 06:53 pm Guido Trotter

Fix iallocator run

The rpc library returns a list, not a tuple, so we'll accept both.

Reviewed-by: iustinp

6657590e 09/08/2008 04:44 pm Guido Trotter

Parallelize LUExportInstance

Unfortunately for the first version we need to lock all nodes. The patch
discusses why this is and discuss ways to improve this in the future.

Reviewed-by: iustinp

31e63dbf 09/08/2008 04:44 pm Guido Trotter

Parallelize LUGrowDisk

Reviewed-by: iustinp

849da276 09/08/2008 04:43 pm Guido Trotter

LURebootInstance: lock only primary when possible

When rebooting an instance and we're not changing it's disks status (all
the cases except in a "full" reboot) we can lock just its primary node.

Reviewed-by: iustinp

a82ce292 09/08/2008 04:43 pm Guido Trotter

Add primary_only flag to _LockInstancesNodes

As the name says when the flag is on (the default is off) only the
primary nodes are locked, as opposed to all of them.

Reviewed-by: iustinp

aa74b828 09/05/2008 06:38 pm Michael Hanselmann

utils.FileLock: Implement timeout

The timeout can be used in ganeti-noded to be more robust against

Reviewed-by: iustinp

e310b019 09/05/2008 02:00 pm Guido Trotter

Add locking.ALL_SET constant and use it

Rather than specifying None in needed_locks every time, with a nice
comment saying to read what we mean rather than what we write, and that
None actually means All, in our magic world, we'll hide this secret
under the ALL_SET constant in the locking module, which has value, you...

45bc5e4a 09/05/2008 01:57 pm Michael Hanselmann

utils.SplitTime: More rounding fixes

SplitTime didn't round the same on different platforms. This patch changes
it to use microseconds and not care about rounding.

Reviewed-by: iustinp

ea47808a 09/04/2008 06:12 pm Guido Trotter

Prevent mistakes using _GetWantedNodes

All the users of _GetWantedNodes have been converted to be concurrent
LUs, and thus cannot call this function with an empty list of nodes
anymore. This patch makes this restriction a part of the function
itself. This prevents mistakes in new concurrent LUs, and creates more...

21a15682 09/04/2008 06:12 pm Guido Trotter

Paralleliza LUQueryNodeVolumes and LUQueryExports

Reviewed-by: iustinp

6bf01bbb 09/04/2008 06:12 pm Guido Trotter

Parallelize LUDiagnoseOS

Reviewed-by: iustinp

895ecd9c 09/04/2008 06:12 pm Guido Trotter

LUQueryExports: make 'node' field mandatory

It turns out this fields was already mandatory. If it hadn't beed valid,
in fact, a value of None would have been passed to _GetWantedNodes which
would have thrown an exception.

Reviewed-by: iustinp

204f2086 09/04/2008 06:11 pm Guido Trotter


Parallel opcodes are not (yet?) supported for chaining. Turns out
though that chaining is used only four times in the code, and twice it's
for querying exports. But what's the need to chain the full opcode, when...

b91a34a5 09/04/2008 06:11 pm Guido Trotter

Fix wrong indentation in LUQueryNodes

Reviewed-by: iustinp

d0c11cf7 09/04/2008 05:53 pm Alexander Schreiber

Merge r1607 from branches/ganeti/ganeti-1.2

Use a default vnc_bind_address if None is specified

Reviewed-by: iustinp

3fb1e1c5 09/02/2008 03:57 pm Alexander Schreiber

merge r1568 from branches/ganeti/ganeti-1.2

Add more fields to gnt-instance list

Reviewed-by: imsnah

6291574d 09/02/2008 03:15 pm Alexander Schreiber

merge r1548 from branches/ganeti/ganeti-1.2

Fix wrong wording of instance rename error message.

Reviewed-by: imsnah

a4273aba 09/02/2008 12:09 pm Alexander Schreiber

merge r1541 from branches/ganeti/ganeti-1.2

more information for VNC console port

Reviewed-by: ultrotter

04c4330c 09/02/2008 11:42 am Alexander Schreiber

merge r1540 from branches/ganeti/ganeti-1.2

Allow access to HVM serial console

Reviewed-by: imsnah

34b6ab97 09/01/2008 07:05 pm Alexander Schreiber

merge r1539 from branches/ganeti/ganeti-1.2

Display VNC console port in gnt-instance info.

Reviewed-by: iustinp

5bc84f33 09/01/2008 05:12 pm Alexander Schreiber

merge r1538 from branches/ganeti/ganeti-1.2

Check HVM device type on instance modify as well.

Reviewed-by: imsnah

cfefe007 09/01/2008 02:37 pm Guido Trotter

Check memory size before setting it

With this change when a user asks for a new memory size for an instance,
the number is checked instead of just applied. The operation fails only
if the instance would not be able to restart on its primary node, but
generates warnings should it be impossible to failover the instance or...

4300c4b6 09/01/2008 02:37 pm Guido Trotter

Pass the force param to SetInstanceParms

It was already allowed in gnt-instance modify, but ignored.
It will be used to force skipping parameter checks.

This is a forward-port from branches/ganeti-1.2

Original-Reviewed-by: imsnah
Reviewed-by: iustinp

5397e0b7 08/29/2008 07:17 pm Alexander Schreiber

Merge r1536 from branches/ganeti/ganeti-1.2

Add HVM device type flags 2/3

Reviewed-by: ultrotter

b77ba978 08/29/2008 06:04 pm Michael Hanselmann

utils.SplitTime: Fix rounding of milliseconds

Reported by Iustin.

It used to return this:


(1234, 1000)

while it should've returned this:


(1235, 0)

Reviewed-by: ultrotter

b894f5a8 08/29/2008 06:01 pm Alexander Schreiber

merge r1535 from branches/ganeti/ganeti-1.2

Add HVM device type flags 1/4

Reviewed-by: ultrotter

5c735209 08/29/2008 04:42 pm Iustin Pop

Make WaitForJobChanges deal with long jobs

This patch alters the WaitForJobChanges luxi-RPC call to have a
configurable timeout, so that the call behaves nicely with long jobs
that have no update.

We do this by adding a timeout parameter in the RPC call, and returning...

3fc175f0 08/29/2008 03:47 pm Alexander Schreiber

merge r997 from branches/ganeti/ganeti-1.2

Fix gnt-instance modify for HVM parameters

This patch makes gnt-instance modify work again for the advanced
HVM parameters after it was broken by other changes.

Reviewed-by: ultrotter

082c5adb 08/28/2008 06:35 pm Michael Hanselmann

Fix error message when masterd is not listening

Reported by Iustin.

Reviewed-by: iustinp

6683bba2 08/28/2008 01:29 pm Guido Trotter

Fix issue when acquiring empty lock sets

By design if an empty list of locks is acquired from a set, no locks are
acquired, and thus release() cannot be called on the set. On the other
hand if None is passed instead of the list, the whole set is acquired,...

5685c1a5 08/27/2008 05:52 pm Michael Hanselmann

jqueue: Replace normal cache dict with weakref dict

A job should only exist once in memory. After the cache is cleaned,
there can still be references to a job somewhere else. If there
are multiple instances, one can get updated while a function is
waiting for changes on another instance. By using...

70552c46 08/27/2008 05:52 pm Michael Hanselmann

jqueue: Keep timestamp of opcode start and end

Reviewed-by: ultrotter

65548ed5 08/27/2008 05:48 pm Michael Hanselmann

jqueue: Reset run_op_idx after job is done

It can be confusing otherwise.

Reviewed-by: ultrotter

6abe9194 08/27/2008 12:55 pm Iustin Pop

Fix a small typo in a constant

Seems noone ran a burnin lately :)

Reviwed-by: amischenko,ultrotter

6c5a7090 08/27/2008 11:34 am Michael Hanselmann

Make sure that client programs get all messages

This is a large patch, but I can't figure out how to split it without
breaking stuff. The old way of getting messages by always getting the
last one didn't bring all messages to the client if they were added...

e67bd559 08/26/2008 06:44 pm Michael Hanselmann

Add simple lock debug output

Currently it can only be enabled by modifying, but we can
add a command line parameter later if needed.

Reviewed-by: schreiberal

35705d8f 08/18/2008 03:51 pm Guido Trotter

Parallelize LUQueryNodes

As for LUQueryInstances the first version just acquires a shared lock on all
nodes. In the future further optimizations are possible, as outlined by
comments in the code.

Reviewed-by: imsnah

7eb9d8f7 08/18/2008 03:51 pm Guido Trotter

Parallelize LUQueryInstances

This first version acquires a shared lock on all requested instances and
their nodes. In the future it can be improved by acquiring less locks if
no dynamic fields have been asked, and/or by locking just primary nodes.

Reviewed-by: imsnah

34ca3914 08/18/2008 03:50 pm Guido Trotter

LockSet: allow lists with duplicate values

If a list with a duplicate value is passed to a lockset what the code
now does is to try to acquire the lock twice, generating a
double-acquire exception in the SharedLock code. This is definitely an
issue. In order to solve it we can either forbit double values in a list...

8a2941c4 08/18/2008 03:49 pm Guido Trotter

Processor: lock all levels even if one is missing

If a locking level wasn't specified locking used to stop. This means
that if one, for example, didn't specify anything at the LEVEL_INSTANCE
level, no locks at the LEVEL_NODE level were acquired either. With this...

0fcc5db3 08/18/2008 03:44 pm Guido Trotter

LURebootInstance: move arg check in ExpandNames

The check for the reboot type can be done without any locks held, so
we'll move it to ExpandNames. Plus, we note in a FIXME that if the
reboot type is not full, we can probably just lock the primary node, and...

34290825 08/18/2008 02:37 pm Michael Hanselmann

LUVerifyCluster: Return boolean indication success

Reviewed-by: schreiberal

9894ece7 08/18/2008 02:12 pm Michael Hanselmann

Use Linux-specific way to name master socket

By using this Linux-specific way we don't have to care about removing the
socket file when quitting or starting (after an unclean shutdown). For a
more detailed description, see the comment in the patch.

Reviewed-by: schreiberal

c4b6c29c 08/15/2008 11:55 am Michael Hanselmann

gnt-node: Add option to always accept peer's SSH key

This option will be used to add nodes to the cluster without
asking the user to confirm the key. Together with key based
authentication this can be used in the QA tests.

Reviewed-by: ultrotter

652d6694 08/15/2008 11:47 am Michael Hanselmann

SshRunner: Add parameter to always accept peer's SSH key

This will be used to add nodes without user interaction, specifically
in QA tests.

Reviewed-by: ultrotter

f6d9f4c3 08/15/2008 11:44 am Michael Hanselmann

Move SSH option building into a function

I'm going to add another option and it would make maintaining
them in constants even more complicated.

Reviewed-by: ultrotter

54ab6aec 08/15/2008 11:44 am Michael Hanselmann

SshRunner.Run: Pass all arguments to BuildCmd

This patch changes SshRunner.Run to pass all arguments to
SshRunner.BuildCmd. They had the same arguments before
and should stay that way. This change makes it easier
to add new or change existing arguments.

4f0afaf5 08/14/2008 01:27 pm Guido Trotter

Pass hypervisor type to the OS scripts

It's handy to make the os scripts know which hypervisor the instance is
going to run under. In order not to change the os API we pass this
information in the environment, where the os scripts can access it if
they're hypervisor-aware....

2557ff82 08/14/2008 01:26 pm Guido Trotter

RunCmd: add optional environment overriding

If the user passes an env dict to RunCmd we'll override the environment
passed to the to-be-executed command with the values in the dict. This
allows us to pass arbitrary environment values to commands we run.

d47d3d38 08/13/2008 07:41 pm Guido Trotter

KVM Hypervisor Cleanup

- Remove a few experiemental code lines left as comments
- Rework first disks' boot=on addition, which was calculated twice
- Remove an empty line
- Remove reference to hvm_pae which doesn't apply to kvm

Reviewed-by: imsnah

eb58f9b1 08/13/2008 05:25 pm Guido Trotter

Add KVM hypervisor code contains the code for ganeti to work under kvm.
This patch also modifies to ship that file, and
lib/hypervisor/ to import it, and add kvm to the
hypervisors map.

Reviewed-by: imsnah

550e49b9 08/13/2008 05:25 pm Guido Trotter

constants: add HT_KVM

Add a new hypervisor type, HT_KVM, to constants, and register it in the

Reviewed-by: imsnah

7e2c5b9e 08/13/2008 05:24 pm Guido Trotter

Add --with-kvm-path configure option

This allows to configure a different path to the kvm binary. By default
/usr/bin/kvm is used, which is the one found in debian and ubuntu.

Reviewed-by: imsnah

a5f723a2 08/13/2008 05:24 pm Guido Trotter

FakeHypervisor: fix a function signature

StartInstance takes 'block_devices', not 'force' as its third argument.
Even if this is not used in the fake hypervisor it's better to have the
correct argument name to avoid confusion.

Reviewed-by: imsnah

e326d4e5 08/13/2008 05:23 pm Guido Trotter

Convert RunCmd to an epydoc docstring

Reviewed-by: imsnah

51144e33 08/13/2008 03:55 pm Michael Hanselmann

Fix adding pristine nodes

If a node hasn't been part of the cluster before being added it'll not
have the cluster's SSH key. This patch makes sure to accept those by
not aliasing the machine name to the cluster name.

Reviewed-by: ultrotter

f56377a3 08/12/2008 08:00 pm Michael Hanselmann

Fix race locking issue in noded

Noded didn't release the job queue lock after initialising it. This
patch makes sure to unlock once the work is done.

Reviewed-by: ultrotter

853e7f3d 08/11/2008 07:28 pm Michael Hanselmann

cli: Use new RPC call instead of polling

This means commands will not take at least one second anymore.

Reviewed-by: ultrotter

dfe57c22 08/11/2008 07:27 pm Michael Hanselmann

Add RPC call to wait for job changes

This way clients can react faster to status or message changes and
don't have to poll anymore.

Reviewed-by: ultrotter