Synnefo » snf-ganeti » ganeti-local

cmdlib: Remove acquired_locks attribute from LUs

The “acquired_locks” attribute in LUs is used to keep a list of acquired
locks at each lock level. This information is already known in the lock
manager, which also happens to be the authoritative source. Removing the...

Increase the lock timeouts before we block-acquire

This has been observed to cause problems on real clusters via the
following mechanism:

- a long job (e.g. a replace-disks) is keeping an exclusive lock on an
instance
- the watcher starts and submits its query instances opcode which...

Split BuildHooksEnv of LUs

Commit dd7f677623 added another call to BuildHooksEnv to provide
post-phase status variables. Since BuildHooksEnv also built the node
lists, that meant they have to be built twice. First a rather strict
check was used, but it turned out to be more tricky. Commit b423c51336...

HooksMaster: Add more assertions for variable names

Also replace explicit loop with dict.update.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

mcpu: Tidy HooksMaster a bit

- Dictionary indentation
- Add empty lines for readability
- Simplify conditional code

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

mcpu: Automatically build the DISPATCH_TABLE

While reviewing dato's interdiff for the OpAssignGroupNodes, I
realised that we can do better. This patch replaces the hand-built
DISPATCH_TABLE with one built from the opcode.OP_MAPPING dict.

Signed-off-by: Iustin Pop <iustin@google.com>...

Add modification of node groups (OpCode/LU/CLI)

With this commit, only modification of the "ndparams" attribute is
supported.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Group operations: OpCode and LU for removing a group

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Adding new OpCode for OOB

Register OpCode and Logical Unit in mcpu.py

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add OpQuery opcode

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

mcpu: Raise directly in _AcquireLocks

Removes code duplication.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

mcpu: Adjust lock acquire strategy

The changes to job queue processing require some changes on this class'
interface. LockAttemptTimeoutStrategy might move to another place, but that'll
be done in a later patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Remove mcpu's ReportLocks callback

This is no longer needed with the new lock monitor. One callback is kept to
check for cancelled jobs.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Provide feedback function for all LU methods

By exposing mcpu's _Feedback function (now renamed to “Log”) to LU's,
methods like ExpandNames can also write to the job execution log.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add opcode to prepare export

To prepare a remote export, the X509 key and certificate need to be generated.
A handshake value is also returned for an easier check whether both clusters
share the same cluster domain secret.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

mcpu: Log lock status with sorted names

Reading and comparing sorted lists is easier when debugging locking problems.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Remove many 'Unused variable' warnings

Note there are some cases left which need extra cleanup.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Add config.DropECReservations

For now this function does nothing, but it gets called by mcpu when the
execution of an LU is done, making sure any pending reservations are
dropped.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

mcpu: Use new timeout class for timeout

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

mcpu: Make sure added locks are released on errors

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Code and docstring style fixes

Found using pylint and epydoc.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

mcpu: Implement lock timeouts

The timeout is always between ~0.1 and ~10.0 seconds. A small
variation of ±5% is added to prevent different jobs from
fighting each other. After 10 attempts to acquire the locks with
a timeout, a blocking acquire is made.

Lock status reporting will be improved in a separate patch....

Remove RpcResult.RemoteFailMsg completely

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Move OpCode processor callbacks into separate class

There are two major arguments for this:
- There will be more callbacks (e.g. for lock debugging) and extending the
parameter list is a lot of work.
- In the jqueue module this allows us to keep per-job or per-opcode variables in...

HooksMaster: fix RunPhase logging

In case of complete failure results is empty, return immediately
(tnx unittests).

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

node-remove post on removed node

Run post phase of node-remove on the removed node as well.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

HooksMaster: list of nodes override

Allow the caller of HooksMaster.RunPhase() to specify an alternative list of
nodes.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Remove extra argument from HooksMaster class

The mcpu.py:HooksMaster class needs to have a proc attribute/argument to
init in ordet to call its LogWarning method. However, this is available
from the 'lu' attribute, so we can remove this dependency.

Signed-off-by: Iustin Pop <iustin@google.com>...

Implement instance recreate-disks

This can be used for a 'plain' type instance when the underlying storage
went away, to recreate the storage (and reinstall) instead of removing
the instance and readding it.

Signed-off-by: Iustin Pop <iustin@google.com>...

Post cluster initialization LU

Add an 'empty' logical unit to run hooks after cluster initialization.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Implement gnt-cluster check-disk-sizes

This patch adds a new opcode and lu for checking disk sizes. Currently
it does only top-level disk verification, and also doesn't check
primary/secondary node size mismatches (these two are added as TODOs in
the Exec() function of the LU)....

Add new opcode to list physical volumes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add new opcode to evacuate nodes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix pylint warnings

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LU execution: implement dry-run framework

This patch adds a new (global) opcode flag 'dry_run' which, when True,
causes early exit from the LU workflow, returning a special value from
the LU object (initialized in the parent LogicalUnit class, and which if...

Convert hooks_runner rpc to new style result

This also converts (and fixes) unittests and mock objects to deal with
this change, and the custom hook verifier in cmdlib.LUClusterVerify.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

An attempt at fixing some encoding issues

This patch unifies the hardcoded re-encoding attempts into a single
function in utils.py. This function is used to take either an unicode or
str object and convert it to a ASCII-only str object which can be safely...

Introduce a very simple LU to force config updates

This LU can be used to force a push of the config in case it's needed,
for example after an upgrade to update the ssconf_release_version file.

Reviewed-by: imsnah

Convert rpc results to a custom type

For a long time we had the problem that both RPC-layer errors and
results from the remote node share the same "valuespace". This is
because we shouldn't raise an exception when only one node failed
(and lose the results from the other nodes)....

Documentation updates for mcpu.py

This is the only change needed to make mcpu epydoc-compliant.

Reviewed-by: ultrotter

Convert mcpu.py to use the logging module

Reviewed-by: imsnah

Implement job 'waiting' status

Background: when we have multiple jobs in the queue (more than just a
few), many of the jobs (up to the number of threads) will be in state
'running', although many of them could be actually blocked, waiting for
some locks. This is not good, as one cannot easily see what is...

Convert mcpu.py

Replacing ssconf with configuration.

Reviewed-by: iustinp

Implement adding/removal of locks by declaration

With this patch LUs can declare locks to be added when they start and/or
removed after they finish. For now locks can only be added in the
acquired state, and removed if owned, and added locks default to be...

Processor: remove ChainOpCode

This function was incompatible with the new locking system, and its
usage has been removed from the code. For now LUs share code by calling
common module-private functions in cmdlib.py, in the future they will
use tasklets (when those will be implemented)....

Processor: lock all levels even if one is missing

If a locking level wasn't specified locking used to stop. This means
that if one, for example, didn't specify anything at the LEVEL_INSTANCE
level, no locks at the LEVEL_NODE level were acquired either. With this...

Fix pylint-detected issues

This is mostly:
- whitespace fix (space at EOL in some files, not all, broken
indentation, etc)
- variable names overriding others (one is a real bug in there)
- too-long-lines
- cleanup of most unused imports (not all)...

Add LogicalUnit.DeclareLocks

This additional LogicalUnit function is optional to implement, but lets
you change your locking needs for one level just before locking it, but
after the previous levels have been already locked. It is useful for
example to calculate what nodes to lock after locking an instance....

Invert nodes/instances locking order

An implementation mistake from the original design caused nodes to be
locked before instances, rather than after. This patch inverts the level
numbering, changing also the relevant unittests and the recursive
locking function starting point....

Processor: Acquire locks before executing an LU

If we're running in a "new style" LU we may need some locks, as required
by the ExpandNames function, to be able to run. We'll walk up the lock
levels present in the needed_locks dictionary and acquire them, then run...