Synnefo » snf-ganeti » ganeti-local

Unit tests for bdev and utils.lvm

Also amended a comment of a tested method.
The tests are for methods introduced in commit 63c73073 (LVM disk creation
uses dedicated PVs).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Return master ip script output in case of failure

Currently only the exit code is returned, which is somewhat pointless:
for failures, we already know it's ≠ 0, so we don't have enough
information (reported on the mailing list). By logging the output as
well (if any), the user will have more information available for...

Add the gnt-storage client

Add a new client called 'gnt-storage'.
The client interacts with the ExtStorage interface, similarly to
the way gnt-os interacts with the OS interface.

For now, only two commands are supported: 'info' and 'diagnose'.

'diagnose' calculates the node status of each provider on each node,...

Fix build breakage after exclusive storage commits

This fixes:

htest/Test/Ganeti/Rpc.hs:50:14:
Couldn't match expected type `Rpc.RpcCallNodeInfo'
against inferred type `Data.Map.Map String Bool
-> Rpc.RpcCallNodeInfo'...

Refactor checks for a new template in LUInstanceSetParams

All the checks have been moved to a private method, so as to make
CheckPrereq() smaller and more readable (pylint was complaining about it).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...

Free disk space reporting follows exclusive storage rules

When exclusive_storage is set, nodes report only the space that could be
used to create new disk according to the rules set in the
design-partitioned doc (i.e., non-empty disks are ignored and the space...

Add exclusive_storage parameter to node_info RPC

The flag is passed to the low-level functions that need it, but it's not
yet used. Also, RPCs get just a default value for now.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Improve assertion message in LUClusterVerifyGroup

In issue 240 this assertion fails. While reproducing was not possible
with the master branch, having a more verbose message can't hurt.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LVM disk creation uses dedicated PVs

When exclusive_storage is set, PVs are allocated according to the
design-partitioned design doc.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add exclusive_storage parameter to blockdev_create RPC

The flag is passed to the low-level functions that need it, but it's not
yet used.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

gnt-network: Fix CLI issues with node groups

The original implementation reserved the name “all” to specify all node
groups. That would prevent the use of a node group actually named “all”.

This patch changes the order and definition of the command line...

Improve network query algorithm

Similar to commit 572a277. Directly iterate over “self.wanted” instead
of iterating through all networks and checking if they're wanted.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Format network field "group_list" in client

The field is now a list of tuples containing the group name, mode and
link.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add function for generating UUIDs in the Haskell codebase

Its first use will be to generate the salt for ConfD requests of the Haskell
client, as in the Python client.

Unit test added as well.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add utility to format dictionary as key=value strings

This will be used in QA to format network interface parameters.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Adjust LUInstanceSetParams wrt the `ext' template

Initial support for the `ext' template in gnt-instance modify.
Take care of all needed checks in CheckArguments and CheckPrereqs
wrt ExtStorage Providers and ext-params.

ExtStorage disk addition and specification of arbitrary ext-params...

Stop all daemons on cluster initialization

Addresses issue 145. When daemons are still running from a previous
cluster instance, starting the daemons again will fail.
“ConfigWriter.IsCluster” only looks at the configuration file, which
could already have been deleted....

More style updates for network-related code

Error message formatting, else/if merging to elif, docstrings.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Detect wrong Python module structure

If, for one reason or another, a module named "ganeti.ganeti" can be
succesfully imported (e.g. through a symlink in the wrong place), error
which are diffcult to debug can occur. This piece of code tests for the
unlikely case of that happening....

Fix network LUs use of error code in OpPrereqError

Currently, most network LUs simply use EINVAL instead of correct error
codes; this patch tries to adjust the obvious cases.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

cmdlib: Factorize network conflict detection

LUNetworkConnect and LUNetworkDisconnect had very similar code to detect
conflicts between instance's network interfaces and networks. This code
factorizes the common part and does some cleanup:

- Remove single-letter variable for lambda (“l”)...

gnt-network add: "--network" is required

Also do some minor code re-formatting.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

query: Factorize code for getting statistics value

This was not only copied for the networking fields in commit 306bed0e,
but commit cfcea7ef fixed wrongly ordered parameters and didn't fix the
original. Either way, this patch merges the two cases again. The newly...

OpNetworkConnect: Check for network mode

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Replace frozenset with compat.UniqueFrozenset

This is not a trivial s/frozenset/compat.UniqueFrozenset/, but rather
only replaces “frozenset” where appropriate. Most of the places are
“static” information that doesn't change after the module has been
loaded....

Remove some unused Python code

This patch removes code which is no longer used due to refactoring:

- http.InitSsl, last usage removed in commit 33231500 (“Convert RPC
client to PycURL”)
- rapi.baserlib.MakeParamsDict, last usage remove in commit 4e5a68f8...

Allow shutting down offline instance

If an instance is offline we definitely shouldn't start it up.
But shutting it down, should it be up by mistake is not "that" bad.
Still, we only allow it with --force, as it still performs an action on
an instance we shouldn't touch. This should make everybody happy....

Fixes and improvements to comments

Some fixes, added more information in a few points, removed a stale (5+
year old) TODO comment.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Make LUNetworkAdd pass _VerifyLocks()

LEVEL_NODE_ALLOC should be aquired too if LEVEL_NODE is ALL_SET.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

hv_kvm: List of SPICE parameters should be module-global

This list is static at runtime and doesn't need to be recreated every
time.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUOobCommand: Always declare NAL in shared mode

Doing so avoids triggering an assertion in mcpu which cross-checks the
node lock and node allocation lock acquisition mode.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add optional formatting for OP_DSC_FIELD

For some opcodes, the output is not "stable", and depends on the exact
input values; this makes it harder to check consistency against
Haskell code.

To compensate for this, we add a way to override the formatting of the...

Very very very basic openvswitch support

This is a "better than nothing" support, just for kvm and just joining
the machine to the opevswitch bridge with the right command.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>...

Stop verifying opcode results in dry_run mode

Commit 1ce03fb1 (“Add ht-based result checks to opcodes”) introduced
infrastructure for checking opcode results, and subsequent commits
improved the list of opcodes which do declare a result, however this
was not tested for dry-run mode operation....

Set watcher pause on all nodes

Instead of just setting the watcher pause file, which isn't replicated,
RPC is used to set it on all nodes (where possible). This means that
after an outage of the master node and a following master-failover, the
watcher will still be paused....

jqueue: Improve inotify error reporting

This addresses issue 218. When the number of inotify watches is
exhausted, for example by being set too low from the beginning or by
other programs, waiting for a job to change would just report a lost job
(e.g. “Error checking job status: Job with id 7817 lost”)....

uidpool: Remove roman number support

Doing so simplifies to code a bit and never had a practical use.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Remove checks wrt IDISK_PARAMS from OpCode level

Change the "--disks" option validation, to just check the format
of the dict and do not check whether the keys are included in the
IDISK_PARAMS constant at OpCode level. This allows the passing of
arbitrary parameters at the CLI, which will then be logically...

Fix ordering of entries in tools.ensure_dirs

Commit ebd437a added two new entries to tools.ensure_dirs, but did so in
the wrong order. Patch forthcoming to improve the unittest's error
message.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

lib/tools/ensure_dirs.py: Code formatting

Wrap lines in a consistent manner (uid/gid on the second line) if
wrapping is necessary at all. “git diff --color-words” shows no
difference at all as only whitespace changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Add utility function to create frozenset with unique values

When used instead of a plain call to “frozenset”, this would have
avoided the issue fixed in commit e2dd6ec. The new function is located
in the “compat” module as it will be used at module load time in most...

Change value for ECODE_TEMP_NORES

Unfortunately there was a bug in commit 518a45e whereby ECODE_TEMP_NORES
had the same value as ECODE_NORES, leading to failures in a Haskell
test. Of course this would also have affected other users of the constant.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Improve error message for when adding inotify watcher fails

Explicitely mention the fs.inotify.max_user_watches sysctl value.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Export error codes from RAPI client module

Until now the error codes were not available from the RAPI client
module. A newly added unit test ensures all error codes are contained in
“ECODE_ALL”, as well as ensuring consistency between the RAPI client and...

cmdlib: Opportunistic locking on instance creation

Adds a new parameter to “OpInstanceCreate” and “OpInstanceMultiAlloc” to
use opportunistic locks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

mcpu: Verify node allocation lock mode

Add verification code to mcpu to check an LU's locks. Two whitelists are
provided to exclude LUs from the two tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

LUBackupExport: Use node allocation lock

Block instance allocations when all node locks will be acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUInstanceMultiAlloc: Use node allocation lock

Avoid conflicts between instance allocations.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUNodeSetParams: Use node allocation lock

Block instance allocations when all node locks will be acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUOobCommand: Use node allocation lock

If no node names are given, all node locks are acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>