Synnefo » snf-ganeti » ganeti-local

hv_kvm: Refactor getting TAP features, add tests

Split retrieving supported features into a dedicated function which can
be mocked. Tests are added for both “_ProbeTapVnetHdr” and
“_GetTunFeatures”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

hv_kvm: Original error message, keyword parameter

- Include original error message when creating TAP interface failed
- Pass keyword parameter as such

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

CLI.hs: fix double spaces in option help strings

Some help strings with continuation backslashes ('\') were providing a
space both before and after the backslash, resulting in double spaces in
help output. Provide it only after the backslash, which fixes the issue and...

Moving network query helper functions to Network.hs

So far, a couple of helper function for the network
queries resided in Config.hs. I figured it makes more
sense to move them to Query/Network.hs, since they are
very tailored to that purpose.

Signed-off-by: Helga Velroyen <helgav@google.com>...

More fields for network queries

This adds more fields to the network queries:
- group_cnt, free_count, reserved_count, and map

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Merge branch 'devel-2.7'

• devel-2.7:
  kvm: fix bug while fetching -device list

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

kvm: fix bug while fetching -device list

_GetKVMOutput expects the command to succeed, but unfortunately on some
versions of kvm "-device ?" will output a correct list of devices, while
exiting with an error code.

To fix this we accept failure in that case (note that this doesn't...

hv_xen: Test stopping an instance

Test stopping an instance, and ensure the configuration file still
exists after an instance couldn't be stopped.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

hv_xen: Test reading non-existent config file

Check whether doing so raises an exception.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

hv_xen: Test removing config from "auto" directory

Test whether any existing configuration file named after the instance is
removed from Xen's “auto” directory.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

hv_xen: Test errors while writing config file

Test error handling while writing Xen configuration files.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

hv_xen: Split StopInstance

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Merge branch 'devel-2.7'

• devel-2.7:
  hv_xen: Remove config after shutdown was successful

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

First part of Network Queries in Haskell

This is the beginning of the implementation of network
queries. This includes establishing all infrastructure
to run the network queries and implement querying of
some simpler fields and the node group listing.

Signed-off-by: Helga Velroyen <helgav@google.com>...

Extend config by networks and networks by UUIDs

For network queries to work, we need to extend the general
config type to include all available networks. Additionally,
we add UUIDs to the network type itself.

Signed-off-by: Helga Velroyen <helgav@google.com>...

hv_xen: Remove config after shutdown was successful

If stopping an instance failed, the configuration would already be gone
and other operations depending on it (e.g. migration) would no longer
work. With this patch the configuration file is only removed once the...

hv_xen: Prepare for unit tests, remove {static,class}method

Unit tests will have to specify a custom Xen configuration directory (a
temporary directory). To this end “hv_xen.XenHypervisor” and its two
derived classes are modified to have fewer static and class methods. A...

hv_xen: Refactor getting node information, add tests

Refactor and add tests for getting node (Domain-0) information.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Merge branch 'devel-2.7'

• devel-2.7:
  hv_*: Always return from Verify, style fixes
  _VerifyErrors()._Error() and _ErrorIf() are now consistent
  Handle the result of QueryGroups() correctly

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Check minimum size of networks on creation

When creating a network, so far no size constraints were checked.
We now limit the size of a network to a /30 or bigger, although
tecnically, the ipaddr library supports even /32 networks.

Signed-off-by: Helga Velroyen <helgav@google.com>...

Network QA

This adds a script for the QA of 'gnt-network'. So far it
covers adding/removing and connecting/disconnecting networks.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

_VerifyErrors()._Error() and _ErrorIf() are now consistent

_Error() didn't contain the logic for demoting errors to warnings and for
marking an operation as failed. Now _ErrorIf() is just a minimal wrapper
for _Error().

Unit tests included.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...

Handle the result of QueryGroups() correctly

If no group is given for the “gnt-network connect“/“… disconnect”
commands, the client uses the result of “QueryGroups()” which is a list
of lists. Use “itertools.chain()” to handle the return value correctly....

hv_xen: Compose file name outside error handling

In _ReadConfigFile, the filename should be prepared outside the
try/except block. Fixes bad code formatting, too.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

hv_base: Remove empty constructor

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Fix format string of KVM output

This fixes a missing 's' in the format string and
the wrong quotes. Those bugs were introduced in
commit 6e043e60.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add a function to get the current time in microseconds

In some cases we need higher resolution that seconds; I've settled on
microseconds as that is what 'threadDelay' wants, for exactly, so it's
easier if we keep the same units.

Signed-off-by: Iustin Pop <iustin@google.com>...

First part of confd timer changes

This patch changes the resolution of the timers: the watcher timer
goes from 60s to 17s, and the polling-mode timer goes from 2 seconds
to 250ms. The code changes a bit more due to the changes in the units
of the various constants....

Start confd in notify mode if we can

This patch changes the default model—where confd is always started in
polling mode—to a model where if possible we enable inotify mode
before starting any of the background threads.

There are some tricky details here: if we enable inotify, we should...

Fix type of 'node_whitelist' request parameter

If opportunistic_locking is used, then 'node_whitelist' parameter passed
to the allocator is set to the LU's owned node locks. However, LU owned_locks
has type of 'set' while IReqInstanceAlloc expects type of...

kvm: deduplicate 'get output' code

We had the same code twice, and were about to add a third time. Better
to collapse it into just one function.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

kvm: remove last version-based feature detection

This was left behind because it required a different kvm invocation.
Now that we can add new ones cheaply (two constants) it's easy to get
rid of it. Differently than in other cases we support old version which...

Makefile.am: Protect placeholders for sed

If the placeholder has exactly the same name as the variable used in
autoconf it would be replaced as well. In the case of
“ENABLE_SPLIT_QUERY” this led to the sed command “s#False#False#g”.
Inserting additional quotes after the first @ sign prevents this....

Text wrap in Haskell code

This adds two functions in Utils.hs: 'wrap' and 'trim' along with some
simple tests. 'wrap' is used in Common.hs for wrapping command
descriptions to a fixed line length, e.g. useful for the "--help"
option.

Signed-off-by: Yiannis Tsiouris <gtsiour@softlab.ntua.gr>...

Makefile: Allow tarball creation when man-in-doc is active

Documentation with included man pages should not be distributed as it
contains fixed paths. Since the documentation with man pages is actually
written to a separate directory, tarball creation is still possible....

Revert "run-in-tempdir: Improve speed"

This reverts commit dd60179dfd8 as it broke coverage for Python.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

cluster-verify checks uniformity of PV sizes

When exclusive_storage is set, cluster-verify complains if the size of PVs
is not uniform across a node group.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

add-node checks PVs

add-node now performs a few checks on LVM before adding a node to the
cluster.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

bdev.GetPVInfo() returns list of LVs

This will be used for checks related to exclusive storage.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Small cleanup in constants.py

- compat.UniqueFrozenset used in one more place, for consistency.
- The order of NV_XXX constants is fixed so that all values are now sorted
by name.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

utils.LvmExclusiveCheckNodePvs() returns more info

utils.LvmExclusiveCheckNodePvs() now returns the size of the smallest and
biggest PVs of a node. They will be used for cross-node consistency checks.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...

run-in-tempdir: Copy doc/* only if needed

To preserve disk space on copying all files to a temporary directory,
the “doc” directory is only copied when building documentation. A
comment describing the need is also added to Makefile.am, as well as a
superfluous direct invocation of Bash removed....

Add reporting infrastructure for data collectors

This commit adds the part of the JSON report generation code that will be
common to all the data collectors, according to the format specified in the
design document.

Signed-off-by: Michele Tartara <mtartara@google.com>...

burnin: Add unittest for supported disk templates

When new a new disk template is added, burnin should be updated.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

make _{Generate,Execute}KVMRuntime accept kvm help

Currently kvm --help is called twice at instance startup, this makes
sure it's called only once.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add KVM machine version as hv param

This is needed in case an instance must be booted with an exact type of
machine version (due to e.g. outdated drivers). If this hv param
explicitly defined (during instance creation/modification) then the kvm
process will be started having this version for the -M option. Otherwise...

Remove special-case of "0" in niceSort

Since now niceSort is more consistently behaved, we don't have the
special casing. This should have been removed in commit a7f0953a, but
I didn't realise it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

test/Types.hs: add an HLint ANN for ignoring camelCase

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Make niceSort a wrapper over niceSortKey id

Currently, niceSort behaves differently than niceSortKey in cases
where the parsed groups are equal but the original not, as it uses
both members of the tuple as opposed to just the first one. Both
niceSortKey and the Python code use only the first one, so in corner...

Option to include man pages in documentation

Before this patch, HTML versions of man pages (man/*.rst) were already
built. However, since they are separate from the normal documentation,
their content is not indexed for Sphinx' search functionality.
Additionally it would simply be nice to have everything in one place....