Synnefo » snf-ganeti » ganeti-local

Add tests for LVs parser

Add unit tests and Quickcheck tests for the LV parser.

Also, fix the alphabetic order of some imports in htest.hs.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Add function for generating arbitrary UUID

Add a function for generating arbitrary UUIDs to be used for QuickCheck tests.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

iallocator: storage processing accepts space_info structure

This is a small refactoring to make the processing of
the storage space information not accept a node info
result, but the more specific space info structure instead.

Signed-off-by: Helga Velroyen <helgav@google.com>...

iallocator: computation of storage data

This patch extracts the processing of the storage-related
data from the RPC call 'node_info' in a separate function
to increase testability. In addition, this patch contains
unit tests for this function.

Signed-off-by: Helga Velroyen <helgav@google.com>...

Index instances by their UUID

No longer index instances by their name but by their UUID in the cluster
config. This change changes large parts of the code, as the following
adjustments were necessary: * Change the index key to UUID in the configuration and the...

Version bump to 2.9.0~alpha1

Bump the version number to 2.9.0~alpha1.

This also requires updating version numbers in cfgupgrade tool.

Furthermore, a data file for test representing a 2.8 configuration had the
version number set to 2.7 in order to prevent tests failing while the current...

Rename directory 'Block' to 'Storage'

This patch renames the 'Block' directory to 'Storage' in
the Haskell code base. The same rename was done in the
python code base earlier this quarter. We generalize the
name, because we needed a place for general storage...

Add tests for check-news

Unit tests verifying the behaviour of the check-news script are added.

This required adding a new make target, "autotools-check" directly called from
"commit-check", because the usual tests are executed in the build directory,
whereas these, being relative to the autotools, cannot be executed there because...

Utility functions for storage types

Handling various storage types for the free space reporting
requires some utility functions. They will be invoked in
later patches. Unit tests are included.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Renamining space reporting for file storage

Before being integrated, the function for space reporting
for file storage is renamed to a more expressive name.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Add a test for hspace to respect instance policy

In this example, the instance policy for disks prevents allocation
of more than one instance per node. The test verifies that tiered
allocation still respects this and doesn't attempt to place policy
violating instances on the cluster....

Convert node indices to UUIDs in configuration

cfgupgrade replaces the occurences of node names (where they are used as
index) with the node UUID.
It also supports the inverse replacement, so downgrades are supported
too.

Tests for cfgupgrade are adapted as well....

Remove XEN_CMD from constants, adjust to PowercycleNode

This patch extend the RPC to powercycle a node by a
hvparams parameter. This parameter is used by the
xen hypervisor to determine which xen command to use.
The respective LU is adjusted as well.

Since this was the last operation depending on the...

Add hvparams to RPC call 'node_info'

This patch adds the hvparams parameter to the RPC call
'node_info'. It also adjusts the related code in noded.py
and Query/Node.hs

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

hypervisors: add hvparams to GetNodeInfo

This patch extends the GetNodeInfo function of the
hypervisors by a hvparams parameter. The parameter
is currently only used by the xen hypervisor to determine
which xen toolstack ('xm' or 'xl') to use.

Signed-off-by: Helga Velroyen <helgav@google.com>...

Include hvparams in ssconf files

The watcher needs access to the cluster's hypervisor
parameters. This patch extends the ssconf infrastructure
by adding a hvparams file for each hypervisor. The format
of the hvparams file is a simple key=value format.

Signed-off-by: Helga Velroyen <helgav@google.com>...

Verify: xen toolstack, hypervisor and hvparams

This patch extends the node verification by:
- Adding a check for the xen toolstack when the hypervisor
is verified.
- Factoring out the hypervisor verification in a
subfunction to increase testability.
- Factoring out the hvparams verification in a...

hv_xen.py: _RunXen and GetInstanceList use hvparams

Propagating the use of hvparams further up in the calling
hierarchy, the functions '_RunXen' and 'GetInstanceList'
use the given hvparams. Unit tests provided. Calling function
'StopInstance' adjusted in this patch as well....

hv_xen._GetCommand: retrieve xen command from hvparams

This patch adds a (so far optional) hvparams parameter
to the '_GetCommand' function. This dictionary is used
to retrieve the xen command (xm or xl). It is optional
for now to make the refactoring possible without breaking...

Fixing docstring of hv_xen_unittest.py

The introductory comment of hv_xen_unittest.py referred to
the wrong hypervisor.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Spindles become part of htools resource spec

Spindles are now part of resource spec. Instances get created with spindles
specified (which are just ignored when exclusive storage is disabled).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Update spindles when moving instances in htools

Spindles get updated, and errors raised when not enough free spindles
exist. No new error is raised when exclusive storage is disabled.

Unit tests included.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...

Load node spindles data in htools

The data structure for nodes gets a new field for free spindles, and the
existing field for total spindles gets renamed to avoid identifying it with
the node parameter that had the same name. These fields get filled with...

Add a force option to the ClusterSetParams Opcode

If set, the op code will, in particular, try to set the master IP
on the new netdev, even if shutting down the master IP on the old
netdev failed.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Add shelltests for diskstats

Check that the behavior of running the diskstats data collector is as
expected.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add function for generating arbitrary non-negative numbers

In many tests it is useful to have the possibility to easily generate
non-negative integer numbers.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix erroneous docstring

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Check real spindles in ipolicies

When exclusive storage is enabled, the spindles in instance disks are used
to check the instance policies (as outlined in design-partitioned.rst).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Load exclusive_storage in htools

The node parameter is loaded into the data structures. No behavior is yet
modified.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add unit test for text backend + fix bug

Test serialization and deserialization of instances. Fix check of secondary
node.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Merge branch 'stable-2.8' into master

• stable-2.8:
  Document users-setup tool
  Adjusting permissions after confd start
  Ensure the queue socket has the right permissions
  Update IAllocator interface documentation
  Add NEWS entry for hail honoring networks...

Parse NIC data from allocation request in hail

Add a NIC type and extend the Instance type by a list of NIC's. Parse
the NIC's in allocation requests and store them for now. Later patches
will make use of this field in order to ensure that the requested
instance is only placed in node groups wich are connected to those...

Parse node group networks

Extend the Group by the network ids it is connected to. Adapt
the IAlloc backend such that the networks are parsed correctly.
This also required the adaption of test data.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add disks_active to configuration

This flag tracks if the disks of an instace are supposed to be active.
That's the case when an instance is running or when its disks got
activated explicitly (and in a couple of other cases).
It will be used by watcher to re-activate disks after a node reboot....

Refactor code for attaching a logical volume

The parsing of "lvs" output is moved into private methods. The code is
slightly more readable and testable. The split in two methods is useful
for the following patches. Unit tests for the new functions are
provided....

Add spindles field to disk object

The field is filled with the value provided on the command line.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Extend hroller tests by options for non-redundant instances

The cluster now consists of 3 nodes, with drbd instances between
nodes 1 and 2, and 2 and 3. Additionally, nodes 1 and 3 each contain
a non-redundant instance, but node 2 cannot hold two additional...

Stash Xen config file after a failed startup

This is a fix for a minor bug.
Currently, a failed Xen VM start results in a stale config file left
behind on the filesystem.
This change introduces a new log directory, where the Xen VM config
file is moved after a failed startup....

cmdlib: Cleanup public/private functions

All functions/classes which are used outside of their defining module
(with tests as an exception) no longer have a leading underscore.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Extract query related logical units from cmdlib

All LUQuery* classes are extracted to query.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Extract node related logical units from cmdlib

All LUNode* classes are extracted to node.py. Common functions are moved
to common.py if used by non-node logical units as well.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Extract cluster related logical units from cmdlib

All LUCluster* classes are extracted to cluster.py. Shared functions are
extracted to common.py, helper functions only used by LUCluster* are
extracted to cluster.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>...

Instance generators

This patch introduces and enhances generators for
instances:
- 'genInstWithNets' is split into the generation of an
arbitrary instance and enhancing an instance with nets
- 'genInst' calls 'genInstWithNets' with an empty set
of initial networks to provide a reasonable default...

Generators for disks

This patch adds generators for Disk instances to the Haskell
test code. It uses somewhat more reasonable generators to
fill the fields instead of just arbitrary values.
'genDiskWithChildren' is a generator that generates a disk
with a specified number of disk children. To avoid shooting...

Backend function for file storage space reporting

This adds functionality to retrieve disk space information
for file storage. It calls the 'df' tool and parses its
output to extract the total and free amount of disk space
on the disk where the given path is located....

Rename dir 'block' to 'storage'

Renaming the 'block' directory to 'storage', because I plan to
place code there that is related to file storage and leaving
it named 'block' would be misleading.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Extend hroller test to also verify tag-based node selection

While the multiple-tags test was added to verify that coloring is done
only after node selection (otherwise it wouldn't be possible to get in
both cases a single reboot group), it can easily be extended to also...

Fix expectation in hroller test

Regular expressions are not shell globs. So "any symbol" is expressed
by a dot, not a question mark. In this case, the confusion lead to a
too liberal expectation, hence the test passed. Fix it nevertheless.

Signed-off-by: Klaus Aehlig <aehlig@google.com>...

Add tests for the -O option of hroller

In hroller, the option -O can be used to mark certain nodes as offline.
These nodes should then not be part of any reboot group. Add tests
to verify this behavior.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Rename DRBD8 to DRBD8Dev

Right now the DRBD8 class has multiple responsibilities: a) it
reprensents a device which can be set up, grown, etc. and b) it
represents the whole DRBD system on a node which has a usermode helper,
which knows how to shut down all devices, etc. Therefore, the DRBD8Dev...