Synnefo » snf-ganeti

Display node names instead of node UUIDs

Display resolved node names instead of node UUIDs in a couple of
warnings and log messages.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

Replace physical_id with dynamic_params

The disk field 'physical_id' has to be kept up to date whenever a disk
object is sent to a node via RPC. This is done with the SetDiskID method
manually, which is a source of bugs.

This patch replaces the use of 'physical_id' with a new field names...

Replace LD_* constants with DT_* constants

LD_* constants are basically like DT_* constants, except
for that both file and shared file were mapped to file.
In order to not having to maintain three slightly different
sets of disk-related constants (DT, LD and ST), we merge...

Extend RPC call 'node_info' by storage parameters

This patch extends the actual RPC call to accept storage
parameters with each storage units. It adjusts all code
which performs this code by integrating the 'exclusive
storage' flag into the lvm storage unit....

iallocator: storage processing accepts space_info structure

This is a small refactoring to make the processing of
the storage space information not accept a node info
result, but the more specific space info structure instead.

Signed-off-by: Helga Velroyen <helgav@google.com>...

iallocator: factor out instance memory summation

The iallocator adjusts the free memory by a calculation
of how much memory is actually currently used by instances
on the node. This calculation is rather complicated and
deserves its own function to increase testability. No...

iallocator: computation of storage data

This patch extracts the processing of the storage-related
data from the RPC call 'node_info' in a separate function
to increase testability. In addition, this patch contains
unit tests for this function.

Signed-off-by: Helga Velroyen <helgav@google.com>...

iallocator: prepare RPC call 'node_info'

This patch does a small refactoring in the allocator. It extracts
the preparation and actual call of the 'node_info' call into a
separate function. This is done for better testability. No functional
changes otherwise....

Refactor NodeInfo RPC regarding storage reporting

The NodeInfo RPC call is refactored to handle now more than
just storage reporting for volume groups.

Since NodeInfo now returns storage space information not
necessarily for volume groups, but also for other storage...

Index nodes by their UUID

No longer index nodes by their name but by their UUID in the cluster
config. This change changes large parts of the code, as the following
adjustments were necessary: * Change the index key to UUID in the configuration and the
ConfigWriter, including all methods....

Adjust all callers of RPC 'call_node_info'

This patch contains all adjustments necessary to code that
uses the 'call_node_info' call. The adjustment is necessary
because the call's signature changed from receiving a list
of hypervisor names to a list of tuples...

Export instance disk spindles

Instance disk spindles can be queried via LUXI and RAPI, and are exported
through the allocator interface. This is a prerequisite for htools to
handle spindles.

The length of a RAPI query including all the instance fields now has become...

Export connected networks to IAllocator

IAllocators should not put instances on nodes in nodegroups which are
not connected to the network the instance should be connected to.
Therefore, export the networks a node group is connected to to the
IAllocator, so it can account for this restriction....

RPC 'node_info': <storage_type,key> instead of vg_names

This replaces the field 'vg_names' in the RPC call of 'node info' by
'storage_units'. A storage unit is a tuple <storage_type,key>
and a generalization of a vg_name. The list of vg names is replaced by...

Allow iallocator to work without LVM storage

Currently, the iallocator interface requires LVM storage, due to the
way it computes the node storage information.

By changing the code to understand that GetVGName() can return None,
and by setting the disk_total/disk_free node parameters to the value...

Call node_info RPCs with the exclusive_storage flag

The flag is read from the configuration and passed to the RPC.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix iallocator for nodegroups with no tags

_ComputeNodeGroupData() raises error if a group has no tags
when list(gdata.tags) is invoked. Use “gdata.GetTags()” instead.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>...

iallocator: Add node whitelist

In the future instance creations might have a lock on all nodes as was
the case until the implementation of opportunistic locking. Nodes for
which the lock is not held will be shown to the iallocator plugin as if
they were marked offline....

Cleanup ht's use of positive/strictpositive

Currently, ht.py uses a bad terminology for positive/non-negative
numbers. Per http://en.wikipedia.org/wiki/Positive_number, this is the
correct terminology:

- A number is positive if it is greater than zero.
- A number is negative if it is less than zero....

Verify the node count just if the allocation was successful

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fix a validation issue

We want to validate the result only if the iallocation run was a
success.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Small style adjustment

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Rework the required_nodes field

This was used to verify the result of instance allocation. However, this
is not logic which belongs to the cmdlib rather than to the iallocator
request itself. So abstract it by its own function. And update the
ValidateResult accordingly....

Adding new multi-allocation request

Now that we've the refactoring done, we can easily add the new mode and
everything works.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move _CalculateGroupIPolicy and _ComputeDiskSize

This makes it possible to better untangle the IAllocator code, which
would otherwise lead to a recursive import structure as we need those
helpers there as well.

Signed-off-by: René Nussbaumer <rn@google.com>...

Add basic support for disk parameters

objects.py: * add disk parameters to Disk, Cluster, NodeGroup.

constants.py: * add dictionaries that will hold types and default values for disk
parameters (for now, empty).

test/ganeti.constants_unittest.py:...

Move _TimeoutExpired to utils

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Instance transfer: add argument for the 'component'

Currently, transfer data is done mainly with just the instance name,
but when we have instances with multiple disks this is not enough to
distinguish between the different transfers being done for the
instance....

Import-export: fix logging of daemon output

In case of failures, the recent daemon output is logged as %r on a
list of unicode strings, which results in the (ugly):

Thu Feb 3 05:13:34 2011 snapshot/0 failed to send data: Exited with status 1 (recent output: [u' DUMP: Date of this level 0 dump: Thu Feb 3 05:13:18 2011', u' DUMP: Dumping /dev/mapper/6369a5f7-1e67-4d0d-a4f0-956b3649c6d7.disk0_data.snap-1 (an unlisted file system) to standard output', u' DUMP: Label: none', u' DUMP: Writing 10 Kilobyte records', u' DUMP: mapping (Pass I) [regular files]', u' DUMP: mapping (Pass II) [directories]', u' DUMP: estimated 54301 blocks.', u' DUMP: Volume 1 started with block 1 at: Thu Feb 3 05:13:19 2011', u' DUMP: dumping (Pass III) [directories]', u' DUMP: dumping (Pass IV) [regular files]', u'socat: E SSL_write(): Connection reset by peer', u"dd: dd: writing `standard output': Broken pipe", u' DUMP: Broken pipe', u' DUMP: The ENTIRE dump is aborted.'])...

Improve import/export timeout settings

With this patch, the exporting node will retry to connect a few times.
The receiving node will make use of the master's increased timeout (see
previous patch).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Use IPv6 for import/export

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

masterd.instance: Add missing argument

_DiskTransferPrivate takes three parameters, not two.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

Remove pred from compat.any/all

This makes it compatible with the python builtin, and we can even use
the builtin when running under the right version of python. The all and
any functions are renamed to _all and _any, so that they can be tested,
and (non)existing unittests are updated (translation: there are no unit...

Disable compression for all intra-cluster imports/exports

Tests have shown that usually we're CPU-bound for intra-cluster
imports/exports. Disabling compression will help with this.

Some versions of OpenSSL, depending on the build options, also
compress transparently. This will need further work in Ganeti....

import/export: Validate remote host/port

The hostname and port received from the remote cluster should
be validated, just in case.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

import/export: Show progress updates to user

With this patch, we show progress updates approx. once per minute.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Put common import/export daemon options into object

The X509 key name and CA are passed from cmdlib all the way to
the backend import/export daemon. With the addition of an option
to choose the compression method, another parameter would have
to be passed all the way. By moving these options to a separate...

Implement opcode changes for remote-export

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LUExportInstance: Move exporting code to helper class

This will simplify the implementation of intra-cluster instance
exports and reduces the number of local variables in
LUExportInstance.Exec.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

cmdlib: Add utility for instance data import/export

Interpreting the backend's import/export daemon status is a bit tricky.
This utility code keeps track of multiple transfers at the same time.
Users can supply callback functions to react to events.

Timeouts are currently hardcoded. Intra-cluster instance moves will likely...