Synnefo » snf-ganeti

Improve handling of storage info

This patch improves the processing of storage information
as result of the 'node_info' call in the following way:
- It removes the override for LVM storage. This was needed
in MakeLegacyNode info to be compatible to the...

Add default disk template selection to man pages

This documents the newly implemented behavior of picking
the disk template from the list of enabled disk templates
on instance creation.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Let instance creation fall back to default disk template

So far, the '-t' option of 'gnt-instance add' is mandatory.
This patch makes it optional. In case, no disk template is
specified, the first of the list of enabled_disk_templates
is used.

Signed-off-by: Helga Velroyen <helgav@google.com>...

Document the 'protocol' disk parameter

Document the new disk parameter 'protocol' which is used to set the DRBD
replication protocol.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Revert "QA: Make Query tests test for '?' values"

This reverts commit
bfbef99b9f056638411aa894e300aa095132e931. The problem is that
in some cases it makes sense for '?' to be displayed. This
is the case for storage reporting on clusters that have only
the 'diskless' disk template enabled. It would be nicer,...

Check DRBD status on verify-disks

Enhance `gnt-cluster verify-disks` such that it checks the DRBD status
of all disks. If the status of at least one disk of an instance is
either StandAlone or Diskless, the instance disks are activated
automatically.

Signed-off-by: Thomas Thrainer <thomasth@google.com>...

QA: Make Query tests test for '?' values

When something goes wrong in the Haskell implementation of
queries, the 'gnt-* list' command does not actually fail,
but it just displays '?' as a sign of failure. We so far
do not test for that, which makes errors in the haskell...

noded.py: remove fallback of 'node_info'

In order to change backend and frontend code of the RPC
call 'node_info' in separate patches, we included a
fallback to the old implementation. Since all RPC callers
are adjusted, this can be removed now.

Signed-off-by: Helga Velroyen <helgav@google.com>...

Add general storage parameters to node info call

As described in the design doc about storage types,
we plan to generalize the RPC call "node info" wrt to
storage types. This patch extends the call by accepting
a list of storage units, that is not only identified...

Prevent LV parser compile error

The LV parser is not compiled correctly by more recent versions of GHC
because of more strict checks.

lvCommand is a surely non-empty list, but the compiler still refuses it
asking for explicitly management of the empty list case....

Load CPUs used by the node OS in htools

A new field is added to the Node type, and it's used to initialize the used
CPUs field.

The signature of Node.create has been split among lines to match the
parameter list.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...

Hypervisors report CPUs used by the node OS

The XenHypervisor's "dom0_cpus" parameter (introduced in 1d60fec6 but never
used) is renamed to "cpu_dom0" for consistency with other parameter
names. The analogous information is returned for the other hypervisors....

Add LV collector to the monitoring daemon

Allow the monitoring daemon to use the LV data collector.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Add LV data collector

This commit adds the LV data collector.

Also, the lvCommand function was not providing the correct value as expected by
the readProcess function, so it was fixed.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Add function testing parsers with QuickCheck

Parsing a string automatically generated by QuickCheck can be useful in many
cases. A function for doing this is added to the common test functions.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Add LV parser

Add the parser for getting the information about the logical volumes in the
system.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Minor fixes to monitoring design doc

Add a missing full stop, and fix a sentence about the storage collectors.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Add test for the --full-evacuation option

In this example cluster, there are four pairs of nodes, each hosting
two instances. Each node has disk capacity for an additional instance.
Therefore, for normal online rolling reboots, there are two node groups,...

Extract a partition functional

Separate the partitionNonRedundant function in hroller into a
general functional that partitions a list of nodes according
to some clearing strategy and the specialization of moving
non-redundant instances out. In this way, we don't have to...

Make hroller not consider offline nodes for evacuation

When planing on where to evacuate the non-redundant instances
of the nodes to be rebooted, it doesn't make sense to consider
offline nodes. So add this restriction to hroller.

Signed-off-by: Klaus Aehlig <aehlig@google.com>...

Clarify meaning of --print-moves in hroller man page

When given the option --print-moves, hroller prints a list of
instance relocations after each reboot group. This sequential
presentation can lead to the wrong impression, that these are
all the moves to be carried out. Therefore, explicitly mention...

Allow customization of the chroot

As different users have different needs about the chroot, allow
customization in its build script. This is achieved as follows.
- The environment variable EXTRA_DEBIAN_PACKAGES is honored,
and the packages named there are additionally installed in...

build_chroot: use APT_INSTALL macro for packet installation

In this way, it is easier to change all the apt calls in the chroot.
In particular, this patch fixes the missing -y option for the installation
of the tools needed to build debian packages.

Signed-off-by: Klaus Aehlig <aehlig@google.com>...

Enable disk growth with exclusive storage

Disk growth is now supported also with exclusive storage, so we let the
operation continue.

Enable disk growth test in qa also with exclusive storage.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add exclusive_storage to blockdev_grow RPC

The flag is passed to the RPC, but it's still ignored by noded.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Make NIC setup more flexible in case of Xen

For every NIC write down a file that contains NIC's
info (MAC, mode, link, network details), that can
be sourced by any vif script. The file location is:
/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>.
...

Document vif-ganeti usage

In order to be able to use it as vif-script one has to
create a symbolic link inside /etc/xen/scripts pointing
to /usr/lib/ganeti/vif-ganeti and then add is a hypervisor
parameter.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...

iallocator: storage processing accepts space_info structure

This is a small refactoring to make the processing of
the storage space information not accept a node info
result, but the more specific space info structure instead.

Signed-off-by: Helga Velroyen <helgav@google.com>...

iallocator: factor out instance memory summation

The iallocator adjusts the free memory by a calculation
of how much memory is actually currently used by instances
on the node. This calculation is rather complicated and
deserves its own function to increase testability. No...

Make NodeInfo (hs) accept arbitrary storage types

So far, the Haskell implementation of NodeInfo just
requests storage information about volume groups.
With this patch, storage info for abitrary storage
types can be requested.

Signed-off-by: Helga Velroyen <helgav@google.com>...

iallocator: computation of storage data

This patch extracts the processing of the storage-related
data from the RPC call 'node_info' in a separate function
to increase testability. In addition, this patch contains
unit tests for this function.

Signed-off-by: Helga Velroyen <helgav@google.com>...

iallocator: factor out 'get_attr' method

This patch extracts an inline 'get_attr' method to extract
data from the result of the RPC call 'node_info'. It was so
far an inline function in the middle of a bigger method, which made
the code hard to read.

Signed-off-by: Helga Velroyen <helgav@google.com>...

Typo in iallocator design doc

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Unset LV name if no LVM disk templates are enabled

If there are no LVM based disk templates enabled in QA, reset the LV
name after the corresponding gnt-cluster modify tests. Otherwise
gnt-cluster verify would complain about the missing volume group.

Also make sure that no disk templates requiring LVM are ever enabled...

Add route to VCluster network in vcluster-setup

This route is required on the Fedora test machine, as otherwise fping
would not work and hang forever.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Merge branch 'stable-2.8' into master

• stable-2.8:
  gnt-cluster info (py): add enabled disk templates
  Version bump to 2.8.0~beta1
  Change version numbers in documentation
  Fix issue with python coverage tests
  Merge branch 'stable-2.7' into stable-2.8...

Provide the right fix for the kvm_extra parameter

With the previous adjustment the arguments were boxed into one extra
array. This actually makes the parameter work (with the same caveats).

The manpage is also updated with the limitations.

Signed-off-by: Guido Trotter <ultrotter@google.com>...

NEWS and version updates for 2.7.0~rc3

Time for a new release.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Provide a special filter for host names

For host names, usually short forms are used, e.g., node1 or node1.sub
instead of the full qualified node1.sub.example.com. Therefore comparing
node names only by equality is too restrictive. This patch provides an...

Fix node modify QA tests for large cluster

If the node count in a cluster exceeds the master candidate pool size
(which is 10 by default), it's no longer possible to remove the master
candidate role from a node without passing --auto-promote (even with
--force). This behavior is perfectly fine, but the TestModifyNode QA...

Change version numbers in documentation

Some of the documents have the version number of Ganeti. This commit updates
it in preparation for the 2.8 version bump.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Fix issue with python coverage tests

The recently introduced check for python libraries required only for testing
(commit 27df5b736ef72b3b12c07f32d64dbac95fe7a5ba) was not correct. This
patch fixes the issue.

Signed-off-by: Michele Tartara <mtartara@google.com>...

grow disk: call SetDiskID before blockdev_getsize

Missing this call caused blockdev_getsize to be called on the wrong
physical device, and fail.

This fixes Issue 497.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Fix daemon QA tests

TestInstanceConsecutiveFailures now leaves the test instance in the same
state it got it.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Better specify what packages to install

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix typo in the documentation index

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Update security document wrt confd access to SSL cert

The fact that confd has been given access to the server.pem certificate
needs to be described in the security document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add a warning about the restriction of gnt-node evacuate

Even though gnt-node evacuate should be able to evacuate all instances
at once, this currently is not the case. Add a warning about this deficit
for the time being.

Signed-off-by: Klaus Aehlig <aehlig@google.com>...

Convert instance indices to UUIDs in configuration

cfgupgrade replaces the occurences of instance names (where they are
used as index) with the instance UUID.

It also supports the inverse replacement, so downgrades are
supported too.

Signed-off-by: Thomas Thrainer <thomasth@google.com>...

Node-UUID related cleanup

Some variables which hold node UUID's are renamed to reflect their
indent better.
Use cfg.GetNodeName(uuid) instead of cfg.GetNodeInfo(uuid).name where
appropriate.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>