Synnefo » snf-ganeti

Allow customization of the chroot

As different users have different needs about the chroot, allow
customization in its build script. This is achieved as follows.
- The environment variable EXTRA_DEBIAN_PACKAGES is honored,
and the packages named there are additionally installed in...

build_chroot: use APT_INSTALL macro for packet installation

In this way, it is easier to change all the apt calls in the chroot.
In particular, this patch fixes the missing -y option for the installation
of the tools needed to build debian packages.

Signed-off-by: Klaus Aehlig <aehlig@google.com>...

Enable disk growth with exclusive storage

Disk growth is now supported also with exclusive storage, so we let the
operation continue.

Enable disk growth test in qa also with exclusive storage.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add exclusive_storage to blockdev_grow RPC

The flag is passed to the RPC, but it's still ignored by noded.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Make NIC setup more flexible in case of Xen

For every NIC write down a file that contains NIC's
info (MAC, mode, link, network details), that can
be sourced by any vif script. The file location is:
/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>.
...

Document vif-ganeti usage

In order to be able to use it as vif-script one has to
create a symbolic link inside /etc/xen/scripts pointing
to /usr/lib/ganeti/vif-ganeti and then add is a hypervisor
parameter.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...

iallocator: storage processing accepts space_info structure

This is a small refactoring to make the processing of
the storage space information not accept a node info
result, but the more specific space info structure instead.

Signed-off-by: Helga Velroyen <helgav@google.com>...

iallocator: factor out instance memory summation

The iallocator adjusts the free memory by a calculation
of how much memory is actually currently used by instances
on the node. This calculation is rather complicated and
deserves its own function to increase testability. No...

Make NodeInfo (hs) accept arbitrary storage types

So far, the Haskell implementation of NodeInfo just
requests storage information about volume groups.
With this patch, storage info for abitrary storage
types can be requested.

Signed-off-by: Helga Velroyen <helgav@google.com>...

iallocator: computation of storage data

This patch extracts the processing of the storage-related
data from the RPC call 'node_info' in a separate function
to increase testability. In addition, this patch contains
unit tests for this function.

Signed-off-by: Helga Velroyen <helgav@google.com>...

iallocator: factor out 'get_attr' method

This patch extracts an inline 'get_attr' method to extract
data from the result of the RPC call 'node_info'. It was so
far an inline function in the middle of a bigger method, which made
the code hard to read.

Signed-off-by: Helga Velroyen <helgav@google.com>...

Typo in iallocator design doc

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Unset LV name if no LVM disk templates are enabled

If there are no LVM based disk templates enabled in QA, reset the LV
name after the corresponding gnt-cluster modify tests. Otherwise
gnt-cluster verify would complain about the missing volume group.

Also make sure that no disk templates requiring LVM are ever enabled...

Add route to VCluster network in vcluster-setup

This route is required on the Fedora test machine, as otherwise fping
would not work and hang forever.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Merge branch 'stable-2.8' into master

• stable-2.8:
  gnt-cluster info (py): add enabled disk templates
  Version bump to 2.8.0~beta1
  Change version numbers in documentation
  Fix issue with python coverage tests
  Merge branch 'stable-2.7' into stable-2.8...

Provide the right fix for the kvm_extra parameter

With the previous adjustment the arguments were boxed into one extra
array. This actually makes the parameter work (with the same caveats).

The manpage is also updated with the limitations.

Signed-off-by: Guido Trotter <ultrotter@google.com>...

NEWS and version updates for 2.7.0~rc3

Time for a new release.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Provide a special filter for host names

For host names, usually short forms are used, e.g., node1 or node1.sub
instead of the full qualified node1.sub.example.com. Therefore comparing
node names only by equality is too restrictive. This patch provides an...

Fix node modify QA tests for large cluster

If the node count in a cluster exceeds the master candidate pool size
(which is 10 by default), it's no longer possible to remove the master
candidate role from a node without passing --auto-promote (even with
--force). This behavior is perfectly fine, but the TestModifyNode QA...

Change version numbers in documentation

Some of the documents have the version number of Ganeti. This commit updates
it in preparation for the 2.8 version bump.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Fix issue with python coverage tests

The recently introduced check for python libraries required only for testing
(commit 27df5b736ef72b3b12c07f32d64dbac95fe7a5ba) was not correct. This
patch fixes the issue.

Signed-off-by: Michele Tartara <mtartara@google.com>...

grow disk: call SetDiskID before blockdev_getsize

Missing this call caused blockdev_getsize to be called on the wrong
physical device, and fail.

This fixes Issue 497.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Fix daemon QA tests

TestInstanceConsecutiveFailures now leaves the test instance in the same
state it got it.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Better specify what packages to install

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix typo in the documentation index

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Update security document wrt confd access to SSL cert

The fact that confd has been given access to the server.pem certificate
needs to be described in the security document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add a warning about the restriction of gnt-node evacuate

Even though gnt-node evacuate should be able to evacuate all instances
at once, this currently is not the case. Add a warning about this deficit
for the time being.

Signed-off-by: Klaus Aehlig <aehlig@google.com>...

Convert instance indices to UUIDs in configuration

cfgupgrade replaces the occurences of instance names (where they are
used as index) with the instance UUID.

It also supports the inverse replacement, so downgrades are
supported too.

Signed-off-by: Thomas Thrainer <thomasth@google.com>...

Node-UUID related cleanup

Some variables which hold node UUID's are renamed to reflect their
indent better.
Use cfg.GetNodeName(uuid) instead of cfg.GetNodeInfo(uuid).name where
appropriate.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix wrong assert

An assert wrongly verified that the node name instead of the node UUID
is in the set of locked nodes. This patch fixes this.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Make build_chroot self-contained

The configuration file are included in build_chroot, and are
automatically created by the script if it is not able to find them
already in the directory pointed by $DATA_DIR.

Note that the configuration file directory is cleaned if it's temporary,...

Fix instance add for missing node specs

Don't pass None to cfg.GetNodeNames. Instead, only resolve the node
names if it's sure that the UUID list is valid.

This fixes issue 495.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Skip Xen monitoring QA for non-Xen clusters

Instead of asserting that the Xen instance status collector fails for
non-Xen clusters just skip the test completely.
Otherwise QA would fail for non-Xen hypervisors if Xen happens to be
installed on the cluster as well....

Adapt VCluster script to current situation

vcluster-setup is adapted to the current Ganeti command line arguments.
Also, a bug for diskless instances in qa_instance is fixed.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Add confd user to masterd group

The server.pem file is 0440 (materd_uid, masterd_gid) and cannot be
accessed by confd.

This patch contains a workaround for 2.8.0~beta1 (adding conf to the
masterd group in the user setup tool, and remembering the users to...

Merge branch 'stable-2.8' into master

• stable-2.8
  (no changes)

• merged from stable-2.7
  More agressively line break man pages

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Merge branch 'stable-2.7' into stable-2.8

• stable-2.7
  More agressively line break man pages

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Merge branch 'stable-2.8' into 'master'

• stable-2.8
  Version bump to 2.8.0~alpha1
  Make cfgupgrade idempotent
  Update config version number when downgrading
  Add test for shrink heuristics over different resources
  Improve hspace shrinking strategy...

Version bump to 2.8.0~alpha1

Now that alpha versions are supported, we can bump the version number
for branch stable-2.8 to 2.8.

This also requires updating the cfgupgrade tool.

Thanks to the previous patches, all the other documents can be upgraded
during the alpha lifetime, before switching to beta (that will enable...

Revert "Make NodeInfo (hs) accept arbitrary storage types"

This reverts commit
e89525a859b2e841c08fce506c0b68b97c7efe61.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Update config version number when downgrading

The downgrade option of the cfgupgrade tool was not changing the version
number in the configuration file of the cluster while performing the
downgrade.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Storage utility functions for Haskell

In order to extend the Haskell version of the NodeInfo
query, we need some utility functions to deal with
disk templates and storage types.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Add test for shrink heuristics over different resources

In this example, memory prevents adding more than one instance on
the first node. Yet, on the other nodes, 2 instances can be placed
each, if disks are shrunk appropriately. This, however, requires
to courageous decision to shrink disks next, even though more nodes...

Convenience function for iterating while the result is Ok

For a function f :: a -> GenericResult a, iterate it (in the sense of the
monad), until the result is Bad; return the list of values occurred.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Disable more version checks for alpha versions

Alpha versions should not check for version numbers in READMEs
and documentation.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add tests for check-news

Unit tests verifying the behaviour of the check-news script are added.

This required adding a new make target, "autotools-check" directly called from
"commit-check", because the usual tests are executed in the build directory,
whereas these, being relative to the autotools, cannot be executed there because...

Improve docstring

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Refactor NodeInfo RPC regarding storage reporting

The NodeInfo RPC call is refactored to handle now more than
just storage reporting for volume groups.

Since NodeInfo now returns storage space information not
necessarily for volume groups, but also for other storage...

Integrate free space reporting for file storage

This patch integrates the free space reporting for file
storage. So far, it is only called, when the default
disk template is a file-based disk template ("file" or
"sharedfile").

Signed-off-by: Helga Velroyen <helgav@google.com>...

Renamining space reporting for file storage

Before being integrated, the function for space reporting
for file storage is renamed to a more expressive name.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>