Thomas Thrainer [Tue, 25 Jun 2013 15:26:13 +0000 (17:26 +0200)]
Replace tempfile by mktemp in QA code
The tempfile command only exists in Debian. In order to run the VCluster
QA also on Fedora, replace its use by mktemp.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Wed, 26 Jun 2013 07:57:25 +0000 (09:57 +0200)]
Merge branch 'stable-2.8' into master
* stable-2.8:
gnt-cluster info (py): add enabled disk templates
Version bump to 2.8.0~beta1
Change version numbers in documentation
Fix issue with python coverage tests
Merge branch 'stable-2.7' into stable-2.8
Disable python test if required libraries are missing
Better specify what packages to install
Improve install guide
Fix typo in the documentation index
Fix typos in the documentation index
Update security document wrt confd access to SSL cert
Add tools for building deb packages to build_chroot
Improve the final message of build_chroot
Make build_chroot self-contained
Conflicts:
configure.ac (trivial)
lib/cmdlib/instance_storage.py (trivial)
The version numbers changed in 'Change version numbers in documentation'
and 'Version bump to 2.8.0~beta1' were adapted to 2.9.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Tue, 25 Jun 2013 07:53:07 +0000 (09:53 +0200)]
Merge branch 'stable-2.7' into stable-2.8
* stable-2.7:
Provide the right fix for the kvm_extra parameter
NEWS and version updates for 2.7.0~rc3
kvm: don't pass whole extra as a single argument
For node queries allow short forms of host names
Provide a special filter for host names
Conflicts:
NEWS: trivial
configure.ac: trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Tue, 25 Jun 2013 06:43:57 +0000 (08:43 +0200)]
Provide the right fix for the kvm_extra parameter
With the previous adjustment the arguments were boxed into one extra
array. This actually makes the parameter work (with the same caveats).
The manpage is also updated with the limitations.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Guido Trotter [Mon, 24 Jun 2013 14:17:47 +0000 (16:17 +0200)]
NEWS and version updates for 2.7.0~rc3
Time for a new release.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Guido Trotter [Mon, 24 Jun 2013 14:45:00 +0000 (16:45 +0200)]
kvm: don't pass whole extra as a single argument
If extra is longer than a single word passing it the way it was passed
won't work. Of course this solution is also only partial, as won't take
into account any form of quoting. :/
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Fri, 21 Jun 2013 13:53:15 +0000 (15:53 +0200)]
For node queries allow short forms of host names
For node queries use the host-name filter instead of the simple
equality-based one.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Fri, 21 Jun 2013 13:51:00 +0000 (15:51 +0200)]
Provide a special filter for host names
For host names, usually short forms are used, e.g., node1 or node1.sub
instead of the full qualified node1.sub.example.com. Therefore comparing
node names only by equality is too restrictive. This patch provides an
alternative makeHostnameFilter to makeSimpleFilter suitable for comparing
host names.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Thu, 20 Jun 2013 09:44:16 +0000 (11:44 +0200)]
gnt-cluster info (py): add enabled disk templates
This fixes issue 485. In the python implementation of
the cluster config info, the enabled_disk_templates
were missing.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Thomas Thrainer [Mon, 24 Jun 2013 09:10:25 +0000 (11:10 +0200)]
Fix node modify QA tests for large cluster
If the node count in a cluster exceeds the master candidate pool size
(which is 10 by default), it's no longer possible to remove the master
candidate role from a node without passing --auto-promote (even with
--force). This behavior is perfectly fine, but the TestModifyNode QA
test didn't account for this.
TestModifyNode is changed such that it first removes the master
candidate role from the tested node with --auto-promote. This will, in
big clusters, promote an additional node to a master candidate which
ensures that the following tests succeed.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Mon, 24 Jun 2013 09:38:26 +0000 (09:38 +0000)]
Version bump to 2.8.0~beta1
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Michele Tartara [Fri, 3 May 2013 09:11:33 +0000 (11:11 +0200)]
Change version numbers in documentation
Some of the documents have the version number of Ganeti. This commit updates
it in preparation for the 2.8 version bump.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Weiwei Jia [Fri, 21 Jun 2013 13:28:39 +0000 (21:28 +0800)]
Doc for GlusterFS Ganeti Support
This patch adds a design document for introducing GlusterFS Support in Ganeti.
Signed-off-by: Weiwei Jia <harryxiyou@gmail.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Fri, 21 Jun 2013 16:15:35 +0000 (16:15 +0000)]
Fix issue with python coverage tests
The recently introduced check for python libraries required only for testing
(commit
27df5b736ef72b3b12c07f32d64dbac95fe7a5ba) was not correct. This
patch fixes the issue.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Thomas Thrainer [Fri, 21 Jun 2013 14:49:51 +0000 (16:49 +0200)]
Merge branch 'stable-2.7' into stable-2.8
* stable-2.7:
Add a warning about the restriction of gnt-node evacuate
Fix permissions of the queue archive dir
Fix daemon QA tests
grow disk: call SetDiskID before blockdev_getsize
Catch DeviceCreationError unhandled exceptions
Conflicts:
lib/cmdlib.py (due to the cmdlib split)
qa/qa_daemon.py (trivial)
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Fri, 21 Jun 2013 13:23:28 +0000 (15:23 +0200)]
Catch DeviceCreationError unhandled exceptions
_CreateBlockDevInner is called twice in TLReplaceDisks. While this
should be fixed, right now this leaves the DeviceCreationError exception
unhandled, which causes a problem due to the fact that this exception
was never supposed to be sent over the wire. For 2.7 we just catch the
exception and convert it to an OpExecError, but we should later do a
more proper cleanup.
This addresses Issue 472.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Fri, 21 Jun 2013 09:19:44 +0000 (11:19 +0200)]
grow disk: call SetDiskID before blockdev_getsize
Missing this call caused blockdev_getsize to be called on the wrong
physical device, and fail.
This fixes Issue 497.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Fri, 21 Jun 2013 09:12:17 +0000 (11:12 +0200)]
Fix daemon QA tests
TestInstanceConsecutiveFailures now leaves the test instance in the same
state it got it.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Fri, 21 Jun 2013 10:51:58 +0000 (12:51 +0200)]
Disable python test if required libraries are missing
Some python libraries are only needed for the tests, but not for running
Ganeti. If those libraries are missing, just disable the Python tests.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Michele Tartara [Fri, 21 Jun 2013 08:49:48 +0000 (08:49 +0000)]
Better specify what packages to install
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Fri, 21 Jun 2013 08:28:14 +0000 (10:28 +0200)]
Improve install guide
Add command for updating the list of cabal packages, before starting
installing them.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Fri, 21 Jun 2013 07:51:39 +0000 (09:51 +0200)]
Fix typo in the documentation index
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Weiwei Jia [Fri, 21 Jun 2013 08:56:32 +0000 (16:56 +0800)]
Fix typos in the documentation index
This patch fixes some typos in the documentation index.
Signed-off-by: Weiwei Jia <harryxiyou@gmail.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Thu, 20 Jun 2013 13:11:57 +0000 (13:11 +0000)]
Update security document wrt confd access to SSL cert
The fact that confd has been given access to the server.pem certificate
needs to be described in the security document.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Guido Trotter [Thu, 20 Jun 2013 16:35:40 +0000 (18:35 +0200)]
Fix permissions of the queue archive dir
The permissions were set to 0740, but of course being a directory the
correct permissions are 0750.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 20 Jun 2013 14:53:31 +0000 (16:53 +0200)]
Add a warning about the restriction of gnt-node evacuate
Even though gnt-node evacuate should be able to evacuate all instances
at once, this currently is not the case. Add a warning about this deficit
for the time being.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 15:03:08 +0000 (17:03 +0200)]
Add gnt-node migrate to QA
Make it possible to optionally run gnt-node migrate during QA.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Fri, 14 Jun 2013 12:56:45 +0000 (14:56 +0200)]
Convert instance indices to UUIDs in configuration
cfgupgrade replaces the occurences of instance names (where they are
used as index) with the instance UUID.
It also supports the inverse replacement, so downgrades are
supported too.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Tue, 11 Jun 2013 15:05:41 +0000 (17:05 +0200)]
Index instances by their UUID
No longer index instances by their name but by their UUID in the cluster
config. This change changes large parts of the code, as the following
adjustments were necessary:
* Change the index key to UUID in the configuration and the
ConfigWriter, including all methods.
* External interfaces (command line interface, IAllocator interface,
hook scripts, etc.) are kept stable.
* Instance UUID's are resolved in ExpandNames and then stored in the
OpCode. This allows to check for instance renames if the OpCode is
reloaded after a cluster restart. This check is currently only done
for single instance parameters.
* Instance locking unfortunately can't use instances UUID as
identifiers. The reasons is that new instances (which have no UUID
yet) have to be locked as well, so the instance name is used.
* Variable names are renamed to follow the following pattern:
- Suffix is 'inst' or 'insts': Variable holds Instance objects
- Suffix is 'name' or 'names': Variable holds Instance names
- Suffix is 'uuid' or 'uuids': Variable holds Instance UUID's
* Tests are adapted.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Mon, 10 Jun 2013 11:59:40 +0000 (13:59 +0200)]
Node-UUID related cleanup
Some variables which hold node UUID's are renamed to reflect their
indent better.
Use cfg.GetNodeName(uuid) instead of cfg.GetNodeInfo(uuid).name where
appropriate.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 20 Jun 2013 12:55:54 +0000 (14:55 +0200)]
Add tools for building deb packages to build_chroot
The chroot only had tools for building and testing Ganeti, but not for
building its Debian packages. They are added by this commit.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 12:15:35 +0000 (14:15 +0200)]
Fix wrong assert
An assert wrongly verified that the node name instead of the node UUID
is in the set of locked nodes. This patch fixes this.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 20 Jun 2013 09:29:25 +0000 (09:29 +0000)]
Improve the final message of build_chroot
Give more precise instructions on the next steps to actually run the
chroot.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 20 Jun 2013 08:57:24 +0000 (08:57 +0000)]
Make build_chroot self-contained
The configuration file are included in build_chroot, and are
automatically created by the script if it is not able to find them
already in the directory pointed by $DATA_DIR.
Note that the configuration file directory is cleaned if it's temporary,
but it is left untouched if it was esplicitly specified by the user.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 11:16:59 +0000 (13:16 +0200)]
Fix testGetHvDefaults testcase
The introduction of the migration_mode parameter to the fake hypervisor
broke the testGetHvDefaults. This patch fixes this by taking the default
parameters of the fake hypervisor into account.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 10:55:40 +0000 (12:55 +0200)]
Fix instance add for missing node specs
Don't pass None to cfg.GetNodeNames. Instead, only resolve the node
names if it's sure that the UUID list is valid.
This fixes issue 495.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Thu, 20 Jun 2013 11:38:28 +0000 (13:38 +0200)]
Merge branch 'stable-2.8' into master
* stable-2.8
In tiered allocation, cut non-promising shrinking tries
Add confd user to masterd group
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 07:21:18 +0000 (09:21 +0200)]
Skip Xen monitoring QA for non-Xen clusters
Instead of asserting that the Xen instance status collector fails for
non-Xen clusters just skip the test completely.
Otherwise QA would fail for non-Xen hypervisors if Xen happens to be
installed on the cluster as well.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Wed, 19 Jun 2013 15:45:33 +0000 (17:45 +0200)]
Add support to migration_mode in fake HV
The fake hypervisor pretends to support migration, but did not support
the migration_mode parameter yet. As this parameter is required during
instance migrations, support for it is added in this patch.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Wed, 19 Jun 2013 15:22:33 +0000 (17:22 +0200)]
Adapt VCluster script to current situation
vcluster-setup is adapted to the current Ganeti command line arguments.
Also, a bug for diskless instances in qa_instance is fixed.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Tue, 18 Jun 2013 14:34:18 +0000 (16:34 +0200)]
Check DRBD status in cluster verify
If the status of DRBD disks is degraded or in a faulty state, issue an
error in cluster verify.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Wed, 19 Jun 2013 14:51:57 +0000 (14:51 +0000)]
Add confd user to masterd group
The server.pem file is 0440 (materd_uid, masterd_gid) and cannot be
accessed by confd.
This patch contains a workaround for 2.8.0~beta1 (adding conf to the
masterd group in the user setup tool, and remembering the users to
actually execute that tool while upgrading).
This partially fixes Issue 292.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Wed, 19 Jun 2013 16:43:21 +0000 (18:43 +0200)]
In tiered allocation, cut non-promising shrinking tries
The heuristics for tiered allocation has been improved in that it
chooses to shrink such a resource next where by shrinking only this
resource a valid allocation can be made, if such a resource exists.
In order to decide whether such a resource exists, all potential
sizes of this resource are verified. We can improve performance by
cutting, as soon a potential allocation of an instance is no longer
blocked on this resource.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Wed, 19 Jun 2013 10:11:50 +0000 (12:11 +0200)]
Merge branch 'stable-2.8' into master
* stable-2.8
(no changes)
* merged from stable-2.7
More agressively line break man pages
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Wed, 19 Jun 2013 08:24:01 +0000 (08:24 +0000)]
Version bump to 2.9.0~alpha1
Bump the version number to 2.9.0~alpha1.
This also requires updating version numbers in cfgupgrade tool.
Furthermore, a data file for test representing a 2.8 configuration had the
version number set to 2.7 in order to prevent tests failing while the current
branch was artificially kept behind its actual version number. Now that alpha
versions are possible, it can be moved back to 2.8 as it was supposed to be.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Wed, 19 Jun 2013 09:02:53 +0000 (11:02 +0200)]
Merge branch 'stable-2.7' into stable-2.8
* stable-2.7
More agressively line break man pages
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Thomas Thrainer [Wed, 19 Jun 2013 07:00:56 +0000 (09:00 +0200)]
Fix reference to NodeInfo.name
This fixes a bug introduced during the move from node names to node
UUIDs as indexes.
Also, a typo is corrected.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Tue, 18 Jun 2013 14:21:20 +0000 (14:21 +0000)]
Merge branch 'stable-2.8' into 'master'
* stable-2.8
Version bump to 2.8.0~alpha1
Make cfgupgrade idempotent
Update config version number when downgrading
Add test for shrink heuristics over different resources
Improve hspace shrinking strategy
Convenience function for iterating while the result is Ok
Provide witness for the sum-type structure of GenericResult
Disable more version checks for alpha versions
Allow alpha versions not to have an entry in the NEWS file
Add tests for check-news
Introduce support for alpha versions
Improve docstring
Fix bug in Makefile.am
Restructure documentation index page
Add missing parenthesis to description of --machine-readable
Update NEWS file for 2.8
Add a --force option to gnt-cluster modify
Add a force option to the ClusterSetParams Opcode
Document the monitoring daemon in the admin guide
Factor out the API of the monitoring daemon
Add monitoring agent info to the security document
Export device UUIDs to hooks and OS scripts
gnt-cluster info: only parameters for enabled hypervisors
Support RAPI QA on an already existing cluster
Conflicts:
Makefile.am:
Trivial, merge of lists of files.
man/gnt-cluster.rst
One parameter added in stable-2.8, one in master
test/hs/shelltests/htools-hspace.test
Tests added in both branches
tools/cfgupgrade
Some upgrade steps are not required anymore, some have been removed.
src/Ganeti/HTools/Instance.hs
Some types changes in the Haskell code because of the patches related to
Ganeti Dedicated.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Tue, 18 Jun 2013 12:11:15 +0000 (15:11 +0300)]
More agressively line break man pages
Newer versions of pandoc (as, e.g., shipped with Ubuntu 13.04)
more agressively quote, in particular minus signs. That sometimes
leads to unintended line breaks, which are a problem, if the new
line starts with a dot.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Fri, 14 Jun 2013 12:57:49 +0000 (14:57 +0200)]
Version bump to 2.8.0~alpha1
Now that alpha versions are supported, we can bump the version number
for branch stable-2.8 to 2.8.
This also requires updating the cfgupgrade tool.
Thanks to the previous patches, all the other documents can be upgraded
during the alpha lifetime, before switching to beta (that will enable
the version number checks).
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Tue, 18 Jun 2013 07:40:12 +0000 (09:40 +0200)]
Revert "Storage utility functions for Haskell"
This reverts commit
88d27b8aa8adc2e5ced773909f1d40812c5a6ea7.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Tue, 18 Jun 2013 07:39:15 +0000 (09:39 +0200)]
Revert "Make NodeInfo (hs) accept arbitrary storage types"
This reverts commit
e89525a859b2e841c08fce506c0b68b97c7efe61.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Michele Tartara [Tue, 18 Jun 2013 07:33:48 +0000 (07:33 +0000)]
Make cfgupgrade idempotent
One of the unit tests checks whether cfgupgrade's downgrade option is
idempotent. It is, but a version number check made it impossible to
actually downgrade multiple times.
With this change, both the current-version version number and the
target-version version number are accepted by the downgrade tool.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Tue, 18 Jun 2013 07:32:31 +0000 (07:32 +0000)]
Update config version number when downgrading
The downgrade option of the cfgupgrade tool was not changing the version
number in the configuration file of the cluster while performing the
downgrade.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Fri, 14 Jun 2013 13:43:47 +0000 (15:43 +0200)]
Make NodeInfo (hs) accept arbitrary storage types
So far, the Haskell implementation of NodeInfo just
requests storage information about volume groups.
With this patch, storage info for abitrary storage
types can be requested.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 13 Jun 2013 17:44:06 +0000 (19:44 +0200)]
Storage utility functions for Haskell
In order to extend the Haskell version of the NodeInfo
query, we need some utility functions to deal with
disk templates and storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Fri, 14 Jun 2013 15:05:19 +0000 (17:05 +0200)]
Rename directory 'Block' to 'Storage'
This patch renames the 'Block' directory to 'Storage' in
the Haskell code base. The same rename was done in the
python code base earlier this quarter. We generalize the
name, because we needed a place for general storage
utility functions and storage code for other storage
types like for example file storage.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Klaus Aehlig [Fri, 14 Jun 2013 12:19:01 +0000 (14:19 +0200)]
Add test for shrink heuristics over different resources
In this example, memory prevents adding more than one instance on
the first node. Yet, on the other nodes, 2 instances can be placed
each, if disks are shrunk appropriately. This, however, requires
to courageous decision to shrink disks next, even though more nodes
fail to allocate more due to memory than to disk.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 14 Jun 2013 13:48:47 +0000 (15:48 +0200)]
Improve hspace shrinking strategy
In tired allocation, hspace shrinks that resource of the instance
next, that causes failure on most nodes. While, this is not a bad
strategy in general, it can lead hspace into a dead end if for a large
number of nodes a particular resource blocks any further allocation of
policy compliant instances. So we improve the heuristics in that it
chooses to shrink such a resource next where by shrinking only this
resource a valid allocation can be made, if such a resource
exists. This improves the results in some cases, while still keeping
the computational complexity of the algorithm low.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 14 Jun 2013 15:42:15 +0000 (17:42 +0200)]
Convenience function for iterating while the result is Ok
For a function f :: a -> GenericResult a, iterate it (in the sense of the
monad), until the result is Bad; return the list of values occurred.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 14 Jun 2013 13:47:55 +0000 (15:47 +0200)]
Provide witness for the sum-type structure of GenericResult
GenericResult, while rightfully a type of its own, is isomorphic
to Either. So, also provide the case analysis function (i.e., the
universal arrow out of the sum).
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Thu, 13 Jun 2013 14:56:30 +0000 (14:56 +0000)]
Disable more version checks for alpha versions
Alpha versions should not check for version numbers in READMEs
and documentation.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 13 Jun 2013 14:15:15 +0000 (14:15 +0000)]
Allow alpha versions not to have an entry in the NEWS file
Alpha versions are still under heavy development, and can therefore not have an
entry in the NEWS file yet. The entry will have to be added before switching to
the first beta version.
Partially fixes Issue 448
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 13 Jun 2013 12:09:40 +0000 (14:09 +0200)]
Add tests for check-news
Unit tests verifying the behaviour of the check-news script are added.
This required adding a new make target, "autotools-check" directly called from
"commit-check", because the usual tests are executed in the build directory,
whereas these, being relative to the autotools, cannot be executed there because
the files of the autotools/ directory are not copied do not belong in the build
directory.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 13 Jun 2013 11:00:12 +0000 (11:00 +0000)]
Introduce support for alpha versions
An alpha version can be used to name a release while a previous one is still
being developed.
E.g. 2.9.0~alpha1 can be out while 2.8.0~rc1 is being worked on.
When a version is marked as alpha, it is allowed to have unreleased versions
before it in the NEWS file, in order to simplify merges of the NEWS file itself
between the two development branches.
Partially fixes Issue 448.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 13 Jun 2013 10:59:47 +0000 (10:59 +0000)]
Improve docstring
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 13 Jun 2013 15:32:53 +0000 (17:32 +0200)]
Fix bug in Makefile.am
There was a typo in the name of the script for launching offline tests.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Thu, 13 Jun 2013 09:21:34 +0000 (11:21 +0200)]
Refactor NodeInfo RPC regarding storage reporting
The NodeInfo RPC call is refactored to handle now more than
just storage reporting for volume groups.
Since NodeInfo now returns storage space information not
necessarily for volume groups, but also for other storage
entities, it is no longer appropriate to call the fields
of the NodeInfo result "vg_free" and "vg_size". We rename
it to "storage_free" and "storage_size" instead.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 13 Jun 2013 09:19:22 +0000 (11:19 +0200)]
Utility functions for storage types
Handling various storage types for the free space reporting
requires some utility functions. They will be invoked in
later patches. Unit tests are included.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Wed, 12 Jun 2013 09:14:49 +0000 (11:14 +0200)]
Integrate free space reporting for file storage
This patch integrates the free space reporting for file
storage. So far, it is only called, when the default
disk template is a file-based disk template ("file" or
"sharedfile").
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Wed, 12 Jun 2013 09:14:20 +0000 (11:14 +0200)]
Unit tests for ApplyStorageSpaceFunction
So far, the function 'ApplyStorageSpaceFunction' was not
covered by unit tests. Before the integration of space
reporting for file storage, we add those missing tests.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Wed, 12 Jun 2013 08:44:48 +0000 (10:44 +0200)]
Renamining space reporting for file storage
Before being integrated, the function for space reporting
for file storage is renamed to a more expressive name.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 13 Jun 2013 09:49:30 +0000 (11:49 +0200)]
Add storage type to NodeInfo result
So far, the storage information returned from RPC call
NodeInfo contained only information about volume groups.
In order to extend the storage reporting to other storage
type, we include another field "type" in the result of
the RPC call to be able to distinguish storage space
information from different storage units.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Fri, 14 Jun 2013 08:36:23 +0000 (10:36 +0200)]
Use cluster's hvparams in LUInstSetParams
Since the instance does not have hvparams at that point,
we have to use the cluster's hvparams here. This fixes
a crash when using 'gnt-instance modify'.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 14 Jun 2013 08:29:40 +0000 (10:29 +0200)]
Merge branch 'stable-2.7' into stable-2.8
* stable-2.7
Make shrinkByType aware of individual disks
Add a test for hspace to respect instance policy
Conflicts: (trival, take union of added files/tests)
Makefile.am
test/hs/shelltests/htools-hspace.test
Semantic conflicts:
test/data/htools/hspace-tiered-ipolicy.data
(adapt to changed text format)
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 13 Jun 2013 14:05:46 +0000 (16:05 +0200)]
Add a test for hspace to respect instance policy
In this example, the instance policy for disks prevents allocation
of more than one instance per node. The test verifies that tiered
allocation still respects this and doesn't attempt to place policy
violating instances on the cluster.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Klaus Aehlig [Thu, 13 Jun 2013 14:05:45 +0000 (16:05 +0200)]
Make shrinkByType aware of individual disks
When shrinking an instances, you can't just get smaller disk footprint
while leaving the individual disks as they are. Make the shrink
heuristic aware of that fact, and decrease all individual disks as
well. Fixes issue 484.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 13 Jun 2013 15:23:54 +0000 (17:23 +0200)]
Fix lookup of xen toolstack in Haskell
There was a bug in the haskell implementation of node
query which made the lookup of the xen toolstack xm/xl
fail.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Thomas Thrainer [Fri, 7 Jun 2013 09:47:38 +0000 (11:47 +0200)]
Inline simple "alias" variables
Remove variables in the form "var = self.var" throughout all LU's. Those
"alias" variables make it harder to read the code (it's no longer
immediately clear what's a field and what's a local variable), and can
introduce subtle bug (especially when not used consistently and when
they are re-assigned).
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Thu, 6 Jun 2013 13:53:07 +0000 (15:53 +0200)]
Convert node indices to UUIDs in configuration
cfgupgrade replaces the occurences of node names (where they are used as
index) with the node UUID.
It also supports the inverse replacement, so downgrades are supported
too.
Tests for cfgupgrade are adapted as well.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Mon, 27 May 2013 14:17:41 +0000 (16:17 +0200)]
Index nodes by their UUID
No longer index nodes by their name but by their UUID in the cluster
config. This change changes large parts of the code, as the following
adjustments were necessary:
* Change the index key to UUID in the configuration and the
ConfigWriter, including all methods.
* Change all cross-references to nodes to use UUID's.
* External interfaces (command line interface, IAllocator interface,
hook scripts, etc.) are kept stable.
* RPC-calls can resolve UUID's as target node arguments, if the RPC
runner is based on a ConfigWriter instance. The result dictionary is
presented in the form the nodes are addressed: by UUID if UUID's were
given, or by name if names were given.
* Node UUID's are resolved in ExpandNames and then stored in the
OpCode. This allows to check for node renames if the OpCode is
reloaded after a cluster restart. This check is currently only done
for single node parameters.
* Variable names are renamed to follow the following pattern:
- Suffix is 'node' or 'nodes': Variable holds Node objects
- Suffix is 'name' or 'names': Variable holds node names
- Suffix is 'uuid' or 'uuids': Variable holds node UUID's
* Tests are adapted.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Wed, 12 Jun 2013 08:29:14 +0000 (10:29 +0200)]
Add cluster name to instance migration RPC
This fixes a 'FIXME' in the xen hypervisor. The instance
migration requires the cluster's name. This should come
from the RPC call, but so far was hackishly taken from
ssconf. Ssconf is intended to be used by scripts only and
not by the backend itself.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Michele Tartara [Wed, 12 Jun 2013 14:48:30 +0000 (14:48 +0000)]
Restructure documentation index page
The index page of the documentation used to be just a list of links.
Now it is a proper introduction to all the other pages.
Fixes Issue 424.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Wed, 12 Jun 2013 11:45:10 +0000 (13:45 +0200)]
Add missing parenthesis to description of --machine-readable
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Klaus Aehlig [Mon, 10 Jun 2013 10:02:22 +0000 (12:02 +0200)]
Support VNC binding to an interface
Currently, the VNC bind address can only be specified as an IP
address. Given that this is a cluster-wide parameter, there
aren't many sensible choices besides 127.0.0.1 and 0.0.0.0. This
patch allows to also specify it as the address of a specifed
interface.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Tue, 14 May 2013 12:42:15 +0000 (13:42 +0100)]
Update NEWS file for 2.8
Add to the NEWS file the list of modifications performed during the 2.8
development cycle.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Helga Velroyen [Thu, 6 Jun 2013 17:08:52 +0000 (17:08 +0000)]
Remove XEN_CMD from configure.ac
Removes the option --with-xen-cmd from the configure
script.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 6 Jun 2013 16:41:01 +0000 (18:41 +0200)]
Remove XEN_CMD from constants, adjust to PowercycleNode
This patch extend the RPC to powercycle a node by a
hvparams parameter. This parameter is used by the
xen hypervisor to determine which xen command to use.
The respective LU is adjusted as well.
Since this was the last operation depending on the
constants.XEN_CMD, it is removed eventually.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Wed, 5 Jun 2013 16:36:10 +0000 (18:36 +0200)]
hv_xen.py: make hvparams mandatory, remove fallbacks
This patch removes the fallback to the xen command from the
auto config and adds an exception instead. Also, the
hvparams parameter of the functions GetCommand, _RunXen and
_GetInstanceList are made mandatory.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 6 Jun 2013 14:35:26 +0000 (16:35 +0200)]
Use hvparams in GetAllInstancesInfo
This patch extends the GetAllInstancesInfo method of the
hypervisors by a hvparams parameter. This change propagates
up to the RPC call all_instances_info, which also needed
to be extended. All callers of this call were adjusted
as well.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 6 Jun 2013 14:22:11 +0000 (16:22 +0200)]
Use hvparams in GetInstanceInfo
This patch extends the GetInstanceInfo function of the
hypervisors by an hvparams parameters. This leads to
an extension of the instance_info RPC call and the
adjustmend of all classes using this call.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 6 Jun 2013 13:08:38 +0000 (15:08 +0200)]
backend.py: remove fallback implementation of GetNodeInfo
This removes the fallback implementation of the backend's
GetNodeInfo function to the old implementation. By now,
all calling code is adjusted to the new hvparams parameter
and thus, the old implementation should not be necessary
anymore.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 6 Jun 2013 11:45:22 +0000 (13:45 +0200)]
Adjust all callers of RPC 'call_node_info'
This patch contains all adjustments necessary to code that
uses the 'call_node_info' call. The adjustment is necessary
because the call's signature changed from receiving a list
of hypervisor names to a list of tuples
(hypervisor_name, hvparams).
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 6 Jun 2013 11:40:10 +0000 (13:40 +0200)]
Add hvparams to RPC call 'node_info'
This patch adds the hvparams parameter to the RPC call
'node_info'. It also adjusts the related code in noded.py
and Query/Node.hs
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 6 Jun 2013 08:51:21 +0000 (10:51 +0200)]
backend.py: use hvparams in GetNodeInfo
This patch extends the GetNodeInfo function of the backend
module by a hvparams parameter. For now, it contains a
fallback solution for when the parameter is None. This
fallback will be removed in a later patch of this series.
It was necessary to be abled to gradually adjust all
calling code without breaking everything at once.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 6 Jun 2013 08:38:56 +0000 (10:38 +0200)]
hypervisors: add hvparams to GetNodeInfo
This patch extends the GetNodeInfo function of the
hypervisors by a hvparams parameter. The parameter
is currently only used by the xen hypervisor to determine
which xen toolstack ('xm' or 'xl') to use.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 6 Jun 2013 12:21:07 +0000 (14:21 +0200)]
hv_xen.py: renaming residual 'xm' occurrences
Some methods still refered to 'xm', which is inaccurate now
that xen can also be run with 'xl'. No functional changes
otherwise.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Wed, 5 Jun 2013 15:15:49 +0000 (17:15 +0200)]
Watcher: Use hvparams when listing instances
This patch makes the watcher use the hvparams from ssconf
to retrieve the list of instances from the hypervisors.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Wed, 5 Jun 2013 15:05:19 +0000 (17:05 +0200)]
Include hvparams in ssconf files
The watcher needs access to the cluster's hypervisor
parameters. This patch extends the ssconf infrastructure
by adding a hvparams file for each hypervisor. The format
of the hvparams file is a simple key=value format.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Wed, 5 Jun 2013 09:08:07 +0000 (11:08 +0200)]
Use hvparams in instance migration
This patch makes use of hvparams on instance migration to
determine the list of instance before the migration. This way, in
xen, the xl/xm choice is respected.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>