Michele Tartara [Tue, 4 Jun 2013 18:25:48 +0000 (14:25 -0400)]
Fix documentation formatting issue
One line was longer than 80 characters.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Klaus Aehlig [Wed, 26 Jun 2013 11:58:36 +0000 (13:58 +0200)]
Add test for the --full-evacuation option
In this example cluster, there are four pairs of nodes, each hosting
two instances. Each node has disk capacity for an additional instance.
Therefore, for normal online rolling reboots, there are two node groups,
of four nodes each, whereas for full evacuation, these have to be refined
to four groups, of two nodes each.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 27 Jun 2013 13:35:46 +0000 (15:35 +0200)]
hroller: option --full-evacuation
Add an option to hroller, to plan for full evacuation of the
nodes to be rebooted, i.e., also plan for replacement secondary
nodes for all instances on the node after migrating out instances
with this node as primary.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 27 Jun 2013 13:34:14 +0000 (15:34 +0200)]
Extract a partition functional
Separate the partitionNonRedundant function in hroller into a
general functional that partitions a list of nodes according
to some clearing strategy and the specialization of moving
non-redundant instances out. In this way, we don't have to
duplicate code when partitioning due to capacity of finding
new secondary nodes.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 27 Jun 2013 10:58:15 +0000 (12:58 +0200)]
Extract functional for greedily clearing nodes
The method clearNodes in hroller greedily clears nodes of
non-redundant instances by moving them to a different node. This patch
separates the greedy clearing algorithm from the specialization to
non-redundant instances; in this way, we don't have to duplicate code
when supporting clearing nodes of secondaries.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Wed, 26 Jun 2013 12:51:24 +0000 (14:51 +0200)]
Make hroller not consider offline nodes for evacuation
When planing on where to evacuate the non-redundant instances
of the nodes to be rebooted, it doesn't make sense to consider
offline nodes. So add this restriction to hroller.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Wed, 12 Jun 2013 09:42:24 +0000 (11:42 +0200)]
Update comments in hroller code
hroller schedules moves of instances to have rebooted nodes
free of instances with this node as primary. Update the comments
to reflect that this move planning is for non-redundant instances
only.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Wed, 26 Jun 2013 16:21:59 +0000 (18:21 +0200)]
Clarify meaning of --print-moves in hroller man page
When given the option --print-moves, hroller prints a list of
instance relocations after each reboot group. This sequential
presentation can lead to the wrong impression, that these are
all the moves to be carried out. Therefore, explicitly mention
the assumption that after each reboot of a group, all instances
are moved to their initial position.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 27 Jun 2013 13:59:15 +0000 (15:59 +0200)]
Remove obsolete TODO
Originally, hroller started as a tool for offline maintenance only.
There it made sense to warn about instances still running. By now,
default planning is to migrate instance off the nodes to be rebooted,
with options for other behavior (like pretending that all instances
will be shut down before the actual reboots start). In that scenario
instances running is the normal situation, not something to be warned
about.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 28 Jun 2013 11:44:34 +0000 (13:44 +0200)]
Allow customization of the chroot
As different users have different needs about the chroot, allow
customization in its build script. This is achieved as follows.
- The environment variable EXTRA_DEBIAN_PACKAGES is honored,
and the packages named there are additionally installed in
the chroot. This should cover the most common cases.
- More complicated customizations can be achieved by setting
the environment variable CHROOT_FINAL_HOOK. This is the last
command executed in the chroot; typically, it would be
a call to a personal script.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Fri, 28 Jun 2013 13:53:20 +0000 (15:53 +0200)]
build_chroot: fix duplicate install
In the last change to the chroot build script, the
apt-get command 'install' was moved to the APT_INSTALL
macro; however, it was forgotten to remove it outside
the macro, causing an error, as there is no package
with name 'install'.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Fri, 28 Jun 2013 12:33:55 +0000 (14:33 +0200)]
build_chroot: use APT_INSTALL macro for packet installation
In this way, it is easier to change all the apt calls in the chroot.
In particular, this patch fixes the missing -y option for the installation
of the tools needed to build debian packages.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Bernardo Dal Seno [Thu, 27 Jun 2013 20:44:49 +0000 (22:44 +0200)]
Don't exceed the standard PV size when growing a disk
With exclusive storage, PVs are supposed to be used only up to the
standard PV size.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Bernardo Dal Seno [Thu, 27 Jun 2013 16:37:36 +0000 (18:37 +0200)]
Enable disk growth with exclusive storage
Disk growth is now supported also with exclusive storage, so we let the
operation continue.
Enable disk growth test in qa also with exclusive storage.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Bernardo Dal Seno [Thu, 27 Jun 2013 16:08:15 +0000 (18:08 +0200)]
Limit the PVs used for disk growth with exclusive storage
The exclusive storage flag is passed along in the backend functions for
disk growth. LVM volumes handles the flag by reusing the same PVs, as disks
are constrained by the number of their spindles.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Bernardo Dal Seno [Thu, 27 Jun 2013 15:06:44 +0000 (17:06 +0200)]
Add exclusive_storage to blockdev_grow RPC
The flag is passed to the RPC, but it's still ignored by noded.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Sebastian Gebhard [Fri, 28 Jun 2013 10:35:04 +0000 (12:35 +0200)]
Design doc for openvswitch support
This patch includes the design document for openvswitch support
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Dimitris Aragiorgis [Wed, 26 Jun 2013 14:38:18 +0000 (17:38 +0300)]
Document vif-ganeti usage
In order to be able to use it as vif-script one has to
create a symbolic link inside /etc/xen/scripts pointing
to /usr/lib/ganeti/vif-ganeti and then add is a hypervisor
parameter.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 26 Jun 2013 14:38:16 +0000 (17:38 +0300)]
Refactor NIC configuration scripts
Add new script vif-ganeti that could be used instead of
default vif-bridge in order NIC customization can be more
flexible.
Extract common code from kvm-ifup and insert it in a new
file net-common that is being sourced by kvm-ifup and
vif-ganeti and located under package lib dir (/usr/lib/ganeti/).
Install vif-ganeti properly with make under /usr/lib/ganeti too.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 26 Jun 2013 14:38:15 +0000 (17:38 +0300)]
Make NIC setup more flexible in case of Xen
For every NIC write down a file that contains NIC's
info (MAC, mode, link, network details), that can
be sourced by any vif script. The file location is:
/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>.
This file is created upon cfg file creation and before
starting the instance and can be sourced by an external
script. Before file creation ensure that all parent dirs
exist.
Cleanup NIC dir when removing config files. Upon Xen
configuration file removal, remove NIC dir as well.
Add new hv param vif_script that allows execution of
user defined networking script and overrides the one
xend is configured with in xend-config.sxp
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Klaus Aehlig [Fri, 28 Jun 2013 08:50:45 +0000 (10:50 +0200)]
Update design document for hroller
The requirement to be able to plan full node evacuation came up after
the initial design. So add a section describing the algorithm, if
full evacuation is requested.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Tue, 25 Jun 2013 12:47:56 +0000 (14:47 +0200)]
iallocator: storage processing accepts space_info structure
This is a small refactoring to make the processing of
the storage space information not accept a node info
result, but the more specific space info structure instead.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Tue, 25 Jun 2013 12:30:27 +0000 (14:30 +0200)]
iallocator: get rid of MakeLegacyNodeInfo
So far, the iallocator makes a node info call, converts
the result into a legacy node info structure and then
retrieves various values from it. This patch removes
the call to MakeLegacyNodeInfo completely, by letting the
consumers of the hypervisor info part of the result
retrieve that part from the result of the node info call
directly.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Tue, 25 Jun 2013 12:15:58 +0000 (14:15 +0200)]
iallocator: factor out instance memory summation
The iallocator adjusts the free memory by a calculation
of how much memory is actually currently used by instances
on the node. This calculation is rather complicated and
deserves its own function to increase testability. No
functional changes in this patch otherwise.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Fri, 21 Jun 2013 11:54:06 +0000 (13:54 +0200)]
iallocator: extract storage info directly from node info
So far, the iallocator made a node info RPC call, converted
the result into a legacy node info format and then extracted
the storage information from that. This patch removes the
middle step of this procedure, skipping the legacy node info
usage.
In later patches, we will completely eliminate the need for
the legacy node info in iallocator.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Fri, 14 Jun 2013 13:43:47 +0000 (15:43 +0200)]
Make NodeInfo (hs) accept arbitrary storage types
So far, the Haskell implementation of NodeInfo just
requests storage information about volume groups.
With this patch, storage info for abitrary storage
types can be requested.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 13 Jun 2013 17:44:06 +0000 (19:44 +0200)]
Storage utility functions for Haskell
In order to extend the Haskell version of the NodeInfo
query, we need some utility functions to deal with
disk templates and storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Fri, 21 Jun 2013 09:55:46 +0000 (11:55 +0200)]
iallocator: computation of storage data
This patch extracts the processing of the storage-related
data from the RPC call 'node_info' in a separate function
to increase testability. In addition, this patch contains
unit tests for this function.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Fri, 21 Jun 2013 10:53:05 +0000 (12:53 +0200)]
rpc.py: override default storage type in node info
This patch changes the semantics of dealing with the result
of the RPC call "node_info" when converting them into a
LegacyNodeInfo. A legacy node info, can only hold storage
information about one storage type, which so far was always
LVM storage. With the recent changes in the RCP call
'node_info', a node info call returns storage info of several
storage types and not necessarily with lvm being the first
in the list. In order to be backward-compatible with tools
that assume that LVM storage is the only storage type, we
make sure, that MakeLegacyNode info can be called
requesting LVM info specifically. This is just an interim
solution to not break everything at once. The long-term
goal is to get rid of MakeLegacyNodeInfo completely.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Fri, 21 Jun 2013 09:45:26 +0000 (11:45 +0200)]
iallocator: factor out 'get_attr' method
This patch extracts an inline 'get_attr' method to extract
data from the result of the RPC call 'node_info'. It was so
far an inline function in the middle of a bigger method, which made
the code hard to read.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Fri, 21 Jun 2013 09:19:35 +0000 (11:19 +0200)]
iallocator: prepare RPC call 'node_info'
This patch does a small refactoring in the allocator. It extracts
the preparation and actual call of the 'node_info' call into a
separate function. This is done for better testability. No functional
changes otherwise.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Sun, 23 Jun 2013 12:47:43 +0000 (14:47 +0200)]
Typo in iallocator design doc
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Wed, 26 Jun 2013 13:52:49 +0000 (15:52 +0200)]
Fix lint error
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Wed, 26 Jun 2013 09:37:00 +0000 (11:37 +0200)]
Unset LV name if no LVM disk templates are enabled
If there are no LVM based disk templates enabled in QA, reset the LV
name after the corresponding gnt-cluster modify tests. Otherwise
gnt-cluster verify would complain about the missing volume group.
Also make sure that no disk templates requiring LVM are ever enabled
without actually providing a LV name.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Wed, 26 Jun 2013 06:36:01 +0000 (08:36 +0200)]
Skip drbd-usermode-helper test if DRBD is disabled
Changing the drbd usermode helper in gnt-cluster modify fails if DRBD is
not installed on the nodes. Thus skip this test if DRBD is not in the
set of enabled disk templates.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Tue, 25 Jun 2013 16:54:12 +0000 (18:54 +0200)]
Add route to VCluster network in vcluster-setup
This route is required on the Fedora test machine, as otherwise fping
would not work and hang forever.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Tue, 25 Jun 2013 15:26:13 +0000 (17:26 +0200)]
Replace tempfile by mktemp in QA code
The tempfile command only exists in Debian. In order to run the VCluster
QA also on Fedora, replace its use by mktemp.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Wed, 26 Jun 2013 07:57:25 +0000 (09:57 +0200)]
Merge branch 'stable-2.8' into master
* stable-2.8:
gnt-cluster info (py): add enabled disk templates
Version bump to 2.8.0~beta1
Change version numbers in documentation
Fix issue with python coverage tests
Merge branch 'stable-2.7' into stable-2.8
Disable python test if required libraries are missing
Better specify what packages to install
Improve install guide
Fix typo in the documentation index
Fix typos in the documentation index
Update security document wrt confd access to SSL cert
Add tools for building deb packages to build_chroot
Improve the final message of build_chroot
Make build_chroot self-contained
Conflicts:
configure.ac (trivial)
lib/cmdlib/instance_storage.py (trivial)
The version numbers changed in 'Change version numbers in documentation'
and 'Version bump to 2.8.0~beta1' were adapted to 2.9.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Tue, 25 Jun 2013 07:53:07 +0000 (09:53 +0200)]
Merge branch 'stable-2.7' into stable-2.8
* stable-2.7:
Provide the right fix for the kvm_extra parameter
NEWS and version updates for 2.7.0~rc3
kvm: don't pass whole extra as a single argument
For node queries allow short forms of host names
Provide a special filter for host names
Conflicts:
NEWS: trivial
configure.ac: trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Tue, 25 Jun 2013 06:43:57 +0000 (08:43 +0200)]
Provide the right fix for the kvm_extra parameter
With the previous adjustment the arguments were boxed into one extra
array. This actually makes the parameter work (with the same caveats).
The manpage is also updated with the limitations.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Guido Trotter [Mon, 24 Jun 2013 14:17:47 +0000 (16:17 +0200)]
NEWS and version updates for 2.7.0~rc3
Time for a new release.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Guido Trotter [Mon, 24 Jun 2013 14:45:00 +0000 (16:45 +0200)]
kvm: don't pass whole extra as a single argument
If extra is longer than a single word passing it the way it was passed
won't work. Of course this solution is also only partial, as won't take
into account any form of quoting. :/
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Fri, 21 Jun 2013 13:53:15 +0000 (15:53 +0200)]
For node queries allow short forms of host names
For node queries use the host-name filter instead of the simple
equality-based one.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Fri, 21 Jun 2013 13:51:00 +0000 (15:51 +0200)]
Provide a special filter for host names
For host names, usually short forms are used, e.g., node1 or node1.sub
instead of the full qualified node1.sub.example.com. Therefore comparing
node names only by equality is too restrictive. This patch provides an
alternative makeHostnameFilter to makeSimpleFilter suitable for comparing
host names.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Thu, 20 Jun 2013 09:44:16 +0000 (11:44 +0200)]
gnt-cluster info (py): add enabled disk templates
This fixes issue 485. In the python implementation of
the cluster config info, the enabled_disk_templates
were missing.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Thomas Thrainer [Mon, 24 Jun 2013 09:10:25 +0000 (11:10 +0200)]
Fix node modify QA tests for large cluster
If the node count in a cluster exceeds the master candidate pool size
(which is 10 by default), it's no longer possible to remove the master
candidate role from a node without passing --auto-promote (even with
--force). This behavior is perfectly fine, but the TestModifyNode QA
test didn't account for this.
TestModifyNode is changed such that it first removes the master
candidate role from the tested node with --auto-promote. This will, in
big clusters, promote an additional node to a master candidate which
ensures that the following tests succeed.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Mon, 24 Jun 2013 09:38:26 +0000 (09:38 +0000)]
Version bump to 2.8.0~beta1
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Michele Tartara [Fri, 3 May 2013 09:11:33 +0000 (11:11 +0200)]
Change version numbers in documentation
Some of the documents have the version number of Ganeti. This commit updates
it in preparation for the 2.8 version bump.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Weiwei Jia [Fri, 21 Jun 2013 13:28:39 +0000 (21:28 +0800)]
Doc for GlusterFS Ganeti Support
This patch adds a design document for introducing GlusterFS Support in Ganeti.
Signed-off-by: Weiwei Jia <harryxiyou@gmail.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Fri, 21 Jun 2013 16:15:35 +0000 (16:15 +0000)]
Fix issue with python coverage tests
The recently introduced check for python libraries required only for testing
(commit
27df5b736ef72b3b12c07f32d64dbac95fe7a5ba) was not correct. This
patch fixes the issue.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Thomas Thrainer [Fri, 21 Jun 2013 14:49:51 +0000 (16:49 +0200)]
Merge branch 'stable-2.7' into stable-2.8
* stable-2.7:
Add a warning about the restriction of gnt-node evacuate
Fix permissions of the queue archive dir
Fix daemon QA tests
grow disk: call SetDiskID before blockdev_getsize
Catch DeviceCreationError unhandled exceptions
Conflicts:
lib/cmdlib.py (due to the cmdlib split)
qa/qa_daemon.py (trivial)
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Fri, 21 Jun 2013 13:23:28 +0000 (15:23 +0200)]
Catch DeviceCreationError unhandled exceptions
_CreateBlockDevInner is called twice in TLReplaceDisks. While this
should be fixed, right now this leaves the DeviceCreationError exception
unhandled, which causes a problem due to the fact that this exception
was never supposed to be sent over the wire. For 2.7 we just catch the
exception and convert it to an OpExecError, but we should later do a
more proper cleanup.
This addresses Issue 472.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Fri, 21 Jun 2013 09:19:44 +0000 (11:19 +0200)]
grow disk: call SetDiskID before blockdev_getsize
Missing this call caused blockdev_getsize to be called on the wrong
physical device, and fail.
This fixes Issue 497.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Fri, 21 Jun 2013 09:12:17 +0000 (11:12 +0200)]
Fix daemon QA tests
TestInstanceConsecutiveFailures now leaves the test instance in the same
state it got it.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Fri, 21 Jun 2013 10:51:58 +0000 (12:51 +0200)]
Disable python test if required libraries are missing
Some python libraries are only needed for the tests, but not for running
Ganeti. If those libraries are missing, just disable the Python tests.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Michele Tartara [Fri, 21 Jun 2013 08:49:48 +0000 (08:49 +0000)]
Better specify what packages to install
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Fri, 21 Jun 2013 08:28:14 +0000 (10:28 +0200)]
Improve install guide
Add command for updating the list of cabal packages, before starting
installing them.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Fri, 21 Jun 2013 07:51:39 +0000 (09:51 +0200)]
Fix typo in the documentation index
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Weiwei Jia [Fri, 21 Jun 2013 08:56:32 +0000 (16:56 +0800)]
Fix typos in the documentation index
This patch fixes some typos in the documentation index.
Signed-off-by: Weiwei Jia <harryxiyou@gmail.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Thu, 20 Jun 2013 13:11:57 +0000 (13:11 +0000)]
Update security document wrt confd access to SSL cert
The fact that confd has been given access to the server.pem certificate
needs to be described in the security document.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Guido Trotter [Thu, 20 Jun 2013 16:35:40 +0000 (18:35 +0200)]
Fix permissions of the queue archive dir
The permissions were set to 0740, but of course being a directory the
correct permissions are 0750.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 20 Jun 2013 14:53:31 +0000 (16:53 +0200)]
Add a warning about the restriction of gnt-node evacuate
Even though gnt-node evacuate should be able to evacuate all instances
at once, this currently is not the case. Add a warning about this deficit
for the time being.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 15:03:08 +0000 (17:03 +0200)]
Add gnt-node migrate to QA
Make it possible to optionally run gnt-node migrate during QA.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Fri, 14 Jun 2013 12:56:45 +0000 (14:56 +0200)]
Convert instance indices to UUIDs in configuration
cfgupgrade replaces the occurences of instance names (where they are
used as index) with the instance UUID.
It also supports the inverse replacement, so downgrades are
supported too.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Tue, 11 Jun 2013 15:05:41 +0000 (17:05 +0200)]
Index instances by their UUID
No longer index instances by their name but by their UUID in the cluster
config. This change changes large parts of the code, as the following
adjustments were necessary:
* Change the index key to UUID in the configuration and the
ConfigWriter, including all methods.
* External interfaces (command line interface, IAllocator interface,
hook scripts, etc.) are kept stable.
* Instance UUID's are resolved in ExpandNames and then stored in the
OpCode. This allows to check for instance renames if the OpCode is
reloaded after a cluster restart. This check is currently only done
for single instance parameters.
* Instance locking unfortunately can't use instances UUID as
identifiers. The reasons is that new instances (which have no UUID
yet) have to be locked as well, so the instance name is used.
* Variable names are renamed to follow the following pattern:
- Suffix is 'inst' or 'insts': Variable holds Instance objects
- Suffix is 'name' or 'names': Variable holds Instance names
- Suffix is 'uuid' or 'uuids': Variable holds Instance UUID's
* Tests are adapted.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Mon, 10 Jun 2013 11:59:40 +0000 (13:59 +0200)]
Node-UUID related cleanup
Some variables which hold node UUID's are renamed to reflect their
indent better.
Use cfg.GetNodeName(uuid) instead of cfg.GetNodeInfo(uuid).name where
appropriate.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 20 Jun 2013 12:55:54 +0000 (14:55 +0200)]
Add tools for building deb packages to build_chroot
The chroot only had tools for building and testing Ganeti, but not for
building its Debian packages. They are added by this commit.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 12:15:35 +0000 (14:15 +0200)]
Fix wrong assert
An assert wrongly verified that the node name instead of the node UUID
is in the set of locked nodes. This patch fixes this.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 20 Jun 2013 09:29:25 +0000 (09:29 +0000)]
Improve the final message of build_chroot
Give more precise instructions on the next steps to actually run the
chroot.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 20 Jun 2013 08:57:24 +0000 (08:57 +0000)]
Make build_chroot self-contained
The configuration file are included in build_chroot, and are
automatically created by the script if it is not able to find them
already in the directory pointed by $DATA_DIR.
Note that the configuration file directory is cleaned if it's temporary,
but it is left untouched if it was esplicitly specified by the user.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 11:16:59 +0000 (13:16 +0200)]
Fix testGetHvDefaults testcase
The introduction of the migration_mode parameter to the fake hypervisor
broke the testGetHvDefaults. This patch fixes this by taking the default
parameters of the fake hypervisor into account.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 10:55:40 +0000 (12:55 +0200)]
Fix instance add for missing node specs
Don't pass None to cfg.GetNodeNames. Instead, only resolve the node
names if it's sure that the UUID list is valid.
This fixes issue 495.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Thu, 20 Jun 2013 11:38:28 +0000 (13:38 +0200)]
Merge branch 'stable-2.8' into master
* stable-2.8
In tiered allocation, cut non-promising shrinking tries
Add confd user to masterd group
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Thomas Thrainer [Thu, 20 Jun 2013 07:21:18 +0000 (09:21 +0200)]
Skip Xen monitoring QA for non-Xen clusters
Instead of asserting that the Xen instance status collector fails for
non-Xen clusters just skip the test completely.
Otherwise QA would fail for non-Xen hypervisors if Xen happens to be
installed on the cluster as well.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Wed, 19 Jun 2013 15:45:33 +0000 (17:45 +0200)]
Add support to migration_mode in fake HV
The fake hypervisor pretends to support migration, but did not support
the migration_mode parameter yet. As this parameter is required during
instance migrations, support for it is added in this patch.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Wed, 19 Jun 2013 15:22:33 +0000 (17:22 +0200)]
Adapt VCluster script to current situation
vcluster-setup is adapted to the current Ganeti command line arguments.
Also, a bug for diskless instances in qa_instance is fixed.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Thomas Thrainer [Tue, 18 Jun 2013 14:34:18 +0000 (16:34 +0200)]
Check DRBD status in cluster verify
If the status of DRBD disks is degraded or in a faulty state, issue an
error in cluster verify.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Wed, 19 Jun 2013 14:51:57 +0000 (14:51 +0000)]
Add confd user to masterd group
The server.pem file is 0440 (materd_uid, masterd_gid) and cannot be
accessed by confd.
This patch contains a workaround for 2.8.0~beta1 (adding conf to the
masterd group in the user setup tool, and remembering the users to
actually execute that tool while upgrading).
This partially fixes Issue 292.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Wed, 19 Jun 2013 16:43:21 +0000 (18:43 +0200)]
In tiered allocation, cut non-promising shrinking tries
The heuristics for tiered allocation has been improved in that it
chooses to shrink such a resource next where by shrinking only this
resource a valid allocation can be made, if such a resource exists.
In order to decide whether such a resource exists, all potential
sizes of this resource are verified. We can improve performance by
cutting, as soon a potential allocation of an instance is no longer
blocked on this resource.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Klaus Aehlig [Wed, 19 Jun 2013 10:11:50 +0000 (12:11 +0200)]
Merge branch 'stable-2.8' into master
* stable-2.8
(no changes)
* merged from stable-2.7
More agressively line break man pages
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Wed, 19 Jun 2013 08:24:01 +0000 (08:24 +0000)]
Version bump to 2.9.0~alpha1
Bump the version number to 2.9.0~alpha1.
This also requires updating version numbers in cfgupgrade tool.
Furthermore, a data file for test representing a 2.8 configuration had the
version number set to 2.7 in order to prevent tests failing while the current
branch was artificially kept behind its actual version number. Now that alpha
versions are possible, it can be moved back to 2.8 as it was supposed to be.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Wed, 19 Jun 2013 09:02:53 +0000 (11:02 +0200)]
Merge branch 'stable-2.7' into stable-2.8
* stable-2.7
More agressively line break man pages
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Thomas Thrainer [Wed, 19 Jun 2013 07:00:56 +0000 (09:00 +0200)]
Fix reference to NodeInfo.name
This fixes a bug introduced during the move from node names to node
UUIDs as indexes.
Also, a typo is corrected.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Tue, 18 Jun 2013 14:21:20 +0000 (14:21 +0000)]
Merge branch 'stable-2.8' into 'master'
* stable-2.8
Version bump to 2.8.0~alpha1
Make cfgupgrade idempotent
Update config version number when downgrading
Add test for shrink heuristics over different resources
Improve hspace shrinking strategy
Convenience function for iterating while the result is Ok
Provide witness for the sum-type structure of GenericResult
Disable more version checks for alpha versions
Allow alpha versions not to have an entry in the NEWS file
Add tests for check-news
Introduce support for alpha versions
Improve docstring
Fix bug in Makefile.am
Restructure documentation index page
Add missing parenthesis to description of --machine-readable
Update NEWS file for 2.8
Add a --force option to gnt-cluster modify
Add a force option to the ClusterSetParams Opcode
Document the monitoring daemon in the admin guide
Factor out the API of the monitoring daemon
Add monitoring agent info to the security document
Export device UUIDs to hooks and OS scripts
gnt-cluster info: only parameters for enabled hypervisors
Support RAPI QA on an already existing cluster
Conflicts:
Makefile.am:
Trivial, merge of lists of files.
man/gnt-cluster.rst
One parameter added in stable-2.8, one in master
test/hs/shelltests/htools-hspace.test
Tests added in both branches
tools/cfgupgrade
Some upgrade steps are not required anymore, some have been removed.
src/Ganeti/HTools/Instance.hs
Some types changes in the Haskell code because of the patches related to
Ganeti Dedicated.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Tue, 18 Jun 2013 12:11:15 +0000 (15:11 +0300)]
More agressively line break man pages
Newer versions of pandoc (as, e.g., shipped with Ubuntu 13.04)
more agressively quote, in particular minus signs. That sometimes
leads to unintended line breaks, which are a problem, if the new
line starts with a dot.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Fri, 14 Jun 2013 12:57:49 +0000 (14:57 +0200)]
Version bump to 2.8.0~alpha1
Now that alpha versions are supported, we can bump the version number
for branch stable-2.8 to 2.8.
This also requires updating the cfgupgrade tool.
Thanks to the previous patches, all the other documents can be upgraded
during the alpha lifetime, before switching to beta (that will enable
the version number checks).
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Tue, 18 Jun 2013 07:40:12 +0000 (09:40 +0200)]
Revert "Storage utility functions for Haskell"
This reverts commit
88d27b8aa8adc2e5ced773909f1d40812c5a6ea7.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Tue, 18 Jun 2013 07:39:15 +0000 (09:39 +0200)]
Revert "Make NodeInfo (hs) accept arbitrary storage types"
This reverts commit
e89525a859b2e841c08fce506c0b68b97c7efe61.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Michele Tartara [Tue, 18 Jun 2013 07:33:48 +0000 (07:33 +0000)]
Make cfgupgrade idempotent
One of the unit tests checks whether cfgupgrade's downgrade option is
idempotent. It is, but a version number check made it impossible to
actually downgrade multiple times.
With this change, both the current-version version number and the
target-version version number are accepted by the downgrade tool.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Tue, 18 Jun 2013 07:32:31 +0000 (07:32 +0000)]
Update config version number when downgrading
The downgrade option of the cfgupgrade tool was not changing the version
number in the configuration file of the cluster while performing the
downgrade.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Fri, 14 Jun 2013 13:43:47 +0000 (15:43 +0200)]
Make NodeInfo (hs) accept arbitrary storage types
So far, the Haskell implementation of NodeInfo just
requests storage information about volume groups.
With this patch, storage info for abitrary storage
types can be requested.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Thu, 13 Jun 2013 17:44:06 +0000 (19:44 +0200)]
Storage utility functions for Haskell
In order to extend the Haskell version of the NodeInfo
query, we need some utility functions to deal with
disk templates and storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Helga Velroyen [Fri, 14 Jun 2013 15:05:19 +0000 (17:05 +0200)]
Rename directory 'Block' to 'Storage'
This patch renames the 'Block' directory to 'Storage' in
the Haskell code base. The same rename was done in the
python code base earlier this quarter. We generalize the
name, because we needed a place for general storage
utility functions and storage code for other storage
types like for example file storage.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Klaus Aehlig [Fri, 14 Jun 2013 12:19:01 +0000 (14:19 +0200)]
Add test for shrink heuristics over different resources
In this example, memory prevents adding more than one instance on
the first node. Yet, on the other nodes, 2 instances can be placed
each, if disks are shrunk appropriately. This, however, requires
to courageous decision to shrink disks next, even though more nodes
fail to allocate more due to memory than to disk.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 14 Jun 2013 13:48:47 +0000 (15:48 +0200)]
Improve hspace shrinking strategy
In tired allocation, hspace shrinks that resource of the instance
next, that causes failure on most nodes. While, this is not a bad
strategy in general, it can lead hspace into a dead end if for a large
number of nodes a particular resource blocks any further allocation of
policy compliant instances. So we improve the heuristics in that it
chooses to shrink such a resource next where by shrinking only this
resource a valid allocation can be made, if such a resource
exists. This improves the results in some cases, while still keeping
the computational complexity of the algorithm low.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 14 Jun 2013 15:42:15 +0000 (17:42 +0200)]
Convenience function for iterating while the result is Ok
For a function f :: a -> GenericResult a, iterate it (in the sense of the
monad), until the result is Bad; return the list of values occurred.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 14 Jun 2013 13:47:55 +0000 (15:47 +0200)]
Provide witness for the sum-type structure of GenericResult
GenericResult, while rightfully a type of its own, is isomorphic
to Either. So, also provide the case analysis function (i.e., the
universal arrow out of the sum).
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Thu, 13 Jun 2013 14:56:30 +0000 (14:56 +0000)]
Disable more version checks for alpha versions
Alpha versions should not check for version numbers in READMEs
and documentation.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 13 Jun 2013 14:15:15 +0000 (14:15 +0000)]
Allow alpha versions not to have an entry in the NEWS file
Alpha versions are still under heavy development, and can therefore not have an
entry in the NEWS file yet. The entry will have to be added before switching to
the first beta version.
Partially fixes Issue 448
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Michele Tartara [Thu, 13 Jun 2013 12:09:40 +0000 (14:09 +0200)]
Add tests for check-news
Unit tests verifying the behaviour of the check-news script are added.
This required adding a new make target, "autotools-check" directly called from
"commit-check", because the usual tests are executed in the build directory,
whereas these, being relative to the autotools, cannot be executed there because
the files of the autotools/ directory are not copied do not belong in the build
directory.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>