Thomas Thrainer [Mon, 18 Nov 2013 10:40:09 +0000 (11:40 +0100)]
Be paranoid about existing keys in cfgupgrade
The cfgupgrade test assumes a very minimal configuration for some of its
tests. This configuration does not contain some of the fields which were
assumed to be present in cfgupgrade. This patch tests for the existence
of those fields before trying to access them.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Thomas Thrainer [Mon, 18 Nov 2013 08:42:12 +0000 (09:42 +0100)]
Downgrade from 2.10 to 2.9 correctly
Add handling for Open vSwitch related configuration parameters as well
as for the "cpuid" and "soundhw" hypervisor parameters.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Thomas Thrainer [Fri, 15 Nov 2013 10:36:23 +0000 (11:36 +0100)]
Merge branch 'stable-2.9' into stable-2.10
Merge branch 'stable-2.9' into stable-2.10
* stable-2.9
Bump revision for 2.9.1
Update NEWS and schedule release for 2.9.1
Fix retrieval of xen command in class method
Fix docstring for ganeti.storage.filestorage_unittest.py
Undo revision bump
Postpone release of 2.9.1
Revision bump for 2.9.1
Update NEWS for 2.9.1 release
Readd nodes as online
When verifying BRBD version, ignore missing values
* stable-2.8
Version bump for 2.8.2
Update NEWS file for 2.8.2 release
DRBD: ensure peers are UpToDate for dual-primary
Conflicts:
NEWS: trivial
configure.ac: ignore version bump
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Thomas Thrainer [Wed, 6 Nov 2013 09:36:37 +0000 (10:36 +0100)]
Don't allow optional node parameters
Ganeti does not support optional fields in parameters
(hypervisor-params, disk-params, etc.). OpenVSwitch related node
parameters were the exception to this rule, which caused numerous
problems related to import/export and (de-)serialization.
The reason for making those parameters optional in the first place was to
disallow them when OpenVSwitch is not used. This was not consistent with
other parts of Ganeti, where we allow parameters to be set even though they
are not actively used.
This patch makes all node parameters mandatory and provides sensible
defaults for them. Checks which make sure that certain parameters are
not set in some cases were removed, and the tests adapted. Also, the
inheritance logic from cluster -> node group -> node was implemented, as
it was missing previously.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Thomas Thrainer [Wed, 6 Nov 2013 09:20:59 +0000 (10:20 +0100)]
Move OVS node parameters to the right place
OpenVSwitch node parameters were passed as separate command line flags.
This was not only inconsistent with all other parameters
(hypervisor-params, disk-params, ...), but also with the documented
behavior in the man pages.
Additionally, the special handling of the OVS related parameters
disabled logic in LUNodeAdd, as ndparams was never None any more.
This patch fixes those problems and treats OVS related parameters like
all other node parameters.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Thomas Thrainer [Wed, 6 Nov 2013 09:12:06 +0000 (10:12 +0100)]
Make NIC VLAN queryable
Add an additional query field which makes it possible to query for
VLANs of instance NICs.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Thomas Thrainer [Wed, 6 Nov 2013 09:09:07 +0000 (10:09 +0100)]
Pass VLAN parameter correctly in moves
Fix the assignment of a wrong value to the VLAN and name parameters of
NICs during instance moves.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Thomas Thrainer [Wed, 6 Nov 2013 09:07:17 +0000 (10:07 +0100)]
Use constant instead of raw string
Use the appropriate constant instead of a raw string.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Jose A. Lopes [Thu, 14 Nov 2013 08:32:06 +0000 (09:32 +0100)]
Merge 'hs2py-constants' into 'hs2py'
Merge code of 'hs2py-constants' into 'hs2py', which requires adding
flag '--constants', and merge target 'src/hs2py-constants' into
'src/hs2py' in 'Makefile.am', also updating dependencies and
variables.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Jose A. Lopes [Thu, 14 Nov 2013 08:34:26 +0000 (09:34 +0100)]
Add flag '--opcodes' to 'hs2py'
This is a transitional step before merging 'hs2py-constants' and
'hs2py'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Jose A. Lopes [Tue, 12 Nov 2013 16:10:56 +0000 (17:10 +0100)]
Sort design docs alphabetically
Sort design docs alphabetically in Makefile.am and fix whitespace.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Jose A. Lopes [Tue, 12 Nov 2013 16:10:02 +0000 (17:10 +0100)]
Missing design docs from distribution
Fixes issue 609.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Klaus Aehlig [Tue, 12 Nov 2013 15:44:22 +0000 (16:44 +0100)]
Bump revision for 2.9.1
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Klaus Aehlig [Tue, 12 Nov 2013 15:43:46 +0000 (16:43 +0100)]
Update NEWS and schedule release for 2.9.1
Now that issue 608 fixed, schedule a new release date
for 2.9.1.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Helga Velroyen [Tue, 12 Nov 2013 14:36:42 +0000 (15:36 +0100)]
Fixing rapi documentation wrt to storage types
This fixes a FIXME in the rapi documentation. The list of
storage types that can be queries for space information was
checked against the complete storage type set and not
against the set of storage types that actually support
storage reporting.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Helga Velroyen [Tue, 12 Nov 2013 13:45:53 +0000 (14:45 +0100)]
Fix retrieval of xen command in class method
This patch fixes issue 608. When introducing the
configurability of the xen toolstack in commit
8ef418bb92, the hypervisor api was accidentally changed
in a way that let to this error in KVM.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Santi Raffa [Mon, 11 Nov 2013 17:44:29 +0000 (18:44 +0100)]
Fix docstring for ganeti.storage.filestorage_unittest.py
Signed-off-by: Santi Raffa <rsanti@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Fri, 8 Nov 2013 09:07:04 +0000 (10:07 +0100)]
Move Haskell constants to proper module
Move Haskell constants from module 'Ganeti.HsConstants', which was a
transitional module part of the Haskell to Python constant generation
infrastructure, to module 'Ganeti.Constants'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Thu, 7 Nov 2013 10:02:42 +0000 (11:02 +0100)]
Tear down Py2Hs constant infrastructure
Tear down Python to Haskell constant conversion infrastructure, which
includes eliminating the autotool 'convert-constants' and the Haskell
module, namely 'Ganeti.PyConstants', which held the converted
contants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Thu, 7 Nov 2013 09:53:15 +0000 (10:53 +0100)]
Hs2Py constants: additional module jstore
Add constants from additional modules ('ganeti.jstore') to the Haskell
to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Thu, 7 Nov 2013 09:50:50 +0000 (10:50 +0100)]
Hs2Py constants: additional module errors
Add constants from module 'ganeti.errors' to the Haskell to Python
constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Thu, 7 Nov 2013 09:48:12 +0000 (10:48 +0100)]
Hs2Py constants: additional module qlang
Add constants from additional modules ('ganeti.qlang') to the Haskell
to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Wed, 6 Nov 2013 10:34:44 +0000 (11:34 +0100)]
Hs2Py constants: additional module luxi
Add constants from additional modules ('ganeti.luxi') to the Haskell
to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Wed, 6 Nov 2013 10:31:06 +0000 (11:31 +0100)]
Remove module 'ganeti.constants' from 'convert-constants'
Given that there are no more constants in 'ganeti.constants' to
convert to Haskell, this module import and conversion logic can be
removed from the autotool 'convert-constants'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Thu, 7 Nov 2013 08:56:34 +0000 (09:56 +0100)]
Fix reference to vcs version in query server
Fix reference to vcs version in query server to take its value from
the Haskell constant in 'Ganeti.Version' instead of using the constant
generated from Python.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 5 Nov 2013 19:33:45 +0000 (20:33 +0100)]
Eliminate unused constants
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 5 Nov 2013 19:32:28 +0000 (20:32 +0100)]
Use configure constants instead of generated
Replace uses of the generated 'AF_INET*' constants with the constants
in Haskell's 'AutoConf'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Fri, 8 Nov 2013 09:04:26 +0000 (10:04 +0100)]
Hs2Py constants: add 'UUID_REGEX'
Add constant 'UUID_REGEX' to the Haskell to Python constant
generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 5 Nov 2013 19:30:26 +0000 (20:30 +0100)]
Add 'AF_INET*' constants to Haskell's 'AutoConf'
Add 'AF_INET*' constants to the Haskell 'AutoConf' module, which
requires extending the 'm4' invocation with additional substitutions,
in the 'Makefile' target for 'src/AutoConf.hs'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 5 Nov 2013 19:28:08 +0000 (20:28 +0100)]
Python autotool to print socket constants
Add Python script to autotools which prints socket related constants.
These constants are a transitional solution for eliminating the Python
to Haskell constant generation given that the final solution will
require a change in the configuration and, therefore, postponed to
2.11.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Fri, 8 Nov 2013 16:06:09 +0000 (17:06 +0100)]
Undo revision bump
Before releasing 2.9.2, we still have issue 608 to fix; if
no release date is set, we still have to be at the lower
version.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Fri, 8 Nov 2013 15:08:55 +0000 (16:08 +0100)]
Postpone release of 2.9.1
...until issue 608 is fixed.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 7 Nov 2013 15:21:46 +0000 (16:21 +0100)]
Revision bump for 2.9.1
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 7 Nov 2013 15:20:23 +0000 (16:20 +0100)]
Update NEWS for 2.9.1 release
Add a section in the file for the new upcoming release. Besides
the fix of the DRBD race condition inherited from 2.8.2, this
also fixes handling and readding of offline nodes.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Thu, 7 Nov 2013 14:19:22 +0000 (15:19 +0100)]
Readd nodes as online
Patch
d0d7d7cf accidentally removed the offline-flag reset
when readding a node. Readd it.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Klaus Aehlig [Thu, 7 Nov 2013 13:42:10 +0000 (14:42 +0100)]
When verifying BRBD version, ignore missing values
When comparing for consistency of the DRBD versions, some
versions might not be available via RPC, typically, if the
node is offline. In this case, leave these nodes out of the
test, instead of failing with an internal python error.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Klaus Aehlig [Thu, 7 Nov 2013 12:37:08 +0000 (13:37 +0100)]
Merge branch 'stable-2.8' into stable-2.9
* stable-2.8
Version bump for 2.8.2
Update NEWS file for 2.8.2 release
DRBD: ensure peers are UpToDate for dual-primary
Conflicts:
NEWS: trivial
configure.ac: ignore version bump on stable-2.8
lib/bdev.py: manually apply the part of commit
73e15b5e that applies to lib/bdev.py to
lib/storage/drbd_info.py, and keep lib/bdev.py
removed
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Wed, 6 Nov 2013 12:26:24 +0000 (12:26 +0000)]
Version bump for 2.8.2
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Michele Tartara [Wed, 6 Nov 2013 12:25:16 +0000 (12:25 +0000)]
Update NEWS file for 2.8.2 release
Add a section in the file for the new upcoming release.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Petr Pudlak [Tue, 5 Nov 2013 14:32:20 +0000 (15:32 +0100)]
When loading configuration fails, include the reason
Before the message why a failure happened (like a parsing error) was lost.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Apollon Oikonomopoulos [Tue, 5 Nov 2013 14:30:45 +0000 (16:30 +0200)]
DRBD: ensure peers are UpToDate for dual-primary
DrbdAttachNet supports both, normal primary/secondary node operation, and
(during live migration) dual-primary operation. When resources are newly
attached, we poll until we find all of them in connected or syncing operation.
Although aggressive, this is enough for primary/secondary operation, because
the primary/secondary role is not changed from within DrbdAttachNet. However,
in the dual-primary ("multimaster") case, both peers are subsequently upgraded
to the primary role. If - for unspecified reasons - both disks are not
UpToDate, then a resync may be triggered after both peers have switched to
primary, causing the resource to disconnect:
kernel: [1465514.164009] block drbd2: I shall become SyncTarget, but I am
primary!
kernel: [1465514.171562] block drbd2: ASSERT( os.conn == C_WF_REPORT_PARAMS )
in /build/linux-rrsxby/linux-3.2.51/drivers/block/drbd/drbd_receiver.c:3245
This seems to be extremely racey and is possibly triggered by some underlying
network issues (e.g. high latency), but it has been observed in the wild. By
logging the DRBD resource state in the old secondary, we managed to see a
resource getting promoted to primary while it was:
WFSyncUUID Secondary/Primary Outdated/UpToDate
We fix this by explicitly waiting for "Connected" cstate and
"UpToDate/UpToDate" disks, as advised in [1]:
"For this purpose and scenario,
you only want to promote once you are Connected UpToDate/UpToDate."
[1] http://lists.linbit.com/pipermail/drbd-user/2013-July/020173.html
Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Tue, 5 Nov 2013 12:47:30 +0000 (13:47 +0100)]
Merge branch 'stable-2.9' into stable-2.10
* stable-2.9
Revision bump for 2.9.0
Schedule release of 2.9.0
* stable-2.8
Improve error message for replace-disks
Conflicts:
NEWS: trivial
configure.ac: drop suffix bump of stable-2.9
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Mon, 4 Nov 2013 09:40:13 +0000 (10:40 +0100)]
Revision bump for 2.9.0
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Mon, 4 Nov 2013 09:39:36 +0000 (10:39 +0100)]
Schedule release of 2.9.0
...and mention the last change pulled in from stable-2.8.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Klaus Aehlig [Mon, 4 Nov 2013 15:36:02 +0000 (16:36 +0100)]
Merge branch 'stable-2.8' into stable-2.9
* stable-2.8
Improve error message for replace-disks
Conflicts:
lib/cmdlib/instance_storage.py
Resolved by manually applying the node name to uuid
transition on the version of stable-2.9.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Mon, 4 Nov 2013 15:20:07 +0000 (15:20 +0000)]
Improve error message for replace-disks
In some conditions, replace-disks will fail if the disks are not properly
activated. Improve the error message suggesting to run activate-disks before
executing replace-disks.
Fixes Issue 606.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Wed, 30 Oct 2013 09:25:42 +0000 (10:25 +0100)]
Fix documentation
Fix documentation in constants containing values in seconds.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Jose A. Lopes [Wed, 30 Oct 2013 09:15:57 +0000 (10:15 +0100)]
Replace all constant definitions with re-exports
Remove all constant definitions in 'lib/constants.py' and re-export
the module 'lib/_constants.py', which contains the generated
constants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Jose A. Lopes [Mon, 28 Oct 2013 18:01:48 +0000 (19:01 +0100)]
Prepare constants for automatic reexport
* separate constants and non-constants in 'lib/constants.py' to make
it simple to replace their re-export definitions with a single
re-export statement at the begining of the module
* fix access 'CONFD_REQFIELD_*' related constants in
'tools/confd-client' to convert them to 'String', rather then doing
it in 'lib/constants.py' in order to allow the replacement of the
re-export definitions with a single re-export statement
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Jose A. Lopes [Mon, 28 Oct 2013 13:45:40 +0000 (14:45 +0100)]
Hs2Py constants: 'hvsParameterTypes' and 'hvsParameters'
Add constants 'hvsParameters' and 'hvsParameterTypes' to the Haskell
to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Petr Pudlak [Thu, 31 Oct 2013 14:35:00 +0000 (15:35 +0100)]
Fix indentation that triggers PEP8 error
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Dimitris Aragiorgis [Wed, 30 Oct 2013 16:56:01 +0000 (18:56 +0200)]
Check if hotplug is supported in CheckPrereq
Introduce new RPC hotplug_supported that invokes the corresponding
hypervisor's method which checks if hotplug is generally
supported. Call this RPC early in CheckPrereq() and abort if
hotplug is not supported. Currently only KVM hypervisor with
qemu versions > 1.0. Please note that this check is unaware of
target device and hotplug action. This latter check is still
done during hotplug_device() RPC.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Wed, 30 Oct 2013 13:07:52 +0000 (14:07 +0100)]
Merge branch 'stable-2.9' into stable-2.10
* stable-2.9
(no changes)
* stable-2.8
Add all dependencies for confd as test dependencies
Add snap-server to the test-relevenat packages
Placate warnings on ganeti.outils_unittest.py
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Klaus Aehlig [Wed, 30 Oct 2013 12:33:58 +0000 (13:33 +0100)]
Merge branch 'stable-2.8' into stable-2.9
* stable-2.8
Add all dependencies for confd as test dependencies
Add snap-server to the test-relevenat packages
Placate warnings on ganeti.outils_unittest.py
Conflicts:
configure.ac: take both additions (and fix)
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Fri, 25 Oct 2013 15:14:17 +0000 (17:14 +0200)]
Remove FIXME + update man pages wrt iallocator
This patch removes a FIXME regarding the retrieval
of the disk template for computation of the cluster
information for the iallocator. After examining the
code, I figured out that it is actually a quite
robust way to just check if the disk template is
available, in particular with respect to extensions
of the iallocator.
In this context, I also updated the man pages of
the operations that use the iallocator, because so
far it was not well documented which disk template
is considered when more than one instance was
allocated. The situation is not ideal (the default
disk template is used), but it is still better than
in 2.9. where only LVM storage was considered.
With the implementation of storage pools, it would
make sense to extend the iallocator protocol to
include storage information for more than one
template.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Wed, 30 Oct 2013 10:13:48 +0000 (11:13 +0100)]
Add all dependencies for confd as test dependencies
Since our tests pull in confd as a dependency, all build dependencies
for confd are also necessary to run the tests.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Sebastian Gebhard [Mon, 14 Oct 2013 13:50:42 +0000 (13:50 +0000)]
Don't check for master netdev when using OVS
This patch disables checking if the given master_netdev exists during cluster
init.
When OVS is used, the needed Open vSwitch will be created later on (with
OpClusterPostInit) and is thus not available at this point of time.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Sebastian Gebhard [Mon, 14 Oct 2013 13:54:04 +0000 (13:54 +0000)]
Fix TypeError in backend/ConfigureOVS
In case of running into a failure during Open vSwitch creation, the error is
not given, but a TypeError is thrown. This is because the number of arguments
for the format string was not equal to the place holders.
This fixes the issue.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Sebastian Gebhard [Mon, 14 Oct 2013 13:54:04 +0000 (13:54 +0000)]
Create Open vSwitch on Master during Cluster init
Currently, Open vSwitch is only created on additional nodes, not on the master
itself.
This fixes the issue and creates and configures an Open vSwitch on the master
node during cluster init.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Dimitris Aragiorgis [Fri, 25 Oct 2013 15:43:06 +0000 (18:43 +0300)]
SimpleRetry on BlockDev.Remove()
Sometimes, upon disk removal, corresponding file descriptors
are kept briefly open by various processes (hypervisor, blkid, etc.).
With this patch, we retry several times before raising the appropriate
error, thus making disk removal more robust against those corner cases.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Klaus Aehlig [Tue, 29 Oct 2013 15:09:14 +0000 (16:09 +0100)]
Add snap-server to the test-relevenat packages
While snap-server is only needed for the optional monitoring daemon,
some tests, notably those testing these optional features, still depend
on it. So, if snap-server is missing, the Haskell tests should not be
run, as they cannot even be build.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Petr Pudlak [Mon, 28 Oct 2013 14:53:28 +0000 (15:53 +0100)]
Show the key in "'None' is not a valid Maybe value"
Currently the error message doesn't say which key is affected, which
makes it kind of useless.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Helga Velroyen [Fri, 25 Oct 2013 15:30:01 +0000 (17:30 +0200)]
Fix usage of xm/xl for gnt-instance modify --new-primary
This fixes issue 519. When calling 'gnt-instance modify
--new-primary', an exception was raised, because a
RPC call was not called with the right hypervisor
parameters which caused the backend function to crash,
because it could not determine which Xen toolstack (xm/xl)
was configured to be used.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Thu, 24 Oct 2013 08:51:10 +0000 (10:51 +0200)]
Hs2Py constants: add 'hvcDefaults'
Add 'hvcDefaults' to the Haskell to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Mon, 28 Oct 2013 09:11:48 +0000 (10:11 +0100)]
Fix documentation
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 15 Oct 2013 16:22:17 +0000 (18:22 +0200)]
Hs2Py constants: add 'diskDtDefaults'
* add constant 'diskDtDefaults' to the Haskell to Python constant
generation
* remove 2 local variables which were used only by 'DISK_DT_DEFAULTS'
and are, therefore, no longer necessary
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Mon, 28 Oct 2013 08:59:29 +0000 (09:59 +0100)]
Fix documentation
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 15 Oct 2013 15:32:55 +0000 (17:32 +0200)]
Hs2Py constants: add 'value*' and 'niccDefaults'
* add group of constants 'value*' and constant 'niccDefaults' to the
Haskell to Python constant generation
* add 'PythonNone' to represent the 'None' is Haskell to Python
generated code
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 15 Oct 2013 15:21:28 +0000 (17:21 +0200)]
Hs2Py constants: add 'ipolicyDefaults'
Add constant 'ipolicyDefaults' to the Haskell to Python constant
generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 15 Oct 2013 12:28:44 +0000 (14:28 +0200)]
Hs2Py constants: add 'mond' latest version
Add 'mond' latest version constant to the Haskell to Python constant
generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 15 Oct 2013 12:20:18 +0000 (14:20 +0200)]
Hs2Py constants: add 'sshDaemonKeyfiles'
Add constant 'sshDaemonKeyfiles' to the Haskell to Python constant
generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 15 Oct 2013 11:33:33 +0000 (13:33 +0200)]
Hs2Py constants: add 'diskLdDefaults'
Add constant 'diskLdDefaults' to the Haskell to Python constant
generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Tue, 15 Oct 2013 09:42:01 +0000 (11:42 +0200)]
Hs2Py constants: add 'ndcDefaults'
Add constant 'ndcDefaults' to the Haskell to Python constant
generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Mon, 14 Oct 2013 08:39:23 +0000 (10:39 +0200)]
Hs2Py constants: add 'becDefaults'
Add constant 'becDefaults' to the Haskell to Python constant
generation. An instance of 'PyValue' is added to the type 'PyValueEx'
in order to allow polymorphic types, namely, 'Map's.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Mon, 14 Oct 2013 08:31:00 +0000 (10:31 +0200)]
Remove 'Show' from 'PyValue' class
Remove 'Show' constraint from the 'PyValue' class which, even though
it allowed to simplify the definition of instances of this class for
primitive types, it made it more complicated to define instances for
other types.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Mon, 14 Oct 2013 08:04:00 +0000 (10:04 +0200)]
Hs2Py constants: add 'drbdValidBarrierOpt'
Add constant 'drbdValidBarrierOpt' to the Haskell to Python constant
generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Mon, 14 Oct 2013 07:56:53 +0000 (09:56 +0200)]
Hs2Py constants: add 'dtsLvm'
Add constant 'dtsLvm' to the Haskell to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Fri, 11 Oct 2013 15:03:20 +0000 (17:03 +0200)]
Hs2Py constants: remove useless constants
Remove unused constants from Python because they were only used by
constants which are currently generated, and move Python comment to
the Haskell side.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Fri, 11 Oct 2013 14:58:47 +0000 (16:58 +0200)]
Hs2Py constants: add 'daemonsExtraLogbase'
Add constant 'daemonsExtraLogbase' to the Haskell to Python constant
generation and update Python reference.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Jose A. Lopes [Fri, 11 Oct 2013 14:49:58 +0000 (16:49 +0200)]
Hs2Py constants: remove unused constant
Remove unused Python constant whose use was replaced by other
constants generated from Haskell.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Thomas Thrainer [Fri, 25 Oct 2013 11:50:13 +0000 (13:50 +0200)]
Fix default value for reason trail
The default value for the reason trail field used to be ht.EmptyList in
2.9 but was incorrectly changed to None in 2.10. This patch corrects
this error which fixes a bug in batch creation of instances.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Dimitris Aragiorgis [Thu, 17 Oct 2013 12:25:12 +0000 (15:25 +0300)]
drive_del after device_del in disk hot-remove
This is needed so that qemu process really releases the
corresponding file descriptor and thus we can dmsetup/lvremove
the device without errors.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Sat, 12 Oct 2013 18:25:37 +0000 (21:25 +0300)]
Minor changes regarding hotplug support
* Mention restrictions in NEWS
- RBD userspace access mode
- In case of a downgrade instances should suffer a reboot to be migrateable
* Bypass interactive verification in NIC modifications via --force option
- Mention it in man page
* Print "modifications take place after restart" message no matter if
--hotplug option is passed. Change cmdlib to append hotplug status info
(if any) to the final result printed out to the user
* Change hotplug option description
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Fri, 11 Oct 2013 00:53:23 +0000 (03:53 +0300)]
Fix in RPC
Use _SingleDiskDictDP() instead of _ObjectDict() for serializing
a disk.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Fri, 11 Oct 2013 00:52:09 +0000 (03:52 +0300)]
block_devices -> kvm_disks renaming
and a couple more styple fixes.
block_devices are misleading so rename them to kvm_disk (just like kvm_nics)
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Fri, 11 Oct 2013 00:50:56 +0000 (03:50 +0300)]
Make hotplug related method raise HotplugError
Hotplug is not supported by default and thus we raise
HotplugError() for hotplug related methods.
If a hypervisor wants to support it, we must override VerifyHotplugSupport()
and implement the Hot* method.
Backend invokes VerifyHotplugSupport() and raises RPCFail if an
exception is raised.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Thu, 10 Oct 2013 13:00:44 +0000 (16:00 +0300)]
Some fixes in _GenerateKVMBlockDevicesOptions()
* Add docstring.
* fix missed usecase of boot_disk=False if kernel_path is given.
* Use devlist arg in GenerateKVMBlockDeviceOptions()
This is previously calculated in _ExecuteKVMRuntime()
so there is no need to rerun the same command.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:40 +0000 (17:45 +0300)]
Add unittests for Hotplug support
Try base InstanceSetParams tests that modify instance's
devices with hotplug=True.
Add some unittests for kvm hypervisor regarding runtime file
manipulation. Based on a sample runtime file search for
specific disk and nic entries based on their UUID.
Add unittest for kvm device id generation.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:39 +0000 (17:45 +0300)]
Extend qa to test hotplug support
Only if default hypervisor is KVM try all hotplug related actions.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:37 +0000 (17:45 +0300)]
Hotplug: Update manpages and NEWS
Add --hotplug entry in gnt-instance manpage.
Add NEWS entry for hotplug support.
In both cases mention that hotplug is only supported for
the KVM Hypervisor version >= 1.0.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:36 +0000 (17:45 +0300)]
Hotplug: client support
Add --hotplug option. Only used in OpInstanceSetParams.
If this is omitted, modifications become effective after reboot.
Ask user confirmation in case NIC modify + hotplug because it will
be done via removing old NIC (and the corresponding tap) and adding
a new one in the same PCI slot.
Corresponding mods in haskell opcode definitions.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:35 +0000 (17:45 +0300)]
Hotplug: cmdlib support
Hotplugging is done by functions invoked by ApplyContainerMods(). In
order hotplugging to take place the --hotplug option must be passed
otherwise the modifications will take place after reboot.
NIC hotplug supports add, remove and modify. The modify is done by removing
the existing NIC and adding a new one in the same pci slot.
Disk hotplug supports add and remove. Before hotplugging a Disk it
must be assembled. Use blockdev_assemble RPC, get the device link_name
and then call hotplug RPC command.
In order to remove a disk (with blockdev_remove) it must be shutdown. So
after unplugging the disk ShutdownDiskInstanceDisks() must be invoked.
For both device types we use the generic RPC call_hotplug_device.
Also adapt unit tests to follow the change.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:34 +0000 (17:45 +0300)]
Return link_name in blockdev_assemble rpc
Until now this RPC returned only dev_path. Since we use it in
hotplug we have to know the simlink of the device so that we
pass it to the corresponding hypervisor command and include it
in block_devices entry in runtime files.
Fix unittest to be aware of changed rpc
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:33 +0000 (17:45 +0300)]
Hotplug: rpc support
Introduce new RPC that eventually invoke hypervisor specific
hotplug functions. In order to be generic it has the following
arguments: device type, action, device, extra info, seq.
Device type can be NIC or DISK, action can be ADD, REMOVE,
device is the NIC or Disk object, extra info is used by Disk
hotplug to point the device path and seq is the device index
(from the master perspective)
Add HotplugSupported() in all Hypervisors
Only KVM hypervisor supports this method. The other hypervisors
raise HypervisorError.
Before trying hotplug in backend layer invoke hyper.HotplugSupported
and abort RPC in case it is not implemented or for some other reason
not supported (disk+chroot, nic without fdsend, etc).
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:32 +0000 (17:45 +0300)]
Introduce hotplug methods (hypervisor inteface)
4 new methods: HotAddDevice, HotDelDevice, HotModDevice, HotplugSupported
All these are the hypervisor interface with the backend.
In general they read current runtime info, do the actual hotplug
and update the runtime file.
NIC hot-add:
- open a tap and get its file descriptor.
- pass fd with SCM rights (using python-fdsend) via monitor socket
- create netdev and device with id=kvm_devid and proper pci info
Disk hot-add:
- create drive with id=kvm_devid
- create device with id=kvm_devid and corresponding pci info
Use MonitorSocket in _PassTapFd in order to connect to monitor socket
and pass the tap's file descriptor using fdsend.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:31 +0000 (17:45 +0300)]
Extract socket related code from QmpMonitor class
Separate unix socket related code from QmpMonitor class and
make the latter extend the newly introduced one: MonitorSocket
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:30 +0000 (17:45 +0300)]
Helper methods that check for hotplug support
Hotplug is currently *not* supported for:
- KVM < 1.0
- existing devices in the cluster
- python-fdsend module is not installed (NIC hotplug)
- chroot (Disk hotplug)
- security mode other than None (Disk hotplug)
For the above reasons raise HotplugError exception and
let masterd handle it.
TODO: modify CallHotplugCommand() so that it parses
monitor output and reports whether the command
succeeded or not.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:29 +0000 (17:45 +0300)]
Use -device with pci info if possible
Extend kvm_cmd with -device option for the case of paravirual
Disks and NICs.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:28 +0000 (17:45 +0300)]
Helper methods for PCI slots and device ids
Device naming:
QEMU monitor expects devices to be uniquely named. Device ids derive
from the following function:
kvm_devid = <device_type>-<part of uuid>-pci-<pci_slot>
Device ids must be reproduce-able when we want to remove them.
For that reason we store the pci slot inside the runtime file and
in case we want to remove a device we obtain its pci slot by
parsing the corresponding runtime enrty and matching the device
by its uuid.
Finding the PCI slot:
For newly added devices Hypervisor parses existing PCI allocations
(via _GetFreePCISlot() and eventually ``info pci`` monitor
command) and decides the PCI slot to plug in the device. During
instance startup hypervisor invokes _UpdatePCISlots() for every
device of the instance. Initial PCI reservations derive from KVM
default setup, that allocates 4 slots for devices other than disks
and NICs.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>
Dimitris Aragiorgis [Wed, 9 Oct 2013 14:45:27 +0000 (17:45 +0300)]
Prepare block_devices runtime entry
With this patch we add another entry in runtime files along with
kvm_cmd, kvm_nics, and hvparams. block_devices that used to be
encapsulated inside kvm_cmd, live now separately just like nics do
but in tupples of (L{objects.Disk}, dev_path).
Introduce also _GetExistingDeviceInfo() helper function to search
for runtime entries. This is going to be useful later in hotplug
methods.
Define new exception: HotplugError.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>