Michael Hanselmann [Mon, 4 Feb 2013 13:38:34 +0000 (14:38 +0100)]
Extract container converters
“objects.ConfigObject” contains two useful functions for working with
containers of serialized objects, “_ContainerToDicts” and
“_ContainerFromDicts”. This patch separates those functions and moves
them into “objectutils” as they'll be useful for converting parts of the
QA configuration to objects. Unittests are added to cover the parts not
already tested through other code paths.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 4 Feb 2013 14:28:51 +0000 (15:28 +0100)]
hv_xen: Fix epydoc error
“utils.RunCmd” is re-exported from “utils.process.RunCmd”. Epydoc
doesn't fully understand this, so we have to refer to the original.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 4 Feb 2013 13:00:40 +0000 (14:00 +0100)]
objects: Improve handling of TCP/UDP port pool
- Handle de-serialization correctly when pool is not defined
- Serialize to empty list when the attribute is None (this should never
happen in reality as the attribute is always set when de-serializing)
- Add tests
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 1 Feb 2013 16:21:55 +0000 (17:21 +0100)]
Merge branch 'devel-2.7'
* devel-2.7: (24 commits)
Fix typo in ganeti-noded man page
QA: Run instance tests with different cluster configurations
QA: Run cluster-verify after instance tests
QA: Refactored launching of the various instance tests
QA: Use lists of nodes as argument to instance tests
QA: Instance tests check the configuration before running
QA: Added functions to allocate/free N nodes at once
QA: Added exclusive_storage to qa_config
QA: Better initialization of a global dictionary
QA: Added disk template to config
QA cleanup: Removed instance-disk-failure test
QA fix: TestInstanceExportWithRemove() updates configuration
QA: Cluster-verify reports shared PVs with exclusive storage
QA: Added constants for LVM volumes
QA: Templates incompatible with exclusive storage reported
QA: Uniformity check for exclusive_storage in cluster-verify
QA: Test for basic features of exclusive storage
QA: Added function to read cluster configuration fields
QA: Added function to check cluster-verify result
Document better gnt-network
...
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michele Tartara [Fri, 1 Feb 2013 13:46:04 +0000 (14:46 +0100)]
Fix typo in ganeti-noded man page
Add a missing "in".
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Mon, 28 Jan 2013 16:26:40 +0000 (17:26 +0100)]
QA: Run instance tests with different cluster configurations
Instance tests are run with or without the exclusive-storage flag set. More
configurations can be added easily.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Sat, 26 Jan 2013 01:40:03 +0000 (02:40 +0100)]
QA: Run cluster-verify after instance tests
We want to make sure that instance operations don't leave the cluster in a
bad state. The "instance-remove-drbd-offline" test leaves some debris
behind, so it's been moved to the last position.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Wed, 23 Jan 2013 16:23:48 +0000 (17:23 +0100)]
QA: Refactored launching of the various instance tests
The code launching instance tests for "plain" and "drbd" templates has been
unified. In this way it's easier to add more templates, and to re-use the same
tests with different configurations.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Fri, 25 Jan 2013 20:39:38 +0000 (21:39 +0100)]
QA: Use lists of nodes as argument to instance tests
Some instance test functions took two node arguments, some took one, and
some took two but the second argument could be None. This patch makes such
functions uniform by using a list of nodes as an argument. This simplifies
the following refactoring.
No test logic has been changed.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Fri, 25 Jan 2013 20:01:23 +0000 (21:01 +0100)]
QA: Instance tests check the configuration before running
Instance test functions check the current configuration, and they will run
the actual tests only if the configuration supports them. This will be
used for refactoring in following patches.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Fri, 25 Jan 2013 18:06:40 +0000 (19:06 +0100)]
QA: Added functions to allocate/free N nodes at once
These make possible to simplify the logic of tests requiring more
nodes. Used in the following patches.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Mon, 28 Jan 2013 12:55:50 +0000 (13:55 +0100)]
QA: Added exclusive_storage to qa_config
QA configuration now tracks the state of the exclusive_storage flag at
cluster level. This will be used to selectively enable tests according to
the configuration.
Also, it's now possible to specify the initial value of the
exclusive_storage flag in the configuration file.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Mon, 28 Jan 2013 15:46:23 +0000 (16:46 +0100)]
QA: Better initialization of a global dictionary
The global 'cfg' variable is initialized to an empty dictionary, so there
is no more need to disable some pylint checks.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Tue, 22 Jan 2013 16:41:12 +0000 (17:41 +0100)]
QA: Added disk template to config
The disk templates used by the instances are tracked in the QA
configuration. This will be used to selectively enable tests according to
the configuration.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Tue, 29 Jan 2013 17:11:31 +0000 (18:11 +0100)]
QA cleanup: Removed instance-disk-failure test
The test was broken, out of sync with the rest of the code, and prone to
crashes. Until someone does a better job, it's better to remove the
test. This simplifies further refactoring.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Sat, 26 Jan 2013 02:38:10 +0000 (03:38 +0100)]
QA fix: TestInstanceExportWithRemove() updates configuration
When the instance is removed from the cluster by this test, it's also freed
in the QA configuration.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Mon, 21 Jan 2013 13:58:55 +0000 (14:58 +0100)]
QA: Cluster-verify reports shared PVs with exclusive storage
Exclusive storage forbids sharing PVs between unrelated LVs. This is a test
that cluster-verify correctly report such cases.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Wed, 30 Jan 2013 21:44:24 +0000 (22:44 +0100)]
QA: Added constants for LVM volumes
The LVM volume group becomes a configurable parameter, and a prefix for
logical volume names used in tests is introduced as a constant.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Fri, 1 Feb 2013 09:21:12 +0000 (10:21 +0100)]
Fix logging arguments formatting
This fixes: “ganeti/hypervisor/hv_kvm.py:93: [W1201, _GetTunFeatures]
Specify string format arguments as logging function parameters”, and
also updates the copyright years.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Bernardo Dal Seno [Fri, 18 Jan 2013 20:00:21 +0000 (21:00 +0100)]
QA: Templates incompatible with exclusive storage reported
Cluster-verify should report instances whose disk template is not
compatible with the exlusive storage setting.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Fri, 18 Jan 2013 17:52:25 +0000 (18:52 +0100)]
QA: Uniformity check for exclusive_storage in cluster-verify
Cluster-verify should report nodes in the same nodegroup with different
values of the exclusive_storage flag.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Wed, 16 Jan 2013 10:49:20 +0000 (11:49 +0100)]
QA: Test for basic features of exclusive storage
Check that creation of instances doesn't trigger any error.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Fri, 18 Jan 2013 16:57:08 +0000 (17:57 +0100)]
QA: Added function to read cluster configuration fields
This function will be used in following patches.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Fri, 18 Jan 2013 15:35:55 +0000 (16:35 +0100)]
QA: Added function to check cluster-verify result
Now it's possible to check that cluster-verify reports all the expected
errors, instead of simply checking that it fails.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Thu, 31 Jan 2013 12:53:12 +0000 (13:53 +0100)]
Document better gnt-network
Specify clearly that the nic parameters depend on the
network-to-nodegroup connection.
Clarify in gnt-network how instance nics are connected to a network.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Guido Trotter [Thu, 31 Jan 2013 12:46:02 +0000 (13:46 +0100)]
Fix the gnt-instance section in the htools manpage
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Guido Trotter [Thu, 31 Jan 2013 11:39:57 +0000 (12:39 +0100)]
update 2.7 NEWS file
Include a handful of important changes that weren't mentioned.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Guido Trotter [Wed, 30 Jan 2013 10:27:54 +0000 (11:27 +0100)]
Update 2.7 design doc
Move two designs that have been implemented in 2.7 to design-2.7 and out
of drafts.
Other designs were partly implemented: explain what part was, and leave
them as drafts pending their completion.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 30 Jan 2013 16:31:25 +0000 (17:31 +0100)]
Disable live-RPC queries under split query
Currently, the node listing RPC is very slow due to missing
parallelisation. For the 2.7 release, we reset these back to masterd,
hoping to revert them by the time 2.8 is ready.
There are a number of queries that I've left pointing to confd, as
they are non-RPC:
- node fields query
- node static fields query (and added a note in one case where it
could become a problem)
All the other converted queries (job listing, group queries, tags,
cluster config) remain pointing to confd, as they don't use RPC.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Dato Simó [Wed, 23 Jan 2013 23:39:59 +0000 (23:39 +0000)]
harep: finish execution with a summary of instance states
The harep tool prints messages for every action that it performs (or that
it doesn't perform). In case nothing is to be done at all, always print
some statistics of the current state of the cluster.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Wed, 23 Jan 2013 22:17:19 +0000 (22:17 +0000)]
harep: do not wait for repair job completion to set tags
Because of instance locks, after submitting a repair job we weren't able to
set the "pending" tag until at least the first opcode of the job finished.
Introduce a small delay in the repair job so as to allow the subsequent
TAGS_SET job to finish immediately, resulting in faster operation from the
user's point of view.
Make the duration of the delay configurable with --job-delay; if zero,
avoid inserting the TestDelay opcode entirely.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Wed, 23 Jan 2013 19:33:53 +0000 (19:33 +0000)]
Document that OpTestDelay/TestDelay take duration in seconds
Also, fix @rtype and @return elements of utils.TestDelay(), which now
returns a tuple but this wasn't being indicated.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Dato Simó [Mon, 21 Jan 2013 21:06:58 +0000 (21:06 +0000)]
harep: create repair jobs
Implement 'doRepair' to create a repair job from a list of opcodes if
the instance's policy allows it (otherwise set an ENOPERM result label),
_and_ the instance was previously healthy (i.e. not in ArFailed or
ArPendingRepair).
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Mon, 21 Jan 2013 21:05:38 +0000 (21:05 +0000)]
harep: pure function to detect brokeness with instances
Add a 'detectBroken' function that determines whether an instance is in an
unhealthy state, and what's needed to repair it. The repair is specified as
an AutoRepairType constant, and a list of opcodes. The opcodes will only be
executed (in following patches) if the repair type is allowed by the policy.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Wed, 16 Jan 2013 21:47:44 +0000 (21:47 +0000)]
harep: check for completed jobs at the start of the program
As a first step before detecting any brokeness with instances, see if any
of our previous repairs have completed, and move instances to ArFailed or
ArHealthy accordingly. Do nothing if there are still running jobs for the
repair.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Wed, 16 Jan 2013 21:33:13 +0000 (21:33 +0000)]
harep: initial parsing of tags
Parse auto-repair tags to set each instance in one of ArHealthy, ArFailed,
or ArPendingRepair. The implementation tries to be well behaved when old
tags have been left behind, which future patches will still try _not_ to do.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Wed, 16 Jan 2013 20:57:30 +0000 (20:57 +0000)]
Program/Harep.hs: add skeleton for the new auto-repair tool
harep(1) detects certain kind of problems with instances and applies the
allowed set of solutions. See doc/design-autorepair.rst.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Wed, 16 Jan 2013 20:46:08 +0000 (20:46 +0000)]
HTools/Types.hs: minor adjustments to auto-repair types
In particular:
- make ArHealthy take an optional AutoRepairData; this allows to
represent the situation where a repair completed successfully, and
hence there's an associated tag we might want to know about.
- add an 'arTag' member to AutoRepairData to store the exact tag this
data was derived from; this allows for easier removal of the tag when
its associated state is superseded. It also stores the tag that will
be written to the cluster when a new AutoRepairData record is created.
- change the default sort order of AutoRepairResult to reflect the
precedence used when reading tags from an object (a failure, if
present, is always the resulting state; a success overrides an
ENOPERM).
- (trivial) make AutoRepairStatus and AutoRepairData derive Eq and Show.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Mon, 14 Jan 2013 17:04:25 +0000 (17:04 +0000)]
Jobs.hs: start with a shorter delay in waitForJobs
Instead of waiting 15 seconds every iteration of the waiting loop, start
with 0.5 seconds and increment exponentially until a certain maximum is
reached. Keep the maximum at 15 seconds.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Wed, 16 Jan 2013 20:45:41 +0000 (20:45 +0000)]
Jobs.hs: add an execJobsWaitOk function
This new 'execJobsWaitOk' returns "Bad" not only if an error occurs, but if
any of the submitted jobs doesn't succeed.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Tue, 22 Jan 2013 16:20:34 +0000 (16:20 +0000)]
Jobs.hs: add a submitJobs function
This new 'submitJobs' function doesn't wait for job completion, it just
returns the job IDs immediately.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Mon, 28 Jan 2013 17:36:57 +0000 (17:36 +0000)]
Fix typo ("Abrreviation" -> "Abbreviation" in Common.hs)
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Wed, 30 Jan 2013 16:56:20 +0000 (16:56 +0000)]
Removes check for conflicts from NetworkDisconnect
This removes the check for conflicts from the Haskell
version of the OpCode NetworkDisconnect. This alignes
the Haskell code with the patch
"Force conflicts check in LUNetworkDisconnect" (which
is currently under review). I will submit these patches
together then.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Helga Velroyen [Wed, 30 Jan 2013 16:35:10 +0000 (17:35 +0100)]
If _UnlockedLookupNetwork() fails raise error
Make _UnlockedLookupNetwork() raise OpPrereqError (instead of returning
None) in case it does not find the requested network. Remove useless and
duplicate code such as:
if net_uuid is None:
raise...
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>
Helga Velroyen [Wed, 30 Jan 2013 16:12:26 +0000 (17:12 +0100)]
Force conflicts check in LUNetworkDisconnect
Until now if one disconnects a network with --no-conflicts-check
and then remove it, there is a possibility to leave instances with NICs
referencing non-existing networks. This causes network queries,
instance removal and modification to fail.
This patch allows a network to be disconnected from a nodegroup,
only if instances residing in the nodegroup do not have NICs attached
to the network. Otherwise OpPrereqError is raised. The
--no-conflicts-check option is removed from the gnt-network disconnect
command as well.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Wed, 30 Jan 2013 16:04:58 +0000 (17:04 +0100)]
hv_kvm: Refactor getting TAP features, add tests
Split retrieving supported features into a dedicated function which can
be mocked. Tests are added for both “_ProbeTapVnetHdr” and
“_GetTunFeatures”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 30 Jan 2013 16:06:16 +0000 (17:06 +0100)]
Merge branch 'devel-2.7'
* devel-2.7:
hv_kvm: Original error message, keyword parameter
CLI.hs: fix double spaces in option help strings
Add new NIC parameter to man page of 'gnt-instance'
Fix spelling issue in hroller.rst
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 30 Jan 2013 15:21:22 +0000 (16:21 +0100)]
hv_kvm: Original error message, keyword parameter
- Include original error message when creating TAP interface failed
- Pass keyword parameter as such
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 29 Jan 2013 16:05:44 +0000 (17:05 +0100)]
Checks for hypervisor constants
Add two tests for hypervisor constants.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dato Simó [Mon, 28 Jan 2013 17:56:38 +0000 (17:56 +0000)]
CLI.hs: fix double spaces in option help strings
Some help strings with continuation backslashes ('\') were providing a
space both before and after the backslash, resulting in double spaces in
help output. Provide it only after the backslash, which fixes the issue and
is consistent with the rest of the file.
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Mon, 28 Jan 2013 16:06:37 +0000 (17:06 +0100)]
Add fields 'inst_cnt' and 'inst_list' to network queries
This adds the fields 'inst_cnt' and 'inst_list' to the
Haskell implementation of the network queries.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Mon, 28 Jan 2013 15:06:57 +0000 (16:06 +0100)]
Moving network query helper functions to Network.hs
So far, a couple of helper function for the network
queries resided in Config.hs. I figured it makes more
sense to move them to Query/Network.hs, since they are
very tailored to that purpose.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Helga Velroyen [Mon, 28 Jan 2013 16:49:51 +0000 (17:49 +0100)]
Add new NIC parameter to man page of 'gnt-instance'
According to doc/design-network.rst, there is a new NIC
parameter 'network', which can be used to link an instance
to a network. This was so far undocumented in the man
page of 'gnt-instance'.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Fri, 25 Jan 2013 13:36:10 +0000 (14:36 +0100)]
More fields for network queries
This adds more fields to the network queries:
- group_cnt, free_count, reserved_count, and map
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Benjamin Kerensa [Tue, 22 Jan 2013 00:32:08 +0000 (16:32 -0800)]
Fix spelling issue in hroller.rst
'explicitly' was spelled the wrong way
Signed-off-by: Benjamin Kerensa <bkerensa@ubuntu.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Fri, 25 Jan 2013 14:13:47 +0000 (15:13 +0100)]
Merge branch 'devel-2.7'
* devel-2.7:
kvm: fix bug while fetching -device list
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Fri, 25 Jan 2013 13:35:43 +0000 (14:35 +0100)]
hv_xen: Fix issues with migration, add tests
Commit 3d942d8 broke instance migration (“self._cmd” was set to None).
This patch fixes that issue, refactors “MigrateInstance” for testing and
adds those tests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Fri, 25 Jan 2013 11:31:22 +0000 (12:31 +0100)]
kvm: fix bug while fetching -device list
_GetKVMOutput expects the command to succeed, but unfortunately on some
versions of kvm "-device ?" will output a correct list of devices, while
exiting with an error code.
To fix this we accept failure in that case (note that this doesn't
change the situation when this command is not supported, as in that case
the output will still not contain the correct string).
This patch also updates the docstring, adding the parameter types and
description.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:12:32 +0000 (15:12 +0100)]
hv_xen: Test various forms of getting instance/node info
Test functions for retrieving instance and node information.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:09:38 +0000 (15:09 +0100)]
hv_xen: Test stopping an instance
Test stopping an instance, and ensure the configuration file still
exists after an instance couldn't be stopped.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:11:47 +0000 (15:11 +0100)]
hv_xen: Test starting an instance
Test starting an instance in normal mode, in paused mode, and in case of
failure.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:11:09 +0000 (15:11 +0100)]
hv_xen: Test reading non-existent config file
Check whether doing so raises an exception.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:10:37 +0000 (15:10 +0100)]
hv_xen: Test verifying hypervisor
Test success and failure.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:10:11 +0000 (15:10 +0100)]
hv_xen: Test removing config from "auto" directory
Test whether any existing configuration file named after the instance is
removed from Xen's “auto” directory.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:08:50 +0000 (15:08 +0100)]
Establish base for testing Xen hypervisor abstraction
There are two separate Xen hypervisors (HVM and PVM), as well as two
different Xen commands (xl and xm). This already provides four different
combinations and future changes might bring even more. For this reason
the test classes for the Xen hypervisor are not defined manually, but
rather generated at module load time. Doing so makes it possible to keep
using the standard unit test infrastructure provided by the “unittest”
module.
Using “super”, which we try to avoid in other places, is necessary to
call functions provided by other classes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:02:42 +0000 (15:02 +0100)]
hv_xen: Test errors while writing config file
Test error handling while writing Xen configuration files.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:01:55 +0000 (15:01 +0100)]
hv_xen: Test using unknown Xen command
This tests “unknown” values of “constants.XEN_CMD”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 14:00:36 +0000 (15:00 +0100)]
hv_xen: Split StopInstance
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Tue, 22 Jan 2013 14:54:53 +0000 (15:54 +0100)]
hv_xen: Abstract running Xen commands
Instead of using the “XEN_CMD” constant in multiple places, that is now
all in a single place and can easily be changed for unit tests (through
a parameter given to the constructor).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Fri, 25 Jan 2013 10:12:30 +0000 (11:12 +0100)]
Merge branch 'devel-2.7'
* devel-2.7:
hv_xen: Remove config after shutdown was successful
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Helga Velroyen [Thu, 24 Jan 2013 14:47:57 +0000 (15:47 +0100)]
List networks: call client explicitely
This way, it is easier to make it call the haskell
implementation of the network queries.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Helga Velroyen [Thu, 24 Jan 2013 14:51:35 +0000 (15:51 +0100)]
First part of Network Queries in Haskell
This is the beginning of the implementation of network
queries. This includes establishing all infrastructure
to run the network queries and implement querying of
some simpler fields and the node group listing.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Helga Velroyen [Thu, 24 Jan 2013 14:50:36 +0000 (15:50 +0100)]
Convert Maybe results to RSUnavail
When displaying query results of type Maybe, one could use the
function rsMaybe. Unfortunately, this function maps 'Nothing'
values to RSNoData which gets displayed as '?' in the list of
query results. These semantics do not fit if the result is of
a Maybe type, because it is an optional field, because in this
case 'Nothing' is not an error (like for example the consequence
of an RPC error), but a legitimate result and as such should
be displayed as '-' in the result list. For this purpose,
this patch introduces another version of rsMaybe, which is
called rsMaybeUnavail to be used for optional fields.
For consistency, rsMaybe gets renamed to rsMaybeNoData.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Helga Velroyen [Thu, 24 Jan 2013 14:46:41 +0000 (15:46 +0100)]
Extend config by networks and networks by UUIDs
For network queries to work, we need to extend the general
config type to include all available networks. Additionally,
we add UUIDs to the network type itself.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Helga Velroyen [Thu, 24 Jan 2013 15:54:34 +0000 (16:54 +0100)]
Typo in comment of network type
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 13:29:18 +0000 (14:29 +0100)]
hv_xen: Remove config after shutdown was successful
If stopping an instance failed, the configuration would already be gone
and other operations depending on it (e.g. migration) would no longer
work. With this patch the configuration file is only removed once the
instance was succesfully stopped or destroyed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michael Hanselmann [Tue, 22 Jan 2013 14:42:07 +0000 (15:42 +0100)]
hv_xen: Simplify writing configuration
Instead of calling _WriteConfigFileStatic from both derived classes,
those now only return the content (minus the “do not edit” header). The
configuration is then written by the base class, XenHypervisor.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Tue, 22 Jan 2013 13:27:48 +0000 (14:27 +0100)]
hv_xen: Prepare for unit tests, remove {static,class}method
Unit tests will have to specify a custom Xen configuration directory (a
temporary directory). To this end “hv_xen.XenHypervisor” and its two
derived classes are modified to have fewer static and class methods. A
new private attribute for the configuration directory is also added (to
be used in test code).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Tue, 22 Jan 2013 12:31:25 +0000 (13:31 +0100)]
hv_xen: Factorize and test disk configuration
The “_GetConfigFileDiskData” function is moved to module level and
cleaned up (module-level constants for letters and file I/O drivers).
Until now only 24 disks would be supported (e.g. “sda” to “sdx”), when
the Latin alphabet actually has 26 characters. Now all 26 letters would
be available for use (“constants.MAX_DISKS” is still 16).
Newly added unit tests provide complete coverage for
“_GetConfigFileDiskData”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Mon, 21 Jan 2013 17:17:42 +0000 (18:17 +0100)]
hv_xen: Refactor getting node information, add tests
Refactor and add tests for getting node (Domain-0) information.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Fri, 18 Jan 2013 14:30:05 +0000 (15:30 +0100)]
hv_xen: Refactor running & parsing "xm list", add tests
This patch refactors “_RunXmList” and adds some tests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 24 Jan 2013 11:46:15 +0000 (12:46 +0100)]
Merge branch 'devel-2.7'
* devel-2.7:
hv_*: Always return from Verify, style fixes
_VerifyErrors()._Error() and _ErrorIf() are now consistent
Handle the result of QueryGroups() correctly
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 23 Jan 2013 17:41:34 +0000 (18:41 +0100)]
hv_*: Always return from Verify, style fixes
Change all “Verify” methods in hypervisor abstractions to explicitely
return None if no problem was detected. Remove punctuation from error
messages. Update docstrings with “@return” and some small mistakes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Helga Velroyen [Mon, 21 Jan 2013 15:20:34 +0000 (16:20 +0100)]
Check minimum size of networks on creation
When creating a network, so far no size constraints were checked.
We now limit the size of a network to a /30 or bigger, although
tecnically, the ipaddr library supports even /32 networks.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Helga Velroyen [Tue, 15 Jan 2013 14:34:55 +0000 (15:34 +0100)]
Network QA
This adds a script for the QA of 'gnt-network'. So far it
covers adding/removing and connecting/disconnecting networks.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Helga Velroyen [Mon, 21 Jan 2013 09:33:35 +0000 (10:33 +0100)]
Fix syntax error in QA's sample config
The additional comma makes this file invalid JSON.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 23 Jan 2013 17:41:01 +0000 (18:41 +0100)]
constants: Add list of known Xen commands
These will be used in unittests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Mon, 21 Jan 2013 21:43:24 +0000 (22:43 +0100)]
_VerifyErrors()._Error() and _ErrorIf() are now consistent
_Error() didn't contain the logic for demoting errors to warnings and for
marking an operation as failed. Now _ErrorIf() is just a minimal wrapper
for _Error().
Unit tests included.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 23 Jan 2013 13:37:42 +0000 (14:37 +0100)]
Merge branch 'master' into devel-2.7
* master:
Handle the result of QueryGroups() correctly
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Tue, 22 Jan 2013 13:58:32 +0000 (15:58 +0200)]
Handle the result of QueryGroups() correctly
If no group is given for the “gnt-network connect“/“… disconnect”
commands, the client uses the result of “QueryGroups()” which is a list
of lists. Use “itertools.chain()” to handle the return value correctly.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit
af3aea481c3b8528239770edcae75d406a57de81)
Dimitris Aragiorgis [Tue, 22 Jan 2013 13:58:32 +0000 (15:58 +0200)]
Handle the result of QueryGroups() correctly
If no group is given for the “gnt-network connect“/“… disconnect”
commands, the client uses the result of “QueryGroups()” which is a list
of lists. Use “itertools.chain()” to handle the return value correctly.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 22 Jan 2013 13:25:52 +0000 (14:25 +0100)]
hv_xen: Compose file name outside error handling
In _ReadConfigFile, the filename should be prepared outside the
try/except block. Fixes bad code formatting, too.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michael Hanselmann [Tue, 22 Jan 2013 15:41:24 +0000 (16:41 +0100)]
testutils: Module-level method for reading test data
This patch moves the functions “_ReadTestData” and “_TestDataFilename”
to module level. They do not depend on a class instance at all.
Some cases of “utils.ReadFile(self._TestDataFilename(…))” are also
corrected.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Tue, 22 Jan 2013 12:50:57 +0000 (13:50 +0100)]
Add test for backend._GetBlockDevSymlinkPath
Add a unit test for the trivial “_GetBlockDevSymlinkPath” function in
backend (small changes in the function were required).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Tue, 22 Jan 2013 12:55:33 +0000 (13:55 +0100)]
hv_base: Remove empty constructor
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Helga Velroyen [Mon, 21 Jan 2013 15:12:04 +0000 (16:12 +0100)]
Fix format string of KVM output
This fixes a missing 's' in the format string and
the wrong quotes. Those bugs were introduced in
commit
6e043e60.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 17 Jan 2013 12:13:02 +0000 (13:13 +0100)]
Renames and cleanup of variable names in confd
The current names are quite confusing; this patch cleans up the
confusion by making sure we use different terms for the two threads,
etc.
No actual code changes besides the renames.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 17 Jan 2013 12:04:13 +0000 (13:04 +0100)]
Change the polling thread sleep behaviour
Currently, the polling thread sleeps at the end of its body; the
original reason was that when the thread initially starts, we want to
load the config immediately (in the old-style config load model).
However, now we explicitly load the config, so we can make this thread
behave like the watcher thread: sleep first, then do the work. This
model improves for example the startup of the thread, by not redoing a
config check immediately after switching to polling mode.
We also increase the max idle poll rounds interval from 2 to 3; the
new behaviour means that we revert from polling to inotify after
around one second of no changes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 17 Jan 2013 11:22:40 +0000 (12:22 +0100)]
First part of confd timer changes
This patch changes the resolution of the timers: the watcher timer
goes from 60s to 17s, and the polling-mode timer goes from 2 seconds
to 250ms. The code changes a bit more due to the changes in the units
of the various constants.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 17 Jan 2013 11:15:39 +0000 (12:15 +0100)]
Increase Haskell log file resolution
Currently the Haskell logging goes with the default time formatting,
which lacks sub-second resolution. This is not good, as it's too
coarse.
The patch adds picoseconds to the log; they will contain usually too
many zeroes, but the other option is to use a log format that (for
precisely zero fractional part) omits the separator, so it's not
consistent from line to line, and which also uses a dot as a
separator, different from the Python code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>