QA: Release instances through method
Instead of having a module-level function in “qa_config” to release aninstance, a new method on instance objects is used.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
QA: Convert instances to objects
Up until now instances were stored as a dictionary. The keys werehardcoded in a lot of places and entries modified directly.
This patch introduces a new class for instances in QA named“_QaInstance”. It still supports accessing details via dictionary...
QA: Release instance in ganeti-qa
Release instances directly from “ganeti-qa” instead of doing so in thethree tests removing instances. This is in line with how nodes arereleased and makes it easier to find and debug double-release issues.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
QA: Stop using "in" operator for node
The new configuration object for nodes will not support it anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
QA: Support entity objects in qa_utils._GetName
This is a temporary solution until nodes and instances are converted toobjects with attributes.
qa_config: Simplify IsTemplateSupported
The same code can be written in a single expression. The earlier patchnamed “Refactor storage of runtime exclusive storage flag in QA”provided the unit test.
Refactor storage of runtime exclusive storage flag in QA
This is a follow-up for “qa_config: Remove exclusive storage flag fromconfig”. Instead of storing the flag in a module-level variable it isnow stored within the new QA configuration class and unit tests are...
Refactor QA configuration code
Ever since its introduction (sometime before commit cec9845 in September2007), the QA configuration was stored in a dictionary at module-levelin “qa/qa_config.py”. The configuration was loaded, verified andevaluated using module-level functions. Since then the configuration has...
qa_config: Remove exclusive storage flag from config
Commit 6a0f22e added the ability to track the status of the cluster-wideflag for exclusive storage. It did so in a way which modifies thein-memory configuration dictionary. This patch changes the code to use a...
vcluster: Make _MakeNodeRoot public
The QA code will also have to generate virtual paths and this functioncomes in handy.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Harep.hs: fix a couple typos in comments and docstrings
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
hv_kvm: Error messages, punctuation, other style fixes
- Capitalize acronyms such as “QMP” or “SPICE”- Remove punctuation from error messages- Improve error reporting when sending a monitor command failed- Put “or” at end of line- Fix indentation for a concatenated string...
hv_kvm: Add TODO regarding monitor commands
Monitor calls should be replaced with QMP once KVM >= 0.14 is required.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix improperly formatted docstring
Change the docstring of chompPrefix to prevent the error"doc comment parse failed" that was raised by some version ofhaddock while generating the documentation for this function.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Merge branch 'devel-2.7'
Redirect output for gnt-* list/list-fields commands
This enabled the redirection; QA output after this patch should becomemuch much smaller (on a 'quick' QA, we are left with ~0.5MB output onstdout and 64MB redirected output).
Signed-off-by: Iustin Pop <iustin@google.com>...
Add function to execute QA commands with redirected output
Some QA commands generate non-trivial output: for a full QA, the gnt-*list commands generate more than 300MB of data. To make the QA logsmore parseable by humans, we need support for redirecting output of...
Implement option to skip logging of QA commands
This patch adds an optional kwarg that allows skipping the "Command:…" line in QA output. This is helpful if we only need to run somemaintenance actions, that are QA-related and not Ganeti-related.
Making ParticalNic's network field of type String
This is yet another fix for type confusion between pythonand haskell. ;) The network field of PartialNic should bea string and not of type Network. This makes it necessaryto add a helper function to look up a network by name...
Fix typo in a comment
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add some more Haskell/Python equivalence tests
This would have caught the log file problem fixed in the previouspatch (9411474b), for example. Also we test user/group equivalence,name only.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix Haskell log file naming after virtual cluster changes
Commit 3329f4de changed the Haskell log file from constants tofunctions, but introduced a bug: it uses now the daemon name insteadof the correct log file, which means "ganeti-confd.log" instead of...
QA: Clean up after "instance-remove-drbd-offline" test
This test used to leave a node with orphan logical volumes and unused DRBDminors. Now, the expected leftovers are cleaned up after the test, andcluster-verify is run.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
qa_config: Remove useless "options" variable
It is only used in one place and not necessary.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Extract container converters
“objects.ConfigObject” contains two useful functions for working withcontainers of serialized objects, “_ContainerToDicts” and“_ContainerFromDicts”. This patch separates those functions and movesthem into “objectutils” as they'll be useful for converting parts of the...
hv_xen: Fix epydoc error
“utils.RunCmd” is re-exported from “utils.process.RunCmd”. Epydocdoesn't fully understand this, so we have to refer to the original.
objects: Improve handling of TCP/UDP port pool
- Handle de-serialization correctly when pool is not defined- Serialize to empty list when the attribute is None (this should never happen in reality as the attribute is always set when de-serializing)- Add tests...
Switch KVM to multi-error verify results
This uses the new _FormatVerifyResults helper function to returnmultiple errors.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Make LXC fail verification if cgroup is not mounted
Since LXC now relies on cgroup memory limits to enforce memory, let'smake hypervisor verification (and thus cluster-verify) return errorswhen the cgroup filesystem is not mounted.
Add a helper function for hypervisor verification
This will allow easier multi-error results from hypervisors; rightnow, we only report the first error, which is not nice.
hv_lxc: fix whitespace errors
The latest lxc patches included a few whitespace style errors, that makelint fail. This patch fixes those.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
s/nic/NIC/ in manpages
For consistency, let's make sure NICs are capitalized always in the sameway.
LXC: add support for the memory controller
Add support for the memory resource controller, useful to actually placememory limits on instances.
Support is still optional, in case the kernel doesn't have it compiledin, or in the case of Debian has it dependent on a kernel command-line...
LXC: adapt hv for newer lxc userspace tools
Currently hv_lxc depends on the behavior and output of older LXC tools,which have since changed, making it unable to function in currentdistributions (e.g. Debian wheezy).
Adapt the tools and expectation for the output and make it into a...
Fix typo in ganeti-noded man page
Add a missing "in".
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
QA: Run instance tests with different cluster configurations
Instance tests are run with or without the exclusive-storage flag set. Moreconfigurations can be added easily.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
QA: Instance tests check the configuration before running
Instance test functions check the current configuration, and they will runthe actual tests only if the configuration supports them. This will beused for refactoring in following patches.
QA: Use lists of nodes as argument to instance tests
Some instance test functions took two node arguments, some took one, andsome took two but the second argument could be None. This patch makes suchfunctions uniform by using a list of nodes as an argument. This simplifies...
QA: Refactored launching of the various instance tests
The code launching instance tests for "plain" and "drbd" templates has beenunified. In this way it's easier to add more templates, and to re-use the sametests with different configurations.
QA: Run cluster-verify after instance tests
We want to make sure that instance operations don't leave the cluster in abad state. The "instance-remove-drbd-offline" test leaves some debrisbehind, so it's been moved to the last position.
QA: Added functions to allocate/free N nodes at once
These make possible to simplify the logic of tests requiring morenodes. Used in the following patches.
QA: Added exclusive_storage to qa_config
QA configuration now tracks the state of the exclusive_storage flag atcluster level. This will be used to selectively enable tests according tothe configuration.
Also, it's now possible to specify the initial value of the...
QA fix: TestInstanceExportWithRemove() updates configuration
When the instance is removed from the cluster by this test, it's also freedin the QA configuration.
QA cleanup: Removed instance-disk-failure test
The test was broken, out of sync with the rest of the code, and prone tocrashes. Until someone does a better job, it's better to remove thetest. This simplifies further refactoring.
QA: Added disk template to config
The disk templates used by the instances are tracked in the QAconfiguration. This will be used to selectively enable tests according tothe configuration.
QA: Better initialization of a global dictionary
The global 'cfg' variable is initialized to an empty dictionary, so thereis no more need to disable some pylint checks.
QA: Cluster-verify reports shared PVs with exclusive storage
Exclusive storage forbids sharing PVs between unrelated LVs. This is a testthat cluster-verify correctly report such cases.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
QA: Added constants for LVM volumes
The LVM volume group becomes a configurable parameter, and a prefix forlogical volume names used in tests is introduced as a constant.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix logging arguments formatting
This fixes: “ganeti/hypervisor/hv_kvm.py:93: [W1201, _GetTunFeatures]Specify string format arguments as logging function parameters”, andalso updates the copyright years.
QA: Templates incompatible with exclusive storage reported
Cluster-verify should report instances whose disk template is notcompatible with the exlusive storage setting.
QA: Uniformity check for exclusive_storage in cluster-verify
Cluster-verify should report nodes in the same nodegroup with differentvalues of the exclusive_storage flag.
QA: Test for basic features of exclusive storage
Check that creation of instances doesn't trigger any error.
QA: Added function to read cluster configuration fields
This function will be used in following patches.
QA: Added function to check cluster-verify result
Now it's possible to check that cluster-verify reports all the expectederrors, instead of simply checking that it fails.
Document better gnt-network
Specify clearly that the nic parameters depend on thenetwork-to-nodegroup connection.
Clarify in gnt-network how instance nics are connected to a network.
Fix the gnt-instance section in the htools manpage
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Update 2.7 design doc
Move two designs that have been implemented in 2.7 to design-2.7 and outof drafts.
Other designs were partly implemented: explain what part was, and leavethem as drafts pending their completion.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
update 2.7 NEWS file
Include a handful of important changes that weren't mentioned.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Disable live-RPC queries under split query
Currently, the node listing RPC is very slow due to missingparallelisation. For the 2.7 release, we reset these back to masterd,hoping to revert them by the time 2.8 is ready.
There are a number of queries that I've left pointing to confd, as...
harep: finish execution with a summary of instance states
The harep tool prints messages for every action that it performs (or thatit doesn't perform). In case nothing is to be done at all, always printsome statistics of the current state of the cluster....
harep: do not wait for repair job completion to set tags
Because of instance locks, after submitting a repair job we weren't able toset the "pending" tag until at least the first opcode of the job finished.Introduce a small delay in the repair job so as to allow the subsequent...
Document that OpTestDelay/TestDelay take duration in seconds
Also, fix @rtype and @return elements of utils.TestDelay(), which nowreturns a tuple but this wasn't being indicated.
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
harep: create repair jobs
Implement 'doRepair' to create a repair job from a list of opcodes ifthe instance's policy allows it (otherwise set an ENOPERM result label),and the instance was previously healthy (i.e. not in ArFailed orArPendingRepair)....
harep: pure function to detect brokeness with instances
Add a 'detectBroken' function that determines whether an instance is in anunhealthy state, and what's needed to repair it. The repair is specified asan AutoRepairType constant, and a list of opcodes. The opcodes will only be...
harep: check for completed jobs at the start of the program
As a first step before detecting any brokeness with instances, see if anyof our previous repairs have completed, and move instances to ArFailed orArHealthy accordingly. Do nothing if there are still running jobs for the...
harep: initial parsing of tags
Parse auto-repair tags to set each instance in one of ArHealthy, ArFailed,or ArPendingRepair. The implementation tries to be well behaved when oldtags have been left behind, which future patches will still try not to do....
Program/Harep.hs: add skeleton for the new auto-repair tool
harep(1) detects certain kind of problems with instances and applies theallowed set of solutions. See doc/design-autorepair.rst.
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
HTools/Types.hs: minor adjustments to auto-repair types
In particular:
- make ArHealthy take an optional AutoRepairData; this allows to represent the situation where a repair completed successfully, and hence there's an associated tag we might want to know about....
Jobs.hs: start with a shorter delay in waitForJobs
Instead of waiting 15 seconds every iteration of the waiting loop, startwith 0.5 seconds and increment exponentially until a certain maximum isreached. Keep the maximum at 15 seconds.
Signed-off-by: Dato Simó <dato@google.com>...
Jobs.hs: add an execJobsWaitOk function
This new 'execJobsWaitOk' returns "Bad" not only if an error occurs, but ifany of the submitted jobs doesn't succeed.
Jobs.hs: add a submitJobs function
This new 'submitJobs' function doesn't wait for job completion, it justreturns the job IDs immediately.
Fix typo ("Abrreviation" -> "Abbreviation" in Common.hs)
Removes check for conflicts from NetworkDisconnect
This removes the check for conflicts from the Haskellversion of the OpCode NetworkDisconnect. This alignesthe Haskell code with the patch"Force conflicts check in LUNetworkDisconnect" (whichis currently under review). I will submit these patches...
If _UnlockedLookupNetwork() fails raise error
Make _UnlockedLookupNetwork() raise OpPrereqError (instead of returningNone) in case it does not find the requested network. Remove useless andduplicate code such as:
if net_uuid is None: raise...
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Force conflicts check in LUNetworkDisconnect
Until now if one disconnects a network with --no-conflicts-checkand then remove it, there is a possibility to leave instances with NICsreferencing non-existing networks. This causes network queries,instance removal and modification to fail....
hv_kvm: Refactor getting TAP features, add tests
Split retrieving supported features into a dedicated function which canbe mocked. Tests are added for both “_ProbeTapVnetHdr” and“_GetTunFeatures”.
hv_kvm: Original error message, keyword parameter
- Include original error message when creating TAP interface failed- Pass keyword parameter as such
Checks for hypervisor constants
Add two tests for hypervisor constants.
CLI.hs: fix double spaces in option help strings
Some help strings with continuation backslashes ('\') were providing aspace both before and after the backslash, resulting in double spaces inhelp output. Provide it only after the backslash, which fixes the issue and...
Add fields 'inst_cnt' and 'inst_list' to network queries
This adds the fields 'inst_cnt' and 'inst_list' to theHaskell implementation of the network queries.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Moving network query helper functions to Network.hs
So far, a couple of helper function for the networkqueries resided in Config.hs. I figured it makes moresense to move them to Query/Network.hs, since they arevery tailored to that purpose.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Add new NIC parameter to man page of 'gnt-instance'
According to doc/design-network.rst, there is a new NICparameter 'network', which can be used to link an instanceto a network. This was so far undocumented in the manpage of 'gnt-instance'.
More fields for network queries
This adds more fields to the network queries:- group_cnt, free_count, reserved_count, and map
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix spelling issue in hroller.rst
'explicitly' was spelled the wrong way
Signed-off-by: Benjamin Kerensa <bkerensa@ubuntu.com>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
hv_xen: Fix issues with migration, add tests
Commit 3d942d8 broke instance migration (“self._cmd” was set to None).This patch fixes that issue, refactors “MigrateInstance” for testing andadds those tests.
kvm: fix bug while fetching -device list
_GetKVMOutput expects the command to succeed, but unfortunately on someversions of kvm "-device ?" will output a correct list of devices, whileexiting with an error code.
To fix this we accept failure in that case (note that this doesn't...
hv_xen: Test various forms of getting instance/node info
Test functions for retrieving instance and node information.
hv_xen: Test stopping an instance
Test stopping an instance, and ensure the configuration file stillexists after an instance couldn't be stopped.
hv_xen: Test starting an instance
Test starting an instance in normal mode, in paused mode, and in case offailure.
hv_xen: Test reading non-existent config file
Check whether doing so raises an exception.
hv_xen: Test verifying hypervisor
Test success and failure.
hv_xen: Test removing config from "auto" directory
Test whether any existing configuration file named after the instance isremoved from Xen's “auto” directory.
Establish base for testing Xen hypervisor abstraction
There are two separate Xen hypervisors (HVM and PVM), as well as twodifferent Xen commands (xl and xm). This already provides four differentcombinations and future changes might bring even more. For this reason...
hv_xen: Test errors while writing config file
Test error handling while writing Xen configuration files.
hv_xen: Test using unknown Xen command
This tests “unknown” values of “constants.XEN_CMD”.
hv_xen: Split StopInstance
hv_xen: Abstract running Xen commands
Instead of using the “XEN_CMD” constant in multiple places, that is nowall in a single place and can easily be changed for unit tests (througha parameter given to the constructor).