Fix improperly formatted docstring
Change the docstring of chompPrefix to prevent the error"doc comment parse failed" that was raised by some version ofhaddock while generating the documentation for this function.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Merge branch 'devel-2.7'
Redirect output for gnt-* list/list-fields commands
This enabled the redirection; QA output after this patch should becomemuch much smaller (on a 'quick' QA, we are left with ~0.5MB output onstdout and 64MB redirected output).
Signed-off-by: Iustin Pop <iustin@google.com>...
Add function to execute QA commands with redirected output
Some QA commands generate non-trivial output: for a full QA, the gnt-*list commands generate more than 300MB of data. To make the QA logsmore parseable by humans, we need support for redirecting output of...
Implement option to skip logging of QA commands
This patch adds an optional kwarg that allows skipping the "Command:…" line in QA output. This is helpful if we only need to run somemaintenance actions, that are QA-related and not Ganeti-related.
Making ParticalNic's network field of type String
This is yet another fix for type confusion between pythonand haskell. ;) The network field of PartialNic should bea string and not of type Network. This makes it necessaryto add a helper function to look up a network by name...
Fix typo in a comment
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add some more Haskell/Python equivalence tests
This would have caught the log file problem fixed in the previouspatch (9411474b), for example. Also we test user/group equivalence,name only.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix Haskell log file naming after virtual cluster changes
Commit 3329f4de changed the Haskell log file from constants tofunctions, but introduced a bug: it uses now the daemon name insteadof the correct log file, which means "ganeti-confd.log" instead of...
QA: Clean up after "instance-remove-drbd-offline" test
This test used to leave a node with orphan logical volumes and unused DRBDminors. Now, the expected leftovers are cleaned up after the test, andcluster-verify is run.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
qa_config: Remove useless "options" variable
It is only used in one place and not necessary.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Extract container converters
“objects.ConfigObject” contains two useful functions for working withcontainers of serialized objects, “_ContainerToDicts” and“_ContainerFromDicts”. This patch separates those functions and movesthem into “objectutils” as they'll be useful for converting parts of the...
hv_xen: Fix epydoc error
“utils.RunCmd” is re-exported from “utils.process.RunCmd”. Epydocdoesn't fully understand this, so we have to refer to the original.
objects: Improve handling of TCP/UDP port pool
- Handle de-serialization correctly when pool is not defined- Serialize to empty list when the attribute is None (this should never happen in reality as the attribute is always set when de-serializing)- Add tests...
Switch KVM to multi-error verify results
This uses the new _FormatVerifyResults helper function to returnmultiple errors.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Make LXC fail verification if cgroup is not mounted
Since LXC now relies on cgroup memory limits to enforce memory, let'smake hypervisor verification (and thus cluster-verify) return errorswhen the cgroup filesystem is not mounted.
Add a helper function for hypervisor verification
This will allow easier multi-error results from hypervisors; rightnow, we only report the first error, which is not nice.
hv_lxc: fix whitespace errors
The latest lxc patches included a few whitespace style errors, that makelint fail. This patch fixes those.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
s/nic/NIC/ in manpages
For consistency, let's make sure NICs are capitalized always in the sameway.
LXC: add support for the memory controller
Add support for the memory resource controller, useful to actually placememory limits on instances.
Support is still optional, in case the kernel doesn't have it compiledin, or in the case of Debian has it dependent on a kernel command-line...
LXC: adapt hv for newer lxc userspace tools
Currently hv_lxc depends on the behavior and output of older LXC tools,which have since changed, making it unable to function in currentdistributions (e.g. Debian wheezy).
Adapt the tools and expectation for the output and make it into a...
Fix typo in ganeti-noded man page
Add a missing "in".
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
QA: Run instance tests with different cluster configurations
Instance tests are run with or without the exclusive-storage flag set. Moreconfigurations can be added easily.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
QA: Instance tests check the configuration before running
Instance test functions check the current configuration, and they will runthe actual tests only if the configuration supports them. This will beused for refactoring in following patches.
QA: Use lists of nodes as argument to instance tests
Some instance test functions took two node arguments, some took one, andsome took two but the second argument could be None. This patch makes suchfunctions uniform by using a list of nodes as an argument. This simplifies...
QA: Refactored launching of the various instance tests
The code launching instance tests for "plain" and "drbd" templates has beenunified. In this way it's easier to add more templates, and to re-use the sametests with different configurations.
QA: Run cluster-verify after instance tests
We want to make sure that instance operations don't leave the cluster in abad state. The "instance-remove-drbd-offline" test leaves some debrisbehind, so it's been moved to the last position.
QA: Added functions to allocate/free N nodes at once
These make possible to simplify the logic of tests requiring morenodes. Used in the following patches.
QA: Added exclusive_storage to qa_config
QA configuration now tracks the state of the exclusive_storage flag atcluster level. This will be used to selectively enable tests according tothe configuration.
Also, it's now possible to specify the initial value of the...
QA fix: TestInstanceExportWithRemove() updates configuration
When the instance is removed from the cluster by this test, it's also freedin the QA configuration.
QA cleanup: Removed instance-disk-failure test
The test was broken, out of sync with the rest of the code, and prone tocrashes. Until someone does a better job, it's better to remove thetest. This simplifies further refactoring.
QA: Added disk template to config
The disk templates used by the instances are tracked in the QAconfiguration. This will be used to selectively enable tests according tothe configuration.
QA: Better initialization of a global dictionary
The global 'cfg' variable is initialized to an empty dictionary, so thereis no more need to disable some pylint checks.
QA: Cluster-verify reports shared PVs with exclusive storage
Exclusive storage forbids sharing PVs between unrelated LVs. This is a testthat cluster-verify correctly report such cases.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
QA: Added constants for LVM volumes
The LVM volume group becomes a configurable parameter, and a prefix forlogical volume names used in tests is introduced as a constant.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix logging arguments formatting
This fixes: “ganeti/hypervisor/hv_kvm.py:93: [W1201, _GetTunFeatures]Specify string format arguments as logging function parameters”, andalso updates the copyright years.
QA: Templates incompatible with exclusive storage reported
Cluster-verify should report instances whose disk template is notcompatible with the exlusive storage setting.
QA: Uniformity check for exclusive_storage in cluster-verify
Cluster-verify should report nodes in the same nodegroup with differentvalues of the exclusive_storage flag.
QA: Test for basic features of exclusive storage
Check that creation of instances doesn't trigger any error.
QA: Added function to read cluster configuration fields
This function will be used in following patches.
QA: Added function to check cluster-verify result
Now it's possible to check that cluster-verify reports all the expectederrors, instead of simply checking that it fails.
Document better gnt-network
Specify clearly that the nic parameters depend on thenetwork-to-nodegroup connection.
Clarify in gnt-network how instance nics are connected to a network.
Fix the gnt-instance section in the htools manpage
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Update 2.7 design doc
Move two designs that have been implemented in 2.7 to design-2.7 and outof drafts.
Other designs were partly implemented: explain what part was, and leavethem as drafts pending their completion.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
update 2.7 NEWS file
Include a handful of important changes that weren't mentioned.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Disable live-RPC queries under split query
Currently, the node listing RPC is very slow due to missingparallelisation. For the 2.7 release, we reset these back to masterd,hoping to revert them by the time 2.8 is ready.
There are a number of queries that I've left pointing to confd, as...
harep: create repair jobs
Implement 'doRepair' to create a repair job from a list of opcodes ifthe instance's policy allows it (otherwise set an ENOPERM result label),and the instance was previously healthy (i.e. not in ArFailed orArPendingRepair)....
Document that OpTestDelay/TestDelay take duration in seconds
Also, fix @rtype and @return elements of utils.TestDelay(), which nowreturns a tuple but this wasn't being indicated.
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
harep: do not wait for repair job completion to set tags
Because of instance locks, after submitting a repair job we weren't able toset the "pending" tag until at least the first opcode of the job finished.Introduce a small delay in the repair job so as to allow the subsequent...
harep: finish execution with a summary of instance states
The harep tool prints messages for every action that it performs (or thatit doesn't perform). In case nothing is to be done at all, always printsome statistics of the current state of the cluster....
harep: initial parsing of tags
Parse auto-repair tags to set each instance in one of ArHealthy, ArFailed,or ArPendingRepair. The implementation tries to be well behaved when oldtags have been left behind, which future patches will still try not to do....
harep: check for completed jobs at the start of the program
As a first step before detecting any brokeness with instances, see if anyof our previous repairs have completed, and move instances to ArFailed orArHealthy accordingly. Do nothing if there are still running jobs for the...
harep: pure function to detect brokeness with instances
Add a 'detectBroken' function that determines whether an instance is in anunhealthy state, and what's needed to repair it. The repair is specified asan AutoRepairType constant, and a list of opcodes. The opcodes will only be...
Program/Harep.hs: add skeleton for the new auto-repair tool
harep(1) detects certain kind of problems with instances and applies theallowed set of solutions. See doc/design-autorepair.rst.
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix typo ("Abrreviation" -> "Abbreviation" in Common.hs)
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Jobs.hs: add a submitJobs function
This new 'submitJobs' function doesn't wait for job completion, it justreturns the job IDs immediately.
Jobs.hs: add an execJobsWaitOk function
This new 'execJobsWaitOk' returns "Bad" not only if an error occurs, but ifany of the submitted jobs doesn't succeed.
Jobs.hs: start with a shorter delay in waitForJobs
Instead of waiting 15 seconds every iteration of the waiting loop, startwith 0.5 seconds and increment exponentially until a certain maximum isreached. Keep the maximum at 15 seconds.
Signed-off-by: Dato Simó <dato@google.com>...
HTools/Types.hs: minor adjustments to auto-repair types
In particular:
- make ArHealthy take an optional AutoRepairData; this allows to represent the situation where a repair completed successfully, and hence there's an associated tag we might want to know about....
Force conflicts check in LUNetworkDisconnect
Until now if one disconnects a network with --no-conflicts-checkand then remove it, there is a possibility to leave instances with NICsreferencing non-existing networks. This causes network queries,instance removal and modification to fail....
If _UnlockedLookupNetwork() fails raise error
Make _UnlockedLookupNetwork() raise OpPrereqError (instead of returningNone) in case it does not find the requested network. Remove useless andduplicate code such as:
if net_uuid is None: raise...
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Removes check for conflicts from NetworkDisconnect
This removes the check for conflicts from the Haskellversion of the OpCode NetworkDisconnect. This alignesthe Haskell code with the patch"Force conflicts check in LUNetworkDisconnect" (whichis currently under review). I will submit these patches...
hv_kvm: Refactor getting TAP features, add tests
Split retrieving supported features into a dedicated function which canbe mocked. Tests are added for both “_ProbeTapVnetHdr” and“_GetTunFeatures”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
hv_kvm: Original error message, keyword parameter
- Include original error message when creating TAP interface failed- Pass keyword parameter as such
Checks for hypervisor constants
Add two tests for hypervisor constants.
CLI.hs: fix double spaces in option help strings
Some help strings with continuation backslashes ('\') were providing aspace both before and after the backslash, resulting in double spaces inhelp output. Provide it only after the backslash, which fixes the issue and...
Add fields 'inst_cnt' and 'inst_list' to network queries
This adds the fields 'inst_cnt' and 'inst_list' to theHaskell implementation of the network queries.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Moving network query helper functions to Network.hs
So far, a couple of helper function for the networkqueries resided in Config.hs. I figured it makes moresense to move them to Query/Network.hs, since they arevery tailored to that purpose.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Add new NIC parameter to man page of 'gnt-instance'
According to doc/design-network.rst, there is a new NICparameter 'network', which can be used to link an instanceto a network. This was so far undocumented in the manpage of 'gnt-instance'.
More fields for network queries
This adds more fields to the network queries:- group_cnt, free_count, reserved_count, and map
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix spelling issue in hroller.rst
'explicitly' was spelled the wrong way
Signed-off-by: Benjamin Kerensa <bkerensa@ubuntu.com>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
hv_xen: Fix issues with migration, add tests
Commit 3d942d8 broke instance migration (“self._cmd” was set to None).This patch fixes that issue, refactors “MigrateInstance” for testing andadds those tests.
kvm: fix bug while fetching -device list
_GetKVMOutput expects the command to succeed, but unfortunately on someversions of kvm "-device ?" will output a correct list of devices, whileexiting with an error code.
To fix this we accept failure in that case (note that this doesn't...
hv_xen: Test various forms of getting instance/node info
Test functions for retrieving instance and node information.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
hv_xen: Test stopping an instance
Test stopping an instance, and ensure the configuration file stillexists after an instance couldn't be stopped.
hv_xen: Test starting an instance
Test starting an instance in normal mode, in paused mode, and in case offailure.
hv_xen: Test reading non-existent config file
Check whether doing so raises an exception.
hv_xen: Test verifying hypervisor
Test success and failure.
hv_xen: Test removing config from "auto" directory
Test whether any existing configuration file named after the instance isremoved from Xen's “auto” directory.
Establish base for testing Xen hypervisor abstraction
There are two separate Xen hypervisors (HVM and PVM), as well as twodifferent Xen commands (xl and xm). This already provides four differentcombinations and future changes might bring even more. For this reason...
hv_xen: Test errors while writing config file
Test error handling while writing Xen configuration files.
hv_xen: Test using unknown Xen command
This tests “unknown” values of “constants.XEN_CMD”.
hv_xen: Split StopInstance
hv_xen: Abstract running Xen commands
Instead of using the “XEN_CMD” constant in multiple places, that is nowall in a single place and can easily be changed for unit tests (througha parameter given to the constructor).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
List networks: call client explicitely
This way, it is easier to make it call the haskellimplementation of the network queries.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
First part of Network Queries in Haskell
This is the beginning of the implementation of networkqueries. This includes establishing all infrastructureto run the network queries and implement querying ofsome simpler fields and the node group listing.
Convert Maybe results to RSUnavail
When displaying query results of type Maybe, one could use thefunction rsMaybe. Unfortunately, this function maps 'Nothing'values to RSNoData which gets displayed as '?' in the list ofquery results. These semantics do not fit if the result is of...
Extend config by networks and networks by UUIDs
For network queries to work, we need to extend the generalconfig type to include all available networks. Additionally,we add UUIDs to the network type itself.
Typo in comment of network type
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_xen: Remove config after shutdown was successful
If stopping an instance failed, the configuration would already be goneand other operations depending on it (e.g. migration) would no longerwork. With this patch the configuration file is only removed once the...
hv_xen: Simplify writing configuration
Instead of calling _WriteConfigFileStatic from both derived classes,those now only return the content (minus the “do not edit” header). Theconfiguration is then written by the base class, XenHypervisor.
hv_xen: Prepare for unit tests, remove {static,class}method
Unit tests will have to specify a custom Xen configuration directory (atemporary directory). To this end “hv_xen.XenHypervisor” and its twoderived classes are modified to have fewer static and class methods. A...
hv_xen: Factorize and test disk configuration
The “_GetConfigFileDiskData” function is moved to module level andcleaned up (module-level constants for letters and file I/O drivers).
Until now only 24 disks would be supported (e.g. “sda” to “sdx”), when...
hv_xen: Refactor getting node information, add tests
Refactor and add tests for getting node (Domain-0) information.
hv_xen: Refactor running & parsing "xm list", add tests
This patch refactors “_RunXmList” and adds some tests.