QA: Convert nodes to objects
Up until now nodes were stored as a dictionary. The keys were hardcodedin a lot of places and entries modified directly.
This patch introduces a new class for nodes in QA named “_QaNode”. Itstill supports accessing details via dictionary syntax, but that will be...
QA: Release instances through method
Instead of having a module-level function in “qa_config” to release aninstance, a new method on instance objects is used.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
QA: Convert instances to objects
Up until now instances were stored as a dictionary. The keys werehardcoded in a lot of places and entries modified directly.
This patch introduces a new class for instances in QA named“_QaInstance”. It still supports accessing details via dictionary...
QA: Release instance in ganeti-qa
Release instances directly from “ganeti-qa” instead of doing so in thethree tests removing instances. This is in line with how nodes arereleased and makes it easier to find and debug double-release issues.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
QA: Stop using "in" operator for node
The new configuration object for nodes will not support it anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
QA: Support entity objects in qa_utils._GetName
This is a temporary solution until nodes and instances are converted toobjects with attributes.
qa_config: Simplify IsTemplateSupported
The same code can be written in a single expression. The earlier patchnamed “Refactor storage of runtime exclusive storage flag in QA”provided the unit test.
Refactor storage of runtime exclusive storage flag in QA
This is a follow-up for “qa_config: Remove exclusive storage flag fromconfig”. Instead of storing the flag in a module-level variable it isnow stored within the new QA configuration class and unit tests are...
Refactor QA configuration code
Ever since its introduction (sometime before commit cec9845 in September2007), the QA configuration was stored in a dictionary at module-levelin “qa/qa_config.py”. The configuration was loaded, verified andevaluated using module-level functions. Since then the configuration has...
qa_config: Remove exclusive storage flag from config
Commit 6a0f22e added the ability to track the status of the cluster-wideflag for exclusive storage. It did so in a way which modifies thein-memory configuration dictionary. This patch changes the code to use a...
Merge branch 'devel-2.7'
Redirect output for gnt-* list/list-fields commands
This enabled the redirection; QA output after this patch should becomemuch much smaller (on a 'quick' QA, we are left with ~0.5MB output onstdout and 64MB redirected output).
Signed-off-by: Iustin Pop <iustin@google.com>...
Add function to execute QA commands with redirected output
Some QA commands generate non-trivial output: for a full QA, the gnt-*list commands generate more than 300MB of data. To make the QA logsmore parseable by humans, we need support for redirecting output of...
Implement option to skip logging of QA commands
This patch adds an optional kwarg that allows skipping the "Command:…" line in QA output. This is helpful if we only need to run somemaintenance actions, that are QA-related and not Ganeti-related.
QA: Clean up after "instance-remove-drbd-offline" test
This test used to leave a node with orphan logical volumes and unused DRBDminors. Now, the expected leftovers are cleaned up after the test, andcluster-verify is run.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
qa_config: Remove useless "options" variable
It is only used in one place and not necessary.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
QA: Run instance tests with different cluster configurations
Instance tests are run with or without the exclusive-storage flag set. Moreconfigurations can be added easily.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
QA: Instance tests check the configuration before running
Instance test functions check the current configuration, and they will runthe actual tests only if the configuration supports them. This will beused for refactoring in following patches.
QA: Use lists of nodes as argument to instance tests
Some instance test functions took two node arguments, some took one, andsome took two but the second argument could be None. This patch makes suchfunctions uniform by using a list of nodes as an argument. This simplifies...
QA: Refactored launching of the various instance tests
The code launching instance tests for "plain" and "drbd" templates has beenunified. In this way it's easier to add more templates, and to re-use the sametests with different configurations.
QA: Run cluster-verify after instance tests
We want to make sure that instance operations don't leave the cluster in abad state. The "instance-remove-drbd-offline" test leaves some debrisbehind, so it's been moved to the last position.
QA: Added functions to allocate/free N nodes at once
These make possible to simplify the logic of tests requiring morenodes. Used in the following patches.
QA: Added exclusive_storage to qa_config
QA configuration now tracks the state of the exclusive_storage flag atcluster level. This will be used to selectively enable tests according tothe configuration.
Also, it's now possible to specify the initial value of the...
QA fix: TestInstanceExportWithRemove() updates configuration
When the instance is removed from the cluster by this test, it's also freedin the QA configuration.
QA cleanup: Removed instance-disk-failure test
The test was broken, out of sync with the rest of the code, and prone tocrashes. Until someone does a better job, it's better to remove thetest. This simplifies further refactoring.
QA: Added disk template to config
The disk templates used by the instances are tracked in the QAconfiguration. This will be used to selectively enable tests according tothe configuration.
QA: Better initialization of a global dictionary
The global 'cfg' variable is initialized to an empty dictionary, so thereis no more need to disable some pylint checks.
QA: Cluster-verify reports shared PVs with exclusive storage
Exclusive storage forbids sharing PVs between unrelated LVs. This is a testthat cluster-verify correctly report such cases.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
QA: Added constants for LVM volumes
The LVM volume group becomes a configurable parameter, and a prefix forlogical volume names used in tests is introduced as a constant.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
QA: Templates incompatible with exclusive storage reported
Cluster-verify should report instances whose disk template is notcompatible with the exlusive storage setting.
QA: Uniformity check for exclusive_storage in cluster-verify
Cluster-verify should report nodes in the same nodegroup with differentvalues of the exclusive_storage flag.
QA: Test for basic features of exclusive storage
Check that creation of instances doesn't trigger any error.
QA: Added function to read cluster configuration fields
This function will be used in following patches.
QA: Added function to check cluster-verify result
Now it's possible to check that cluster-verify reports all the expectederrors, instead of simply checking that it fails.
Network QA
This adds a script for the QA of 'gnt-network'. So far itcovers adding/removing and connecting/disconnecting networks.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix syntax error in QA's sample config
The additional comma makes this file invalid JSON.
Merge branch 'devel-2.6'
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
QA: Use instance name for rename test
Reported in issue 341. In some setups the instance live in a differentnetblock from the cluster. Therefore a the configuration-global “rename”name shouldn't be used for them. Instead another instance name is used....
QA: Factorize retrieving non-existent groups from config
Also add a check to make sure there are enough groups defined.
QA: Support master-netdev and default NIC parameters
Requested in issue 337.
The parameter “bridge” was not documented and is therefore silentlyreplaced with “master-netdev”. A note is added to “qa-sample.json”describing how comments work.
This is a cherry-pick of master commit...
QA: Do not pass "--bridge" to "gnt-cluster init"
Commit ec0652a (June 2009) removed the option.
This is a cherry-pick of master commit784537390cd5b1c4f5f8fb9401a62eb23acce39b
Fix QA for --offline semantics modification
The previous patch series made it simpler to deal with ERROR_up offlineinstances, but didn't fix qa, that had the old expectations. This fixesthat, and adds more testing about the expected behavior.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Replace frozenset with compat.UniqueFrozenset
This is not a trivial s/frozenset/compat.UniqueFrozenset/, but ratheronly replaces “frozenset” where appropriate. Most of the places are“static” information that doesn't change after the module has beenloaded....
Remove some unused Python code
This patch removes code which is no longer used due to refactoring:
- http.InitSsl, last usage removed in commit 33231500 (“Convert RPC client to PycURL”)- rapi.baserlib.MakeParamsDict, last usage remove in commit 4e5a68f8...
Fixes and improvements to comments
Some fixes, added more information in a few points, removed a stale (5+year old) TODO comment.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fixes to pass pep8 (make lint)
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Iustin Pop <iustin@google.com>
Add a small QA check in instance renames for tags update
We also need to change the signature for _GetInstanceInfo, since wedon't have access to the instance dictionary in the rename instancetest.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
QA: TestAssignNodesIncludingSplit can be disabled
The test can now be enabled/disabled with the "group-rwops" flag.
QA: never put a master node offline
Before, it could happen that the master node was chosen to be set offlinein the "instance-remove-drbd-offline" test. Also changed a finally clauseto make sure that an exception while making the node online doesn't break...
Add option to disable iallocator use in QA
By default some QA tests use iallocators. It is now possible to disableiallocators in QA, so that QA doesn't force having iallocators set up.The affected tests have now alternatives pathways.
qa_utils: Stop hardcoding path to /etc/hosts
- Simplified shell command for adding hosts, too- The exception handling is also improved by re-raising any exceptions instead of swallowing qa_error.Error
Fixed QA for replace-disks
Bug introduced in commit cc4b14f07: Part of a statement was missing. Thismakes pylint-qa fail.
Instance QA uses default iallocator
QA for recreate-disk and replace-disks now run also with the defaultiallocator.
gnt-instance reinstall: Don't always exit with success
If one or more jobs failed the exit status should be set accordingly.
Always_failover doesn't require --allow-failover anymore
If an administrator sets always_failover, it means that there is no needfor another explicit approval to failover instead of migrating.
Migrate qa/*.py from constants to pathutils
File system paths moved from constants to pathutils.
QA for instance migration made more compact
Also, the value of always_failover is not hard-coded any more, and it'spossible to disable changing the always_failover parameter, if needed.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Fix warnings/errors with newer pylint
To help developing Ganeti on newer distributions, let's try to fixpylint warnings/errors. I'm using pylint from current Debian wheezy:pylint 0.25.1, astng 0.23.1, common 0.58.0, and we have 3 things thatneeds fixing....
Fix decorator uses which crash newer pylint
Pylint version:
pylint 0.25.1, astng 0.23.1, common 0.58.0
crashes when passing the fully-qualified decorator name with:
File "/usr/lib/pymodules/python2.7/pylint/checkers/base.py", line 161, in visit_function...
Bump pep8 version to 1.2
Debian Wheezy will ship with this version, and it has many improved checks compared to 0.6, so let's:
- bump version in the docs- silence some new checks that are wrong due to our indent=2 instead of 4- fix lots of errors in the code where the indentation was wrong by 1...
Switch job IDs to numeric
This has been a long-standing cleanup item, which we've alwaysrefrained from doing due to the high estimated effort needed.
In reality, it turned out that after some infrastructure improvements(the previous patches), the actual job queue-related changes are quite...
Add QA test for node group modification of ndparams
Additionally, we run hbal if the htools tests are enabled (newkey). Note that hbal will directly exit, since there are no instances,but at least it will load and parse the group details.
QA for allocator in recreate-disks
QA for recreate-disks called it twice with a list of node; the first callnow uses an allocator.
Signed-off-by: Adam Ingrassia <api@google.com>Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'stable-2.6'
Add some rudimentary node group ipolicy checks
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add a simple QA test for gnt-node list-drbd
This will be run only if 'haskell-confd' QA configuration has beenenabled.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
QA updated to test instance removing when sec. is offline
Added test checking that it is possible to remove an instance with drbddisk when non-primary node is offline.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Ensure that --wait-for-sync is used in QA
We don't have a specific test for activate disks, so let's add it inthe cases where we run (incidentally) activate-disks.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fixed doclint warning
Two blank lines are needed between functions.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
QA: Added test for gnt-instance recreate-disks
The test is disabled by default, as it takes a long time.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
QA function do destroy disks
This is useful to simulate HW problems during QA.
QA: Fix modification of HV parameters via RAPI
This is basically another case of commit d488adb.
QA: Fix breakage on cluster initialization
The hypervisors are a list and need to be comma-separated.
QA: Test some instance parameters depending on hypervisor
For example, the “initrd_path” parameter is only available with xen-pvm,whereas “boot_order” is only available with xen-hvm.
QA: Verify hypervisors, make available through functions
Some parts of QA should only be run with certain hypervisors (e.g.testing instance HV parameters). To make that happen we always definethe cluster's hypervisors and provide a function to get the default...
Fixed typos, formatting, etc and added a comment
This patch only touches comments and man pages.
Fix QA import of existing instances
Commit f346a7d9 broke the TestInstanceImport test in QA, since itdropped the "always generate" parameter for backup import. We add anoverride mac parameter so that, no matter what is defined in theconfig file, imported instances (can) have a different MAC....
QA: Add configuration options for static MAC address
In some QA environments static MAC addresses must be used.
QA: Ignore tags using regular expression
In some QA environments external scripts may add tags. With this patch aregular expression can be used to ignore them during QA runs.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
QA: add toggle for fewer common instance tests
Currently, the "common" instance tests (shutdown, rapi stoppedconsole, stopped modify, startup, rapi shutdown, rapi startup, list,info, modify, rapi modify , console, rapi console, reinstall, rename,rapi rename, rename and back, rapi rename and back, grow disk, reboot,...
QA: stop logging SSH arguments for each invocation
Currently, this is how the QA log looks like:
Command: ssh -oEscapeChar=none -oBatchMode=yes -l root -t -oStrictHostKeyChecking=yes -oClearAllForwardings=yes -oForwardAgent=yes -oControlPath=/tmp/ganeti-qa-multiplexer.DMzkuH -oControlMaster=no node14 exit...
Switch QA to tty-less mode
This makes ssh generate much less noise when QA is run itself withouta tty, as otherwise we get thousands of lines of:
Pseudo-terminal will not be allocated because stdin is not a terminal.
in the QA logs (which are already big enough)....
qa_utils: Factorize code for getting entity name
This reduces code duplication and provides better error messages ifsomething is incorrect while getting a node or instance name.
qa_utils: Accept string as instance for check
Some tests don't have access to an instance dictionary.
QA: Cleanup after commit 5fa0375e4a
Some tests had wrong assumptions on the instance status and someneeded explicit start or stop commands.
QA: Decorate tests with instance checks
Start using “InstanceCheck” decorator where appropriate. Use direct callfor check in places where the instance status changes during the test.
The parameters of “TestInstanceImport” had to be re-ordered to have the...
QA: Infrastructure for hook script to check instance status
This script can be used to check if an instance is running or stopped atvarious points during a QA run. Environment variables are used to passthe most essential information.
QA: Support passing keyword arguments to test functions
Until now this wasn't possible.
Add missing qa/__init__.py file
The file was there and listed in Makefile.am, but I forgot to add it tocommit a0c3e7264a.
QA: Fix shutdown for reinstall/rename tests
The “RunTestIf” uses an AND association between the passed test names,so the instance would be be shut down if both rename and reinstall wereenabled. Since this is not necessarily the case, OR must be used.
QA: Enable use of OR conditions in test checks
Until now “TestRunIf” and “TestEnabled” could only handle AND. With thispatch a new class named “Either” is added to “qa_config” and allows OR.The name “Either” was chosen instead of “Or” as the latter is very close...
Add grow-disk tests in QA