Fix 'gnt-group add' output in QA
The command 'gnt-group add' might return a list of jobs, therefore, wecan print the job's output during the QA, in a way similar to that of'gnt-cluster verify', for example.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Merge branch 'stable-2.11' into master
Test node certificate renewal in QA
This extends the QA by explicitely testing the renewalof SSL client certificates.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Use node UUID as client certificate serial number
It turns out, that some implementations of OpenSSL are morepedantic in checking the certficates than others. In thisparticular case, the SSL connection could not beestablished when the serial number of the certificates...
Fix watcher tampering with instance userdown QA
The watcher was tampering with the instance user down QA because, whenthe watcher sees an instance marked an 'USER_down', it will cleanupthe instance, that is it will stop the instance, thus marking it as...
Instance communication mechanism QA (for cluster only)
Extend QA with tests for the instance communication mechanism. Thesetests cover only the cluster creating the instance communicationnetwork and connecting it to the existing node group, and also if new...
Fix bracket style in rapi-workload
Pep8 complained about bracket indentation. This fixes it to satisfy Pep8as well as our style guide.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>(cherry-picked from ccded668c8a93fa25aaf21ddf5105270a04fe517)...
Merge branch 'stable-2.10' into stable-2.11
Fix 'UserDown' QA
Increase instance shutdown timeout and make call 'pkill' morereliable, which should fix the KVM QA.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Merge branch 'stable-2.9' into stable-2.10
Add QA tests for RAPI multi-instance allocation
The instance multi-allocation had no tests to detect its breakage, andthis patch fixes that.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Merge branch 'stable-2.8' into stable-2.9
Fix QA flakiness
The newly added job QA has some flakiness with respect to its use ofgnt-job watch. Fix this by waiting until the canceling status isreplaced with the canceled status, or a timeout is reached.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Linting fix: remove unused import
Add QA test for job cancellation
This patch introduces a QA test in which a job is cancelled whilewaiting.
QA for KVM instance shutdown
Add QA for instance shutdown for KVM.
Clean up epydoc comments
Add missing colons, and improve descriptions of parameters.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Use options for turning functionality on/off
Two command-line options are added: one for confirming that the testhas been started intentionally, and one for showing the methodinvocation output, which is useful, but not always needed.
Add job cancellation workload
To examine if jobs can be cancelled correctly, provide workload relatedto this as well.
Add cluster parameter change workload
One of the few leftover unused RAPI methods is the cluster modifymethod. This patch tests it by setting and unsetting various safevalues.
Make an instance move workload that works in 2.6
The instance move workload present before this patch works on 2.11, butfails on 2.6. The 2.11 workload will still be useful should any laterversion of Ganeti use it as reference, but a 2.6 workload has to be...
Add instance move workload
Through the use of functions provided by the rapi QA, all the requestsrelated to instance moves can be exercised.
Allow the skipping of checks for inter-cluster move test
The inter-cluster instance move test is very interesting for the RAPIcompatibility tests, as it uses many RAPI requests that are otherwisehard to exercise. It uses no command-line functionality apart from...
Make the finish function return the error status explicitly
The earlier version of the Finish function assumed that checking if thevalue of the response is None would suffice to check if any errors haveoccurred. This is not, and this patch adjusts the function to expose...
Add migration and failover workload
This patch introduces additional calls adding migration and failoverRAPI operations, moving a DRBD-disk template instance between nodes.
Add tracking of used client methods
As a helper or a warning to anyone extending the RAPI client, theclient wrapper now warns of unused methods or method arguments.
Add network workload
This patch exercises the network RAPI commands.
Add miniature query filtering workload
As query filtering was not a part of the previous workloads, this patchadds a single example of its use.
Add per-resource query workload
The query requests are done to receive data about a certain resourcetype. With tests for all the resources barring networks in place, thequery workloads can be added at the point where the existence of enoughresources in the system can be confirmed, making the results of the...
Add group-related workload
This patch further extends the RAPI workload by exercising all thegroup-related functionality.
Add node-related workload
This patch further expands the workload by performing various nodeoperations.
Add warning about the RecreateInstanceDisks invocation
A test relying on RAPI alone cannot exercise the RecreateInstanceDisksfunctionality properly - simply because it cannot damage an instanceto the point where its disks would be missing and in need of...
Add various single instance operations
To further expand the number of RAPI methods in the workload, thesingle instance operations are added in this patch. An instance iscreated, deleted, shutdown, restarted, reinstalled, renamed, andhas its disks activated and deactivated and grown....
Add tag method testing
This patch adds a generic way to test tagging of various entities viaRAPI. More tags testing will be added as other entitt tests are added.
Add helper function that waits for jobs to finish
Some RAPI calls result in the creation of a long-running job,returning a job id to be used to extract the results later. To reducethe amount of boilerplate, introduce a simple function to do thewaiting....
Add simple retrieval operations to workload
This patch expands the RAPI workload with simple Get* commands.
Add the first version of the RAPI workload script
The RAPI workload script supplies work for the RAPI compatibilitytests. The initial version does very little, but can be expandedas needed.
Make the qa_rapi setup method return the RAPI client
Move RAPI secret lookup to qa_rapi
The RAPI secret lookup is a helper function used by the Ganeti QA toretrieve the RAPI password of an already setup cluster. As this couldbe useful to other utilities performing QA, move it to the qa_rapimodule.
Catch exceptions when calling curses.setupterm() in QA
If it's running on a non-standard terminal, such asrxvt-unicode-256color, the call fails with an exception. Instead, catchthe exception and proceed without coloring warnings/errors.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Fix types for queries in QA
Due to the actual implementation of the '?' operatorin our query language, it happily accepted essentiallyany value that was not 0 or False as being true. However,it was always only specified to work on boolean values.Therefore, our QA shouldn't test for this unspecified...
Gluster: add the Shared File storage type
The shared file and gluster disk templates should not report their diskspace information like file does, because they do not behave the same.
If a cluster pulls from the same, shared source of storage then it is...
gnt-cluster verify: demote orphan volume error to warning
Ganeti checks for orphan volume by making sure that it knows about allvolumes on disk; any additional orphan volume, even if created by theadministrator, causes a failure in gnt-cluster verify. Given that...
Add network tag tests to QA
The QA did not have a test for network tags until now, and this patchremedies the situation.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Activate QA for rapi queries via luxi
This patch enables QA testing for rapi queries for thenewly transformed queries from python to haskell(groups, instances, nodes, export, and networks). So far,the QA did not distinguish between resources that cannot be...
Use the generic query field test for network QA
Instead of just using sporadic 'gnt-network list' commands,use the more exhaustive generic query test functions forlist and list-fields.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add gnt-network list to network QA
The QA for networks so far did not cover network queriesand as such issues with those could not be discovered.
Add patching QA configuration files on buildbots
Add an IPv6 check for the QA test of custom SSH ports
If any of the nodes involved would be reached through IPv6, the test isskipped. This is because the required testing setup using "iptables" issuiteble only for IPv4 and doesn't work for IPv6 connections....
Show running times of whole test blocks
Similarly to simple tests, test blocks are now demarcated (in boldface)and measured.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Add more color options to QA
Allow combination of colors, background colors, bold, reverse video,etc.
Add more thorough check for terminal color support to QA
It uses `tput` to check if the current terminal has enough colors.
Add a simple QA test for a group with a custom SSH port
The test is turned on by setting `"group-custom-ssh-port": true` inthe config.
Unlike `"ssh-port": ...` in "options", this is a single test thatdoesn't affect the rest of the tests.
Export `NodeAdd` and `NodeRemove` from qa_node
This is needed for testing SSH ports so that we can remove a node froma group, add it to a group with a custom SSH port, and then undo thechange.
Allow QA node acquisition and group creation using `with`
For acquiring nodes use `with AcquireManyNodes(num): ...`. The nodeswill be released automatically.For creating a new group use `with NewGroupCtx() as group: ...`. Thegroup will be removed automatically....
Update QA to test custom SSH ports
This patch adds the `ssh-port` option. If set to a non-standard port,the QA script sets up the default node group with this port, and beforerunning tests it adds `iptable` rules to all nodes so that the nodessee each other's SSH servers as running on this port....
Add QA tests for instance shutdown detection
Add QA tests for instance shutdown detection with Xen PVM and HVMhypervisors.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Remove hardcoded references to File, SharedFile templates
DTS_FILEBASED is a constant that exists and this commit makes surethat it is used whenever sensible, rather than resorting to hardcodingthe pair of templates in very many files.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Add visual/color marks to the QA script
Add the function name to textual descriptions of tests
It makes it a bit easier to find the corresponding functions.
Signed-off-by: Petr Pudlak <pudlak@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Merge branch 'stable-2.10' into master
Minor changes regarding hotplug support
Extend qa to test hotplug support
Only if default hypervisor is KVM try all hotplug related actions.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
QA: run all the instances during upgrade
The parameter 'upgrade-instances' was meant that one instance ofeach of these templates is left alive over the upgrade procedure.Threfore, breaking after the first suitable template is found isnot helpful.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Fix lint on qa_cluster.py
pep8 wants two blank lines after TestUpgrade. The previous commit gaveit one. This commit gives it the second.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Fix end-of-line whitespace
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Run the cluster upgrade test in qa, if enabled
The test is not run directly after the cluster related tests,but only after basic istance creation is verified, as this testalso creates instances.
Add QA routine to test 'gnt-cluster upgrade'
The subtle points of upgrading a cluster can only be found inQA, so add a method to do so. A prerequisite is, that bothversions are installed on the test cluster. To have a realisticscenario, instances are created on both versions....
QA: Factor out the list of available instance test
The knowledge about which instance creating tests areavailable is best kept in the module on istances. Moreover,it is not only used by RunInstanceTests, but willalso be needed by the upgrade test.
Verify that ConfD is running after master-failover
This patch adds additional 'gnt-node list' commands tothe testing of 'gnt-cluster master-failover' in order totest if ConfD (or LuxiD) is still running after amaster-failover.
Signed-off-by: Helga Velroyen <helgav@google.com>...
QA: remove subsumed QA test wrt disk templates
This patch removes a QA test that was supposed to testthat unused disk templates can be disabled safely. Thetest is now subsumed by unit tests and thus we remove ithere. Besides that, it was actually testing the wrong...
Use 'DTS_LVM' when possible
This patch replaces all usages of the utility function'GetLvmDiskTemplate' by the new 'DTS_LVM' constantto make it consistant with the usage of other DTS_*constants.
Additionally, it provides a unit tests to ensureconsistancy between DTS_LVM and the mapping of disk...
Fix indentation for newer lints
Newer versions of pylint are more strict regarding indentation rules. Thiscommit fixes a couple of indentations that they consider wrong.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Merge branch 'stable-2.9' into master
Hs2Py constants: update Python references
Update references to constants that were previously in the generated'lib/_autoconf.py' and since the previous patch have been included inthe Hs2Py constant generation.
Fix some wrong indentations in the code
Fix all instances of pep8's error: "E128 continuation line under-indented forvisual indent".
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix an improperly escaped string
Mark a string representing a "sed" command including a regular expression as araw string, so that the entire escaping is performed by "sed" itself.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Make QaConfig a proper container
Newer pylint versions complain that QaConfig is not a proper container becauseit does not implement the len(), delitem() and setitem() methods.This commit implements them.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Provide PYTHONPATH to burnin
One feature of the new private-module layout is that a pythonscript cannot simply 'import ganeti'. The reason is that theinterfaces to these modules are not stable and hence should notbe used outside Ganeti. However, this is what the burnin test...
Replace LD_* constants with DT_* constants
LD_* constants are basically like DT_* constants, exceptfor that both file and shared file were mapped to file.In order to not having to maintain three slightly differentsets of disk-related constants (DT, LD and ST), we merge...
QA: fix file storage QA wrt ipolicy
Ipolicy-allowed disk templates are now dependend on thecluster-wide enabled disk-templates. The QA for filestorage did not consider this updated logic so far.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
CLI: actually remove the --no-drbd-storage option
This removes the '--no-drbd-storage' option completelyfrom the command line interface. The hugepages designdoc still had a left-over reference to --no-lvm-storage,which is removed in this patch as well....
cluster init/modify: set helper without DRBD being enabled
The logic around the DRBD usermode helper so far was thatsetting it was only possible when DRDB was not explicitelydisabled. This patches changes it in a way that it isconsistent to how Ganeti handles the volume group name....
QA: adapt changes wrt DRBD usermode helper
This adjusts and extends the QA for 'gnt-cluster modify'with respect to the changes regarding the DRBDusermode helper.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Add 'SF_NODE' to 'VALID_STORAGE_FIELDS'
Constant 'SF_NODE' is a storage field and, therefore, should be in'VALID_STORAGE_FIELDS'. This patch fixes this and also reference tothese constants, namely, in 'cmdlib' and 'qa'.
Remove storage field duplication in 'qa'
QA was adding 'SF_TYPE' to a list containing all storage fields (i.e.,'VALID_STORAGE_FIELDS'). However, 'SF_TYPE' is already part of'VALID_STORAGE_FIELDS', which means this element was occurring twicein the list....
QA: make ipolicy test respect enabled disk templates
This makes the 'gnt-cluster modify --ipolicy-*' testsrespect the enabled disk templates on the QA cluster.If the required disk templates are not enabled, thetest is skipped.
QA: adjust tests wrt to ipolicy disk templates
This adjusts the cluster QA with respect to the new checksbetween the ipolicy disk templates and the enabled disktemplates.
Use exact option name --enabled-disk-templates consistently
It is possible to use both, --enabled-disk-template(singular) or --enabled-disk-tempaltes (plural),but the usage of it has been documented and used in QAinconsistently. This is fixed by this patch now....
QA: skip lvm-based tests if lvm disabled
QA tests should make sure not to be run if the necessaryenvironment for them is not present. This patch fixes thatfor a couple of tests that fail to check that before beingexecuted.