Make QaConfig a proper container
Newer pylint versions complain that QaConfig is not a proper container becauseit does not implement the len(), delitem() and setitem() methods.This commit implements them.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Replace LD_* constants with DT_* constants
LD_* constants are basically like DT_* constants, exceptfor that both file and shared file were mapped to file.In order to not having to maintain three slightly differentsets of disk-related constants (DT, LD and ST), we merge...
QA: fix file storage QA wrt ipolicy
Ipolicy-allowed disk templates are now dependend on thecluster-wide enabled disk-templates. The QA for filestorage did not consider this updated logic so far.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
QA: make ipolicy test respect enabled disk templates
This makes the 'gnt-cluster modify --ipolicy-*' testsrespect the enabled disk templates on the QA cluster.If the required disk templates are not enabled, thetest is skipped.
Signed-off-by: Helga Velroyen <helgav@google.com>...
QA: adjust tests wrt to ipolicy disk templates
This adjusts the cluster QA with respect to the new checksbetween the ipolicy disk templates and the enabled disktemplates.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Use exact option name --enabled-disk-templates consistently
It is possible to use both, --enabled-disk-template(singular) or --enabled-disk-tempaltes (plural),but the usage of it has been documented and used in QAinconsistently. This is fixed by this patch now....
QA: skip lvm-based tests if lvm disabled
QA tests should make sure not to be run if the necessaryenvironment for them is not present. This patch fixes thatfor a couple of tests that fail to check that before beingexecuted.
QA: test for --{shared,}-file-storage-dir
This adds tests for 'gnt-cluster modify' wrt to theoptions --shared-file-storage-dir=... and--file-storage-dir. It also separates the two directories'specification in the QA configuration.
Rename VALID_STORAGE_TYPES to STORAGE_TYPES
For consistency, this patch renames 'VALID_STORAGE_TYPES'to 'STORAGE_TYPES', because the set of valid disktemplates is also just names 'DISK_TEMPLATES'.
Rename disk_template/storage_type map + cleanup
This patch renames the mapping of disk templates to storagetypes to MAP_DISK_TEMPLATE_STORAGE_TYPE, which is a moreexpressive name since it indicates that it is a mapping anddoes not have the inconsistent usage of singular and plural...
gnt-cluster {init, modify} --file-storage-dir
This patch implements consistent usage and behavior ofthe --file-storage-dir option in 'gnt-cluster init'and 'gnt-cluster modify'. It includes a bunch of unittests as well.
Additionally, it enables the previously written unit...
Prepare QA for shared file storage
This prepares the QA for running tests regardingshared file storage.
Merge branch 'stable-2.8' into stable-2.9
Rename queryd to luxid
As queryd will, in the future, handle all LUXI request, queue jobs andmost likely perform various other tasks, it is renamed to luxid already.This will safe some headache when upgrading Ganeti installations, as wedon't have to deal with a daemon rename....
Add ganeti-queryd to QA env test
Test for the newly introduces ganeti-queryd in the environment test ofQA.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Revert "Disable failing QA test"
This reverts commit b2525349f0da75c74432f2dd85abbcad56e56fed.
Issue 516 was fixed, so now the QA test can be re-enabled again.
QA: Adjust QA for file storage
This improves the QA code wrt to file storage in thefollowing ways:- It fixes a FIXME to make the file storage directory customizable in the QA config. Fallback is the default value.- It makes QA derive the enabled storage types from...
QA: Adjust test whether file storage is enabled
This patch fixes the node storage QA, which checked whetherfile storage is enabled by looking at the constant'ENABLE_FILE_STORAGE'. QA should rather retrieve thisinformation from the QA config.
Disable failing QA test
See issue 516 for an explanation of the problem, and why it was requiredto disable this test.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Disable enabled-template-test for exclusive storage
This patch disables the tests regarding enabled disktemplates for exclusive storage, because the semanticsthere are different and not covered in the test.
Revert "QA: Make Query tests test for '?' values"
This reverts commitbfbef99b9f056638411aa894e300aa095132e931. The problem is thatin some cases it makes sense for '?' to be displayed. Thisis the case for storage reporting on clusters that have onlythe 'diskless' disk template enabled. It would be nicer,...
Add QA test for verify-disks with broken DRBD
gnt-cluster verify-disks supports automatic activation of broken DRBDdisks. This new QA test verifies that it actually finds instances withbroken DRBD disks and activates the disks of those instances.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
QA: Make Query tests test for '?' values
When something goes wrong in the Haskell implementation ofqueries, the 'gnt-* list' command does not actually fail,but it just displays '?' as a sign of failure. We so fardo not test for that, which makes errors in the haskell...
Enable disk growth with exclusive storage
Disk growth is now supported also with exclusive storage, so we let theoperation continue.
Enable disk growth test in qa also with exclusive storage.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix lint error
Unset LV name if no LVM disk templates are enabled
If there are no LVM based disk templates enabled in QA, reset the LVname after the corresponding gnt-cluster modify tests. Otherwisegnt-cluster verify would complain about the missing volume group.
Also make sure that no disk templates requiring LVM are ever enabled...
Skip drbd-usermode-helper test if DRBD is disabled
Changing the drbd usermode helper in gnt-cluster modify fails if DRBD isnot installed on the nodes. Thus skip this test if DRBD is not in theset of enabled disk templates.
Replace tempfile by mktemp in QA code
The tempfile command only exists in Debian. In order to run the VClusterQA also on Fedora, replace its use by mktemp.
Merge branch 'stable-2.8' into master
Fix node modify QA tests for large cluster
If the node count in a cluster exceeds the master candidate pool size(which is 10 by default), it's no longer possible to remove the mastercandidate role from a node without passing --auto-promote (even with--force). This behavior is perfectly fine, but the TestModifyNode QA...
Merge branch 'stable-2.7' into stable-2.8
Fix daemon QA tests
TestInstanceConsecutiveFailures now leaves the test instance in the samestate it got it.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add gnt-node migrate to QA
Make it possible to optionally run gnt-node migrate during QA.
Skip Xen monitoring QA for non-Xen clusters
Instead of asserting that the Xen instance status collector fails fornon-Xen clusters just skip the test completely.Otherwise QA would fail for non-Xen hypervisors if Xen happens to beinstalled on the cluster as well....
Adapt VCluster script to current situation
vcluster-setup is adapted to the current Ganeti command line arguments.Also, a bug for diskless instances in qa_instance is fixed.
Merge branch 'stable-2.8' into 'master'
Export node spindles
Node spindles (queried live) are exported through the LUXI, RAPI, andiallocator interfaces.
Support RAPI QA on an already existing cluster
We support running most of the QA tests on an existing cluster. One bigexception was RAPI, as the credentials for the QA RAPI user were created atcluster setup. This patch allows to recover the password for the QA user...
Export instance disk spindles
Instance disk spindles can be queried via LUXI and RAPI, and are exportedthrough the allocator interface. This is a prerequisite for htools tohandle spindles.
The length of a RAPI query including all the instance fields now has become...
Spindles are mandatory with exclusive storage
If they are not specified in the command line an error is reported. Also,disk creation would fail without them.
QA has been updated.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Fix release of instances in QA
Also, fix some uses of keyword arguments for arguments with a defaultvalue, as required by the style guide, and a typo.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix missing "raise" in QA
The bug was introduced in 345d395d.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
QA for spindles in creating disks
"gnt-instance add" and "gnt-instance recreate-disks" are tested giving thenumber of spindles, when supported. Also, QA for "gnt-instancerecreate-disks" now covers the case where disks are resized.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Add QA for recreating single instance disks
So far QA only recreated the whole set of disks at once.
Add QA for gnt-instance modify --disk
Just a very basic test that adds and then removes a disk.
Monitoring QA: Remove superfluous import
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Non-Xen support for monitoring QA
The QA tests the Xen instance status collector, but that is expected to failwhen run on machines that do not use Xen.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add QA for instance status collector
This commit introduces the QA for the instance status collector.
Begin the first QA for a monitoring-related component, the files and somefunctions are named after monitoring because they are meant to containfuture monitoring QAs as well....
QA: factor out some instance management functions
Some functions for managing instances will have to be used by new upcomingunit tests, so they are taken out of the instances QA file and put in a newutilities file accessible by other QA files as well....
QA checks suitability for exclusive_storage tests
When instance tests with exclusive_storage are enabled, the number of PVson nodes are checked to see if it's suitable for those tests.
This fixes issue 437.
Merge remote-tracking branch 'origin/stable-2.8'
Conflicts: lib/bdev.py lib/cmdlib.py
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Disk templates are "allowed" and not "enabled" in ipolicies
The way disk templates were displayed in instance policies became confusingsince the introduction of the enabled_disk_templates cluster parameter.
Fix issue 440.
Cleanup DRBD 8.4 devices properly during QA
QA cleans up DRBD 8.4 devices via drbdsetup, which currently only workedfor DRBD 8.3. This (rather ugly) command makes sure that DRBD 8.4devices are cleaned up as well.
QA: Test multiple instance specs
The test of the interaction between policy and instances is extended.
QA: Handle multiple instance specs
Existing tests are updated to cope with the new instance specs format.
QA: Transpose instance specs
The format used to store instance specs in QA is changed to better handlemultiple instance specs, a feature contained in the following patches.
Make DEFAULT_ENABLED_DISK_TEMPLATES a list
This patch makes the constant DEFAULT_ENABLED_DISK_TEMPLATESa list instead of a set, because the order of the elements actually mattersas the first one is planned to be the default disk template for instancecreation. Also, this constant is always used like a list and thus providing it...
gnt-cluster modify/init: deprecate --no-lvm-storage
This patch does three things:
1. It deprecates the option '--no-lvm-storage' of 'gnt-cluster modify'and 'gnt-cluster init'. Technically, it is not fully removed, but kept in orderto warn the user that it is no longer supported and that she should use...
QA: Update tests for new ipolicy specs command-line options
The old command-line options weren't working any more. Utility functionsare updated, and broken tests are removed (new tests are coming in thefollowing patches).
QA: Split function to set and parse instance policies
The main parts, moved to qa_util, will be used for node groups too.
QA: More tests for instance policies in groups
Instance policies are changed and checked for changes.
QA for --ipolicy-xxx-specs and show-ispecs-cmd
The tests check the interactions between the two commands.
Also refactored group QA code to separate tests about ipolicy, as thosetests are going to grow in complexity.
QA: Skip RAPI setup when RAPI is disabled
This is similar to 405692ef, reverted in aeefb4cd.
QA: Fix error regarding missing names of disk configs
This fixes an error in case there are no disk names in the QA config(which was the case till recently). Right now, QA tries to use 'None'as disk name and that makes instance creation fail.
Revert "Make invocation of rapi test configurable ..."
This reverts commit 405692eff3d6f8400834f2df93d597bc7e4dd89f, because itbreaks QA.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Make invocation of rapi test configurable by QA config
This does not solve the underlying problem of the rapi test notbeing runnable locally, but it makes it at least configurableto avoid having to patch in/out all the time.
Update QA tests to check disk and NIC names
Modify existing tests to use the name option for instance disks. Theconfiguration options 'disk' and 'disk-growth' are merged into a single'disks' option, which is a list of dictionaries with 'size', 'growth' and...
QA: Test enabled disk templates with known configuration
The test failed because it was executed also in a configuration that wasn'texpected, i.e., with exclusive storage enabled.
Also the docstring is shortened, as it was being truncated in the QA output....
QA: Decision about disk templates to use is more modular
qa_config.IsTemplateSupported() is already used to decide which disktemplates are supported by QA. Enabled templates now are part of thatdecision.
Unit tests are updated.
QA: creating instances of dis/enabled disk templates
This patch makes sure that instance creation is only possiblefor instances that use enabled disk templates.
QA: dis/enabling of used disk templates
This extends the QA for 'gnt-cluster modify'. It ensuresthat disabling of a disk template is only possible if thecluster does not have any instances using the template.
qa_instance: refactoring wrt disk templates
This patch is a preparation for later patches in QA in this seriesit refactors the instance QA to make it more flexible regardingcreation of instances with different disk templates. Right now weonly support creation of instances with disk templates 'drbd', 'plain',...
QA: Test cluster init/modify wrt enabled disk templates
Fixes the QA of 'gnt-cluster init' and 'gnt-cluster modify' with respectto enabling disk templates.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtarara@google.com>
qa: fix TestInstanceModifyPrimaryAndBack flakiness
The test tries to scp the instance disks from the primary node to thenew destination, but connects to the master node instead. Thus it worksonly when the primary node is the master node. Fixing by correcting to...
qa: Fix mistake in _DestroyInstanceDisks
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
qa: fix instance recreate disks for ST_FILE disks
More lvm commands "sprinkled" in qa :/ We really have to factor them outin the storage layer, once we have a good one.
qa: skip gnt-backup for file instances
Note that this fixes the "current issue" but doesn't fix the underlyingproblem. :/
qa: fix RemoveInstanceOfflineNode for file-based
- The cleanup should only happen for non externally mirrored disk templates- It should behave differently for file and lvm based instances- It should definitely not be explicit in the qa file :/ (but it's...
qa: disable inter-cluster-instance-move for file
This actually hides a real issue that was discovered by enablingfile-based qa, so bug was filed for it. On the other hand it's betterthan keeping the qa broken, since this functionality was broken anyway....
qa: fix rename instance for non-lvm backed storage
The rename instance test performs an lvm-specific check for tags on theinstance volume. This should be in the future abstracted inside thetemplate class, and not explicitly called from qa.
This would allow other disks to perform the checks that make sense for...
qa: fix modify primary test
This test uses a non-existing config value "file-storage-dir" and fails.Fixing it by using the default value, which is currently what qa does.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
qa: pass some extra options to make scp work
scp by itself won't work on a cluster: a few extra options are needed totell it where to find the "right" ssh global known hosts file managed byGaneti.
gnt-node info uses a revised format
The code is more modular, and the output is YAML-compliant. QA has beenupdated to reflect the new format.
Add QA for gnt-instance modify --new-primary
Add a simple walk through the intended use case ofgnt-instance modify --new-primary for the disk templatefile. The tested scenario is shutting down an instance,manually moving the disk to the new node, setting the...
Additionally support disk-type file in QA
Also perform QA tests for the file disk template. The main reasonis that testing gnt-instance modify --new-primary requires amanual move of disks, which is most easily simulated for files.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Refactor ispecs in ipolicy structures
Minimum and maximum instance specs are put together into a single elementof the instance policy. This is in preparation for introducing multiplemin/max specs.
QA: Fix assertion
The assertion was always true because "all" was missing.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
gnt-instance info uses a revised format
The code is more modular and the output is YAML-compliant. QA updated totake advantage of it.
gnt-cluster info uses a revised format
The code is more modular and the output is YAML-compliant.
Added function in QA that uses PyYAML to parse the command output, and QAis updated to take advantage of it (instead of using lots of complicatedREs).
QA: Extend cluster QA wrt enabled storage types
This extends the QA scripts to test the changes in 'gnt-cluster init','gnt-cluster info', and 'gnt-cluster modify' with respect to enabledstorage types.
Add constants for storage types to constants.py
This patch adds the currently known and used storage methods to theconstants.py file. Note that storage types are different from disktemplates. For example both disk templates, drbd and plain, use lvmstorage types....
Fix QA: skip reinstall for diskless instances
Diskless instances cannot really be reinstalled, not even through RAPI.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.7'
Add QA for instance creation with policy violation
When instance policy is violated, creation fails.
Add QA for policy-instance interactions
Violations on policy changes are checked.
Add QA for cluster policies
qa_cluster.TestClusterSetISpecs() is exported as it will be used in futuretests.