Rename/make uniform the other query entities
Following the new naming style introduced in Exports.hs, this patchrenames the other resources to export non-qualified names (fieldMap asopposed to nodeFieldMap), and to use qualified module imports.
Also fixes a haddock issue in a docstring....
Test AsyncStreamServer with abstract unix sockets
This was meant to be since "the beginning" but was never submitted assomehow it failed with python 2.4. Now that the minimum python versionhas been increased it can be added.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
qa_config: Support callable test checks
Until now all tests depended on simple AND/OR conditions. With theintroduction of virtual cluster support, some tests can only run ifvirtual clusters are not in use (e.g. not yet supported or notpossible). Instead of introducing more logic for enabling/disabling...
QA: Add config entries for virtual cluster
If these new options are set, a virtual cluster is created and used.Virtual clusters consist of 1..N virtual nodes on a single physical (orvirtual) node. The design of virtual clusters is described in“doc/design-virtual-clusters.rst”....
Fix style violations in commit 4a90bd4, small changes
- Commit 4a90bd4 contained a rather large number of style violations: - Indentation/sequence formatting - Wrapping of long lines - Docstrings whose first line was wrapped - A stray backslash in a docstring...
RAPI: Add flag to require authentication
Most RAPI resources do not require authentication for the “GET” method.In some setups it can be desirable to always require authentication.This patch adds a command line parameter to always require it.
Some unrelated minor typos in the “ganeti-rapi” man page are also fixed....
Remove test of obsolete getNetworkUuid method
This will remove the tests for the getNetworkUuid method.The method will be obsolete after the changes regardingidentification of networks by UUID and not name are mergedinto master. Will send another patch for removing the...
Add two utility functions for handling Either lists
These two functions permit operating in bulk on only the Left or Rightvalues in the original list, then reassembling the list back in theoriginal order.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Simplify RPC error cases
This patch removes the node from the RPC error constructursCurlLayerError and OfflineNodeError. The rationale is that we anywayreturn tuples (node, result), and removing this duplication allowssimplified signatures/calls in the execution of RPC calls....
qa_config.AcquireNode: Use sort key function
Use a sort key function instead of using a comparing function (“cmp=…”).The latter is not supported in Python 3 and using a sort key function iseasier.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
qa_config: Add repr for instance/node classes
Having a readable representation makes debugging easier.
Test "GetNodeInfo" for Xen hypervisor
This patch adds unit tests for success and failure of the “GetNodeInfo”method for the Xen hypervisor abstractions.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
QA: Add validation for cluster name
Merge branch 'devel-2.7'
baserlib: Make "_OPCODE_ATTRS" public
"_OPCODE_ATTRS" will also be used in the Sphinx extension.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Helper to retrieve access permissions for RAPI resource
Merge similar code into a helper function.
Status change reason support for Reboot
Add support to the Reboot command for specifying the reason for the laststatus change.
Some features are implemented as functions, even if used only once, becausethey will be used by the future patches introducing reason support for all...
Infrastructure for specifying instance status change reason
This patch introduces some infrastructural modifications that will be used bythe following commits to implement the support for specifying the reason forthe last status change of an instance....
Unit tests for Query/Network.hs
This patch adds a couple of unit tests for Query/Network.hs.Note that they'll need to be adapted, once issue 362 is addressed.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add Haskell parser for "xm uptime"
In order to fetch precise information about the uptime of the VMsrunning in Xen, we need to analyze the output of the "xm uptime" command.
This commit adds the parser to do that, and its tests.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add Haskell parser for "xm list --long"
In order to fetch precise information about the status of the VMs running inXen, we need to analyze the output of the "xm list --long" command.
Add IQ_NETWORKS in query tests
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Rename lib/objectutils to outils.py
Back when this was introduced, I mentioned that it breaks heavily tabcompletion (ob<TAB> doesn't work anymore), but at that moment I didn'thave a suggestion what to name it. I think outils is good and shortenough, and doesn't conflict with anything else, so here it goes....
Add functions to parse CLI-level format of priorities
The current serialisation format for submit priorities isinteger-based, same as the opcode json serialisation. But for CLIlevel, we need to support a string-based format, so we add functionsto parse and format this representation....
Enable use of the priority option in hbal
This patch adds the option to hbal, and uses it to tweak the submittedjobs. There are also two small shelltests for testing the parsing.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Finish the remote→restricted commands rename
The documentation still points to /etc/ganeti/remote-commands,although the code is already using restricted-command. Update thedocumentation and a few docstrings accordingly.
Signed-off-by: Iustin Pop <iustin@google.com>...
Removes check for conflicts from NetworkDisconnect
This removes the check for conflicts from the Haskellversion of the OpCode NetworkDisconnect. This alignesthe Haskell code with the patch"Force conflicts check in LUNetworkDisconnect" (whichis currently under review). I will submit these patches...
Add unit tests for RADOSBLockDevice
Add unit tests for the RADOSBlockDevice JSON output parsing function,and modify the unit tests for the plain output parsing function towork with the new code.
Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>Reviewed-by: Guido Trotter <ultrotter@google.com>
Remove network_type slot (Issue 363)
This slot was not used by Ganeti so the same info can beprovided via tags. In order not to break configuration datawe add a FromDict() method in Network config object thatremoves the deprecated network_type (if found) and then invoke...
Moved uniformity check for exclusive_storage flag
Cluster-verify used to check that the value of exclusive_storage is uniformwithin node groups. Now, it's impossible to change the flag for a singlenode, so that check has been removed and an equivalent one has been added...
"exclusive_storage" cannot be changed on single nodes
There's never been support for a configuration where nodes in the same nodegroup have different values of the exclusive_storage flag. This patchdisables the possibility to change the flag for individual nodes....
Upgrades made on loading the configuration are always saved
Before, only some upgrades were written back to the configuration file. Alittle refactoring of _UpgradeConfig() has been done to write unit tests.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
QA: Convert nodes to objects
Up until now nodes were stored as a dictionary. The keys were hardcodedin a lot of places and entries modified directly.
This patch introduces a new class for nodes in QA named “_QaNode”. Itstill supports accessing details via dictionary syntax, but that will be...
QA: Release instances through method
Instead of having a module-level function in “qa_config” to release aninstance, a new method on instance objects is used.
QA: Convert instances to objects
Up until now instances were stored as a dictionary. The keys werehardcoded in a lot of places and entries modified directly.
This patch introduces a new class for instances in QA named“_QaInstance”. It still supports accessing details via dictionary...
Refactor storage of runtime exclusive storage flag in QA
This is a follow-up for “qa_config: Remove exclusive storage flag fromconfig”. Instead of storing the flag in a module-level variable it isnow stored within the new QA configuration class and unit tests are...
Refactor QA configuration code
Ever since its introduction (sometime before commit cec9845 in September2007), the QA configuration was stored in a dictionary at module-levelin “qa/qa_config.py”. The configuration was loaded, verified andevaluated using module-level functions. Since then the configuration has...
vcluster: Make _MakeNodeRoot public
The QA code will also have to generate virtual paths and this functioncomes in handy.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Remove use of 'head' and add hlint warning for it
Since 'head' is unsafe to use in most cases, this patch removes itsuse from most of the code, adds a lint warning for it (and for tail aswell), and adds override annotations in the few cases where it'sactually OK to use it (mainly when using head over the result of...
Add some more Haskell/Python equivalence tests
This would have caught the log file problem fixed in the previouspatch (9411474b), for example. Also we test user/group equivalence,name only.
Extract container converters
“objects.ConfigObject” contains two useful functions for working withcontainers of serialized objects, “_ContainerToDicts” and“_ContainerFromDicts”. This patch separates those functions and movesthem into “objectutils” as they'll be useful for converting parts of the...
objects: Improve handling of TCP/UDP port pool
- Handle de-serialization correctly when pool is not defined- Serialize to empty list when the attribute is None (this should never happen in reality as the attribute is always set when de-serializing)- Add tests...
Add a helper function for hypervisor verification
This will allow easier multi-error results from hypervisors; rightnow, we only report the first error, which is not nice.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Program/Harep.hs: add skeleton for the new auto-repair tool
harep(1) detects certain kind of problems with instances and applies theallowed set of solutions. See doc/design-autorepair.rst.
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_kvm: Refactor getting TAP features, add tests
Split retrieving supported features into a dedicated function which canbe mocked. Tests are added for both “_ProbeTapVnetHdr” and“_GetTunFeatures”.
Checks for hypervisor constants
Add two tests for hypervisor constants.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_xen: Fix issues with migration, add tests
Commit 3d942d8 broke instance migration (“self._cmd” was set to None).This patch fixes that issue, refactors “MigrateInstance” for testing andadds those tests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
hv_xen: Test various forms of getting instance/node info
Test functions for retrieving instance and node information.
hv_xen: Test stopping an instance
Test stopping an instance, and ensure the configuration file stillexists after an instance couldn't be stopped.
hv_xen: Test starting an instance
Test starting an instance in normal mode, in paused mode, and in case offailure.
hv_xen: Test reading non-existent config file
Check whether doing so raises an exception.
hv_xen: Test verifying hypervisor
Test success and failure.
hv_xen: Test removing config from "auto" directory
Test whether any existing configuration file named after the instance isremoved from Xen's “auto” directory.
Establish base for testing Xen hypervisor abstraction
There are two separate Xen hypervisors (HVM and PVM), as well as twodifferent Xen commands (xl and xm). This already provides four differentcombinations and future changes might bring even more. For this reason...
hv_xen: Test errors while writing config file
Test error handling while writing Xen configuration files.
hv_xen: Test using unknown Xen command
This tests “unknown” values of “constants.XEN_CMD”.
Extend config by networks and networks by UUIDs
For network queries to work, we need to extend the generalconfig type to include all available networks. Additionally,we add UUIDs to the network type itself.
Signed-off-by: Helga Velroyen <helgav@google.com>...
hv_xen: Factorize and test disk configuration
The “_GetConfigFileDiskData” function is moved to module level andcleaned up (module-level constants for letters and file I/O drivers).
Until now only 24 disks would be supported (e.g. “sda” to “sdx”), when...
hv_xen: Refactor getting node information, add tests
Refactor and add tests for getting node (Domain-0) information.
hv_xen: Refactor running & parsing "xm list", add tests
This patch refactors “_RunXmList” and adds some tests.
_VerifyErrors()._Error() and _ErrorIf() are now consistent
_Error() didn't contain the logic for demoting errors to warnings and formarking an operation as failed. Now _ErrorIf() is just a minimal wrapperfor _Error().
Unit tests included.
testutils: Module-level method for reading test data
This patch moves the functions “_ReadTestData” and “_TestDataFilename”to module level. They do not depend on a class instance at all.
Some cases of “utils.ReadFile(self._TestDataFilename(…))” are also...
Add test for backend._GetBlockDevSymlinkPath
Add a unit test for the trivial “_GetBlockDevSymlinkPath” function inbackend (small changes in the function were required).
hv_xen: Add test for CPU pinning configuration
Add a unittest for a function formatting CPU pinning information forXen's configuration.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Text wrap in Haskell code
This adds two functions in Utils.hs: 'wrap' and 'trim' along with somesimple tests. 'wrap' is used in Common.hs for wrapping commanddescriptions to a fixed line length, e.g. useful for the "--help" option.
Signed-off-by: Yiannis Tsiouris <gtsiour@softlab.ntua.gr>...
utils.LvmExclusiveCheckNodePvs() returns more info
utils.LvmExclusiveCheckNodePvs() now returns the size of the smallest andbiggest PVs of a node. They will be used for cross-node consistency checks.
burnin: Add unittest for supported disk templates
When new a new disk template is added, burnin should be updated.
Remove special-case of "0" in niceSort
Since now niceSort is more consistently behaved, we don't have thespecial casing. This should have been removed in commit a7f0953a, butI didn't realise it.
test/Types.hs: add an HLint ANN for ignoring camelCase
Replace nodegroups' PartialNic by PartialNicParams
This fixes a bug that corrups the config when one adds anetwork to a node group. I wil soon extend QA by some commandsthat would have detected this.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Loader.hs: ignore expired ArSuspended policies
At the moment, because 'mergeData' is pure, it may set instance auto-repairpolicies that are of the form `ArSuspended $ Until timestamp_in_the_past`.If later on the auto-repair tool notices this, it has lost access to what...
Utils.hs: add a clockTimeToString function
This function allows to easily convert a ClockTime object to a stringrepresentation of its timestamp (seconds-only).
Utils.hs: function to chomp prefix + separator from a string
Some auto-repair tags are composed of a prefix and then an optionalargument; the new "chompPrefix" function in Utils.hs allows to strip aprefix, allowing the last character (the separator) to be absent if there...
Add initial constants and Haskell ADTs for auto repair
In this commit, the AutoRepairType and AutoRepairResult types are defined,with the possible values specified in doc/design-autorepair.rst.
kvm: add two help dump files
One is the --help output for kvm 1.1.2 and the second one is the same as0.9.1 but with a fake option added between -drive and its boot=on|offoption: this tests that if boot=on|off appears in another option after-drive it is not accepted by our regexp....
kvm: use help based feature detection
As discussed on the list some people backport features to different kvmversions, making the version based detection system unreliable. Wechange wherever we can to detection on the help output.
This also fixes Issue 250...
test/Utils.hs: fix capitalization in docstring comment
Unit tests for bdev and utils.lvm
Also amended a comment of a tested method.The tests are for methods introduced in commit 63c73073 (LVM disk creationuses dedicated PVs).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Support instance-minor pairing in the DRBD collector
This commits enables the DRBD data collector to use the Confd client togather information about the pairing between DRBD minors and instances.
For testing purposes, the DRBD data collector now requires either zero...
Support integrating instance information in the DRBD parser
This commit modifies the DRBD parser and its data structures to includeinformation about the instance a DRBD minor belongs to.
Test files have been updated as well, to support the new field in the data...
Add "use_locking" parameter to network query opcode
This was extracted from a patch by Dimitris Aragiorgis with the subject“Add locking to _NetworkQuery”. The rest of the patch did no longerapply and will require more work.
Move src/Ganeti/HTools/Program.hs to Program/Main.hs
This removes one more tab conflict; this is the last module in ourcode where we have both x.hs and x/.
Furthermore, we collapse all actual code into the new Main.hs module,leaving the htools.hs basically empty (will allow better testing in...
Remove duplicate test file functions
This patch unifies the functions that generated the treereorganisation discussion in the first place :)
Rename test/hs/test to htest
The current base name of 'test' means that this binary will generateprofiling, coverage, etc. files with the name 'test.*', result intab-completion conflicts with the test directory. Let's slightlychange its base name for typing easiness....
Move htest/ files under the test/ tree
htest/data becomes test/data/htools (basically reverting commit8feabc89), and htest/* becomes test/hs/*.
Most changes beside the rename are trivial s/…/…, with the exceptionof autotools/run-in-tempdir, which needed some more changes now that...
Move python test files to test/py
This is the first step of the test files reorganisation: moving test/*(except test/data) to new directory test/py/.
Add the gnt-storage client
Add a new client called 'gnt-storage'.The client interacts with the ExtStorage interface, similarly tothe way gnt-os interacts with the OS interface.
For now, only two commands are supported: 'info' and 'diagnose'.
'diagnose' calculates the node status of each provider on each node,...
gnt-* list-fields: Show friendly field type
For people writing query filters (documented in ganeti(7)) knowing a field'stype can be useful.
$ gnt-instance list-fields name be/memoryName Type Title Descriptionname Text Instance Instance name...
Add utility to format dictionary as key=value strings
This will be used in QA to format network interface parameters.
cmdlib: Additional opportunistic locking check
If opportunistic locking is requested in the opcode, but no iallocatoris used (not specified or no default), an error will be raised.
Replace frozenset with compat.UniqueFrozenset
This is not a trivial s/frozenset/compat.UniqueFrozenset/, but ratheronly replaces “frozenset” where appropriate. Most of the places are“static” information that doesn't change after the module has beenloaded....
Add exclusive_storage node parameter
Unit tests updated and expanded with an inheritance check.
The flag has no effect yet.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix error during cluster initialization due to hv_kvm
Commit 141d148 was a bit too enthusiastic. The three parameters added tothe list of parameters to be checked default to a value not evaluatingto false, leading to a failure on cluster initialization....
Add test for SPICE parameter list, add missing ones
“_SPICE_ADDITIONAL_PARAMS” is supposed to be the full list ofSPICE-related KVM hypervisor parameters with the exception of“HV_KVM_SPICE_BIND”. The new test checks if all parameters starting with“HV_KVM_SPICE_*” are included. Three previously missing parameters are...
Add optional formatting for OP_DSC_FIELD
For some opcodes, the output is not "stable", and depends on the exactinput values; this makes it harder to check consistency againstHaskell code.
To compensate for this, we add a way to override the formatting of the...
Add RPC for setting watcher pause
The watcher pause file should be set/unset on all nodes at once, notonly the master node. For that a new RPC is needed.