Refactor ispecs in ipolicy structures
Minimum and maximum instance specs are put together into a single elementof the instance policy. This is in preparation for introducing multiplemin/max specs.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
cfgupgrade: Add --downgrade option
It's now possible to go back to the previous stable version. Unit testsprovided.
This is mostly useful during development, when going from master tostable/devel.
Unit test for cli.CreateIPolicyFromOpts()
Useful for subsequent patches.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Generic function to print data for gnt-xxx info
The function produces a human-readable output, which is also a valid YAMLfile, from an intermediate data structure. This will be used by "gnt-xxxinfo" commands. The output of "gnt-xxx info" was almost YAML-compliant, and...
Fix bug in rlib2 unit tests
The "queryargs" parameter of the init function of rlib2 classes shouldbe a dictionary, as defined in the ResourceBase parent class (inlib/rapi/baserlib.py).
In the rlib2 and baserlib unit tests, when the queryargs parameter is not...
gnt-cluster modify: dis/enabling storage types
This patch extends the 'gnt-cluster modify' command to manipulate the listof enabled storage types. Note that this currenlty does no validationwith respect to whether or not there are instances currently using a storage...
Add the core of the monitoring daemon
This commit adds the core infrastructure of the monitoring daemon,and integrates it in the build and test systems.
The actual functionality of the monitoring daemon is still completelymissing.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add Mond to the list of possible daemons
Also, add its logfiles and extra log files.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add infrastructure for allowing additional logfiles
Some daemons will need more than the single logfile that is currentlyallowed. This patch introduces the infrastructure to allow this.
Make gnt-node list -o(p|s)inst_list output stable
Currently, both the Python and Haskell code return the internalinstance list unsorted, which means the output can vary depending onthe phase of the moon (well, the Haskell code actually uses internallya tree, sorted by the instance name, but it's implementation detail)....
Expand TestHelper to allow non-underscore prefixes
HLint 1.8.28 requires us to always add "ignore CamelCase", which canbe problematic sometimes (e.g. when using OverloadedStrings). Let'sexpand TestHelper to also support less-standard 'caseFooBar' names, so...
Change to CamelCase format in Test/Ganeti/Objects.hs
This is in preparation for using OverloadedString extensions in thisfile, which conflicts with hlint 1.8.28's handling of annotations.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add simple Ip4Address/Ip4Network types
This patch adds some very simple IPv4 address/network types, and usesthem in the 'Network' config object.
We need these in order to properly compute the reserved IP addresses,without depending on an external library (which I haven't found, by...
Merge branch 'devel-2.7'
Unit tests for objects.InstancePolicy + a fix
Tests for: objects.InstancePolicy.CheckParameterSyntax() objects.InstancePolicy.CheckDiskTemplates() objects.InstancePolicy.CheckISpecSyntax()
Instance policies with an empty disk-template list now are reported....
Unit tests for objects.FillIPolicy() + small fix
IPOLICY_DEFAULTS is now a legal policy (the disk-templates entry was a setinstead of a list, before).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix upgrade of policy in objects.Cluster
Unknown elements were silently removed on startup. This means that asoftware upgrade could result in lost configuration information ifcfgupgrade wasn't run promptly.
Added unit test for Cluster.UpgradeConfig() to cover this case....
Fix instance policy checks for default back-end parameters
Policy violations of back-end parameters that used the cluster defaultvalue were not reported in cluster-verify.
Unit tests for cmdlib._GetUpdatedIPolicy()
Not 100% coverage, though.
Fix policy check for disk templates
Instance disk template is checked against the policy, and disklessinstances aren't checked for the number of disks.
Conflicts: Makefile.am (curl changes and new hs directories)...
Expand Haddock to run over test files as well
This patch does multiple enhancements to the way we build the Haddockdocs, motivated by the fact that if we don't run Haddock over testfiles as well, bad formatting can be submitted and will accumulateover time (without any checks)....
Switch LispConfig double comparison to relative error
This further improves the comparison for "non-trivial" numbers. Without this patch, there are still cases where the absoluteerror is too big, and we need to switch to relative error.
Concept has been taken from...
Make the XmParser config test runtime more consistent
Currently, the test uses a frequency of 5 string/5 double/1 list forgenerating Arbitrary instances of ListConfig. However, the list casehas simply a "choose (1, 20)" `vectorOf` arbitrary, which means it...
Improve output of the XmParser config test
Currently, this tests and its helper function 'isAlmostEqual' usesplain booleans to signify failures, which means you can't really debuga failed test. The patch changes the call chain to use annotatedproperties all through, which results in messages like:...
Fix node partial name matching in Haskell code
This implements QffHostname and fixes the node listing (as well asexport listing when filtering on node name).
This bug was hidden by the fact that node listing with "gnt-node listaa" works if you don't have live queries (as it was originally), as...
Fix bug in group queries related to node/instance fields
Since we use the primitive string type for group UUIDs, the groupfields have a bug where we pass the group name as filter for nodetests, whereas the nodes themselves use the group UUID. This results...
Rename/make uniform the other query entities
Following the new naming style introduced in Exports.hs, this patchrenames the other resources to export non-qualified names (fieldMap asopposed to nodeFieldMap), and to use qualified module imports.
Also fixes a haddock issue in a docstring....
Add export lists for files which didn't have them
Two test helper files didn't have export lists (lost during the splitof tests). This patch finally reintroduces them, to hopefully make theexport list saner and with fewer changes for purely-internal changes....
Fix Haskell compatibility tests with disabled file storage
When file storage is disabled at ./configure time, we shouldn't passopcodes containing DTFile/DTSharedFile/StorageFile to Python forvalidation, as they will fail.
This patch implements this by simply tweaking the Arbitrary instances...
Allow rpc.MakeLegacyNodeInfo to parse non-LVM results
'MakeLegacyNodeInfo' is not the best place for this, but we'd have toduplicate it if we wanted a LVM-less version, so the easiest is to addan optional parameter that allows it to accept/skip LVM-less results....
Test AsyncStreamServer with abstract unix sockets
This was meant to be since "the beginning" but was never submitted assomehow it failed with python 2.4. Now that the minimum python versionhas been increased it can be added.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
qa_config: Support callable test checks
Until now all tests depended on simple AND/OR conditions. With theintroduction of virtual cluster support, some tests can only run ifvirtual clusters are not in use (e.g. not yet supported or notpossible). Instead of introducing more logic for enabling/disabling...
QA: Add config entries for virtual cluster
If these new options are set, a virtual cluster is created and used.Virtual clusters consist of 1..N virtual nodes on a single physical (orvirtual) node. The design of virtual clusters is described in“doc/design-virtual-clusters.rst”....
Fix style violations in commit 4a90bd4, small changes
- Commit 4a90bd4 contained a rather large number of style violations: - Indentation/sequence formatting - Wrapping of long lines - Docstrings whose first line was wrapped - A stray backslash in a docstring...
RAPI: Add flag to require authentication
Most RAPI resources do not require authentication for the “GET” method.In some setups it can be desirable to always require authentication.This patch adds a command line parameter to always require it.
Some unrelated minor typos in the “ganeti-rapi” man page are also fixed....
Remove test of obsolete getNetworkUuid method
This will remove the tests for the getNetworkUuid method.The method will be obsolete after the changes regardingidentification of networks by UUID and not name are mergedinto master. Will send another patch for removing the...
Add two utility functions for handling Either lists
These two functions permit operating in bulk on only the Left or Rightvalues in the original list, then reassembling the list back in theoriginal order.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Simplify RPC error cases
This patch removes the node from the RPC error constructursCurlLayerError and OfflineNodeError. The rationale is that we anywayreturn tuples (node, result), and removing this duplication allowssimplified signatures/calls in the execution of RPC calls....
qa_config.AcquireNode: Use sort key function
Use a sort key function instead of using a comparing function (“cmp=…”).The latter is not supported in Python 3 and using a sort key function iseasier.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
qa_config: Add repr for instance/node classes
Having a readable representation makes debugging easier.
Test "GetNodeInfo" for Xen hypervisor
This patch adds unit tests for success and failure of the “GetNodeInfo”method for the Xen hypervisor abstractions.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
QA: Add validation for cluster name
baserlib: Make "_OPCODE_ATTRS" public
"_OPCODE_ATTRS" will also be used in the Sphinx extension.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Helper to retrieve access permissions for RAPI resource
Merge similar code into a helper function.
Status change reason support for Reboot
Add support to the Reboot command for specifying the reason for the laststatus change.
Some features are implemented as functions, even if used only once, becausethey will be used by the future patches introducing reason support for all...
Infrastructure for specifying instance status change reason
This patch introduces some infrastructural modifications that will be used bythe following commits to implement the support for specifying the reason forthe last status change of an instance....
Unit tests for Query/Network.hs
This patch adds a couple of unit tests for Query/Network.hs.Note that they'll need to be adapted, once issue 362 is addressed.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add Haskell parser for "xm uptime"
In order to fetch precise information about the uptime of the VMsrunning in Xen, we need to analyze the output of the "xm uptime" command.
This commit adds the parser to do that, and its tests.
Add Haskell parser for "xm list --long"
In order to fetch precise information about the status of the VMs running inXen, we need to analyze the output of the "xm list --long" command.
Add IQ_NETWORKS in query tests
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Rename lib/objectutils to outils.py
Back when this was introduced, I mentioned that it breaks heavily tabcompletion (ob<TAB> doesn't work anymore), but at that moment I didn'thave a suggestion what to name it. I think outils is good and shortenough, and doesn't conflict with anything else, so here it goes....
Add functions to parse CLI-level format of priorities
The current serialisation format for submit priorities isinteger-based, same as the opcode json serialisation. But for CLIlevel, we need to support a string-based format, so we add functionsto parse and format this representation....
Enable use of the priority option in hbal
This patch adds the option to hbal, and uses it to tweak the submittedjobs. There are also two small shelltests for testing the parsing.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Finish the remote→restricted commands rename
The documentation still points to /etc/ganeti/remote-commands,although the code is already using restricted-command. Update thedocumentation and a few docstrings accordingly.
Signed-off-by: Iustin Pop <iustin@google.com>...
Removes check for conflicts from NetworkDisconnect
This removes the check for conflicts from the Haskellversion of the OpCode NetworkDisconnect. This alignesthe Haskell code with the patch"Force conflicts check in LUNetworkDisconnect" (whichis currently under review). I will submit these patches...
Add unit tests for RADOSBLockDevice
Add unit tests for the RADOSBlockDevice JSON output parsing function,and modify the unit tests for the plain output parsing function towork with the new code.
Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>Reviewed-by: Guido Trotter <ultrotter@google.com>
Remove network_type slot (Issue 363)
This slot was not used by Ganeti so the same info can beprovided via tags. In order not to break configuration datawe add a FromDict() method in Network config object thatremoves the deprecated network_type (if found) and then invoke...
Moved uniformity check for exclusive_storage flag
Cluster-verify used to check that the value of exclusive_storage is uniformwithin node groups. Now, it's impossible to change the flag for a singlenode, so that check has been removed and an equivalent one has been added...
"exclusive_storage" cannot be changed on single nodes
There's never been support for a configuration where nodes in the same nodegroup have different values of the exclusive_storage flag. This patchdisables the possibility to change the flag for individual nodes....
Upgrades made on loading the configuration are always saved
Before, only some upgrades were written back to the configuration file. Alittle refactoring of _UpgradeConfig() has been done to write unit tests.
QA: Convert nodes to objects
Up until now nodes were stored as a dictionary. The keys were hardcodedin a lot of places and entries modified directly.
This patch introduces a new class for nodes in QA named “_QaNode”. Itstill supports accessing details via dictionary syntax, but that will be...
QA: Release instances through method
Instead of having a module-level function in “qa_config” to release aninstance, a new method on instance objects is used.
QA: Convert instances to objects
Up until now instances were stored as a dictionary. The keys werehardcoded in a lot of places and entries modified directly.
This patch introduces a new class for instances in QA named“_QaInstance”. It still supports accessing details via dictionary...
Refactor storage of runtime exclusive storage flag in QA
This is a follow-up for “qa_config: Remove exclusive storage flag fromconfig”. Instead of storing the flag in a module-level variable it isnow stored within the new QA configuration class and unit tests are...
Refactor QA configuration code
Ever since its introduction (sometime before commit cec9845 in September2007), the QA configuration was stored in a dictionary at module-levelin “qa/qa_config.py”. The configuration was loaded, verified andevaluated using module-level functions. Since then the configuration has...
vcluster: Make _MakeNodeRoot public
The QA code will also have to generate virtual paths and this functioncomes in handy.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Remove use of 'head' and add hlint warning for it
Since 'head' is unsafe to use in most cases, this patch removes itsuse from most of the code, adds a lint warning for it (and for tail aswell), and adds override annotations in the few cases where it'sactually OK to use it (mainly when using head over the result of...
Add some more Haskell/Python equivalence tests
This would have caught the log file problem fixed in the previouspatch (9411474b), for example. Also we test user/group equivalence,name only.
Extract container converters
“objects.ConfigObject” contains two useful functions for working withcontainers of serialized objects, “_ContainerToDicts” and“_ContainerFromDicts”. This patch separates those functions and movesthem into “objectutils” as they'll be useful for converting parts of the...
objects: Improve handling of TCP/UDP port pool
- Handle de-serialization correctly when pool is not defined- Serialize to empty list when the attribute is None (this should never happen in reality as the attribute is always set when de-serializing)- Add tests...
Add a helper function for hypervisor verification
This will allow easier multi-error results from hypervisors; rightnow, we only report the first error, which is not nice.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Program/Harep.hs: add skeleton for the new auto-repair tool
harep(1) detects certain kind of problems with instances and applies theallowed set of solutions. See doc/design-autorepair.rst.
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_kvm: Refactor getting TAP features, add tests
Split retrieving supported features into a dedicated function which canbe mocked. Tests are added for both “_ProbeTapVnetHdr” and“_GetTunFeatures”.
Checks for hypervisor constants
Add two tests for hypervisor constants.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_xen: Fix issues with migration, add tests
Commit 3d942d8 broke instance migration (“self._cmd” was set to None).This patch fixes that issue, refactors “MigrateInstance” for testing andadds those tests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
hv_xen: Test various forms of getting instance/node info
Test functions for retrieving instance and node information.
hv_xen: Test stopping an instance
Test stopping an instance, and ensure the configuration file stillexists after an instance couldn't be stopped.
hv_xen: Test starting an instance
Test starting an instance in normal mode, in paused mode, and in case offailure.
hv_xen: Test reading non-existent config file
Check whether doing so raises an exception.
hv_xen: Test verifying hypervisor
Test success and failure.
hv_xen: Test removing config from "auto" directory
Test whether any existing configuration file named after the instance isremoved from Xen's “auto” directory.
Establish base for testing Xen hypervisor abstraction
There are two separate Xen hypervisors (HVM and PVM), as well as twodifferent Xen commands (xl and xm). This already provides four differentcombinations and future changes might bring even more. For this reason...
hv_xen: Test errors while writing config file
Test error handling while writing Xen configuration files.
hv_xen: Test using unknown Xen command
This tests “unknown” values of “constants.XEN_CMD”.
Extend config by networks and networks by UUIDs
For network queries to work, we need to extend the generalconfig type to include all available networks. Additionally,we add UUIDs to the network type itself.
Signed-off-by: Helga Velroyen <helgav@google.com>...
hv_xen: Factorize and test disk configuration
The “_GetConfigFileDiskData” function is moved to module level andcleaned up (module-level constants for letters and file I/O drivers).
Until now only 24 disks would be supported (e.g. “sda” to “sdx”), when...
hv_xen: Refactor getting node information, add tests
Refactor and add tests for getting node (Domain-0) information.
hv_xen: Refactor running & parsing "xm list", add tests
This patch refactors “_RunXmList” and adds some tests.
_VerifyErrors()._Error() and _ErrorIf() are now consistent
_Error() didn't contain the logic for demoting errors to warnings and formarking an operation as failed. Now _ErrorIf() is just a minimal wrapperfor _Error().
Unit tests included.
testutils: Module-level method for reading test data
This patch moves the functions “_ReadTestData” and “_TestDataFilename”to module level. They do not depend on a class instance at all.
Some cases of “utils.ReadFile(self._TestDataFilename(…))” are also...
Add test for backend._GetBlockDevSymlinkPath
Add a unit test for the trivial “_GetBlockDevSymlinkPath” function inbackend (small changes in the function were required).
hv_xen: Add test for CPU pinning configuration
Add a unittest for a function formatting CPU pinning information forXen's configuration.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Text wrap in Haskell code
This adds two functions in Utils.hs: 'wrap' and 'trim' along with somesimple tests. 'wrap' is used in Common.hs for wrapping commanddescriptions to a fixed line length, e.g. useful for the "--help" option.
Signed-off-by: Yiannis Tsiouris <gtsiour@softlab.ntua.gr>...