Make retry tests independent of actual time
Even in the tests, real time is used. While, generally, the assumptionsabout execution time are pretty safe, in some rare circumstances, e.g.,on machines with extremely heavy load they do not hold true, thus rendering...
Add unit test for GetLinuxNodeInfo
The function was not unit tested previously.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.7' into stable-2.8
Support DSA SSH keys in bootstrap
As outlined in issue 338, Ganeti failed to initialize a cluster if noRSA SSH key is present on the master node. This patch extends Ganetissupport to DSA keys, so clusters with only DSA keys are possible now.
This fixes issue 338....
ganeti.backend_unittest: chmod restricted cmd dir
TestVerifyRestrictedCmdDirectory.testNormal implicitly relies on the currentumask to check the behaviour of backend._VerifyRestrictedCmdDirectory. However,when run under a more relaxed umask (as in the case of the Debian buildd's...
Add cleanup parameter to instance failover
Most of the code is shared with instance migrate, so we actually only needto add the parameter and pass its value along the the common code.
Also, tests and harep are updated to support the right set of options to...
Fix permission problem related to Issue 477
Commit 91525dee856951ace940c78b6254a1c7344b4803 fixed Issue 477 but broke"gnt-cluster info".
This commit offers a solution to both problems, by changing the permissionof the socket instead of changing the permission the confd process is run...
Rename queryd to luxid
As queryd will, in the future, handle all LUXI request, queue jobs andmost likely perform various other tasks, it is renamed to luxid already.This will safe some headache when upgrading Ganeti installations, as wedon't have to deal with a daemon rename....
Document the rapi client not to have a QueryNetworks method
While there is a luxi query QueryNetworks, the rapi client does notprovide such a method. Document this by declaring it in theKNOWN_UNUSED_LUXI calls.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add queryd daemon (split from confd)
queryd is added as a new daemon which handles configuration queries overLUXI. This functionality was removed from confd, which now only queriesover the network.
The queryd user is added to the master group such that it can access...
Conflicts:...
Allow modify_etc_hosts to be changed
The modify_etc_hosts options, enabling the cluster to modify the /etc/hostsfiles of nodes, and to keep them in sync, could only be set at cluster inittime.
With this commit it can now be changed through modify_etc_hosts as well....
Add luxiReqQueryNetworks to LuxiOp
When the QueryNetwork was introduced as a method, apparentlyit was forgotten in the Haskell world. Add it here as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Expose bulk parameter for GetJobs in RAPI client
This patch exposes the bulk argument of the jobs resource onthe RAPI python wrapper, making it possible to retrieve statusinformation about all jobs with a single call.
Signed-off-by: Leon Handreke <lhandreke@google.com>...
Add tests for CanRead
Verify that the CanRead function is actually able to check whether a file hasthe proper permissions.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add support for querying network timestamps
Add creation and modified timestamps when creating a new network, andextend the available query fields for networks with these fields,namely 'ctime' and 'mtime'.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>...
Only generate node lists with nodes having different names
genNodeList is used in testing to generate list of nodesthat could come up in cluster configurations. Since namesare used to assign indices to nodes, they have to be unique;this is also the case in all real clusters....
Also remove prop_IterateAlloc_sane from test list
In f4d1bb7 that test was removed, but forgotten to remove itfrom the list of tests to be executed. Fix that.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix documentation for prop_Alloc_sane
As discussed in the last commit, placing a new instance on the clustercan lead to a cluster that can be improved by moving previously addedinstances. For an empty cluster, however, there are no previousinstances. So add this to the test description to make obvious why...
Remove IterateAllocSane test
The test is testing for a property that just isn't true. Iteratedallocation greedily place one instance at a time taking the locallymost balanced solution. Then it is tested whether the resulting globalallocation can be improved....
Add test for shrink heuristics over different resources
In this example, memory prevents adding more than one instance onthe first node. Yet, on the other nodes, 2 instances can be placedeach, if disks are shrunk appropriately. This, however, requiresto courageous decision to shrink disks next, even though more nodes...
Add tests for check-news
Unit tests verifying the behaviour of the check-news script are added.
This required adding a new make target, "autotools-check" directly called from"commit-check", because the usual tests are executed in the build directory,whereas these, being relative to the autotools, cannot be executed there because...
Conflicts: (trival, take union of added files/tests) Makefile.am test/hs/shelltests/htools-hspace.test...
Add a test for hspace to respect instance policy
In this example, the instance policy for disks prevents allocationof more than one instance per node. The test verifies that tieredallocation still respects this and doesn't attempt to place policyviolating instances on the cluster....
Make shrinkByType aware of individual disks
When shrinking an instances, you can't just get smaller disk footprintwhile leaving the individual disks as they are. Make the shrinkheuristic aware of that fact, and decrease all individual disks aswell. Fixes issue 484....
Add a force option to the ClusterSetParams Opcode
If set, the op code will, in particular, try to set the master IPon the new netdev, even if shutting down the master IP on the oldnetdev failed.
Add tests for network-aware allocation
hail-alloc-invalid-network defines a cluster with two nodegroups and anallocation request which does not fit on any of the groups. Group 1 hasinvalid disk-templates while Group 2 is not connected to the rightnetworks....
Parse NIC data from allocation request in hail
Add a NIC type and extend the Instance type by a list of NIC's. Parsethe NIC's in allocation requests and store them for now. Later patcheswill make use of this field in order to ensure that the requestedinstance is only placed in node groups wich are connected to those...
Support group networks in Text backend
The Text backend now parses network UUID (comma separated) andserializes them in the same form.The test data is adapted to the new format.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Parse node group networks
Extend the Group by the network ids it is connected to. Adaptthe IAlloc backend such that the networks are parsed correctly.This also required the adaption of test data.
Make the disks_active flag queryable
gnt-instance list now also supports the disks_active field.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add disks_active to configuration
This flag tracks if the disks of an instace are supposed to be active.That's the case when an instance is running or when its disks gotactivated explicitly (and in a couple of other cases).It will be used by watcher to re-activate disks after a node reboot....
Stash Xen config file after a failed startup
This is a fix for a minor bug.Currently, a failed Xen VM start results in a stale config file leftbehind on the filesystem.This change introduces a new log directory, where the Xen VM configfile is moved after a failed startup....
cmdlib: Cleanup public/private functions
All functions/classes which are used outside of their defining module(with tests as an exception) no longer have a leading underscore.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
cmdlib: Extract storage related functionality
Split instance.py further by extracting storage related logical unitsand functions to instance_storage.py.
Extract query related logical units from cmdlib
All LUQuery* classes are extracted to query.py.
Extract instance related logical units from cmdlib
All LUInstance* classes are extracted to instance.py. Common functionsare moved to common.py if used by non-instance logical units as well.Additionally, helper functions which are only used by LUBackup* and...
Extract node related logical units from cmdlib
All LUNode* classes are extracted to node.py. Common functions are movedto common.py if used by non-node logical units as well.
Extract group related logial units from cmdlib
All LUGroup* classes are moved to group.py. Common functions areextracted to common.py.
Extract cluster related logical units from cmdlib
All LUCluster* classes are extracted to cluster.py. Shared functions areextracted to common.py, helper functions only used by LUCluster* areextracted to cluster.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Compatibility test for instances
This patch introduces a test to check the compatibilityof the Haskell and the Python representation of instances.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Instance generators
This patch introduces and enhances generators forinstances:- 'genInstWithNets' is split into the generation of an arbitrary instance and enhancing an instance with nets- 'genInst' calls 'genInstWithNets' with an empty set of initial networks to provide a reasonable default...
Annotate every arbitrary instance field
The Arbitrary instance of the 'Instance' object is writtenusing the <*> syntax. Since it often uses the 'arbitrary'generator for the instance's fields it is hard to figureout which 'arbitrary' fills which instance field. This...
Generators for disks
This patch adds generators for Disk instances to the Haskelltest code. It uses somewhat more reasonable generators tofill the fields instead of just arbitrary values.'genDiskWithChildren' is a generator that generates a diskwith a specified number of disk children. To avoid shooting...
Disk templates are "allowed" and not "enabled" in ipolicies
The way disk templates were displayed in instance policies became confusingsince the introduction of the enabled_disk_templates cluster parameter.
Fix issue 440.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Unit test for FillIPolicy keeping unknown keys
This makes sure that issue 401 doesn't come back.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add reason trail to haskell opcode format
The haskell type definition of opcodes should remain aligned with the pythonone.
Fix bug in group queries related to node/instance fields
Since we use the primitive string type for group UUIDs, the groupfields have a bug where we pass the group name as filter for nodetests, whereas the nodes themselves use the group UUID. This results...
Add shelltests for hspace allocation
Both tiered and standard allocations are tested, with a single and a doublemin/max instance specification.
Unit test for cli.FormatPolicyInfo()
We try to parse the string obtained by formatting a policy, and see if weget the same policy.
Add command-line support for multiple specs in ipolicy
Command line options accept multiple min/max specifications in instancepolicies. The output of show-ispecs-cmd is updated accordingly.
Add multiple min/max specs in instance policy
Now instance policies can contain more than one min/max specs. This is themain element of the "Constrained instance sizes" section in the"Partitioned Ganeti" design doc.
This is a big patch, but changing the type of a configuration item requires...
Separate checks for std spec compliance
This is needed to be able to validate the std spec against multiple min/maxspec pairs (appearing in next patches).
Conflicts: NEWS doc/rapi.rst
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Reason trail implementation for "start"
Opcode-specific implementation of the reason trail for the instancestartup operation.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Reason trail implementation for "shutdown"
Opcode-specific implementation of the reason trail for the instance shutdownoperation.
Add unit tests for cfgupgrade with a real configuration
Test upgrade and downgrade from a realistic 2.7 configuration.
Split functions in cfupgrade unit tests
This makes easier to test real configurations.
Limit specs in ipolicy command lines are always complete
Command line options are brought in line with the specs change of previouspatch. Old options are still allowed in gnt-cluster init, where thesemantic will remain non-ambiguous even after introducing multiple specs....
Refactor tests in ipolicy validation
This makes the code more modular, in preparation for the following patches.
Limit specs in instance policies are always complete
Specs used to specify limits in instance policies are always complete, evenat group level (only the whole limit set can be overridden). This is inpreparation for introducing multiple limits.
Add --ipolicy-xxx-specs options
These options allow to specify whole instance policy specs. This is neededfor the upcoming changes that tend to threat specs as monolithic objects.
Add command to print ipolicy options
The output of this command can be used to create an exact copy of thecurrent instance policy specs. The command could be expanded to print allthe options used to create a group or the cluster.
New CLI input type: list of key/value pairs
This will be used for the new instance specs options.
Allow "default" as a value for ipolicy templates
Previously the string value was being corrupted.
Extend the simulation backend to also simulate a master node
In a simulated cluster as created by the simulation backend tothe htools, make the first node of the first node group the masternode. In this way, hools (like hroller) that require a master node...
Add unit tests for hroller
Add a unit test for hroller in a situation where the orderis completely determined. There are 3 nodes where node 2 has,for each of the other nodes, an instanced shared with it. Sonode 2 has to be a reboot group of its own. There are no more...
Reason trail implementation for instance reboot
This commits allows ganeti to correctly forward the reason trail informationregarding instance reboot.
Remove old "reason" implementation
Remove the useless parts of the old, partial, implementation of the support fortracking the reason of instances state change, before implementing the newreason trail support, as per the design document.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Make hroller insist on finding precisely one master node
As people rely on the master node being the last node of the lastgroup, make hroller fail, if no master node could be found in thecluster. This happens, e.g., if a backend format is used that does not...
Add nic/disk name and UUID fields in Haskell code
Add name and uuid fields to Disk and nic Haskell objects. Also,since they now have an UUID, make them instances of UuidObject.
Signed-off-by: Christor Stavrakakis <cstavr@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Add name to INicParams and IDiskParams
This commits adds name to NIC and Disk modification definition.
Signed-off-by: Chris Stavrakakis <cstavr@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Update QA tests to check disk and NIC names
Modify existing tests to use the name option for instance disks. Theconfiguration options 'disk' and 'disk-growth' are merged into a single'disks' option, which is a list of dictionaries with 'size', 'growth' and...
Check that device names are unique and valid
Extend the CheckArguments phase of LUInstanceCreate and CheckPrereqphase of LUInstanceSetParams to also check if the name parameters ofdisks and NICs are unique and valid.
Support UUIDs and names when refering to a device
Modify _ApplyContainerMods function to lookup NICs/Disks not only by theirindex inside the container, but also by their UUID or name. Abstract thelookup code in new GetItemFromContainer function.
Make type of identifier in "opcode._TestInstSetParamsModList" to be...
Move HooksMaster out of the mcpu module
We need to do this, so that backend.py doesn't need to import mcpu, andthus indirectly cmdlib. This reduces the size of the node daemon byabout half, which is very important as it is pinned in memory.
This solves Issue 419....
QA: Decision about disk templates to use is more modular
qa_config.IsTemplateSupported() is already used to decide which disktemplates are supported by QA. Enabled templates now are part of thatdecision.
Unit tests are updated.
Adapt test data to changed ipolicy format
The test data for hail allocating a multi-disk instance wasoriginally added in the 2.7 branch. On master the formatof the ipolicy had changed; "min" and "max" are no longertop-level of the ipolicy, but instead below the "minmax"...
Merge branch 'devel-2.7'
Removes obsolete 'enabled storage types'
Since managing of different storage units is now done usingdisk templates and not storage types, we remove the obsoleteenabled storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtarara@google.com>
gnt-cluster modify: modify enabled disk templates
Adds setting of the list of enabled disk templates to 'gnt-cluster modify'.Note that this does not yet include any verification regarding disk templatescurrently used by instances.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Add shelltests verifying hail applies disk ipolicy per disk
We test on two minimal examples, a positive and a negative one, wherethe possibility to allocate a node depends on whether the disk policyis checked on a per-disk level or on the total amount of disk space...
Make the disks parameter available to the constructor
In that way, tools building on Instance will benefit from the correctedverification semantics of the instance policy on disk space.
Properly export errors while reading job list
In case of problems while reading the job list from disk (such as permissionerrors) confd would silently fail, writing a warning on the log file butsending an empty list and no error message to the client....
Fix a bug in the Runtime tests
A group name was erroneusly inserted in the list of user names.
Add parameter pnode to OpInstanceSetParams
This parameter will be used to set the new primary nodeof an instance, assuming that the disks have been movedby other means (outside ganeti's control).
Unit tests for config.ConfigWriter.VerifyConfig()
The test is aimed mostly at instance policies, which changed recently.
Refactor ispecs in ipolicy structures
Minimum and maximum instance specs are put together into a single elementof the instance policy. This is in preparation for introducing multiplemin/max specs.
cfgupgrade: Add --downgrade option
It's now possible to go back to the previous stable version. Unit testsprovided.
This is mostly useful during development, when going from master tostable/devel.
Unit test for cli.CreateIPolicyFromOpts()
Useful for subsequent patches.
Generic function to print data for gnt-xxx info
The function produces a human-readable output, which is also a valid YAMLfile, from an intermediate data structure. This will be used by "gnt-xxxinfo" commands. The output of "gnt-xxx info" was almost YAML-compliant, and...
Fix bug in rlib2 unit tests
The "queryargs" parameter of the init function of rlib2 classes shouldbe a dictionary, as defined in the ResourceBase parent class (inlib/rapi/baserlib.py).
In the rlib2 and baserlib unit tests, when the queryargs parameter is not...
gnt-cluster modify: dis/enabling storage types
This patch extends the 'gnt-cluster modify' command to manipulate the listof enabled storage types. Note that this currenlty does no validationwith respect to whether or not there are instances currently using a storage...
Add the core of the monitoring daemon
This commit adds the core infrastructure of the monitoring daemon,and integrates it in the build and test systems.
The actual functionality of the monitoring daemon is still completelymissing.
Add Mond to the list of possible daemons
Also, add its logfiles and extra log files.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add infrastructure for allowing additional logfiles
Some daemons will need more than the single logfile that is currentlyallowed. This patch introduces the infrastructure to allow this.
Make gnt-node list -o(p|s)inst_list output stable
Currently, both the Python and Haskell code return the internalinstance list unsorted, which means the output can vary depending onthe phase of the moon (well, the Haskell code actually uses internallya tree, sorted by the instance name, but it's implementation detail)....
Expand TestHelper to allow non-underscore prefixes
HLint 1.8.28 requires us to always add "ignore CamelCase", which canbe problematic sometimes (e.g. when using OverloadedStrings). Let'sexpand TestHelper to also support less-standard 'caseFooBar' names, so...
Change to CamelCase format in Test/Ganeti/Objects.hs
This is in preparation for using OverloadedString extensions in thisfile, which conflicts with hlint 1.8.28's handling of annotations.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add simple Ip4Address/Ip4Network types
This patch adds some very simple IPv4 address/network types, and usesthem in the 'Network' config object.
We need these in order to properly compute the reserved IP addresses,without depending on an external library (which I haven't found, by...