Haskell to Python constants: more constants
Wrap 'Set' in 'ListSet' for the opcodes
In what Haskell to Python opcodes are concerned, a Haskell 'Set' istranslated into a Python 'list'. In other words, currently, opcodesthat handle sets of parameters are actually handling lists becausethis is how sets are currently encoded. This patch introduces a new...
Fix opcodes and parameters
Merge branch 'stable-2.9'
Merge branch 'stable-2.8' into stable-2.9
Remove TagObject
The 'TagObject' datatype became useless because it was replaced by'TagKind'. This patch removes 'TagObject', its 'Arbitrary' instance,and related functions.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Update Harep, Query server, and tests
Update Harep, Haskell query server, and tests concerning Luxi andopcodes to reflect the changes to Haskell to Python opcodegeneration. This change is necessary because TagObject is replaced byTagKind and some types in opcodes and parameters changed to be...
Add Python opcode generation
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Eliminate Haskell constant for opcode names
Modify 'autotools/convert-constants' not to generate the Haskellconstant that holds all opcode names. By having Haskell generate thePython opcopdes, it becomes impossible to generate this constant dueto a cyclic dependency. However, this constant is used only in tests,...
Add Haskell types for opcodes and parameters
Add Haskell types for IP addresses (version 4 and 6), IP networkaddresses, tag kind, and query result code.
Add cleanup parameter to instance failover
Most of the code is shared with instance migrate, so we actually only needto add the parameter and pass its value along the the common code.
Also, tests and harep are updated to support the right set of options to...
gnt-cluster modify --shared-file-storage-dir
This patch introduces to 'gnt-cluster modify' the option'--shared-file-storage-dir' to change the default directoryfor instances using shared file storage at cluster runtime.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Remove obsolete autoconf variable from remaining files
This removes the obsolete autoconf variable'ENABLE_SHARED_FILE_STORAGE' from all remaining files.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Extend gnt-instance show to show vlan parameter
Extensions for "gnt-instance show" to output the vlan parameter whichwas introduced by my recent patch series.This is done by extending the nic object that is used to output theinfo. Every other occurrence of this object is adopted as well....
gnt-cluster {init, modify} --file-storage-dir
This patch implements consistent usage and behavior ofthe --file-storage-dir option in 'gnt-cluster init'and 'gnt-cluster modify'. It includes a bunch of unittests as well.
Additionally, it enables the previously written unit...
Conflicts: src/Ganeti/Utils.hs (trivial)trivial
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix permission problem related to Issue 477
Commit 91525dee856951ace940c78b6254a1c7344b4803 fixed Issue 477 but broke"gnt-cluster info".
This commit offers a solution to both problems, by changing the permissionof the socket instead of changing the permission the confd process is run...
Rename queryd to luxid
As queryd will, in the future, handle all LUXI request, queue jobs andmost likely perform various other tasks, it is renamed to luxid already.This will safe some headache when upgrading Ganeti installations, as wedon't have to deal with a daemon rename....
Add queryd daemon (split from confd)
queryd is added as a new daemon which handles configuration queries overLUXI. This functionality was removed from confd, which now only queriesover the network.
The queryd user is added to the master group such that it can access...
Merge branch 'stable-2.8' into master
Merge branch 'stable-2.7' into stable-2.8
Conflicts:...
Remove file storage path from autoconf
This patch removes the option to configure the filestorage path at configure time, because it is nowreplaced by a runtime option for gnt-cluster init.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Allow modify_etc_hosts to be changed
The modify_etc_hosts options, enabling the cluster to modify the /etc/hostsfiles of nodes, and to keep them in sync, could only be set at cluster inittime.
With this commit it can now be changed through modify_etc_hosts as well....
Add luxiReqQueryNetworks to LuxiOp
When the QueryNetwork was introduced as a method, apparentlyit was forgotten in the Haskell world. Add it here as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add support for querying network timestamps
Add creation and modified timestamps when creating a new network, andextend the available query fields for networks with these fields,namely 'ctime' and 'mtime'.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>...
Only generate node lists with nodes having different names
genNodeList is used in testing to generate list of nodesthat could come up in cluster configurations. Since namesare used to assign indices to nodes, they have to be unique;this is also the case in all real clusters....
Add "includeLogicalId" function for Disks
This function checks whether a disk contains a given Lvm logical ID,directly or through its children.
Unit tests are added as well.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add "instance" field to LVInfo
Extend the LVInfo data structure with the field for storing the name ofthe instance it is paired with.
Update the tests accordingly.
Also remove prop_IterateAlloc_sane from test list
In f4d1bb7 that test was removed, but forgotten to remove itfrom the list of tests to be executed. Fix that.
Fix documentation for prop_Alloc_sane
As discussed in the last commit, placing a new instance on the clustercan lead to a cluster that can be improved by moving previously addedinstances. For an empty cluster, however, there are no previousinstances. So add this to the test description to make obvious why...
Remove IterateAllocSane test
The test is testing for a property that just isn't true. Iteratedallocation greedily place one instance at a time taking the locallymost balanced solution. Then it is tested whether the resulting globalallocation can be improved....
Turn 'exclusive_storage' into storage parameter (hs)
This is the haskell implementation of my patch "Extend RPC call'node_info' by storage parameters". It turns the 'exclusivestorage' flag into a storage parameter of the LVM storage types.Besides that, this patch moves some types into the Types.hs....
Load CPUs used by the node OS in htools
A new field is added to the Node type, and it's used to initialize the usedCPUs field.
The signature of Node.create has been split among lines to match theparameter list.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Add tests for LVs parser
Add unit tests and Quickcheck tests for the LV parser.
Also, fix the alphabetic order of some imports in htest.hs.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add function testing parsers with QuickCheck
Parsing a string automatically generated by QuickCheck can be useful in manycases. A function for doing this is added to the common test functions.
Add function for generating arbitrary UUID
Add a function for generating arbitrary UUIDs to be used for QuickCheck tests.
Index instances by their UUID
No longer index instances by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the...
Merge branch 'stable-2.8' into 'master'
Rename directory 'Block' to 'Storage'
This patch renames the 'Block' directory to 'Storage' inthe Haskell code base. The same rename was done in thepython code base earlier this quarter. We generalize thename, because we needed a place for general storage...
Conflicts: (trival, take union of added files/tests) Makefile.am test/hs/shelltests/htools-hspace.test...
Make shrinkByType aware of individual disks
When shrinking an instances, you can't just get smaller disk footprintwhile leaving the individual disks as they are. Make the shrinkheuristic aware of that fact, and decrease all individual disks aswell. Fixes issue 484....
Index nodes by their UUID
No longer index nodes by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the ConfigWriter, including all methods....
Add hvparams to RPC call 'node_info'
This patch adds the hvparams parameter to the RPC call'node_info'. It also adjusts the related code in noded.pyand Query/Node.hs
Spindles become part of htools resource spec
Spindles are now part of resource spec. Instances get created with spindlesspecified (which are just ignored when exclusive storage is disabled).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
htools cluster score takes spindles into account
When exclusive storage is enabled, spindles are used instead of disk spaceto compute the cluster score.
Comments and variable names in computePDsk has been changed to match theactual code.
Update spindles when moving instances in htools
Spindles get updated, and errors raised when not enough free spindlesexist. No new error is raised when exclusive storage is disabled.
Unit tests included.
Unit tests for htools and exclusive storage
The existing tests are run also on nodes with exclusive storage enabled. Thevalues for spindles and exclusive storage are set in a consistent way, forboth nodes and instances.
Load node spindles data in htools
The data structure for nodes gets a new field for free spindles, and theexisting field for total spindles gets renamed to avoid identifying it withthe node parameter that had the same name. These fields get filled with...
Add a force option to the ClusterSetParams Opcode
If set, the op code will, in particular, try to set the master IPon the new netdev, even if shutting down the master IP on the oldnetdev failed.
Fix shadowing of library function
The "reads" field shadows a library function from Prelude. This commitfixes the problem.
Add unit tests for the Diskstats parser
Add one test parsing an actual /proc/diskstats file and one QuickChecktest.
Add function for generating arbitrary non-negative numbers
In many tests it is useful to have the possibility to easily generatenon-negative integer numbers.
Factor out and rename testFile function
The testFile function used in the DRBD parser test file can actually beuseful in any case where a parser has to be tested.
This patch moves it to the TestCommon file, and renames it to a more meaningfultestParser....
Fix erroneous docstring
Check real spindles in ipolicies
When exclusive storage is enabled, the spindles in instance disks are usedto check the instance policies (as outlined in design-partitioned.rst).
Add spindles to instance disks in htools
A new data type is introduced for disks to store both size and spindles.When available, spindles are filled with input data. Except for loading andstoring, spindles are ignored.
Load exclusive_storage in htools
The node parameter is loaded into the data structures. No behavior is yetmodified.
New function to load JSON arrays of optional values
This will be needed to load spindles in some htools backends. Unit testsprovided.
Add unit test for text backend + fix bug
Test serialization and deserialization of instances. Fix check of secondarynode.
Parse NIC data from allocation request in hail
Add a NIC type and extend the Instance type by a list of NIC's. Parsethe NIC's in allocation requests and store them for now. Later patcheswill make use of this field in order to ensure that the requestedinstance is only placed in node groups wich are connected to those...
Parse node group networks
Extend the Group by the network ids it is connected to. Adaptthe IAlloc backend such that the networks are parsed correctly.This also required the adaption of test data.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add disks_active to configuration
This flag tracks if the disks of an instace are supposed to be active.That's the case when an instance is running or when its disks gotactivated explicitly (and in a couple of other cases).It will be used by watcher to re-activate disks after a node reboot....
Add spindles field to disk object
The field is filled with the value provided on the command line.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Compatibility test for instances
This patch introduces a test to check the compatibilityof the Haskell and the Python representation of instances.
Instance generators
This patch introduces and enhances generators forinstances:- 'genInstWithNets' is split into the generation of an arbitrary instance and enhancing an instance with nets- 'genInst' calls 'genInstWithNets' with an empty set of initial networks to provide a reasonable default...
Annotate every arbitrary instance field
The Arbitrary instance of the 'Instance' object is writtenusing the <*> syntax. Since it often uses the 'arbitrary'generator for the instance's fields it is hard to figureout which 'arbitrary' fills which instance field. This...
Generators for disks
This patch adds generators for Disk instances to the Haskelltest code. It uses somewhat more reasonable generators tofill the fields instead of just arbitrary values.'genDiskWithChildren' is a generator that generates a diskwith a specified number of disk children. To avoid shooting...
Merge remote-tracking branch 'origin/stable-2.8'
Conflicts: lib/bdev.py lib/cmdlib.py
Add reason trail to haskell opcode format
The haskell type definition of opcodes should remain aligned with the pythonone.
Fix bug in group queries related to node/instance fields
Since we use the primitive string type for group UUIDs, the groupfields have a bug where we pass the group name as filter for nodetests, whereas the nodes themselves use the group UUID. This results...
Add test for DRBD 8.4 /proc/drbd parsing
The current parser already supports the output of DRBD 8.4 /proc/drbd,so add tests which actually verify this behaviour.Note that the empty lines in proc_drbd84.txt are actually present in theoutput of DRBD 8.4, they always appear when minors are not contiguous....
Add test for empty version in /proc/drbd
The previous patch removed a (rather implicit) test for /proc/drbd notcontaining a version line. This adds an explicit test for this case.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Signed-off-by: Michele Tartara <mtartara@google.com>...
Extract /proc/drbd parsing code into DRBD8Info
As the DRBD8 class got bigger due to the previous merge of BaseDRBD, nowparts of it are ripped out into DRBD8Info. This new class parses/proc/drbd and exposes the information in an easily accessible way. This...
Add multiple min/max specs in instance policy
Now instance policies can contain more than one min/max specs. This is themain element of the "Constrained instance sizes" section in the"Partitioned Ganeti" design doc.
This is a big patch, but changing the type of a configuration item requires...
Limit specs in instance policies are always complete
Specs used to specify limits in instance policies are always complete, evenat group level (only the whole limit set can be overridden). This is inpreparation for introducing multiple limits.
Remove old "reason" implementation
Remove the useless parts of the old, partial, implementation of the support fortracking the reason of instances state change, before implementing the newreason trail support, as per the design document.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add nic/disk name and UUID fields in Haskell code
Add name and uuid fields to Disk and nic Haskell objects. Also,since they now have an UUID, make them instances of UuidObject.
Signed-off-by: Christor Stavrakakis <cstavr@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Add name to INicParams and IDiskParams
This commits adds name to NIC and Disk modification definition.
Signed-off-by: Chris Stavrakakis <cstavr@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'devel-2.7'
Removes obsolete 'enabled storage types'
Since managing of different storage units is now done usingdisk templates and not storage types, we remove the obsoleteenabled storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtarara@google.com>
gnt-cluster modify: modify enabled disk templates
Adds setting of the list of enabled disk templates to 'gnt-cluster modify'.Note that this does not yet include any verification regarding disk templatescurrently used by instances.
Make the disks parameter available to the constructor
In that way, tools building on Instance will benefit from the correctedverification semantics of the instance policy on disk space.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Properly export errors while reading job list
In case of problems while reading the job list from disk (such as permissionerrors) confd would silently fail, writing a warning on the log file butsending an empty list and no error message to the client....
Fix a bug in the Runtime tests
A group name was erroneusly inserted in the list of user names.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add parameter pnode to OpInstanceSetParams
This parameter will be used to set the new primary nodeof an instance, assuming that the disks have been movedby other means (outside ganeti's control).
Refactor ispecs in ipolicy structures
Minimum and maximum instance specs are put together into a single elementof the instance policy. This is in preparation for introducing multiplemin/max specs.
gnt-cluster modify: dis/enabling storage types
This patch extends the 'gnt-cluster modify' command to manipulate the listof enabled storage types. Note that this currenlty does no validationwith respect to whether or not there are instances currently using a storage...
Add Mond to the list of possible daemons
Also, add its logfiles and extra log files.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Expand TestHelper to allow non-underscore prefixes
HLint 1.8.28 requires us to always add "ignore CamelCase", which canbe problematic sometimes (e.g. when using OverloadedStrings). Let'sexpand TestHelper to also support less-standard 'caseFooBar' names, so...
Change to CamelCase format in Test/Ganeti/Objects.hs
This is in preparation for using OverloadedString extensions in thisfile, which conflicts with hlint 1.8.28's handling of annotations.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add simple Ip4Address/Ip4Network types
This patch adds some very simple IPv4 address/network types, and usesthem in the 'Network' config object.
We need these in order to properly compute the reserved IP addresses,without depending on an external library (which I haven't found, by...
Conflicts: Makefile.am (curl changes and new hs directories)...
Expand Haddock to run over test files as well
This patch does multiple enhancements to the way we build the Haddockdocs, motivated by the fact that if we don't run Haddock over testfiles as well, bad formatting can be submitted and will accumulateover time (without any checks)....