Hs2Py constants: add 'devConsole' and syslog related
Add 'devConsole' and syslog related constants to the Haskell to Pythonconstant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add Template Haskell 'declareLADT'
Add 'declareLADT' in Template Haskell module to declare Haskelldatatypes using 'String's directly as values for the JSONserialization, as opposed to 'Name's which is what the current'declareADT' allows. To achieve this, 'genFromRaw' must be...
Haskell to Python constants: add exit code constants
Add exit code constants to Haskell to Python constant generationinfrastructure. The module 'Ganeti.ConstantUtils' must be importedqualified in order to avoid a clash between'Ganeti.ConstantUtils.exitFailure' and...
Haskell to Python constants: add 'daemonsLogbase'
Move constant 'DAEMONS_LOGBASE' from Python to Haskell to beautomatically generated.
Flip dependency between Haskell types and constants
Before this patch, Haskell types, such as, 'GanetiDaemon' and'GanetiGroup', and related functions were taking their values fromHaskell constants. However, given that the role of Haskell to Pythonconstants is to leverage Haskell and its typesystem, it makes sense to...
Replace import in 'Ganeti.Path'
Given that Haskell has its own generated 'AutoConf' module, it makessense that other Haskell modules import 'configure' variables directlyfrom 'AutoConf' instead of importing the Python to Haskell generatedconstants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Fix monitoring daemon extra log filepaths
Haskell constants 'daemonsExtraLogfilesGanetiMondAccess' and'daemonsExtraLogfilesGanetiMondError' cannot be constants becausetheir Python counterparts are calculated through'pathutils.GetLogFilename', which indirectly depends on the...
Make CPUload Data Collector aware of clock ticks
Modify CPUload data collector to get the number of clock ticksper second from Posix.Unistd. Since CPUload collector convertsclock ticks (jiffies) to seconds and the clock ticks to secondsratio varies between linux distributions, it must be read from...
Make hbal support the option --ignore-dynu
This will allow to do balancing based only on static information.In particular, the effect of balancing towards same number ofinstances per node that the dynamic usage consideration hasif no usage file is provided, can be turned off, if this is...
Make ExtLoader honor the --ignore-dynu option
Make the external data loader correctly honor the --ignore-dynu optionby clearing the dynamic utilisation data from all instances if the optionis given.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Provide utility function to clear utilisation data
Using this utility function, the dynamic utilisation of all instancescan be set to 0. This will have the effect of all utilisation being ignored.
Define a new option --ignore-dynu
This option, if set, will tell htools to ignore any dynamicuse data.
Merge branch 'stable-2.9' into master
Add 'daemonsPorts' and related constants
Add 'Protocol' datatype
Add 'Protocol' datatype which is necessary for the 'daemonsPorts'constant.
Fix instances of 'PyValue'
Fix instances of 'PyValue', namely, for tuples and maps.
Haskell to Python constants: more constants
Merge branch 'stable-2.8' into stable-2.9
Add user separation constants
Add user separation constants, such as, 'daemonsGroup', 'rapiGroup',and 'rapiUser', to the Haskell to Python constant generationinfrastructure.
Add AutoConf.hs.in
Add 'AutoConf.hs.in' which is the template for generating the file'AutoConf.hs'. This file mimics the Python equivalent'lib/_autoconf.py' and it is necessary for Haskell constants to beable to retrieve their values from 'configure'....
Haskell to Python constants: add 'adminstAll'
Add constant 'adminstAll' to Haskell to Python constant generation.
Add 'FrozenSet'
Add type 'FrozenSet' that wrap a Haskell 'Set', and a guardedconstructor for this type and a 'PyValue' instance.
Fix module documentation
Fix 'Ganeti.Hs2Py.ListContants.hs.in' template module documentation towarn about automatic module generation.
Fix 'HsConstants' module documentation to warn about proper moduleusage.
Wrap 'Set' in 'ListSet' for the opcodes
In what Haskell to Python opcodes are concerned, a Haskell 'Set' istranslated into a Python 'list'. In other words, currently, opcodesthat handle sets of parameters are actually handling lists becausethis is how sets are currently encoded. This patch introduces a new...
Fix 'List' instance of 'PyValue'
Fix 'List' instance of 'PyValue' to properly convert Haskell lists toPython lists.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Mark the DSA host pubkey as optional
Commit a9542a4 introduced support for DSA SSH keys. However, the dsahostkeypubfield added to the config is not marked as optional in the Haskell components.As a result, luxid thinks the config file is corrupt and refuses to start. We...
Haskell to Python opcodes: fix Unit
Fix how Haskell to Python opcode generation handles unit. It seemsthat, in 'template-haskell-2.7.0', unit is represented by TemplateHaskell as a constructor of the form '(ConT name)', where 'name' isthe constructor name for 'Unit'. However, in 'template-haskell-2.8.0'...
Import original constants instead of generated
Haskell to Python constant generation introduced duplicated constantsin Haskell. This patch eliminates the duplication and fixes Haskellto import the correct constants, instead of the generated ones.
Setup constant generation infrastructure
Setup a transitional Haskell to Python constant generationinfrastructure. This infrastructure is transitional and Pythonconstants will not be migrated to Haskell all at once. Instead, someconstants will be incrementally pulled from Python and moved to...
Rename 'Constants.hs' to 'PyConstants.hs'
Rename the generated 'Constants.hs' module, which contains the Haskellconstants generated from Python, to 'PyConstants.hs' in order toeliminate duplicated constants in Haskell, a problem introduced by theHaskell to Python constant generation, and to circumvent a problem...
Add Template Haskell code for constant generation
Add Template Haskell functions that generate a list in Haskellcontaining all the constants to be generated in Python. This list isan association list mapping the constant name to its Python value,which is stored as a string....
Move some 'PyValue' instances to a separate file
Move some 'PyValue' instances to a separate file so they can beavailable to other modules which need these instances but do not wantto depend on the constants module, which led to compilationconstraints....
Replace LD_* constants with DT_* constants
LD_* constants are basically like DT_* constants, exceptfor that both file and shared file were mapped to file.In order to not having to maintain three slightly differentsets of disk-related constants (DT, LD and ST), we merge...
Add new node parameters to Object.hs
In commit 807d8853, new node parameters were introduced in the python world.Add them to the haskell world as well to restore consistency.
Make the DRBD collector more failure-resilient
If information about instances is not available, just log the error andcontinue without it.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add function to unwrap Results logging failures
Add logWarningIfBad, a utility function similar to exitIfBad, that logs awarning and returns a default value instead of just crashing the program ifthe unpacked value is Bad.
Signed-off-by: Michele Tartara <mtartara@google.com>...
disk_template param is optional in instance create
The disk_template parameter is optional during disk create (the firstenabled disk template is taken in this case), so don't require it.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Use FQDN to check master node status
The master node name in SS conf is stored as FQDN, so also use the FQDNon each node to check if it is the master node.
This fixes issue 551.
Fix opcodes and parameters
Merge branch 'stable-2.9'
Add hs2py program that generates Python opcodes
Add hs2py which is a Haskell program that uses the 'Hs2Py' module tooutput the actual Python opcode strings.
Generate Python opcodes from opcode descriptors
Add Haskell module Hs2Py which contains the helper functions thatgenerate the Python opcodes as strings from the Haskell opcodedescriptors produced by Template Haskell.
Remove TagType
By removing 'TagObject' in the previous patch, 'TagType' also becameuseless, since it was only used by 'TagObject'. This patch removes 'TagType'.
Remove TagObject
The 'TagObject' datatype became useless because it was replaced by'TagKind'. This patch removes 'TagObject', its 'Arbitrary' instance,and related functions.
Update Harep, Query server, and tests
Update Harep, Haskell query server, and tests concerning Luxi andopcodes to reflect the changes to Haskell to Python opcodegeneration. This change is necessary because TagObject is replaced byTagKind and some types in opcodes and parameters changed to be...
Add Python opcode generation
Add opcode documentation
Add Haskell module containing opcode documentation for all theopcodes.
Add Haskell types for opcodes and parameters
Add Haskell types for IP addresses (version 4 and 6), IP networkaddresses, tag kind, and query result code.
Support DSA SSH keys in bootstrap
As outlined in issue 338, Ganeti failed to initialize a cluster if noRSA SSH key is present on the master node. This patch extends Ganetissupport to DSA keys, so clusters with only DSA keys are possible now.
This fixes issue 338....
Include VCS version in `gnt-cluster version`
Also print the VCS version in the output of `gnt-cluster version`. Thismakes the VCS version also available over RAPI, etc.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
mond: Introduce a new threading mechanism
Change mond's interface in order to be able to call periodically thecollection functions of the data collectors that support suchfunctionality. The data of such a collector is stored in a map with thedata collector's name as the key....
Add CPUload collector
Introduce CPUload data collector as described in the correspondingdesign document. Add two data types in Datacollectors's Types.
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>Signed-off-by: Michele Tartara <mtartara@google.com>...
Introduce LoadParser and Types for a new Collector
Introduce the Parser and Types for a new Data Collector.
Signed-off-by: Spyros Trigazis <strigazi@gmail.com>Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Factor out utility functions from DiskstatsP
Remove the utility functions from DiskstatsP and import them fromParsers. Also modify the parser implementation to a more readable form.
Add generic Parsers file
Extract the utility functions of Diskstats's collectorParser so other collector Parsers can be able to use them.
Add cleanup parameter to instance failover
Most of the code is shared with instance migrate, so we actually only needto add the parameter and pass its value along the the common code.
Also, tests and harep are updated to support the right set of options to...
gnt-cluster verify: consider shared file storage
This patch enhances 'gnt-cluster verify' in a way that itnow validates the acceptance and existance of the sharedstorage directory.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
gnt-cluster modify --shared-file-storage-dir
This patch introduces to 'gnt-cluster modify' the option'--shared-file-storage-dir' to change the default directoryfor instances using shared file storage at cluster runtime.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Extend gnt-instance show to show vlan parameter
Extensions for "gnt-instance show" to output the vlan parameter whichwas introduced by my recent patch series.This is done by extending the nic object that is used to output theinfo. Every other occurrence of this object is adopted as well....
gnt-cluster {init, modify} --file-storage-dir
This patch implements consistent usage and behavior ofthe --file-storage-dir option in 'gnt-cluster init'and 'gnt-cluster modify'. It includes a bunch of unittests as well.
Additionally, it enables the previously written unit...
Remove extra whitespace
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Fix permission errors for split users
Correctly set ownership and permissions for daemon log files, correctthe name of the luxid logfile and set the ownership of the query socketcorrectly.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Conflicts: src/Ganeti/Utils.hs (trivial)trivial
Let ReqNodeInstances work with node UUIDs
The "primaryNode" and "secondaryNode" fields of "Instance" entities in thecluster configuration where changed to use UUIDs instead of names. TheReqNodeInstances query inside Confd was not upgraded yet, and was thus...
Add documentation line to getNodeInstances
Add debug logging to Confd
Knowing the replies actually sent helps tracking down problems much moreefficiently.
Extending nicparams to support vlan parameter
This parameter is used to store the vlans each nic is connected to.It is only used if the network is in openvswitch mode.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Fix permission problem related to Issue 477
Commit 91525dee856951ace940c78b6254a1c7344b4803 fixed Issue 477 but broke"gnt-cluster info".
This commit offers a solution to both problems, by changing the permissionof the socket instead of changing the permission the confd process is run...
Add hs function to easily change file ownership
The Haskell library functions only allow to change file ownership usinguid/gid. A function for doing that with explicit names is added by thiscommit.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Rename queryd to luxid
As queryd will, in the future, handle all LUXI request, queue jobs andmost likely perform various other tasks, it is renamed to luxid already.This will safe some headache when upgrading Ganeti installations, as wedon't have to deal with a daemon rename....
Add queryd daemon (split from confd)
queryd is added as a new daemon which handles configuration queries overLUXI. This functionality was removed from confd, which now only queriesover the network.
The queryd user is added to the master group such that it can access...
Extract ConfigReader from Confd/Server.hs
Confd's functionality to watch the Ganeti configuration file isextracted to the ConfigReader module. No functional changes areintroduced.
This extraction makes will enable us to split queryd from confd, asqueryd will have to use the same functionality....
Add timestamps to haskell network query fields
Add timestamp fields to the list of available network query fields inthe Haskell code.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.8' into master
Merge branch 'stable-2.7' into stable-2.8
Conflicts:...
Verify file storage path
This patch adds two verification steps to 'gnt-clusterverify':- The configured file storage directory is checked against the allowed file storage directories file.- We check whether the configured file storage directory is existing and writable on each node....
Allow modify_etc_hosts to be changed
The modify_etc_hosts options, enabling the cluster to modify the /etc/hostsfiles of nodes, and to keep them in sync, could only be set at cluster inittime.
With this commit it can now be changed through modify_etc_hosts as well....
Add luxiReqQueryNetworks to LuxiOp
When the QueryNetwork was introduced as a method, apparentlyit was forgotten in the Haskell world. Add it here as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Log received message at debug level
At debug level, we can well afford to have a detailed entryfor each message received by a server.
Log RPC errors from inside executeRpcCall
executeRpcCall is the function to be used for executing RPCs, so it makes senseto use it as the single point for logging all thte RPC call errors.
Fixes Issue 293.
Factor out the logRpcErrors function
This function can be useful to multiple RPC calls, therefore it is movedto the file containing the common RPC functions.
Also, it is made more generic by changing its signature.
Add support for querying network timestamps
Add creation and modified timestamps when creating a new network, andextend the available query fields for networks with these fields,namely 'ctime' and 'mtime'.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>...
Prevent silent failure in case of connection problems
While running "gnt-node list", if a query to ConfD fails (especiallybecause of permission problems) it used to just fail silently, with gnt-nodeshowing question marks instead of data.
With this patch, ConfD records the error in its log file, together with a...
Include "instance" information in LV data collector
This commit enables the logical volume data collector to get information aboutthe instances and to link it to the information about logical volumes.
The list of parameters accepted by the collector is expanded to allow proper...
Add "includeLogicalId" function for Disks
This function checks whether a disk contains a given Lvm logical ID,directly or through its children.
Unit tests are added as well.
Add option for loading serialized instances
Monitoring CLI tools might have to load serialized lists of instances(mainly for testing reasons). This patch adds an option to allow that.
Factor out lv info gathering function
The buildJsonReport function will soon have to perform the coupling ofinstance data with LVInfo data. In preparation for that, in order to makeit more readable, the instructions for obtaining LVInfos are factored out...
Add "instance" field to LVInfo
Extend the LVInfo data structure with the field for storing the name ofthe instance it is paired with.
Update the tests accordingly.
Factor out the getInstances function
The getInstances function can be useful in general, but is defined insidethe InstStatus data collector. This commit takes it out and adds it toa proper (newly created) library.
Extraction of storage info by type
There was a bug in the node queries. It was assumed thatthe returned storage space information was in a particularorder. With the changes in the storage reporting, thisorder is not that reliable anymore, in particular, the...