Add data type for data collector category
Also, update the DRBD data collector to use and export it.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Export the dcVersionInformation for the Drbd collector
Also, update the JSON output (and the design document) so that it is notin camelcase anymore. This is part of a bigger effort to remove camelcasefrom the exposed JSON.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add data collector version data type
Define the new data type and update the DRBD data collector to use it.
Fix comment line position in DRBD data collector
Export dcName information of Drbd data collector
Change format of the empty status message
Having 'null' instead of an empty string for an empty status messageincreases the complexity of the code with no real improvement.
This patch modifies the monitoring agent design document solving this issue.
Improve INSTALL documentation
Additional clarification note added.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Document more installation dependencies
A couple of them were missing and are added by this patch.
Unit tests for config.ConfigWriter.VerifyConfig()
The test is aimed mostly at instance policies, which changed recently.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
cfgupgrade: Add support for newly refactored ipolicies
Instance policies in the configuration are upgraded and downgraded. Bothoperations are idempotent.
Refactor ispecs in ipolicy structures
Minimum and maximum instance specs are put together into a single elementof the instance policy. This is in preparation for introducing multiplemin/max specs.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
cfgupgrade: Add --downgrade option
It's now possible to go back to the previous stable version. Unit testsprovided.
This is mostly useful during development, when going from master tostable/devel.
cfgupgrade: Refactor code
All the upgrading code is now in one function. No functionality has beenchanged.
Configuration is verified on load
If there are errors in the configuration when master daemon is started,they are now logged and reported. Previously they were logged sometimes atstartup, and when performing any change to the configuration.
Refactor cli.CreateIPolicyFromOpts()
The code relative to instance specs has been moved into a differentfunction. This in preparation for further patches to have a more clear codestructure.
Remove duplicate filling of instance policy
Instance policy fields were manually filled in CreateIPolicyFromOpts() andthen filled again by the caller. Now they are filled once inside thefunction.
Unit test for cli.CreateIPolicyFromOpts()
Useful for subsequent patches.
Amend partitioned design doc for multiple ispecs
There will be only one standard specification in instance policies.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
QA: Fix assertion
The assertion was always true because "all" was missing.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
More info on using the RAPI
Add some additional information about how to use/test the RAPI,including how to specify authentication credentials.
gnt-instance info uses a revised format
The code is more modular and the output is YAML-compliant. QA updated totake advantage of it.
gnt-cluster info uses a revised format
The code is more modular and the output is YAML-compliant.
Added function in QA that uses PyYAML to parse the command output, and QAis updated to take advantage of it (instead of using lots of complicatedREs).
Generic function to print data for gnt-xxx info
The function produces a human-readable output, which is also a valid YAMLfile, from an intermediate data structure. This will be used by "gnt-xxxinfo" commands. The output of "gnt-xxx info" was almost YAML-compliant, and...
Fix bug in rlib2 unit tests
The "queryargs" parameter of the init function of rlib2 classes shouldbe a dictionary, as defined in the ResourceBase parent class (inlib/rapi/baserlib.py).
In the rlib2 and baserlib unit tests, when the queryargs parameter is not...
Remove use of deprecated function
cgi.parse_qs() is deprecated in python 2.6 and substituted byurlparse.parse_qs().
This patch modifies the code to use the new function.
gnt-cluster modify: limit storage type consistency check
This fixes the currently broken Q&A. The consistency checkfor storage types is now only run when the 'enabled_storage_types'are actually changed by 'gnt-cluster modify'.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Check if instances use disk templates that get disabled
When disk templates get disabled via 'gnt-cluster modify', so farwe did not check if any instances still use a disk template of thattype. This patch adds the necessariy check.It also includes adding a map of disk templates to storage types...
Fixing some mistakes in design doc for space reporting
- Use entities as examples for storage types that are actually storagetypes.- Fix mixing up 'external_storage' and 'exclusive_storage'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Use 'storage type' consistently in design doc
To minimize the confusion between whether or not a storage typeis the same as a storage method, I updated the design doc touse 'storage type' instead of 'storage method' consistently.
Upgrade configuration wrt enabled storage types
This implements an online update of the configuration for when a configurationis loaded that does not yet contain the 'enabled_storage_types' attribute.Note that this will require more changes as dis/enabling of (shared) file...
QA: Extend cluster QA wrt enabled storage types
This extends the QA scripts to test the changes in 'gnt-cluster init','gnt-cluster info', and 'gnt-cluster modify' with respect to enabledstorage types.
gnt-cluster init: add enabled_storage_types parameter
This patch extends the 'gnt-cluster init' command to set the listof enabled storage types on cluster creation.
gnt-cluster modify: dis/enabling storage types
This patch extends the 'gnt-cluster modify' command to manipulate the listof enabled storage types. Note that this currenlty does no validationwith respect to whether or not there are instances currently using a storage...
gnt-cluster info: show enabled storage types
This extends the 'gnt-cluster info' command to list the storage typesthat are enabled on the cluster. It also fixes the broken indentationin the 'handleCall' function.
Add 'enabled_storage_types' to the cluster config
This patch adds the cluster's new field 'enabled_storage_types'to the configuration objects in python and haskell.
Add constants for storage types to constants.py
This patch adds the currently known and used storage methods to theconstants.py file. Note that storage types are different from disktemplates. For example both disk templates, drbd and plain, use lvmstorage types....
Fix Mond man page
Remove some erroneous formatting from the Mond man page.
Add monitoring HTTP API structure
Add all the supported commands to the API.The actual response is still to be implemented.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add basic HTTP server functionalities to Mond
Add a stub implementation of the Mond HTTP server to Mond using the Haskellsnap-server library.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Update documentation for monitoring daemon
This commit updates the installation instructions and the user guide.
Merge branch 'devel-2.7'
Trivial merge: no conflicts...
Add the core of the monitoring daemon
This commit adds the core infrastructure of the monitoring daemon,and integrates it in the build and test systems.
The actual functionality of the monitoring daemon is still completelymissing.
cfgupgrade: Fix error messages
Messages were inconsistent. You could get both an error and a messagesaying that everything was fine.
Update "FIXME" string in RAPI
We are not ready for this change yet. Let's push it to 2.8.
Typo 'repot' in Server.hs
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix typo 'VolumeN' in bdev.py
Add design document for the "reason trail"
This commit adds the design document for introducing "reason trails",tracing the reason why opcodes are executed, step by step.
Add Mond to the list of possible daemons
Also, add its logfiles and extra log files.
Add infrastructure for allowing additional logfiles
Some daemons will need more than the single logfile that is currentlyallowed. This patch introduces the infrastructure to allow this.
Add user and group for the monitoring agent
This commit adds the user and group variables for the monitoring agent in allthe configuration files where they are required.
rapi client: add target_node to migrate instance
This allows migrating to any node, as it is already possible forfailover, when instances are externally mirrored.
Signed-off-by: Daniel Krambrock <danielk_lists@z9d.de>Signed-off-by: Guido Trotter <ultrotter@google.com>...
Make diskless instances externally mirrored
This addresses Issue 237.
Mirroring no disk is a no-op. As such we can treat them like mirroredinstances, since the data they need (none) will be present on all nodes.
This is definitely enough to failover or migrate instances with a manual...
Fix migrate/failover -n for ext mirror storage
This fixes issue 396.
- Fix a wrong comment that mentions drbd8 when actually the code acts only on externally mirrored instances.- Fix a wrong assert that requires failover/migrate to acquire the NAL on externally mirrored instances: this is the case only when a...
Enable split queries for the network
Now that all fields are implemented, and (I think) behave equivalentto the Python implementation, we can switch on the split queries fornetworks.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add a simple tool for checking split-query equivalence
This is not run automatically (although it could/should), but is veryuseful during development.
Fix network query field types/names in the Haskell code
The headers/type/descriptions had some differences from the Pythoncode, when checked for exact equivalence.
Introduce better item getter helper in query.py
And also use it for simple network fields. Currently the Pythonlibrary returns jsnull, which is formatted as 'None' by the Pythonclient, which (IMHO) is not nice, since these fields are strings (someof them) or "other". An undeclared gateway is easier to diagnose when...
Fix gnt-backup list -o node via confd
Currently, the 'node' field is declared as a simple config field, sowhen only selecting this fields, the runtime gathering is no longerrun and it's presumed that all nodes have a backup. So the output isnot truthful (instead of just listing the nodes with at least one...
Make gnt-node list -o(p|s)inst_list output stable
Currently, both the Python and Haskell code return the internalinstance list unsorted, which means the output can vary depending onthe phase of the moon (well, the Haskell code actually uses internallya tree, sorted by the instance name, but it's implementation detail)....
Sort instance list in gnt-group list -opinst_list
The Python code currently sorts this, but the Haskell code not.
This should maybe have a test, but I'm not sure how far we want toencode such properties in tests… (and the real reason I'm not addingone is that we don't have a way to generate a random cluster with...
Change node disk/hv_state query in confd
Currently, the Python code returns either FS_UNAVAIL (if theseattributes are None) or the proper dicts. As we don't allow editing ofthese attributes, in most cases they will therefore be FS_UNAVAIL onthe client....
A few style fixes in Ganeti.Network
Side-effects of working on some other network-related stuff…
Expand TestHelper to allow non-underscore prefixes
HLint 1.8.28 requires us to always add "ignore CamelCase", which canbe problematic sometimes (e.g. when using OverloadedStrings). Let'sexpand TestHelper to also support less-standard 'caseFooBar' names, so...
Change to CamelCase format in Test/Ganeti/Objects.hs
This is in preparation for using OverloadedString extensions in thisfile, which conflicts with hlint 1.8.28's handling of annotations.
Add simple Ip4Address/Ip4Network types
This patch adds some very simple IPv4 address/network types, and usesthem in the 'Network' config object.
We need these in order to properly compute the reserved IP addresses,without depending on an external library (which I haven't found, by...
Add missing external_reservations query field in confd
Based on the implemented Ip4Network/Address types, we can now computethe (external) reservations.
Update design doc about free space reporting
This patch updates the design doc "Design correct reporting of storagefree space". The modifications were chosen to not conflict with anyfuture changes of Ganeti regarding storage pools.
Fix QA: skip reinstall for diskless instances
Diskless instances cannot really be reinstalled, not even through RAPI.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix typo in the monitoring agent design doc
Fix style error in hconfd
The first line of a function should be blank, unless it is able to contain thewhole function.
Add QA for instance creation with policy violation
When instance policy is violated, creation fails.
Add QA for policy-instance interactions
Violations on policy changes are checked.
Add QA for cluster policies
qa_cluster.TestClusterSetISpecs() is exported as it will be used in futuretests.
Unit tests for objects.InstancePolicy + a fix
Tests for: objects.InstancePolicy.CheckParameterSyntax() objects.InstancePolicy.CheckDiskTemplates() objects.InstancePolicy.CheckISpecSyntax()
Instance policies with an empty disk-template list now are reported....
Unit tests for objects.FillIPolicy() + small fix
IPOLICY_DEFAULTS is now a legal policy (the disk-templates entry was a setinstead of a list, before).
Fix upgrade of policy in objects.Cluster
Unknown elements were silently removed on startup. This means that asoftware upgrade could result in lost configuration information ifcfgupgrade wasn't run promptly.
Added unit test for Cluster.UpgradeConfig() to cover this case....
Fix instance policy checks for default back-end parameters
Policy violations of back-end parameters that used the cluster defaultvalue were not reported in cluster-verify.
Fix restoring default instance specs in group policies
"default" was not accepted as a valid input value for instance specs ingroup policies, due to a bug introduced in 2cc673a3e (and released with2.6.0). Added QA for this and another similar case.
Unit tests for cmdlib._GetUpdatedIPolicy()
Not 100% coverage, though.
Fix policy check for disk templates
Instance disk template is checked against the policy, and disklessinstances aren't checked for the number of disks.
Fix merge 8e09e801 that resulted in duplicated code
A fragment in LUInstanceCreate.CheckPrereq() removed in commit ba147ff8 wasreintroduced in merge 8e09e801 due to a change in df28c49b.
GanetiRapiClient: fix the no_remember option
There was a typo which prevented the correct option from being passed toRAPI
Signed-off-by: Daniel Krambrock <danielk_lists@z9d.de>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>...
Change hbal behaviour in case of early exit
Currently, hbal exits with status 1 if early exit is requested, evenwhen all jobs are successful. This is counter-intuitive behaviour, solet's fix it (Issue 386).
Note that the man page had conflicting information already, so it's a...
Fix "reason" opcode parameter description
The "reason" opcode parameter can now apply to multiple opcodes, not just theone for rebooting instances.
Fix build/sphinx_ext.py with tuple defaults for op params
When an OpCode's parameter has a tuple as default value, this codewill break:
buf.write("defaults to ``%s``" % default)
The patch fixes this and other potential cases by always passing atuple to '%'....
Fix bug in man build rule
Currently the man page build rule is run without ‘-e’, so the actualchecks in there are just warnings (check-man-warnings,check-man-dashes).
The patch fixes this and also fixes the man pages which made me see theproblem.
Additionally, check-man-dashes is now verbose, otherwise it's hard to...
Fix hscolour style sheet building
Currently we only symlink this into two hard-coded directories, whichbreaks the source code viewing for all the others.
The patch replaces this with automatic building; as playing withrelative symlinks is non-trivial, I just generate it many times (the...
Make "reason" a common opcode attribute
The "reason" attribute will be common to many opcodes.This patch factors it out so that it will be possible for the next commitsto use it without code duplication.
Conflicts: Makefile.am (curl changes and new hs directories)...
Update required pylint version
This should help a lot of users, since pylint 0.21.1 was quite an oldversion…
The tree is "clean" w.r.t. the new pylint version.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Constants.hs.in: improve Haddock markup in the template
convert-constants: generate better Haddock markup
Currently the Python source constants are used as-is, which is notnice on the eyes (especially for unqualified names, it was hard toseparate them visually).
Expand Haddock to run over test files as well
This patch does multiple enhancements to the way we build the Haddockdocs, motivated by the fact that if we don't run Haddock over testfiles as well, bad formatting can be submitted and will accumulateover time (without any checks)....
Switch LispConfig double comparison to relative error
This further improves the comparison for "non-trivial" numbers. Without this patch, there are still cases where the absoluteerror is too big, and we need to switch to relative error.
Concept has been taken from...
Make the XmParser config test runtime more consistent
Currently, the test uses a frequency of 5 string/5 double/1 list forgenerating Arbitrary instances of ListConfig. However, the list casehas simply a "choose (1, 20)" `vectorOf` arbitrary, which means it...
Switch Attoparsec parser from double to rational
According to the documentation, “This function is almost ten timesfaster than rational, but is slightly less accurate. For 94.2% ofnumbers, this function and rational give identical results, but forthe remaining 5.8%, this function loses precision around the 15th...
Improve output of the XmParser config test
Currently, this tests and its helper function 'isAlmostEqual' usesplain booleans to signify failures, which means you can't really debuga failed test. The patch changes the call chain to use annotatedproperties all through, which results in messages like:...