Michele Tartara [Thu, 14 Mar 2013 12:47:24 +0000 (12:47 +0000)]
Add Kind data type for data collectors
Also, add it to the DRBD data collector, and export it from there.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michele Tartara [Thu, 14 Mar 2013 12:39:23 +0000 (12:39 +0000)]
Add data type for data collector category
Also, update the DRBD data collector to use and export it.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michele Tartara [Thu, 14 Mar 2013 12:02:26 +0000 (12:02 +0000)]
Export the dcVersionInformation for the Drbd collector
Also, update the JSON output (and the design document) so that it is not
in camelcase anymore. This is part of a bigger effort to remove camelcase
from the exposed JSON.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michele Tartara [Thu, 14 Mar 2013 11:50:41 +0000 (11:50 +0000)]
Add data collector version data type
Define the new data type and update the DRBD data collector to use it.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michele Tartara [Thu, 14 Mar 2013 13:41:43 +0000 (13:41 +0000)]
Fix comment line position in DRBD data collector
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michele Tartara [Thu, 14 Mar 2013 10:40:52 +0000 (10:40 +0000)]
Export dcName information of Drbd data collector
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michele Tartara [Thu, 7 Mar 2013 17:36:42 +0000 (18:36 +0100)]
Change format of the empty status message
Having 'null' instead of an empty string for an empty status message
increases the complexity of the code with no real improvement.
This patch modifies the monitoring agent design document solving this issue.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michele Tartara [Thu, 28 Mar 2013 12:43:50 +0000 (12:43 +0000)]
Improve INSTALL documentation
Additional clarification note added.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Thu, 28 Mar 2013 10:10:35 +0000 (10:10 +0000)]
Document more installation dependencies
A couple of them were missing and are added by this patch.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Wed, 20 Feb 2013 15:14:31 +0000 (16:14 +0100)]
Unit tests for config.ConfigWriter.VerifyConfig()
The test is aimed mostly at instance policies, which changed recently.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Tue, 26 Mar 2013 22:22:06 +0000 (23:22 +0100)]
cfgupgrade: Add support for newly refactored ipolicies
Instance policies in the configuration are upgraded and downgraded. Both
operations are idempotent.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Tue, 19 Feb 2013 21:14:13 +0000 (22:14 +0100)]
Refactor ispecs in ipolicy structures
Minimum and maximum instance specs are put together into a single element
of the instance policy. This is in preparation for introducing multiple
min/max specs.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Wed, 27 Feb 2013 14:28:12 +0000 (15:28 +0100)]
cfgupgrade: Add --downgrade option
It's now possible to go back to the previous stable version. Unit tests
provided.
This is mostly useful during development, when going from master to
stable/devel.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Wed, 27 Feb 2013 14:21:59 +0000 (15:21 +0100)]
cfgupgrade: Refactor code
All the upgrading code is now in one function. No functionality has been
changed.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Wed, 27 Feb 2013 18:05:21 +0000 (19:05 +0100)]
Configuration is verified on load
If there are errors in the configuration when master daemon is started,
they are now logged and reported. Previously they were logged sometimes at
startup, and when performing any change to the configuration.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Wed, 13 Mar 2013 20:46:07 +0000 (21:46 +0100)]
Refactor cli.CreateIPolicyFromOpts()
The code relative to instance specs has been moved into a different
function. This in preparation for further patches to have a more clear code
structure.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Wed, 6 Mar 2013 23:19:19 +0000 (00:19 +0100)]
Remove duplicate filling of instance policy
Instance policy fields were manually filled in CreateIPolicyFromOpts() and
then filled again by the caller. Now they are filled once inside the
function.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Thu, 14 Mar 2013 16:46:57 +0000 (17:46 +0100)]
Unit test for cli.CreateIPolicyFromOpts()
Useful for subsequent patches.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Wed, 20 Mar 2013 00:48:07 +0000 (01:48 +0100)]
Amend partitioned design doc for multiple ispecs
There will be only one standard specification in instance policies.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Wed, 27 Mar 2013 10:22:49 +0000 (11:22 +0100)]
QA: Fix assertion
The assertion was always true because "all" was missing.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Tue, 26 Mar 2013 16:21:43 +0000 (16:21 +0000)]
More info on using the RAPI
Add some additional information about how to use/test the RAPI,
including how to specify authentication credentials.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Wed, 13 Mar 2013 17:22:58 +0000 (18:22 +0100)]
gnt-instance info uses a revised format
The code is more modular and the output is YAML-compliant. QA updated to
take advantage of it.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Mon, 25 Mar 2013 00:58:16 +0000 (01:58 +0100)]
gnt-cluster info uses a revised format
The code is more modular and the output is YAML-compliant.
Added function in QA that uses PyYAML to parse the command output, and QA
is updated to take advantage of it (instead of using lots of complicated
REs).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Bernardo Dal Seno [Wed, 13 Mar 2013 17:22:50 +0000 (18:22 +0100)]
Generic function to print data for gnt-xxx info
The function produces a human-readable output, which is also a valid YAML
file, from an intermediate data structure. This will be used by "gnt-xxx
info" commands. The output of "gnt-xxx info" was almost YAML-compliant, and
only minor adjustments are needed to make it fully compliant.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Thu, 21 Mar 2013 15:54:43 +0000 (16:54 +0100)]
Fix bug in rlib2 unit tests
The "queryargs" parameter of the __init__ function of rlib2 classes should
be a dictionary, as defined in the ResourceBase parent class (in
lib/rapi/baserlib.py).
In the rlib2 and baserlib unit tests, when the queryargs parameter is not
actually needed, it is sometimes erroneously initialized with an empty list or a
"None" value instead of an empty dictionary.
This commit solves the problem and introduces an assertion to prevent it from
happening again in the future.
The use of the assertion is safe, because in production code the __init__
function is only called by the RAPI server, with the queryargs parameter
initialized by the Mapper.getController function (lib/rapi/connector.py) that
always returns a dictionary. It can only affect test code and future code.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Fri, 22 Mar 2013 11:37:18 +0000 (11:37 +0000)]
Remove use of deprecated function
cgi.parse_qs() is deprecated in python 2.6 and substituted by
urlparse.parse_qs().
This patch modifies the code to use the new function.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Helga Velroyen [Fri, 22 Mar 2013 10:31:46 +0000 (11:31 +0100)]
gnt-cluster modify: limit storage type consistency check
This fixes the currently broken Q&A. The consistency check
for storage types is now only run when the 'enabled_storage_types'
are actually changed by 'gnt-cluster modify'.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Thu, 21 Mar 2013 12:06:01 +0000 (13:06 +0100)]
Check if instances use disk templates that get disabled
When disk templates get disabled via 'gnt-cluster modify', so far
we did not check if any instances still use a disk template of that
type. This patch adds the necessariy check.
It also includes adding a map of disk templates to storage types
and reordering of storage types and disk templates in alphabetical order.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Wed, 20 Mar 2013 14:25:33 +0000 (15:25 +0100)]
Fixing some mistakes in design doc for space reporting
- Use entities as examples for storage types that are actually storage
types.
- Fix mixing up 'external_storage' and 'exclusive_storage'.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Wed, 20 Mar 2013 10:10:28 +0000 (11:10 +0100)]
Use 'storage type' consistently in design doc
To minimize the confusion between whether or not a storage type
is the same as a storage method, I updated the design doc to
use 'storage type' instead of 'storage method' consistently.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Wed, 20 Mar 2013 13:51:23 +0000 (14:51 +0100)]
Upgrade configuration wrt enabled storage types
This implements an online update of the configuration for when a configuration
is loaded that does not yet contain the 'enabled_storage_types' attribute.
Note that this will require more changes as dis/enabling of (shared) file
storage at configure time will be removed.
This patch includes adding a mapping from disk templates to storage types. While
I was on it, I also reordered the disk template and storage type parameters
alphabetically, to make dealing with those easier.
Also, this mentions the configuration change in the NEWS file.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Thu, 14 Mar 2013 14:09:35 +0000 (15:09 +0100)]
QA: Extend cluster QA wrt enabled storage types
This extends the QA scripts to test the changes in 'gnt-cluster init',
'gnt-cluster info', and 'gnt-cluster modify' with respect to enabled
storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Thu, 14 Mar 2013 10:49:21 +0000 (11:49 +0100)]
gnt-cluster init: add enabled_storage_types parameter
This patch extends the 'gnt-cluster init' command to set the list
of enabled storage types on cluster creation.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Wed, 13 Mar 2013 15:28:45 +0000 (16:28 +0100)]
gnt-cluster modify: dis/enabling storage types
This patch extends the 'gnt-cluster modify' command to manipulate the list
of enabled storage types. Note that this currenlty does no validation
with respect to whether or not there are instances currently using a storage
type that is being removed from the list.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Wed, 13 Mar 2013 14:38:17 +0000 (15:38 +0100)]
gnt-cluster info: show enabled storage types
This extends the 'gnt-cluster info' command to list the storage types
that are enabled on the cluster. It also fixes the broken indentation
in the 'handleCall' function.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Wed, 13 Mar 2013 13:44:20 +0000 (14:44 +0100)]
Add 'enabled_storage_types' to the cluster config
This patch adds the cluster's new field 'enabled_storage_types'
to the configuration objects in python and haskell.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Wed, 13 Mar 2013 12:21:41 +0000 (13:21 +0100)]
Add constants for storage types to constants.py
This patch adds the currently known and used storage methods to the
constants.py file. Note that storage types are different from disk
templates. For example both disk templates, drbd and plain, use lvm
storage types.
The rapi documentation asserts that valid storage types are limited
to lvm and file. With the extension of storage types, this is not true
anymore, but the implementation of listing storage units is not
implemented for the new types. We disable the assert for now, adding
a fixme to extend it once the listing of storage units is extended
for the new types as well.
'gnt-node list-storage' currently does not support any other storage
type than 'lvm' and 'file'. We limit the QA to storage types that are
currently implemented for this operation.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Thu, 21 Mar 2013 11:18:26 +0000 (12:18 +0100)]
Fix Mond man page
Remove some erroneous formatting from the Mond man page.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Thu, 7 Mar 2013 15:36:48 +0000 (15:36 +0000)]
Add monitoring HTTP API structure
Add all the supported commands to the API.
The actual response is still to be implemented.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Fri, 8 Mar 2013 18:27:06 +0000 (18:27 +0000)]
Add basic HTTP server functionalities to Mond
Add a stub implementation of the Mond HTTP server to Mond using the Haskell
snap-server library.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michele Tartara [Fri, 8 Mar 2013 18:35:41 +0000 (19:35 +0100)]
Update documentation for monitoring daemon
This commit updates the installation instructions and the user guide.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Thu, 21 Mar 2013 09:42:16 +0000 (10:42 +0100)]
Merge branch 'devel-2.7'
* devel-2.7
cfgupgrade: Fix error messages
Update "FIXME" string in RAPI
rapi client: add target_node to migrate instance
Make diskless instances externally mirrored
Fix migrate/failover -n for ext mirror storage
Trivial merge: no conflicts
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Wed, 20 Feb 2013 10:39:17 +0000 (11:39 +0100)]
Add the core of the monitoring daemon
This commit adds the core infrastructure of the monitoring daemon,
and integrates it in the build and test systems.
The actual functionality of the monitoring daemon is still completely
missing.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Wed, 20 Mar 2013 13:24:49 +0000 (14:24 +0100)]
cfgupgrade: Fix error messages
Messages were inconsistent. You could get both an error and a message
saying that everything was fine.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Michele Tartara [Wed, 20 Mar 2013 11:16:47 +0000 (12:16 +0100)]
Update "FIXME" string in RAPI
We are not ready for this change yet. Let's push it to 2.8.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Wed, 20 Mar 2013 10:42:48 +0000 (11:42 +0100)]
Typo 'repot' in Server.hs
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Helga Velroyen [Wed, 20 Mar 2013 10:22:57 +0000 (11:22 +0100)]
Fix typo 'VolumeN' in bdev.py
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michele Tartara [Mon, 11 Mar 2013 14:43:57 +0000 (15:43 +0100)]
Add design document for the "reason trail"
This commit adds the design document for introducing "reason trails",
tracing the reason why opcodes are executed, step by step.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Fri, 8 Mar 2013 17:28:10 +0000 (17:28 +0000)]
Add Mond to the list of possible daemons
Also, add its logfiles and extra log files.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michele Tartara [Fri, 8 Mar 2013 14:41:38 +0000 (14:41 +0000)]
Add infrastructure for allowing additional logfiles
Some daemons will need more than the single logfile that is currently
allowed. This patch introduces the infrastructure to allow this.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michele Tartara [Thu, 21 Feb 2013 16:25:36 +0000 (16:25 +0000)]
Add user and group for the monitoring agent
This commit adds the user and group variables for the monitoring agent in all
the configuration files where they are required.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Daniel Krambrock [Tue, 12 Mar 2013 17:17:24 +0000 (18:17 +0100)]
rapi client: add target_node to migrate instance
This allows migrating to any node, as it is already possible for
failover, when instances are externally mirrored.
Signed-off-by: Daniel Krambrock <danielk_lists@z9d.de>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Tue, 12 Mar 2013 15:07:04 +0000 (16:07 +0100)]
Make diskless instances externally mirrored
This addresses Issue 237.
Mirroring no disk is a no-op. As such we can treat them like mirrored
instances, since the data they need (none) will be present on all nodes.
This is definitely enough to failover or migrate instances with a manual
target node. Further work might be needed for support in hail.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 12 Mar 2013 15:24:44 +0000 (16:24 +0100)]
Fix migrate/failover -n for ext mirror storage
This fixes issue 396.
- Fix a wrong comment that mentions drbd8 when actually the code acts
only on externally mirrored instances.
- Fix a wrong assert that requires failover/migrate to acquire the NAL
on externally mirrored instances: this is the case only when a
iallocator is used, not if a node is manually specified.
Extra qa should be added, but this can only be done if an externally
mirrored instance or a diskless instance is tested. It is easier to add
it on the master branch, where we have diskless instances support.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 11 Mar 2013 14:44:17 +0000 (15:44 +0100)]
Add a simple tool for checking split-query equivalence
This is not run automatically (although it could/should), but is very
useful during development.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 11:53:36 +0000 (12:53 +0100)]
Enable split queries for the network
Now that all fields are implemented, and (I think) behave equivalent
to the Python implementation, we can switch on the split queries for
networks.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 11:52:16 +0000 (12:52 +0100)]
Add missing external_reservations query field in confd
Based on the implemented Ip4Network/Address types, we can now compute
the (external) reservations.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 11:50:38 +0000 (12:50 +0100)]
Add simple Ip4Address/Ip4Network types
This patch adds some very simple IPv4 address/network types, and uses
them in the 'Network' config object.
We need these in order to properly compute the reserved IP addresses,
without depending on an external library (which I haven't found, by
the way). Currently the only operation supported is 'get next IP
address', which is enough for us.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 14:37:15 +0000 (15:37 +0100)]
Change to CamelCase format in Test/Ganeti/Objects.hs
This is in preparation for using OverloadedString extensions in this
file, which conflicts with hlint 1.8.28's handling of annotations.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 14:34:59 +0000 (15:34 +0100)]
Expand TestHelper to allow non-underscore prefixes
HLint 1.8.28 requires us to always add "ignore CamelCase", which can
be problematic sometimes (e.g. when using OverloadedStrings). Let's
expand TestHelper to also support less-standard 'caseFooBar' names, so
that we can remove the annotations.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 11:43:00 +0000 (12:43 +0100)]
A few style fixes in Ganeti.Network
Side-effects of working on some other network-related stuff…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 10:26:32 +0000 (11:26 +0100)]
Change node disk/hv_state query in confd
Currently, the Python code returns either FS_UNAVAIL (if these
attributes are None) or the proper dicts. As we don't allow editing of
these attributes, in most cases they will therefore be FS_UNAVAIL on
the client.
The Haskell code however returns missingRuntime, which is FS_NODATA,
so a mismatch (that also incurs a RPC call). Fixing it is easy, and
makes the output consistent.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 10:20:29 +0000 (11:20 +0100)]
Sort instance list in gnt-group list -opinst_list
The Python code currently sorts this, but the Haskell code not.
This should maybe have a test, but I'm not sure how far we want to
encode such properties in tests… (and the real reason I'm not adding
one is that we don't have a way to generate a random cluster _with_
instances).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 10:17:46 +0000 (11:17 +0100)]
Make gnt-node list -o(p|s)inst_list output stable
Currently, both the Python and Haskell code return the internal
instance list unsorted, which means the output can vary depending on
the phase of the moon (well, the Haskell code actually uses internally
a tree, sorted by the instance name, but it's implementation detail).
By adding an explicit sort step, we guarantee both stability of the
output and consistency between the two query paths.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 10:08:56 +0000 (11:08 +0100)]
Fix gnt-backup list -o node via confd
Currently, the 'node' field is declared as a simple config field, so
when only selecting this fields, the runtime gathering is no longer
run and it's presumed that all nodes have a backup. So the output is
not truthful (instead of just listing the nodes with at least one
export, all are listed).
By simply making it a runtime field, filtering works properly and we
get the same output as the masterd code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Mon, 11 Mar 2013 09:13:24 +0000 (10:13 +0100)]
Introduce better item getter helper in query.py
And also use it for simple network fields. Currently the Python
library returns jsnull, which is formatted as 'None' by the Python
client, which (IMHO) is not nice, since these fields are strings (some
of them) or "other". An undeclared gateway is easier to diagnose when
properly reported by the server as missing, rather than "existing but
null".
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Fri, 8 Mar 2013 16:31:44 +0000 (17:31 +0100)]
Fix network query field types/names in the Haskell code
The headers/type/descriptions had some differences from the Python
code, when checked for exact equivalence.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Helga Velroyen [Thu, 21 Feb 2013 10:21:46 +0000 (11:21 +0100)]
Update design doc about free space reporting
This patch updates the design doc "Design correct reporting of storage
free space". The modifications were chosen to not conflict with any
future changes of Ganeti regarding storage pools.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Tue, 12 Mar 2013 13:11:32 +0000 (14:11 +0100)]
Fix QA: skip reinstall for diskless instances
Diskless instances cannot really be reinstalled, not even through RAPI.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michele Tartara [Tue, 12 Mar 2013 11:02:55 +0000 (12:02 +0100)]
Fix typo in the monitoring agent design doc
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Tue, 12 Mar 2013 11:07:11 +0000 (12:07 +0100)]
Merge branch 'devel-2.7'
* devel-2.7
Add QA for instance creation with policy violation
Add QA for policy-instance interactions
Add QA for cluster policies
Unit tests for objects.InstancePolicy + a fix
Unit tests for objects.FillIPolicy() + small fix
Fix upgrade of policy in objects.Cluster
Fix instance policy checks for default back-end parameters
Fix restoring default instance specs in group policies
Unit tests for cmdlib._GetUpdatedIPolicy()
Fix policy check for disk templates
Fix merge
8e09e801 that resulted in duplicated code
GanetiRapiClient: fix the no_remember option
Conflicts:
qa/qa_cluster.py
qa/qa_instance.py
Conflicts are due to QA config in master using objects instead of
dictionaries. Also updated some new QA code in devel for the same reason.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Mon, 11 Mar 2013 17:11:48 +0000 (17:11 +0000)]
Fix style error in hconfd
The first line of a function should be blank, unless it is able to contain the
whole function.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Wed, 27 Feb 2013 10:47:02 +0000 (11:47 +0100)]
Add QA for instance creation with policy violation
When instance policy is violated, creation fails.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Fri, 22 Feb 2013 18:07:33 +0000 (19:07 +0100)]
Add QA for policy-instance interactions
Violations on policy changes are checked.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Thu, 21 Feb 2013 17:58:20 +0000 (18:58 +0100)]
Add QA for cluster policies
qa_cluster.TestClusterSetISpecs() is exported as it will be used in future
tests.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Tue, 19 Feb 2013 21:13:27 +0000 (22:13 +0100)]
Unit tests for objects.InstancePolicy + a fix
Tests for:
objects.InstancePolicy.CheckParameterSyntax()
objects.InstancePolicy.CheckDiskTemplates()
objects.InstancePolicy.CheckISpecSyntax()
Instance policies with an empty disk-template list now are reported.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Tue, 19 Feb 2013 13:56:05 +0000 (14:56 +0100)]
Unit tests for objects.FillIPolicy() + small fix
IPOLICY_DEFAULTS is now a legal policy (the disk-templates entry was a set
instead of a list, before).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Wed, 27 Feb 2013 17:50:17 +0000 (18:50 +0100)]
Fix upgrade of policy in objects.Cluster
Unknown elements were silently removed on startup. This means that a
software upgrade could result in lost configuration information if
cfgupgrade wasn't run promptly.
Added unit test for Cluster.UpgradeConfig() to cover this case.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Thu, 7 Mar 2013 02:14:56 +0000 (03:14 +0100)]
Fix instance policy checks for default back-end parameters
Policy violations of back-end parameters that used the cluster default
value were not reported in cluster-verify.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Mon, 25 Feb 2013 13:16:40 +0000 (14:16 +0100)]
Fix restoring default instance specs in group policies
"default" was not accepted as a valid input value for instance specs in
group policies, due to a bug introduced in
2cc673a3e (and released with
2.6.0). Added QA for this and another similar case.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Mon, 18 Feb 2013 17:02:06 +0000 (18:02 +0100)]
Unit tests for cmdlib._GetUpdatedIPolicy()
Not 100% coverage, though.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Thu, 7 Mar 2013 00:46:14 +0000 (01:46 +0100)]
Fix policy check for disk templates
Instance disk template is checked against the policy, and diskless
instances aren't checked for the number of disks.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Thu, 7 Mar 2013 00:34:35 +0000 (01:34 +0100)]
Fix merge
8e09e801 that resulted in duplicated code
A fragment in LUInstanceCreate.CheckPrereq() removed in commit
ba147ff8 was
reintroduced in merge
8e09e801 due to a change in
df28c49b.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Daniel Krambrock [Mon, 11 Mar 2013 13:05:21 +0000 (14:05 +0100)]
GanetiRapiClient: fix the no_remember option
There was a typo which prevented the correct option from being passed to
RAPI
Signed-off-by: Daniel Krambrock <danielk_lists@z9d.de>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 8 Mar 2013 15:10:42 +0000 (16:10 +0100)]
Merge branch 'devel-2.7'
* devel-2.7:
Change hbal behaviour in case of early exit
Fix build/sphinx_ext.py with tuple defaults for op params
Fix bug in man build rule
Fix hscolour style sheet building
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Wed, 6 Mar 2013 09:58:53 +0000 (10:58 +0100)]
Change hbal behaviour in case of early exit
Currently, hbal exits with status 1 if early exit is requested, even
when all jobs are successful. This is counter-intuitive behaviour, so
let's fix it (Issue 386).
Note that the man page had conflicting information already, so it's a
good thing to clean this up.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Wed, 6 Mar 2013 15:09:48 +0000 (15:09 +0000)]
Fix "reason" opcode parameter description
The "reason" opcode parameter can now apply to multiple opcodes, not just the
one for rebooting instances.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 6 Mar 2013 11:04:12 +0000 (12:04 +0100)]
Fix build/sphinx_ext.py with tuple defaults for op params
When an OpCode's parameter has a tuple as default value, this code
will break:
buf.write("defaults to ``%s``" % default)
The patch fixes this and other potential cases by always passing a
tuple to '%'.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Wed, 6 Mar 2013 09:43:51 +0000 (10:43 +0100)]
Fix bug in man build rule
Currently the man page build rule is run without ‘-e’, so the actual
checks in there are just warnings (check-man-warnings,
check-man-dashes).
The patch fixes this and also fixes the man pages which made me see the
problem.
Additionally, check-man-dashes is now verbose, otherwise it's hard to
find out actually where in the page the error is.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Tue, 5 Mar 2013 17:13:14 +0000 (18:13 +0100)]
Fix hscolour style sheet building
Currently we only symlink this into two hard-coded directories, which
breaks the source code viewing for all the others.
The patch replaces this with automatic building; as playing with
relative symlinks is non-trivial, I just generate it many times (the
file is small enough (<250) that it shouldn't matter, size-wise, and
it takes 2ms to do it).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michele Tartara [Wed, 27 Feb 2013 10:51:28 +0000 (11:51 +0100)]
Make "reason" a common opcode attribute
The "reason" attribute will be common to many opcodes.
This patch factors it out so that it will be possible for the next commits
to use it without code duplication.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 5 Mar 2013 16:05:11 +0000 (17:05 +0100)]
Merge branch 'devel-2.7'
* devel-2.7:
Update required pylint version
Constants.hs.in: improve Haddock markup in the template
convert-constants: generate better Haddock markup
Expand Haddock to run over test files as well
Conflicts:
Makefile.am (curl changes and new hs directories)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Tue, 5 Mar 2013 15:00:50 +0000 (16:00 +0100)]
Update required pylint version
This should help a lot of users, since pylint 0.21.1 was quite an old
version…
The tree is "clean" w.r.t. the new pylint version.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Tue, 5 Mar 2013 13:12:42 +0000 (14:12 +0100)]
Constants.hs.in: improve Haddock markup in the template
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Tue, 5 Mar 2013 13:11:24 +0000 (14:11 +0100)]
convert-constants: generate better Haddock markup
Currently the Python source constants are used as-is, which is not
nice on the eyes (especially for unqualified names, it was hard to
separate them visually).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Tue, 5 Mar 2013 13:07:50 +0000 (14:07 +0100)]
Expand Haddock to run over test files as well
This patch does multiple enhancements to the way we build the Haddock
docs, motivated by the fact that if we don't run Haddock over test
files as well, bad formatting can be submitted and will accumulate
over time (without any checks).
The patch does:
- replace manual built ALL_APIDOC_HS_DIRS with automatically built one
(from HS_DIRS)
- change Haddock so that it runs from the top directory (instead of
from src/)
- change HsColour target file to be built via bash parameter
substitution, rather than sed (I don't know how to do it in one go,
so I use 2 intermediate variables)
- change 'hs-apidoc' target so that it depends on the real target
file; in case no source file has been modified, running 'make
hs-apidoc' twice will not result in two runs
- run HsColour/Haddock under en_US.UTF-8 locale, otherwise they can't
parse correctly the Unicode chars in the test files
Additionally, wrong formatting (oops) in a test file has been
corrected.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Tue, 5 Mar 2013 09:21:15 +0000 (10:21 +0100)]
Merge branch 'devel-2.7'
* devel-2.7: (23 commits)
Add export lists for files which didn't have them
Fix Haskell compatibility tests with disabled file storage
Fix QA with disabled file storage
Fix convert-constants handling of booleans
Fix handling of disabled (shared) file storage
Fix low verbosity levels in htools
Fix improperly formatted docstring
Allow iallocator to work without LVM storage
Allow rpc.MakeLegacyNodeInfo to parse non-LVM results
Fix LUTestAllocator with instance alloc
Fix confd issue regarding --no-lvm-storage
Fix networks in _PrepareNicModifications()
Fix sphinx label namespace
Clarify use of move-instance with self-signed certificates
Remove early returns in network LUs
Fix HooksDict() in case of no tags
Add networks to _AllIDs()
Fix locking in LUNetworkConnect()
Fix networks in LUInstanceSetParams()
Fix another docstring typo
...
Conflicts:
src/Ganeti/Query/Node.hs (trivial; function rename in master
and 'vgs' change in devel-2.7)
also exported new entity from TestCommon.hs (used in master)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Thu, 28 Feb 2013 13:34:28 +0000 (14:34 +0100)]
Switch LispConfig double comparison to relative error
This further improves the comparison for "non-trivial"
numbers. Without this patch, there are still cases where the absolute
error is too big, and we need to switch to relative error.
Concept has been taken from
<http://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/>.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 28 Feb 2013 10:05:45 +0000 (11:05 +0100)]
Switch Attoparsec parser from double to rational
According to the documentation, “This function is almost ten times
faster than rational, but is slightly less accurate. For 94.2% of
numbers, this function and rational give identical results, but for
the remaining 5.8%, this function loses precision around the 15th
decimal place. For 0.001% of numbers, this function will lose
precision at the 13th or 14th decimal place.”. What happens is that
for our tests, it can happen that “Attoparsec.double (show a_double)”
is quite different from “a_double”, such that we have much more than
1e-12 absolute difference.
Since our xm lists should not be too big, I think switching to
rational is better. Next patch also changes the way we compare
doubles, so maybe this patch is not really needed…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Thu, 28 Feb 2013 10:31:35 +0000 (11:31 +0100)]
Make the XmParser config test runtime more consistent
Currently, the test uses a frequency of 5 string/5 double/1 list for
generating Arbitrary instances of ListConfig. However, the list case
has simply a "choose (1, 20)" `vectorOf` arbitrary, which means it
could recurse forever.
Manually running only this test gives runtime as such:
- ~100-200ms: very often
- ~1-2s: often
- ~5s: rare
- ~20s: very rare (but I hit this when running < 30 times the test,
so…)
On average, this makes this test one of the slowest ones, which is
annoying.
By changing to a sized generator, we can control the depth of the
recursion, ensuring that we have a consistent runtime: out of 100
runs, one is 229ms, one is 164ms, the other are 80-120ms.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>