Synnefo » snf-ganeti

Add two utility functions

These both are work with/on the Result type, so we add them to
BasicTypes. The functions will be used as more generic versions of
some more specialised functions that are right now spread across the
modules.

Signed-off-by: Iustin Pop <iustin@google.com>...

Typo fix: s/aditional/additional/

Trivial fix for a typo in message output of LUInstanceSetParams

Signed-off-by: Alexander Schreiber <als@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Export the AF_INET/AF_INET6 constants

These are needed to ensure that htools uses the same numeric values as
Python.

By the way, I'm not sure what's the best option: importing socket into
constants (this patch), or adding custom exports to
autotools/convert-constants....

htools: fix long version of --port for daemons

The dashes do not need to be in, if they are then the resulting option
is:

-p PORT  ----port=PORT     Network port (default: 1814)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Update gnt-node evacuate man page/help text

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

hspace: Make use of the spindle_use

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add tests for custom text files

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Enable pipefail mode for shell tests

These allow more accurate checks.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Enhance the RAPI backend to accept file:// URLs

This will allow offline testing of this backend (except for the actual
curl part), also in the case where we didn't compile against curl.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

htools: Reflect the rename of spindle_usage to spindle_use

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add a helper function for mirrorType computations

We most always use `templateMirrorType . diskTemplate`, so let's add a
helper function for this.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Enable all disk templates in unit-tests

After the many patches implementing the shared storage functionality
in iallocator, balances, etc., and after the patches preparing the
unit-tests to do the right thing for shared storage, we can finally
flip the bits that make instances have "random" disk templates as...

Replace explicit case with 'either'

Instead of manually case-ing on the Either contents, let's just use
either with const functions (not really readable, but…).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Abstract the secondary evac into a separate function

This is generic enough (no longer depending on the replace type) that
it will be used for other replace modes too. So all we need to do is
to abstract it to a separate function (that can be called from...

Extend the possible move types with FailoverToAny

This brings together all previous pieces and allows balancing to work
(with the known caveats related to disk space calculation) for shared
storage disk templates.

Signed-off-by: Iustin Pop <iustin@google.com>...

ipolicy: Keep track of spindle usage

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

htools: Convert all the other backends as this is trivial

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

htools: Adding spindle usage to Instance

For now we just pass in "1" as spindle usage and adapt the backend
later.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

constants: Add new spindle_usage parameter to BE

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Enable lvmstrap to run under Linux 3.x

Extend the kernel version check to also accept Linux 3.x as valid.

Signed-off-by: Alexander Schreiber <als@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add more rebalance offline tests

These test that we can actually rebalance (and that it results in
replace-disks and failovers/migrations), and that we don't
double-rebalance, etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Build a coverage-enabled version of the tools

This allows the offline tests to also generate coverage data, and the
hs-coverage target is changed to show both unit-test and offline-tests
coverage.

The downside is that now we build yet-another-binary, which makes the...

Add a simple binary tester for htools

This only tests the command line basic options. In the future, it can
be expanded to check other things.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

htools: add confd daemon binary

This is currently not installed anywhere, just built.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

htools: add basic daemon-related functionality

This is not complete for now, just the basic functionality has been
implemented:

- daemonize
- check we're running under the correct user
- call setup logging

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

htools: add runtime functionality

This duplicates the lib/runtime.py functionality, allowing us to check
for runtime users and groups consistency.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

htools: add definitions for confd types

While we have some of these as plain types in Constants.hs, we add
proper ADT definitions for them in a new file. Furthermore, we add the
ConfdRequest and ConfdReply types here (in Python they are in
objects.py).

Signed-off-by: Iustin Pop <iustin@google.com>...

htools: add partial implementation of lib/objects.py

This is partial since not all object types can be easily converted for
now (will need some changes on the Python side for this).

Most importantly, the *Params types do not have a good solution now:
the Python code, due to its dynamic typing, hides the fact that we...

Fix lint error in commit 035b33e2

Commit 035b33e2 forgot one blank space, and current pylint in Debian
Sid doesn't run on our code… so I didn't realise this until after
commit, sorry.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix IAlloc.processRelocate to support shared storage

Currently, this function expects (but doesn't really check! only via
secondary node being defined) a DRBD8 template type.

This patch (not very readable, sorry) changes it so that we check
whether we're internally mirrored, externally mirrored, or not...

Small improvement for nicer unit-testing

In case you just run "make htools/test" to recompile the unittests,
the old .tix file should be removed, otherwise you get the message
about inconsistent tick boxes. Let's make the compilation process
automatically remove this, for easier testing....

Add a simple type alias for nicer type signatures

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix test Node_addSec_idempotent

Another test that only works for DRBD8 instances.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Split the Node_addOffline test into pri/sec

Currently, this (single) test checks whether we can add an instance as
primary and secondary to a given node. This won't work for instance
templates other than DRBD8, so we split it into:

- an add to primary test, which should work for all disk templates...

iPolicy: enable by default all storage templates

… otherwise we can't run tests or simulations using non-default
templates. Tests still pass.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix printSolutionLine to handle non-DRBD steps

Currently Cluster.printSolutionLine function always wants to print the
old and new secondary, and as such it fails (ugly) for moves of
non-DRBD instances. Change it so that it looks explicitly for
noSecondary nodes, and format the output different in that case....

Small simplification in tryBalance

Instead of computing the bad/good node lists separately (in effect
iterating twice over the all_nodes list and also creating that list
twice from the container), let's use 'partition' and do a small
improvement.

Signed-off-by: Iustin Pop <iustin@google.com>...

Do not rely on no-secondary for movable tests

Currently, the code in Loader.hs marks as un-movable any instances
which don't have a secondary. This is not valid for any shared storage
templates, so let's remove that check completely. The initial state of...

Fix tempfile reset code & test on newer Python

Python 2.7.3 (rc status) and 3.2.3/3.3 (rc, respectively alpha status)
have fixed http://bugs.python.org/issue12856 which we worked around
ourselves.

This means two things:

- we don't need to manually reset the module...

Split check-man-warnings into two

This moves the dashes check to a new check-man-dashes script.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Makefile: improve the fix for .DELETE_ON_ERROR

I've investigated more and it seems that .DELETE_ON_ERROR not working
is a bug/regression in some versions of make; testing with make 3.82
from Debian experimental makes this special target work as
expected. Also, make 3.81 works on trivial Makefiles, but not on our...

htools/CLI: Adding new options to use ialloc as source

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

htools: Refactor some of IAlloc.readRequest to Hail

This makes it possible to reuse that function when we want to use IAlloc
as a data source backend

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

hinfo: Gather and print group statistics

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

More -- fixes

Because .DELETE_ON_ERROR doesn't work consistently (works when the
target we're talking about is a final target, but not for
intermediates), I missed a lot of man checks.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fixup a Node unittest

This fixes the prop_Node_rMem test, which cares about memory only;
otherwise we could fail due to other problems (e.g. not enough disk).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Add two more unittests for Node add/remove

These just test that add/remove are idempotent. This is not perfect,
as we use unsorted lists for some values (instead of sets), so when
using non-empty nodes this would break (but for empty nodes, a
1-element list is sorted, so it's fine)....

Track instance-used spindles in node set/add/remove

This simply tracks the instance-used spindles (using hard-coded '1'
per instance), and additionally prevents additions in soft-mode when
we go over the limit.

Note: there's an assymetry between addSec and removeSec (basically...

Simplify logic in Node operations

This patch adds two helper functions and uses them for a somewhat
simplified logic in the add/remove pri/sec node functions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

htools: support spindles in text load/save

If missing, we'll default to 1 spindle.

This also updates the text file format with the ipolicy, which was
forgotten.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>