Iustin Pop [Mon, 21 Nov 2011 15:33:00 +0000 (16:33 +0100)]
htools: add runtime functionality
This duplicates the lib/runtime.py functionality, allowing us to check
for runtime users and groups consistency.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 20 Nov 2011 17:16:18 +0000 (18:16 +0100)]
htools: add a small module for crypto functions
This adds a dependency on the following new libraries:
- Data.HMAC module from the 'crypto' library
- Data.Text from the 'text' library
- Data.ByteString from the 'bytestring' library
In Debian, the bytestring library comes with the ghc compiler, whereas
the other two are available separately in wheezy/unstable.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 18 Nov 2011 10:36:31 +0000 (11:36 +0100)]
htools: add definitions for confd types
While we have some of these as plain types in Constants.hs, we add
proper ADT definitions for them in a new file. Furthermore, we add the
ConfdRequest and ConfdReply types here (in Python they are in
objects.py).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 18 Nov 2011 10:13:37 +0000 (11:13 +0100)]
htools: add a very basic lib/config.py equivalent
This only is able to load the configuration data for now.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 18 Nov 2011 09:59:12 +0000 (10:59 +0100)]
htools: add partial implementation of lib/objects.py
This is partial since not all object types can be easily converted for
now (will need some changes on the Python side for this).
Most importantly, the *Params types do not have a good solution now:
the Python code, due to its dynamic typing, hides the fact that we
actually have two different types at play: a full type which needs to
have all keys, and the 'partial' type which has slightly different
behaviour. I've implemented these in Haskell as two different types,
Full* and Partial*, which are derived automatically from a single
Parameter type, together with the associated Fill* functions.
Furthermore, HVParams is even more special, as its contents is not
fixed but varies per hypervisor type, plus it has the HV_GLOBALS part
which should not be customisable at instance type (yay for
exceptions). As such, this should be written in Haskell as a
multi-constructor type, but it's the only one so far and thus we don't
have support for it yet.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
René Nussbaumer [Fri, 2 Mar 2012 12:15:13 +0000 (13:15 +0100)]
htools: Make it possible to load an IAlloc request
This is only useful in the stand-alone command line tools:
- hbal
- hspace
- hinfo
This change also documents the flag in all man-pages.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 7 Mar 2012 21:43:50 +0000 (23:43 +0200)]
Fix lint error in commit
035b33e2
Commit
035b33e2 forgot one blank space, and current pylint in Debian
Sid doesn't run on our code… so I didn't realise this until after
commit, sorry.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 20:55:49 +0000 (22:55 +0200)]
Fix relocation test ClusterAllocRelocate
As for the evacuate tests, we require a mirrored disk template, but
otherwise the test should work for both mirror types.
Additionally, we perform a simplification that was left as TODO.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 6 Mar 2012 23:24:28 +0000 (01:24 +0200)]
Fix IAlloc.processRelocate to support shared storage
Currently, this function expects (but doesn't really check! only via
secondary node being defined) a DRBD8 template type.
This patch (not very readable, sorry) changes it so that we check
whether we're internally mirrored, externally mirrored, or not
mirrored at all. It then proceeds to do the right thing™ per the
mirror type, so that it can work well with both.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 6 Mar 2012 22:20:32 +0000 (00:20 +0200)]
Fix up Cluster nodeEvac unittests for shared storage
Currently, these presume that the instance being generated always is
DRBD/relocatable (but won't be in the future), and also presume that
all relocation modes can be used (hint: not ☺).
So we add a couple of helper functions to restrict the used disk
templates/relocation modes, and these two tests are ready for shared
storage.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 6 Mar 2012 22:11:19 +0000 (00:11 +0200)]
Small improvement for nicer unit-testing
In case you just run "make htools/test" to recompile the unittests,
the old .tix file should be removed, otherwise you get the message
about inconsistent tick boxes. Let's make the compilation process
automatically remove this, for easier testing.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 6 Mar 2012 20:32:40 +0000 (22:32 +0200)]
Rework evacDrbdSecondaryInner to take the imove
Instead of hard-coding the instance move operation, let's accept is as
a parameter from the caller. This is the single thing that makes this
DRBD-specific, so removing it will help for future
patches. Furthermore, this abstract the imove op type into a single
place instead of two (the inner function and the nodeEvacInstance).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 6 Mar 2012 20:13:24 +0000 (22:13 +0200)]
Add a simple type alias for nicer type signatures
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 20:46:56 +0000 (22:46 +0200)]
Fix a few tests that hard-coded the number of nodes
Instead of randomly generating the number of required nodes or
hard-coding it, compute it from the disk template of the
instance. Otherwise tests will fail when the disk template is not
DRBD8 and we ask for allocation on 2 nodes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 20:43:42 +0000 (22:43 +0200)]
Fix test Node_addSec_idempotent
Another test that only works for DRBD8 instances.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 20:41:38 +0000 (22:41 +0200)]
Fix the Node_rMem test to specify DRBD8 templates
… since memory reservation only is used for this template.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 20:39:37 +0000 (22:39 +0200)]
Split the Node_addOffline test into pri/sec
Currently, this (single) test checks whether we can add an instance as
primary and secondary to a given node. This won't work for instance
templates other than DRBD8, so we split it into:
- an add to primary test, which should work for all disk templates
- an add to secondary test, for which we set the disk template to DRBD
explicitly
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 13:54:19 +0000 (14:54 +0100)]
Use the mirror type in Cluster.possibleMoves
Currently the function possibleMoves always presumes DtDrbd, which is
wrong for shared storage. To make it work with all disk templates, we
additionally pass the mirror type to it (we assume that the move
behaviour only depends on the mirror type, and not the individual
disk template; currently this is true, and we should aim to keep it
that way).
For none and external mirrors, we don't give any solutions; for
internal mirrors, we keep the previous algorithm.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 13:50:25 +0000 (14:50 +0100)]
iPolicy: enable by default all storage templates
… otherwise we can't run tests or simulations using non-default
templates. Tests still pass.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 13:11:15 +0000 (14:11 +0100)]
Fix printSolutionLine to handle big scores
Currently we don't set a boundary on the length of the score field,
which breaks all our nicely-aligned output efforts.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 13:06:57 +0000 (14:06 +0100)]
Fix printSolutionLine to handle non-DRBD steps
Currently Cluster.printSolutionLine function always wants to print the
old and new secondary, and as such it fails (ugly) for moves of
non-DRBD instances. Change it so that it looks explicitly for
noSecondary nodes, and format the output different in that case.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 12:55:54 +0000 (13:55 +0100)]
Add a helper type and function for the mirror type
Currently we track the mirroring type as simply yes/no, with "yes"
meaning DRBD. This is not enough for handling shared storage, so we
need to add a type that gives the exact mirroring type (none,
internal, external), and a function that ties the disk template type
to the mirroring type.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 12:09:24 +0000 (13:09 +0100)]
Small simplification in tryBalance
Instead of computing the bad/good node lists separately (in effect
iterating twice over the all_nodes list and also creating that list
twice from the container), let's use 'partition' and do a small
improvement.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 11:58:47 +0000 (12:58 +0100)]
Mark Rbd as a movable disk template
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 19 Feb 2012 21:15:17 +0000 (22:15 +0100)]
Do not rely on no-secondary for movable tests
Currently, the code in Loader.hs marks as un-movable any instances
which don't have a secondary. This is not valid for any shared storage
templates, so let's remove that check completely. The initial state of
the instance will reflect correctly the movable status, based on the
disk template of the instance, so we don't need to re-do this check.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 19 Feb 2012 20:59:48 +0000 (21:59 +0100)]
Handle nodes without local storage
Our current cluster model is based on per-node storage, and as such we
don't handle shared disk templates. Until we can improve the model,
let's handle nodes without local storage better: in order to still
compute a valid cluster score, we need to handle nodes with tDsk == 0.
For this, we abstract the in-line computation of pDsk (which is used
in the cluster score) to a separate function which, for tDsk == 0,
returns instead 1 (i.e. disk fully used).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 6 Mar 2012 19:38:18 +0000 (21:38 +0200)]
Fix tempfile reset code & test on newer Python
Python 2.7.3 (rc status) and 3.2.3/3.3 (rc, respectively alpha status)
have fixed http://bugs.python.org/issue12856 which we worked around
ourselves.
This means two things:
- we don't need to manually reset the module
- we can't test for the no-reset case
Unfortunately current Debian Sid has the 2.7.3 RC but still labeled as
2.7.2+ ☹, so we have to manually do an extra check (in the unit test
only); I expect Debian will update to official 2.7.3 as soon as it's
released, and then we can remove this override.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 5 Mar 2012 21:48:16 +0000 (23:48 +0200)]
Merge remote-tracking branch 'origin/devel-2.5'
Conflicts:
Makefile.am (trivial, test data files added on both branches)
lib/opcodes.py (trivial, master renamed filter→qfilter, 2.5
fixed the type of the parameter)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 5 Mar 2012 10:49:41 +0000 (11:49 +0100)]
Split check-man-warnings into two
This moves the dashes check to a new check-man-dashes script.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 5 Mar 2012 10:44:51 +0000 (11:44 +0100)]
Rename check-man to check-man-warnings
Currently the check-man test is dependent on the MAN_HAS_WARNINGS
test, which made sense as long as the script only did the warnings
check.
However, we should run the [em] tests always, and in general we could
run other tests too, even if man doesn't support --warnings; so let's
split the check man. As a first prerequisite, we rename check-man to
check-man-warnings (and the respective Makefile variables).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 5 Mar 2012 10:38:35 +0000 (11:38 +0100)]
Makefile: improve the fix for .DELETE_ON_ERROR
I've investigated more and it seems that .DELETE_ON_ERROR not working
is a bug/regression in some versions of make; testing with make 3.82
from Debian experimental makes this special target work as
expected. Also, make 3.81 works on trivial Makefiles, but not on our
complex one.
In the meantime, let's improve the fix for the building of man pages:
make the removal implicit, instead of explicit, via the use of 'trap'
on EXIT. I just hope this behaves consistently across all bash
versions :)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Fri, 2 Mar 2012 10:52:30 +0000 (11:52 +0100)]
htools/ExtLoader: Bring the pieces together
This make it possible to load from an Ialloc file
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 2 Mar 2012 10:51:32 +0000 (11:51 +0100)]
htools/CLI: Adding new options to use ialloc as source
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 2 Mar 2012 10:50:55 +0000 (11:50 +0100)]
htools/IAlloc: Adding loadData method for use as backend
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 2 Mar 2012 10:49:34 +0000 (11:49 +0100)]
htools: Refactor some of IAlloc.readRequest to Hail
This makes it possible to reuse that function when we want to use IAlloc
as a data source backend
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Thu, 1 Mar 2012 09:54:54 +0000 (10:54 +0100)]
hinfo: Adjust the verbosity
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 29 Feb 2012 15:29:44 +0000 (16:29 +0100)]
hinfo: Gather and print group statistics
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 29 Feb 2012 14:55:58 +0000 (15:55 +0100)]
Explicitly remove the .x.in man pages on failure
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 29 Feb 2012 14:55:18 +0000 (15:55 +0100)]
More -- fixes
Because .DELETE_ON_ERROR doesn't work consistently (works when the
target we're talking about is a final target, but not for
intermediates), I missed a lot of man checks.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Wed, 29 Feb 2012 13:33:49 +0000 (14:33 +0100)]
htools: Adding printTable helper
This helper abstract away all the unlines and unwords call you've to do
after you got a formatTable back.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 21 Feb 2012 09:42:32 +0000 (10:42 +0100)]
Fixup a Node unittest
This fixes the prop_Node_rMem test, which cares about memory only;
otherwise we could fail due to other problems (e.g. not enough disk).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 20 Feb 2012 12:45:24 +0000 (13:45 +0100)]
Allow overriding the target test count
This is not perfect, as the override applies identical target test
values to both the 'fast' and 'slow' tests (making the slow tests take
a long time), but it allows a quick override for manual runs.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 20 Feb 2012 12:30:04 +0000 (13:30 +0100)]
Add two more unittests for Node add/remove
These just test that add/remove are idempotent. This is not perfect,
as we use unsorted lists for some values (instead of sets), so when
using non-empty nodes this would break (but for empty nodes, a
1-element list is sorted, so it's fine).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 20 Feb 2012 12:14:27 +0000 (13:14 +0100)]
Use the spindles metric in cluster scores
This makes balancing use the new metric.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 20 Feb 2012 11:18:15 +0000 (12:18 +0100)]
Track instance-used spindles in node set/add/remove
This simply tracks the instance-used spindles (using hard-coded '1'
per instance), and additionally prevents additions in soft-mode when
we go over the limit.
Note: there's an assymetry between addSec and removeSec (basically
secondaries should only exist with DTDrbd8, so the check for uses_disk
is superflous), will be cleaned up later.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 20 Feb 2012 10:43:50 +0000 (11:43 +0100)]
Shorten some function names
Since we use the functions always module-qualified, let's drop the
`instance` prefix and use a shorter one.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sun, 19 Feb 2012 20:43:40 +0000 (21:43 +0100)]
Simplify logic in Node operations
This patch adds two helper functions and uses them for a somewhat
simplified logic in the add/remove pri/sec node functions.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 16 Feb 2012 12:43:01 +0000 (13:43 +0100)]
htools: support spindles in simu backend
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 16 Feb 2012 12:34:02 +0000 (13:34 +0100)]
htools: support spindles in text load/save
If missing, we'll default to 1 spindle.
This also updates the text file format with the ipolicy, which was
forgotten.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 16 Feb 2012 11:57:51 +0000 (12:57 +0100)]
htools: add support for listing spindles for nodes
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 16 Feb 2012 11:52:54 +0000 (12:52 +0100)]
htools: add spindle-related attribute to nodes
This adds the spindleCount, hiSpindles and instSpindles attributes.
The spindleCount is equivalent to spindle_count on the ganeti side (a
node parameter). hiSpindles is the maximum instance-used spindles, and
instSpindles will be used to track spindles as used by instances (in
later patches).
The patch also reads spindleCount from the live backends. The text and
simu backends currently set it to 1, hard-coded.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Fri, 27 Jan 2012 16:21:24 +0000 (17:21 +0100)]
Add spindle ratio to htool's IPolicy
Just the usual data/type declarations, read/save in the text backend,
etc. Also does a bit of unit-test cleanup (the way we build
ipolicies).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Wed, 29 Feb 2012 09:59:22 +0000 (10:59 +0100)]
hinfo: Adding man page
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 28 Feb 2012 14:39:24 +0000 (15:39 +0100)]
hinfo: Reorganize the code to separate functions
Also rename some functions as they don't reflect their actions anymore
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 28 Feb 2012 12:50:14 +0000 (13:50 +0100)]
Two tiny fixed related to runtime functionality
Adds an assert and fixes a typo.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Mon, 27 Feb 2012 09:33:08 +0000 (10:33 +0100)]
hinfo: Adding basic skeleton based on hbal
It prints the information which hbal prints in a nice formatted manner and of
course without doing any actions. An example output looks like this:
$ hinfo -m xen.example.com
Loaded 4 nodes, 2 instances
Cluster has 1 node group(s)
Cluster coefficients:
Field Value Weight
free_mem_cv 0.
00195342 x1.00
free_disk_cv 0.
00000000 x1.00
n1_cnt 0.
00000000 x1.00
reserved_mem_cv 0.
00195342 x1.00
offline_all_cnt 0.
00000000 x4.00
offline_pri_cnt 0.
00000000 x16.00
vcpu_ratio_cv 0.
12500000 x1.00
cpu_load_cv 0.
50000000 x1.00
mem_load_cv 0.
50000000 x1.00
disk_load_cv 0.
00000000 x1.00
net_load_cv 0.
50000000 x1.00
pri_tags_score 0.
00000000 x2.00
Cluster score: 1.
62890685
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 27 Feb 2012 14:51:15 +0000 (15:51 +0100)]
Add automated checking for non-escaped --
This checks to see if any man output has [em] in it, as that will not
be correctly show in ASCII. Unfortunately the check will depend on
whether the other man check is done, but I didn't want to introduce
yet another short script for this (can be done, no issue).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 27 Feb 2012 14:48:05 +0000 (15:48 +0100)]
Fix man pages to not use unescaped --
I've seen that man pages, as generated by the version of pandoc we
use, show single dashes in option names instead of double ones (-
versus --). After bringing it up with upstream
(http://groups.google.com/group/pandoc-discuss/browse_thread/thread/
9c4589a4001d42f9/
95ee8dae8932dc93),
it seems that this is a known behaviour of pandoc that has been
improved in newer versions.
Until then, let's use correctly double dashes; from the two options in
the above thread, I chose to use \-- as that doesn't change the actual
output; whereas ``--nodes`` make this a code block, which will look
differently from a short option and could change how the output looks
(e.g. when in a bold span).
Additionally, I've removed two cases where unescape em was explicitly
intended, as that makes automated checking harder and we can use other
formatting.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 24 Feb 2012 11:22:02 +0000 (12:22 +0100)]
Add job result descriptions to RAPI documentation
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 27 Feb 2012 13:40:44 +0000 (14:40 +0100)]
Remove a superfluous warning in LUNodeRemove
Since we run the post-hooks explicitly in the Exec() function (via
_RunPostHook) after we removed the target node from the config, we
will get a:
WARNING Node 'node2', which is about to be removed, was not found in
the list of all nodes
in the logs every time we remove a node. The patch just removes the
warning, as actually invalid configurations (for the pre hook) will be
checked correctly elsewhere.
Additionally, the docstrings for BuildHooksEnv and BuildHooksNodes are
corrected/switched.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 23 Feb 2012 15:43:35 +0000 (16:43 +0100)]
opcodes: Remove dependency on query module
There's no need to verify the field definitions on every query. They are
static for all intents and purposes anyway and verified at module load
time.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 23 Feb 2012 15:48:18 +0000 (16:48 +0100)]
Merge branch 'stable-2.5' into devel-2.5
* stable-2.5:
Fix Makefile.am compatibility with automake 1.11.2
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 21 Feb 2012 19:14:46 +0000 (20:14 +0100)]
http.server: Factorize request handling even more
This splits even more parts of the request handling code into a separate
class. Doing so allows us to reuse this part of the code for tests (e.g.
mocks). Unlike before now the error handling can also be reused.
The patch became a bit more convoluted than intended, but the end result
is easier to read than the original code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Tue, 21 Feb 2012 17:23:37 +0000 (18:23 +0100)]
http.server: Move error message formatting to handler class
Like before this patch moves more functionality from the actual server
class into a separate handler class. At the same time the function is
changed to return both content-type and body instead of relying on a
class attribute.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Tue, 21 Feb 2012 16:40:20 +0000 (17:40 +0100)]
noded: Shorter docstring for mlockall(2) executor
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Tue, 21 Feb 2012 16:39:46 +0000 (17:39 +0100)]
Split handling HTTP requests into separate class
Until now HTTP requests were handled in the same class as incoming
connections (http.server.HttpServer). With this change the request
handling is delegated to a separate class which can be re-used in tests
without creating a socket, etc.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Tue, 21 Feb 2012 16:13:50 +0000 (17:13 +0100)]
http.server: Factorize request handling
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Wed, 22 Feb 2012 11:58:05 +0000 (12:58 +0100)]
Remove deprecated “QueryLocks” LUXI request
This has been deprecated since Ganeti 2.4 and hasn't been used by
Ganeti's code since. I'm not aware of any external users.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 21 Feb 2012 15:19:48 +0000 (16:19 +0100)]
Add result checks for OpTags*
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Tue, 21 Feb 2012 09:05:29 +0000 (10:05 +0100)]
opcodes: Annotate the OP_RESULT of query operations
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 21 Feb 2012 13:37:58 +0000 (14:37 +0100)]
Replace single- with double-quotes
In at least two cases "%s" is replaced with str(), too.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 21 Feb 2012 08:36:04 +0000 (09:36 +0100)]
Fix asyncnotifier unit tests
On my workstation, if I run the following two tests in this exact
order:
- TestSingleFileEventHandler.testError
- TestSingleFileEventHandler.testReplace
the second test will fail, because there is no "unregister" of
previous tests (and the poll() done by asyncore will fire on the
previous files which are now removed, etc. etc.).
While investigating this, I realised that we don't do any cleanup of
the various handlers/notifiers we setup for this test, so if we simply
add a proper cleanup, the tests will work nicely (in any order :).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Tue, 21 Feb 2012 07:42:09 +0000 (08:42 +0100)]
Small fixup to rapi docstrings
The fast that most classes don't override the @cvars means that direct
references to FillOpcode can fail (they do on my workstation, but not
on buildbot?). Anyway, for safety, it's best to qualify the name.
Additionally a small typo is fixed.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 20 Feb 2012 14:40:46 +0000 (15:40 +0100)]
Small update to LogicalUnit.ExpandNames docstring
… to make even more obvious what's the difference between a declared
lock level with an empty list of locks and no lock level.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 16 Feb 2012 09:56:40 +0000 (10:56 +0100)]
Export ndparams in RAPI node query
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 3 Feb 2012 15:55:01 +0000 (16:55 +0100)]
Add ndp/* fields to group/node query
Also change the meaning of group ndparams to mean the actual (not
custom) nd params, and add custom_ndparams for the current meaning of
ndparams.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 27 Jan 2012 13:45:10 +0000 (14:45 +0100)]
Export ndparams in iallocator
Strangely, these were not exported at all before.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 27 Jan 2012 13:38:31 +0000 (14:38 +0100)]
Add ipolicy parameter spindle_ratio
This will represent the instance (count) per spindle_count of their
node(s). Debatable whether we need to add a per-instance
spindle_weight.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 27 Jan 2012 13:29:31 +0000 (14:29 +0100)]
Add new spindle_count node parameter
Currently this is not handled by Ganeti, just recorded.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 27 Jan 2012 13:26:35 +0000 (14:26 +0100)]
Fix upgrading of ndparams
Currently, we only upgrade the ndparams if they are missing
completely, which creates problems if we add any new parameters on an
already-upgraded cluster.
Fix this by adding an UpgradeNDParams function.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 27 Jan 2012 13:25:03 +0000 (14:25 +0100)]
Small fixes to objects.UpgradeDiskParams
Fix a typo, and cleanup the code a bit.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 20 Feb 2012 16:21:51 +0000 (17:21 +0100)]
opcodes: Add result checks for OpBackup*
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 20 Feb 2012 12:29:38 +0000 (13:29 +0100)]
Improve opcode result check test to have a whitelist
Only whitelisted opcodes may not have a result check.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 17 Feb 2012 16:49:35 +0000 (17:49 +0100)]
config: Add check for disk's “iv_name”
This check verifies “iv_name” of all instance disks. If one is wrong
(which shouldn't happen in the first place), cluster verification will
warn:
“ERROR: cluster: Instance 'inst.example.com' has wrongly named disks:
name of disk 1 should be 'disk/1', but is 'disk/4'”
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Mon, 20 Feb 2012 10:35:11 +0000 (11:35 +0100)]
opcodes: Fix OP_RESULT for OobCommand
The result is a list of a list with elements with size of 2.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Sun, 19 Feb 2012 19:58:42 +0000 (20:58 +0100)]
Fix Makefile.am compatibility with automake 1.11.2
Automake 1.11.2 made the following change:
* Long-standing bugs:
- Automake now warns about more primary/directory invalid combinations,
such as "doc_LIBRARIES" or "pkglib_PROGRAMS".
Unfortunately, this breaks our Makefile.am (issue 216) exactly because
we were relying on pkglib_SCRIPTS.
This patch works around this by adding a new myexeclibdir variable
(exec so that it is intalled at `install-exec` time, the same as the
pkglibdir), and switches to that.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Fri, 17 Feb 2012 10:49:00 +0000 (11:49 +0100)]
cli: Handle negative numbers when parsing key-value assignments
To remove the last disk, or to add a NIC to the end, one can use the
index -1. This wouldn't work as intended as “-” is a special prefix.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 17 Feb 2012 12:42:04 +0000 (13:42 +0100)]
utils.text: Add function to truncate string
The function adds an ellipse if the string was actually truncated. Also
start using it in mcpu for result checks (where the message is also
slightly changed to use a colon).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 17 Feb 2012 12:46:55 +0000 (13:46 +0100)]
Fix breakage introduced by
fa6dd6bb56
Forgot “enumerate”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 15 Feb 2012 16:52:47 +0000 (17:52 +0100)]
LUInstanceSetParams: Update disk's “iv_name”
When modifications are made, disks may not have the same index anymore.
Updating all disks fixes this.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Wed, 15 Feb 2012 17:50:08 +0000 (18:50 +0100)]
ApplyContainerMods: Fix issues with indices
When adding an item the index given to the callback function would be
incorrect under certain conditions. This patch also adds assertions and
more tests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Wed, 15 Feb 2012 16:36:36 +0000 (17:36 +0100)]
cmdlib: Remove some users of “iv_name”
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Wed, 15 Feb 2012 16:09:20 +0000 (17:09 +0100)]
ConfigWriter: Stop using “iv_name”
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Wed, 15 Feb 2012 16:06:29 +0000 (17:06 +0100)]
ConfigWriter.RenameInstance: Stop using iv_name, safer operation
Stop using the disk index encoded in “iv_name” when renaming an instance.
This patch also changes the code to operate on a copy of the instance
until the major changes have been applied. In the case of a failure we
won't loose the instance object anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Tue, 14 Feb 2012 12:41:57 +0000 (13:41 +0100)]
gnt-instance modify: Support new-style NIC/disk modifications
This patch adds support for adding/removing NICs/disks at arbitrary
indices on the command line. To add a disk at a specified index, use
“--disk 3:size=16G”. To remove the second disk, use “--disk 2:remove”.
Unittests are included.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 10 Feb 2012 14:51:53 +0000 (15:51 +0100)]
opcodes: Adding missing OP_RESULTs
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Mon, 13 Feb 2012 09:14:49 +0000 (10:14 +0100)]
mcpu: Make the op result exception more verbose
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 15 Feb 2012 16:02:33 +0000 (17:02 +0100)]
Reconcile Makefile.am and test data files
Sorry, forgot this in previous commit.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 15 Feb 2012 12:14:04 +0000 (13:14 +0100)]
Workaround changed LVM behaviour
The vgreduce command has changed behaviour from when we initially
wrote the code (2.02.02 versus 2.02.66, 4 years delta):
- if there are LVs which will be impacted, it requires --force
- otherwise refuses to proceed, but it still returns exit code 0
We handle this by looking to see if it returns "Wrote out consistent
volume group" (behaviour unchanged), or if it complains about
"--force"; in the case it didn't complete, we retry the operation.
We improve a bit the checking of "vgs", as it uses to fail silently
and we didn't detect it.
New tests for this function should test, I believe, all the expected
variations; at the least we now have data files with the expected
output.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 15 Feb 2012 13:23:49 +0000 (14:23 +0100)]
cmdlib: De-duplicate code in _GenerateDiskTemplate
There has been a lot of duplicated code in _GenerateDiskTemplate,
and some cases of very similar, but not quite same duplicates. This
patch merges them.
Generating a disk's “logical_id” attribute is done via a
lambda/function. Maybe the ID's could be pre-computed and stored in a
list.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>