Iustin Pop [Fri, 8 Jul 2011 09:45:10 +0000 (11:45 +0200)]
htools: switch hscan to the generic binary
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 8 Jul 2011 09:41:01 +0000 (11:41 +0200)]
htools: switch hbal to the generic binary
In addition, the patch adds a separate Makefile variable for holding
the binary roles to make it more clear what we symlink.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 8 Jul 2011 09:33:52 +0000 (11:33 +0200)]
htools: switch hail to the generic binary
This converts the first binary to the generic 'htools' binary.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 21:43:07 +0000 (23:43 +0200)]
htools: add a generic binary
This is the start of a series of patches that will unify all the
binaries currently in use in a single one, which can perform different
roles based on the name it is installed as.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 21:00:45 +0000 (23:00 +0200)]
htools: add a compatibility module
When compiling with the parallel-3.x library, we get a deprecation
warning, which makes understanding any other error messages
harder. This patch adds a compatibility module that will hold such
code for transitioning libraries.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 20:27:34 +0000 (22:27 +0200)]
htools: remove no-longer-needed tryMG* functionality
… which was deprecated by the previous patch.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 20:23:23 +0000 (22:23 +0200)]
htools: remove ialloc/relocate and multi-evacuate
Since the new node-evacuate mode does both their work and also
supports better multi-group clusters (including handling split
instances).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 20:06:26 +0000 (22:06 +0200)]
htools: fix potential bug in ialloc/change-group
Currently, the ChangeAll mode of nodeEvac computes the primary group
of the instance and then uses the resulting group index for computing
the group score. However, during the change-group operation (which
also uses ChangeAll), the group of the instance at the beginning and
end of the operation differs, so we can't do that and we must instead
pass the final group to the function.
This patch does that, by passing the group from the caller of
nodeEvac.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 18:00:40 +0000 (20:00 +0200)]
htools: run IAllocator input through checkData
As the IAllocator backend is using a different data path than the
others, it doesn't get the full functionality that loadExternalData
does. This results in the current situation where checkData is not run
on the input cluster state, which means the node memory properties are
not correctly set, leading to no Xmem being set correctly.
This patch fixes this bug by adding a manual call to checkData.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 17:59:45 +0000 (19:59 +0200)]
htools: abstract a function for displaying warnings
This will make it possible to reuse this in IAllocator too.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 17:13:04 +0000 (19:13 +0200)]
htools: use maybePrintNodes in hail.hs
This eliminates duplication of codes (and was forgotten back when
maybePrintNodes was added).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 17:09:12 +0000 (19:09 +0200)]
htools: add cluster state saving support to hail
This add support for saving the cluster state (both pre- and
post-iallocator run) to a text file such that it can be fed back into
any of the htools commands.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 16:59:04 +0000 (18:59 +0200)]
htools: return the final instance map in ialloc
Similar to the previous patch, this returns the final instance map
from the iallocator run, which will allow saving the cluster state for
further examination/post-processing.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 16:48:30 +0000 (18:48 +0200)]
htools: implement post-alloc cluster status display
This patch changes the IAllocator result formatting workflow to return
the final node list, which can be then used to display the final node
status too—currently only the initial status can be shown, which is
only half useful.
Note that as the FIXME in the code says, doing this right for the
evacuate mode is hard; however, as that mode is deprecated, we can
live it for the moment.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Mon, 11 Jul 2011 19:02:58 +0000 (21:02 +0200)]
Update node group iallocator design to use job dependencies
While working on a function to submit jobsets, I realized that we
actually don't need them anymore. With the new job dependencies, the
iallocator plugin can just generate the right dependencies and gets the
same effect as with jobsets. This saves us from having to implement
jobsets in Ganeti core.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 14 Jul 2011 20:49:34 +0000 (22:49 +0200)]
Fix assertion error on unclean master shutdown
Commit
66bd7445 added an assertion to ensure a finalized job has its
“end_timestamp” attribute set. Unfortunately it didn't cover a case when
the queue is recovering from an unclean master shutdown.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 13 Jul 2011 23:51:45 +0000 (01:51 +0200)]
Make SharedLock._is_owned public
This will be useful for assertions. GanetiLockManager._is_owned is
exported, too.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 7 Jul 2011 16:36:24 +0000 (18:36 +0200)]
htools: return new state from new IAllocator modes
The old modes already return the node list (as part of AllocSolution),
this patch makes the new modes provide this new information.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 13:45:57 +0000 (15:45 +0200)]
htools: update the node-evac/change-group result
… per the new design changes. This also requires changes to
updateEvacSolution, as passing in the original instance is not
helpful; we just pass the instance index now and look up the actual
instance in the modified instance map.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 13:45:05 +0000 (15:45 +0200)]
htools: add function for getting all nodes of an instance
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 12:11:36 +0000 (14:11 +0200)]
htools: add change-group IAllocator mode
This builds a lot upon the node-evacuation mode, and its
implementation is therefore quite easy (once node-evac was added).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 11:58:30 +0000 (13:58 +0200)]
htools: add group restrictions to findBestAllocGroup
While a normal multi-group allocation doesn't need to restrict the
list of target groups beyond the unallocable status of some groups,
when we relocate instances from one group to the other we need to
limit the subset of searched groups.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 7 Jul 2011 11:35:48 +0000 (13:35 +0200)]
htools: generalise and rename availableLocalNodes
Currently this function works in the context of an instance, computing
its local group nodes. We can use generalise it to take a group index
argument, so that it's not bound to an instance, and we rename it
accordingly.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 6 Jul 2011 22:57:14 +0000 (00:57 +0200)]
htools: add ChangeGroup to IAllocator types/loader
This patch updates the types and functions in IAllocator.hs and
Loader.hs to the new design changes (elimination of multi-relocate
mode and replacement with change-group).
It also removes an extra re-export of EvacMode from Loader.hs.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 6 Jul 2011 22:44:21 +0000 (00:44 +0200)]
htools: split tryMGAlloc in two
Currently, this function does both identify the best group for an
instance, and format the results (partially). The former functionality
is useful in other contexts too, so let's split the function in two.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 6 Jul 2011 22:13:27 +0000 (00:13 +0200)]
htools: add node-evacuation of DRBD all nodes
This is much more complicated than either change primary (which is
trivial) and change secondary (which is also simple).
The main strategy is:
- choose a new secondary (that will become the final primary) and
replace-disks to it
- failover
- choose a new secondary again (this will be the final secondary)
However, if the original primary was offline, then we first have to
failover to the secondary of the instance.
Note: currently this evacuation mode, while producing correct
evacuation results, does not result in optimal placement of the
instance; it will need attention and possibly an algorithm change.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 6 Jul 2011 21:29:57 +0000 (23:29 +0200)]
htools: small optimisation in Cluster.collapseFailures
This makes fewer temporary data structures, and thus brings a bit of
improvement in total runtime.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 6 Jul 2011 20:30:42 +0000 (22:30 +0200)]
htools: add optional limit to allocation count
Currently, the tieredAlloc/iterateAlloc functions will not return
until the allocation fails; this means unit-testing their
functionality (e.g. that an instance can be allocated) is slow, since
they will allocate all possible instances.
This patch adds an optional limit that allows allocation to return
early; this makes the cluster unittests twice as fast.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 6 Jul 2011 14:29:14 +0000 (16:29 +0200)]
htools: add node-evacuation of DRBD secondaries
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 6 Jul 2011 14:52:01 +0000 (16:52 +0200)]
htools: rework compCV so that we can use [Node] too
In some cases, we have a [Node.Node] list, instead of a Node.List
structure. Instead of converting [Node.Node] to the latter, let's
rework compCV and add a compCVNodes such that we skip this
(potentially expensive) step.
This already allows a simplification in a unittest, and will simplify
consumers of Node.computeGroups.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Stephen Shirley [Fri, 24 Jun 2011 11:29:48 +0000 (13:29 +0200)]
Adding a wrapper around connecting to kvm console
The wrapper will connect to the console, and check in the background if
the instance is paused, unpausing it as necessary.
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Stephen Shirley [Thu, 23 Jun 2011 09:42:23 +0000 (11:42 +0200)]
Adding a wrapper around "xm console"
The wrapper will connect to the console, and check in the background if
the instance is paused, unpausing it as necessary.
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 27 May 2011 10:50:15 +0000 (12:50 +0200)]
Add opcode attribute for chained jobs
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 18 May 2011 14:38:12 +0000 (16:38 +0200)]
Design for chained jobs
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Fri, 8 Jul 2011 13:32:23 +0000 (15:32 +0200)]
htools: fix a doc bug in the hbal man page
Similar to the previous hspace fix.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 8 Jul 2011 14:01:49 +0000 (16:01 +0200)]
Mark htools/README as obsolete
And point to the main INSTALL file for up-to-date information.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 14 Jul 2011 09:32:22 +0000 (11:32 +0200)]
Fix htool unittest failure due to invalid prereqs
Currently, the test prop_Node_rMem does some reserved memory checks on
nodes, and for the test to work correctly (can add more instances) we
require that the node free memory is greater than zero via a
prerequisite.
However, if the node free memory is exactly 1, the call to
setInstanceSmallerThanNode will set the instance memory to "free
memory `div` 2", and since this is integer division, the instance
memory will get set to 0. Which means that the node reserved memory
will not increase when we add this instance as secondary.
To work around, we simply require that the node memory is high enough
(we use our 'base' unit of memory).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 5 Jul 2011 21:32:19 +0000 (23:32 +0200)]
htools: add node-evacuate of DRBD8 primaries
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 3 Jul 2011 14:17:46 +0000 (16:17 +0200)]
htools: add more IAlloc data types and functions
These move the 'not supported' message deeper into the workflow;
assuming that the workflow as added by this patch is correct, all that
remains is:
- for disk template DRBD8, plug in the currently existing mechanisms
to implement relocation
- for other disk templates, first modify the Node function to handle
node changes correctly for each disk template, and then add support
for them in nodeEvacInstance
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 6 Jul 2011 22:12:28 +0000 (00:12 +0200)]
htools: add two monad-conversion utility functions
For monadic operations, these function will make it easier to convert
other monads (or monad-like structure) to our 'Result' type.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 4 Jul 2011 21:41:52 +0000 (23:41 +0200)]
htools: add a function for computing evacuated nodes
The new IAllocator interface is based on instances, not nodes, so we
need to backtrack and compute on which nodes we can't allocate
instances during the current operation. This patch adds a function for
computing this node set.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 4 Jul 2011 12:31:04 +0000 (14:31 +0200)]
htools: rename an internal constant
'emptySolution' is fine while we have only one Solution type
(AllocSolution), but in the future we'll introduce another one and
thus we should rename it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 3 Jul 2011 21:16:06 +0000 (23:16 +0200)]
htools: abstract and simplify a multigroup function
In the future, this 'split/associated instance idx list' functionality
will be used by other code, so let's move it to its own code.
The new method of computing the split also requires exporting one more
function from IntMap in Container.hs.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 3 Jul 2011 13:53:57 +0000 (15:53 +0200)]
hspace: also display the starting tiered specs
These are useful for checking what instance specifications we start
with (as they are not necessarily the first entry in the TSPEC map).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 3 Jul 2011 13:46:10 +0000 (15:46 +0200)]
hspace: add support for units in the options
This patch converts hspace to allow units in all disk and memory
specifications, and updates the man page accordingly. It also correct
a mistake in the man page for the --max-cpu option.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 3 Jul 2011 13:18:42 +0000 (15:18 +0200)]
htools: add parseUnit function
This is similar, but not identical, to Utils.ParseUnit. The biggest
difference is that we don't round up/down; as we only use integral
types, the result will always be rounded down.
Moreover, since (real-world) disk sizes come in SI units, the function
differentiates between SI and binary prefixes, using lower-case for
binary and upper-case for SI (similar to lvm usage). This distinction
should be ported to the Python code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Wed, 13 Jul 2011 15:03:01 +0000 (15:03 +0000)]
htools: simplify select tests
Rather than running all possible tests in one go, split them out in
three cases: "normal", undefined default value, undefined list values
(both of which require a valid response before them).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Stephen Shirley [Wed, 13 Jul 2011 15:58:50 +0000 (17:58 +0200)]
Set startup_paused to False when restarting
This fixes the lint error:
E1120:1220:InstanceReboot: No value passed for parameter
'startup_paused' in function call
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Thu, 7 Jul 2011 13:38:06 +0000 (14:38 +0100)]
htools: lookup instance names in select/exclude
Currently we just match the passed name with fqdns. With this change we
do a proper search for either the fqdn or the initial name component, as
in Ganeti itself.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 7 Jul 2011 13:37:25 +0000 (14:37 +0100)]
htools: infrastructure for looking up names
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 30 Jun 2011 21:45:26 +0000 (17:45 -0400)]
htools: Add if' and select as functions
if' is just a functional version of the if-then-else construct. Using it
we can manipulate ifs as functions, and also build other functionality.
select allows to choose a result depending on truth value of statements,
with a final default, and can be used to substitute cascaded-nested
if-then-else statements.
Both come with unit tests.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Sat, 2 Jul 2011 22:08:00 +0000 (00:08 +0200)]
htools: remove obsolete option INodes
hspace no longer uses it, hence it can be removed.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sat, 2 Jul 2011 22:11:35 +0000 (00:11 +0200)]
hspace: also report the disk template
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 22:06:10 +0000 (00:06 +0200)]
hspace: ask for the disk_template not number of nodes
This makes more sense, as the disk template is more meaningful than
the number of nodes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 21:44:22 +0000 (23:44 +0200)]
htools: add an utility function for disk templates
In the future we'll use this instead of requiring the number of nodes
from the user.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 21:41:25 +0000 (23:41 +0200)]
htools: define the --disk-template option
Also sorts the export list correctly.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sat, 2 Jul 2011 21:35:46 +0000 (23:35 +0200)]
htools: read/save the disk template in Text backend
This requires that we change the signature of loadInst in order to
properly annotate the error messages, which in turn requires more
unittest changes.
Also, this invalidates yet again saved data files…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 21:22:11 +0000 (23:22 +0200)]
htools: read the disk_template in the Luxi backend
We need to request the disk template in the Luxi query and then use it
when parsing the response.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 21:20:43 +0000 (23:20 +0200)]
htools: read the disk template in Luxi and Rapi
Both these backends already have the data, we just need to use it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 21:18:25 +0000 (23:18 +0200)]
htools: add DiskTemplate to instance definition
Currently always fixed to DRBD8, as that is what we assume already.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 21:09:38 +0000 (23:09 +0200)]
htools: add some unittests for Types.hs
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 20:54:06 +0000 (22:54 +0200)]
htools: add a DiskTemplate type
This will be used later in the Instance type.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 19:56:38 +0000 (21:56 +0200)]
htools: small simplification to parseUtilisation
We eliminate the local variable as it's used only once.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 19:51:59 +0000 (21:51 +0200)]
htools: rename Instance.auto_balance
… to autoBalance, per the recommended style.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 08:24:21 +0000 (10:24 +0200)]
htools: simplify some JSON-related code
Instead of manually encoding the JSValue(s), we can use showJSON as
long as we use data types that can be auto-converted.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 08:18:46 +0000 (10:18 +0200)]
htools: allow different result types
This patch extends the previous patch changes to allow that each
Cluster function returns a different result type, as long as we have
an auxiliary function that processes that into the standard
IAllocResult type.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 07:29:50 +0000 (09:29 +0200)]
htools: start reorganising the IAllocator workflow
Currently, all IAllocator requests return the same result type: a list
of strings (usually nodes, but can also be instance:nodes). This won't
be true in the future, with the new request types that will return
opcode lists.
Therefore we will need to support multiple result types in the
workflow; the first step is to make the final formatResponse step
result-type-indepedent, so we take out the formatRVal invocation out
of this function.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 06:18:22 +0000 (08:18 +0200)]
htools: split more code from hail/main
The logic of request processing is not separated into its own
function, and (beside CLI interaction, e.g. verbosity handling) all
logic is now in IAllocator.hs.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 2 Jul 2011 06:12:18 +0000 (08:12 +0200)]
htools: move code from hail.hs to IAllocator.hs
This will make it easier to unittest the code, and keeps all login in a
single module.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 30 Jun 2011 10:05:21 +0000 (12:05 +0200)]
htools: add parsing of input node-evac messages
This is a lot of duplication with modeMreloc, which will get sorted
out once we convert mreloc to change-group.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 4 Jul 2011 14:20:38 +0000 (16:20 +0200)]
htools: remove pre-2.4 compatibility from RAPI
This was potentially hiding issues in the group query, so let's remove
it since we're now bound to the current Ganeti version.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 29 Jun 2011 16:02:44 +0000 (18:02 +0200)]
htools: introduce new data type for node-evac
The evac-mode goes into the Types.hs module since it needs to be
available to Cluster.hs too.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 4 Jul 2011 14:05:47 +0000 (16:05 +0200)]
htools: docstring fixes and improvements
No code changes (except one definition being moved around in QC.hs)
are contained in this patch.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sun, 3 Jul 2011 21:36:53 +0000 (23:36 +0200)]
Fix some issues in htools apidoc generation
First, auto-generated files were not processed if HsColour thus the
'source' link in Haddock documentation was dangling.
Second, we document the original Python constant name in the
Ganeti.Constants source, so that it's easily retrievable (as opposed
to undoing the name transformation).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 8 Jul 2011 09:59:36 +0000 (11:59 +0200)]
Replace an mkdir -p with @mkdir_p@ in Makefile.am
Oops, this shouldn't have gone in in the first place…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sat, 2 Jul 2011 19:45:12 +0000 (21:45 +0200)]
Exclude auto-generated Haskell code from coverage
Especially for Ganeti.Constants, which has many declarations, it
doesn't make sense to check for coverage.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 7 Jul 2011 13:25:43 +0000 (15:25 +0200)]
Update multi-reloc design doc with enhanced results
In order to increase the amount of information returned from the
IAllocator plugin, we enhance the return values from the node-evac and
change-group operations.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 11 Jul 2011 21:49:30 +0000 (23:49 +0200)]
gnt-cluster {command|copyfile}: Support per-group operations
This patch allows commands to be run on and files to be copied to all
nodes within a specific group.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 11 Jul 2011 21:48:12 +0000 (23:48 +0200)]
cli.GetOnlineNodes: Support node group filter, use query2
This patc changes cli.GetOnlineNodes to use query2, which does the
filtering in the master daemon, and adds a new parameter to filter by
node group.
Unittests were added for the old implementation and then adopted to
ensure no functionality was lost.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 11 Jul 2011 19:00:09 +0000 (21:00 +0200)]
Remove references to multi-relocate from iallocator doc
For some reason these were still in tere.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 8 Jul 2011 22:26:27 +0000 (00:26 +0200)]
ht.WithDesc: Work around pylint warning
Explicitely defining “__call__” silences a pylint warning when wrapped
type check functions are used directly. I had no idea pylint is this
intelligent.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 11 Jul 2011 14:50:27 +0000 (16:50 +0200)]
Merge branch 'devel-2.4'
* devel-2.4:
ht: Add new check for numbers
Fix off-by-one bug in job serial generation
Shorten some unbreakable lines in man pages
Correct some spelling mistakes
Fix bug in recreate-disks for DRBD instances
Fix a lint warning
KVM: configure bridged NICs at migration start
Fix RAPI documentation regarding master role
Fix bug in drbd8 replace disks on current nodes
Conflicts:
lib/cmdlib.py: Trivial
lib/opcodes.py: Trivial
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 8 Jul 2011 19:33:16 +0000 (21:33 +0200)]
ht: Add new check for numbers
Places which receive floats can usually also deal with integers, e.g.
OpTestDelay. Tests are added and the new check function is used for the
aforementioned opcode and verifying query results.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 7 Jul 2011 19:10:20 +0000 (21:10 +0200)]
Fix off-by-one bug in job serial generation
Commit
009e73d0 (September 2009) changed the job queue to generate
multiple job serials at once. Ever since it would return one more than
requested.
The “serial” file in the job queue directory is defined to contain the
“last job ID used” (design-2.0). With the change above, the serial file
would always contain the next serial number. The first value returned by
the generating function was the one contained in the file, so during the
switch in 2009 one job may have been overwritten.
This patch changes the code to always return the exact number of
serials, to keep the last used serial on disk and adds an assertion.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 11 Jul 2011 13:42:17 +0000 (15:42 +0200)]
Reverts the patch series about console wrappers
This reverts commits
030a9cb8022b83bf43ec14dfbafd943299bc01c4 and
ae082df0000a785b693b2f4aa434650a81a94bdf.
There are two problems:
- Makefile.am breakage, which is trivial to revert
- unittest breakage, which honestly I'm not sure how to fix and how
serial consoles interact with the unpause helper
After the reset, the startup --paused still works but won't unpause
the instance automatically (if I understood the code correctly).
Furthermore, the code also fixes a style issue in hv_kvm.py (too long
line) introduced by the next commit after the above two.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Stephen Shirley [Thu, 23 Jun 2011 15:15:28 +0000 (17:15 +0200)]
Add gnt-instance start --pause
Creates the instance, but pauses execution before booting. This combined
with 'gnt-instance console' unpausing instances means that the entire
boot process can be viewed and monitored.
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Stephen Shirley [Fri, 24 Jun 2011 11:29:48 +0000 (13:29 +0200)]
Adding a wrapper around connecting to kvm console
The wrapper will connect to the console, and check in the background if
the instance is paused, unpausing it as necessary.
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Stephen Shirley [Thu, 23 Jun 2011 09:42:23 +0000 (11:42 +0200)]
Adding a wrapper around "xm console"
The wrapper will connect to the console, and check in the background if
the instance is paused, unpausing it as necessary.
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 5 Jul 2011 22:54:42 +0000 (00:54 +0200)]
Fix lint error
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Mon, 27 Jun 2011 22:13:23 +0000 (00:13 +0200)]
RAPI: Document all feature strings
- Use constants and an assertion
- Update documentation for node migration
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 23 May 2011 13:58:29 +0000 (15:58 +0200)]
Remove old node evacuation opcode
LUNodeEvacStrategy has been replaced with LUNodeEvacuate.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Mon, 23 May 2011 12:43:33 +0000 (14:43 +0200)]
Change RAPI for new node evacuation opcode
The change is not backwards compatible, see the updated NEWS file.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 20 May 2011 14:31:54 +0000 (16:31 +0200)]
Change “gnt-node evacuate” to use new opcode
By default it'll now evacuate all instances from the node, not
just secondaries.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 20 May 2011 13:30:33 +0000 (15:30 +0200)]
Add new opcode to evacuate node
This new opcode will replace LUNodeEvacStrategy, which used to return a
list of instances and new secondary nodes. With the new opcode the
iallocator (if available) is tasked to generate the necessary operations
in the form of opcodes. This moves some logic from the client to the
master daemon.
At the same time support is added to evacuate primary instances, which
are also evacuated by default.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Guido Trotter [Tue, 5 Jul 2011 16:35:25 +0000 (17:35 +0100)]
Alias gnt-job show to gnt-job info
Am I the only one to make that mistake 10 times a week?
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 29 Jun 2011 13:06:12 +0000 (15:06 +0200)]
One Haskell and integer sizes fix
Haskell has two main integer types:
- Int, which is a native-type, and is guaranteed to have at least
[-2²⁹, 2²⁹-1] range; on 64-bit platforms, it has much higher range
- Integer, which is a software type (implemented using libgmp), and
thus unbounded
For performance reasons, the node/instance properties use Int for
their attributes (and Double for some, but that's another story). This
is all fine and doesn't cause problems. However, the CStats type which
holds the overall cluster resources starts to fail when we analyse
clusters with more than around 400 nodes and big memory/disk sizes on
32 bit platforms.
The simple fix would be to restrict cluster sizes, but that's no
nice. I've benchmarked and changing to Integer doesn't show a visible
slowdown on 64-bit platforms (as far as I can read on the internets,
GHC knows to optimise Integer and only use software types when the
values are large enough), and it also fixes the 32-bit problem. So
this patch changes the CStats types to Integer, except for the
instance count (which I don't expect to overflow 2²⁹ anytime soon).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 1 Jul 2011 10:25:26 +0000 (12:25 +0200)]
Fix cluster verify for empty node groups
There were some implicit assertions in the code that all node groups
have nodes, which is not necessarily true.
Additionally, the patch does a wrapping change.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 30 Jun 2011 11:19:42 +0000 (13:19 +0200)]
Fix a typo and punctuation in iallocator.rst
Beside the 'dscription' typo, also make the punctuation more
consistent.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 30 Jun 2011 15:15:51 +0000 (17:15 +0200)]
Shorten some unbreakable lines in man pages
In order to make the display right on 80-columns terminals.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 30 Jun 2011 15:07:34 +0000 (17:07 +0200)]
Correct some spelling mistakes
New lintian is even smarter:
- overriden → overridden
- allows to → allows one to
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>