Statistics
| Branch: | Tag: | Revision:

root / src / Ganeti / HTools / Program @ 9d049fb4

# Date Author Comment
9d049fb4 11/26/2013 03:30 pm Klaus Aehlig

Rename LuxiSocket to MasterSocket

Rename the constants to name the socket to connect masterd,
as the name LuxiSocket hints on luxid, which is different
from masterd.

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Petr Pudlak <>

81879d92 11/25/2013 05:50 pm Klaus Aehlig

hsqueeze: fix position of option in gnt-node power

hsqueeze can produce a shell script with the commands to squeeze
the cluster; in the script, fix the position of the '-f' option
in the 'gnt-node power' command.

Signed-off-by: Klaus Aehlig <>...

268fb0bd 11/15/2013 05:40 pm Klaus Aehlig

hsqueeze: add option to show or save commands

Add an option to hsqueeze to show, or save in a file, the commands
that have to be carried out.

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Hrvoje Ribicic <>

bbc6620d 11/15/2013 05:40 pm Klaus Aehlig

hsqueeze: when balancing also keep the move sequence

In hsqueeze, when computing the balancing sequence, also
remember the sequence of moves that lead there.

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Hrvoje Ribicic <>

7c3a6391 11/15/2013 05:39 pm Klaus Aehlig

Move saving of a command list to CLI

Move the function that saves a list of a command in a file
to CLI.hs. In this way, it is reusable by other htools.

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Hrvoje Ribicic <>

c407510c 10/31/2013 06:00 pm Klaus Aehlig

hsqueeze: support planing for onlining nodes

If the amount of free resources falls below a given threshold,
hsqueeze will suggest putting standby nodes back online until
the minimum of free resources is reached, or all standby nodes
are online.

Signed-off-by: Klaus Aehlig <>...

5f6e9cb9 10/29/2013 02:26 pm Klaus Aehlig

Add hsqueeze planning for compression

Add a new htool, hsqueeze, for dynamic power management. This
commit only implements the first useful part: plan (but not
executing) taking nodes offline nodes still keeping within the
resource limit.

Signed-off-by: Klaus Aehlig <>...

c62bec27 09/30/2013 03:20 pm Spyros Trigazis

Import MonD data from file

Implement functionality to import MonD data from a mock file to
HTools (mainly for testing purposes) with the --mond-data
option.

Signed-off-by: Spyros Trigazis <>
Signed-off-by: Michele Tartara <>...

c32c4e4d 09/30/2013 03:20 pm Spyros Trigazis

Fetch data from MonD to HTools

Contact all MonDs from HTools to fetch data from its Data
Collectors (only CPUload Data Collector is queried at the
moment). This information is available to all HTools with the
--mond option and can be ignored if the --ignore-dynu option is...

14b4e045 09/12/2013 06:26 pm Klaus Aehlig

Make hbal support the option --ignore-dynu

This will allow to do balancing based only on static information.
In particular, the effect of balancing towards same number of
instances per node that the dynamic usage consideration has
if no usage file is provided, can be turned off, if this is...

5cbf7832 08/08/2013 10:31 am Jose A. Lopes

Fix opcodes and parameters

  • fix some return types in opcodes and some types and documentation in
    parameters.
  • fix serialization of the default value of the 'name' parameter in
    the opcodes related to tags when tag kind is 'cluster'
  • revert changes to 'pShutdownTimeout'' and its references because the...
1ca326c8 08/07/2013 06:46 pm Thomas Thrainer

Merge branch 'stable-2.9'

  • stable-2.9
    Fix batch creation of instances
    Fix documentation of gnt-instance batch-create
    Support multi instance allocs without iallocator
    Support DSA SSH keys in bootstrap
    Include VCS version in `gnt-cluster version`...
804d72eb 08/07/2013 05:41 pm Thomas Thrainer

Merge branch 'stable-2.8' into stable-2.9

  • stable-2.8
    Fix batch creation of instances
    Fix documentation of gnt-instance batch-create
    Support multi instance allocs without iallocator
    Support DSA SSH keys in bootstrap
    Include VCS version in `gnt-cluster version`...
6e94b75c 08/07/2013 10:41 am Jose A. Lopes

Update Harep, Query server, and tests

Update Harep, Haskell query server, and tests concerning Luxi and
opcodes to reflect the changes to Haskell to Python opcode
generation. This change is necessary because TagObject is replaced by
TagKind and some types in opcodes and parameters changed to be...

aa7a5c90 07/30/2013 04:12 pm Michele Tartara

Add cleanup parameter to instance failover

Most of the code is shared with instance migrate, so we actually only need
to add the parameter and pass its value along the the common code.

Also, tests and harep are updated to support the right set of options to...

a12b230c 07/01/2013 10:53 am Klaus Aehlig

hroller: option --full-evacuation

Add an option to hroller, to plan for full evacuation of the
nodes to be rebooted, i.e., also plan for replacement secondary
nodes for all instances on the node after migrating out instances
with this node as primary.

Signed-off-by: Klaus Aehlig <>...

fdbdeaa3 07/01/2013 10:53 am Klaus Aehlig

Extract a partition functional

Separate the partitionNonRedundant function in hroller into a
general functional that partitions a list of nodes according
to some clearing strategy and the specialization of moving
non-redundant instances out. In this way, we don't have to...

a947a583 07/01/2013 10:53 am Klaus Aehlig

Extract functional for greedily clearing nodes

The method clearNodes in hroller greedily clears nodes of
non-redundant instances by moving them to a different node. This patch
separates the greedy clearing algorithm from the specialization to
non-redundant instances; in this way, we don't have to duplicate code...

634a1460 07/01/2013 10:52 am Klaus Aehlig

Make hroller not consider offline nodes for evacuation

When planing on where to evacuate the non-redundant instances
of the nodes to be rebooted, it doesn't make sense to consider
offline nodes. So add this restriction to hroller.

Signed-off-by: Klaus Aehlig <>...

a8cbe1d7 07/01/2013 10:52 am Klaus Aehlig

Update comments in hroller code

hroller schedules moves of instances to have rebooted nodes
free of instances with this node as primary. Update the comments
to reflect that this move planning is for non-redundant instances
only.

Signed-off-by: Klaus Aehlig <>...

34ad1d7c 07/01/2013 10:52 am Klaus Aehlig

Remove obsolete TODO

Originally, hroller started as a tool for offline maintenance only.
There it made sense to warn about instances still running. By now,
default planning is to migrate instance off the nodes to be rebooted,
with options for other behavior (like pretending that all instances...

da4a52a3 06/20/2013 05:04 pm Thomas Thrainer

Index instances by their UUID

No longer index instances by their name but by their UUID in the cluster
config. This change changes large parts of the code, as the following
adjustments were necessary: * Change the index key to UUID in the configuration and the...

1c3231aa 06/13/2013 12:32 pm Thomas Thrainer

Index nodes by their UUID

No longer index nodes by their name but by their UUID in the cluster
config. This change changes large parts of the code, as the following
adjustments were necessary: * Change the index key to UUID in the configuration and the
ConfigWriter, including all methods....

e7aa0b03 06/10/2013 06:04 pm Klaus Aehlig

Add type annotation to avoid monomorphism restriction

Even though we need the let-bound variable showMoves only
at type [(String, String)] -> IO (), it's most general type
would be (PrintfArg a, PrintfArg b) => [(a, b)] -> IO ().
This causes the monomorphism restriction apply to that binding,...

30ce253e 06/10/2013 04:57 pm Klaus Aehlig

add option --print-moves to hroller

If non-redundant instances are present in the cluster, hroller will
plan for them to move to other nodes while the group is rebooted.
This adds an option to also show this plan.

Signed-off-by: Klaus Aehlig <>...

0cc3d742 06/07/2013 04:11 pm Bernardo Dal Seno

hspace prints info about spindles

Statistics about spindles are tracked. In human-readable output, spindles
are printed only when used (i.e., exclusive storage is enabled). For
machine-oriented output, they are always there.

Signed-off-by: Bernardo Dal Seno <>...

914c6df4 06/07/2013 04:11 pm Bernardo Dal Seno

Spindles become part of htools resource spec

Spindles are now part of resource spec. Instances get created with spindles
specified (which are just ignored when exclusive storage is disabled).

Signed-off-by: Bernardo Dal Seno <>
Reviewed-by: Klaus Aehlig <>

2724417c 06/03/2013 04:15 pm Bernardo Dal Seno

Add spindles to instance disks in htools

A new data type is introduced for disks to store both size and spindles.
When available, spindles are filled with input data. Except for loading and
storing, spindles are ignored.

Signed-off-by: Bernardo Dal Seno <>...

f2e50930 06/03/2013 11:25 am Klaus Aehlig

Restrict instance moves in hroller to the same node group

When scheduling rolling reboots, hroller looks for nodes to evacuate
the non-redundant instances to. This is done by greedily moving
instances to other nodes that can take them, policy wise and capacity...

342a9b3b 05/31/2013 04:36 pm Bernardo Dal Seno

Merge branch 'stable-2.8' into master

  • stable-2.8:
    Document users-setup tool
    Adjusting permissions after confd start
    Ensure the queue socket has the right permissions
    Update IAllocator interface documentation
    Add NEWS entry for hail honoring networks...
908c2f67 05/29/2013 10:52 am Thomas Thrainer

Parse NIC data from allocation request in hail

Add a NIC type and extend the Instance type by a list of NIC's. Parse
the NIC's in allocation requests and store them for now. Later patches
will make use of this field in order to ensure that the requested
instance is only placed in node groups wich are connected to those...

23247a73 05/22/2013 12:08 pm Klaus Aehlig

hroller: option to ignore non-redundant instances

Add an option to hroller restoring the old behavior on not taking
any non-redundant instances into account when forming reboot
groups.

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Guido Trotter <>

86c346db 05/22/2013 12:08 pm Klaus Aehlig

Make hroller also plan for non-redundant instances

Non-redundant instances need to be moved to a different node
before maintenance of the node. Even though they can be moved to
any node, there must be enough capacity to host the instances of the
reboot group to be evacuated....

89363f98 05/22/2013 12:07 pm Klaus Aehlig

hroller: option to skip nodes with non-redundant instances

So far, hroller ignores the fact, that non-redundant instances exist.
One option to deal is non-redundant instances is to not schedule those
nodes for reboot. This is supported by adding the option --skip-non-redundant....

3409c0af 05/22/2013 12:07 pm Klaus Aehlig

Remove trailing whitespace

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Guido Trotter <>

8d38fb72 05/10/2013 01:55 pm Klaus Aehlig

Support online-maintenance in hroller

Make hroller take into account the nodes (redundant) instances
will be migrated to. This be behavior can be overridden by the
--offline-maintenance option which will make hroller plan under
the assumption that all instances will be shutdown before starting...

2207220d 05/10/2013 01:55 pm Klaus Aehlig

Add option --one-step-only to hroller

Add a new option to hroller to only output information about the first
reboot group. Together with the option --node-tags this allows for the
following work flow. First tag all nodes; then repeatedly compute the
first node group, handle these nodes and remove the tags. In between...

a39779f6 05/10/2013 01:54 pm Klaus Aehlig

Sort reboot groups by size

Make hroller output the node groups not containing the master node
sorted by size, largest group first. The master node still remains
the last node of the last reboot group. In this way, most progress
is made when switching back to normal cluster operations after the...

004398d0 05/07/2013 07:40 pm Klaus Aehlig

Fix lint errors (redundant bracket)

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Guido Trotter <>

313fdabc 05/07/2013 06:59 pm Klaus Aehlig

Add option to hroller to select nodes based on tags

Add option --node-tags to tell hroller to consider only nodes
with these tags. A use case would be a tag tracking on which
nodes the maintenance has not yet been carried out, e.g., if
rolling reboots are interleaved with other cluster operations....

442d5aae 05/07/2013 06:58 pm Klaus Aehlig

Make hroller filter the nodes before coloring the graph

Hroller used to first compute a coloring of the node graph and then
filter out the nodes that it had to work on. While the only filtering
was according to node groups this did not make a difference, as there...

53822ec4 04/29/2013 06:54 pm Bernardo Dal Seno

hspace: Handle multiple ipolicy specs

With tiered allocation, hspace uses all the max specs in turn as the
initial instance spec.

Signed-off-by: Bernardo Dal Seno <>
Reviewed-by: Helga Velroyen <>

41044e04 04/29/2013 06:53 pm Bernardo Dal Seno

Add multiple min/max specs in instance policy

Now instance policies can contain more than one min/max specs. This is the
main element of the "Constrained instance sizes" section in the
"Partitioned Ganeti" design doc.

This is a big patch, but changing the type of a configuration item requires...

7dbe4c72 04/19/2013 02:54 pm Klaus Aehlig

Make hroller insist on finding precisely one master node

As people rely on the master node being the last node of the last
group, make hroller fail, if no master node could be found in the
cluster. This happens, e.g., if a backend format is used that does not...

5b658b83 04/19/2013 02:54 pm Klaus Aehlig

Make Hroller present master node last

If in the list of nodes to be scheduled for maintaince,
one is marked as being the master node, schedule it
as the last node in the last group.

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Guido Trotter <>

ffc18bb2 04/17/2013 10:29 pm Thomas Thrainer

Fix warnings hlint 1.8.43 complained about

These lines are ok according to previous versions of hlint but trigger
an error with version 1.8.43.

Signed-off-by: Thomas Thrainer <>
Reviewed-by: Helga Velroyen <>

723ec678 04/12/2013 12:31 am Guido Trotter

Merge branch 'devel-2.7'

  • devel-2.7: (26 commits)
    Fix burnin install path
    Fix format of the NEWS file
    NEWS: Add news entry for the hail disk policy fix
    Add shelltests verifying hail applies disk ipolicy per disk
    Make the disks parameter available to the constructor...
241cea1e 04/11/2013 05:10 pm Klaus Aehlig

Make the disks parameter available to the constructor

In that way, tools building on Instance will benefit from the corrected
verification semantics of the instance policy on disk space.

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Guido Trotter <>

da5f09ef 03/27/2013 12:46 pm Bernardo Dal Seno

Refactor ispecs in ipolicy structures

Minimum and maximum instance specs are put together into a single element
of the instance policy. This is in preparation for introducing multiple
min/max specs.

Signed-off-by: Bernardo Dal Seno <>...

4f73cfc9 03/08/2013 05:46 pm Iustin Pop

Merge branch 'devel-2.7'

  • devel-2.7:
    Change hbal behaviour in case of early exit
    Fix build/sphinx_ext.py with tuple defaults for op params
    Fix bug in man build rule
    Fix hscolour style sheet building

Signed-off-by: Iustin Pop <>...

2d6bdcc5 03/08/2013 02:50 pm Iustin Pop

Change hbal behaviour in case of early exit

Currently, hbal exits with status 1 if early exit is requested, even
when all jobs are successful. This is counter-intuitive behaviour, so
let's fix it (Issue 386).

Note that the man page had conflicting information already, so it's a...

491d02ca 03/05/2013 01:56 pm Iustin Pop

Merge branch 'devel-2.7'

  • devel-2.7: (23 commits)
    Add export lists for files which didn't have them
    Fix Haskell compatibility tests with disabled file storage
    Fix QA with disabled file storage
    Fix convert-constants handling of booleans
    Fix handling of disabled (shared) file storage...
bfa99f7a 03/04/2013 12:35 pm Iustin Pop

Fix low verbosity levels in htools

In a few cases, we tested the verbosity level for (== 0), instead of
higher/lower than a certain value. If the user passes multiple
"--quiet" options, this can result in negative verbosity levels, which
behave like "extra verbosity"....

2fce67b6 02/25/2013 08:05 pm Guido Trotter

HRoller: allow filtering by node group

Accept the -G option, and if it's passed require that it matches a
nodegroup, then only output nodes belonging to that group.

Signed-off-by: Guido Trotter <>
Reviewed-by: Iustin Pop <>

544029d3 02/25/2013 08:05 pm Guido Trotter

HRoller: print only online nodes

To make the graphs work even when instances live on offline nodes (eg.
because we're offlining them just to exclude them, or because they have
instance still on them) we just filter them out at the end, when we're
going to print out the result....

fe502d25 02/15/2013 03:04 pm Iustin Pop

Merge branch 'devel-2.7'

  • devel-2.7:
    Rename lib/objectutils to outils.py
    Fix typo in gnt-group manpage
    Fix wrong type in a docstring of the RAPI subsystem
    Finish the remote→restricted commands rename
    Enable use of the priority option in hbal
    Add CLI-level option to override the priority...
a81ca843 02/12/2013 03:47 pm Iustin Pop

Enable use of the priority option in hbal

This patch adds the option to hbal, and uses it to tweak the submitted
jobs. There are also two small shelltests for testing the parsing.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

d41f6558 02/12/2013 03:47 pm Iustin Pop

Make hbal opcode annotation more generic

Currently, hbal code always uses annotateOpCode function, which means
we would have to pass the options data to all function in the call
chain if we wanted to make this more flexible.

By abstracting the type of the annotator and passing it as an argument...

3c296f56 02/12/2013 03:25 pm Michael Hanselmann

Merge branch 'devel-2.7'

  • devel-2.7: (23 commits)
    QA: Support additional arguments for initialization
    qa_utils: Fix order of arguments passed to _AssertRetCode
    Improve reporting on errors.AddressPoolError exceptions
    Add note about lv-tags rename...
72747d91 02/07/2013 06:14 pm Iustin Pop

Remove use of 'head' and add hlint warning for it

Since 'head' is unsafe to use in most cases, this patch removes its
use from most of the code, adds a lint warning for it (and for tail as
well), and adds override annotations in the few cases where it's
actually OK to use it (mainly when using head over the result of...

546a1dcf 02/07/2013 05:02 pm Dato Simó

Harep.hs: fix a couple typos in comments and docstrings

Signed-off-by: Dato Simó <>
Reviewed-by: Guido Trotter <>

41238d27 01/30/2013 07:39 pm Dato Simó

harep: finish execution with a summary of instance states

The harep tool prints messages for every action that it performs (or that
it doesn't perform). In case nothing is to be done at all, always print
some statistics of the current state of the cluster....

8af72964 01/30/2013 07:39 pm Dato Simó

harep: do not wait for repair job completion to set tags

Because of instance locks, after submitting a repair job we weren't able to
set the "pending" tag until at least the first opcode of the job finished.
Introduce a small delay in the repair job so as to allow the subsequent...

8072af6c 01/30/2013 07:39 pm Dato Simó

harep: create repair jobs

Implement 'doRepair' to create a repair job from a list of opcodes if
the instance's policy allows it (otherwise set an ENOPERM result label),
and the instance was previously healthy (i.e. not in ArFailed or
ArPendingRepair).
...

4ba5f148 01/30/2013 07:39 pm Dato Simó

harep: pure function to detect brokeness with instances

Add a 'detectBroken' function that determines whether an instance is in an
unhealthy state, and what's needed to repair it. The repair is specified as
an AutoRepairType constant, and a list of opcodes. The opcodes will only be...

a5e58418 01/30/2013 07:39 pm Dato Simó

harep: check for completed jobs at the start of the program

As a first step before detecting any brokeness with instances, see if any
of our previous repairs have completed, and move instances to ArFailed or
ArHealthy accordingly. Do nothing if there are still running jobs for the...

3416e3e7 01/30/2013 07:39 pm Dato Simó

harep: initial parsing of tags

Parse auto-repair tags to set each instance in one of ArHealthy, ArFailed,
or ArPendingRepair. The implementation tries to be well behaved when old
tags have been left behind, which future patches will still try not to do....

b6d9bec8 01/30/2013 07:38 pm Dato Simó

Program/Harep.hs: add skeleton for the new auto-repair tool

harep(1) detects certain kind of problems with instances and applies the
allowed set of solutions. See doc/design-autorepair.rst.

Signed-off-by: Dato Simó <>
Reviewed-by: Iustin Pop <>

ef947a42 01/14/2013 05:54 pm Dato Simó

Loader.hs: ignore expired ArSuspended policies

At the moment, because 'mergeData' is pure, it may set instance auto-repair
policies that are of the form `ArSuspended $ Until timestamp_in_the_past`.
If later on the auto-repair tool notices this, it has lost access to what...

f6b9c6e5 12/28/2012 05:50 pm Iustin Pop

Fix a bad data type in Hcheck.hs

While trying to understand why some code was not being tested, I
realised that we have a bad data type in Hcheck.hs.

We have "data Level = GroupLvl | ClusterLvl", but then we need to pass
the group name/index as well, so we have functions that look like the...

2fd5a116 12/28/2012 05:50 pm Iustin Pop

Move src/Ganeti/HTools/Program.hs to Program/Main.hs

This removes one more tab conflict; this is the last module in our
code where we have both x.hs and x/.

Furthermore, we collapse all actual code into the new Main.hs module,
leaving the htools.hs basically empty (will allow better testing in...

3add7574 12/24/2012 11:38 pm Iustin Pop

Rename htools/ to src/

Per offline discussions, this is the first patch of the
renames. Tested with "make distcheck", seems to work fine.

The only change outside of the renaming is a bit of simplification in
the .gitignore rules; otherwise, simply s/htools/src/....