| Branch: | Tag: | Revision:

root / Ganeti @ 57587760

# Date Author Comment
57587760 03/31/2010 12:54 pm Iustin Pop

Fix iallocator crash when no solutions exist

Commit 5436576 added an un-guarded `head' call, which crashes with
“Prelude.head: empty list” when no results exists for the per-instance
allocation/relocation calls.

This patch fixes this, and also adds another check for an unguarded...

934c62dc 03/31/2010 12:51 pm Iustin Pop

Fix IAllocator multi-evacuate message

Since Ganeti passes full host names (not common-suffix-stripped), we
need to remove the suffix from the evac_nodes keys too. In case one node
is not part of the cluster, it will lead to a wrong error message, but
for now it fixes the problem.

12b0511d 02/22/2010 04:19 pm Iustin Pop

Add a tryEvac function

This will be used by the node evacuate IAllocator request type.

Signed-off-by: Iustin Pop <>

54365762 02/22/2010 04:19 pm Iustin Pop

Implement IAllocator node evacuate request

This patch adds the new request loading/execution (trivial), but the
actual response formatting becomes more difficult as now the response
type differs by request.

Signed-off-by: Iustin Pop <>

20c891d0 02/22/2010 04:19 pm Iustin Pop

IAllocator: move some keys into per-request data

Since not all structures will have these keys in the future, we move
them into per-structure keys.

Signed-off-by: Iustin Pop <>

23f9ab76 02/22/2010 04:19 pm Iustin Pop

Change an internal type from Maybe to list

In preparation for multiple responses, we change from Maybe to List
(both used in the container sense).

This allows us to keep the same workflow for all kind of requests.

Signed-off-by: Iustin Pop <>

1fe81531 02/22/2010 04:19 pm Iustin Pop

Move a type declaration to Node.hs

We'll need AllocElement in both Cluster and IAlloc in the future, so we
move it to Node.hs which is imported by both.

Signed-off-by: Iustin Pop <>

2e28ac32 02/22/2010 03:50 pm Iustin Pop

Implement evacuation mode in hbal

This mode restricts the list of instances to be moved to the instances
living on the offline (and drained) nodes.

Signed-off-by: Iustin Pop <>

f0f21ec4 02/22/2010 03:50 pm Iustin Pop

Add an evac mode CLI option

Signed-off-by: Iustin Pop <>

df18fdfe 02/22/2010 03:50 pm Iustin Pop

Reorder options in CLI.hs

This should be no code change, just reordering of the options.

Signed-off-by: Iustin Pop <>

146b37eb 02/03/2010 01:29 pm Iustin Pop

Fix secondary node selection for existing N+1

In case a secondary node is already N+1 failed, currently the node
selection will accept a node that cannot start (at all) the new instance
as valid. This is wrong, so we add a new simple check to prevent the...

a4a6e623 02/03/2010 10:24 am Iustin Pop

Rewrite the node add checks for simpler layout

This will make it clearer than many if…then choices.

a804261a 01/14/2010 06:38 pm Iustin Pop

Move instance relocation test upper in the chain

Currently we test each instance for relocation in checkMove; however, it
is a little more clear if we pass only the relocatable instances to
checkMove. The patch also slightly rewrites (indendation/style) the...

5ad86777 01/14/2010 06:05 pm Iustin Pop

Split the balancing function in two parts

Currently in the balancing function we do two thing:

- take the decision where to do a new balancing round or not
- and actually computing the balancing round

This is not nice, as the two parts are conceptually separate, so this...

71e635f3 01/12/2010 12:18 pm René Nussbaumer

Fixing a typo in option description

Signed-off-by: René Nussbaumer <>
Reviewed-by: Michael Hanselmann <>
Signed-off-by: Iustin Pop <>

16c2369c 01/07/2010 01:57 pm Iustin Pop

Switch the text file format to single-file

This patch changes from the two separate files to a single file, with
sections separated by a blank line. Currently only the node and instance
data is accepted, later the cluster tags will be read too via this

f5197d89 01/07/2010 12:44 pm Iustin Pop

Change the signatures of the text loader slightly

This is in preparation for the text format changes.

0ccaab44 12/28/2009 12:09 pm Iustin Pop

Fix small typo

This was found, of all things, via lintian during the Debian packaging…

0c860cff 12/11/2009 07:01 pm Iustin Pop

Convert n1_score metric from % to count

This increases the priority of fixing N+1 failures compared to balancing

8ce618f3 12/11/2009 06:54 pm Iustin Pop

Merge branch 'master' into next

  • master:
    Use the oper_ram field if available
    rapi, luxi: treat drained nodes as offline
673f0f00 12/11/2009 06:47 pm Iustin Pop

Metric: count of primary instances/offline nodes

This helps with evacuation/failover of instances on 2-node clusters with
one one offline.

e4d31268 12/11/2009 06:43 pm Iustin Pop

Offline instance metric: change from % to count

Currently we use the offline instance percentage (with range [0, 1]),
but this is not good, since we want the evacuation of such instances to
have a high priority; therefore we change this to a count of offline...

6402a260 12/11/2009 06:17 pm Iustin Pop

Use the oper_ram field if available

For the RAPI and LUXI backends, we can get the actual memory usage (if
instances are running) via the oper_ram, whereas backend/memory only
tell what the instance will use at the next boot.

Not using oper_ram means that the node model is flawed and we consider...

b45222ce 12/09/2009 12:30 pm Iustin Pop

rapi, luxi: treat drained nodes as offline

Commit e97f211 changed the iallocator backend to handle drained nodes as
offline. This commit completes that change by making the rapi and luxi
backend do the same (the text backend ignores any '?' values which are...

1cea2e1e 12/02/2009 04:58 pm Iustin Pop

Fix typo breaking LUXI backend

This really shows the need for actual dist-time full testing (not

434c15d5 12/02/2009 12:57 pm Iustin Pop

Fix unittests after instance tags addition

f5e67f55 12/01/2009 02:49 pm Iustin Pop

Configure exclusion tags via the cluster tags

This patch adds reading of the exclusion tags from the cluster tags: any
tags starting with htools:iextags: will convert their suffix into an
exclusion tags prefix. In other words, "htools:iextags:service" will...

669ea132 12/01/2009 01:24 pm Iustin Pop

Read cluster tags in the IAllocator backend

f89235f1 12/01/2009 12:47 pm Iustin Pop

Read cluster tags in the LUXI backend

ea017cbc 12/01/2009 11:53 am Iustin Pop

Read cluster tags in the RAPI backend

This also shows them in hbal in verbose mode.

94e05c32 11/27/2009 05:13 pm Iustin Pop

Introduce support for reading the cluster tags

While these are not actually populated from the backends, and all the
programs ignore them, this patch contains the changes in the function
types required.

0f15cc76 11/17/2009 11:44 am Iustin Pop

Add a command-line option to filter exclusion tags

Since we don't want all instance tags to be used for exclusion, we add a
command line option to filter on these. Since the iallocator protocol
cannot accept command line options, currently it's not possible to...

b2999982 11/17/2009 11:44 am Iustin Pop

Add a new node list field

This patch adds a new node list field (ptags), showing the primary
instance tags.

1e4b5230 11/17/2009 11:44 am Iustin Pop

Node: add function for conflicting primary count

d844fe88 11/17/2009 11:44 am Iustin Pop

Use conflicting primaries count in cluster score

This small patch adds the number of conflicting primaries in the cluster
score. This is different from the other non-CV metrics where we usually
compute the percentage of failing instances (for that metric); but for a...

e27eb8ab 11/17/2009 11:44 am Iustin Pop

Specialize the math functions

The statistics functions are currently defined as polymorphic with a
Floating constraint. Changing this to monomorphic on Double type makes
them stricter and much more performant (~70% speedup). This is a cheap
way to recoup some of the loses incurred by the recent proliferation of...

185297fa 11/17/2009 11:44 am Iustin Pop

Collapse the statistical functions into one

This allows us to get rid of two duplicate list length computations,
with a minor speedup.

5f0b9579 11/17/2009 11:44 am Iustin Pop

Introduce tag-based exclusion of primary instances

This patch introduces exclusion of primary instances based on tags. This
is incomplete as currently all tags are being excluded, and we don't
optimise towards relocation of instances sharing tags on the same node.

17e7af2b 11/11/2009 12:10 pm Iustin Pop

Add a tags attribute to instances

… and read it in all the loaders. hscan is modified to save it to the
files it generates.

The attribute is not yet used in any place.

27671a61 11/11/2009 11:39 am Iustin Pop

Small change in some list arguments

This is simpler than the concat operator.

e98fb766 11/10/2009 02:59 pm Iustin Pop

Allow overriding the field list in -p

The print nodes option can now accept an optional field list to
customise the output. This is ugly, since the field names do not match
the header names, but it is at least barely customisable (at runtime).

76354e11 11/09/2009 05:49 pm Iustin Pop

Move more node-listing functionality in Node.hs

This will prepare for the runtime-selectable field list.

a7a8f280 11/09/2009 04:51 pm Iustin Pop

Change the default dynamic usage to baseUtil

This fixed the unbalanced secondary instances on partially empty
clusters, and helps in general for the cases where real utilisation data
is not available.

daee4bed 11/09/2009 03:43 pm Iustin Pop

Add a few comments in the scoring function

117dc2d8 11/06/2009 07:23 pm Iustin Pop

Enhance the error reporting for Rapi and Luxi

Currently the JSON conversion in Rapi and Luxi are giving something
Error: failed to load data. Details:
Unable to read Double

This doesn't tell one where the error is (in a node specification? and...

262f3e6c 11/06/2009 07:23 pm Iustin Pop

Change the Utils.fromObj signature

Currently the fromObj function takes a JSON object which is then
converted into a list of (String, JSValue) in which we make a lookup.
However, most of the callers of this function call it repeatedly on the
same object, which means we do the object→list conversion repeatedly....

7f4e37f0 11/06/2009 07:13 pm Iustin Pop

Make some CLI options more consistent

Both the simulate and the tiered allocation mode take a machine spec on
input via a comma-separated list. This patch makes this a little bit
more consistent (always use disk,ram,cpu in this order).

83ad1f3c 11/06/2009 07:13 pm Iustin Pop

hspace: show tiered-alloc stats in the output

This is a first attempt to get a readable output of tiered allocation
stats in hspace's output. Not very nice, but it should be somewhat

12e6776a 11/06/2009 07:13 pm Iustin Pop

A small style change in Node.hs

This imports PeerMap as P and reindents some lines.

c8db97e5 11/06/2009 07:12 pm Iustin Pop

Add support for shrinking instance specs

This patch adds a function that, for some given failure modes, shrinks a
given instance in the hope that allocation will succeed when retried
with the new spec.

2f567ac0 10/30/2009 01:23 pm Iustin Pop

Convert option parsing to a monadic flow

This allows us to do verification of option arguments in the assignment
functions themselves.

1f9066c0 10/30/2009 01:23 pm Iustin Pop

Rework the instance spec CLI options

This patch reworks the internal handling of the instance spec CLI
option, and adds a tiered spec option that will be used in hspace to
enable the (auxiliary) tiered-spec allocation mode.

It also introduces a new data type for holding the instance...

ce0392e6 10/21/2009 02:06 pm Iustin Pop

Some cleanup of Loader.mergeData

This doesn't need to be a monadic function, let's make it a simpler one.

a5f8dcdc 10/21/2009 11:58 am Iustin Pop

hbal: ignore unknown instance in dynload file

Since the utilisation file might be generated at a different time from
the hbal run, and instances could dissapear in the meantime, it's better
to simply ignore unknown instances rather than abort.

30ff0c73 10/21/2009 11:47 am Iustin Pop

Expand the --print-instances output

This adds run status, resource parameters and load parameters for

e7724ccc 10/19/2009 01:08 am Iustin Pop

Change the Container.findByName function

This patch changes the signature and implementation of the function;
returning the item makes more sense (saves a lookup later again in the
container, and applying idx is cheap), and the previous implementation
was ugly.

bbd8efd2 10/19/2009 01:08 am Iustin Pop

Some small style fixes

8c9af2f0 10/19/2009 12:17 am Iustin Pop

Simplify the cstats initializer

Since all values are initialized to zero, the exact ordering is not
important and thus we can use the positional mode for simpler code.

The patch also adds docstrings to the cstats functions.

668c03b3 10/19/2009 12:11 am Iustin Pop

Simplify Cluster.computeMoves

Since we now have an actual type for describing the instance moves
(IMove), it's simpler to convert this into the move description/move
commands, rather than re-computing the move based on initial and final
nodes. This makes the shell commands computation and over-Luxi command...

eb2598ab 10/18/2009 11:20 pm Iustin Pop

Remove obsolete export

The ‘Placement’ type has been moved to Types.hs but we kept exporting it
from Cluster, which is not needed.

c5f7412e 10/18/2009 08:21 pm Iustin Pop

Generalise the node/instance listing

This patch introduces a generic formatTable function (based on, and
similar to the Ganeti one, but different and more FP in style) and
changes the node and instance listing to it.

The node list (due to the many variables) is still a little bit hackish...

ad6cffe4 10/18/2009 07:38 pm Iustin Pop

Fix instance listing for non-redundant case

c4ef235b 10/16/2009 05:24 pm Iustin Pop

Fix two haddoc/happy docstring issues

ee9724b9 10/16/2009 04:59 pm Iustin Pop

Start using the utilisation scores in balancing

This enables the per-node load/total available capacity scores to be
used in balancing. Note that the total available capacity is currently
fixed at zero and cannot be changed by the user.

aa8d2e71 10/16/2009 04:59 pm Iustin Pop

Add loading and processing of utilisation data

This patch adds loading and processing the utilisation data during
instance moves. While the data is not yet used, it is correctly modified
by instance changes between nodes.

hbal has the new ‘-U’ command line argument for this. The format of the...

4f83a560 10/16/2009 02:54 pm Iustin Pop

Add an option to input utilisation data

a488a217 10/16/2009 02:43 pm Iustin Pop

Merge the Node.setPri and Node.addCpus functions

The latter is only used right after the former in the Loader module, and
we'll need more of this 'update not with the data of this instance'
functionality (which is different than addPri where all information must...

5b763470 10/16/2009 10:41 am Iustin Pop

Move some utility functions to Utils.hs

These were already duplicate (Text and Simu) and we need tryRead in more places.

183a9c3d 10/16/2009 10:09 am Iustin Pop

Show the load on nodes in node lists

The strange printf usage is due to some limitation (it seems) in ghc for
very long argument lists. The whole printout should be rewritten later.

2180829f 10/15/2009 05:05 pm Iustin Pop

Add initial structure for utilisation balancing

This patch adds the datatypes and modifies the nodes and instance types to have
such attributes. They are not used yet in any way.

507fda3f 10/15/2009 05:00 pm Iustin Pop

Allow displaying the instance map in hbal

This is similar to --print-nodes, but with much fewer fields.

181d4e04 10/15/2009 11:49 am Iustin Pop

Add an explicit export list to Instance.hs

This exports all functions, but it's still good to have.

3a3c1eb4 10/15/2009 11:33 am Iustin Pop

More hlint fixes

This makes (for now) the code hlint-clean. This is per se not a huge
gain, but it allows easier tracking of regressions in style later
(one-two new violations are easier to diagnose when not hidden among 20
“known” ones).

c15f7183 10/14/2009 04:43 pm Iustin Pop

Style change: camel-casing of unittests

f5b553da 10/14/2009 04:41 pm Iustin Pop

Style change: cluster CStats camel-casing

This is again the cs_x to csX name change.

2060348b 10/14/2009 04:41 pm Iustin Pop

Style change: node and instance attributes

This changes from a_b to aB in all node and instance attributes, to
match the standard Haskell style. Also attributes that should have been
camel-cased but weren't were changed (e.g. plist → pList, pnode →

fca250e9 10/14/2009 01:45 pm Iustin Pop

Modify the internals of the detailed CV scores

Before we used a tuple; since we'll need more metrics in the future,
it's simpler to transform this into a list of doubles, whose elements
are handled homogeneously by all the code that needs them.

0df5a1b4 10/14/2009 11:56 am Iustin Pop

Add a command line option for executing jobs

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

9a2ff880 10/14/2009 11:56 am Iustin Pop

Add two specialized Luxi calls

This are higher level wrappers over the basic callMethod.

Signed-off-by: Iustin Pop <>

dfbbd43a 10/14/2009 11:56 am Iustin Pop

Change iMoveToJob to properly create migrates

The current Cluster.iMoveToJob always creates failovers, which is not
what we want. This simply used the original instances status to select
between these two (this is not optimal by the way, since the status...

924f9c16 10/14/2009 11:55 am Iustin Pop

Extend the MoveJob type to hold the instance index

This will be needed in order to generate the proper instance move commands.

Signed-off-by: Iustin Pop <>

13b6cb3f 10/14/2009 11:53 am Iustin Pop

Add a simple module for dealing with Ganeti jobs

This holds for now just the job status definitions and its serialisation
to/from JSON.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

66dac8e0 10/12/2009 03:13 pm Iustin Pop

Fix haddock issues with tuple members

It seems that haddock cannot document tuple members - but arguably, once
one needs to do that, tuples should not be used anymore.

This just moves the comments to the tuple comment.

Signed-off-by: Iustin Pop <>...

e97f211e 10/08/2009 01:06 pm Guido Trotter

parseNode: don't lookup values in drained nodes

Currently parseNode skips looking for values in offline nodes, but tries
to read them for drained ones. With this patch we treat offline and
drained nodes in the same way (which is compatible with the iallocator...

a2e90275 10/02/2009 06:54 pm Iustin Pop

Store the instance move in the MoveJobs

This will automatically sort our Ganeti jobs into the independent job
sets, and then we can submit them separately.

92e32d76 10/02/2009 06:48 pm Iustin Pop

Move some more type definitions to Types.hs

6b20875c 10/02/2009 06:37 pm Iustin Pop

Add a function converting Placements into Jobs

This converts from htools-specific Placements into Ganeti standard
OpCodes, which will later allow execution via Luxi.

702a4ee0 10/02/2009 06:35 pm Iustin Pop

Add a small implementation of OpCodes

These are just a few opcodes we need for executing instance moves.

3173c987 10/02/2009 05:52 pm Iustin Pop

Record the move being performed in a Placement

This will allow a more descriptive output later in the solution list, as
opposed to trying to reconstruct the move from the node indices.

The patch also documents the Placement members.

a1b5eeaf 10/02/2009 03:54 pm Iustin Pop

Add definitions for more Luxi calls

6583e677 10/02/2009 03:54 pm Iustin Pop

Split the Luxi generic parts from the loader

The Luxi loader implements both a generic Ganeti Luxi client and the
loader; it is better if these two are separated. The patch adds a
Ganeti/Luxi.hs (not under HTools!) since that is generic for Ganeti, and
not related necessarily to htools.

0e8ae201 10/02/2009 02:56 pm Iustin Pop

hbal: Implement grouping of moves into jobsets

Since moving two instances between different node-quadruples (inst X: A,
B → C, D and inst Y: E, F → G, H) can be parallelised by Ganeti, it
makes sense to split the operation list into jobsets whose execution...

1cf97474 09/30/2009 03:12 am Iustin Pop

Change ExtLoader to only handle I/O errors

Due to the Control.Exception changes between 6.8 and 6.10, using it
portably is difficult. Since we're only interested in handling I/O
errors, we can use prelude's catch and not have to deal with
Control.Exception at all....

cf924b6d 09/29/2009 04:43 pm Iustin Pop

Brown-paper-bag release fixing haddock issues

Haddock doesn't like pre-processed files (at least not in all versions).
Thus we need to remove the ExtLoader module from the haddock-procesed
file list.

685f5bc6 09/28/2009 05:09 pm Iustin Pop

Brown bag fix: invert a test

During testing I used the test inversely to see it triggers correctly,
and committed by mistake the inverted test. Fixing it.

fbb95f28 09/28/2009 05:09 pm Iustin Pop

Turn on, and fix, more warnings

The Makefile was intented to be -Wall and not simply -W, but I missed
that. This enables more warnings and also enables -Werror (except for
the tests).

45ab6a8d 09/28/2009 04:16 pm Iustin Pop

Add support for building without curl

Since curl is not always needed (e.g. when only using luxi or less
likely file backends only) and is also not always available, it is
useful for building without it. This of course disabled the RAPI

This patch changes ExtLoader to build with the ‘-cpp’ option which makes...

e8f89bb6 09/28/2009 03:50 pm Iustin Pop

Split the exernal data loader out of CLI.hs

Currently the external data loader is in CLI.hs, which makes all
programs that need cli functionality (options, etc.) link against the
network modules (most importantly curl). This patch splits this
functionality into a new module such that (for example) hail which only...

084b2502 09/03/2009 01:58 am Iustin Pop

Fix luxi recvMsg for messages bigger than 4K

This patch fixes a logic bug in luxi that breaks receive of messages
bigger than 4096 bytes. The send message is not impacted as it uses a
different algorithm.

cf35a869 09/01/2009 01:54 am Iustin Pop

Test some cases for the cluster score computation