Statistics
| Branch: | Tag: | Revision:

root / htools / Ganeti / HTools / Cluster.hs @ bebe7a73

History | View | Annotate | Download (62 kB)

# Date Author Comment
bebe7a73 03/07/2012 11:24 pm Iustin Pop

Add a simple type alias for nicer type signatures

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

5f4464db 03/07/2012 11:05 pm Iustin Pop

Use the mirror type in Cluster.possibleMoves

Currently the function possibleMoves always presumes DtDrbd, which is
wrong for shared storage. To make it work with all disk templates, we
additionally pass the mirror type to it (we assume that the move
behaviour only depends on the mirror type, and not the individual...

255d140d 03/07/2012 11:05 pm Iustin Pop

Fix printSolutionLine to handle big scores

Currently we don't set a boundary on the length of the score field,
which breaks all our nicely-aligned output efforts.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

e85444d0 03/07/2012 11:05 pm Iustin Pop

Fix printSolutionLine to handle non-DRBD steps

Currently Cluster.printSolutionLine function always wants to print the
old and new secondary, and as such it fails (ugly) for moves of
non-DRBD instances. Change it so that it looks explicitly for
noSecondary nodes, and format the output different in that case....

73d12eab 03/07/2012 11:04 pm Iustin Pop

Small simplification in tryBalance

Instead of computing the bad/good node lists separately (in effect
iterating twice over the all_nodes list and also creating that list
twice from the container), let's use 'partition' and do a small
improvement.

Signed-off-by: Iustin Pop <>...

c3024b7e 02/29/2012 04:22 pm René Nussbaumer

htools: Adding printTable helper

This helper abstract away all the unlines and unwords call you've to do
after you got a formatTable back.

Signed-off-by: René Nussbaumer <>
Reviewed-by: Iustin Pop <>

084565ac 02/29/2012 04:18 pm Iustin Pop

Use the spindles metric in cluster scores

This makes balancing use the new metric.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

7959cbb9 02/29/2012 04:17 pm Iustin Pop

Shorten some function names

Since we use the functions always module-qualified, let's drop the
`instance` prefix and use a shorter one.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

2922d2c5 02/28/2012 02:26 pm René Nussbaumer

hinfo: Adding basic skeleton based on hbal

It prints the information which hbal prints in a nice formatted manner and of
course without doing any actions. An example output looks like this:

$ hinfo -m xen.example.com
Loaded 4 nodes, 2 instances
Cluster has 1 node group(s)...

bdd6931c 02/13/2012 07:17 pm Guido Trotter

htools: allow rbd disk template

Notes:
- it'd be nice if there was a way to automatically generate the
DiskTemplate list instead of manually specifying it. After all we
have C.diskTemplates
- Of course this actually does nothing, as for other...

33e17565 01/26/2012 03:34 pm Iustin Pop

Re-indent code for readability

Otherwise the docstrings for members take too many lines.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

90c2f1e8 01/26/2012 03:34 pm Iustin Pop

Track 'normalised' used CPUs in Cluster.CStats

This will be used in hspace for proper tracking of used/pool/unavail
npus.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

80d7d8a1 01/26/2012 03:34 pm Iustin Pop

Switch over some hspace stats from RSpec to AllocInfo

This patch changes the allocation delta stats from RSpec to
AllocInfo. There's nothing interesting about it, but it should allow
more correct computations of Npus in hspace.

I also moved the AllocStats type alias from Cluster.hs to Types.hs...

6a855aaa 01/13/2012 03:24 pm Iustin Pop

Extend node evacuation unittests

Currently they only check for change secondary, via iallocator's
relocate method. We add two separate unittests that checks all the
node evacuate modes (ChangeAll, ChangePrimary) and also the group
change request. All of these are done only for DRBD instances....

16670b57 01/13/2012 03:24 pm Iustin Pop

Remove some dead code

While wondering why Cluster.tryReloc is not unit-tested, I realised
that this is no longer uses. Apparently, last summer's new IAllocator
modes and related changes have silently deprecated this function.

Signed-off-by: Iustin Pop <>...

dce9bbb3 01/13/2012 03:24 pm Iustin Pop

Add a complex allocation/serialisation/load test

This is unfortunately a complex test that checks if:

- starting from a empty cluster
- allocating a few instances on it
- serialising it via the Text backend
- loading it back into internal data structures...

aa5b2f07 01/13/2012 03:17 pm Iustin Pop

Obey instance policy in allocation routines

This currently checks the minimum/maximum sizes and the disk
template. Unittests will come later.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

74c35f2f 01/13/2012 03:17 pm Iustin Pop

Trivial performance optimisation

This is not in the hot-path, but still, no sense in doing O(n) instead
of O(1).

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

3603605a 12/08/2011 10:00 am Iustin Pop

Cleanup hlint errors

First, we update the recommended hlint version to what I used to get a
clean output (1.8.15). Most of the changes are:

- remove unneeded parentheses
- some simplifications (intercalate " " → unwords, maybe … id →
fromMaybe, etc.)
- removal of some duplicate code (in previous patches)...

255f55a9 12/08/2011 10:00 am Iustin Pop

Abstract some common Cluster.hs code into a function

Signed-off-by: Iustin Pop <>
Reviewed-by: Agata Murawska <>

9fc18384 11/17/2011 03:49 pm Iustin Pop

htools: finish re-indenting Cluster.hs

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

f23f21c3 11/16/2011 03:59 pm Iustin Pop

Start converting the indentation to 2-chars

… also do some other small style cleanups.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

8a8ed513 11/16/2011 03:16 pm Agata Murawska

htools: remove 'running' from instance options

Signed-off-by: Agata Murawska <>
Reviewed-by: Iustin Pop <>

5f828ce4 10/26/2011 05:35 pm Agata Murawska

Generalize the generation of ADT from raw types

Signed-off-by: Agata Murawska <>
Reviewed-by: Iustin Pop <>

e86f7f65 10/14/2011 01:37 pm Iustin Pop

Further cleanup in hspace

This moves the checking of results from the allocation functions to a
separate function, so that we have less code duplication. It also does
a bit of simplification in the printing functions.

Signed-off-by: Iustin Pop <>...

7eda951b 10/14/2011 01:37 pm Iustin Pop

Add a type synonym for the allocation function sig

Both iterateAlloc and tieredAlloc share the same signature, but it's
not documented nor exported (needed for refactoring).

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

00976304 10/12/2011 02:51 pm Michael Hanselmann

Merge branch 'devel-2.5'

  • devel-2.5:
    rpc: Disable HTTP client pool and reduce memory consumption
    Preserve bridge MTU in KVM ifup script
    hail: Fix result for node evacuation
    Fix assertion error on unclean master shutdown

Signed-off-by: Michael Hanselmann <>...

5b0ac1a5 10/12/2011 02:35 pm Michael Hanselmann

Merge branch 'stable-2.5' into devel-2.5

  • stable-2.5:
    rpc: Disable HTTP client pool and reduce memory consumption
    hail: Fix result for node evacuation
    Fix assertion error on unclean master shutdown

Signed-off-by: Michael Hanselmann <>...

1ab94e48 10/07/2011 01:00 pm Michael Hanselmann

hail: Fix result for node evacuation

According to the iallocator documentation the “node-evacuate” call needs
to return a list of jobs, not a list of lists of jobs.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

2c9336a4 10/03/2011 12:17 pm Iustin Pop

Rename some helper functions for consistency

This changes the names for some helper functions so that future
patches are touching less unrelated code. The change replaces
shortened prefixes with the full type name.

Signed-off-by: Iustin Pop <>...

1bf6d813 09/29/2011 10:16 am Iustin Pop

Small simplification in tryAlloc

Signed-off-by: Iustin Pop <>
Reviewed-by: Agata Murawska <>

b0631f10 09/29/2011 10:16 am Iustin Pop

Change how node pairs are generated/used

Currently, the node pairs used for allocation are a simple [(primary,
secondary)] list of tuples, as this is how they were used before the
previous patch. However, for that patch, we use them separately per
primary node, and we have to unpack this list right after generation....

f828f4aa 09/29/2011 10:15 am Iustin Pop

Parallelise instance allocation/capacity computation

This patch finally enables parallelisation in instance placement.

My original try for enabling this didn't work well, but it took a
while (and liberal use of threadscope) to understand why. The attempt...

d7339c99 09/29/2011 10:15 am Iustin Pop

Abstract comparison of AllocElements

This is moved outside of the concatAllocs as it will be needed in
another place in the future.

Signed-off-by: Iustin Pop <>
Reviewed-by: Agata Murawska <>

129734d3 09/29/2011 10:14 am Iustin Pop

Change type of Cluster.AllocSolution

Originally, this data type was used both by instance allocation (1
result), and by instance relocation (many results, one per
instance). As such, the field 'asSolutions' was a list, and the
various code paths checked whether the length of the list matches the...

6804faa0 09/14/2011 05:42 pm Iustin Pop

htools: remove dead code

The tryEvac/evacuateInstance functions are no longer used in the new
multi-group world order, so we remove them and change the unit-test to
test the actual IAllocator function.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

7073b3a8 09/14/2011 05:42 pm Iustin Pop

hail: don't select the primary as new secondary

This just adds the primary node of the instance as 'non-allocable'
during the choosing of the new secondary.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

97da6b71 07/22/2011 02:33 pm Iustin Pop

htools: rework the algorithm for ChangeAll mode

I think I've identified the problem with the current ChangeAll
mode. The current algorithm works as follows:

- identify a new primary by choosing the node which gives best score
as new secondary
- failover to it...

77ecfa82 07/22/2011 01:53 pm Iustin Pop

htools: replace two hardcoded uses of pri+sec nodes

These two cases use explicit uses of primary and secondary nodes with
Instance.allNodes, which means the code is more flexible if the
internal layout of the instance changes.

I've verified that the output of involvedNodes is not required to be...

bbe9758d 07/22/2011 01:53 pm Iustin Pop

htools: add target_node member to migrate opcode

… and failover too. Not many changes otherwise except for
serialisation and unittests.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

1b0a6356 07/21/2011 02:51 pm Iustin Pop

htools: a few more hlint fixes

Tested only on GHC 7.x, will test on 6.1x too before commit.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

179c0828 07/21/2011 02:50 pm Iustin Pop

htools: further docstring fixes

This adds parameter documentation for Cluster.iMoveToJob (I think it
was not clear if the new or old node list is needed) and fixes other
docstring style issues.

After this patch, all modules except for CLI.hs (which has many...

756df409 07/19/2011 05:54 pm Iustin Pop

htools: move tiered spec map helper to Hspace.hs

This is used just in hspace, so let's help in making Cluster.hs
smaller. We also split the function in two, as computing the spec map
and formatting it are two different tasks.

Signed-off-by: Iustin Pop <>...

1adec4be 07/19/2011 05:12 pm Iustin Pop

htools: add a compatibility module

When compiling with the parallel-3.x library, we get a deprecation
warning, which makes understanding any other error messages
harder. This patch adds a compatibility module that will hold such
code for transitioning libraries....

a5f248ac 07/19/2011 05:12 pm Iustin Pop

htools: remove no-longer-needed tryMG* functionality

… which was deprecated by the previous patch.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

a86fbf36 07/19/2011 05:12 pm Iustin Pop

htools: fix potential bug in ialloc/change-group

Currently, the ChangeAll mode of nodeEvac computes the primary group
of the instance and then uses the resulting group index for computing
the group score. However, during the change-group operation (which...

4036f63a 07/15/2011 06:56 pm Iustin Pop

htools: return new state from new IAllocator modes

The old modes already return the node list (as part of AllocSolution),
this patch makes the new modes provide this new information.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

5440c877 07/15/2011 06:56 pm Iustin Pop

htools: update the node-evac/change-group result

… per the new design changes. This also requires changes to
updateEvacSolution, as passing in the original instance is not
helpful; we just pass the instance index now and look up the actual
instance in the modified instance map....

20b376ff 07/15/2011 04:25 pm Iustin Pop

htools: add change-group IAllocator mode

This builds a lot upon the node-evacuation mode, and its
implementation is therefore quite easy (once node-evac was added).

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

d72ff6c3 07/15/2011 04:24 pm Iustin Pop

htools: add group restrictions to findBestAllocGroup

While a normal multi-group allocation doesn't need to restrict the
list of target groups beyond the unallocable status of some groups,
when we relocate instances from one group to the other we need to
limit the subset of searched groups....

c9a9b853 07/15/2011 04:13 pm Iustin Pop

htools: generalise and rename availableLocalNodes

Currently this function works in the context of an instance, computing
its local group nodes. We can use generalise it to take a group index
argument, so that it's not bound to an instance, and we rename it...

8fd09137 07/15/2011 04:10 pm Iustin Pop

htools: split tryMGAlloc in two

Currently, this function does both identify the best group for an
instance, and format the results (partially). The former functionality
is useful in other contexts too, so let's split the function in two.

Signed-off-by: Iustin Pop <>...

d52d41de 07/15/2011 04:10 pm Iustin Pop

htools: add node-evacuation of DRBD all nodes

This is much more complicated than either change primary (which is
trivial) and change secondary (which is also simple).

The main strategy is:

- choose a new secondary (that will become the final primary) and...

b4bae394 07/15/2011 04:00 pm Iustin Pop

htools: small optimisation in Cluster.collapseFailures

This makes fewer temporary data structures, and thus brings a bit of
improvement in total runtime.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

8f48f67d 07/15/2011 03:59 pm Iustin Pop

htools: add optional limit to allocation count

Currently, the tieredAlloc/iterateAlloc functions will not return
until the allocation fails; this means unit-testing their
functionality (e.g. that an instance can be allocated) is slow, since
they will allocate all possible instances....

db56cfc4 07/15/2011 03:33 pm Iustin Pop

htools: add node-evacuation of DRBD secondaries

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

9bb5721c 07/15/2011 03:32 pm Iustin Pop

htools: rework compCV so that we can use [Node] too

In some cases, we have a [Node.Node] list, instead of a Node.List
structure. Instead of converting [Node.Node] to the latter, let's
rework compCV and add a compCVNodes such that we skip this
(potentially expensive) step....

bef83fd1 07/14/2011 12:05 pm Iustin Pop

htools: add node-evacuate of DRBD8 primaries

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

47eed3f4 07/14/2011 12:05 pm Iustin Pop

htools: add more IAlloc data types and functions

These move the 'not supported' message deeper into the workflow;
assuming that the workflow as added by this patch is correct, all that
remains is:

- for disk template DRBD8, plug in the currently existing mechanisms...

63a78055 07/14/2011 12:05 pm Iustin Pop

htools: add a function for computing evacuated nodes

The new IAllocator interface is based on instances, not nodes, so we
need to backtrack and compute on which nodes we can't allocate
instances during the current operation. This patch adds a function for...

97936d51 07/14/2011 12:05 pm Iustin Pop

htools: rename an internal constant

'emptySolution' is fine while we have only one Solution type
(AllocSolution), but in the future we'll introduce another one and
thus we should rename it.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

1f4ae205 07/14/2011 12:05 pm Iustin Pop

htools: abstract and simplify a multigroup function

In the future, this 'split/associated instance idx list' functionality
will be used by other code, so let's move it to its own code.

The new method of computing the split also requires exporting one more...

0e09422b 07/13/2011 01:14 pm Iustin Pop

htools: rename Instance.auto_balance

… to autoBalance, per the recommended style.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

525bfb36 07/13/2011 01:01 pm Iustin Pop

htools: docstring fixes and improvements

No code changes (except one definition being moved around in QC.hs)
are contained in this patch.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

7034694d 07/01/2011 06:06 pm Iustin Pop

One Haskell and integer sizes fix

Haskell has two main integer types:

- Int, which is a native-type, and is guaranteed to have at least
[-2²⁹, 2²⁹-1] range; on 64-bit platforms, it has much higher range
- Integer, which is a software type (implemented using libgmp), and...

88253d03 06/28/2011 06:35 pm Iustin Pop

htools: rewrite Cluster.filterMGResults

filterMGResults was built using a sequence of map and filter calls;
while this was logically correct, it used some incomplete pattern
matching which with the new GHC 7 triggers a warning.

The patch rewrites it using a single foldl that does both the filtering...

e4491427 06/28/2011 06:35 pm Iustin Pop

htools: use the new Group.isAllocable

… instead of the hardcoded test against AllocUnallocable.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

e08424a8 05/31/2011 07:38 pm Guido Trotter

Implement balancing with no instance moves

Note that --no-disk-moves and --no-instance-moves are not incompatible,
but if both are used no solution can possibly exist.

Signed-off-by: Guido Trotter <>
Reviewed-by: Iustin Pop <>

93439b1c 04/14/2011 11:33 am Iustin Pop

Show the auto_balance flag in the instance listing

Signed-off-by: Iustin Pop <>
Reviewed-by: Adeodato Simo <>

10028866 04/07/2011 01:56 pm René Nussbaumer

htools: Make opcode naming consistent with Ganeti codebase

This patch just cleans up the htools codebase to make it more consistent
with the naming of the Ganeti codebase.

Signed-off-by: René Nussbaumer <>
Reviewed-by: Iustin Pop <>

8d66f58a 04/06/2011 08:01 pm René Nussbaumer

htools: Use OpMigrateInstance with allow_failover option

Before hbal decided on the fly if an instance is migratable or not. As
we implemented failover fallback in commit d5cafd31456 we can start to
use that.

Signed-off-by: René Nussbaumer <>...

2a8e2dc9 03/23/2011 06:35 pm Iustin Pop

Fix some lint warnings in htools code

hlint gives more suggestions, but some make the code (IMHO) harder to
read.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

d5072e4c 03/23/2011 06:35 pm Iustin Pop

Fix lint errors in the htools code

These are just changes from hlint suggestions. Still compiles and
passes unittests.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

dc384cf0 03/17/2011 06:04 pm Iustin Pop

Merge remote branch 'htools/master'

  • htools/master: (605 commits)
    Update NEWS file for version 0.3.1
    Update copyright years in the man pages
    Remove obsolete env vars from the manpages
    Fix dist archive generation
    luxi backend: show attribute names in errors...