Statistics
| Branch: | Tag: | Revision:

root / htools / Ganeti / HTools / QC.hs @ 084565ac

History | View | Annotate | Download (59.1 kB)

# Date Author Comment
7959cbb9 02/29/2012 04:17 pm Iustin Pop

Shorten some function names

Since we use the functions always module-qualified, let's drop the
`instance` prefix and use a shorter one.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

8bc34c7b 02/29/2012 04:16 pm Iustin Pop

htools: add spindle-related attribute to nodes

This adds the spindleCount, hiSpindles and instSpindles attributes.

The spindleCount is equivalent to spindle_count on the ganeti side (a
node parameter). hiSpindles is the maximum instance-used spindles, and...

c22d4dd4 02/29/2012 04:15 pm Iustin Pop

Add spindle ratio to htool's IPolicy

Just the usual data/type declarations, read/save in the text backend,
etc. Also does a bit of unit-test cleanup (the way we build
ipolicies).

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

3c1e4af0 01/26/2012 08:45 pm Iustin Pop

Fix integer overflow in Node unittest

When using a really random "extra cpus" variable, this can overflow
(especially on 32 bit) and thus the final value can be < 0, thus the
test will fail to fail.

It doesn't make sense to test with huge values, so let's just generate...

e8fa4ff6 01/26/2012 03:34 pm Iustin Pop

Add vcpu_ratio definition to the IPolicy type

This will be "naturally" read in the JSON-based backends (Luxi/Rapi),
and we add explicit code for handling it in the Text backend.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

487e1962 01/26/2012 03:34 pm Iustin Pop

Remove Node.mCpu, replaced with the ipol vcpu_ratio

Since the iPolicy contains the vcpu_ratio now, we can remove the
per-node mCpu value (which was the initial way to set this attribute).

Furthermore, we need to re-set the hiCpu value when changing the
policy....

37483aa5 01/26/2012 03:34 pm Iustin Pop

Expand the tiered alloc unittest to check allocation stats

This was missing before, so with the new metric (NCpus), it's a good
time to test this.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

468b828e 01/16/2012 02:50 pm Iustin Pop

Another small optimisation in test data generation

It seems that QuickCheck will happily generate huge data structures,
if that's what one requests. For this test, where we ask for [[Char]],
it could happen that we get a huge number of chars, so let's limit...

15e3d31c 01/16/2012 02:50 pm Iustin Pop

Generate tags using the helper in Node tests

Otherwise we test very big lists (slow) without any advantage in doing
so.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

45c4d54d 01/16/2012 02:50 pm Iustin Pop

Combine two Node unittests

These two tests are very similar, so let's run them together instead
of generating the data twice. Also improve the test a bit.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

a292b4e0 01/16/2012 02:49 pm Iustin Pop

Add unit-tests for --help/--version as common options

After reorganizing the way we process options, we can finally add a
test that all program personalities accept --help and --version and
that they exit early for these two.

This trivial patch has a higher than expected impact on the coverage,...

89298c04 01/16/2012 02:49 pm Iustin Pop

Add some very basic options test

Just tests that passing a string argument returns it correctly. We're
not interested in the flags themselves that much just that the
function itself parses some options at least correctly.

Signed-off-by: Iustin Pop <>...

a7ea861a 01/16/2012 02:49 pm Iustin Pop

Add test for CLI.parseYesNo

Also fix a tiny typo in the docstring.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

8b5a517a 01/16/2012 02:49 pm Iustin Pop

Fix bug in CLI.parseISpecString and add test

'zip' finishes on the shortest list, so parseISpecString accepted any
string of 3 values or more, which is not good. Fix this and add test
for both successful and failed parsing.

Signed-off-by: Iustin Pop <>...

e1dde6ad 01/16/2012 02:49 pm Iustin Pop

Add a unittest for the 'Simu' backend

This only test successful parsing, but for this case it tests
reasonably well the resulting cluster properties, I think (coverage
is >70% for expressions).

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

59ed268d 01/16/2012 02:49 pm Iustin Pop

Further optimise instance test data generation

Following on the lessons from this week (commit 7806125e), this patch
replaces generation of instances `suchThat` they are smaller than on
node with direct generation of instances with upper bounds on
memory/disk/cpu....

7806125e 01/13/2012 04:58 pm Iustin Pop

Fix very slow unit-test data generation in some cases

Learning, learning. It turns out that for objects with even just five
fields, generating them and filtering can be costly, especially when
the filtering is nested (as it was in this case, arbitrary1 →...

746b7aa6 01/13/2012 04:58 pm Iustin Pop

Fix one bad unit-test data generation case

It could happen that the node is generated with 0 disk available, in
which case the operation will fail due to disk issues, not CPU. Fix
this by using the recently-introduced genOnlineNode helper.

Signed-off-by: Iustin Pop <>...

aa1d552d 01/13/2012 03:24 pm Iustin Pop

Further JSON unit-tests for various Types.hs types

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

96bc2003 01/13/2012 03:24 pm Iustin Pop

Add a unit-test helper function

… instead of always using "printTestCase … False".

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

6a855aaa 01/13/2012 03:24 pm Iustin Pop

Extend node evacuation unittests

Currently they only check for change secondary, via iallocator's
relocate method. We add two separate unittests that checks all the
node evacuate modes (ChangeAll, ChangePrimary) and also the group
change request. All of these are done only for DRBD instances....

a2a0bcd8 01/13/2012 03:24 pm Iustin Pop

Fix a few issues in unittest data generation

This patch fixes two ugly but non-critical issues:

- the getFQDN wrongly generated only the first component as
limited-length name, the rest could be any length, and that makes
the generated name bad for display purposes...

3074ccaf 01/13/2012 03:24 pm Iustin Pop

Fix another data generation issue in tests

Another "good" seed value manages to generate nodes with identical
names, which of course makes the sets be of smaller value than the
lists. Avoid this issue by generating unique node names.

Signed-off-by: Iustin Pop <>...

d6f9f5bd 01/13/2012 03:24 pm Iustin Pop

Add a help function for generating test nodes

This abstracts the concept of "give an sane, online node" in a single
place, rather than spread around the tests. We also change the
signature for isNodeBig for easier usage.

Signed-off-by: Iustin Pop <>...

e73c5fe2 01/13/2012 03:24 pm Iustin Pop

Generate clusters with unique node names in tests

Nothing breaks otherwise in the cluster routines, but serialisation
to/from text gets confused otherwise.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

dce9bbb3 01/13/2012 03:24 pm Iustin Pop

Add a complex allocation/serialisation/load test

This is unfortunately a complex test that checks if:

- starting from a empty cluster
- allocating a few instances on it
- serialising it via the Text backend
- loading it back into internal data structures...

b37f4a76 01/13/2012 03:24 pm Iustin Pop

Implement saving/restoring of policies in Text backend

This adds a new optional (so we're backwards-compat) section to the
cluster data so that we can reliably save/restore the policy data.

Note that different from the other sections, we save all policies...

bcd17bf0 01/13/2012 03:24 pm Iustin Pop

Further unittests for ipolicy/ispec text serialisation

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

00b70680 01/13/2012 03:17 pm Iustin Pop

Unittests for the instance policy checks on allocation

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

d6eec019 01/13/2012 03:17 pm Iustin Pop

Set group policy on nodes

In order to be able to use the group policy during allocation, we have
a few choices:

- pass the group policy as a separate argument to all involved
functions, or
- add the policy as an attribute to nodes, or
- add the group (not only the group index) as an attribute to nodes...

6cff91f5 01/13/2012 03:17 pm Iustin Pop

Add ipolicy attribute to group objects

This is straightforward; as before, so far the text and simu backends
have no way to use a non-default ipolicy.

Furthermore, we do some unit-test preparation for future changes
(these seem a no-op for now, but they are needed for later patches)....

71375ef7 01/13/2012 03:16 pm Iustin Pop

Extend ClusterData with the cluster instance policy

This attribute is always initialised to the default, and is not (yet)
read/saved in the various backends.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

b69be409 01/13/2012 03:16 pm Iustin Pop

Stop exporting JSON functionality from Utils.hs

This completes the Utils/JSON split started in commit f047f90f. The
import graph should be cleaner now.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

f3f76ccc 01/13/2012 03:16 pm Iustin Pop

More reshuffling of code

Following the split Types/BasicTypes, we can remove the last
JSON-related stuff from Utils.hs, and do some more cleanup.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

ba1260ba 12/08/2011 12:03 pm Iustin Pop

Some small test simplifications

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

3603605a 12/08/2011 10:00 am Iustin Pop

Cleanup hlint errors

First, we update the recommended hlint version to what I used to get a
clean output (1.8.15). Most of the changes are:

- remove unneeded parentheses
- some simplifications (intercalate " " → unwords, maybe … id →
fromMaybe, etc.)
- removal of some duplicate code (in previous patches)...

b99d1638 11/17/2011 03:49 pm Iustin Pop

Fix unittest bug related to offline instances

Currently, the code in Node.hs is overly strict: once a node's free
memory reaches 0, it will refuse to add any instances (offline or
not). I think this is a safe safeguard (I don't expect nodes to run
without at least 1MB of free memory), so rather than change this...

d5dfae0a 11/17/2011 03:49 pm Iustin Pop

htools: re-indent QC.hs

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

e1bf27bb 11/16/2011 03:16 pm Agata Murawska

htools: Nicer arbitrary instantiation

Signed-off-by: Agata Murawska <>
Reviewed-by: Iustin Pop <>

61bbbed7 11/16/2011 03:16 pm Agata Murawska

htools: change behaviour for offline instances

Signed-off-by: Agata Murawska <>
Reviewed-by: Iustin Pop <>

8a8ed513 11/16/2011 03:16 pm Agata Murawska

htools: remove 'running' from instance options

Signed-off-by: Agata Murawska <>
Reviewed-by: Iustin Pop <>

7dd14211 11/16/2011 03:15 pm Agata Murawska

htools: change runSt type to InstanceStatus

Signed-off-by: Agata Murawska <>
Reviewed-by: Iustin Pop <>

5f828ce4 10/26/2011 05:35 pm Agata Murawska

Generalize the generation of ADT from raw types

Signed-off-by: Agata Murawska <>
Reviewed-by: Iustin Pop <>

72bb6b4e 10/03/2011 12:17 pm Iustin Pop

A few minor test improvements

This patch adds a few niceties to the test suite:

- allows matching test groups case insensitive and emit warnings when
we give test group names that don't match anything
- add a new operator that is similar to assertEqual in Python: it...

23fe06c2 10/03/2011 12:17 pm Iustin Pop

Use TemplateHaskell to decorate tests with names

This makes error message change from "Test 4 failed …" to "Test
prop_Loader_mergeData failed", which is much more readable. It also
removes the duplication of test suite names in the test.hs file.

Signed-off-by: Iustin Pop <>...

2c9336a4 10/03/2011 12:17 pm Iustin Pop

Rename some helper functions for consistency

This changes the names for some helper functions so that future
patches are touching less unrelated code. The change replaces
shortened prefixes with the full type name.

Signed-off-by: Iustin Pop <>...

129734d3 09/29/2011 10:14 am Iustin Pop

Change type of Cluster.AllocSolution

Originally, this data type was used both by instance allocation (1
result), and by instance relocation (many results, one per
instance). As such, the field 'asSolutions' was a list, and the
various code paths checked whether the length of the list matches the...

6804faa0 09/14/2011 05:42 pm Iustin Pop

htools: remove dead code

The tryEvac/evacuateInstance functions are no longer used in the new
multi-group world order, so we remove them and change the unit-test to
test the actual IAllocator function.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

bbe9758d 07/22/2011 01:53 pm Iustin Pop

htools: add target_node member to migrate opcode

… and failover too. Not many changes otherwise except for
serialisation and unittests.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

0047d4e2 07/21/2011 02:52 pm Iustin Pop

htools: add two more small unittests

This adds tests for the opToResult and eitherToResult functions from
Types.hs, and changes two other tests for the same module to test JSON
serialisation (which automatically also tests the lower-level to/from
string conversion functions)....

1b0a6356 07/21/2011 02:51 pm Iustin Pop

htools: a few more hlint fixes

Tested only on GHC 7.x, will test on 6.1x too before commit.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

33b9d92d 07/19/2011 05:54 pm Iustin Pop

htools: import the program modules in QC.hs

This adds the binaries code to the coverage, and thus the coverage
finally shows the real coverage over all logic code (except for the
htools.hs code, which is not logic code related to the algorithms, so
it doesn't matter — plus it's also very small)....

8f48f67d 07/15/2011 03:59 pm Iustin Pop

htools: add optional limit to allocation count

Currently, the tieredAlloc/iterateAlloc functions will not return
until the allocation fails; this means unit-testing their
functionality (e.g. that an instance can be allocated) is slow, since
they will allocate all possible instances....

9bb5721c 07/15/2011 03:32 pm Iustin Pop

htools: rework compCV so that we can use [Node] too

In some cases, we have a [Node.Node] list, instead of a Node.List
structure. Instead of converting [Node.Node] to the latter, let's
rework compCV and add a compCVNodes such that we skip this
(potentially expensive) step....

3158250d 07/14/2011 01:14 pm Iustin Pop

Fix htool unittest failure due to invalid prereqs

Currently, the test prop_Node_rMem does some reserved memory checks on
nodes, and for the test to work correctly (can add more instances) we
require that the node free memory is greater than zero via a
prerequisite....

1cb92fac 07/14/2011 12:05 pm Iustin Pop

htools: add parseUnit function

This is similar, but not identical, to Utils.ParseUnit. The biggest
difference is that we don't round up/down; as we only use integral
types, the result will always be rounded down.

Moreover, since (real-world) disk sizes come in SI units, the function...

22fac87d 07/13/2011 07:45 pm Guido Trotter

htools: simplify select tests

Rather than running all possible tests in one go, split them out in
three cases: "normal", undefined default value, undefined list values
(both of which require a valid response before them).

Signed-off-by: Guido Trotter <>...

efe98965 07/13/2011 04:52 pm Guido Trotter

htools: infrastructure for looking up names

Signed-off-by: Guido Trotter <>
Reviewed-by: Iustin Pop <>

bfe6c954 07/13/2011 04:52 pm Guido Trotter

htools: Add if' and select as functions

if' is just a functional version of the if-then-else construct. Using it
we can manipulate ifs as functions, and also build other functionality.

select allows to choose a result depending on truth value of statements,...

6429e8d8 07/13/2011 01:40 pm Iustin Pop

htools: read/save the disk template in Text backend

This requires that we change the signature of loadInst in order to
properly annotate the error messages, which in turn requires more
unittest changes.

Also, this invalidates yet again saved data files…...

d25643d1 07/13/2011 01:37 pm Iustin Pop

htools: add DiskTemplate to instance definition

Currently always fixed to DRBD8, as that is what we assume already.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

3c002a13 07/13/2011 01:37 pm Iustin Pop

htools: add some unittests for Types.hs

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

0e09422b 07/13/2011 01:14 pm Iustin Pop

htools: rename Instance.auto_balance

… to autoBalance, per the recommended style.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

525bfb36 07/13/2011 01:01 pm Iustin Pop

htools: docstring fixes and improvements

No code changes (except one definition being moved around in QC.hs)
are contained in this patch.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

752635d3 06/28/2011 06:37 pm Iustin Pop

htools: fix prop_Node_rMem corner case

This patch fixes a bug in the test specification where we allowed nodes
with zero free memory (hence no instance can be added, at all) and adds
a simple labeling of the way this test can fail.

Signed-off-by: Iustin Pop <>...

309e7c9a 06/28/2011 06:37 pm Iustin Pop

htools: fix corner case in prop_Text_Load_Instance

This unittest had a corner case where it could fail if the same
primary/secondary node names were generated.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

a1cd7c1e 06/28/2011 06:37 pm Iustin Pop

htools: improve more unittests

Using new functionality in QuickCheck 2 (the suchThat function), we
generate now better test cases, such that (heh) we have no longer
incomplete tests.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

00c75986 06/28/2011 06:37 pm Iustin Pop

htools: more fixes to unsatisfiable tests

Currently the way we generate nodes in some cases is by creating a
totally random node, then restricting the test based on whether the node
'size' (as defined by multiples of base unit) satifies some high/low
rules. This results in hard-to-satisfy conditions, so we change this...

41085bd3 06/28/2011 06:37 pm Iustin Pop

htools: Rework some unittests

The new scaffolding which replaced the batch driver of QuickCheck 1 now
shows how many passes we have for incomplete tests. Some tests show very
low pass counts, so we rework them to have more actually valid test
cases.

Signed-off-by: Iustin Pop <>...

8e4f6d56 06/28/2011 06:37 pm Iustin Pop

htools: Switch to QuickCheck 2.x

Since current distros don't package anymore QuickCheck 1.x, let's move
to 2.x.

This requires also a few changes to the code:

- Test.QuickCheck.Batch doesn't exist anymore, so we need to write some
scaffolding code to replace it...

cc532bdd 06/28/2011 06:35 pm Iustin Pop

htools: some lint fixes

Removal of duplicate parantheses, removal of extra 'do', conversion from
nested if to guards, use hierarchical imports. All per hlint.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

2d1708e0 06/17/2011 11:38 am Guido Trotter

Add instance selection list to Loader.mergeData

Signed-off-by: Guido Trotter <>
Reviewed-by: Iustin Pop <>

e08424a8 05/31/2011 07:38 pm Guido Trotter

Implement balancing with no instance moves

Note that --no-disk-moves and --no-instance-moves are not incompatible,
but if both are used no solution can possibly exist.

Signed-off-by: Guido Trotter <>
Reviewed-by: Iustin Pop <>

e82271f8 05/24/2011 06:22 pm Iustin Pop

Replace instance states hardcoded with constants

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

9cbc1edb 04/14/2011 11:35 am Iustin Pop

Add some tests for the auto_balance attribute

It tests node add/remove secondary, rather than cluster-level N+1
checks, but it's better than nothing.

Signed-off-by: Iustin Pop <>
Reviewed-by: Adeodato Simo <>

bc782180 04/14/2011 11:35 am Iustin Pop

Read/write auto_balance via Text

This also means another change in the text format; we really should
move to json…

The unittests are also update for the new 9-column layout and
additionally a bit of improvement is done.

Signed-off-by: Iustin Pop <>...

c352b0a9 04/12/2011 11:55 am Iustin Pop

Add a new attribute to Instance.Instance

This will mirror Ganeti's be/auto_balance one, which we need to use to
properly match N+1 computations.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

10028866 04/07/2011 01:56 pm René Nussbaumer

htools: Make opcode naming consistent with Ganeti codebase

This patch just cleans up the htools codebase to make it more consistent
with the naming of the Ganeti codebase.

Signed-off-by: René Nussbaumer <>
Reviewed-by: Iustin Pop <>

a810ad21 04/07/2011 12:36 pm Iustin Pop

htools: add an utility function for JSON parsing

This allows extracting values from a JSON object that might miss, but
have a well-defined default value.

Signed-off-by: Iustin Pop <>
Reviewed-by: René Nussbaumer <>

8d66f58a 04/06/2011 08:01 pm René Nussbaumer

htools: Use OpMigrateInstance with allow_failover option

Before hbal decided on the fly if an instance is migratable or not. As
we implemented failover fallback in commit d5cafd31456 we can start to
use that.

Signed-off-by: René Nussbaumer <>...

c478f837 03/23/2011 06:35 pm Iustin Pop

Another attempt at fixing htools build without curl

OK, my previous small fix was not good. There is another issue: haddoc
(the documentation generator) needs to pass the same compiler options
(i.e. in our case, -DNO_CURL) to ghc. But in case of no curl, then it...

2aa65bf0 03/23/2011 03:18 pm Iustin Pop

Fix Haskell unittests without RAPI

Since we don't test for now the RAPI backend directly, we can simply
skip the import. Later we can make a conditional import if needed.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michael Hanselmann <>

dc384cf0 03/17/2011 06:04 pm Iustin Pop

Merge remote branch 'htools/master'

  • htools/master: (605 commits)
    Update NEWS file for version 0.3.1
    Update copyright years in the man pages
    Remove obsolete env vars from the manpages
    Fix dist archive generation
    luxi backend: show attribute names in errors...