Synnefo » snf-ganeti

Luxi: read the node uuid from the cluster

This makes the code incompatible with Ganeti pre-2.4.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

Utils: add a default UUID

This will be used as a placeholder for the cases when we need a UUID
(any UUID), but we don't have one handy.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

Improve the standard deviation computation

This does just two passes, instead of three, over the list. This reduces
the overall runtime well enough (~25%) in some tests, but it's not
reproducible using profiling, so I don't know how much the function
itself is being sped-up....

Luxi loader: split parsing from loading

Text loader: split parsing from loadData

This change, which will be followed by similar changes in the other
loaders, splits the parsing of the data from the actual loading from
disk. Since the parsing doesn't usually involve IO actions, we will be
able to better test the parsing. The loading becomes a smaller part of...

Fix tag exclusion weight

Currently, the tag exclusion metric has a weight of one, which means
there might be cases where we won't move instances around because it
upsets the cluster metrics. However, we do want to make a higher effort
for cleaning up tag collisions, so we increase the weight to an...

Improve the error message for tiered alloc option

Add new CLI options for min gain during balancing

Recent hbal seems to run many steps for small improvements (< 1e-3), so
we should stop early in this case.

We add a new option (-g), that will be used for the minimum gain during
balancing. This check will only become active when the cluster score is...

Fix ReplaceSecondary moves for offline nodes

The addition of a new secondary on a node is doing two memory tests:
- in strict mode, reject if we get into N+1 failure
- reject if the new instance memory is greater than the free memory (not
available memory) on the node...

Abstract the cluster serialization from hscan.hs

This is currently hardcoded in an internal function in hscan.hs, and we
move it to Text.hs for later use.

Add unittest for Node text serialization

This checks that the Node text serialization and deserialization
operations are idempotent when combined other.

Move text serialization functions to Text.hs

Currently these are in hscan, and cannot be reused easily.

Fix another haddock issue

Add some more imports to QC.hs

This is needed so that in the coverage report we list all modules, even
the ones we don't test at all, such that we get the complete results.

Introduce per-metric weights

Currently all metrics have the same weight (we just sum them together).
However, for the hard constraints (N+1 failures, offline nodes, etc.)
we should handle the metrics differently based on their meaning. For
example, an instance living on a primary offline node is worse than an...

Introduce a relaxed add instance mode

In case an instance is living on an offline node, it doesn't make sense
to refuse moving it because that would create N+1 failures; failing N+1
is still much better than not running at all. Similarly, if the
secondary node of an instance is offline, meaning the instance doesn't...

hbal: print short names in steps list

This was a regression from the name handling changes, as we started
using the original names for the solution list (which is not designed
for parsing/feeding back into ganeti).

Allow '+' in node list fields

When the field list is prefixed with a plus sign, this will extend the
default field list, instead of replacing it entirely.

Cleanup a node's peer map when possible

If the last secondary instance of a peer is deleted (detected by the new
peer memory value being equal to zero), then the pair (pdx, 0) should be
deleted completely. This is not optimization per se, but rather cleanup...

Remove JOB_STATUS_GONE and add unittests

… for the serialization/deserialization of the job and opcode status.

Job status 'gone' was not actually used. It can be reintroduced if
needed.

Change the Luxi operations structure

Currently, we define the LuxiOp type as a simple enumeration, and leave
the arguments structure to the users of the Ganeti.Luxi module. This is
suboptimal for a couple of reasons: first, we decouple the operation
type from operation arguments, and that means we don't use the type...

Add a few Loader tests

These are not comprehensive, but at least we have a start.

Introduce OpCode unittests

Replace fromJResult with annotateJResult

This patch removes all old uses of fromJResult with the annotated
version, and removes the non-annotated version. All JSON parsing points
should now have annotated errors.

Change fromObj error messages

Currently fromObj doesn't detail what we're trying to read, which can
lead to cryptic messages: "Cannot read Int". The patch changes this
function to annotate the error messages with the key/value we're trying
to convert, by using a new version of fromJResult....

Add more unittests

Instance, Node and Text modules have improved coverage.

Move two functions from hspace to Cluster.hs

This is done so we can test a longer pipeline.

Another haddoc fix…

Stop modifying names for internal computations

Currently the name used internally is modified and holds the shortened
name of the nodes/instances. This has caused issues before, since we
always have to strip the suffix from input data and reapply it if we...

Change some test constants

First, we reduce the max size of the disks, since Int on 32bits will
overflow for big simulated clusters. This is a real issue, that will
need fixing in real life, but for now we just "silence" this test.

Second, we increase the amount of time a test is allowed to run,...

Add more unit tests

This increases the overall coverage by 5%-10% (depending on coverage
type). Some modules are still not unittested at all, as HUnit is a
better choice for them.

Remove the noLimit values and always use limits

This patch moves from allowing no-limits for disk/cpu ratios, and always
use a real limit. For disk, it's simple since we use 0, which means no
reservations for disks. For CPU, we set an (arbitrary) limit of 64 v/p,...

Fix Node hiCpu computation

In case we're not enabling limits, let's restrict this to -1, instead of
-1 times the number of pcpus.

Introduce total vcpu tracking in CStats

We add a new field that tracks the available virtual cpus (expressed as
node cpus times the vcpu ratio).

Fix iallocator crash when no solutions exist

Commit 5436576 added an un-guarded `head' call, which crashes with
“Prelude.head: empty list” when no results exists for the per-instance
allocation/relocation calls.

This patch fixes this, and also adds another check for an unguarded...

Fix iallocator crash when no solutions exist

Commit 5436576 added an un-guarded `head' call, which crashes with
“Prelude.head: empty list” when no results exists for the per-instance
allocation/relocation calls.

This patch fixes this, and also adds another check for an unguarded...

Abstract instance running states into a list

This removes some manual checks from a few places in the code with a
single list defined once.

Fix unused-do-binds for ghc 6.12

GHC 6.12 has some new warnings, which are valid in most cases except
(IMHO) printf usage.

hscan: implement LUXI backend scanning

This allows hscan to work also with NO_CURL (but only for the local
machine, of course).

balance function: use the movable flag directly

Instead of deciding based on secondary node, use the new flag.

Add a 'movable' flag on instances

This will be used instead of checking for no secondary and for
simplifying 'do not touch' instances.

Implement IAllocator node evacuate request

This patch adds the new request loading/execution (trivial), but the
actual response formatting becomes more difficult as now the response
type differs by request.

Signed-off-by: Iustin Pop <iustin@google.com>

Move a type declaration to Node.hs

We'll need AllocElement in both Cluster and IAlloc in the future, so we
move it to Node.hs which is imported by both.

Signed-off-by: Iustin Pop <iustin@google.com>

IAllocator: move some keys into per-request data

Since not all structures will have these keys in the future, we move
them into per-structure keys.

Signed-off-by: Iustin Pop <iustin@google.com>

Add an evac mode CLI option

Signed-off-by: Iustin Pop <iustin@google.com>

Fix secondary node selection for existing N+1

In case a secondary node is already N+1 failed, currently the node
selection will accept a node that cannot start (at all) the new instance
as valid. This is wrong, so we add a new simple check to prevent the...

Move instance relocation test upper in the chain

Currently we test each instance for relocation in checkMove; however, it
is a little more clear if we pass only the relocatable instances to
checkMove. The patch also slightly rewrites (indendation/style) the...

Fixing a typo in option description

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>

Change the signatures of the text loader slightly

This is in preparation for the text format changes.

Merge branch 'master' into next

• master:
  Use the oper_ram field if available
  rapi, luxi: treat drained nodes as offline