Synnefo » snf-ganeti

Split the balancing function in two parts

Currently in the balancing function we do two thing:

- take the decision where to do a new balancing round or not
- and actually computing the balancing round

This is not nice, as the two parts are conceptually separate, so this...

Metric: count of primary instances/offline nodes

This helps with evacuation/failover of instances on 2-node clusters with
one one offline.

Use conflicting primaries count in cluster score

This small patch adds the number of conflicting primaries in the cluster
score. This is different from the other non-CV metrics where we usually
compute the percentage of failing instances (for that metric); but for a...

Move more node-listing functionality in Node.hs

This will prepare for the runtime-selectable field list.

Expand the --print-instances output

This adds run status, resource parameters and load parameters for
instances.

Simplify Cluster.computeMoves

Since we now have an actual type for describing the instance moves
(IMove), it's simpler to convert this into the move description/move
commands, rather than re-computing the move based on initial and final
nodes. This makes the shell commands computation and over-Luxi command...

Generalise the node/instance listing

This patch introduces a generic formatTable function (based on, and
similar to the Ganeti one, but different and more FP in style) and
changes the node and instance listing to it.

The node list (due to the many variables) is still a little bit hackish...

Start using the utilisation scores in balancing

This enables the per-node load/total available capacity scores to be
used in balancing. Note that the total available capacity is currently
fixed at zero and cannot be changed by the user.

Allow displaying the instance map in hbal

This is similar to --print-nodes, but with much fewer fields.

Style change: node and instance attributes

This changes from a_b to aB in all node and instance attributes, to
match the standard Haskell style. Also attributes that should have been
camel-cased but weren't were changed (e.g. plist → pList, pnode →
pNode).

Change iMoveToJob to properly create migrates

The current Cluster.iMoveToJob always creates failovers, which is not
what we want. This simply used the original instances status to select
between these two (this is not optimal by the way, since the status...

Store the instance move in the MoveJobs

This will automatically sort our Ganeti jobs into the independent job
sets, and then we can submit them separately.

Add a function converting Placements into Jobs

This converts from htools-specific Placements into Ganeti standard
OpCodes, which will later allow execution via Luxi.

hbal: Implement grouping of moves into jobsets

Since moving two instances between different node-quadruples (inst X: A,
B → C, D and inst Y: E, F → G, H) can be parallelised by Ganeti, it
makes sense to split the operation list into jobsets whose execution...

Split the balancing algorithm in two parts

Currently the computation, recursing part and the IO part (progress
updates) of the balancing main function (iterateDepth) are all in the
same function, which makes it hard to test. This patch moves the
decision/computation part (whether to proceed one more round, whether we...

Use migrate or failover based on instance state

While we can't guarantee that the instance will be in the same state by
the time the migrate/failover command will be run, we can at least try
to do the right thing assuming no other changes to the cluster state....

Fix a haddoc issue

Change the tryAlloc/tryReloc workflow

Currently, the tryAlloc and tryReloc function return a list with all the
results, both failures and successes. This is fine for hail, which does
one round of allocations, but is not so good for hspace, which does
iterative rounds; since at each (successful) step we only take the best...

Slight change to the internal allocation results

Currently the Cluster.AllocSolution type is defined as a list of
‘(OpResult Node.list, …)’ and the results for applyMove are defined as
‘(OpResult Node.List, …)’. Both these means that the failure/success
indication is hidden in the first elements of this tuple, which makes is...

Export more stats in hspace

This patch changes Cluster.totalResources to compute more resources and
prints them in hspace.

Simplify Cluster.computeMoves

This patch changes the function Cluster.computeMoves to use guards and a
couple of subexpressions in order to greatly simplify it.

Introduce a new type for allocation results

Currently the allocation/move operations workflow return ‘Maybe a’,
which is very convenient but loses all details about the failure mode.

This patch introduces a new data type which encodes the specific failure...

Fix totalResources avail disk computation

This uses the newly-added Node.availDisk to compute the actual available
disk correctl, and display the total allocatable disk in hspace.

Add display of more stats in hspace

This patch changes Cluster.totalResources to compute more details about
the cluster status, and enhances hspace to display more of these.

Fix the various monomorphism warning

In a few places (e.g. tryRead or any printf call) it's a little bit hard
to add the correct type signatures, but in the it is possible to fix
these warnings (which can bite one in subtle cases).

Add cpu ratio to cluster calculation

Fix the ReplacePrimary instance move

During a replace-primary instance move, on the real cluster the instance
is temporarily started on the secondary, and as such we must check that
the secondary node can hold it for this duration. Currently the code
does not, and depending on cluster scoring it will put instances on such...

Add copyright/license information

This doc-patch adds copyright and license information to (hopefully) all
needed files.

Move some alloc functions from hail into Cluster

These are generic enough to be used from multiple places, they belong
better in Cluster.hs than in the hail source.

Lots of documentation updates

This patch does only doc build changes, doc changes and function move
around (for more logical documentation). It should have no impact at all
on the code.

Add type synonyms for the node/instance indices

This is a first step towards full datatype renaming. That requires more
changes, so at first we only want to document clearly what is a node
index, what is an instance index, and what is a plain Int.

hail: Implement non-mirrored instance allocation

This patch implements non-mirrored instance allocation, by allocating as
secondary node “noSecondary”.

Working implementation if relocate

This patch completes the implementation of hail relocate. It maps all
valid destination nodes through a ReplaceSecondary IMove, filters out
the failed relocations, computes the resulting scores and picks the
lowest one.

Remove some extraneous uses of ktn/kti

Since we have Node/Instance.name, we can now simplify a few constructs.

More code reorganizations

This new big patch does a couple of more cleanups in the loading of data
chapter:
- introduce a Types module that holds most types (except the base
Node/Instance/etc.) so that multiple other modules can use these
(instead of only Cluster and its users)...

Experimental support for non-redundant instances

This patch adds experimental support to hbal for non-redundant instances
(i.e. instances with only one node). They are currently handled as
non-moveable, and as such the algorithm simply ignores them.

Supports needs to be added when reading from RAPI via hscan, and...

Introduce nice errors on invalid input fields

This patch switches from plain read to a wrapper over readsPrec that
returns better error messages than the buildin 'Prelude: no parse'.

Add initial validation checks in Cluster.loadData

This patch converts loadTabular and loadData to a monadic form, thus
allowing meaningful error messages from the node/instance load routines.

Don't consider offline nodes as N+1 failed

This is just a cosmetic (I hope) change; the nodes shouldn't be used
anyway, and we only correct the display message.

hbal: Add a new min-score option

This new parameter causes the algorithm to finish (or even not start at
all) if we reach/have a score better than it.

Increase allowed missing memory to 512MB

Since Xen seems to “steal” some amounts of memory (depending on total
node memory), we increase the maximum allowed missing memory to 512MB,
based on gathered data from multiple machines.

Add checks for missing disk space

This small patch adds disk space checks to the Cluster.checkData
function, and simplifies a little the warning messages.

Add a new instance field denoting run status

This patch modifies Rapi, the Cluster.loadData and hscan serialization to load
and save the instance run status. At instance level, we add both a boolean
field denoting the true/false run status, and a string field which holds the...

Add functions to check and fix cluster data

This patch adds a checkData function which goes over the node list and computes
the unaccounted memory, returning a list of warning messages (if any) and the
update nodes.

Pass actual types to node/instance constructors

This patch changes the parameters passed to the node and instance
constructors from generic Strings (which are then parsed via “read”) to
the actual used types, by converting them earlier in Cluster.loadData.

Add a separate type for the [(Int, String)] list

This is added for better readability, since this is very often used in
declarations.

Show offline nodes in the node status list

This patch adds a new ‘-’ flag for the node status which denotes offline
nodes.

Some updates to the apidoc rules