Synnefo » snf-ganeti » ganeti-local

Add a function converting Placements into Jobs

This converts from htools-specific Placements into Ganeti standard
OpCodes, which will later allow execution via Luxi.

Split the Luxi generic parts from the loader

The Luxi loader implements both a generic Ganeti Luxi client and the
loader; it is better if these two are separated. The patch adds a
Ganeti/Luxi.hs (not under HTools!) since that is generic for Ganeti, and
not related necessarily to htools.

Change ExtLoader to only handle I/O errors

Due to the Control.Exception changes between 6.8 and 6.10, using it
portably is difficult. Since we're only interested in handling I/O
errors, we can use prelude's catch and not have to deal with
Control.Exception at all....

Brown bag fix: invert a test

During testing I used the test inversely to see it triggers correctly,
and committed by mistake the inverted test. Fixing it.

Add support for building without curl

Since curl is not always needed (e.g. when only using luxi or less
likely file backends only) and is also not always available, it is
useful for building without it. This of course disabled the RAPI
backend.

This patch changes ExtLoader to build with the ‘-cpp’ option which makes...

Fix luxi recvMsg for messages bigger than 4K

This patch fixes a logic bug in luxi that breaks receive of messages
bigger than 4096 bytes. The send message is not impacted as it uses a
different algorithm.

Test some cases for the cluster score computation

Implement support for 'cheap' moves only

This patch adds support for cheap (failover/migrate) operations only in
the balancing algorithm and in the hbal command line options.

This allows a very quick balancing (compared to allowing replace-disks)
which can be useful as a scheduled operation.

Use migrate or failover based on instance state

While we can't guarantee that the instance will be in the same state by
the time the migrate/failover command will be run, we can at least try
to do the right thing assuming no other changes to the cluster state....

Add a simulated cluster data loader

This is useful especially for hspace, where we might want to simulate a
hypothetical cluster to check allocation beforehand.

Unify the command line options and structures

This patch moves all the command line options and their internal
representation into CLI.hs. This means that duplicated options between
any two binaries are no longer declared twice, and that we no longer
need the two *Option classes.

CLI: Prevent incompatible options to be selected

This patch makes CLI abort if more than one backend is selected.

Initial commit of the luxi backend

This patch adds a luxi backend that allows direct query of the master
daemon on the local node. This patch doesn't enable the backend to be
used.

There are a couple of things still missing in the implementation:
- we don't have a master timeout in reads and writes, only a...

Fix a haddoc issue

Change the tryAlloc/tryReloc workflow

Currently, the tryAlloc and tryReloc function return a list with all the
results, both failures and successes. This is fine for hail, which does
one round of allocations, but is not so good for hspace, which does
iterative rounds; since at each (successful) step we only take the best...

Slight change to the internal allocation results

Currently the Cluster.AllocSolution type is defined as a list of
‘(OpResult Node.list, …)’ and the results for applyMove are defined as
‘(OpResult Node.List, …)’. Both these means that the failure/success
indication is hidden in the first elements of this tuple, which makes is...

hspace: move instance count and score into CStats

Currently the instance count and cluster score are separated from the
other initial/final phase stats, even though they are very similar. This
patch moves computation of these two into totalResources/CStats and...

Export more stats in hspace

This patch changes Cluster.totalResources to compute more resources and
prints them in hspace.

Fix score calculation to work with empty clusters

Currently the cluster score calculation includes an offline instance
percentage, expressed as “offline inst / (offline + online inst)”, which
results in NaN for empty clusters. This patch changes the calculation...

Take the foldl out of Loader.fixNodes

Currently Loader.fixNodes is foldl' with a complicated function. It
makes more sense to take foldl' out of this function (and put it into
the caller) and let fixNodes be only this internal function.

Fix hlint-generated warnings

This big patch cleans up the code per hlint indications. Many removals
of extra parentheses, replacements of concat . map with concabtMap,
extra dollar signs, eta reductions, etc. were performed.

The code still compiles and passes a couple of manual tests on sample...

Return correct failure data from Node.add*

This patch alters the Node.addPri/addSec to return correct failure data.
It removes the computeFailN1 function from the module as that used to
combine both mem and disk checks in the same function and thus the real...

Remove hn1 and related code

hn1 was deprecated for a while and this patch removes it altogether. The
support code in Cluster.hs is also removed.

Add an availDisk node function

This function returns the amount of available disk, which depends on
whether a low disk limit has been configured or not and on the free disk
space of the node.

Add a new type for cluster statistics

Currently totalResources returns a 5-tuple of integers. This is not easy
to handle, as each change on the return type means that each caller must
be updated.

This patch adds a new type for cluster stats and uses that instead as...

Fix a haddock/docstring issue

Add two new node attributes

Two new min disk free ratio and max cpu usage attributes are added to the
nodes. These will be used in the future to restrict allocation.

Fix the various monomorphism warning

In a few places (e.g. tryRead or any printf call) it's a little bit hard
to add the correct type signatures, but in the it is possible to fix
these warnings (which can bite one in subtle cases).

Add cpu ratio to cluster calculation

Add cpu-count-related attributes to nodes

This patch adds cpu-count related attributes to nodes:
- total cpus
- cpus in use
- ratio of virtual:physical cpus

We also set correctly the cpu values at load time, but we don't do
anything yet while moving instances around. The cpu ratio is shown in...

Fix reading of total disk space in iallocator

IAllocator currently uses a wrong key name for reading the total disk
space (‘disk_usage’ which was copied from RAPI, but the actual
iallocator key is ‘disk_space_total’).

This patch fixes that and also makes iallocator always use this key,...

Rework the tryAlloc/tryReloc functions

Currently tryAlloc/tryReloc do not return the new instance, as this is
not needed for IAllocator alloc/reloc requests. However, for computing
the space, the new instance is useful, so we modify these functions to
return this information too....

Small doc change

And an alignment issue.

Add copyright/license information

This doc-patch adds copyright and license information to (hopefully) all
needed files.

Move some alloc functions from hail into Cluster

These are generic enough to be used from multiple places, they belong
better in Cluster.hs than in the hail source.

Move the RqType and Request types to Loader.hs

These two will be more generic than now, and belong somewhere else -
Loader.hs is a generic module for data loading, thus we move them there.

Lots of documentation updates

This patch does only doc build changes, doc changes and function move
around (for more logical documentation). It should have no impact at all
on the code.

Add some very trivial Instance tests

This is more of an exercise in QuickCheck than strong testing.

Add test infrastructure and initial tests

This patch adds a QuickCheck-based test infrastructure and initial tests
based on it. The PeerMap module has a 100% coverage ☺

Side-note: one has to read the source of QuickCheck to see how to use it
(especially the Batch submodule), the docs are not enough…

Remove unused parameters from PeerMap creation

We remove some unused arguments (added way back for compatibility with
Arrays, which we didn't use in the end). This makes the code clearer
(and doesn't need the Ndx type to be an instance of Num).

Add a separate function for looking up instances

Currently we (wrongly) use lookupNode to lookup instances, just because
the name assoc list has the same type. This patch adds a separate
function for it.

Change the module import hierarchy

This patch makes the Types module a base module, and Node/Instance ones
import it, from the previous (opposite) situation. This will allow in
the future to use newtypes for the index and name types.

Fix loading of plain instances via iallocator

Currently iallocator is broken when reading single-node instances (and
with an ugly error message). This patch fixes this case, by marking them
with secondary node “noSecondary” like the rest of the code.

hail: do not allocate on offline/drained nodes

This patch implements filtering out of the offline/drained nodes and
fixes a bug in IAllocator.hs parsing (similar to an older bug in Rapi.hs
from where the code was copied).

Implement hail allocate (for 2-node requests)

This patch implements allocate for two node requests. One node requests
can be done as soon as we have a valid allocateOn function for single
nodes.

Start implementing the hail functionality

This patch implements a very stupid (and broken) version of hail
‘allocate’.

Remove the ktn/kti from second half of loading

This removes the return of ktn/kti from Loader.mergeData and associated
functions.

Add some utility functions for kt deprecation

These will be used to remove even more uses of ktn/kti in non-critical
paths.

Strip the suffix from the names in the objects

This strips the suffix from the objects themselves, not only from the
ktn/kti vars.

Move common loading sequence in CLI

This patch moves the common loading sequence to CLI, such that hbal/hn1
and possible future scripts that take the input from same sources can
use it.