Synnefo » snf-ganeti

Modify the internals of the detailed CV scores

Before we used a tuple; since we'll need more metrics in the future,
it's simpler to transform this into a list of doubles, whose elements
are handled homogeneously by all the code that needs them.

Implement first version of job execution

This not so nice patch adds submit/wait-for-completion functionality in
hbal. When enabled via ‘-X’ (only working on Luxi), it executes the jobs
(as split into separate jobsets by Cluster.splitJobs) and waits until...

Add two specialized Luxi calls

This are higher level wrappers over the basic callMethod.

Signed-off-by: Iustin Pop <iustin@google.com>

Extend the MoveJob type to hold the instance index

This will be needed in order to generate the proper instance move commands.

Signed-off-by: Iustin Pop <iustin@google.com>

Only update the version file on make dist

Since on regular compiles the version information is not critical, it's
better to only update it on make dist; furthermore, there it is indeed
needed to regenerate it automatically, without requiring the use of make...

makefile: regenerate version when it changes

If we're building on a different version, the version file needs to be
regenerated.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Store the instance move in the MoveJobs

This will automatically sort our Ganeti jobs into the independent job
sets, and then we can submit them separately.

Add a function converting Placements into Jobs

This converts from htools-specific Placements into Ganeti standard
OpCodes, which will later allow execution via Luxi.

Record the move being performed in a Placement

This will allow a more descriptive output later in the solution list, as
opposed to trying to reconstruct the move from the node indices.

The patch also documents the Placement members.

Split the Luxi generic parts from the loader

The Luxi loader implements both a generic Ganeti Luxi client and the
loader; it is better if these two are separated. The patch adds a
Ganeti/Luxi.hs (not under HTools!) since that is generic for Ganeti, and
not related necessarily to htools.

Change ExtLoader to only handle I/O errors

Due to the Control.Exception changes between 6.8 and 6.10, using it
portably is difficult. Since we're only interested in handling I/O
errors, we can use prelude's catch and not have to deal with
Control.Exception at all....

Update NEWS file for the 0.1.7 release

Turn on, and fix, more warnings

The Makefile was intented to be -Wall and not simply -W, but I missed
that. This enables more warnings and also enables -Werror (except for
the tests).

Add support for building without curl

Since curl is not always needed (e.g. when only using luxi or less
likely file backends only) and is also not always available, it is
useful for building without it. This of course disabled the RAPI
backend.

This patch changes ExtLoader to build with the ‘-cpp’ option which makes...

Fix luxi recvMsg for messages bigger than 4K

This patch fixes a logic bug in luxi that breaks receive of messages
bigger than 4096 bytes. The send message is not impacted as it uses a
different algorithm.

Add some more instance tests

This include instance text load tests.

Implement support for 'cheap' moves only

This patch adds support for cheap (failover/migrate) operations only in
the balancing algorithm and in the hbal command line options.

This allows a very quick balancing (compared to allowing replace-disks)
which can be useful as a scheduled operation.

Use migrate or failover based on instance state

While we can't guarantee that the instance will be in the same state by
the time the migrate/failover command will be run, we can at least try
to do the right thing assuming no other changes to the cluster state....

Improve the error message for command line errors

Instead of using ioError . userError, we format the error ourselves.
This is nicer - no ‘)’ at the end of the output.

Fix a typo in hbal.hs

Signed-off-by: Guido Trotter <ultrotter@google.com>

Unify the command line options and structures

This patch moves all the command line options and their internal
representation into CLI.hs. This means that duplicated options between
any two binaries are no longer declared twice, and that we no longer
need the two *Option classes.

Fix a few hlint errors

CLI: Prevent incompatible options to be selected

This patch makes CLI abort if more than one backend is selected.

Add support for luxi backend in CLI/hspace/hbal

This patch changes the backend selection method in CLI to prefer, in order:
- a RAPI specification
- a Luxi specification
- and finally the node/instance files

It also modifies hspace and hbal to provide a ‘-L’ command line option...

Introduce timeout in RAPI queries

The patch adds two constants in Types.hs for connect and query timeout,
then modifies Rapi.hs to use them as the connect and general curl
timeout.

Rapi could be improved more, as currently we wait double the total
timeout due to not aborting early in case the node queries failed.

Fix a haddoc issue

Change the tryAlloc/tryReloc workflow

Currently, the tryAlloc and tryReloc function return a list with all the
results, both failures and successes. This is fine for hail, which does
one round of allocations, but is not so good for hspace, which does
iterative rounds; since at each (successful) step we only take the best...

Slight change to the internal allocation results

Currently the Cluster.AllocSolution type is defined as a list of
‘(OpResult Node.list, …)’ and the results for applyMove are defined as
‘(OpResult Node.List, …)’. Both these means that the failure/success
indication is hidden in the first elements of this tuple, which makes is...

hspace: switch output to shell-script format

This (big) patch changes the output of hspace from text-format
(separated by ‘: ’) to a shell-snippet, in ‘key=value’ format.

This will allow sourcing the output or parsing it via awk/sed/etc.

Fix unittests

The recent OpResult and CPU values additions broke unittests.

Show errors on stderr instead of stdout

Currently many of the exit and warning conditions mistakenly display error
messages on stdout, which makes parsing the output of programs harder. This
patch attempts to fix such occurrences.

hspace: convert N1 error exit into FailN1 result

Currently hspace exits with an error if the cluster is not N+1 compliant
at the beginning of the run. This patch changes hspace such that this
condition is instead treated as a zero-allocation-possible, FailN1 mode....

hspace: add display of instance spec

This is mostly for user-friendliness in the default mode, when we don't
specify the instance parameters.

Take the foldl out of Loader.fixNodes

Currently Loader.fixNodes is foldl' with a complicated function. It
makes more sense to take foldl' out of this function (and put it into
the caller) and let fixNodes be only this internal function.

Fix hlint-generated warnings

This big patch cleans up the code per hlint indications. Many removals
of extra parentheses, replacements of concat . map with concabtMap,
extra dollar signs, eta reductions, etc. were performed.

The code still compiles and passes a couple of manual tests on sample...

Return correct failure data from Node.add*

This patch alters the Node.addPri/addSec to return correct failure data.
It removes the computeFailN1 function from the module as that used to
combine both mem and disk checks in the same function and thus the real...

Remove hn1 and related code

hn1 was deprecated for a while and this patch removes it altogether. The
support code in Cluster.hs is also removed.

Fix totalResources avail disk computation

This uses the newly-added Node.availDisk to compute the actual available
disk correctl, and display the total allocatable disk in hspace.

Add two new autocomputed vars to Nodes

Currently we track the max disk usage/max vcpus as percentages, however
sometimes it's easier to check against minimum free disk or maximum
number of cpus, as units instead of percentages.

This patch adds two new variables, lo_dsk, hi_cpu, which are recomputed...

Enhance hspace resource display

The display of cluster resources is extracted into a separate function
and enhanced to display more stats.

Update NEWS file for the 0.1.4 release

Fix some hscan bugs

Currently hscan has a number of bugs:
- doesn't add the common suffix (csf) to the instance's nodes
- doesn't export the cpus for neither nodes nor instances
- doesn't support single-node instances

This patch fixes these issues.

Add cpu/disk limits in hbal

Implement cpu/disk limits in instance moves

We modify Node.addPri/addSec to take into account the limits on instance
adds.

Fix 'unused X' warnings

This removes some unused functions and imports to cleanup the warnings.

Small changes to the node list output

This is just some cleanup of the node list output, adding pcpu/vcpu
counters, and making the display slightly nicer.

Update cpu counters correctly after pinst changes

The cpu counters are update on primary instance adds/removes.

Add a new vcpus attribute to instances

This patch adds reading of vcpu count for instances, in preparation for
using the vcpu ratio in cluster scoring.

Update NEWS and README for the 0.1.3 release

Small updates to the documentation and make a new small release.

Update NEWS file for the 0.1.2 release