Statistics
| Branch: | Tag: | Revision:

root / src / Ganeti @ 8920fa09

# Date Author Comment
1ca709c1 02/20/2013 06:47 pm Iustin Pop

Switch the curl bindings from optional to required

Currently, we support curl being optional via some sporting exercises:
ifdefs in the code, data types that represent 'Curl is disabled'
state, etc. However, with the future work on RPC, we would have to
even make the dependencies list conditional on it, etc. This is too...

85f6a869 02/20/2013 06:47 pm Iustin Pop

Reduce duplication of curl options computation

Some curl option are request-specific, but not node specific: e.g. rpc
timeout, etc. The patch changes the HttpClientRequest type so that we
can pre-seed such options, instead of rebuilding the list in each
individual request execution....

9c0a27d0 02/20/2013 06:47 pm Iustin Pop

Simplify RPC error cases

This patch removes the node from the RPC error constructurs
CurlLayerError and OfflineNodeError. The rationale is that we anyway
return tuples (node, result), and removing this duplication allows
simplified signatures/calls in the execution of RPC calls....

da9e2aff 02/20/2013 06:47 pm Iustin Pop

Add two utility functions for handling Either lists

These two functions permit operating in bulk on only the Left or Right
values in the original list, then reassembling the list back in the
original order.

Signed-off-by: Iustin Pop <>
Reviewed-by: Michele Tartara <>

cc40185c 02/20/2013 06:47 pm Iustin Pop

Add a Ganeti-specific implementation of Curl Multi

As we want to be able to run queries against multiple nodes in
parallel, and furthermore in parallel with other work, we need to
implement the Curl Multi interface (see libcurl-multi(3)).

This patch adds a Ganeti-specific such implementation, to be used...

8920fa09 02/20/2013 06:47 pm Iustin Pop

Switch the RPC module over to the multi interface

This replaces the very-basic parMap of IO actions (fully serialised,
as parMap won't work here), to the multi interface.

This makes a simple "time gnt-node list" on a 6-node cluster go from
3.2s to ~0.9s, and allows even better parallelisation - before,...

4a90bd4f 02/19/2013 04:27 pm Michele Tartara

Status change reason support for Reboot

Add support to the Reboot command for specifying the reason for the last
status change.

Some features are implemented as functions, even if used only once, because
they will be used by the future patches introducing reason support for all...

a59d5fa1 02/19/2013 04:27 pm Michele Tartara

Infrastructure for specifying instance status change reason

This patch introduces some infrastructural modifications that will be used by
the following commits to implement the support for specifying the reason for
the last status change of an instance.
...

834bea99 02/18/2013 06:08 pm Helga Velroyen

Unit tests for Query/Network.hs

This patch adds a couple of unit tests for Query/Network.hs.
Note that they'll need to be adapted, once issue 362 is addressed.

Signed-off-by: Helga Velroyen <>
Reviewed-by: Iustin Pop <>

7dc27988 02/18/2013 05:35 pm Michele Tartara

Make Confd client usable for testing

Allow the Confd client to be able to connect to an arbitrary server
instead of just the real one running on a cluster.

This is meant to be used for testing.

If a data collector using the Confd client is tested with shelltest, it will...

c5a957c3 02/18/2013 05:35 pm Michele Tartara

Add Haskell parser for "xm uptime"

In order to fetch precise information about the uptime of the VMs
running in Xen, we need to analyze the output of the "xm uptime" command.

This commit adds the parser to do that, and its tests.

Signed-off-by: Michele Tartara <>...

b8585908 02/18/2013 05:35 pm Michele Tartara

Add Haskell parser for "xm list --long"

In order to fetch precise information about the status of the VMs running in
Xen, we need to analyze the output of the "xm list --long" command.

This commit adds the parser to do that, and its tests.

Signed-off-by: Michele Tartara <>...

332a83ca 02/18/2013 05:34 pm Michele Tartara

Add request type to Confd server for getting instance list

Add to Confd server a new request type (and its implementation) to ask for
the list of instances in a node.

Signed-off-by: Michele Tartara <>
Reviewed-by: Iustin Pop <>

fe502d25 02/15/2013 03:04 pm Iustin Pop

Merge branch 'devel-2.7'

  • devel-2.7:
    Rename lib/objectutils to outils.py
    Fix typo in gnt-group manpage
    Fix wrong type in a docstring of the RAPI subsystem
    Finish the remote→restricted commands rename
    Enable use of the priority option in hbal
    Add CLI-level option to override the priority...
551b44e2 02/12/2013 03:47 pm Iustin Pop

Add a function to change an OpCode's priority

This simply updates the metaopcode submit priority.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

37fe56e0 02/12/2013 03:47 pm Iustin Pop

Add functions to parse CLI-level format of priorities

The current serialisation format for submit priorities is
integer-based, same as the opcode json serialisation. But for CLI
level, we need to support a string-based format, so we add functions
to parse and format this representation....

e247747c 02/12/2013 03:47 pm Iustin Pop

Add CLI-level option to override the priority

This just defined the new priority, with the same name as the Python one.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

a81ca843 02/12/2013 03:47 pm Iustin Pop

Enable use of the priority option in hbal

This patch adds the option to hbal, and uses it to tweak the submitted
jobs. There are also two small shelltests for testing the parsing.

Signed-off-by: Iustin Pop <>
Reviewed-by: Guido Trotter <>

d41f6558 02/12/2013 03:47 pm Iustin Pop

Make hbal opcode annotation more generic

Currently, hbal code always uses annotateOpCode function, which means
we would have to pass the options data to all function in the call
chain if we wanted to make this more flexible.

By abstracting the type of the annotator and passing it as an argument...

bb63ccb0 02/12/2013 03:39 pm Helga Velroyen

Removes check for conflicts from NetworkDisconnect

This removes the check for conflicts from the Haskell
version of the OpCode NetworkDisconnect. This alignes
the Haskell code with the patch
"Force conflicts check in LUNetworkDisconnect" (which
is currently under review). I will submit these patches...

3c296f56 02/12/2013 03:25 pm Michael Hanselmann

Merge branch 'devel-2.7'

  • devel-2.7: (23 commits)
    QA: Support additional arguments for initialization
    qa_utils: Fix order of arguments passed to _AssertRetCode
    Improve reporting on errors.AddressPoolError exceptions
    Add note about lv-tags rename...
5cfa6c37 02/11/2013 05:31 pm Dimitris Aragiorgis

Remove network_type slot (Issue 363)

This slot was not used by Ganeti so the same info can be
provided via tags. In order not to break configuration data
we add a FromDict() method in Network config object that
removes the deprecated network_type (if found) and then invoke...

48616625 02/11/2013 05:31 pm Dimitris Aragiorgis

Remove family and size from network objects

This info is not used by Ganeti and therefore is removed.

Signed-off-by: Dimitris Aragiorgis <>
Reviewed-by: Guido Trotter <>

5ba35653 02/08/2013 01:40 pm Guido Trotter

Make ParticalNic's network field of type String

This was applied to "master" along with extra changes affecting the
master branch only. Cherry-picking just the Objects.hs change.

Signed-off-by: Guido Trotter <>
Reviewed-by: Iustin Pop <>

72747d91 02/07/2013 06:14 pm Iustin Pop

Remove use of 'head' and add hlint warning for it

Since 'head' is unsafe to use in most cases, this patch removes its
use from most of the code, adds a lint warning for it (and for tail as
well), and adds override annotations in the few cases where it's
actually OK to use it (mainly when using head over the result of...

546a1dcf 02/07/2013 05:02 pm Dato Simó

Harep.hs: fix a couple typos in comments and docstrings

Signed-off-by: Dato Simó <>
Reviewed-by: Guido Trotter <>

94042ae4 02/06/2013 02:16 pm Michele Tartara

Fix improperly formatted docstring

Change the docstring of chompPrefix to prevent the error
"doc comment parse failed" that was raised by some version of
haddock while generating the documentation for this function.

Signed-off-by: Michele Tartara <>...

83251115 02/05/2013 06:55 pm Michael Hanselmann

Merge branch 'devel-2.7'

  • devel-2.7:
    Redirect output for gnt-* list/list-fields commands
    Add function to execute QA commands with redirected output
    Implement option to skip logging of QA commands
    Fix typo in a comment
    Add some more Haskell/Python equivalence tests...
0c6d6a52 02/05/2013 03:37 pm Helga Velroyen

Making ParticalNic's network field of type String

This is yet another fix for type confusion between python
and haskell. ;) The network field of PartialNic should be
a string and not of type Network. This makes it necessary
to add a helper function to look up a network by name...

6d3d13ab 02/05/2013 03:14 pm Iustin Pop

Fix typo in a comment

Signed-off-by: Iustin Pop <>
Reviewed-by: Michele Tartara <>

9411474b 02/05/2013 10:36 am Iustin Pop

Fix Haskell log file naming after virtual cluster changes

Commit 3329f4de changed the Haskell log file from constants to
functions, but introduced a bug: it uses now the daemon name instead
of the correct log file, which means "ganeti-confd.log" instead of...

8072af6c 01/30/2013 07:39 pm Dato Simó

harep: create repair jobs

Implement 'doRepair' to create a repair job from a list of opcodes if
the instance's policy allows it (otherwise set an ENOPERM result label),
and the instance was previously healthy (i.e. not in ArFailed or
ArPendingRepair).
...

8af72964 01/30/2013 07:39 pm Dato Simó

harep: do not wait for repair job completion to set tags

Because of instance locks, after submitting a repair job we weren't able to
set the "pending" tag until at least the first opcode of the job finished.
Introduce a small delay in the repair job so as to allow the subsequent...

41238d27 01/30/2013 07:39 pm Dato Simó

harep: finish execution with a summary of instance states

The harep tool prints messages for every action that it performs (or that
it doesn't perform). In case nothing is to be done at all, always print
some statistics of the current state of the cluster....

3416e3e7 01/30/2013 07:39 pm Dato Simó

harep: initial parsing of tags

Parse auto-repair tags to set each instance in one of ArHealthy, ArFailed,
or ArPendingRepair. The implementation tries to be well behaved when old
tags have been left behind, which future patches will still try not to do....

a5e58418 01/30/2013 07:39 pm Dato Simó

harep: check for completed jobs at the start of the program

As a first step before detecting any brokeness with instances, see if any
of our previous repairs have completed, and move instances to ArFailed or
ArHealthy accordingly. Do nothing if there are still running jobs for the...

4ba5f148 01/30/2013 07:39 pm Dato Simó

harep: pure function to detect brokeness with instances

Add a 'detectBroken' function that determines whether an instance is in an
unhealthy state, and what's needed to repair it. The repair is specified as
an AutoRepairType constant, and a list of opcodes. The opcodes will only be...

b6d9bec8 01/30/2013 07:38 pm Dato Simó

Program/Harep.hs: add skeleton for the new auto-repair tool

harep(1) detects certain kind of problems with instances and applies the
allowed set of solutions. See doc/design-autorepair.rst.

Signed-off-by: Dato Simó <>
Reviewed-by: Iustin Pop <>

f5606cda 01/30/2013 07:37 pm Dato Simó

Fix typo ("Abrreviation" -> "Abbreviation" in Common.hs)

Signed-off-by: Dato Simó <>
Reviewed-by: Guido Trotter <>

3f68544e 01/30/2013 07:37 pm Dato Simó

Jobs.hs: add a submitJobs function

This new 'submitJobs' function doesn't wait for job completion, it just
returns the job IDs immediately.

Signed-off-by: Dato Simó <>
Reviewed-by: Iustin Pop <>

7d8b651e 01/30/2013 07:37 pm Dato Simó

Jobs.hs: add an execJobsWaitOk function

This new 'execJobsWaitOk' returns "Bad" not only if an error occurs, but if
any of the submitted jobs doesn't succeed.

Signed-off-by: Dato Simó <>
Reviewed-by: Iustin Pop <>

b8e76da8 01/30/2013 07:37 pm Dato Simó

Jobs.hs: start with a shorter delay in waitForJobs

Instead of waiting 15 seconds every iteration of the waiting loop, start
with 0.5 seconds and increment exponentially until a certain maximum is
reached. Keep the maximum at 15 seconds.

Signed-off-by: Dato Simó <>...

de9d75c8 01/30/2013 07:37 pm Dato Simó

HTools/Types.hs: minor adjustments to auto-repair types

In particular:

- make ArHealthy take an optional AutoRepairData; this allows to
represent the situation where a repair completed successfully, and
hence there's an associated tag we might want to know about....
0ae4b355 01/30/2013 07:26 pm Helga Velroyen

Removes check for conflicts from NetworkDisconnect

This removes the check for conflicts from the Haskell
version of the OpCode NetworkDisconnect. This alignes
the Haskell code with the patch
"Force conflicts check in LUNetworkDisconnect" (which
is currently under review). I will submit these patches...

b76d4aaf 01/30/2013 06:13 pm Michael Hanselmann

Merge branch 'devel-2.7'

  • devel-2.7:
    hv_kvm: Original error message, keyword parameter
    CLI.hs: fix double spaces in option help strings
    Add new NIC parameter to man page of 'gnt-instance'
    Fix spelling issue in hroller.rst

Signed-off-by: Michael Hanselmann <>...

516c52f2 01/29/2013 05:08 pm Dato Simó

CLI.hs: fix double spaces in option help strings

Some help strings with continuation backslashes ('\') were providing a
space both before and after the backslash, resulting in double spaces in
help output. Provide it only after the backslash, which fixes the issue and...

125df9a8 01/29/2013 03:07 pm Helga Velroyen

Add fields 'inst_cnt' and 'inst_list' to network queries

This adds the fields 'inst_cnt' and 'inst_list' to the
Haskell implementation of the network queries.

Signed-off-by: Helga Velroyen <>
Reviewed-by: Michele Tartara <>

2d3d843d 01/29/2013 03:06 pm Helga Velroyen

Moving network query helper functions to Network.hs

So far, a couple of helper function for the network
queries resided in Config.hs. I figured it makes more
sense to move them to Query/Network.hs, since they are
very tailored to that purpose.

Signed-off-by: Helga Velroyen <>...

ae960018 01/25/2013 06:12 pm Helga Velroyen

More fields for network queries

This adds more fields to the network queries:
- group_cnt, free_count, reserved_count, and map

Signed-off-by: Helga Velroyen <>
Reviewed-by: Guido Trotter <>

05092772 01/24/2013 08:30 pm Helga Velroyen

First part of Network Queries in Haskell

This is the beginning of the implementation of network
queries. This includes establishing all infrastructure
to run the network queries and implement querying of
some simpler fields and the node group listing.

Signed-off-by: Helga Velroyen <>...

a64cc96b 01/24/2013 08:30 pm Helga Velroyen

Convert Maybe results to RSUnavail

When displaying query results of type Maybe, one could use the
function rsMaybe. Unfortunately, this function maps 'Nothing'
values to RSNoData which gets displayed as '?' in the list of
query results. These semantics do not fit if the result is of...

b43064d0 01/24/2013 08:30 pm Helga Velroyen

Extend config by networks and networks by UUIDs

For network queries to work, we need to extend the general
config type to include all available networks. Additionally,
we add UUIDs to the network type itself.

Signed-off-by: Helga Velroyen <>...

3673a326 01/24/2013 06:00 pm Helga Velroyen

Typo in comment of network type

Signed-off-by: Helga Velroyen <>
Reviewed-by: Iustin Pop <>

d316b880 01/21/2013 01:58 pm Iustin Pop

Renames and cleanup of variable names in confd

The current names are quite confusing; this patch cleans up the
confusion by making sure we use different terms for the two threads,
etc.

No actual code changes besides the renames.

Signed-off-by: Iustin Pop <>...

a6e054a8 01/21/2013 01:57 pm Iustin Pop

Add a function to get the current time in microseconds

In some cases we need higher resolution that seconds; I've settled on
microseconds as that is what 'threadDelay' wants, for exactly, so it's
easier if we keep the same units.

Signed-off-by: Iustin Pop <>...

ce817701 01/21/2013 01:57 pm Iustin Pop

Increase Haskell log file resolution

Currently the Haskell logging goes with the default time formatting,
which lacks sub-second resolution. This is not good, as it's too
coarse.

The patch adds picoseconds to the log; they will contain usually too
many zeroes, but the other option is to use a log format that (for...

7e7fa841 01/21/2013 01:57 pm Iustin Pop

First part of confd timer changes

This patch changes the resolution of the timers: the watcher timer
goes from 60s to 17s, and the polling-mode timer goes from 2 seconds
to 250ms. The code changes a bit more due to the changes in the units
of the various constants....

01eea342 01/21/2013 01:57 pm Iustin Pop

Change the polling thread sleep behaviour

Currently, the polling thread sleeps at the end of its body; the
original reason was that when the thread initially starts, we want to
load the config immediately (in the old-style config load model).

However, now we explicitly load the config, so we can make this thread...

c62df702 01/21/2013 01:57 pm Iustin Pop

Start confd in notify mode if we can

This patch changes the default model—where confd is always started in
polling mode—to a model where if possible we enable inotify mode
before starting any of the background threads.

There are some tricky details here: if we enable inotify, we should...

3190ad64 01/21/2013 01:24 pm Iustin Pop

Explicitly load the config at confd startup

Currently, confd main function doesn't load the config, but simply
forks the individual timer threads, which will—as a side effect-also
load the configuration as soon as they start.

However, this makes codes to the startup more complex, so let's...

9fb621af 01/18/2013 12:41 pm Yiannis Tsiouris

Text wrap in Haskell code

This adds two functions in Utils.hs: 'wrap' and 'trim' along with some
simple tests. 'wrap' is used in Common.hs for wrapping command
descriptions to a fixed line length, e.g. useful for the "--help"
option.

Signed-off-by: Yiannis Tsiouris <>...

e71c47d3 01/16/2013 03:56 pm Michele Tartara

Integrate DRBD with the report generation infrastructure

With this commit, the DRBD data collector does not only print the data
it extracts, but includes them in the proper JSON structure common to
all the data collectors, as prescribed by the design document....

13d3acab 01/16/2013 03:56 pm Michele Tartara

Add reporting infrastructure for data collectors

This commit adds the part of the JSON report generation code that will be
common to all the data collectors, according to the format specified in the
design document.

Signed-off-by: Michele Tartara <>...

ace37e24 01/16/2013 03:56 pm Michele Tartara

Move getCurrentTime from Confd.Utils to Utils

The getCurrentTime function is of wide interest, so its moved to a more
appropriate position.

Signed-off-by: Michele Tartara <>
Reviewed-by: Iustin Pop <>

a7f0953a 01/15/2013 04:27 pm Iustin Pop

Make niceSort a wrapper over niceSortKey id

Currently, niceSort behaves differently than niceSortKey in cases
where the parsed groups are equal but the original not, as it uses
both members of the tuple as opposed to just the first one. Both
niceSortKey and the Python code use only the first one, so in corner...

2f3a3365 01/15/2013 11:42 am Helga Velroyen

Replace nodegroups' PartialNic by PartialNicParams

This fixes a bug that corrups the config when one adds a
network to a node group. I wil soon extend QA by some commands
that would have detected this.

Signed-off-by: Helga Velroyen <>
Reviewed-by: Guido Trotter <>

ef947a42 01/14/2013 05:54 pm Dato Simó

Loader.hs: ignore expired ArSuspended policies

At the moment, because 'mergeData' is pure, it may set instance auto-repair
policies that are of the form `ArSuspended $ Until timestamp_in_the_past`.
If later on the auto-repair tool notices this, it has lost access to what...

e79f576c 01/14/2013 05:54 pm Dato Simó

Loader.hs: rewrite extractExTags to use chompPrefix

Signed-off-by: Dato Simó <>
Reviewed-by: Iustin Pop <>

55416810 01/14/2013 05:54 pm Dato Simó

Loader.hs: set instance auto-repair policy in mergeData

'getArPolicy' and 'setArPolicy' follow the precedence rules introduced in
b1eb71c: within an object, the most restrictive tag wins; across object,
the nearest tag wins.

Signed-off-by: Dato Simó <>...

23594127 01/14/2013 05:54 pm Dato Simó

Instance.hs: add an 'arPolicy' field for auto-repair policy

Signed-off-by: Dato Simó <>
Reviewed-by: Iustin Pop <>

b6aeda4a 01/14/2013 05:54 pm Dato Simó

Utils.hs: add a clockTimeToString function

This function allows to easily convert a ClockTime object to a string
representation of its timestamp (seconds-only).

Signed-off-by: Dato Simó <>
Reviewed-by: Iustin Pop <>

b009f682 01/14/2013 05:54 pm Dato Simó

Utils.hs: function to chomp prefix + separator from a string

Some auto-repair tags are composed of a prefix and then an optional
argument; the new "chompPrefix" function in Utils.hs allows to strip a
prefix, allowing the last character (the separator) to be absent if there...

3e77a36c 01/14/2013 05:52 pm Dato Simó

Add initial constants and Haskell ADTs for auto repair

In this commit, the AutoRepairType and AutoRepairResult types are defined,
with the possible values specified in doc/design-autorepair.rst.

Signed-off-by: Dato Simó <>
Reviewed-by: Iustin Pop <>

7ec2f76b 01/14/2013 05:52 pm Dato Simó

HTools/Types.hs: more auto-repair types

AutoRepairPolicy, AutoRepairStatus, and other auxiliary types are added.
These are used only internally by the auto-repair tool, and parsed from the
various object tags as defined in the design doc.

Signed-off-by: Dato Simó <>...

c816a371 01/10/2013 02:44 pm Michele Tartara

Fix typo in the DRBD parser types file

A missing letter has been added to a docstring.

Signed-off-by: Michele Tartara <>
Reviewed-by: Michael Hanselmann <>

d78970ba 01/08/2013 06:16 pm Michele Tartara

Support instance-minor pairing in the DRBD collector

This commits enables the DRBD data collector to use the Confd client to
gather information about the pairing between DRBD minors and instances.

For testing purposes, the DRBD data collector now requires either zero...

eb62691c 01/08/2013 06:16 pm Michele Tartara

Support integrating instance information in the DRBD parser

This commit modifies the DRBD parser and its data structures to include
information about the instance a DRBD minor belongs to.

Test files have been updated as well, to support the new field in the data...

7ce2f8ee 01/08/2013 11:26 am Michele Tartara

Fix a bug in the generation of completion information

Completion information for multi-command binaries were not generated in
the correct way, so the build-bash-complation script failed when such a
binary was actually found.

This commit fixes this issue, and a related docstring typo as well....

8d459129 01/07/2013 07:00 pm Michael Hanselmann

Add "use_locking" parameter to network query opcode

This was extracted from a patch by Dimitris Aragiorgis with the subject
“Add locking to _NetworkQuery”. The rest of the patch did no longer
apply and will require more work.

Signed-off-by: Michael Hanselmann <>...

2663321c 01/04/2013 10:47 am Michele Tartara

Improve the reception of replies in the confd client

If a received reply is valid and there is still no valid reply,
keep it, no matter how new it is.

Signed-off-by: Michele Tartara <>
Reviewed-by: Guido Trotter <>

f6b9c6e5 12/28/2012 05:50 pm Iustin Pop

Fix a bad data type in Hcheck.hs

While trying to understand why some code was not being tested, I
realised that we have a bad data type in Hcheck.hs.

We have "data Level = GroupLvl | ClusterLvl", but then we need to pass
the group name/index as well, so we have functions that look like the...

2fd5a116 12/28/2012 05:50 pm Iustin Pop

Move src/Ganeti/HTools/Program.hs to Program/Main.hs

This removes one more tab conflict; this is the last module in our
code where we have both x.hs and x/.

Furthermore, we collapse all actual code into the new Main.hs module,
leaving the htools.hs basically empty (will allow better testing in...

3add7574 12/24/2012 11:38 pm Iustin Pop

Rename htools/ to src/

Per offline discussions, this is the first patch of the
renames. Tested with "make distcheck", seems to work fine.

The only change outside of the renaming is a bit of simplification in
the .gitignore rules; otherwise, simply s/htools/src/....