Synnefo » snf-ganeti » ganeti-local

Add a luxi call for multi-job submit

As a workaround for the job submit timeouts that we have, this patch
adds a new luxi call for multi-job submit; the advantage is that all the
jobs are added in the queue and only after the workers can start
processing them....

Merge branch 'master' into branch-2.1

Move to data-based hvparam checks instead of code

Currently the hypervisor parameters are checked using hard-coded snippets in
each hypervisor. However, most parameter checks fall into three cases:
- file check
- directory check
- string value in a set...

Move more hypervisor strings into constants

This patch adds constants for the mouse and boot order strings; while
there are still some issues remaining, we're trying to cleanup hardcoded
strings from the hypervisors.

Since the formatting of frozensets is currently wrong, we also add an...

IAllocator: export total disk size for instances

This patch adds for current instance a ‘disk_space_total’ key, similar
to the key for the new instance in case of new allocations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

call_instance_start: add optional hv/be parameters

This patch modifies the rpc.call_instance_start - the master side - to
take optional hv/be parameters. The noded side is unchanged and
oblivious to the change.

This will allow implementation of single-user capability and such on...

Instance reinstall: don't mix up errors

If the remote info rpc call fails we can't assume that the instance is
up.

Signed-off-by: Guido Trotter <ultrotter@google.com>

2.1 design: add VNC console password changes

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Move HVM's device_model to a hypervisor parameter

This moves yet another hardcoded value to a hypervisor parameter. I
removed the 64/32 difference as it doesn't seem valid to me - it's more
of a local site config rather than arch config.

Signed-off-by: Iustin Pop <iustin@google.com>...

2.1 design: propose redistribute config changes

This patch proposes a mini-design to improve redistribute-config and
integrate it better with other logical units.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LUSetClusterParams: improve volume group removal

Currently LUSetClusterParams will remove the volume group if the vg_name
field passed in is not true, but not None. Setting the target volume
group to False or the empty string, though, is a bad idea because it's...

LUQueryClusterInfo: return a few more fields

Some fields can be set at cluster init, and perhaps even modifed with
SetClusterParams but there's no way to know them. With this patch we
export them in the cluster info query.

Signed-off-by: Guido Trotter <ultrotter@google.com>...

KVM: Abstract runtime file removal in a function

This removes some code which was duplicated in shutdown and migrate.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some small doc updates

We change some formatting to sphinx-specific, to show how the
documentation can be improved.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUSetInstanceParam: don't assume memory is integer

LUSetInstanceParam currently assumes that the 'memory' value of a
call_instance_info result is an integer, while the rest of the code
explicitely converts it to int(). Converting it to int works around a...

Convert from auto-generated RAPI docs to static

This patch removes the autogeneration of the RAPI docs from the code
(based on docstrings) and moves the current autogenerated output to
the rapi.rst file.

The reasons behind this are multiple:
- the build system becomes a little more simple (this could have been...

Fix QA and documentation about no initrd case

In Ganeti 1.2, “none” was used to signify no initrd. In 2.0 we have
changed to “no_” as a prefix (i.e. “-H no_initrd_path”) and thus we
document in the manpage this.

The QA suite is changed accordingly.

Signed-off-by: Iustin Pop <iustin@google.com>...

Exporting the instance network_port on the RAPI

Patch for adding network_port to the instance attributes exported by the
RAPI.

[iustin@google.com: slightly changed the formatting]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Small doc change in README

The version is 2.0, and we don't build PDFs by default, only HTML
files.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Make Python interpreter selectable for test scripts

The Python interpreter used to run the test cases is hard-coded to be
/usr/bin/python. If we use the first one from $PATH instead, it is
much easier to test ganeti with other Python versions.

Signed-off-by: Iustin Pop <iustin@google.com>...

Add initial 2.1 design doc

This document contains a skeleton for the 2.1 design process.
For now it just has introductory paragraphs and a structure for the
various areas' design, but some sections still don't have a text, as
we're still in the early design phases....

ganeti.initd: include defaults file, if present

In the example init script we'll execute an optional defaults file to
make it easier to add local customizations to the ganeti startup.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin <iustin@google.com>

Avoid DeprecationWarning on Python >= 2.6

Python 2.6 complains about module 'sha' being deprecated. It makes
execution of Ganeti commands a bit annoying, and when you run
'ganeti-watcher' in cron jobs, you get a mail message after every
execution.

Tests pass under under Python 2.6 and Python 2.4....

ganeti-noded: add bind address option

This allows ganeti-noded to bind only on one interface rather than all
the ones on the machine. The default behaviour doesn't change.

Signed-off-by: Guido Trotter <ultrotter@google.com>

Fix compatibility with DRBD 8.3

DRBD 8.3 changes two more things compared to 8.2:
- /proc/drbd format changed in multiple ways; the part we're
interested is the ‘st:’ to ‘ro:‘ change (in the changelog named as
“Renamed 'state' to 'role'”
- “drbdsetup /dev/drbdN show” changed the ‘device’ stanza from:...

Abstract Linux node information in hv_base

Currently both hv_fake and hv_kvm implement practically identical code
to get the node information. Since future container-like hypervisors
will also need this functionality, this patch moves it into the base
class (as a separate function) which can then be called from classes...

Small optimisation in utils.WriteFile

Currently we always try to remove the new file, even if the rename
succeeded. This patch tracks the existence of the new file and doesn't
try to remove it if we managed to rename it.

Signed-off-by: Iustin Pop <iustin@google.com>...

Allow gnt-debug submit-job to take multiple args

Currently “gnt-debug submit-job” takes a single argument and has
non-trivial startup-costs; in order to exercise the job system, it is
better to be able to submit multiple jobs with a single invocation of
the script....

Fix gnt-cluster getmaster on non-master nodes

The current implementation of “gnt-cluster getmaster” doesn't work on
non-master nodes, which is a regression from 1.2. This patch implements
it (again) via ssconf.

Signed-off-by: Iustin Pop <iustin@google.com>...

Update gnt-instance(8) for info

Add the --all argument, and reword a bit the basic information.

Reviewed-by: iustinp

LUDiagnoseOS: change locking and error handling

Since the “list OSes” call is exported via RAPI, this can be used pretty
easily to DOS the master daemon during long jobs.

The implementation of LUDiagnoseOS makes an RPC call to all nodes; we
lock nodes here in order to prevent node removal....

Prevent errors when xenvg is broken cluster verify

When vg_name is not returned at all, we currently abort with an internal
error. This is because we don't catch KeyError.

This patch adds a custom message for this case, and also adds KeyError
to the list of catched exceptions, just for safety....

Trivial typo fix in error message

Reviewed-by: iustinp

Distribute built documentation

This patch changes the way documentation is built in order to distribute
the generated output in the 'dist' archive, and thus no longer
requiring the presence of the docbook/rst toolchains during build time.
This will lower the requirements for installation and also makes the...

Fix the output of watcher on non-master nodes

Currently the watcher spews errors message on non-master nodes. This
cleans it up.

Reviewed-by: imsnah

Fix Xen soft reboot via polling

This patch fixes the Xen soft reboot ("xm reboot") via polling for a specific
time for either changed domain ID or decreased CPU run-time.

This sould prevent the race-conditions discussed on the mailing list for
reboots.
...

Add some more debugging info to masterd

This patch will log data about queries, which are today completely
invisible (at the default log level) in the master log file.

Reviewed-by: imsnah

Fix _NOQUOTE regexp

Allow expressions longer than one character to match.

Reviewed-by: imsnah

Raise on invalid gnt-cluster queue commands

1. gnt-cluster queue foo
   Failure: prerequisites not met for this operation:
   Command 'foo' is not valid.

Reviewed-by: iustinp

Add the 2.0-specific node flags to the design doc

This patch adds the newly-introduced node flags to the design document,
as they currently are missing from there.

The patch also reduces the TOC depth to 3, as it was too big.

Reviewed-by: ultrotter

Xen: Remove one hardcoded constant

s/"vnc_bind_address"/constants.HV_VNC_BIND_ADDRESS/

Reviewed-by: imsnah

Handle ghost instances in temp DRBD map

Currently cluster-verify doesn't handle the (admitedly invalid) case where we
have reservation for instances that were removed in the meantime.

This patch adds a check for this and prevents code errors in cluster-verify in...

Fix serial_no field on instances

The instance objects did not get a serial_no field. This patch adds a
new constants for the field name and uses it for all three cases
(cluster, nodes, instances).

Reviewed-by: imsnah

Complete the cfgupgrade script for 2.0 migrations

This patch makes the cfgupgrade script to handle:
- instance changes
- disk changes
- further cluster fixes
- adds configuration checks at the end, in non-dry-run mode

Reviewed-by: ultrotter

Fix bash completion for cluster copyfile/command

“copyfile” takes a file argument, so we enable file-completion for it.
“gnt-cluster command” takes a command, so we enable command completion.

Reviewed-by: imsnah

Export tags to cluster verify hooks

This patch export the cluster and node tags to the cluster verify hook
scripts. The tags are exported as a space-separated list, which allows
easy parsing from the shell (e.g. “for tag in $GANETI_CLUSTER_TAGS; do
...”) and therefore requires the previous “Don't allow spaces in tag...

Update the iallocator documentation

This updates the iallocator documentation to 2.0, bumps up the
iallocator version (and moves a constants to lib/constants.py), and
fixes a style on install.rst.

Reviewed-by: ultrotter

A doc update and a small indentation fix

This adds a small paragraph about the “master” role of a node, and fixes
a wrong indentation in the bash completion file.

Reviewed-by: imsnah

Create runtime dir in bootstrap

Some hypervisors (KVM) need RUN_GANETI_DIR to exist even at cluster init
time. This patch creates it in InitCluster just before hv parameter
checking. Since the code to make list of directories is already repeated
twice in the code, and this would be the third time, we abstract it into...

Convert the RAPI document to restructured text

This patch changes the RAPI document, and the RAPI resources
autogenerated-documentation to restructured text. This meant changing
the autogen tool.

The new fragment can be included via RST directives, and doesn't need...