Synnefo » snf-ganeti

Improve the node add operation

Currently, the node add operation uses a job to query the node name and
the bootstrap function directly reads the config file for the cluster
name.

This patch changes to that both the cluster name and the verification of
the node is done via queries to the master....

Allow network-less instances

Currently there's no way to specify no NICs for an instance, even though
this is a supported configuration.

The patch adds a --no-nics option to gnt-instance add.

Reviewed-by: amishchenko

Fix instance creation

This patch fixes the diskless and drbd/file based instances. Sorry :(

Reviewed-by: ultrotter

Show disk access mode in gnt-instance info

The mode parameter needs to be exported and shown in the info output.

Reviewed-by: imsnah

Enable auto-unit formatting in script output

This patch enables by default the old 'human-readable' option, but in a
slightly different model.

The option is now called "units" and takes either:
- 'h' for automatic formatting
- 'm', 'g' or 't' for mebi/gibi/tebibytes...

Make cli.py use FieldSet for matching fields

This changes cli.py to FieldSet usage so that gnt-instance list will
format nicely the disk.size/*, and the count of disks/nics.

Reviewed-by: ultrotter

Reuse HTTP client pool for RPC

ganeti-masterd: Add initialization and shutdown of RPC pool. It needs
to be shutdown before forking.

ganeti.cli: Add decorator function to initialize and shutdown RPC pool.

ganeti.rpc: Add functions to initialize and shutdown RPC pool. Throw...

Initial multi-disk/multi-nic support

This patch adds support for mult-disk/multi-nic in:
- instance add
- burnin

The start/stop/failover/cluster verify work as expected. Replace disk
and grow disk are TODO.

There's also a change gnt-job to allow dictionaries to be listed in...

Documentation updates for gnt-debug and gnt-os

Reviewed-by: imsnah

Documentation updates for gnt-backup

Reviewed-by: imsnah

Documentation updates for gnt-node

The patch also adds a 'return 0' to a function that was missing it.

Reviewed-by: imsnah

Documentation updates for gnt-instance

Reviewed-by: imsnah

convert remaining print statements to ToStderr in gnt-cluster

Reviewed-by: imsnah

Add mac option to gnt-backup import

Before 'auto' was the only allowed possibility

Forward-port-of: r1885, Reviewed-by: iustinp

Reviewed-by: imsnah

Convert the gnt scripts to ToStdout/err

Currently the gnt-* scripts are using a mix of print, logger.ToStd* and
sys.stderr.write. We convert them all to using cli.ToStdout/err. This
way, we can easily change the implementation for all at once.

Reviewed-by: imsnah

Remove --hypervisor-type from gnt-cluster.

We no longer use a single, cluster-wide hypervisor, but configure the
actual to be used hypervisor on the instance level.

Reviewed-by: imsnah

Enable gnt-cluster modify to hv/beparams

This patch enables the cluster modify to change:
- enabled hypervisor list
- hvparams (per hypervisor)
- beparams (only the default group)

Syntax:
gnt-cluster modify -B vcpus=3 -H xen-pvm:no_initrd_path

Validation for parameters is somewhat missing - the individual...

Add an interface for the drain flag changes/query

This adds the set/reset in the jqueue and luxi modules, and a way to
query it in OpQueryConfigValues, and also the comand line interface for
it:
$ gnt-cluster queue info
The drain flag is unset
$ gnt-cluster queue drain...

gnt-backup: update for cluster parameters

- add backend and hypervisor parameters
- fix beparams validation/passing
- pass hypervisor and hvparams
- remove deprecated flags

Reviewed-by: iustinp

gnt-instance remove deprecated flags

kernel, initrd, hvm_boot_order and vnc_bind_address are now hypervisor
parameters and should not have their own flag. Moreover querying of
vnc_bind_address should of course pass through the hv/ namespace.

Reviewed-by: iustinp

Some fixes related to auto_balance

Change the constant name to match the value (autobalance ->
auto_balance).

Also add the auto_balance header so that gnt-instance can list it.

Reviewed-by: ultrotter

Allow instance info to only query the config file

This patch adds a new '-s' parameter to ‘gnt-instance info’ that makes
it return only 'static' information. This is much faster, especially for
drbd instances.

This is a forward-port of rev 1570 on the ganeti-1.2 branch, resending...

Change gnt-instance modify to the hvparams model

Reviewed-by: imsnah

Switch instance hypervisor parameters to hvparams

This big patch changes instance create to the new hvparams structure.
Old parameters are removed, so old jobs or old instances file will break
current clusters.

Reviewed-by: ultrotter

Move the hypervisor attribute to the instances

This (big) patch moves the hypervisor type from the cluster to the
instance level; the cluster attribute remains as the default hypervisor,
and will be renamed accordingly in a next patch. The cluster also gains...

Implement job 'waiting' status

Background: when we have multiple jobs in the queue (more than just a
few), many of the jobs (up to the number of threads) will be in state
'running', although many of them could be actually blocked, waiting for
some locks. This is not good, as one cannot easily see what is...

Change SshRunner usage

Currently the SshRunner uses a SimpleConfigReader instance, however this
is not best. We change it to use the cluster name directly (and its
constructor now takes this as parameter, instead of SCR), and its
callers are change to pass the name directly....

Convert ssh.py

Get rid of ssconf and convert to configuration instead.

Reviewed-by: iustinp

Add output of job/opcode timestamps

This patch adds posibility of selection of job/opcode timestamps in
gnt-job list and info.

The code handling the possible cases (None or a valid timestamps) are
ugly though...

Reviwed-by: imsnah

Add opcode execution log in job info

This patch adds the job execution log in “gnt-job info” and also allows
its selection in “gnt-job list” (however here it's not very useful as
it's not easy to parse). It does this by adding a new field in the query
job call, named ‘oplog’....

Implement job summary in gnt-job list

It is not currently possibly to show a summary of the job in the output
of “gnt-job list”. The closes is listing the whole opcode(s), but that
is too verbose. Also, the default output (id, status) is not very
useful, unless one looks for (and knows about) an exact job ID....

gnt-instance: fix tags commands online help

It used to refer to "nodes", which was confusing.

Reviewed-by: iustinp

Do not use jobs in gnt-instance _ExpandNames()

In the gnt-instance script, _ExpandNames() uses jobs to query instance
names. This is not optimal, so we change it to use queries.

Reviewed-by: ultrotter

OpVerifyDisks returns a list, not a tuple

Fixing the check in gnt-cluster, or gnt-cluster verify-disks is broken.
Since the version in 1.2 used to return a tuple we'll accept both.

Reviewed-by: iustinp

merge r1568 from branches/ganeti/ganeti-1.2

Add more fields to gnt-instance list

Reviewed-by: imsnah

merge r1539 from branches/ganeti/ganeti-1.2

Display VNC console port in gnt-instance info.

Reviewed-by: iustinp

Merge r1537 from branches/ganeti/ganeti-1.2

Add HVM device type flags 3/4

Reviewed-by: ultrotter

LUVerifyCluster: Return boolean indication success

Reviewed-by: schreiberal

Allow kvm hypervisor in gnt-cluster init

Reviewed-by: imsnah

Use new query function for exports in gnt-backup

Reviewed-by: iustinp

Use new query RPC call in “gnt-instance list”

Reviewed-by: iustinp

Fix cluster destroy

With the recent startup/shutdown changes (and with the master daemon in
place), the cluster destroy needs some fixing.

This patch moves the finalization of the destroy out from cmdlib into
bootstrap, so we can nicely shutdown the rapi and master daemons....

Rework master startup/shutdown/failover

This (big) patch reworks the master startup/shutdown and the fixes the
master failover.

What does the patch do?

For master start/stop:
- remove the old ganeti-master script and its associated man page
- moves the ip start/stop directly into the backend.(Start|Stop)Master...

Make “gnt-debug delay” work again

The old API is no longer working.

Reviewed-by: ultrotter

gnt-job: Don't treat job IDs as numbers

Reviewed-by: iustinp

Make "gnt-job list" work again

"gnt-job list" was broken after my recent changes in the RPC
between clients and the master. This patch makes it work again.

Reviewed-by: iustinp

Implement “gnt-job list -o +...”

This adds the same “-o +...” functionality in gnt-job as in the node and
instance scripts.

Reviewed-by: imsnah

AddNode: Check for node existance

In the "new world" we'll need to setup ganeti-noded via ssh on the node
before calling the AddNode opcode. Before doing it we'll check that the
node is not already in the cluster, if --readd was not passed. This
guarantees we're not going to restart ganeti-noded on a running node....

Fix gnt-cluster “command” and “copyfile”

Since the disabling of forking in the master daemon, the two ssh-based
subcommands were not working anymore. However, there is no need at all
for the commands to be run from the master daemon (permissions to read
the cluster private ssh key notwithstanding), they can be run directly...

Use a single Makefile.am instead of many

This change allows us to use cleaner dependencies between
directories. The build system is basically rewritten in large parts
and may contain bugs.

Reviewed-by: iustinp