Synnefo » snf-ganeti » ganeti-local

Instance parameters: force typing

We want all the hv/be parameters to have a known type, rather than a
random mix of empty string, boolean values, and None, so we declare the
type of each variable and we enforce/convert it.

- Add some new constants for enforceable value types...

Allow query of the drained node attribute

This patch exports the drained attribute:
- LUQueryNodes accepts now the drained field
- RAPI exports it for node objects
- gnt-node info shows it now (along newly-added master_candidate and
offline flags)...

Export the cpu nodes and sockets from Xen

This is a hand-picked forward patch of commit 1755 on the 1.2 branch
(hand-picked since the trees diverged too much since then):

The patch changed the xen hypervisor to compute the number of cpu
    sockets/nodes and enables the command line and the RAPI to show this...

Enable lockless node queries

Similar to the instance list, this patch enables lockless node queris.
“gnt-node list” accepts now the “--sync” flag which enables locking, the
default is lockless.

Reviewed-by: imsnah

Implement lockless query operations

This patch adds the framework for, and enables lockless OpQueryInstances. This
means that instances will be shown in ERROR_up or ERROR_down state, even though
this is not an error (but just an in-progress job).

The framework is implemented as follows:...

Add gnt-node migrate

This is a (modified) forward-port of commit 1190 on the 1.2 branch:

This is the same as gnt-node failover, and is also a cut&paste of its
  code (almost). It will be really really useful to quickly empty a
  healthy node. I can be persuaded to merge MigrateNode and FailoverNode...

gnt-instance: support no_PARAMETER value

Since parameters get set to False if a no_ is prefixed don't try to
interpret those boolean values, and pass them unchanged.

Reviewed-by: iustinp

Check that instance exists before confirm. queries

Currently we ask the user for confirmation, and only after (try to)
remove, failover or migrate the instance. This doesn't work nicely if
the instance doesn't exist, so we make a query for the instance before...

Sort the instance names in batcher

In case we submit multiple instances via batcher, it's nicer to have the
sorted nicely.

Reviewed-by: imsnah

Make iallocator work with offline nodes

This patch changes the iallocator framework to work with and properly
export to plugins offline nodes. It does this by only exporting the
static configuration data for those nodes, and not attempting to parse
the runtime data....

Fix a couple of epydoc warnings

Reviewed-by: ultrotter

Forward port the live migration from 1.2 branch

This is forward port via copy (and not individual patches cherry-pick)
of the latest code on the 1.2 branch related to the migration.

The changes compared to 1.2 are the fact that we don't need the
IdentifyDisks step anymore (the drbd rpc calls are independent now), and...

Heavy redo of gnt-instance info output

In 2.0, we have more parameters in drbd's logical_id, and passing the
results over json makes them unicode which looks worse with the default
formatting. As such, a redo of the output is needed.

This patch:
- adds a separate function to format the logical_id of devices...

Cleanup replace-disks modes and options

In 1.2, due to the md+drbd7 legacy, we had a complex choice of replace
modes, and the new drbd8 modes where forced into this syntax, with some
complicated rules of transition from one mode to another (if REPLACE_ALL...

Fix epydoc format warnings

This patch should fix all outstanding epydoc parsing errors; as such, we
switch epydoc into verbose mode so that any new errors will be visible.

Reviewed-by: imsnah

node list: add the offline field

Reviewed-by: ultrotter

Extract the ListNodes headers and use them in help

Currently we have to update both the ListNodes headers and the online
help for the full field list. This patch uses the headers keys for the
help, thus removing duplicating places to update, and adding hope that...

SetInstanceParams: handle default/none values

If the hv/be parameter lowercase value is set to "default" we'll pass
constants.VALUE_DEFAULT, if it's set to "none" we'll pass
constants.VALUE_NONE.

Reviewed-by: imsnah

Add cluster candidate pool size parameter

This patch adds a new cluster paramater "candidate_pool_size" which
tracks the desired size of the list of nodes with the master_candidate
flag set.

Reviewed-by: imsnah

Add master/master_candidate fields to node list

This patch adds listing of the master_candidate field (as Y/N) and of
the master role (again Y/N) for nodes.

Reviewed-by: imsnah

gnt-job: Print message from CancelJob to standard output

Reviewed-by: iustinp

Improve the node add operation

Currently, the node add operation uses a job to query the node name and
the bootstrap function directly reads the config file for the cluster
name.

This patch changes to that both the cluster name and the verification of
the node is done via queries to the master....

Allow network-less instances

Currently there's no way to specify no NICs for an instance, even though
this is a supported configuration.

The patch adds a --no-nics option to gnt-instance add.

Reviewed-by: amishchenko

Fix instance creation

This patch fixes the diskless and drbd/file based instances. Sorry :(

Reviewed-by: ultrotter

Show disk access mode in gnt-instance info

The mode parameter needs to be exported and shown in the info output.

Reviewed-by: imsnah

Enable auto-unit formatting in script output

This patch enables by default the old 'human-readable' option, but in a
slightly different model.

The option is now called "units" and takes either:
- 'h' for automatic formatting
- 'm', 'g' or 't' for mebi/gibi/tebibytes...

Make cli.py use FieldSet for matching fields

This changes cli.py to FieldSet usage so that gnt-instance list will
format nicely the disk.size/*, and the count of disks/nics.

Reviewed-by: ultrotter

Reuse HTTP client pool for RPC

ganeti-masterd: Add initialization and shutdown of RPC pool. It needs
to be shutdown before forking.

ganeti.cli: Add decorator function to initialize and shutdown RPC pool.

ganeti.rpc: Add functions to initialize and shutdown RPC pool. Throw...

Initial multi-disk/multi-nic support

This patch adds support for mult-disk/multi-nic in:
- instance add
- burnin

The start/stop/failover/cluster verify work as expected. Replace disk
and grow disk are TODO.

There's also a change gnt-job to allow dictionaries to be listed in...

Documentation updates for gnt-debug and gnt-os

Reviewed-by: imsnah

Documentation updates for gnt-backup

Reviewed-by: imsnah

Documentation updates for gnt-node

The patch also adds a 'return 0' to a function that was missing it.

Reviewed-by: imsnah

Documentation updates for gnt-instance

Reviewed-by: imsnah

convert remaining print statements to ToStderr in gnt-cluster

Reviewed-by: imsnah

Add mac option to gnt-backup import

Before 'auto' was the only allowed possibility

Forward-port-of: r1885, Reviewed-by: iustinp

Reviewed-by: imsnah

Convert the gnt scripts to ToStdout/err

Currently the gnt-* scripts are using a mix of print, logger.ToStd* and
sys.stderr.write. We convert them all to using cli.ToStdout/err. This
way, we can easily change the implementation for all at once.

Reviewed-by: imsnah

Remove --hypervisor-type from gnt-cluster.

We no longer use a single, cluster-wide hypervisor, but configure the
actual to be used hypervisor on the instance level.

Reviewed-by: imsnah

Enable gnt-cluster modify to hv/beparams

This patch enables the cluster modify to change:
- enabled hypervisor list
- hvparams (per hypervisor)
- beparams (only the default group)

Syntax:
gnt-cluster modify -B vcpus=3 -H xen-pvm:no_initrd_path

Validation for parameters is somewhat missing - the individual...

Add an interface for the drain flag changes/query

This adds the set/reset in the jqueue and luxi modules, and a way to
query it in OpQueryConfigValues, and also the comand line interface for
it:
$ gnt-cluster queue info
The drain flag is unset
$ gnt-cluster queue drain...

gnt-backup: update for cluster parameters

- add backend and hypervisor parameters
- fix beparams validation/passing
- pass hypervisor and hvparams
- remove deprecated flags

Reviewed-by: iustinp

gnt-instance remove deprecated flags

kernel, initrd, hvm_boot_order and vnc_bind_address are now hypervisor
parameters and should not have their own flag. Moreover querying of
vnc_bind_address should of course pass through the hv/ namespace.

Reviewed-by: iustinp

Some fixes related to auto_balance

Change the constant name to match the value (autobalance ->
auto_balance).

Also add the auto_balance header so that gnt-instance can list it.

Reviewed-by: ultrotter

Allow instance info to only query the config file

This patch adds a new '-s' parameter to ‘gnt-instance info’ that makes
it return only 'static' information. This is much faster, especially for
drbd instances.

This is a forward-port of rev 1570 on the ganeti-1.2 branch, resending...

Change gnt-instance modify to the hvparams model

Reviewed-by: imsnah

Switch instance hypervisor parameters to hvparams

This big patch changes instance create to the new hvparams structure.
Old parameters are removed, so old jobs or old instances file will break
current clusters.

Reviewed-by: ultrotter

Move the hypervisor attribute to the instances

This (big) patch moves the hypervisor type from the cluster to the
instance level; the cluster attribute remains as the default hypervisor,
and will be renamed accordingly in a next patch. The cluster also gains...

Implement job 'waiting' status

Background: when we have multiple jobs in the queue (more than just a
few), many of the jobs (up to the number of threads) will be in state
'running', although many of them could be actually blocked, waiting for
some locks. This is not good, as one cannot easily see what is...

Change SshRunner usage

Currently the SshRunner uses a SimpleConfigReader instance, however this
is not best. We change it to use the cluster name directly (and its
constructor now takes this as parameter, instead of SCR), and its
callers are change to pass the name directly....

Convert ssh.py

Get rid of ssconf and convert to configuration instead.

Reviewed-by: iustinp

Add output of job/opcode timestamps

This patch adds posibility of selection of job/opcode timestamps in
gnt-job list and info.

The code handling the possible cases (None or a valid timestamps) are
ugly though...

Reviwed-by: imsnah