Check for tabs and long lines in Python code

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

.gitignore: Ignore distcheck directories

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Wrap lines over 80 characters

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

cmdlib: Fix typo in variable name

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Makefile: Move lengthy variable list to separate variable

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Move LoadModule function to ganeti.build

It should only be used at build-time.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add new ganeti.build namespace

This will be used to store build-time Python code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

upload: Use more compatible regular expression

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

mcpu: formatting/indenting fix

Small fix for a mistake done by bad editor settings.

Signed-off-by: Luca Bigliardi <shammash@google.com>

Add bash completion for --node node1[:node2]

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Add more debugging to bash completion

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Add simple unittest for manpages

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Fix small error in gnt-job manpage

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Add locking doc to Ganeti documentation

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Add ipsec hook to .gitignore

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Even more style fixes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

KVM hypervisor: Use ReadFile

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

A few style fixes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Add RAPI resource to redistribute config

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Add simple unittest for remote API docs

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

rapi doc: Add …/nodes/[name] and …/instances/[name]/reinstall

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Add simple unittest for hooks documentation

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Update hooks documentation

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Move LoadModule function to utils

It can be used by unittests for daemons/* or scripts/*.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

utils.CalculateDirectorySize: Don't redefine builtin

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Use ReadFile/WriteFile in more places

This survived QA, burnin and unittests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Makefile: add IPsec hook

Enable creation and dist for IPsec hook.

Signed-off-by: Luca Bigliardi <shammash@google.com>

Example IPsec hook

This hook is a simple example which automatically configures a basic IPsec link
among all the nodes of a cluster.

Signed-off-by: Luca Bigliardi <shammash@google.com>

gnt-cluster watcher: Show more information

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

confd: add cluster master query

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

confd/querylib: fix docstrings

They were cut&paste from the first example queries, rather than
documenting the current ones.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

confd: avoid spamming the logfile

When confd is disabled we don't want to be noticed every timer interval.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Remove SimpleConfigReader.GetHypervisorType

We haven't had that config file entry for a while.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Merge commit 'origin/next' into branch-2.1

* commit 'origin/next':
  Return cluster tags from LUQueryClusterInfo

Conflicts:
lib/cmdlib.py: Trivial

Replace BINDIR variable as well

Add bindir to the list of variables replaced at make time.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Return cluster tags from LUQueryClusterInfo

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

devel/upload: Build in parallel

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Merge branch 'next' into branch-2.1

* next:
  Add script to clean archived jobs after 21 days
  rapi: export more static node information
  Pass the correct signal to handlers
  cli: Use ToStdout/ToStderr instead of print
  Fix small typo in gnt-node
  Simplify handling of boolean args in rapi
  Fix checks in LUSetNodeParms for the master node
  Improve the example startup script
  Fix insserv dependencies

Conflicts:
Makefile.am: added ganeti-cleaner
doc/examples/ganeti.initd.in: Iustin's init script improvements
lib/cli.py: Trivial (ToStderr changes)

Add more bash completion metadata for options

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Bash completion: Implement dynamic option value completion

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Bash completion: Add function to get all iallocators

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Bash completion: Simplify option completion

This makes the output file quite a bit smaller and is needed
in preparation for non-static option completion (e.g. list of
nodes).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Bash completion: Support for --foo=bar option format

That is with the equal sign.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Bash completion: Small optimizations

$2 doesn't contain the correct value when completing something like
"--disk-template=…". Getting it via COMP_WORDS is better.

Short options (e.g. -I) can't have an equal sign.

Also add useful debugging commands for development.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Fix QA test for “gnt-node evacuate”

Before the arguments weren't valid.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Confd: don't fail if the config doesn't load

Rather than quitting we'll just continue to poll the config at a slow
rate, hoping that sooner or later we'll get it back. This allows also
working on non-MC nodes, and smoothly transitioning from MC to non-MC,
other than dealing with a few temporary breakages.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

confd: start in polling mode

This allows us not to enable the inotify handler immediately, and thus
to make it easier for us should the config file not exist at all.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

ConfdProcessor: add disabled state

This is a state the processor will get in, if it fails to load the
config.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Move SimpleConfigReader creation to ConfdProcessor

This will be useful to make ConfdProcessor aware of a config failure,
without quitting confd.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

confd: s/confd_event_handler/inotify_handler/

In a case we don't encounter frequently (file modified but not
overwritten) the notify handler we use is called with a wrong name.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add script to clean archived jobs after 21 days

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Implement timers in confd

Timers are used both for checking for inotify failures, and for polling,
should inotify notices become too frequent.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

constants: add confd reload and rate limit times

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

ConfdInotifyEventHandler.enable: use InotifyError

Rather than raising ConfdFatalError directly
ConfdInotifyEventHandler.enable raises InotifyError should it not be
able to configure inotify, allowing the caller to decide what to do.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add errors.InotifyError

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

ConfdInotifyEventHandler, move to a callback

ConfdInotifyEventHandler used to reload the config whenever a
notification arrived. Moving to a callback system, so that
ConfdConfigurationReloader can be responsible for that functionality.

Additionally the inotify class no longer reenables itself automatically,
but just notifies the callback if it's been disabled, and waits for its
call to enable(). This allows, should ConfdConfigurationReloader decide
it wants to move to polling, to avoid having a double enable()/disable()
call.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move creation of inotify handler to a new class

This class will be responsible for managing inotify notifications,
timers, and rate-limiting reloads. For now none of these features is
implemented. :)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

ConfdInotifyEventHandler: add enable/disable

Make possible to enable and disable the inotify event handler.  The
inotify handler will remain enabled, unless explicitely told to disable
itself.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

SimpleConfigReader: handle stat() errors

Also, catch EnvironmentError, rather than IOError, when trying to read
the file.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Encode the actual exception raised by LU execution

Currently, the actual exception raised during an LU execution (one of
OpPrereqError, OpExecError, HooksError, etc.) is lost because the
jqueue.py code simply sets that to a str(err), and the code in cli.py
simply passes that string to OpExecError.

This patch moves to encoding the errors as per errors.EncodeError and
changes the cli code to parse and raise that (if possible).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the luxi error handling into errors.py

Currently the luxi error handling is hardcoded as special encoding on
the masterd-side and special decoding on the client side. This patch
moves it to errors.py such that other parts of the code can reuse the
same encoding.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Remove watcher pause file 1h past end time

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

ganeti-watcher: Don't run if paused

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

gnt-cluster: Add CLI for watcher pause

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add file to pause watcher for a certain duration

This can be used during maintenance work.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

cli._Argument: Remove unused, optional “suggest” parameter

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

bash completion: Allow arguments with min < max arguments

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

ganeti-masterd: Master voting in separate process

One shouldn't fork a Python process after using threads. Master
voting is done before forking (utils.Daemonize), but it also uses
threads. Hence it's now called from a separate process.

This patch also fixes the check function to actually exit if
the current machine is not the master (introduced in 04ccf5e9).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

ganeti-masterd: Add helper to run function in separate process

This will be used to do the master voting.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

GenerateSecret: specify the length of the secret

Add a parameter to specify the length of the bytes sequence represented by the
returned hex string. Defaulting to 20 so it's compatible with previous SHA1
hash.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

cmdlib: Add some messages to LUExportInstance

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Style fixes for ganeti-*

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

ganeti-noded: Close listening socket in child

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Document cluster destroy hook

Add documentation about new cluster-destroy-post hook.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LUDestroyCluster: run hooks

Run post hooks on master node before it's removed.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LUDestroyCluster: hooks support

We're going to call hooks from its Exec method so LUDestroyCluster has to
support them.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Run unittests in a temporary directory

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix building in a different directory

Also add build-bash-completion to EXTRA_DIST.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add script to run build commands in temporary directory

Python always compiles imported modules. By running these targets
in a temporary directory we don't pollute the source tree.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Makefile: Use BUILT_SOURCES

These are always built before anything else.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Simplify manpage building

This eliminates one temporary directory in the process.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Makefile: Fix docbook2man error message

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Handle more errors in bash completion

Don't print error messages if cluster hasn't been initialized yet.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

HooksMaster: fix RunPhase logging

In case of complete failure results is empty, return immediately
(tnx unittests).

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

LURemoveNode: no logs running post on removed node

Do not log results from hooks ran on the removed node as now it's done by
RunPhase.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

HooksMaster: logging hooks in RunPhase

Extend RunPhase so it will log hooks results in POST phase.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

rapi: export more static node information

Currently not all static node information is exported over rapi, and
this makes it less complete than gnt-node list. This patch attempts to
reduce the difference.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Document post cluster initialization LU

Update hooks documentation with the new opcode (not deleting OP_INIT_CLUSTER
from obsolete operations because the name is different).

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

node-remove post on removed node

Run post phase of node-remove on the removed node as well.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

HooksMaster: document raised exception

HooksAbort is raised, but not documented.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

HooksMaster: list of nodes override

Allow the caller of HooksMaster.RunPhase() to specify an alternative list of
nodes.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add burnin support for instance moves

This patch adds support for instance moves in burnin. This means that
non-drbd instances finally get support for being moved.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fix error output in LUSetClusterParams

Before:
  gnt-cluster modify --enabled-hypervisors=xen-hvm,xem-pvm
  Failure: command execution error:
  Enabled hypervisors contains invalid entries: set([u'xem-pvm'])

After:
  gnt-cluster modify --enabled-hypervisors=xen-hvm,xem-pvm
  Failure: command execution error:
  Enabled hypervisors contains invalid entries: 'xem-pvm'

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add ArgHost class

It expands to the hostnames known by bash.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Re-introduce constants for most often used arguments

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Use script to generate bash completion

Completion for tools/burnin is not yet implemented. It needs to be
converted to definition-based options handling first.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add tools directory to constants

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Convert “burnin” to definition-based options

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add more auto-completion metadata

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Make all options use cli_option

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>