ganeti-local
14 years agoUnify the “--backend-parameters” option
Iustin Pop [Thu, 17 Sep 2009 09:21:31 +0000 (11:21 +0200)]
Unify the “--backend-parameters” option

This is not straight-forward. First, gnt-cluster had
“--backend-parameters” wheres the rest of the script had just
“--backend”. I settled on the full form, since one can always abbreviate
options, but not vice-versa.

Second, the help in some cases was different, e.g. start instance
had “Temporary backend parameters” and modify had “Change backend
parameters”. I think this kind of distinctions are better served in the
man page. Alternatively, if we could copy options easily, one could
re-instantiate them with a changed help line.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoUnify the “--os-type” option
Iustin Pop [Thu, 17 Sep 2009 09:11:22 +0000 (11:11 +0200)]
Unify the “--os-type” option

This was already unified inside gnt-instance, I just moved that
definition (and upper-cased it) into cli.py.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoUnify the “--iallocator” option
Iustin Pop [Thu, 17 Sep 2009 09:07:23 +0000 (11:07 +0200)]
Unify the “--iallocator” option

This patch also changes the gnt-debug's “--algorithm” into
IALLOCATOR_OPT, since it was the only non-standard use of this (due to
historical reasons). Since it's just a debug option,
backwards-compatibility is not critical here.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoUnify the “--file-driver” option
Iustin Pop [Thu, 17 Sep 2009 09:04:00 +0000 (11:04 +0200)]
Unify the “--file-driver” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoUnify the “--file-storage-dir” option
Iustin Pop [Thu, 17 Sep 2009 09:02:07 +0000 (11:02 +0200)]
Unify the “--file-storage-dir” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoUnify the “--no-nics” option
Iustin Pop [Thu, 17 Sep 2009 08:59:55 +0000 (10:59 +0200)]
Unify the “--no-nics” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoUnify the “--disk-template” option
Iustin Pop [Thu, 17 Sep 2009 08:57:46 +0000 (10:57 +0200)]
Unify the “--disk-template” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoUnify the “--no-wait-for-sync” option
Iustin Pop [Thu, 17 Sep 2009 08:57:03 +0000 (10:57 +0200)]
Unify the “--no-wait-for-sync” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoExplain the reason for fqdn hostnames
Guido Trotter [Thu, 17 Sep 2009 20:58:23 +0000 (21:58 +0100)]
Explain the reason for fqdn hostnames

The install guide asks the user to use the fqdn in the hostname, but
doesn't explain why. This has been asked multiple times, so adding an
explanation there.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agoRemove RpcResult.RemoteFailMsg completely
Michael Hanselmann [Wed, 16 Sep 2009 16:16:05 +0000 (18:16 +0200)]
Remove RpcResult.RemoteFailMsg completely

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoAdd locking improvements to Ganeti 2.1 design document
Michael Hanselmann [Wed, 16 Sep 2009 16:10:12 +0000 (18:10 +0200)]
Add locking improvements to Ganeti 2.1 design document

Also increase the table of contents' depth to 4.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoAdd an error-simulation mode to cluster verify
Iustin Pop [Thu, 17 Sep 2009 08:33:48 +0000 (10:33 +0200)]
Add an error-simulation mode to cluster verify

One of the issues we have in ganeti is that it's very hard to test the
error-handling paths; QA and burnin only test the OK code-path, since
it's hard to simulate errors.

LUVerifyCluster is special amongst the LUs in the fact that a) it has a
lot of error paths and b) the error paths only log the error, they don't
do any rollback or other similar actions. Thus, it's enough for this LU
to separate the testing of the error condition from the logging of the
error condition.

This patch does this by replacing code blocks of the form:

  if x:
    log_error()
    [y]

into:

  log_error_if(x)
  [if x:
    y
  ]

After this change, it's simple enough to turn on logging of all errors
by adding a special case inside log_error_if such that if the incoming
opcode has a special ‘debug_simulate_errors’ attribute and it's true, it
will log unconditionally the error.

Surprisingly this also turns into an absolute code reduction, since some
of the if blocks were simplified. The only downside to this patch is
that the various _VerifyX() functions are now stateful (modifying an
attribute on the LU instance) instead of returning a boolean result.

Last note: yes, this discovered some error cases in the logging.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoIntroduce parseable error codes in LUVerifyCluster
Iustin Pop [Wed, 16 Sep 2009 09:55:07 +0000 (11:55 +0200)]
Introduce parseable error codes in LUVerifyCluster

Currently the output of cluster verify can be parsed for 'ERROR'
messages, but that is the only indication we get (error or no error). In
order to allow monitoring tools to separate different error conditions,
this patch introduces a new output format (“gnt-cluster verify
--error-codes”) that changes the output from human-friendly to
machine-friendly. In this mode, an error line changes from:
  ERROR: node node1: drbd minor 1 of instance inst1.is not active

to:
  ERROR:ENODEDRBD:node:node1:drbd minor 1 of instance inst1 is not active

i.e. the error message is a ‘:’-separated field, with ERROR in the first
place, the error code in the second, the object type (cluster, node,
instance) in the third, the name of the object (for nodes/instances) in
the fourth, and then the text message.

The patch also removes some of the verbosity of the operation
(“Verifying instance X”, “Verifying node X”) since on big clusters these
informational messages can quickly fill up an entire screen. The
original behaviour can be restored via the ‘--verbose’ option.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoUnify the multiple 'verbose' options
Iustin Pop [Tue, 15 Sep 2009 09:42:51 +0000 (11:42 +0200)]
Unify the multiple 'verbose' options

Currently multiple files define a 'verbose' option. This patch moves all
these definitions to a single VERBOSE_OPT option which is exported from
cli.py.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoRewrite the lib/cli.py export list
Iustin Pop [Tue, 15 Sep 2009 09:40:02 +0000 (11:40 +0200)]
Rewrite the lib/cli.py export list

Right now, the export list is a mess, it's not sorted, and the exported
items are in random places. This patches does grow the length of the
list, but at least it's more clear.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoShow "--help" in bash completion
Michael Hanselmann [Wed, 16 Sep 2009 15:39:03 +0000 (17:39 +0200)]
Show "--help" in bash completion

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoRemove RpcResult.failed attribute
Michael Hanselmann [Wed, 16 Sep 2009 13:15:22 +0000 (15:15 +0200)]
Remove RpcResult.failed attribute

It's redundant with RpcResult.fail_msg. This change survived QA, burnin and
unittests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoConfd: add primary IPs queries
Luca Bigliardi [Wed, 16 Sep 2009 13:19:18 +0000 (14:19 +0100)]
Confd: add primary IPs queries

Extend confd to answer queries about nodes primary IPs and master candidates
primary IPs.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoganeti-confd: remove partial imports
Guido Trotter [Wed, 16 Sep 2009 11:00:03 +0000 (12:00 +0100)]
ganeti-confd: remove partial imports

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoConfd client library
Guido Trotter [Tue, 15 Sep 2009 16:33:23 +0000 (17:33 +0100)]
Confd client library

Initial confd client library implementation.  This initial version uses
asyncore, and supports answers via a callback.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoMove fourcc packing/unpacking to main confd module
Guido Trotter [Tue, 15 Sep 2009 12:50:19 +0000 (13:50 +0100)]
Move fourcc packing/unpacking to main confd module

This way it can be used by the client as well

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoAdd unittests for confd constants
Guido Trotter [Tue, 15 Sep 2009 11:20:40 +0000 (12:20 +0100)]
Add unittests for confd constants

Checking that the fourcc code is indeed a fourcc code, and that request
codes and reply statuses are distinct.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoganeti-confd: explicitely log failed big sends
Guido Trotter [Mon, 14 Sep 2009 16:13:45 +0000 (17:13 +0100)]
ganeti-confd: explicitely log failed big sends

Make sure that if we try to send packages which are too big (which
shouldn't happen) this gets properly logged in the config file.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoAdd a magic fourcc code to confd packets
Guido Trotter [Mon, 14 Sep 2009 15:59:03 +0000 (16:59 +0100)]
Add a magic fourcc code to confd packets

This will make it easier to change the protocol later on

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoAsyncUDPSocket: Move to a well defined UDP size
Guido Trotter [Mon, 14 Sep 2009 15:42:33 +0000 (16:42 +0100)]
AsyncUDPSocket: Move to a well defined UDP size

Currently we read maximum 4K packets, and don't check packets when
sending them. With this patch we move to a well defined maximum size of
60K.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoConfdAsyncUDPServer: fix a docstring
Guido Trotter [Mon, 14 Sep 2009 14:54:37 +0000 (15:54 +0100)]
ConfdAsyncUDPServer: fix a docstring

It refers to an older input variable

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoAbstract AsyncUDPSocket to daemon
Guido Trotter [Mon, 14 Sep 2009 13:13:17 +0000 (14:13 +0100)]
Abstract AsyncUDPSocket to daemon

This allows this extended asyncore+udp module to be used also in other
daemons, and in the confd client library

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoConfdAsyncUDPServer: defer handling writes
Guido Trotter [Mon, 14 Sep 2009 10:49:49 +0000 (11:49 +0100)]
ConfdAsyncUDPServer: defer handling writes

Currently if we fail writing to the socket (perhaps because a signal was
delivered) we lose the data we were sending. Although this is not too
bad (it's udp, and data may get lost anyway) we try to avoid this by
just putting the packets to write in a queue and handling that through
the normal asyncore methods.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoConfdAsyncUDPServer: handle signals at read time
Guido Trotter [Mon, 14 Sep 2009 10:40:36 +0000 (11:40 +0100)]
ConfdAsyncUDPServer: handle signals at read time

Currently if a signal is delivered during an attempted read, an
exception is logged in the logfile. There is no need for this, so we
handle this case explicitely.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoFix authorized_keys generation at cluster init
Luca Bigliardi [Wed, 16 Sep 2009 12:37:09 +0000 (13:37 +0100)]
Fix authorized_keys generation at cluster init

Copy pub_key in authorized_keys.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoMove developer notes into normal documentation
Michael Hanselmann [Tue, 15 Sep 2009 12:14:23 +0000 (14:14 +0200)]
Move developer notes into normal documentation

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agojqueue: Remove unused run_op_index attribute
Michael Hanselmann [Mon, 14 Sep 2009 15:51:25 +0000 (17:51 +0200)]
jqueue: Remove unused run_op_index attribute

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoExport new lock_status field to gnt-job
Michael Hanselmann [Tue, 15 Sep 2009 11:09:31 +0000 (13:09 +0200)]
Export new lock_status field to gnt-job

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoKeep lock status with every job
Michael Hanselmann [Tue, 15 Sep 2009 11:00:41 +0000 (13:00 +0200)]
Keep lock status with every job

This can be useful for debugging locking problems.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoMove OpCode processor callbacks into separate class
Michael Hanselmann [Tue, 15 Sep 2009 11:02:56 +0000 (13:02 +0200)]
Move OpCode processor callbacks into separate class

There are two major arguments for this:
- There will be more callbacks (e.g. for lock debugging) and extending the
  parameter list is a lot of work.
- In the jqueue module this allows us to keep per-job or per-opcode variables in
  a separate class. Instead of having to clean up the worker class after
  processing one job, these references will automatically go out of scope.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoReplace last hardcoded 'nodelist' with NV_NODELIST
Iustin Pop [Mon, 14 Sep 2009 11:32:54 +0000 (13:32 +0200)]
Replace last hardcoded 'nodelist' with NV_NODELIST

LUAddNode has this usage remaining, as it's the only other LU to use
call_node_verify (beside LUVerifyCluster).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoTreat virtual LVs as inexistent
Iustin Pop [Mon, 14 Sep 2009 11:25:18 +0000 (13:25 +0200)]
Treat virtual LVs as inexistent

Currently, “gnt-cluster verify” and “gnt-cluster verify-disks” use the
list of LVs as returned by backend.GetVolumeList to determine whether an
LV exists or not. However, LVs can also be ‘virtual’, which is handled
correctly (i.e. as missing) by the bdev code, but not by this function.

This patch changed GetVolumeList to simply skip virtual LVs; this makes
cluster verify and verify-disks report these correctly as missing. The
only downside is that an user could get confused (lvs reports the volume
as existing, but ganeti as missing). However, this is better than simply
considering virtual LVs as “good”.

No other code beside these two gnt-cluster operations uses the
GetVolumeList function, so we don't change the behaviour of the rest of
the code (e.g. replace-disks, instance info, etc.).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agolocking: Acquire SharedLock in shared mode in separate function
Michael Hanselmann [Mon, 14 Sep 2009 11:05:21 +0000 (13:05 +0200)]
locking: Acquire SharedLock in shared mode in separate function

This is for some more symetry with
SharedLock.__exclusive_acquire.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoRpcResult: Fix cases where “failed” wouldn't be set to True
Michael Hanselmann [Wed, 9 Sep 2009 15:51:35 +0000 (17:51 +0200)]
RpcResult: Fix cases where “failed” wouldn't be set to True

This broke “gnt-instance replace-disks --auto” when the instance
is down.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agolocking: Add level name dict
Michael Hanselmann [Mon, 7 Sep 2009 16:40:53 +0000 (18:40 +0200)]
locking: Add level name dict

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agoMerge commit 'origin/next' into branch-2.1
Michael Hanselmann [Mon, 14 Sep 2009 10:26:17 +0000 (12:26 +0200)]
Merge commit 'origin/next' into branch-2.1

* commit 'origin/next':
  Fix gnt-node modify online help
  Fix gnt-job info entry in gnt-job(8)
  locking: Don't swallow exceptions
  Add check for duplicate MACs in instance add
  scripts/gnt-node: fix a help string
  Optimise multi-job submit
  Extend gnt-debug with more debugging options

Conflicts:
scripts/gnt-debug: Trivial
scripts/gnt-node: Trivial

14 years agoFix gnt-node modify online help
Guido Trotter [Mon, 14 Sep 2009 10:12:39 +0000 (11:12 +0100)]
Fix gnt-node modify online help

This completes what was began in commit
862b1b2b4c698cec8ad816793172fac9aa7f4cdf

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoFix gnt-job info entry in gnt-job(8)
Guido Trotter [Fri, 11 Sep 2009 12:33:07 +0000 (13:33 +0100)]
Fix gnt-job info entry in gnt-job(8)

It currently reports "cancel" in its syntax, which is obviously a
cut&paste error.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agolocking: Don't swallow exceptions
Michael Hanselmann [Fri, 11 Sep 2009 14:28:13 +0000 (16:28 +0200)]
locking: Don't swallow exceptions

This is an indentation bug.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agocheck-python-code: Report EOL whitespace
Michael Hanselmann [Fri, 11 Sep 2009 12:31:45 +0000 (14:31 +0200)]
check-python-code: Report EOL whitespace

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoCheck for tabs and long lines in Python code
Michael Hanselmann [Fri, 11 Sep 2009 11:01:26 +0000 (13:01 +0200)]
Check for tabs and long lines in Python code

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years ago.gitignore: Ignore distcheck directories
Michael Hanselmann [Fri, 11 Sep 2009 12:35:49 +0000 (14:35 +0200)]
.gitignore: Ignore distcheck directories

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoWrap lines over 80 characters
Michael Hanselmann [Fri, 11 Sep 2009 12:28:00 +0000 (14:28 +0200)]
Wrap lines over 80 characters

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agocmdlib: Fix typo in variable name
Michael Hanselmann [Fri, 11 Sep 2009 10:06:14 +0000 (12:06 +0200)]
cmdlib: Fix typo in variable name

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoMakefile: Move lengthy variable list to separate variable
Michael Hanselmann [Thu, 10 Sep 2009 16:21:55 +0000 (18:21 +0200)]
Makefile: Move lengthy variable list to separate variable

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoMove LoadModule function to ganeti.build
Michael Hanselmann [Thu, 10 Sep 2009 15:42:27 +0000 (17:42 +0200)]
Move LoadModule function to ganeti.build

It should only be used at build-time.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoAdd new ganeti.build namespace
Michael Hanselmann [Thu, 10 Sep 2009 15:41:41 +0000 (17:41 +0200)]
Add new ganeti.build namespace

This will be used to store build-time Python code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoupload: Use more compatible regular expression
Michael Hanselmann [Mon, 7 Sep 2009 17:29:14 +0000 (19:29 +0200)]
upload: Use more compatible regular expression

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agomcpu: formatting/indenting fix
Luca Bigliardi [Wed, 9 Sep 2009 09:46:25 +0000 (10:46 +0100)]
mcpu: formatting/indenting fix

Small fix for a mistake done by bad editor settings.

Signed-off-by: Luca Bigliardi <shammash@google.com>

14 years agoAdd check for duplicate MACs in instance add
Iustin Pop [Tue, 8 Sep 2009 12:29:26 +0000 (14:29 +0200)]
Add check for duplicate MACs in instance add

Currently LUAddInstance doesn't check for duplicate MACs, and it fails
during the Exec() phase when trying to add the instance to the config
(ConfigWriter checks for this). This patch copies the code from
LUModifyInstance (which already does it).

This fixed issue #70.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agoscripts/gnt-node: fix a help string
Guido Trotter [Mon, 7 Sep 2009 13:11:58 +0000 (14:11 +0100)]
scripts/gnt-node: fix a help string

gnt-node modify says it wants an "<instance>". Changing it to
"<node_name>" as for the other commands.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoOptimise multi-job submit
Iustin Pop [Mon, 7 Sep 2009 13:02:07 +0000 (15:02 +0200)]
Optimise multi-job submit

Currently, on multi-job submits we simply iterate over the
single-job-submit function. This means we grab a new serial, write and
replicate (and wait for the remote nodes to ack) the serial file, and
only then create the job file; this is repeated N times, once for each
job.

Since job identifiers are ‘cheap’, it's simpler to simply grab at the
start a block of new IDs, write and replicate the serial count file a
single time, and then proceed with the jobs as before. This is a cheap
change that reduces I/O and reduces slightly the CPU consumption of the
master daemon: submit time seems to be cut in half for big batches of
jobs and the masterd cpu time by (I can't get consistent numbers)
between 15%-50%.

Note that this doesn't change anything for single-job submits and most
probably for < 5 job submits either.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoExtend gnt-debug with more debugging options
Iustin Pop [Mon, 7 Sep 2009 12:05:44 +0000 (14:05 +0200)]
Extend gnt-debug with more debugging options

This patch extends gnt-debug to be able to submit multiple copies of the
input jobs and job contents, in order to simplify testing. It also adds
a timing mode, and splits the execution into separate submit and
execution stages (for timing purposes).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoAdd bash completion for --node node1[:node2]
Michael Hanselmann [Fri, 4 Sep 2009 15:08:51 +0000 (17:08 +0200)]
Add bash completion for --node node1[:node2]

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoAdd more debugging to bash completion
Michael Hanselmann [Fri, 4 Sep 2009 15:07:28 +0000 (17:07 +0200)]
Add more debugging to bash completion

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoAdd simple unittest for manpages
Michael Hanselmann [Fri, 4 Sep 2009 11:52:00 +0000 (13:52 +0200)]
Add simple unittest for manpages

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoFix small error in gnt-job manpage
Michael Hanselmann [Fri, 4 Sep 2009 11:51:38 +0000 (13:51 +0200)]
Fix small error in gnt-job manpage

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoAdd locking doc to Ganeti documentation
Michael Hanselmann [Fri, 4 Sep 2009 11:29:27 +0000 (13:29 +0200)]
Add locking doc to Ganeti documentation

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoAdd ipsec hook to .gitignore
Michael Hanselmann [Fri, 4 Sep 2009 11:24:07 +0000 (13:24 +0200)]
Add ipsec hook to .gitignore

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoEven more style fixes
Michael Hanselmann [Thu, 3 Sep 2009 16:15:50 +0000 (18:15 +0200)]
Even more style fixes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoKVM hypervisor: Use ReadFile
Michael Hanselmann [Thu, 3 Sep 2009 15:41:41 +0000 (17:41 +0200)]
KVM hypervisor: Use ReadFile

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoA few style fixes
Michael Hanselmann [Thu, 3 Sep 2009 15:40:51 +0000 (17:40 +0200)]
A few style fixes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoAdd RAPI resource to redistribute config
Michael Hanselmann [Thu, 3 Sep 2009 14:05:33 +0000 (16:05 +0200)]
Add RAPI resource to redistribute config

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoAdd simple unittest for remote API docs
Michael Hanselmann [Thu, 3 Sep 2009 10:28:14 +0000 (12:28 +0200)]
Add simple unittest for remote API docs

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agorapi doc: Add …/nodes/[name] and …/instances/[name]/reinstall
Michael Hanselmann [Wed, 2 Sep 2009 11:56:16 +0000 (13:56 +0200)]
rapi doc: Add …/nodes/[name] and …/instances/[name]/reinstall

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoAdd simple unittest for hooks documentation
Michael Hanselmann [Thu, 3 Sep 2009 14:35:09 +0000 (16:35 +0200)]
Add simple unittest for hooks documentation

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoUpdate hooks documentation
Michael Hanselmann [Wed, 2 Sep 2009 15:45:39 +0000 (17:45 +0200)]
Update hooks documentation

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoMove LoadModule function to utils
Michael Hanselmann [Thu, 3 Sep 2009 10:45:04 +0000 (12:45 +0200)]
Move LoadModule function to utils

It can be used by unittests for daemons/* or scripts/*.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoutils.CalculateDirectorySize: Don't redefine builtin
Michael Hanselmann [Wed, 2 Sep 2009 14:28:41 +0000 (16:28 +0200)]
utils.CalculateDirectorySize: Don't redefine builtin

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoUse ReadFile/WriteFile in more places
Michael Hanselmann [Thu, 3 Sep 2009 10:43:13 +0000 (12:43 +0200)]
Use ReadFile/WriteFile in more places

This survived QA, burnin and unittests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

14 years agoMakefile: add IPsec hook
Luca Bigliardi [Mon, 31 Aug 2009 12:34:54 +0000 (13:34 +0100)]
Makefile: add IPsec hook

Enable creation and dist for IPsec hook.

Signed-off-by: Luca Bigliardi <shammash@google.com>

14 years agoExample IPsec hook
Luca Bigliardi [Mon, 10 Aug 2009 16:25:35 +0000 (17:25 +0100)]
Example IPsec hook

This hook is a simple example which automatically configures a basic IPsec link
among all the nodes of a cluster.

Signed-off-by: Luca Bigliardi <shammash@google.com>

14 years agognt-cluster watcher: Show more information
Michael Hanselmann [Mon, 31 Aug 2009 15:59:25 +0000 (17:59 +0200)]
gnt-cluster watcher: Show more information

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

14 years agoconfd: add cluster master query
Guido Trotter [Mon, 31 Aug 2009 14:54:20 +0000 (15:54 +0100)]
confd: add cluster master query

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoconfd/querylib: fix docstrings
Guido Trotter [Mon, 31 Aug 2009 14:58:05 +0000 (15:58 +0100)]
confd/querylib: fix docstrings

They were cut&paste from the first example queries, rather than
documenting the current ones.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoconfd: avoid spamming the logfile
Guido Trotter [Fri, 28 Aug 2009 15:18:17 +0000 (18:18 +0300)]
confd: avoid spamming the logfile

When confd is disabled we don't want to be noticed every timer interval.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoRemove SimpleConfigReader.GetHypervisorType
Guido Trotter [Mon, 31 Aug 2009 15:01:41 +0000 (16:01 +0100)]
Remove SimpleConfigReader.GetHypervisorType

We haven't had that config file entry for a while.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoMerge commit 'origin/next' into branch-2.1
Michael Hanselmann [Mon, 31 Aug 2009 13:57:39 +0000 (15:57 +0200)]
Merge commit 'origin/next' into branch-2.1

* commit 'origin/next':
  Return cluster tags from LUQueryClusterInfo

Conflicts:
lib/cmdlib.py: Trivial

14 years agoReplace BINDIR variable as well
Luca Bigliardi [Mon, 31 Aug 2009 12:27:04 +0000 (13:27 +0100)]
Replace BINDIR variable as well

Add bindir to the list of variables replaced at make time.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

14 years agoReturn cluster tags from LUQueryClusterInfo
Michael Hanselmann [Mon, 31 Aug 2009 10:07:52 +0000 (12:07 +0200)]
Return cluster tags from LUQueryClusterInfo

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agodevel/upload: Build in parallel
Michael Hanselmann [Mon, 31 Aug 2009 09:37:26 +0000 (11:37 +0200)]
devel/upload: Build in parallel

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agoMerge branch 'next' into branch-2.1
Michael Hanselmann [Fri, 28 Aug 2009 15:33:37 +0000 (17:33 +0200)]
Merge branch 'next' into branch-2.1

* next:
  Add script to clean archived jobs after 21 days
  rapi: export more static node information
  Pass the correct signal to handlers
  cli: Use ToStdout/ToStderr instead of print
  Fix small typo in gnt-node
  Simplify handling of boolean args in rapi
  Fix checks in LUSetNodeParms for the master node
  Improve the example startup script
  Fix insserv dependencies

Conflicts:
Makefile.am: added ganeti-cleaner
doc/examples/ganeti.initd.in: Iustin's init script improvements
lib/cli.py: Trivial (ToStderr changes)

14 years agoAdd more bash completion metadata for options
Michael Hanselmann [Thu, 27 Aug 2009 17:30:09 +0000 (19:30 +0200)]
Add more bash completion metadata for options

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agoBash completion: Implement dynamic option value completion
Michael Hanselmann [Thu, 27 Aug 2009 17:29:23 +0000 (19:29 +0200)]
Bash completion: Implement dynamic option value completion

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agoBash completion: Add function to get all iallocators
Michael Hanselmann [Thu, 27 Aug 2009 17:26:36 +0000 (19:26 +0200)]
Bash completion: Add function to get all iallocators

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agoBash completion: Simplify option completion
Michael Hanselmann [Thu, 27 Aug 2009 16:43:41 +0000 (18:43 +0200)]
Bash completion: Simplify option completion

This makes the output file quite a bit smaller and is needed
in preparation for non-static option completion (e.g. list of
nodes).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agoBash completion: Support for --foo=bar option format
Michael Hanselmann [Thu, 27 Aug 2009 15:36:22 +0000 (17:36 +0200)]
Bash completion: Support for --foo=bar option format

That is with the equal sign.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoBash completion: Small optimizations
Michael Hanselmann [Thu, 27 Aug 2009 15:35:36 +0000 (17:35 +0200)]
Bash completion: Small optimizations

$2 doesn't contain the correct value when completing something like
"--disk-template=…". Getting it via COMP_WORDS is better.

Short options (e.g. -I) can't have an equal sign.

Also add useful debugging commands for development.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

14 years agoFix QA test for “gnt-node evacuate”
Michael Hanselmann [Fri, 28 Aug 2009 10:44:36 +0000 (12:44 +0200)]
Fix QA test for “gnt-node evacuate”

Before the arguments weren't valid.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoConfd: don't fail if the config doesn't load
Guido Trotter [Fri, 28 Aug 2009 12:03:39 +0000 (15:03 +0300)]
Confd: don't fail if the config doesn't load

Rather than quitting we'll just continue to poll the config at a slow
rate, hoping that sooner or later we'll get it back. This allows also
working on non-MC nodes, and smoothly transitioning from MC to non-MC,
other than dealing with a few temporary breakages.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoconfd: start in polling mode
Guido Trotter [Fri, 28 Aug 2009 13:59:07 +0000 (16:59 +0300)]
confd: start in polling mode

This allows us not to enable the inotify handler immediately, and thus
to make it easier for us should the config file not exist at all.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoConfdProcessor: add disabled state
Guido Trotter [Fri, 28 Aug 2009 11:57:38 +0000 (14:57 +0300)]
ConfdProcessor: add disabled state

This is a state the processor will get in, if it fails to load the
config.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoMove SimpleConfigReader creation to ConfdProcessor
Guido Trotter [Fri, 28 Aug 2009 11:05:54 +0000 (14:05 +0300)]
Move SimpleConfigReader creation to ConfdProcessor

This will be useful to make ConfdProcessor aware of a config failure,
without quitting confd.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoconfd: s/confd_event_handler/inotify_handler/
Guido Trotter [Fri, 28 Aug 2009 12:07:01 +0000 (15:07 +0300)]
confd: s/confd_event_handler/inotify_handler/

In a case we don't encounter frequently (file modified but not
overwritten) the notify handler we use is called with a wrong name.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

14 years agoAdd script to clean archived jobs after 21 days
Michael Hanselmann [Fri, 28 Aug 2009 14:37:14 +0000 (16:37 +0200)]
Add script to clean archived jobs after 21 days

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>