ganeti-local
13 years agoRestrict blacklisted OSes in instance installation
Iustin Pop [Thu, 16 Sep 2010 15:42:51 +0000 (17:42 +0200)]
Restrict blacklisted OSes in instance installation

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoAdd two new cluster settings
Iustin Pop [Thu, 16 Sep 2010 14:34:15 +0000 (16:34 +0200)]
Add two new cluster settings

The new variables are:

- a list of hidden OSes, that should not be displayed to the users in
  interactive selection (e.g. reinstall); however, if they are selected, they
  can be used
- a list of OSes that should be hidden and blocked from install-time selection

The filtering will apply at pure OS name level, not OS+variant level.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoAbstract OS name/variant functions
Iustin Pop [Thu, 16 Sep 2010 14:00:25 +0000 (16:00 +0200)]
Abstract OS name/variant functions

Currently, the computation of the 'pure' name or the variant is
hardcoded and spread around the functions that need it. This is not
nice, and in the future we'd spread it even more with more usage of
variants/pure os names.

This patch abstracts these functions into the OS class, and then
replaces the hardcoded uses with the new functions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoAdd OS new states to the design doc
Iustin Pop [Tue, 21 Sep 2010 08:38:04 +0000 (10:38 +0200)]
Add OS new states to the design doc

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoRemove the RPC changes from the 2.2 design
Iustin Pop [Tue, 21 Sep 2010 08:26:02 +0000 (10:26 +0200)]
Remove the RPC changes from the 2.2 design

These were not implemented.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoRemove 'Detailed Design' from design-2.2.rst
Iustin Pop [Tue, 21 Sep 2010 08:24:18 +0000 (10:24 +0200)]
Remove 'Detailed Design' from design-2.2.rst

This also bumps up the rest of the headings.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoMerge branch 'stable-2.2' into devel-2.2
Michael Hanselmann [Wed, 22 Sep 2010 14:33:34 +0000 (16:33 +0200)]
Merge branch 'stable-2.2' into devel-2.2

* stable-2.2:
  NEWS: Add 2.2.0~rc2 release date
  Bump version to 2.2.0~rc2
  Remove two old aliases
  Fix migration on new KVMs
  Prepare 2.2.0~rc2 release

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agoMerge branch 'devel-2.1' into devel-2.2
Michael Hanselmann [Wed, 22 Sep 2010 12:58:37 +0000 (14:58 +0200)]
Merge branch 'devel-2.1' into devel-2.2

* devel-2.1:
  Fix mac checker regex

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agoNEWS: Add 2.2.0~rc2 release date
Michael Hanselmann [Wed, 22 Sep 2010 12:49:52 +0000 (14:49 +0200)]
NEWS: Add 2.2.0~rc2 release date

This was forgotten when preparing the release.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agoBump version to 2.2.0~rc2 v2.2.0rc2
Michael Hanselmann [Wed, 22 Sep 2010 10:44:17 +0000 (12:44 +0200)]
Bump version to 2.2.0~rc2

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix mac checker regex
Iustin Pop [Tue, 21 Sep 2010 18:24:28 +0000 (20:24 +0200)]
Fix mac checker regex

Currently, the mac checker regex could match a corner case of
11:22:33:44:55:66: (one extra colon at the end). We fix this, and we
also move the regex compilation outside of this function, at module
level.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoRemove two old aliases
Iustin Pop [Tue, 21 Sep 2010 07:09:08 +0000 (09:09 +0200)]
Remove two old aliases

These were deprecated since Ganeti 2.0, it's time to remove them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoFix migration on new KVMs
Vitaly Kuznetsov [Thu, 16 Sep 2010 20:27:24 +0000 (20:27 +0000)]
Fix migration on new KVMs

New KVMs (0.12.1.2-el6 and 0.13.5 tested) exit immediately after
unsuccessful network connection when they are in "-incoming" mode. The
simple check netutils.TcpPing causes remote kvm to exit so the migration
will always fail. This check is also redundant by the way as if the
network port is closed migration will fail with guarantee. This commit
removes this check.

Signed-off-by: Vitaly Kuznetsov <vitty@altlinux.ru>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix typo in ganeti man page
Michael Hanselmann [Fri, 17 Sep 2010 15:51:58 +0000 (17:51 +0200)]
Fix typo in ganeti man page

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agocli: Pass options in {Add,Remove}Tags
Michael Hanselmann [Fri, 17 Sep 2010 15:35:36 +0000 (17:35 +0200)]
cli: Pass options in {Add,Remove}Tags

They'll be used for job priorities. Also add an empty line to
gnt-os where it's missing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoPrepare 2.2.0~rc2 release
Michael Hanselmann [Fri, 17 Sep 2010 11:45:57 +0000 (13:45 +0200)]
Prepare 2.2.0~rc2 release

Update NEWS, but not yet configure.ac.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoQA: Test more tag operations via RAPI
Michael Hanselmann [Thu, 16 Sep 2010 10:46:32 +0000 (12:46 +0200)]
QA: Test more tag operations via RAPI

This ensures tag operations work via RAPI. Also print requested URI to
stdout for debugging.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoqa_rapi: Fix indentation error
Michael Hanselmann [Wed, 15 Sep 2010 19:38:23 +0000 (21:38 +0200)]
qa_rapi: Fix indentation error

If no function to verify the result was specified, qa_rapi._DoTests
wouldn't return the result.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoMerge branch 'stable-2.2' into devel-2.2
Michael Hanselmann [Thu, 16 Sep 2010 10:38:18 +0000 (12:38 +0200)]
Merge branch 'stable-2.2' into devel-2.2

* stable-2.2:
  cmdlib: Fix type of “name” parameter for tag operations
  rlib2: Set tag operation param “name” to None for cluster tags
  Fix OS_VARIANT variable setting

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoopcode summary: improve display for list summaries
Iustin Pop [Mon, 23 Aug 2010 14:20:26 +0000 (16:20 +0200)]
opcode summary: improve display for list summaries

Currently, opcodes like NODE_EVAC_STRATEGY look bad:

  89684 error   NODE_EVAC_STRATEGY([u'node3'])

With this patch, we try to render list arguments a little bit better:

  89684 error   NODE_EVAC_STRATEGY(node3)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoMerge branch 'devel-2.1' into devel-2.2
Iustin Pop [Thu, 16 Sep 2010 07:24:55 +0000 (09:24 +0200)]
Merge branch 'devel-2.1' into devel-2.2

* devel-2.1:
  Fix case of MAC special-values

(no conflicts, took LGTM from original commit)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoFix case of MAC special-values
Iustin Pop [Mon, 13 Sep 2010 09:30:03 +0000 (11:30 +0200)]
Fix case of MAC special-values

This fixes Debian bug #596347. An alternative would be to also allow
the uppercase form, but it's not nice I think.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agocmdlib: Fix type of “name” parameter for tag operations
Michael Hanselmann [Wed, 15 Sep 2010 19:37:38 +0000 (21:37 +0200)]
cmdlib: Fix type of “name” parameter for tag operations

The parameter “name” is be None for cluster tags.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agorlib2: Set tag operation param “name” to None for cluster tags
Michael Hanselmann [Wed, 15 Sep 2010 19:39:05 +0000 (21:39 +0200)]
rlib2: Set tag operation param “name” to None for cluster tags

Otherwise parameter verification in the master daemon fails.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoRemove mcpu's ReportLocks callback
Michael Hanselmann [Mon, 13 Sep 2010 14:46:41 +0000 (16:46 +0200)]
Remove mcpu's ReportLocks callback

This is no longer needed with the new lock monitor. One callback is kept to
check for cancelled jobs.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoRevert "jqueue: Resume jobs from “waitlock” status"
Michael Hanselmann [Fri, 10 Sep 2010 15:28:30 +0000 (17:28 +0200)]
Revert "jqueue: Resume jobs from “waitlock” status"

This reverts commit 4008c8edae31a3971fa8c4b200238afc8005d3d4.

While it worked in my initial tests, I've now found cases where this doesn't
work properly as it is. More work is needed and will be done as part of the
Ganeti 2.3 job queue changes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix OS_VARIANT variable setting
Vitaly Kuznetsov [Wed, 8 Sep 2010 16:57:45 +0000 (16:57 +0000)]
Fix OS_VARIANT variable setting

This was introduced in efaa9b06d1e1e6d1678d0edd75b1ba37cf0de3d9.

 in OSCoreEnv:
  inst_os.name is pure operating system name (without variant) as variant is stripped
   in OSFromDisk(). So we always get variant = inst_os.supported_variants[0] (first
   variant in variants list).
  Adding argument os_name with full name (including variant) solves this problem.

Signed-off-by: Vitaly Kuznetsov <vitty@altlinux.ru>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
[modified by iustin to handle the call to OSCoreEnv from ValidateOS too]
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoMerge branch 'stable-2.2' into devel-2.2
Michael Hanselmann [Fri, 10 Sep 2010 12:06:22 +0000 (14:06 +0200)]
Merge branch 'stable-2.2' into devel-2.2

* stable-2.2:
  Fix pylint warning in http/__init__.py
  Allow SSL ciphers to be overridden in HTTP server
  If we had any errors in setup in one of the hosts, exit with non-zero

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix pylint warning in http/__init__.py
Michael Hanselmann [Fri, 10 Sep 2010 11:48:53 +0000 (13:48 +0200)]
Fix pylint warning in http/__init__.py

My bad for not seeing this before:
R0201:614:HttpBase.GetSslCiphers: Method could be a function

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoAllow SSL ciphers to be overridden in HTTP server
Michael Hanselmann [Fri, 10 Sep 2010 11:14:04 +0000 (13:14 +0200)]
Allow SSL ciphers to be overridden in HTTP server

Users of this class, such as the RAPI server, might want to override or adjust
the default SSL cipher defined in a constant.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agojqueue: Resume jobs from “waitlock” status
Michael Hanselmann [Thu, 9 Sep 2010 16:17:25 +0000 (18:17 +0200)]
jqueue: Resume jobs from “waitlock” status

After an unclean restart of ganeti-masterd, jobs in the “waitlock” status can
be safely restarted. They hadn't modified the cluster yet.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agojqueue: Move queue inspection into separate function
Michael Hanselmann [Wed, 8 Sep 2010 17:25:07 +0000 (19:25 +0200)]
jqueue: Move queue inspection into separate function

This makes the __init__ function a lot smaller while not changing
functionality.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agojqueue: Don't update file in MarkUnfinishedOps
Michael Hanselmann [Thu, 9 Sep 2010 13:53:56 +0000 (15:53 +0200)]
jqueue: Don't update file in MarkUnfinishedOps

This reduced the number of updates to the job files. It's used in two places
while processing a job and the file is updated just afterwards.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agolocking.SharedLock: Update class docstring
Michael Hanselmann [Tue, 7 Sep 2010 14:04:00 +0000 (16:04 +0200)]
locking.SharedLock: Update class docstring

This was already outdated when the initial version of SharedLock was added
in commit 162c1c1f1 (February 2008).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoIf we had any errors in setup in one of the hosts, exit with non-zero
René Nussbaumer [Thu, 9 Sep 2010 11:13:08 +0000 (13:13 +0200)]
If we had any errors in setup in one of the hosts, exit with non-zero

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoFix the output of the key fingerprint from binary to hex
René Nussbaumer [Thu, 9 Sep 2010 11:09:41 +0000 (13:09 +0200)]
Fix the output of the key fingerprint from binary to hex

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoMerge branch 'stable-2.2' into devel-2.2
Michael Hanselmann [Tue, 7 Sep 2010 09:53:12 +0000 (11:53 +0200)]
Merge branch 'stable-2.2' into devel-2.2

* stable-2.2:
  Fix breakage introduced by commit 8044bf655
  Remove “dry_run” from opcodes.OpCreateInstance
  Disable the RAPI CA checks in watcher
  move-instance: Fix parameter order in _CreateInstance

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agocli: Use list of options shared between commands
Michael Hanselmann [Tue, 7 Sep 2010 09:44:26 +0000 (11:44 +0200)]
cli: Use list of options shared between commands

The completion script for bash has to know about these options. Until now
the list was in two places--once in cli.py and once in
autotools/build-bash-completion. A shared list is used with this patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agojqueue: Use separate function for encoding errors
Michael Hanselmann [Mon, 6 Sep 2010 19:58:31 +0000 (21:58 +0200)]
jqueue: Use separate function for encoding errors

Comes with unittest.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix some epydoc warnings
Manuel Franceschini [Fri, 3 Sep 2010 10:47:51 +0000 (12:47 +0200)]
Fix some epydoc warnings

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoFix breakage introduced by commit 8044bf655
Michael Hanselmann [Thu, 2 Sep 2010 16:34:38 +0000 (18:34 +0200)]
Fix breakage introduced by commit 8044bf655

Note to self: even patches removing one line can break everything.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

13 years agoRemove “dry_run” from opcodes.OpCreateInstance
Michael Hanselmann [Thu, 2 Sep 2010 15:52:44 +0000 (17:52 +0200)]
Remove “dry_run” from opcodes.OpCreateInstance

I'ts declared twice, once in opcodes.OpCode and here, and this one
is redundant.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoDisable the RAPI CA checks in watcher
Iustin Pop [Thu, 2 Sep 2010 09:18:27 +0000 (11:18 +0200)]
Disable the RAPI CA checks in watcher

Since the RAPI certificate is not necessarily self-signed, and we
currently don't have any configuration variable for the real CA file, we
disable for now the CA checks. This fixes the 'restart RAPI every 5
minutes' problem with non-self-signed certs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agomove-instance: Fix parameter order in _CreateInstance
Manuel Franceschini [Wed, 1 Sep 2010 09:34:30 +0000 (11:34 +0200)]
move-instance: Fix parameter order in _CreateInstance

This patch fixes a bug, which caused the primary and secondary
destination nodes to be revered on the destination cluster after an
instance move.

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoShow list of pending acquires in “gnt-debug locks”
Michael Hanselmann [Thu, 26 Aug 2010 16:17:20 +0000 (18:17 +0200)]
Show list of pending acquires in “gnt-debug locks”

This is accomplished by keeping a list of waiting threads instead
of just their number inside the lock-internal condition. A few
other tweaks to the output format are also made.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoAdding host key verification to setup-ssh
René Nussbaumer [Tue, 24 Aug 2010 11:54:01 +0000 (13:54 +0200)]
Adding host key verification to setup-ssh

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoAdding a paramiko fingerprint format helper
René Nussbaumer [Tue, 24 Aug 2010 14:21:37 +0000 (16:21 +0200)]
Adding a paramiko fingerprint format helper

And provide unittests for them

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoMerge branch 'devel-2.1' into devel-2.2
Guido Trotter [Tue, 24 Aug 2010 17:06:33 +0000 (18:06 +0100)]
Merge branch 'devel-2.1' into devel-2.2

* devel-2.1:
  Increase version to 2.1.7
  Update NEWS for the 2.1.7 release

Conflicts:
NEWS
  - merge
configure.ac
  - keep the 2.2 version

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoIncrease version to 2.1.7 v2.1.7
Guido Trotter [Tue, 24 Aug 2010 16:46:43 +0000 (17:46 +0100)]
Increase version to 2.1.7

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoUpdate NEWS for the 2.1.7 release
Guido Trotter [Fri, 20 Aug 2010 09:39:29 +0000 (10:39 +0100)]
Update NEWS for the 2.1.7 release

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix race condition in locking unittest
Michael Hanselmann [Tue, 24 Aug 2010 15:22:29 +0000 (17:22 +0200)]
Fix race condition in locking unittest

While writing unittests for the new lock monitor, I made a small mistake and
didn't synchronize acquiring locks properly. This patch takes care of this
issue by adding additional synchronization.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoAdd simple lock monitor
Michael Hanselmann [Mon, 23 Aug 2010 17:32:54 +0000 (19:32 +0200)]
Add simple lock monitor

This patch adds an initial implementation of a lock monitor, accessible
for the user through “gnt-debug locks”. It currently shows all resource
locks: BGL, nodes and instances. Config and job queue locks could be
shown too, but wouldn't be of much help.  The current owner(s) and mode
are also shown.

Showing pending acquires will require further changes on the SharedLock
internals and is not yet implemented.

Example output:
$ gnt-debug locks -o name,mode,owner
Name            Mode      Owner
BGL/BGL         shared    JobQueue19/Job147
instances/inst1 exclusive JobQueue19/Job147
instances/inst2 -         -
instances/inst3 -         -
instances/inst4 -         -
nodes/node1     exclusive JobQueue19/Job147
nodes/node2     exclusive JobQueue19/Job147

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoworkerpool: Allow setting task name
Michael Hanselmann [Mon, 23 Aug 2010 11:31:55 +0000 (13:31 +0200)]
workerpool: Allow setting task name

With this patch, the task name is added to the thread name and will show up in
logs. Log messages from jobs will look like “pid=578/JobQueue14/Job13 mcpu:289
DEBUG LU locks acquired/cluster/BGL/shared”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoUse one function to parse “--fields” option values
Michael Hanselmann [Mon, 23 Aug 2010 15:14:11 +0000 (17:14 +0200)]
Use one function to parse “--fields” option values

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agolocking.LockSet: Use function to get member lock name
Michael Hanselmann [Mon, 23 Aug 2010 12:15:46 +0000 (14:15 +0200)]
locking.LockSet: Use function to get member lock name

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFinalize removal of “lock_status” job field
Michael Hanselmann [Mon, 23 Aug 2010 14:54:22 +0000 (16:54 +0200)]
Finalize removal of “lock_status” job field

When removing the field in commit 9bdab62198, I forgot to remove some parts.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoPrepare 2.2.0~rc1 release v2.2.0rc1
Michael Hanselmann [Mon, 23 Aug 2010 11:58:41 +0000 (13:58 +0200)]
Prepare 2.2.0~rc1 release

Update NEWS and version number.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agosetup-ssh: fix updating of authorized_keys
Iustin Pop [Sun, 22 Aug 2010 07:00:20 +0000 (09:00 +0200)]
setup-ssh: fix updating of authorized_keys

Due to what seems like a bug (or inconsistency) in paramiko, files
opened with a+ over SFTP need a seek() in order for the user to be able
to read data from them. We implement this, and rely on the fact that we
do iterate over all lines before writing and that the file is opened in
append mode (which at least on Linux should work correctly).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agosetup-ssh: Also use keys from the ssh-agent
Iustin Pop [Sun, 22 Aug 2010 06:37:32 +0000 (08:37 +0200)]
setup-ssh: Also use keys from the ssh-agent

Currently, setup-ssh only uses one disk-based key. This means that any
setup where we use keys from ssh-agent (which do not necessarily exist
on disk) will break when moving from the old method to setup-ssh.

This patch moves the SSH key handling to separate functions, and uses
both the disk key (first) and the agent keys for login.

The patch also fixes the root_logger setup level (I tried to hard to
reduce noise and broke the debug level, sorry).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agosetup-ssh: try to use key auth first
Iustin Pop [Fri, 20 Aug 2010 13:12:02 +0000 (15:12 +0200)]
setup-ssh: try to use key auth first

This patch changes the setup-ssh workflow to try key authentication
first, and then fall-back to password authentication. The password is
also read lazily, with no prompts if we can authenticate via keys.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agosetup-ssh: redo the logging levels
Iustin Pop [Fri, 20 Aug 2010 13:02:12 +0000 (15:02 +0200)]
setup-ssh: redo the logging levels

There is not enough logged by default on stderr, and way too much
information in the log file.

Since we don't want to debug paramiko itself, we filter only warnings
and above.

Otherwise, we make WARNINGS the default level, with verbose meaning INFO
and debug, well, DEBUG.

Additionally, a message is improved.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agosetup-ssh: only read the ssh port once
Iustin Pop [Fri, 20 Aug 2010 12:09:44 +0000 (14:09 +0200)]
setup-ssh: only read the ssh port once

Then reuse it. With certain setups, this could be slow when pre-seeding
an entire cluster.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agosetup-ssh: fix the logging error message
Iustin Pop [Fri, 20 Aug 2010 11:54:54 +0000 (13:54 +0200)]
setup-ssh: fix the logging error message

This took a bit of interesting debugging, but the issue seems to be that
on older python/logging/paramiko (I'm not sure which is at fault here),
the other threads are finishing later than the main thread, so the
logging module closes the logging file (but doesn't unregister it)
before the paramiko thread(s) get a change to finish their logging.

Tested on both old versions (Dapper) and new versions (Lucid) of these
three pieces of the puzzle.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agoUse Sphinx' :rfc: extension to refer to RFCs
Michael Hanselmann [Fri, 20 Aug 2010 13:13:51 +0000 (15:13 +0200)]
Use Sphinx' :rfc: extension to refer to RFCs

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoDocument non-standard usage of JSON in RAPI
Michael Hanselmann [Fri, 20 Aug 2010 13:13:11 +0000 (15:13 +0200)]
Document non-standard usage of JSON in RAPI

This was requested in issue 118.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix small spelling mistake
Manuel Franceschini [Fri, 20 Aug 2010 09:46:24 +0000 (11:46 +0200)]
Fix small spelling mistake

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoExplicitly add dry-run to some commands
Iustin Pop [Thu, 19 Aug 2010 12:25:50 +0000 (14:25 +0200)]
Explicitly add dry-run to some commands

Based on manual inspection (that the command only does a submit of some
jobs/opcodes), we re-add the dry-run option on a subset of the existing
commands.

A few more commands could use dry-run, but the code doesn't deal nicely
with no results, so fixing those is left for later patches.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoStop adding the dry-run option by default
Iustin Pop [Thu, 19 Aug 2010 10:40:22 +0000 (12:40 +0200)]
Stop adding the dry-run option by default

Currently cli.py unconditionally adds the dry-run option. This patch
disables this, and exports dry-run as a normal option.

The other alternative I tried to implement (adding a new fake option for
disabling the auto-add per individual command) would require changes in
more places, as the list of options is no longer a homogeneous list.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoFix a few commands behaviour with dry-run
Iustin Pop [Thu, 19 Aug 2010 13:51:00 +0000 (15:51 +0200)]
Fix a few commands behaviour with dry-run

These commands use or display the result from the LU, so in case of
dry-run, they will crash or display just 'None'. At least checking that
the result is 'true' (in the boolean sense) will make them work better.

As for gnt-os modify, it didn't pass the 'opts' parameter properly to
SubmitOpCode, so the dry-run option was silently ignored.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agojqueue: Remove lock status field
Michael Hanselmann [Wed, 18 Aug 2010 15:19:44 +0000 (17:19 +0200)]
jqueue: Remove lock status field

With the job queue changes for Ganeti 2.2, watched and queried jobs are
loaded directly from disk, rendering the in-memory “lock_status” field
useless. Writing it to disk would be possible, but has a huge cost at
runtime (when tested, processing 1'000 opcodes involved 4'000 additional
writes to job files, even with replication turned off).

Using an additional in-memory dictionary to just manage this field turned
out to be a complicated task due to the necessary locking.

The plan is to introduce a more generic lock debugging mechanism in the
near future. Hence the decision is to remove this field now instead of
spending a lot of time to make it working again.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoQA: Run simple job queue test
Michael Hanselmann [Wed, 18 Aug 2010 13:01:31 +0000 (15:01 +0200)]
QA: Run simple job queue test

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoMerge branch 'devel-2.1' into devel-2.2
Guido Trotter [Thu, 19 Aug 2010 13:56:40 +0000 (14:56 +0100)]
Merge branch 'devel-2.1' into devel-2.2

* devel-2.1:
  Don't ignore secondary node silently

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoDon't ignore secondary node silently
Guido Trotter [Thu, 19 Aug 2010 12:59:55 +0000 (13:59 +0100)]
Don't ignore secondary node silently

Currently on non-mirrored disk templates the secondary node is ignored
silently. This patch adds a check for this case, and warns the user
should this be happening. This solves issue 113.

The patch also moves a prereq check to an argument check. This is ok
because the iallocator framework already checks it has a secondary node,
when it needs one.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoetags: force Python as a language
Iustin Pop [Thu, 19 Aug 2010 11:41:04 +0000 (13:41 +0200)]
etags: force Python as a language

etags has cannot properly identify the scripts/gnt-* files as Python, so
we statically force the language to Python.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoMerge branch 'devel-2.1' into devel-2.2
Guido Trotter [Wed, 18 Aug 2010 16:44:12 +0000 (17:44 +0100)]
Merge branch 'devel-2.1' into devel-2.2

* devel-2.1:
  Fix --master-netdev arg name in gnt-cluster(8)
  Restore 'tablet mouse on vnc' behavior
  Document the usb_mouse hv parameter
  Revert "Add -usbdevice tablet to KVM when using vnc"

Conflicts:
man/gnt-instance.sgml
  - merge

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoRAPI client: Support modifying instances
Michael Hanselmann [Tue, 17 Aug 2010 16:50:37 +0000 (18:50 +0200)]
RAPI client: Support modifying instances

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoRAPI: Allow modifying instance
Michael Hanselmann [Tue, 17 Aug 2010 16:50:11 +0000 (18:50 +0200)]
RAPI: Allow modifying instance

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoSmall fixes for instance creation via RAPI documentation
Michael Hanselmann [Tue, 17 Aug 2010 15:16:38 +0000 (17:16 +0200)]
Small fixes for instance creation via RAPI documentation

- Inconsistencies
- Missing types

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agognt-debug: Extend job queue tests
Michael Hanselmann [Fri, 13 Aug 2010 10:22:43 +0000 (12:22 +0200)]
gnt-debug: Extend job queue tests

Test multiple opcodes, also with failure.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agojqueue: Mark opcodes following failed ones as failed, too
Michael Hanselmann [Tue, 17 Aug 2010 13:52:11 +0000 (15:52 +0200)]
jqueue: Mark opcodes following failed ones as failed, too

When an opcode fails, the job queue would leave following opcodes as “queued”,
which can be quite confusing. With this patch, they're all marked as failed and
assertions are added to check this.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agojqueue: Work around race condition between job processing and archival
Michael Hanselmann [Tue, 17 Aug 2010 13:33:52 +0000 (15:33 +0200)]
jqueue: Work around race condition between job processing and archival

This is a simplified version of a patch I sent earlier to make sure the job
file is only written once with a finalized status.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agojqueue: More checks for cancelling queued job
Michael Hanselmann [Fri, 13 Aug 2010 17:18:36 +0000 (19:18 +0200)]
jqueue: More checks for cancelling queued job

We can also check when the lock status is updated. This will
improve job cancelling.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoerrors: Function to check whether value is encoded error
Michael Hanselmann [Fri, 13 Aug 2010 17:19:07 +0000 (19:19 +0200)]
errors: Function to check whether value is encoded error

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agojqueue: Add more debug output
Michael Hanselmann [Thu, 12 Aug 2010 11:08:46 +0000 (13:08 +0200)]
jqueue: Add more debug output

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agognt-backup: Pass error code to OpPrereqError
Michael Hanselmann [Mon, 16 Aug 2010 13:53:41 +0000 (15:53 +0200)]
gnt-backup: Pass error code to OpPrereqError

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoMerge branch 'devel-2.1'
Iustin Pop [Tue, 17 Aug 2010 12:42:58 +0000 (14:42 +0200)]
Merge branch 'devel-2.1'

* devel-2.1:
  Fix path in ganeti-rapi man page
  Adjust message in case ~/.ssh is no directory

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoRe-add the 'live' parameter to migration opcodes
Iustin Pop [Tue, 17 Aug 2010 12:18:34 +0000 (14:18 +0200)]
Re-add the 'live' parameter to migration opcodes

This patch reintroduces the live parameter, for backwards compatibility
at the Luxi level. This way, clients can work transparently with both
2.1 and 2.2, even though sub-optimally.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoFix --master-netdev arg name in gnt-cluster(8)
Guido Trotter [Thu, 12 Aug 2010 15:50:47 +0000 (11:50 -0400)]
Fix --master-netdev arg name in gnt-cluster(8)

This fixes Issue 114.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoRestore 'tablet mouse on vnc' behavior
Guido Trotter [Mon, 9 Aug 2010 18:30:17 +0000 (14:30 -0400)]
Restore 'tablet mouse on vnc' behavior

We needed to revert commit 5b062a58ac76b39c2dc6a7e1543affdf43dc7ee7
because it was in conflict with the usb_mouse hv parameter. Here we
reintroduce its functionality only when usb_mouse is not specified.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoDocument the usb_mouse hv parameter
Guido Trotter [Mon, 9 Aug 2010 16:19:07 +0000 (12:19 -0400)]
Document the usb_mouse hv parameter

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoRevert "Add -usbdevice tablet to KVM when using vnc"
Guido Trotter [Mon, 9 Aug 2010 16:11:07 +0000 (12:11 -0400)]
Revert "Add -usbdevice tablet to KVM when using vnc"

This reverts commit 5b062a58ac76b39c2dc6a7e1543affdf43dc7ee7.
This fixes issue 109. The mouse type can be set with the usb_mouse
hv parameter.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoFix path in ganeti-rapi man page
Michael Hanselmann [Mon, 16 Aug 2010 13:54:26 +0000 (15:54 +0200)]
Fix path in ganeti-rapi man page

This takes care of issue 116.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoworkerpool: Don't keep reference to task arguments
Michael Hanselmann [Fri, 13 Aug 2010 10:26:31 +0000 (12:26 +0200)]
workerpool: Don't keep reference to task arguments

The workerpool should not keep any reference to task arguments after
they were processed by RunTask. Doing so led to jobs being cached
by the job queue's WeakValueDictionary for longer than they should've
been.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

13 years agocli.SubmitOpCode: Pass keyword parameter as keyword
Michael Hanselmann [Fri, 13 Aug 2010 10:21:28 +0000 (12:21 +0200)]
cli.SubmitOpCode: Pass keyword parameter as keyword

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

13 years agognt-backup: Don't show confusing message w/o target node
Michael Hanselmann [Tue, 10 Aug 2010 15:54:42 +0000 (17:54 +0200)]
gnt-backup: Don't show confusing message w/o target node

“gnt-backup export” requires the target node. Until now, the master
daemon would complain that the “parameter
'OP_BACKUP_EXPORT.target_node' fails validation”. With this patch,
an additional check is done in the client program.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

13 years agomasterd.instance: Add missing argument
Michael Hanselmann [Tue, 10 Aug 2010 14:41:50 +0000 (16:41 +0200)]
masterd.instance: Add missing argument

_DiskTransferPrivate takes three parameters, not two.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

13 years agoAdjust message in case ~/.ssh is no directory
Michael Hanselmann [Tue, 10 Aug 2010 13:51:11 +0000 (15:51 +0200)]
Adjust message in case ~/.ssh is no directory

Use actual path, not something hardcoded.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoRAPI client: Fix docstring for migrating instance
Michael Hanselmann [Fri, 30 Jul 2010 18:47:50 +0000 (20:47 +0200)]
RAPI client: Fix docstring for migrating instance

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoQA: Test renaming instance via RAPI
Michael Hanselmann [Fri, 30 Jul 2010 17:52:43 +0000 (19:52 +0200)]
QA: Test renaming instance via RAPI

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoRAPI client: Support renaming instances
Michael Hanselmann [Fri, 30 Jul 2010 17:33:31 +0000 (19:33 +0200)]
RAPI client: Support renaming instances

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>