ganeti-local
13 years agoutils: Move ResetTempfileModule to wrappers module
Michael Hanselmann [Tue, 11 Jan 2011 11:12:46 +0000 (12:12 +0100)]
utils: Move ResetTempfileModule to wrappers module

It's not exactly a wrapper, but this seemed like the best place.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move RunningTimeout to utils.algo
Michael Hanselmann [Tue, 11 Jan 2011 11:07:24 +0000 (12:07 +0100)]
utils: Move RunningTimeout to utils.algo

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move code manipulating /etc/hosts to separate file
Michael Hanselmann [Tue, 11 Jan 2011 15:01:48 +0000 (16:01 +0100)]
utils: Move code manipulating /etc/hosts to separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix utils.io unittest
Michael Hanselmann [Tue, 11 Jan 2011 14:45:18 +0000 (15:45 +0100)]
Fix utils.io unittest

“import signal” was missing, but of course it didn't show during my
testing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move X509-related code into separate file
Michael Hanselmann [Mon, 10 Jan 2011 17:36:04 +0000 (18:36 +0100)]
utils: Move X509-related code into separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move I/O-related code into separate file
Michael Hanselmann [Mon, 10 Jan 2011 19:44:41 +0000 (20:44 +0100)]
utils: Move I/O-related code into separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move code related to file locking into separate file
Michael Hanselmann [Mon, 10 Jan 2011 16:58:52 +0000 (17:58 +0100)]
utils: Move code related to file locking into separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move wrappers into separate file
Michael Hanselmann [Mon, 10 Jan 2011 16:51:46 +0000 (17:51 +0100)]
utils: Move wrappers into separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Use function to disable fork
Michael Hanselmann [Mon, 10 Jan 2011 18:59:45 +0000 (19:59 +0100)]
utils: Use function to disable fork

Use a function instead of a variable written by another module.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Change RemovePidFile to take path, not name
Michael Hanselmann [Mon, 10 Jan 2011 19:40:43 +0000 (20:40 +0100)]
utils: Change RemovePidFile to take path, not name

This avoids having to monkey-patch the utils module for unittests.
Monkey patching is evil and caused a bug while moving code around.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix a typo in backend.py
René Nussbaumer [Tue, 11 Jan 2011 13:03:28 +0000 (14:03 +0100)]
Fix a typo in backend.py

Sorry I thought I did run commit-check but must not have paid attention
to its output. There was a typo in the docstring. This patch fixes this.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoPause/Resume sync accordingly before wiping disk
René Nussbaumer [Fri, 7 Jan 2011 13:24:04 +0000 (14:24 +0100)]
Pause/Resume sync accordingly before wiping disk

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoAdd RPC calls for pause/resume sync of devices
René Nussbaumer [Fri, 7 Jan 2011 13:18:12 +0000 (14:18 +0100)]
Add RPC calls for pause/resume sync of devices

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoAdd backend method for pause/resume sync of devices
René Nussbaumer [Fri, 7 Jan 2011 13:11:37 +0000 (14:11 +0100)]
Add backend method for pause/resume sync of devices

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoMake it possible to pause/resume sync of mirrored dev
René Nussbaumer [Fri, 7 Jan 2011 13:10:48 +0000 (14:10 +0100)]
Make it possible to pause/resume sync of mirrored dev

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix some doc issues introduced in recent commits
Iustin Pop [Tue, 11 Jan 2011 10:16:24 +0000 (11:16 +0100)]
Fix some doc issues introduced in recent commits

The hooks is due to me not checking the recent patch from David, and
the cli.py change is due to the utils.py split.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoMore fixes to hooks documentation
David Knowles [Mon, 10 Jan 2011 16:17:15 +0000 (11:17 -0500)]
More fixes to hooks documentation

Signed-off-by: David Knowles <dknowles@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix docstring for "cmd" in qa_utils.GetSSHCommand
Adeodato Simo [Mon, 10 Jan 2011 15:05:30 +0000 (15:05 +0000)]
Fix docstring for "cmd" in qa_utils.GetSSHCommand

Docstring indicated "cmd" should be a list, when it's expected that
it'll be a string.

Also, converted docstring to current epydoc-style list of function
arguments.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoAdd QA tests for OpAssignGroupNodes
Adeodato Simo [Fri, 7 Jan 2011 21:56:39 +0000 (21:56 +0000)]
Add QA tests for OpAssignGroupNodes

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoExpose OpAssignGroupNodes over RAPI and RAPI client
Adeodato Simo [Fri, 7 Jan 2011 21:10:25 +0000 (21:10 +0000)]
Expose OpAssignGroupNodes over RAPI and RAPI client

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoCluster verify warns if there are split instances
Adeodato Simo [Thu, 16 Dec 2010 16:20:43 +0000 (16:20 +0000)]
Cluster verify warns if there are split instances

Emit a warning for each instance with a mirrored disk template that has
its primary and secondary nodes living in different node groups.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoAdd `gnt-group assign-nodes` in the CLI
Adeodato Simo [Fri, 7 Jan 2011 13:26:33 +0000 (13:26 +0000)]
Add `gnt-group assign-nodes` in the CLI

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoAdd unit tests for CheckAssignmentForSplitInstances
Adeodato Simo [Fri, 7 Jan 2011 13:25:55 +0000 (13:25 +0000)]
Add unit tests for CheckAssignmentForSplitInstances

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoAdd OpAssignGroupNodes and LUAssignGroupNodes
Adeodato Simo [Fri, 7 Jan 2011 13:24:27 +0000 (13:24 +0000)]
Add OpAssignGroupNodes and LUAssignGroupNodes

This operation allows to assign nodes to a group, verifying whether
the change would leave any mirrored instance split between different
node groups.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoValidate RAPI resource paths
Michael Hanselmann [Mon, 10 Jan 2011 16:11:00 +0000 (17:11 +0100)]
Validate RAPI resource paths

To stay consistent, RAPI resources should only use a restricted set of
characters. No uppercase, no “_”, etc. This is enforced by this patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Remove unused variables
Michael Hanselmann [Mon, 10 Jan 2011 15:14:48 +0000 (16:14 +0100)]
utils: Remove unused variables

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move hashing-related code into separate file
Michael Hanselmann [Mon, 10 Jan 2011 15:13:53 +0000 (16:13 +0100)]
utils: Move hashing-related code into separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Rename _CloseFDNoErr to CloseFdNoError
Michael Hanselmann [Mon, 10 Jan 2011 15:05:37 +0000 (16:05 +0100)]
utils: Rename _CloseFDNoErr to CloseFdNoError

This makes it a public function, which is necessary for
moving code into separate files.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move logging-related code into separate file
Michael Hanselmann [Mon, 10 Jan 2011 15:02:05 +0000 (16:02 +0100)]
utils: Move logging-related code into separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move Mlockall into separate file
Michael Hanselmann [Mon, 10 Jan 2011 14:56:34 +0000 (15:56 +0100)]
utils: Move Mlockall into separate file

The unittest file is renamed to match the other utils.* tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move text-related functions into separate file
Michael Hanselmann [Mon, 10 Jan 2011 14:48:00 +0000 (15:48 +0100)]
utils: Move text-related functions into separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Split Retry & co. into separate file
Michael Hanselmann [Fri, 7 Jan 2011 17:08:03 +0000 (18:08 +0100)]
utils: Split Retry & co. into separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Split algorithms into separate file
Michael Hanselmann [Fri, 7 Jan 2011 17:06:32 +0000 (18:06 +0100)]
utils: Split algorithms into separate file

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoutils: Move to separate module
Michael Hanselmann [Fri, 7 Jan 2011 16:15:42 +0000 (17:15 +0100)]
utils: Move to separate module

This is preparing for splitting the whole units module.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoAdd unit test for ganeti.rapi.client's ModifyGroup
Adeodato Simo [Fri, 7 Jan 2011 20:49:36 +0000 (20:49 +0000)]
Add unit test for ganeti.rapi.client's ModifyGroup

This was forgotten in f18fab7.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agomcpu: Automatically build the DISPATCH_TABLE
Iustin Pop [Mon, 10 Jan 2011 11:11:09 +0000 (12:11 +0100)]
mcpu: Automatically build the DISPATCH_TABLE

While reviewing dato's interdiff for the OpAssignGroupNodes, I
realised that we can do better. This patch replaces the hand-built
DISPATCH_TABLE with one built from the opcode.OP_MAPPING dict.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoGeneralize the OpCode-should-be-in-mcpu test
Iustin Pop [Mon, 10 Jan 2011 10:51:35 +0000 (11:51 +0100)]
Generalize the OpCode-should-be-in-mcpu test

Currently, the unittest TestDispatchTable in mcpu unittest does a
hard-coded approach to test whether an opcode should be included or
not in the mcpu.Processor dispatch table. This is not flexible, so we
replace it with two changes:

- first, we do not return the base OpCode in opcodes.OP_MAPPING;
  rationale being that we shouldn't need to serialise or deserialise
  this opcode during normal operation
- second, we add a new attribute WITH_LU (other name suggestions
  welcome) to the base opcode set to True, and we explicitly set it to
  False in OpTestDummy, thus automatic skipping of such LUs in the
  unittest (and in other places)

To correct for the new behaviour, the tests in the opcode unittests
are updated to include OpCode explicitly.

Note: we also replace assert_ with assertTrue in the affected unittest
(as assert_ is deprecated in 2.7).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoDocument the intended use of (vm|master)_capable
Iustin Pop [Fri, 7 Jan 2011 14:27:55 +0000 (15:27 +0100)]
Document the intended use of (vm|master)_capable

This patch adds the intended usage scenarios of these two flags.

Additionally it corrects the tag valid charset.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoIAllocator: fix breakage for non-vm_capable nodes
Iustin Pop [Fri, 7 Jan 2011 11:24:48 +0000 (12:24 +0100)]
IAllocator: fix breakage for non-vm_capable nodes

Currently, the IAllocator doesn't know that non-vm_capable nodes
should not be queried for hypervisor/instance information.

This patch updates the node data computation to generate basic info,
but not the runtime data for such nodes. Furthermore, the
documentation is updated to note that such nodes will node have
runtime data.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoFix syntax error with Python 2.4
Michael Hanselmann [Fri, 7 Jan 2011 16:00:40 +0000 (17:00 +0100)]
Fix syntax error with Python 2.4

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoUse new console information in hypervisor abstraction
Michael Hanselmann [Fri, 7 Jan 2011 15:34:19 +0000 (16:34 +0100)]
Use new console information in hypervisor abstraction

This makes use of the new way of returning console information from the
master daemon. Unittests are included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agoIntroduce verbose opcode result for console
Michael Hanselmann [Fri, 7 Jan 2011 11:31:02 +0000 (12:31 +0100)]
Introduce verbose opcode result for console

With this patch OpConnectConsole will no longer just return a command
with arguments, but rather a detailed description about how to connect
to an instance's console. Unittests for some parts are included. Another
patch will follow to adjust the hypervisor abstractions for the new
model.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agoluxi.Client: Add function to close connection
Michael Hanselmann [Thu, 6 Jan 2011 18:16:21 +0000 (19:16 +0100)]
luxi.Client: Add function to close connection

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoMerge branch 'devel-2.3'
Michael Hanselmann [Fri, 7 Jan 2011 10:12:05 +0000 (11:12 +0100)]
Merge branch 'devel-2.3'

* devel-2.3:
  Remove unused import from client.gnt_instance
  gnt-instance console: Improve error reporting
  Increase timeout for connection on remote import
  import-export: Improve timeout error reporting

Conflicts:
lib/cmdlib.py: Trivial
lib/opcodes.py: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoRemove unused import from client.gnt_instance
Michael Hanselmann [Thu, 6 Jan 2011 19:34:26 +0000 (20:34 +0100)]
Remove unused import from client.gnt_instance

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

13 years agoUpdating hooks documentation with missing environment variables
David Knowles [Thu, 6 Jan 2011 22:09:47 +0000 (17:09 -0500)]
Updating hooks documentation with missing environment variables

Signed-off-by: David Knowles <dknowles@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agognt-instance console: Improve error reporting
Michael Hanselmann [Thu, 6 Jan 2011 15:38:31 +0000 (16:38 +0100)]
gnt-instance console: Improve error reporting

If the SSH command fails, this will give a more detailed error
message than before.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoIncrease timeout for connection on remote import
Michael Hanselmann [Thu, 25 Nov 2010 19:47:44 +0000 (20:47 +0100)]
Increase timeout for connection on remote import

The source cluster has to shut down an instance before it can be
exported. Doing so can take a while, but the default connection timeout
is only 60 seconds. Adding the shutdown timeout on the receiving cluster
should help.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit dae91d021ecba089c478163b25dc426abb589351)

13 years agoimport-export: Improve timeout error reporting
Michael Hanselmann [Thu, 6 Jan 2011 16:36:38 +0000 (17:36 +0100)]
import-export: Improve timeout error reporting

When the source cluster takes too long to create a snapshot, the
destination would time out. Unfortunately no good error message was
written unless debug logging was enabled, not even to the log file. This
will be improved with this patch.

Another patch to be backported from master will hopefully avoid this
situation completely.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoList recorded powered state in gnt-node info
René Nussbaumer [Thu, 6 Jan 2011 13:41:27 +0000 (14:41 +0100)]
List recorded powered state in gnt-node info

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoSupport query of node field 'powered'
René Nussbaumer [Thu, 6 Jan 2011 12:13:23 +0000 (13:13 +0100)]
Support query of node field 'powered'

This field is based on OOB support and is only available if there's oob
support for that node.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoqa_group.py: reimplement query tests with qa_utils
Adeodato Simo [Wed, 5 Jan 2011 17:57:10 +0000 (17:57 +0000)]
qa_group.py: reimplement query tests with qa_utils

Now that group queries use query2 infrastructure, update the QA tests to
use the generic functions in qa_utils.py.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoganeti.query_unittest.py: add tests for group queries
Adeodato Simo [Wed, 5 Jan 2011 16:42:59 +0000 (16:42 +0000)]
ganeti.query_unittest.py: add tests for group queries

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoConvert “gnt-group list” to query2
Adeodato Simo [Wed, 5 Jan 2011 12:04:15 +0000 (12:04 +0000)]
Convert “gnt-group list” to query2

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agocmdlib.py: convert LUQueryGroups to query2
Adeodato Simo [Wed, 5 Jan 2011 12:40:51 +0000 (12:40 +0000)]
cmdlib.py: convert LUQueryGroups to query2

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agocmdlib.py: move _GetQueryImplementation to end of file
Adeodato Simo [Wed, 5 Jan 2011 11:58:19 +0000 (11:58 +0000)]
cmdlib.py: move _GetQueryImplementation to end of file

_GetQueryImplementation() uses _QUERY_IMPL, which list all query type
implementations. By moving it to the end of the file, we ensure all such
classes are defined by the time we list them.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoquery.py: add definitions for node group queries
Adeodato Simo [Wed, 5 Jan 2011 11:56:49 +0000 (11:56 +0000)]
query.py: add definitions for node group queries

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoconstants.py: define QR_GROUP resource
Adeodato Simo [Wed, 5 Jan 2011 12:01:43 +0000 (12:01 +0000)]
constants.py: define QR_GROUP resource

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoganeti.query_unittest.py: test lock fields too
Adeodato Simo [Thu, 6 Jan 2011 12:27:40 +0000 (12:27 +0000)]
ganeti.query_unittest.py: test lock fields too

Additionally, change TestQueryFields.testSomeFields() to handle lists of
fields shorter than 20 elements.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agolvmstrap: also test sysfs holders
Iustin Pop [Wed, 5 Jan 2011 13:19:29 +0000 (14:19 +0100)]
lvmstrap: also test sysfs holders

If a device has entries in its holder directory
(/sys/block/$name/holders), it means that some kernel system "uses"
that device, and hence should not be considered available.

This patch adds a new 'in-use' check based on this sysfs test, and
introduces an overal InUse function for devices.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agolvmstrap: add support for non-partitioned md disks
Iustin Pop [Wed, 5 Jan 2011 12:56:00 +0000 (13:56 +0100)]
lvmstrap: add support for non-partitioned md disks

This patch, originally written by Marc Schmitt <mschmitt@google.com>,
adds support for MD devices (used in a non-partitioned mode). I
abstracted all the original startswith('md') checks into separate
functions, and also moved the supported disk types to a list.

Proper "in-use" detection also needs another check, which will come in
a subsequent patch.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoRPC: mark jobqueue functions as URGENT
Iustin Pop [Tue, 4 Jan 2011 09:59:05 +0000 (10:59 +0100)]
RPC: mark jobqueue functions as URGENT

Recently, we've seen more and more cases of a specific breakage
pattern in Ganeti: master candidates which are semi-alive (as in, they
respond to ping, they can complete a TCP/SSL handshake, but otherwise
the root filesystem is broken) cause lots of confusion within masterd.

My analysis shows that waiting up to 5 minutes for a reply from such a
broken master candidate is too long, and this long wait breaks other
timeouts (e.g. the Luxi timeout), making standard recovery from this
situation very hard. It's much easier to kill the master daemon, edit
manually the config file and mark the node as regular, then restart
the master daemon.

The proposal is therefore to reduce the timeout for the job queue
functions to TMO_URGENT (1 minute), which should be more balanced
between a working but overloaded node and a broken node.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoMerge branch 'devel-2.3'
Michael Hanselmann [Thu, 6 Jan 2011 10:27:23 +0000 (11:27 +0100)]
Merge branch 'devel-2.3'

* devel-2.3:
  cfgupgrade: Remove unused “program” variable
  cfgupgrade: Check master name, clarify question
  Makefile: Merge build-time reST copying
  Move doc/upgrade.rst to UPGRADE, copy at build-time
  Import upgrade notes into documentation

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agocfgupgrade: Remove unused “program” variable
Michael Hanselmann [Thu, 6 Jan 2011 10:25:32 +0000 (11:25 +0100)]
cfgupgrade: Remove unused “program” variable

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agoQA: Remove 'oob_program=default' on gnt-cluster modify
René Nussbaumer [Thu, 6 Jan 2011 10:06:34 +0000 (11:06 +0100)]
QA: Remove 'oob_program=default' on gnt-cluster modify

On cluster level there's no 'default' because it's the highest cascading
level. Due to this 'default' is a valid value and doesn't mean to remove
the value from the dict like it does on the other levels. To overcome
this we just empty the string.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoConvert “gnt-debug locks” to query2
Michael Hanselmann [Thu, 6 Jan 2011 10:08:11 +0000 (11:08 +0100)]
Convert “gnt-debug locks” to query2

Locks can now be queried using “Query(what="lock", …)” over LUXI.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agocfgupgrade: Check master name, clarify question
Michael Hanselmann [Wed, 5 Jan 2011 17:41:59 +0000 (18:41 +0100)]
cfgupgrade: Check master name, clarify question

- Check hostname and abort if it doesn't match contents of
  “ssconf_master_node”, can be overridden using “--ignore-hostname”
  parameter.
- Clarify confirmation question and don't mention instances anymore.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoMakefile: Merge build-time reST copying
Michael Hanselmann [Wed, 5 Jan 2011 17:52:29 +0000 (18:52 +0100)]
Makefile: Merge build-time reST copying

No need to copy this snippet around, “make” can work harder for us.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoMove doc/upgrade.rst to UPGRADE, copy at build-time
Michael Hanselmann [Wed, 5 Jan 2011 17:48:29 +0000 (18:48 +0100)]
Move doc/upgrade.rst to UPGRADE, copy at build-time

This will allow distributions to install the file as text documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoImport upgrade notes into documentation
Michael Hanselmann [Wed, 5 Jan 2011 15:22:33 +0000 (16:22 +0100)]
Import upgrade notes into documentation

This patch formats the upgrade notes currently in the wiki[1] as reST
and adds them to the documentation.

[1] http://code.google.com/p/ganeti/wiki/UpgradeNotes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix OpSetInstanceParams.disk_template check
Michael Hanselmann [Wed, 5 Jan 2011 16:45:54 +0000 (17:45 +0100)]
Fix OpSetInstanceParams.disk_template check

When moving the opcode parameters I moved two or three checks from an
opcode's CheckArguments function to the type checks. This was one of
them and unfortunately I didn't notice the parameter can be None.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoRAPI: Add resource to grow instance disk
Michael Hanselmann [Wed, 5 Jan 2011 11:54:15 +0000 (12:54 +0100)]
RAPI: Add resource to grow instance disk

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoReword "one of hmgt" as "one of h/m/g/t" for clarity
Adeodato Simo [Tue, 4 Jan 2011 21:01:29 +0000 (21:01 +0000)]
Reword "one of hmgt" as "one of h/m/g/t" for clarity

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoQA: Adding new cluster verify cases
René Nussbaumer [Tue, 4 Jan 2011 15:21:32 +0000 (16:21 +0100)]
QA: Adding new cluster verify cases

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoout of band verification in gnt-cluster verify
René Nussbaumer [Tue, 4 Jan 2011 12:34:49 +0000 (13:34 +0100)]
out of band verification in gnt-cluster verify

This adds the verify tests for out of band management

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoAdding additional VerifyNode checks to backend
René Nussbaumer [Tue, 4 Jan 2011 10:15:53 +0000 (11:15 +0100)]
Adding additional VerifyNode checks to backend

This adds checks for out of band support. The helpers have to exist and
they have to be executable.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoRAPI: Add resource to modify cluster
Michael Hanselmann [Tue, 4 Jan 2011 17:30:29 +0000 (18:30 +0100)]
RAPI: Add resource to modify cluster

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agobaserlib: Add function for filling opcodes
Michael Hanselmann [Tue, 4 Jan 2011 16:29:50 +0000 (17:29 +0100)]
baserlib: Add function for filling opcodes

This function makes use of the opcode parameters which now live
directly in the opcode. A number of RAPI resources can now be simplified.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoImprove opcode summary tests
Michael Hanselmann [Fri, 31 Dec 2010 15:42:49 +0000 (16:42 +0100)]
Improve opcode summary tests

Test full summary instead of just format.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoMigrate code verifying opcode parameters to base class
Michael Hanselmann [Fri, 31 Dec 2010 15:39:53 +0000 (16:39 +0100)]
Migrate code verifying opcode parameters to base class

This allows the function to be used in other places as well.
An optional parameter is added to control whether default
values should be set. Unittests are added, providing full
coverage.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoImprove tests for OP_ID
Michael Hanselmann [Thu, 30 Dec 2010 17:22:05 +0000 (18:22 +0100)]
Improve tests for OP_ID

… by detecting duplicates.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agocmdlib: Remove opcode parameters
Michael Hanselmann [Thu, 30 Dec 2010 15:50:43 +0000 (16:50 +0100)]
cmdlib: Remove opcode parameters

Remove the parameter definitions and use those from the opcode classes
instead. Small style changes are also made (empty lines, wrapping).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoopcodes: Add opcode parameter definitions
Michael Hanselmann [Thu, 30 Dec 2010 15:50:17 +0000 (16:50 +0100)]
opcodes: Add opcode parameter definitions

This is the first step for migrating them from cmdlib. A metaclass is
used to define “__slots__” upon class creation time (not instantiation).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoquery2: Add new field status “offline”
Michael Hanselmann [Wed, 29 Dec 2010 17:03:36 +0000 (18:03 +0100)]
query2: Add new field status “offline”

This allows “gnt-node list” to show the difference between modes marked
offline and nodes with e.g. RPC errors (“(nodata)”). node1 is the
master, node2's node daemon crashed and node3 is marked offline:

$ gnt-node list -o name,offline,dtotal,dfree
Node              Offline    DTotal     DFree
node1.example.com N            1.3T      1.3T
node2.example.com N        (nodata)  (nodata)
node3.example.com Y       (offline) (offline)

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoQA: Fix out-of-band tests
Michael Hanselmann [Thu, 30 Dec 2010 17:48:36 +0000 (18:48 +0100)]
QA: Fix out-of-band tests

- Handle situations with no non-master node
- Expand node name to make test work when configuration just has short
  names (e.g. “node1” instead of “node1.example.com”)

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agoAdd unittests for ht module
Michael Hanselmann [Tue, 4 Jan 2011 14:32:16 +0000 (15:32 +0100)]
Add unittests for ht module

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoht.TInt: Exclude boolean values
Michael Hanselmann [Tue, 4 Jan 2011 14:40:40 +0000 (15:40 +0100)]
ht.TInt: Exclude boolean values

See inline comment.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoCleanup bootstrap.SetupNodeDaemon
Michael Hanselmann [Tue, 4 Jan 2011 12:45:04 +0000 (13:45 +0100)]
Cleanup bootstrap.SetupNodeDaemon

- Code formatting
- Use ShellQuote for one argument
- Remove variables no longer used after commit 9294514d

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

13 years agoMerge branch 'devel-2.3'
Michael Hanselmann [Fri, 31 Dec 2010 12:39:10 +0000 (13:39 +0100)]
Merge branch 'devel-2.3'

* devel-2.3:
  Fix typo in gnt-instance manpage
  jqueue: Fix cancelling while in waitlock in queue
  cli: Extend message for LUXI timeouts
  Fix timeout handling in LUXI client

Conflicts:
man/gnt-instance.sgml: Trivial merge in gnt-instance.rst

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix build errors with ganeti-listrunner
Michael Hanselmann [Fri, 31 Dec 2010 12:08:16 +0000 (13:08 +0100)]
Fix build errors with ganeti-listrunner

- Remove non-ASCII character from manpage
- Reformat docstring for epydoc in script

These caused build breakage on some but not all distributions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoFix typo in gnt-instance manpage
Michael Hanselmann [Fri, 31 Dec 2010 12:11:05 +0000 (13:11 +0100)]
Fix typo in gnt-instance manpage

s/os-name/os-type/. This was reported in issue 133.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agocli: Change “<…>” in query output to “(…)”
Michael Hanselmann [Wed, 29 Dec 2010 16:42:28 +0000 (17:42 +0100)]
cli: Change “<…>” in query output to “(…)”

This should reduce the amount of damage in case of accidential copy &
paste.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoInitial import of listrunner
Michael Hanselmann [Wed, 29 Dec 2010 16:35:52 +0000 (17:35 +0100)]
Initial import of listrunner

This tool was used and worked on internally for quite a long time. We
decided to include it in Ganeti.

Known issues:
- Code doesn't match rest of Ganeti (e.g. using “print” all over the
  place, hardcoded calls to sys.exit deep in functions)
- Code duplication from Ganeti library (e.g. PingByTcp/netutils.TcpPing,
  GetHosts/utils.ReadFile)
- Using ssh-agent doesn't work with more than one worker (Paramiko keeps
  the socket open and the file descriptor is used from different
  workers)
  - No clear separation between parent and child process in code
- Uses getopt instead of optparse

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agojqueue: Fix cancelling while in waitlock in queue
Michael Hanselmann [Tue, 21 Dec 2010 18:10:32 +0000 (19:10 +0100)]
jqueue: Fix cancelling while in waitlock in queue

Since the recent change to leave jobs in the “waitlock” status (commit
5fd6b6947), cancelling a job while it's back in the queue would break.
This patch handles these cases and adds a unittest.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoLUInstanceRename: log result of name resolving
Iustin Pop [Fri, 24 Dec 2010 08:27:15 +0000 (09:27 +0100)]
LUInstanceRename: log result of name resolving

While the LU does return the final name, it's useful to log the actual
DNS resolving process (input and output) in order to help with the
diagnose of failures.

The patch also fixes the docstring of the Exec() function.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

13 years agoFix QA for “list-fields” commands
Michael Hanselmann [Tue, 21 Dec 2010 16:39:42 +0000 (17:39 +0100)]
Fix QA for “list-fields” commands

The list of fields is not only sorted, but sorted in a nice way.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoRemove utils.FormatTimestampWithTZ
Michael Hanselmann [Tue, 21 Dec 2010 16:37:25 +0000 (17:37 +0100)]
Remove utils.FormatTimestampWithTZ

Long story short: time.strftime("%Z", time.localtime()) doesn't work,
even though it's documented to be equivalent to time.strftime("%Z").

$ TZ=America/Sao_Paulo python -c 'import time; print
time.strftime("%Z"), time.strftime("%Z", time.localtime())'
BRST LMT

References:
http://bugs.python.org/issue762963
https://bugs.launchpad.net/ubuntu/+source/python2.6/+bug/564607
http://stackoverflow.com/questions/4367896/issue-with-timezone-with-time-strftime

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoEnsure temp files from RunCmd tests are removed
Michael Hanselmann [Tue, 21 Dec 2010 16:34:43 +0000 (17:34 +0100)]
Ensure temp files from RunCmd tests are removed

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

13 years agoAllow customisation of the disk index separator
Iustin Pop [Tue, 21 Dec 2010 13:18:39 +0000 (14:18 +0100)]
Allow customisation of the disk index separator

As per issue 124, some Xen versions (or packaging) don't deal nicely
with the colon being part of a disk name. Therefore we add a
configure-time option for customising this.

Note: setting the separator to interesting values like / is not
handled by the code. This being a configure-time option (e.g. to be
set by distribution packagers), we assume the person building the code
knows what they are doing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

13 years agoutils: Timezone fixes and tests
Michael Hanselmann [Fri, 17 Dec 2010 17:38:40 +0000 (18:38 +0100)]
utils: Timezone fixes and tests

- Update docstrings to explicitely mention Epoch
- Fix timezone bug in FormatTimestampWithTZ, where it would
  use GMT/UTC when it should use the local timezone
- Add unittests for time formatting functions

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>