Michael Hanselmann [Mon, 31 Jan 2011 16:03:18 +0000 (17:03 +0100)]
utils.SetupLogging: Make program a mandatory argument
It's passed in by most users (daemons, CLI scripts) and for the others
(burnin, watcher) it certainly doesn't hurt, especially when using
syslog.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Mon, 31 Jan 2011 15:46:21 +0000 (16:46 +0100)]
utils.log: Restrict I/O error handling coverage
The I/O error will occur while opening the file, not while adding
and configuring the handler.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Mon, 31 Jan 2011 15:43:28 +0000 (16:43 +0100)]
utils.log: Split formatter building into separate function
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Mon, 31 Jan 2011 13:58:26 +0000 (14:58 +0100)]
burner: Trivial code cleanup
- Use constant for exit value
- Configure logging from main function, not from class' “__init__”
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Mon, 31 Jan 2011 13:54:38 +0000 (14:54 +0100)]
burnin: Reuse existing function for debug value
Instead of using its own, burnin can use cli.SetGenericOpcodeOpts.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Stephen Shirley [Tue, 1 Feb 2011 12:07:31 +0000 (13:07 +0100)]
Merge node groups from other cluster
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Stephen Shirley [Mon, 31 Jan 2011 16:07:08 +0000 (17:07 +0100)]
Enforce that new node groups have unique names
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Stephen Shirley [Mon, 31 Jan 2011 16:00:03 +0000 (17:00 +0100)]
Add _UnlockedLookupNodeGroup()
This allows calling of _UnlockedLookupNodeGroup() from within
AddNodeGroup()
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Stephen Shirley [Mon, 31 Jan 2011 14:19:48 +0000 (15:19 +0100)]
cluster-merge should refuse to merge own cluster
Also fix type of Merger.cluster_name from list to string. This would
have triggered an error in sshRunner if cluster keys were in use.
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Stephen Shirley [Mon, 31 Jan 2011 13:49:03 +0000 (14:49 +0100)]
Minor grammar fix in QuitGanetiException docstring
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Stephen Shirley [Mon, 31 Jan 2011 13:18:48 +0000 (14:18 +0100)]
Fix grammar of var naming
flatten is the verb, flattened is the adjective.
Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 31 Jan 2011 12:52:39 +0000 (13:52 +0100)]
Introduce re-openable log record handler
This patch adds a new log handler class based on the standard library's
BaseRotatingHandler. This new class allows the log file to be re-opened,
e.g. upon receiving a SIGHUP signal. The latter will be implemented in
forthcoming patches. The patch does not change the behaviour regarding
writing to /dev/console.
Quite a bit of code had to be changed to unittest the log handlers.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Fri, 28 Jan 2011 15:53:40 +0000 (16:53 +0100)]
Re-create instance disk symlinks on activate
This patch implements recreation of instance disk symlinks when the
activate-disks operation is run. Until now, it was not possible to
re-create these symlinks without stopping and starting or migrating an
instance as the RPC call where this is done was in instance startup
and migration.
In order to do this, the blockdev_assemble rpc call needs the disk
index too, which is added to the protocol. This is a change from 2.3
and makes instance startup incompatible (FYI).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 28 Jan 2011 14:21:04 +0000 (15:21 +0100)]
Add RAPI resource for instance console
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 28 Jan 2011 13:10:09 +0000 (14:10 +0100)]
Export console information as query field
This makes it possible to get the console information via a LUXI query.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Stephen Shirley [Fri, 28 Jan 2011 13:49:34 +0000 (14:49 +0100)]
manpage: gnt-group remove cannot remove last group
Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Fri, 28 Jan 2011 13:05:09 +0000 (14:05 +0100)]
ConfigWriter: add checks for be/nd/nic params
This adds checking (in the configuration) for invalid be, nd and nic
params. The code is a bit tricky as nd params are at cluster,
nodegroup and node level, nicparams are at cluster and nic level,
whereas beparams are at cluster and instance level.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Fri, 28 Jan 2011 13:04:22 +0000 (14:04 +0100)]
ConfigWriter: simplify _UnlockedVerifyConfig
This just adds a 'cluster' local variable for reducing duplication.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Guido Trotter [Fri, 28 Jan 2011 12:32:30 +0000 (13:32 +0100)]
Add e1000 nic support for HVM
Closes issue: 130
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Stephen Shirley [Fri, 28 Jan 2011 12:24:30 +0000 (13:24 +0100)]
Prevent removal of last node group
- Add check in ConfigWriter to prevent last node group from being
removed
- Tidy up error message a bit
Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 28 Jan 2011 10:31:55 +0000 (11:31 +0100)]
Fix instance list for instances running multiple times
If for some reason (e.g. failed migration) one instance is running
on multiple nodes the output can become inconsistent. To get that error
and make it consistent between runs we make the call on the secondary
too and look if it's running there. If so we report the instance as
ERROR_wrongnode.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 27 Jan 2011 19:24:18 +0000 (20:24 +0100)]
Small QA fixes: groups via RAPI, cluster OOB
Add “cluster-oob” to sample configuration file. Don't run RAPI group
tests if disabled.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 27 Jan 2011 17:07:09 +0000 (18:07 +0100)]
Merge branch 'devel-2.3' into devel-2.4
* devel-2.3:
Wait for master to become available on initialization
Start all daemons on cluster initialization
Clarify job processing order in admin guide
Improve option descriptions
Remove two unused variables
Fix LUOSDiagnose and non-vm_capable nodes
Rephrasing two error messages for auto promotion
storage: Check that mapper is either used or None
Fix bug in “gnt-node list-storage”
Improve import/export timeout settings
Increase remote import/export timeout
Conflicts:
lib/constants.py: Trivial
lib/objects.py: Trivial
qa/qa_node.py: Trivial
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 27 Jan 2011 15:44:17 +0000 (16:44 +0100)]
cluster verify: add hvparams verification
Currently, the validity of the hypervisor parameters is only checked
at init/modification time, and not in the cluster verify. This is bad,
as it can lead to inconsistent state that is only detected when the
next modification (which can be unrelated) is made, leading to
unexpected error messages.
This patch adds both syntax verification (in masterd) and validity
verification on remote nodes. The downside of the patch is that on
clusters with many instances which have custom parameters, it will be
slow. A possible improvement would be to detect duplicate, identical
set of parameters, and collapse these into a single verification, but
that is left as a TODO (in case it becomes problematic).
An additional change is in utils.ForceDict, where we said 'key',
whereas this function is always used with parameter dicts, so I
changed it to "Unknown parameter".
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Thu, 27 Jan 2011 13:47:18 +0000 (14:47 +0100)]
Remove dumb-allocator
- Remove the actual code
- Remove mentions of it from iallocator.rst, and use hail instead
- Also remove mentions of "etch-image" and use "debootstrap+default"
- Mention htools as the reference implementation in iallocator.rst
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Stephen Shirley [Thu, 27 Jan 2011 14:16:09 +0000 (15:16 +0100)]
Open other clusters' config in foreign mode
Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Stephen Shirley [Thu, 27 Jan 2011 15:18:48 +0000 (16:18 +0100)]
Add (unused) arg to _OfflineClusterMerge
cli._RunWhileClusterStoppedHelper.Call passes (self, *args) to functions
called via cli.RunWhileClusterStoppedHelper(). The code in cluster-merge
was broken by commit
d8aab233.
Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 27 Jan 2011 13:04:41 +0000 (14:04 +0100)]
Fix unittest breakage on Python 2.4/2.5
Commit
70b0d2a29 broke unittests on Python 2.4 and 2.5. Turns out that
Python 2.6 and above allow classes to be passed as custom test runners,
whereas earlier versions don't.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 26 Jan 2011 18:36:45 +0000 (19:36 +0100)]
Check for duplicate RAPI URIs and handlers
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 26 Jan 2011 18:18:16 +0000 (19:18 +0100)]
Ensure all resources are used by RAPI client
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Wed, 26 Jan 2011 18:17:18 +0000 (19:17 +0100)]
RAPI client: De-/activating instance disks
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Wed, 26 Jan 2011 18:09:28 +0000 (19:09 +0100)]
RAPI client: Wrap /2/redistribute-config resource
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Wed, 26 Jan 2011 18:04:19 +0000 (19:04 +0100)]
Add unittest for RAPI client's ModifyInstance
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Thu, 27 Jan 2011 09:22:50 +0000 (10:22 +0100)]
Watcher: Fix endless repair tries for broken secondary
In cases where secondary was offline and not evacuated watcher tried
to activate-disks in an endless manner, but this is useless, as the
secondary is offline and therefore not responding to this approach.
This patch skips activation of the disk if the secondary is bad but
instance up and running.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 26 Jan 2011 17:54:36 +0000 (18:54 +0100)]
Verify disks: increase parallelism and other fixes
The recent work on multi-VG support has converted LUClusterVerifyDisks
into doing serialised calls to each node, as each node can have
different VGs. This is suboptimal, especially for big clusters, where
this LU is executed by the watcher very often.
This patch changes the logic based on the observation that querying a
node for its VGs and then requesting a LV list for those VGs is
equivalent to simply asking for all LVs, without specifying the VG
name(s). So backend.py needs changes to accept an empty VG list, and
the LU itself partially reverts to the previous version.
Additionally, we do two other fixes to this LU:
- small improvement in getting the instance list from the config
- MapLVsByNode works for all disk types, hence no need to restrict to
the DRBD template, especially as today we can "recreate" disks for
plain volumes too (the warning message in gnt-cluster is updated
too)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 26 Jan 2011 17:51:19 +0000 (18:51 +0100)]
gnt-cluster verify-disks: fix VG name
Recent multi-VG work already exports the missing LV names as vg/lv,
not simply lv. So the query and addition of the VG name in gnt-cluster
verify-disks is redundant, and even wrong for non-default-VG
instances.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 26 Jan 2011 16:45:10 +0000 (17:45 +0100)]
Deactivate disks: allow skipping hypervisor checks
In some cases (e.g. the hypervisor not running at all), we might want
to force disk deactivation, skipping the hypervisor checks. I believe
this is not a good thing to do all the time, so this patch adds the
force option to allow manual selection of this operation mode.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 26 Jan 2011 15:46:56 +0000 (16:46 +0100)]
Wait for master to become available on initialization
This is analogue to the existing check for a responsive node daemon.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 26 Jan 2011 15:45:11 +0000 (16:45 +0100)]
Start all daemons on cluster initialization
At least ganeti-confd was not started. It got started a few minutes
later by ganeti-watcher. Also move one pylint disable to the effective
line.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 26 Jan 2011 12:17:33 +0000 (13:17 +0100)]
Clarify job processing order in admin guide
The fact that jobs don't necessarily execute in order has been source
for some confusion. Hopefully this update will clarify things.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 26 Jan 2011 12:08:59 +0000 (13:08 +0100)]
Improve option descriptions
Also replace hardcoded “xenvg” with constant.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 26 Jan 2011 13:13:09 +0000 (14:13 +0100)]
Remove two unused variables
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 26 Jan 2011 13:39:39 +0000 (14:39 +0100)]
Show hidden/blacklisted OSes in cluster info
Since we can blacklist/hide non-existing OSes (for preseeding), we
cannot query easily the OSes themselves for this status. Hence we
export the entire lists in cluster info (which should be cheaper than
gnt-os diagnose).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Wed, 26 Jan 2011 12:46:07 +0000 (13:46 +0100)]
Further man page updates for OS parameters
Also replace one UTF-8 char with the ASCII equivalent, not all Pandoc
versions support it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Wed, 26 Jan 2011 12:44:16 +0000 (13:44 +0100)]
Change the Makefile to use bash as SHELL
This is because we want, whenever we use sequences of commands, to set
pipefail, otherwise detecting build failures is difficult.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Wed, 26 Jan 2011 12:10:24 +0000 (13:10 +0100)]
Add documentation for OS parameters
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 26 Jan 2011 12:07:59 +0000 (13:07 +0100)]
Fix LUOSDiagnose and non-vm_capable nodes
This skips non-vm_capable nodes in the OS diagnose search, since such
OSes will not be used anyway on those nodes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Wed, 26 Jan 2011 10:51:33 +0000 (11:51 +0100)]
Rephrasing two error messages for auto promotion
Using auto_promote or auto-promote can lead to confusion on using the
user facing interfaces. While auto-promote is fine for CLI it's not for
RAPI and vice-versa. This patch should eliminate this confusion.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 26 Jan 2011 10:19:03 +0000 (11:19 +0100)]
lvmstrap: fix logic bug for partition reread
The if structure in CheckReread is broken, and makes partitions reread
be full of race issues (esp. after updating them).
Also fix a small message bug.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 26 Jan 2011 09:01:41 +0000 (10:01 +0100)]
storage: Check that mapper is either used or None
This is a followup patch to the one moving GetAllocatable out to
module level.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 25 Jan 2011 18:28:59 +0000 (19:28 +0100)]
Fix bug in “gnt-node list-storage”
LVM PV storage units would always show as allocatable, even when they
weren't. For some reason I have not been able to determine, the function
parsing the attributes (“_GetAllocatable”) was not even called and the
list opcode simply returned the attribute string as the value (e.g.
“a-”). Removing “@staticmethod” did the trick and then I just moved it
to module level.
A QA test is included.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 25 Jan 2011 12:53:37 +0000 (13:53 +0100)]
Fix payload check for out-of-band health
This logic error was not detected before as health has not been
implemented on the cli and therefore no QA code existed for that.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 25 Jan 2011 10:18:41 +0000 (11:18 +0100)]
Fix premature abort of LUOobCommand due to result.Raise
This is a bug I recognized while doing tests on gnt-node health. A left
over result.Raise line causes premature abort of LUOobCommand on the
first node failing the RPC call. This is not expected behaviour for
multiple nodes.
This bug is not critical for the current use case as for a single node
this just works as expected. Though for multi node this will fail,
that's the reason why QA and tests passed as currently we just had
single node invocation. QA for multi node will be included in the
gnt-node health, as this is the first command using multi node.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 25 Jan 2011 16:59:36 +0000 (17:59 +0100)]
ht: Add TMaybeDict check
This replaces a number of equal “ht.TOr(ht.TDict, ht.TNone)” checks.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 24 Jan 2011 11:01:16 +0000 (12:01 +0100)]
QA: replace ping with fping
This allows a lot of simplification in the TestIcmpPing, as fping can
take multiple arguments so we don't need anymore to create many
commands joined with &&.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Thu, 20 Jan 2011 12:47:40 +0000 (13:47 +0100)]
Modify LUOobCommand to support multiple nodes
This will change the result of this LU to a query like result. A list of
tuples with information about the state of the data.
It also includes the modification to the commands calling this opcode.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Fri, 21 Jan 2011 15:22:18 +0000 (16:22 +0100)]
Merge branch 'devel-2.4'
* devel-2.4:
Another fix for LUClusterVerifyDisks
QA: also run gnt-cluster verify-disks
Fix disk adoption breakage
Fix typo in query2 design document
Improve documentation for QRFS_UNAVAIL
lvmstrap: add PV-on-partition support
lvmstrap: abstract a little the sysfs paths
lvmstrap: ignore small-sized partitions
lvmstrap: add explicit test for swap backends
lvmstrap: add an explicit test for mounted devices
lvmstrap: add more excluded FS types
lvmstrap: fix very old contact information
Instance query: replace duplicates with aliases
query: Add alias support in _PrepareFieldList
Fix disk count check in LUSetInstanceParams
Document iallocator change (alloc_policy)
Conflicts:
lib/constants.py (due to QRFS→RS on master and doc on 2.4)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Fri, 21 Jan 2011 08:42:38 +0000 (09:42 +0100)]
Rename QRFS_* to RS_*
This patch renames QRFS_* to RS_* fields so they can be used in other
places (i.e. LUs) without confusion, as this was initially meant for
query operations.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 20 Jan 2011 16:55:35 +0000 (17:55 +0100)]
Another fix for LUClusterVerifyDisks
The LVM queries should only be done for vm_capable nodes. In order to
do this, we also add a new ConfigWriter method to abstract that query.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 20 Jan 2011 15:04:00 +0000 (16:04 +0100)]
QA: also run gnt-cluster verify-disks
The bug recently reported by Apollon Oikonomopoulos was missed because
we don't test this command at all.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Apollon Oikonomopoulos [Tue, 18 Jan 2011 10:30:12 +0000 (12:30 +0200)]
Fix disk adoption breakage
Disk adoption is currently broken by
84d7e26b, which added multiple LVM
volume group support. This patch fixes the calls to rpc.call_vg_list,
which are multi-node calls but were handled as single-node calls in
84d7e26b.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 20 Jan 2011 14:58:41 +0000 (15:58 +0100)]
Fix typo in query2 design document
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 20 Jan 2011 12:05:58 +0000 (13:05 +0100)]
Improve import/export timeout settings
With this patch, the exporting node will retry to connect a few times.
The receiving node will make use of the master's increased timeout (see
previous patch).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 20 Jan 2011 11:24:14 +0000 (12:24 +0100)]
Increase remote import/export timeout
It's been shown that 60 seconds may not be enough to establish a
connection.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 20 Jan 2011 11:31:46 +0000 (12:31 +0100)]
Improve documentation for QRFS_UNAVAIL
IMHO this should have been named QRFS_NA or QRFS_UNSUPPORTED, but
UNAVAIL is good enough.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 19 Jan 2011 16:52:10 +0000 (17:52 +0100)]
lvmstrap: add PV-on-partition support
This is a not-so-nice change, adding support for partitions to be used
as PVs.
The not-nice part is that partitions live in a separate place in
sysfs, whereas in dev they live at the same level as disks. We
workaround this via a new SysfsName function that computes the correct
sysfs base path for a given disk.
The other rule is that if a disk is not in use, we ignore its
partitions completely, as the disk will be re-partitioned anyway. Only
if the disk is busy, we consider each of its partitions for the
free/busy list.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 19 Jan 2011 16:49:26 +0000 (17:49 +0100)]
lvmstrap: abstract a little the sysfs paths
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 19 Jan 2011 16:11:40 +0000 (17:11 +0100)]
lvmstrap: ignore small-sized partitions
This patch changes lvmstrap to ignore “small” partitions. Currently
extended partitions are reported as unused as with a size of 1024
(bytes), and this confuses lvmstrap. Since a very small partition
won't help anyway (below hundred of PE size is not helpful), let's
restrict it to 1GB.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 19 Jan 2011 13:49:50 +0000 (14:49 +0100)]
lvmstrap: add explicit test for swap backends
Similar to mounted filesystems, recent kernel/userland report swap
backends:
root@node4:~# fuser -avm /dev/sda6
USER PID ACCESS COMMAND
/dev/sda6: root kernel swap /dev/sda6
But old ones not:
node1# fuser -avm /dev/sda6
USER PID ACCESS COMMAND
/dev/sda6:
So we add an explicit test for swap backends using /proc/swaps.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 19 Jan 2011 13:46:44 +0000 (14:46 +0100)]
lvmstrap: add an explicit test for mounted devices
Recent kernels/userland report a mounted filesystem as follows:
root@node4:~# fuser -avm /dev/sda5
USER PID ACCESS COMMAND
/dev/sda5: root kernel mount /srv/ganeti
But old ones do not:
node1# fuser -avm /dev/sda5
USER PID ACCESS COMMAND
/dev/sda5:
So in order to cover both, we add an explicit test for this. We
already compute the device numbers of all mounted filesystems, so it
is a trivial addition (GetMountInfo() is cheap, so we don't need to
cache it).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 19 Jan 2011 11:10:36 +0000 (12:10 +0100)]
lvmstrap: add more excluded FS types
Also moves the list of excluded types to the top level and make it a
frozenset.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 19 Jan 2011 17:02:55 +0000 (18:02 +0100)]
lvmstrap: fix very old contact information
A memory from the past: this was left over from before the 1.2 release
or so…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 19 Jan 2011 09:17:52 +0000 (10:17 +0100)]
Instance query: replace duplicates with aliases
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Wed, 19 Jan 2011 09:16:10 +0000 (10:16 +0100)]
query: Add alias support in _PrepareFieldList
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Tue, 18 Jan 2011 15:22:06 +0000 (16:22 +0100)]
Adding a basic oob helper as an example
This is just a plain stupid and simple out-of-band helper without
anything fancy. It uses plain ssh to power off / power cycle the
machine, does not support power on. It support power status using fping
to check if the host replies.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Apollon Oikonomopoulos [Tue, 18 Jan 2011 10:31:58 +0000 (12:31 +0200)]
Fix disk count check in LUSetInstanceParams
LUSetInstanceParams checked instance.nics (and not instance.disks)
against constants.MAX_DISKS.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Balazs Lecz [Wed, 19 Jan 2011 14:29:05 +0000 (14:29 +0000)]
Document iallocator change (alloc_policy)
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 18 Jan 2011 16:58:33 +0000 (17:58 +0100)]
Add check-news to EXTRA_DIST
This was missing from commit
7385c51d.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 18 Jan 2011 16:26:26 +0000 (17:26 +0100)]
Add script checking release dates in NEWS
This will detect human errors when setting a release date in NEWS.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Sun, 16 Jan 2011 15:57:53 +0000 (16:57 +0100)]
query: use the actual types for BE/HV parameters
This patch exposes the VTYPE kind of BE/HV parameters, instead of
returning QFT_OTHER. The current situation makes a query like:
gnt-instance list -o name,be/memory,oper_ram
very strange looking.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sun, 16 Jan 2011 16:28:33 +0000 (17:28 +0100)]
query: return UNAVAIL for "wrong" HV parameters
If a HV parameter is required that does not apply for an instance,
currently the code returns None. This is bad, as it means we cannot
switch to the actual HV parameter types and validate correctly this
field.
This patch changes it so that in this case we return QRFS_UNAVAIL;
ideally we would use a NOT_APPROPRIATE or similar field, but UNAVAIL is
good enough (the call cannot fail in another way).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sun, 16 Jan 2011 16:22:37 +0000 (17:22 +0100)]
query: change (debug-mode) field validation errors
Currently, the single assert just checks that the entire row is
consistent (true/false), and dumps the row and field definitions as an
accompanying message. This makes it very hard to understand what failed.
This patch changes this validation to show descriptive messages, which
makes it much faster in diagnosing invalid result.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sun, 16 Jan 2011 15:23:45 +0000 (16:23 +0100)]
Rename a few instance field titles
The conversion to querylib introduced unique field titles, which however
did an inconsistent rename of the instance vcpus/memory fields.
This patch makes both BE_MEMORY and _VCPUS be named with prefix “Config”
(which before was “Configured_”), and drops the “Runtime” (note no
underscore) prefix on the oper_ ones.
Rationale: for numeric fields, the width of the title defines the width
of the columns, as the numbers are usually small. Hence these columns
(more likely used than the BE ones) would be very wide, leading to a
not-so-nice display. I happened upon this when my terminals didn't
manage to fit a gnt-instance list anymore.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sun, 16 Jan 2011 15:02:25 +0000 (16:02 +0100)]
query: Change internal result computation
While looking at the query library, I realized that while we have five
field statuses, making this a 5-dimensional space, four of them are
shrunk to a single possible value (None). Hence it should be possible to
convert this into a single value space plus extra 4 special constants.
This patch implements this, making (IMHO) the return value of normal
functions much simpler: you simply return the desired value, instead of
(QRFS_NORMAL, value); for the special results, you simply return
_FS_UNAVAIL, instead of (QRFS_UNAVAIL, None). This I believe does
simplify the code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sun, 16 Jan 2011 14:17:40 +0000 (15:17 +0100)]
Import QRFS_* and QFT_* in query.py
Currently, the QRFT/QFT constants are used with the module (e.g.
constants.QRFS_NORMAL) in the query library. Given that there are 176
uses of these constants in a module that has roughly ~1200 lines in
total (including whitespace), it makes sense to import these directly to
improve readability.
This patch does this and (IMHO) it does indeed make the code more to the
point; the entire query library is about queries, so using these
constants directly doesn't cause any misunderstandings.
Note: the epydoc links are not changed, since we want the generated docs
to point to the right place.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 18 Jan 2011 12:43:52 +0000 (13:43 +0100)]
Build epydoc's module list from Makefile
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Mon, 17 Jan 2011 15:50:03 +0000 (15:50 +0000)]
KVM: don't fail removing non-existing nic path
This shouldn't normally happen, but if it does no need to fail.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Apollon Oikonomopoulos [Mon, 17 Jan 2011 18:42:51 +0000 (20:42 +0200)]
KVM: Perform network configuration in Ganeti
This patch introduces network configuration for KVM in Ganeti.
There are three problems with having KVM perform network configuration via ifup
scripts:
a) Ganeti never gets to know the tap interface that is associated with an
instance's NIC
b) Migration of routed instances will cause network problems because the
incoming KVM side configures the network as soon as it is spawned and not
as soon as the migration finishes. This means that all routing
configuration will be present in both, primary and secondary, nodes at the
same time, possibly causing network disruption during the migration.
c) We never get to know if the network configuration succeeded or not.
This patch moves network configuration from KVM to Ganeti, using KVM's ability
to receive already open tap devices as file descriptors.
_WriteNetScript is removed from hv_kvm.py, together with its unit tests.
Minor modifications are made to _ExecKVMRuntime to handle tap device
initialization. NIC <-> tap associations are stored under a new directory,
_ROOT_DIR/nic in a file-per-nic fashion.
The end-user semantics remain the same: The user can override the network
configuration by providing _KVM_NET_SCRIPT. If this is not present or
executable, the default constants.KVM_IFUP script is run.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Apollon Oikonomopoulos [Mon, 17 Jan 2011 10:50:05 +0000 (12:50 +0200)]
Ship default kvm-ifup script
In order to retain backwards compatibility, we ship the shell script previously
generated by _WriteNetScript as a stand-alone script under PKGLIBDIR.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Apollon Oikonomopoulos [Mon, 17 Jan 2011 18:41:34 +0000 (20:41 +0200)]
KVM: Add auxiliary functions to handle tap devices
Add two new functions, _OpenTap and _ProbeTapVnetHdr, to
hypervisors.hv_kvm.
_ProbeTapVnetHdr checks if the host kernel supports the virtio net header
(IFF_VNET_HDR) feature. This feature allows KVM to ask the the host kernel to
perform checksumming and segmentation offload of the data passing through a
guest's virtio NIC, using a special header[1] prepended to data sent through
the tap interface. A qemu-kvm version later than 0.12 is required for this
feature to work.
_OpenTap opens a new tap device suitable for use with qemu-kvm's "-net
tap,fd=n" option. It also enables vnet_hdr support if requested and safe to do
so.
[1] struct virtio_net_hdr in include/linux/virtio_net.h
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Tue, 18 Jan 2011 11:54:56 +0000 (11:54 +0000)]
epydoc: specify files in tools/
This way we can add there non-python scripts
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Sat, 15 Jan 2011 14:32:54 +0000 (15:32 +0100)]
Remove hard-coded OP_ID definitions
The removal requires adding a few pylint ignores, as it cannot know that
OP_ID will be defined at runtime.
Also switch the asserts to check for non-declaration.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Sat, 15 Jan 2011 14:25:24 +0000 (15:25 +0100)]
Check consistency of the class names and OP_ID
As the class names should be now consistent with the OP_IDs, we add a
check for wrongly-defined OP_IDs.
However, the future removal of the hand-coded OP_IDs will render this
obsolete, so this check is introduced just to make sure that the
previous renaming patches did the right job, and it will then be
removed.
The consistency checks require renaming the test opcodes, which were
using arbitrary names, depending on test author. They are now all
standardized on OpTest (local scope).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Sat, 15 Jan 2011 12:07:20 +0000 (13:07 +0100)]
Rename OpTestJobqueue and LUTestJobqueue
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sat, 15 Jan 2011 12:06:57 +0000 (13:06 +0100)]
Rename OpAddTags and LUAddTags
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sat, 15 Jan 2011 12:06:57 +0000 (13:06 +0100)]
Rename OpSearchTags and LUSearchTags
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sat, 15 Jan 2011 12:06:57 +0000 (13:06 +0100)]
Rename OpGetTags and LUGetTags
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sat, 15 Jan 2011 12:06:57 +0000 (13:06 +0100)]
Rename OpDelTags and LUDelTags
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sat, 15 Jan 2011 12:06:57 +0000 (13:06 +0100)]
Rename OpDiagnoseOS and LUDiagnoseOS
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Sat, 15 Jan 2011 12:06:57 +0000 (13:06 +0100)]
Rename OpSetNodeParams and LUSetNodeParams
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>