Iustin Pop [Mon, 14 May 2012 14:47:14 +0000 (16:47 +0200)]
Allow units in ipolicy disk/mem CLI changes
This makes "gnt-cluster modify --specs-mem-size max=16g" work. The
downside (due to how we iterate) is that error messages are not very
clear:
$ gnt-cluster modify --specs-mem-size max=16a
Failure: prerequisites not met for this operation:
error type: wrong_input, error details:
Invalid disk ({}) or memory ({'max': '16a'}) size in policy: Unknown unit: a
Also updates the help of these commands, which was very confusing
before (at least for me).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 14 May 2012 14:39:57 +0000 (16:39 +0200)]
Move some ipolicy functions from objects to cli
These are only used in CLI programs to parse command line options and
such they do not belong in object.py (I will change them more and I
don't want to add more code in objects.py).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 14 May 2012 13:33:05 +0000 (15:33 +0200)]
Beautify a couple of error messages
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 14 May 2012 13:31:37 +0000 (15:31 +0200)]
Fix _ComputeNewInstanceViolations logic
This function did the opposite: was computing which old instance
violated the specs but no longer do it now. new - old is the expected
behaviour.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 14 May 2012 13:21:27 +0000 (15:21 +0200)]
Beautify disk ipolicy violations in cluster-verify
Currently, we only get:
instance3: ['disk-size value 512 is not in range [1024, 1048576]'
which doesn't explain which disk we are talking about. This patch
extends the verification functions to take an additional parameter
that qualifies the disk:
instance3: ['disk-size/0 value 512 is not in range [1024, 1048576]'
Future patch will make the formatting of the list better.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 15 May 2012 09:07:08 +0000 (11:07 +0200)]
qa_utils: Factorize code for getting entity name
This reduces code duplication and provides better error messages if
something is incorrect while getting a node or instance name.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 10 May 2012 10:59:06 +0000 (12:59 +0200)]
NEWS: Style fixes
- Quote where appropriate
- Small wording changes
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 14 May 2012 13:50:35 +0000 (15:50 +0200)]
qa_utils: Accept string as instance for check
Some tests don't have access to an instance dictionary.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 14 May 2012 13:51:51 +0000 (15:51 +0200)]
QA: Cleanup after commit
5fa0375e4a
Some tests had wrong assumptions on the instance status and some
needed explicit start or stop commands.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 14 May 2012 13:46:22 +0000 (15:46 +0200)]
LUInstanceCreate: Run rename script on instance import
If an instance is imported with a different name, network settings may have to
be changed. Since import scripts may not already to the right thing, we decided
to run the rename script. The same technique is already used for inter-cluster
instance moves.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Mon, 14 May 2012 13:33:36 +0000 (15:33 +0200)]
ganeti.query_unittest: Adding testcase for diskparams
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Mon, 14 May 2012 08:32:09 +0000 (10:32 +0200)]
gnt-group add: Fix diskparam fill
This was a pretty non-obvious bug. A cluster looks sane after
gnt-cluster init, however on a daemon restart the diskparameters had the
default filled in. The same applies to gnt-group add. This is due to the
nature that UpgradeConfig() from NodeGroups did just populate them with
defaults if something was set on it.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 11 May 2012 14:56:49 +0000 (16:56 +0200)]
gnt-group modify: Fix an update issue with diskparams
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 14 May 2012 09:33:04 +0000 (11:33 +0200)]
Make x509 unittest testClockSkew a bit less flaky
Since the tested function actually uses time.time(), it cannot be make
fully stable, but 1 second is very dangerous; let's just test SKEW * 2
and higher since that should be good (if the delta between _GenCert
and VerifyX509Certificate is 5 minutes, then…).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Fri, 11 May 2012 13:27:58 +0000 (15:27 +0200)]
gnt-group: Add info command
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 11 May 2012 13:15:35 +0000 (15:15 +0200)]
cli: Make FormatParameterDict recursive
This is useful if you have nested parameter dicts like in diskparams.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 11 May 2012 11:55:46 +0000 (13:55 +0200)]
query: Expose diskparamters through query
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 11 May 2012 09:47:35 +0000 (11:47 +0200)]
gnt-cluster info: Print and format disk parameters
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Fri, 11 May 2012 13:48:32 +0000 (15:48 +0200)]
Merge branch 'devel-2.5'
* devel-2.5:
Update NEWS and bump version for 2.5.1 release
Fix gnt-group --help display
Fix hardcoded Xen kernel path
Fix grow-disk handling of invalid units
Accept both PUT and POST in noded
Preserve bridge MTU in KVM ifup script
Update synopsis for “gnt-cluster repair-disk-sizes”
Reconcile Makefile.am and test data files
Workaround changed LVM behaviour
Enable lvmstrap to run under Linux 3.x
Add a default PATH variable to OS scripts env
Move hooks PATH environment variable to constants
Add note to the install doc about bridge MAC issues
Fix exception re-raising in Python Luxi clients
Fix LVM volume listing with newer LVM
Conflicts:
NEWS (trivial)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Fri, 11 May 2012 13:05:58 +0000 (15:05 +0200)]
Merge commit 'v2.5.1' into devel-2.5
* commit 'v2.5.1':
Update NEWS and bump version for 2.5.1 release
Fix gnt-group --help display
Fix hardcoded Xen kernel path
Fix grow-disk handling of invalid units
Accept both PUT and POST in noded
Preserve bridge MTU in KVM ifup script
Update synopsis for “gnt-cluster repair-disk-sizes”
Reconcile Makefile.am and test data files
Workaround changed LVM behaviour
Enable lvmstrap to run under Linux 3.x
Add a default PATH variable to OS scripts env
Move hooks PATH environment variable to constants
Add note to the install doc about bridge MAC issues
Fix exception re-raising in Python Luxi clients
Fix LVM volume listing with newer LVM
Conflicts:
lib/client/gnt_group.py (trivial, fixup conflicts from
6bc3ed14)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 10 May 2012 18:26:39 +0000 (20:26 +0200)]
Remove some duplicate code in the Xen config writing
Should be straightforward; the only difference from before is that we
will remove the 'auto' file even during migration finalise.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 11 May 2012 09:32:27 +0000 (11:32 +0200)]
Update NEWS and bump version for 2.5.1 release
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 16 Mar 2012 10:59:01 +0000 (11:59 +0100)]
Fix gnt-group --help display
Copy-paste mismatch :)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
(cherry picked from commit
36c70d4ddc508dd1ffdcc806d617d5100f4cb265)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 23 Mar 2012 09:08:41 +0000 (09:08 +0000)]
Fix hardcoded Xen kernel path
We already have a ./configure-time variable for this, but it seems to
be actually unused.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit
3c4afa2e93d9499ce39c1aed575dacb549d35083)
Signed-off-by: Iustin Pop <iustin@google.com>
(trivial patch, let's cherry-pick it)
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 27 Apr 2012 10:04:59 +0000 (12:04 +0200)]
Fix grow-disk handling of invalid units
The reason why grow-disk was doing:
$ gnt-instance grow-disk instance3 0 -64
Unhandled Ganeti error: Invalid format
Is because it does it's own ParseUnit call, and doesn't transform that
into a nicer message.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
(cherry picked from commit
c8bde61ea8360da67fc09339f1fe29ddd0acc374)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 21 Oct 2011 21:51:39 +0000 (23:51 +0200)]
Accept both PUT and POST in noded
This is a partial cherry-pick from
7530364ddbe949bc34fc26f25ba3f5d921beb021 on master:
Currently, noded requires PUT, even though the semantics of the RPC
calls do not match a PUT. We change the code accept both PUT and POST,
with the intention to remove the PUT support in a later version.
Additionally, we add a message to the HttpBadRequest exception to make
clear the failure mode (not seeing any error message was what made me
send this patch…). This was the only description-less use of this
exception, by the way.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
(cherry picked from commit
7530364ddbe949bc34fc26f25ba3f5d921beb021)
What was not cherry-picked is the rpc change (to switch to PUT). The
reason I want to backport this to devel-2.5 is that when upgrading to
2.6, having noded accept both makes for an easier upgrade path.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit
5d0566deaf5cd50f487c97ae6884bef0939db1c5)
Signed-off-by: Iustin Pop <iustin@google.com>
Yet another cherry-pick (must go deeper!); since we might not make a
new release from the devel-2.5 branch, let's add this to stable-2.5.
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Andrea Spadaccini [Tue, 11 Oct 2011 13:39:05 +0000 (14:39 +0100)]
Preserve bridge MTU in KVM ifup script
Closes: #201 - KVM_IFUP does not set bridge-MTU on tap devices
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit
a1ec8695a6b453acdc2fa746a27be73c614b2e87)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 3 Nov 2011 15:40:44 +0000 (16:40 +0100)]
Update synopsis for “gnt-cluster repair-disk-sizes”
Mention that instances can be passed on the CLI when “--help” is used.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
(cherry picked from commit
eb5ac108d146644200df98b9f90dae003dcea426)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 15 Feb 2012 16:02:33 +0000 (17:02 +0100)]
Reconcile Makefile.am and test data files
Sorry, forgot this in previous commit.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit
1a1e7ab3f7cfe156152fb69961115a2c85b2a82d)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 15 Feb 2012 12:14:04 +0000 (13:14 +0100)]
Workaround changed LVM behaviour
The vgreduce command has changed behaviour from when we initially
wrote the code (2.02.02 versus 2.02.66, 4 years delta):
- if there are LVs which will be impacted, it requires --force
- otherwise refuses to proceed, but it still returns exit code 0
We handle this by looking to see if it returns "Wrote out consistent
volume group" (behaviour unchanged), or if it complains about
"--force"; in the case it didn't complete, we retry the operation.
We improve a bit the checking of "vgs", as it uses to fail silently
and we didn't detect it.
New tests for this function should test, I believe, all the expected
variations; at the least we now have data files with the expected
output.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit
048eeb2b8faa15bfb94205f908538ae729d6ca27)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Alexander Schreiber [Tue, 13 Mar 2012 16:34:52 +0000 (17:34 +0100)]
Enable lvmstrap to run under Linux 3.x
Extend the kernel version check to also accept Linux 3.x as valid.
Signed-off-by: Alexander Schreiber <als@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit
1bf72492d381aacb5c488f1a87ac7665b9ddc6c7)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Thu, 10 May 2012 11:53:56 +0000 (13:53 +0200)]
apidoc: Fix some typos and errors introduced by my previous patches
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 8 May 2012 09:16:11 +0000 (11:16 +0200)]
bdev: Add a guard to verify disk parameters are set
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 4 Apr 2012 12:41:06 +0000 (14:41 +0200)]
LUGroup*: Fix inheritance of disk parameters
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 4 Apr 2012 12:19:53 +0000 (14:19 +0200)]
bdev: Take the params directly from disk without refill
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 4 May 2012 11:33:12 +0000 (13:33 +0200)]
Special case blockdev_find
Similiar to blockdev_create we sometimes do find on children. This fixes
those cases. However, this is not very nice.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Thu, 3 May 2012 09:08:38 +0000 (11:08 +0200)]
Special case blockdev_create
This is due to the nature of bdev. We spread some logic into cmdlib and
deal for example with it's children recursively. This makes it hard to
annotate the disk parameters in a generic way as we don't always deal
with the top most disk. But the disk parameters are depending on the top
device not on the children.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 8 May 2012 08:10:35 +0000 (10:10 +0200)]
cmdlib: Adding annotation helper for special cases
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 4 Apr 2012 12:26:43 +0000 (14:26 +0200)]
cmdlib: Remove all diskparams calculations not required anymore
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 4 Apr 2012 11:34:45 +0000 (13:34 +0200)]
cmdlib: Adapt the rpc calls
The following (blockdev) RPC calls are not converted yet (as they are
not straight forward or need more research):
* bdev_sizes
* blockdev_remove
* blockdev_shutdown
* blockdev_removechildren
* blockdev_close
* blockdev_getsize
* drbd_disconnect_net
* blockdev_rename (has already a special encoder, needs further research
if needed at all)
* blockdev_getmirrorstatus (not sure if we have everywhere a clear link
to the instance the disk belongs)
* blockdev_getmirrorstatus_multi (same here, further research)
Then special cases where we take care later in the patch series:
* blockdev_create (special cased)
* blockdev_find (special cased, like blockdev_create)
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 3 Apr 2012 16:44:16 +0000 (18:44 +0200)]
rpc: Adapt the callbacks to the new encoder type
We change the cmdlib side on the next patch.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 3 Apr 2012 11:25:22 +0000 (13:25 +0200)]
rpc: Adding new encoders for annotating disk parameters
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 3 Apr 2012 12:10:23 +0000 (14:10 +0200)]
rpc: Adding helper to annotate disk params
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 30 Mar 2012 12:58:52 +0000 (14:58 +0200)]
config: Adding convenience method for disk parameters
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 30 Mar 2012 12:40:12 +0000 (14:40 +0200)]
objects: Add helper function to fill disk parameters
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 10 May 2012 09:54:47 +0000 (11:54 +0200)]
QA: Decorate tests with instance checks
Start using “InstanceCheck” decorator where appropriate. Use direct call
for check in places where the instance status changes during the test.
The parameters of “TestInstanceImport” had to be re-ordered to have the
instance as the first argument.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 10 May 2012 09:44:02 +0000 (11:44 +0200)]
QA: Infrastructure for hook script to check instance status
This script can be used to check if an instance is running or stopped at
various points during a QA run. Environment variables are used to pass
the most essential information.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 10 May 2012 06:15:48 +0000 (08:15 +0200)]
QA: Support passing keyword arguments to test functions
Until now this wasn't possible.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 9 May 2012 03:12:48 +0000 (05:12 +0200)]
Add a default PATH variable to OS scripts env
In commit
896a03f6 I cleaned up the environment for OS scripts,
however I think that was a bit too extreme - it breaks our own
instance-debootstrap hooks, because for example dpkg (called from the
grub script) requires PATH to be set.
Instead of requiring every OS to define a path, let's set a default
PATH for the OS scripts, which should cover most common uses. A more
specialised PATH can be set, if needed, in the OS scripts.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Andrea Spadaccini [Tue, 1 Nov 2011 18:45:27 +0000 (18:45 +0000)]
Move hooks PATH environment variable to constants
Move the contents of the PATH environment variable for hooks to
constants, and use its value in the code and in the hooks documentation.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit
fe5ca2bbc103d25d159bc23b00c3acdfb024c0d2)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 8 May 2012 11:06:10 +0000 (13:06 +0200)]
Add note to the install doc about bridge MAC issues
Thanks to Faidon Liambotis for explaining this on the external IRC
channel.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Faidon Liambotis <paravoid@gmail.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 8 May 2012 12:29:07 +0000 (14:29 +0200)]
Fix exception re-raising in Python Luxi clients
Commit
e687ec01 (present in 2.5 since the 2.5 beta 3) did consistency
fixes across the code-base. Unfortunately this was done without enough
checks on the actual meaning of one of the fixes, which means error
re-raising in lib/errors.py is broken.
The problem is that:
raise cls, args
is different than:
raise cls(args)
And our unit-tests didn't catch this (this patch updates the tests).
This breakage is usually trivial, like wrong error messages:
$ gnt-instance remove no-such-instance
Failure: prerequisites not met for this operation:
("Instance 'no-such-instance' not known", 'unknown_entity')
versus:
$ gnt-instance remove no-such-instance
Failure: prerequisites not met for this operation:
error type: unknown_entity, error details:
Instance 'no-such-instance' not known
or:
$ gnt-instance add … no-such-instance
Failure: prerequisites not met for this operation:
('The given name (no-such-instance) does not resolve: Name or service not known', 'resolver_error')
versus:
$ gnt-instance add … no-such-instance
Failure: prerequisites not met for this operation:
error type: resolver_error, error details:
The given name (no-such-instance) does not resolve: Name or service not known
But in some cases where we rely on a certain data representation
(e.g. HooksAbort), this actually breaks because we try to iterate over
the wrong type:
File "/usr/lib/python2.6/dist-packages/ganeti/cli.py", line 1907, in FormatError
for node, script, out in err.args[0]:
ValueError: need more than 1 value to unpack
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 9 May 2012 08:42:59 +0000 (10:42 +0200)]
Allow clock skews in certificate verification
Currently we allow for up to NODE_MAX_CLOCK_SKEW time difference
between nodes in some operations, but not everywhere: SSL certificate
verification (import/export, both intra and inter-cluster) has a zero
limit (downwards), and a week upwards. This can cause even
intra-cluster backup problems, if the source node has a time even two
seconds in the future.
To fix this, when we verify certificates compare with a time offset
with the max skew, which fixes the lower bound and reduces the upper
bound by an insignificant amount (0.04%).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 8 May 2012 10:54:54 +0000 (12:54 +0200)]
Improve formatting in the hooks documentation
The pyeval role generates literal (e.g. ``OP_CLUSTER…``) formatting,
which differs from the other headers. This is confusing, so let's add
a note that these are virtual operations, which makes the output more
consistent.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 7 May 2012 10:56:32 +0000 (12:56 +0200)]
Add decoding of Luxi calls and unittests for LuxiOp
This patch adds a hand-coded decoder for LuxiCall arguments, as the
data-structure is not uniform enough for automated generation (even
for the serialisation, we had to add hints for some fields,
de-serialisation is even harder).
It also fixes a tiny issue with WaitForJobChange job ID encoding, and
adds unittests for the encoding/decoding of LuxiOp structures.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 7 May 2012 10:07:10 +0000 (12:07 +0200)]
Some unit tests improvements
In preparation for unit-testing the Luxi definitions, this improves
the auto-generation of OpCodes: currently this uses unbounded plain
Strings for names and fields, and this creates too big test sizes
(e.g. when automatically generating job sets).
Furthermore, there are some more cleanups: switching from liftM to
applicative style for easier definition, and some improvements in
generators.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Sat, 5 May 2012 04:18:39 +0000 (06:18 +0200)]
Auto-define a LuxiReq data type
We currently auto-generate a LuxiOp data type, which holds the entire
operation (including parameters). However, having a data type just for
the method call would be useful, so let's change THH to also
defineSADT for the Luxi constructors.
Currently I don't know how to match automatically a LuxiReq to its
LuxiOp counterpart (not even sure we need that), so any matching will
remain manual.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Sat, 5 May 2012 04:14:12 +0000 (06:14 +0200)]
Replace hardcoded constants with Constants.hs names
Now that luxi.py constants are exported, we can use them for more
consistency.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Sat, 5 May 2012 03:51:03 +0000 (05:51 +0200)]
Fix Makefile dependencies for building Constants.hs
I was wondering why my changes to luxi.py did not trigger the rebuild…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Sat, 5 May 2012 03:42:48 +0000 (05:42 +0200)]
Correct capitalisation of two Luxi calls
Two Luxi calls have inconsistent an name/value mapping (in the Python
code):
- REQ_AUTOARCHIVE_JOBS versus AutoArchiveJobs (versus AutoarchiveJobs)
- REQ_QUEUE_SET_DRAIN_FLAG versus SetDrainFlag (no Queue)
While these are only a consistency issue, let's fix them so that the
Haskell code (which uses the auto-generated camel-case form) doesn't
need to handle them case specially, and looks more like the Python
code (hah, joke!).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Sat, 5 May 2012 03:32:53 +0000 (05:32 +0200)]
Add a new JSON function
And its associated unittests.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Wed, 18 Apr 2012 16:15:51 +0000 (18:15 +0200)]
Add design document for query path splitting
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 30 Apr 2012 22:33:20 +0000 (15:33 -0700)]
Fix LVM volume listing with newer LVM
Per commit 0304f0e, newer LVM has extended the lv_attr field. However,
that commit was incomplete as we examine this attribute in another
place in the code.
Thanks to user alperhome, the _LVSLINE_REGEX in lib/backend.py also
needs fixing. I've used the same change as in the above commit: accept
at minimum 6 characters, but allow for more.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Fri, 4 May 2012 06:39:40 +0000 (23:39 -0700)]
Implement reverse mapping of values to names
This adds a bit of dumb mapping of values to names, while trying to be
safe. This is the best we can do without resorting to parsing or
interpreting ASTs.
The difference in the output is:
-- | Converted from Python list or set ADMINST_ALL
adminstAll :: [String]
-adminstAll = ["down", "offline", "up"]
+adminstAll = [adminstDown, adminstOffline, adminstUp]
Since for most such values we use strings, we don't gain in type
safety on the Haskell side. But it makes the output more readable and
it might open up other opportunities later.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 4 May 2012 06:11:30 +0000 (23:11 -0700)]
Implement support for multi-module export
This add support for exporting constants from multiple modules (as
opposed to hard-coding constants), and also makes the output more
readable by skipping things we know for sure we don't want to convert
(as opposed to things we would like to but don't know _how_ to
convert).
Additionally, we export the constants from the luxi module too.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 4 May 2012 12:34:16 +0000 (14:34 +0200)]
Add missing qa/__init__.py file
The file was there and listed in Makefile.am, but I forgot to add it to
commit
a0c3e7264a.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 26 Apr 2012 17:57:29 +0000 (19:57 +0200)]
QA: Fix shutdown for reinstall/rename tests
The “RunTestIf” uses an AND association between the passed test names,
so the instance would be be shut down if both rename and reinstall were
enabled. Since this is not necessarily the case, OR must be used.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 26 Apr 2012 17:35:46 +0000 (19:35 +0200)]
QA: Enable use of OR conditions in test checks
Until now “TestRunIf” and “TestEnabled” could only handle AND. With this
patch a new class named “Either” is added to “qa_config” and allows OR.
The name “Either” was chosen instead of “Or” as the latter is very close
to the reserved keyword “or”.
Examples:
["rapi", Either(["instance-rename", "instance-reboot"])]
Either(["node-list", "instance-list", "job-list"])
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 30 Apr 2012 23:53:59 +0000 (16:53 -0700)]
Further fixes for new-style exception handling
Commit
30d25dd8 moved the htools code to new-style exception handling,
but the hconfd code hasn't been, which fails when compiling on newer
GHC versions.
This patch does the rest of the move; however, the situation is not
very nice, we should have a better way to handle this (introduce
catchIO? or something like that).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 30 Apr 2012 21:33:18 +0000 (14:33 -0700)]
Add NEWS entry for paramiko/pycrypto warnings
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Fri, 27 Apr 2012 10:04:59 +0000 (12:04 +0200)]
Fix grow-disk handling of invalid units
The reason why grow-disk was doing:
$ gnt-instance grow-disk instance3 0 -64
Unhandled Ganeti error: Invalid format
Is because it does it's own ParseUnit call, and doesn't transform that
into a nicer message.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Fri, 27 Apr 2012 09:33:12 +0000 (11:33 +0200)]
Fix rapi.testutils unittest
Since we use a testutils.InputTestClient(), then the actual error
expected is VerificationError, and not GanetiApiError (which is used
at real run-time).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 26 Apr 2012 15:03:36 +0000 (17:03 +0200)]
Make setup-ssh behave more friendly
So while testing my previous patch I run setup-ssh manually without
any arguments, and it did nothing (no complains, no messages,
anything). That was very surprising, so let's make it behave better if
no nodes are passed.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 26 Apr 2012 14:56:58 +0000 (16:56 +0200)]
Silence Paramiko deprecation warnings
This works around a problem in Paramiko, but which looks like it's a
Ganeti problem:
[cluster] root@node4:~# gnt-node add --readd node3
/usr/lib/python2.6/dist-packages/Crypto/Util/randpool.py:40:
RandomPool_DeprecationWarning: This application uses RandomPool, which
is BROKEN in older releases. See http://www.pycrypto.org/randpool-broken
RandomPool_DeprecationWarning)
To silence this, we do an ugly thing (since Python 2.4 doesn't support
context managers): manually reset the warnings filter, which resets
any -W command line flags (hopefully noone calls this tool manually so
often).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 10 Apr 2012 21:09:57 +0000 (23:09 +0200)]
Add example script for using RAPI test utilities
This script shows a few examples on how to use the RAPI input test
client. It is also run at “make check” time to ensure it's not
completely broken.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 30 Mar 2012 14:52:30 +0000 (16:52 +0200)]
Add more RAPI test utilities
This patch adds a mock RAPI client to test input values to methods. All
methods either raise an exception if there was a problem or return None.
Third-party code can use this to test their input values to the RAPI
client.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 30 Mar 2012 14:41:04 +0000 (16:41 +0200)]
rapi.testutils.FakeCurl: Add header support
With this patch headers are constructed from the PycURL options
and passed to the mock implementation.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Mon, 30 Jan 2012 10:25:54 +0000 (11:25 +0100)]
Move _FakeCurl from tests/ganeti.rapi.client to ganeti.rapi.testutils
This is preparation for the mock system, where we need the same cURL
mock.
Signed-off-by: René Nussbaumer <rn@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 30 Mar 2012 11:58:33 +0000 (13:58 +0200)]
errors: Add exception for RAPI testing utilities
This exception is raised to abort before actually sending a LUXI call
(there is no LUXI server involved in the test). The testing utilities
catch the exception to report a success (i.e. the code didn't throw
an exception before due to invalid types, etc.). To allow the exception
to be thrown all the way to the test utilities, the HTTP server library
must ignore it. Also some overly generic exception handling is removed
from the RAPI request handler.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 30 Mar 2012 11:51:18 +0000 (13:51 +0200)]
luxi: Add list of all requests
This will be used in a unittest for RAPI testing utilities.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 29 Mar 2012 15:31:08 +0000 (17:31 +0200)]
server.rapi: Factorize RAPI user loading
By moving the code loading RAPI users from a file into a standalone
function it becomes easier to use the RAPI request handler in
tests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 29 Mar 2012 13:04:50 +0000 (15:04 +0200)]
RemoteApiHandler: Add support for mocked LUXI client
This will be used for providing a type-checking RAPI client for tests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 26 Apr 2012 08:40:03 +0000 (10:40 +0200)]
Man page update for grow-disk
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 25 Apr 2012 12:47:46 +0000 (14:47 +0200)]
Add grow-disk tests in QA
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 25 Apr 2012 11:59:33 +0000 (13:59 +0200)]
Add absolute grow-disk mode at CLI level
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 25 Apr 2012 11:58:47 +0000 (13:58 +0200)]
Add 'absolute' grow-disk mode at OpCode/LU level
This also improves the log messages for the (default) relative mode
("by %s to %s").
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 25 Apr 2012 11:42:47 +0000 (13:42 +0200)]
Ensure that the grow disk amount is positive
The CLI currently doesn't permit this, but a direct job submission
would allow it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 26 Apr 2012 09:50:17 +0000 (11:50 +0200)]
Log all external commands execution
This logs all external commands in normal (non-debug) mode. This will
leak the DRBD secrets, but in any case we do log failed commands, so
it's not a significant change.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 20 Apr 2012 18:53:42 +0000 (20:53 +0200)]
QA: Add default setting for tests
Commit
1010ec70b1 enabled all tests by default. In some scenarios where
the configuration is static one may not want newly added tests to be
run.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Chris Schrier [Fri, 20 Apr 2012 16:22:26 +0000 (12:22 -0400)]
Update NEWS to mention that CertificateError change.
Signed-off-by: Chris Schrier <schrierc@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Chris Schrier [Fri, 20 Apr 2012 16:22:25 +0000 (12:22 -0400)]
Change CertificateError to subclass GanetiApiError
Signed-off-by: Chris Schrier <schrierc@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 20 Apr 2012 17:51:04 +0000 (19:51 +0200)]
Add opcode parameter descriptions
Fixes QA after commit
cfdf561dade7.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Chris Schrier <schrierc@google.com>
Michael Hanselmann [Tue, 20 Mar 2012 16:57:12 +0000 (17:57 +0100)]
Stop acquiring BGL for LUXI queries
Short description: This fixes an issue whereby masterd would become
unresponsive on the LUXI socket, leading to client timeouts. While made
worse in 2.5, the underlying issue was already present in 2.4.
Longer description: Until now all LUXI queries would acquire the BGL
(big Ganeti lock) in shared mode. With the exception of OpNodeAdd and
OpNodeRemove, this was also the case for all opcodes before version 2.5.
In 2.5 we split OpClusterVerify into multiple opcodes, one of which
(OpClusterVerifyConfig) now acquires the BGL in exclusive mode. Whether
or not doing so is good is a separate discussion: OpNodeAdd and
OpNodeRemove, as of this writing, still require an exclusive BGL.
OpClusterVerifyConfig is run more often than OpNodeAdd or OpNodeRemove
in normal clusters, which is why we only recognized this issue in 2.5.
What would happen is that once OpClusterVerifyConfig tried to acquire
its exclusive BGL while it was actually held by other opcodes (e.g.
OpInstanceReplaceDisks), the locking code would not grant shared
acquires for the BGL, even when the exclusive acquire is removed from
the queue for a short amount of time after a timeout. This is necessary
to prevent lock starvation.
In this situation further LUXI queries requiring the BGL in shared mode,
e.g. OpClusterQuery, would block and the client eventually time out.
Over time they fill the client request workerpool's queue and at that
point even requests not requiring the BGL stop working. Once the
long-running operation(s) holding the BGL in shared mode finished,
OpClusterVerifyConfig gets it in exclusive mode and everything returns
to normal. LUXI recovers very soon too.
I'd like to thank Bernardo Dal Seno for his contribution to this bugfix.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit
0fa753bad2cf5a0cf88953347e5da3aebbf21956)
Michael Hanselmann [Wed, 18 Apr 2012 21:15:38 +0000 (23:15 +0200)]
Stop using locks in LUXI "QueryTags"
Also mark it as deprecated in NEWS as normal queries can be used
instead.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 19 Apr 2012 18:03:12 +0000 (20:03 +0200)]
Convert LUClusterConfigQuery to query2
The main intention of this patch is to make it possible to retrieve
cluster tags via query2. While at it I decided to convert
LUClusterConfigQuery right away. Some of the values returned by
LUClusterQuery are also included, but the conversion of LUClusterQuery
is not yet complete.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 19 Apr 2012 18:10:18 +0000 (20:10 +0200)]
Fix RAPI QA with exports via query2
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 19 Apr 2012 20:14:49 +0000 (22:14 +0200)]
Remove unused constants
These are not used anywhere in Python or Haskell.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 18 Apr 2012 16:38:14 +0000 (18:38 +0200)]
Convert listing exports to query2
This solves one case where locks are acquired during LUXI queries.
Pretty late into the transition I noticed that OpBackupQuery had a
“use_locking” parameter for a long time, but didn't use it. Since
most of the other changes were already and this allows exports to
be listed via RAPI (/2/query) I decided to finish.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 18 Apr 2012 16:39:38 +0000 (18:39 +0200)]
utils.algo: Use str.isdigit instead of regular expression
str.isdigit is about 4x faster than using a regular expression ("\d+").
This is in the inner sorting code so speed matters.
$ python -m timeit -s 'import re; s = re.compile("^\d+$")' \
's.match(""); s.match("Hello World"); s.match("1234")'
1000000 loops, best of 3: 0.937 usec per loop
$ python -m timeit '"".isdigit(); "Hello World".isdigit(); "1234".isdigit()'
1000000 loops, best of 3: 0.218 usec per loop
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Chris Schrier [Wed, 18 Apr 2012 22:10:50 +0000 (18:10 -0400)]
Include PycURL error code in GanetiApiError.
Signed-off-by: Chris Schrier <schrierc@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>