Iustin Pop [Thu, 29 Dec 2011 15:19:37 +0000 (16:19 +0100)]
Add a new disk-template ipolicy option
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 29 Dec 2011 15:18:11 +0000 (16:18 +0100)]
Add a new CLI option type 'list'
This simply splits the value in the option parser, instead of needing
to do it in the client code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 29 Dec 2011 15:15:13 +0000 (16:15 +0100)]
Fix handling of errors from InstancePolicy.Check...
This code raises a configuration error, but we need to transform it
into a prereq error (or possibly exec error), depending on when we
call this function.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 29 Dec 2011 11:29:37 +0000 (12:29 +0100)]
Switch hspace defaults to the cluster policy
This changes from the current hardcoded defaults to the cluster
policy. The command line options now override the defaults from the
cluster, and the tiered spec mode is always enabled.
Also fixes a tiny typo in the man page (together with the man page
updates).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 29 Dec 2011 10:53:42 +0000 (11:53 +0100)]
Add a helper function converting ispecs to rspecs
The rspec type is simpler than the ispec one; most likely it should be
deprecated later.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 29 Dec 2011 10:49:24 +0000 (11:49 +0100)]
Abstract creation of instance from a spec
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 29 Dec 2011 10:22:13 +0000 (11:22 +0100)]
Load cluster ipolicy via Rapi
This requires changing from querying the /tags resource to the /info
resource.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 29 Dec 2011 09:43:42 +0000 (10:43 +0100)]
Update memory/maxmem reading in Rapi backend
Recent changes to the instance beparams have replaced memory with
maxmem in Rapi bulk queries. Until this is either reverted (for
backwards compat) or we decide to go ahead with only maxmem, we change
the backend to read both; this only affects the "instance down" code
path (where ``oper_ram`` is missing).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Fri, 23 Dec 2011 11:01:51 +0000 (12:01 +0100)]
Load cluster ipolicy via Luxi
Also show it in hbal's verbose output (helpful for debugging).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Fri, 23 Dec 2011 10:38:38 +0000 (11:38 +0100)]
Extend ClusterData with the cluster instance policy
This attribute is always initialised to the default, and is not (yet)
read/saved in the various backends.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Fri, 23 Dec 2011 10:22:06 +0000 (11:22 +0100)]
Add default ipolicy declarations
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Fri, 23 Dec 2011 10:13:34 +0000 (11:13 +0100)]
Add support for RE patterns to convert constants
This is a trivial conversion.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Fri, 23 Dec 2011 09:43:50 +0000 (10:43 +0100)]
Add support for lists/frozensets in convert-constants
Unfortunately, we only support lists of simple types, and not even
lists of tuples. If we actually needed those, it would be possible to
implement them, with a bit more complexity in the converter.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 22 Dec 2011 17:16:24 +0000 (18:16 +0100)]
Add support for tuples in convert-constants
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 22 Dec 2011 16:39:35 +0000 (17:39 +0100)]
More improvements to convert-constants
This prepares for tuple and other conversions.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 22 Dec 2011 16:32:15 +0000 (17:32 +0100)]
Improve convert-constants to handle dictionaries
The two main drawbacks for convert-constants are the fact that it
can't handle sets/frozensets (mainly due to the fact that I don't know
how useful this would be to the Haskell code) and that it cannot
export dictionaries.
To fix the second case, the current patch changes the code to support
flattening (potentially nested) dictionaries into single name
space. Yes, this could generate conflicts, but they will be detected
at compile time.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 22 Dec 2011 15:27:05 +0000 (16:27 +0100)]
Add object definitions for the ispec and ipolicy
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 22 Dec 2011 14:58:30 +0000 (15:58 +0100)]
Stop exporting JSON functionality from Utils.hs
This completes the Utils/JSON split started in commit
f047f90f. The
import graph should be cleaner now.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 22 Dec 2011 14:50:23 +0000 (15:50 +0100)]
More reshuffling of code
Following the split Types/BasicTypes, we can remove the last
JSON-related stuff from Utils.hs, and do some more cleanup.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Mon, 21 Nov 2011 11:44:30 +0000 (12:44 +0100)]
htools: split parts of HTools/Types.hs into BasicTypes.hs
The 'Result' type is common and:
- might be used outside of HTools-specific code too
- is better split as we need these basic types for building the more
complex ones in Types.hs
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Constantinos Venetsanopoulos [Wed, 11 Jan 2012 15:03:18 +0000 (17:03 +0200)]
Fix acquisition of node lock in LUInstanceGrowDisk
Ensure node level locks are recalculated properly
in LUInstanceGrowDisk.
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 22 Dec 2011 10:38:37 +0000 (11:38 +0100)]
Fix cluster ispecs on upgrade
Cluster-level parameters must *not* be empty. Plus two more minor
changes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 22 Dec 2011 10:38:04 +0000 (11:38 +0100)]
Better max ispec defaults
Instead of being equal to the minimum ones, these are reusing current
constants.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 22 Dec 2011 10:09:50 +0000 (11:09 +0100)]
Rename optISpec to optStdSpec
More consistency with the tiered allocation mode.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 22 Dec 2011 10:04:50 +0000 (11:04 +0100)]
Change how hspace reads cmdline args for std specs
For the standard specs, hspace currently takes separate options for
the memory, disk and VCPUs. For the tiered specs, which were
introduced later, it takes a single option with all three values
combined.
This patch adds a backward-incompatible change to the standard spec,
basically moving it to the same single option format, which is a bit
more simple (both in the code and on the command line). This is
"needed" for future changes, where the command line will just
overwrite what we get from cluster, instead of setting the starting
point.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Wed, 21 Dec 2011 16:52:45 +0000 (17:52 +0100)]
Abstract tiered specs command line parsing
This will be used in the future for the standard specs too, so let's
abstract it away.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Tue, 10 Jan 2012 13:47:20 +0000 (14:47 +0100)]
cmdlib._ComputeMinMaxSpec: Add unittest for this function
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 10 Jan 2012 13:18:16 +0000 (14:18 +0100)]
cmdlib: Rename _CheckMinMaxSpecs to _ComputeMinMaxSpec
This is to reflect the fact that _Check functions do raise Op*Error
which is not the case for _CheckMinMaxSpec.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 10 Jan 2012 12:45:03 +0000 (13:45 +0100)]
cmdlib: _VerifyInstancePolicy has been replaced and is dead code
This removes this method
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 10 Jan 2012 14:20:05 +0000 (15:20 +0100)]
utils.ResetTempfileModule: Improve performance
This function is called for after every fork (e.g. for handling an RPC
request). With the changes in this patch generating the next random
filename is about 30% faster.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 10 Jan 2012 12:38:03 +0000 (13:38 +0100)]
LUInstanceCreate: Release node resource lock
… after running iallocator. Otherwise an assertion later in the code
would fail (for a good reason).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Mon, 9 Jan 2012 14:38:50 +0000 (15:38 +0100)]
_CheckTargetNodeIPolicy: instance.primary_node is not objects.Node
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Mon, 9 Jan 2012 09:43:43 +0000 (10:43 +0100)]
cmdlib: node.group is not a nodegroup object
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Fri, 6 Jan 2012 15:44:48 +0000 (16:44 +0100)]
LUInstanceCreate: self.disks is a dict not a disk object
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Fri, 6 Jan 2012 14:00:32 +0000 (15:00 +0100)]
qa-sample.json: Fix a typo
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 10 Jan 2012 11:58:27 +0000 (12:58 +0100)]
Fix wrong variable name
Commit
bc5d0215 added support for disk params, but due to copy-paste
it tries to enforce the hvparams into disk params values, leading to:
$ gnt-cluster modify -H kvm:initrd_path=/boot/initrd-2.6-kvmU
Parameter Error: Unknown parameter 'initrd_path'
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
Guido Trotter [Mon, 12 Dec 2011 15:45:32 +0000 (15:45 +0000)]
InstanceStartup: use overridden beparams in prereq
Without doing this memory checks happen on instance default parameters
rather than any overridden ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 9 Jan 2012 14:12:50 +0000 (15:12 +0100)]
gen-coverage: Change filter
It is not necessary to run a Python script to generate the filter, which
didn't work very well anyway on systems with modules in
/usr/share/pyshared.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 6 Jan 2012 15:01:49 +0000 (16:01 +0100)]
rpc: More unittests
- Fix: Don't check st_atime for temporary file for upload tests
- Add tests for five encoders
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Thu, 5 Jan 2012 14:05:29 +0000 (15:05 +0100)]
Document "gnt-node add" behavior in case of failure
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 13 Dec 2011 09:39:23 +0000 (10:39 +0100)]
cmdlib: Adapt LUClusterVerifyGroup for instance policy
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 13 Dec 2011 09:35:40 +0000 (10:35 +0100)]
gnt-node: Add instance policy to migrate
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Fri, 16 Dec 2011 13:51:29 +0000 (14:51 +0100)]
gnt-backup: Adding force option to import
This makes it possible to bypass the instance policy
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
René Nussbaumer [Wed, 14 Dec 2011 13:39:49 +0000 (14:39 +0100)]
gnt-instance: Adding instance policy to add
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Fri, 16 Dec 2011 14:01:30 +0000 (15:01 +0100)]
gnt-instance: Adding verification of instance policy to modify
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 13 Dec 2011 09:34:27 +0000 (10:34 +0100)]
gnt-instance: Adding instance policy to move
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 13 Dec 2011 09:31:04 +0000 (10:31 +0100)]
gnt-instance: Adding instance policy to migrate
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 13 Dec 2011 09:12:49 +0000 (10:12 +0100)]
gnt-instance: Adding instance policy to failover
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 13 Dec 2011 08:38:47 +0000 (09:38 +0100)]
cmdlib: Adding helper for instance policy
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 6 Jan 2012 12:29:41 +0000 (13:29 +0100)]
rpc.ConfigRunner: Fix uploading file
Commit 601dfcb made some changes to the default encoders. This made
“ConfigRunner.call_upload_file” fail due to a missing encoder. This
patch applies the necessary changes to ConfigRunner and updates the
unittest for “call_upload_file”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Thu, 5 Jan 2012 15:55:40 +0000 (15:55 +0000)]
Update GrowDisk docstring
It forgot about one argument.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 6 Jan 2012 11:38:29 +0000 (11:38 +0000)]
Merge branch 'devel-2.5'
* devel-2.5:
KVM: support version reported by 1.0
doc/admin: Clarify archived jobs
Conflicts:
NEWS: trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 20:23:51 +0000 (21:23 +0100)]
rpc._PrepareFileUpload: Use FileStatHelper
Use fstat(2) on the file handle instead of doing a separate stat(2).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 20:22:45 +0000 (21:22 +0100)]
Move helper class from watcher to utils.io
“FileStatHelper” can be used together with “ReadFile” to a file's status
while it's opened. This avoids certain race conditions.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 20:12:24 +0000 (21:12 +0100)]
Add unittest for uploading file via RPC
After some preparing patches this unittest can finally be added.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 20:11:04 +0000 (21:11 +0100)]
rpc: Change signature of RpcRunner
Instead of receiving the whole masterd context, it only gets the
configuration and the lock monitor callback. This simplifies
unittesting.
Additionaly, two new arguments for testing are added.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 20:09:00 +0000 (21:09 +0100)]
runtime: Style fix in docstring
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 19:00:34 +0000 (20:00 +0100)]
Add unittests for RPC client
This patch adds a number of unittests for the RPC client base class.
Some small changes were necessary in “rpc.py” to allow for better
testing.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 18:58:39 +0000 (19:58 +0100)]
rpc._RpcClientBase: Add check for number of arguments
Just an additional check, useful in unittests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 16:47:38 +0000 (17:47 +0100)]
Add unittest for RPC compression
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 18:58:06 +0000 (19:58 +0100)]
rpc: Style fix
“assert” is a statement, not a function.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Fri, 6 Jan 2012 10:50:45 +0000 (10:50 +0000)]
Merge branch 'stable-2.5' into devel-2.5
* stable-2.5:
KVM: support version reported by 1.0
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
Guido Trotter [Thu, 5 Jan 2012 18:05:06 +0000 (18:05 +0000)]
check-news/NEWS: impose more standards
- Always two empty lines before a version.
- Standardize how version names should be called.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Thu, 5 Jan 2012 16:20:24 +0000 (16:20 +0000)]
KVM: support version reported by 1.0
This of course was working for all the rcs, but broke with 1.0 itself.
In addition:
- split between running kvm --version and parsing its output
- unittest parsing for various known --help outputs
- updated NEWS file
- happy 2012 wishes
- the hope to finish this patch before it's time to say happy easter
:)
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Andrea Spadaccini [Thu, 5 Jan 2012 14:56:25 +0000 (14:56 +0000)]
qa: fix disk parameters tests
Fix an error in the disk parameters tests (the arguments to gnt-cluster
were not passed correctly) and move them to separate functions.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 12:54:37 +0000 (13:54 +0100)]
man/gnt-node: Improve powercycle documentation
Reword a bit, capitalize “SSH”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 13:05:08 +0000 (14:05 +0100)]
rpc_defs: Enable “node_powercycle” for offline nodes
With this change it's possible to powercycle nodes marked as offline. Of
course the node might not be running the node daemon anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 13:03:47 +0000 (14:03 +0100)]
rpc._NodeConfigResolver: Support resolving offline nodes
This is needed to powercycle a node marked offline.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 4 Jan 2012 19:33:55 +0000 (20:33 +0100)]
rpc: Pass resolver options to actual resolver
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 5 Jan 2012 14:23:49 +0000 (15:23 +0100)]
rpc: Add support for resolver options callback
This adds support for a callback returning name resolver options. This
is required for powercycling offline nodes. While it would be possible
to implement this using a separate RPC client class, doing so would
require some code duplication with the standard “RpcRunner” class.
The callback can generate the options based on the call arguments.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 4 Jan 2012 18:59:13 +0000 (19:59 +0100)]
rpc._RpcProcessor: Make read timeout a non-keyword argument
It is always used and can be set to “None” if intended.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Thu, 5 Jan 2012 10:18:37 +0000 (11:18 +0100)]
QA: Make the ispec values configurable
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 4 Jan 2012 13:15:50 +0000 (14:15 +0100)]
Update the resource model DD with current state
On my work for instance policies we decided not to overload --force
more, instead we go with a new flag --ignore-ipolicy. This is just a
small patch to reflect this decision.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Thu, 22 Dec 2011 13:49:24 +0000 (14:49 +0100)]
Fix failure in distcheck and coverage tests
autotools/testrunner was missing in distcheck. Now some changes from
7004106119c5c56c93085ccfb1250b3561b5b6ad have been reverted, and testrunner
is not built anymore. coverage now uses fakeroot when needed.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 22 Dec 2011 18:19:28 +0000 (19:19 +0100)]
Makefile: Fix parallel distcheck
Running “make -jN distcheck” for any N > 1 would reliably break when
building the RPC client wrappers. This was due to wrong dependencies in
Makefile.am, fixed by this patch.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 22 Dec 2011 17:12:59 +0000 (18:12 +0100)]
doc/admin: Clarify archived jobs
Also mention that archived jobs can be viewed using “gnt-job info”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 22 Dec 2011 16:43:40 +0000 (17:43 +0100)]
Merge branch 'devel-2.5'
* devel-2.5:
jqueue: Factorize checking job processor's result
jqueue unittest: Rename simple fake-job class
jqueue: Fix epylint errors introduced in
37d76f1e4
jqueue: Fix deadlock between job queue and dependency manager
locking: Add “__repr__” to SharedLock and PipeCondition
daemon.GenericMain: Don't generate backtrace on conflicting daemons
utils.io.WritePidFile: Improve error reporting
utils.ListVisibleFiles: Hide “lost+found” directories
Fix race condition in test for *FileID functions
Conflicts:
lib/utils/io.py: Trivial
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 22 Dec 2011 16:35:03 +0000 (17:35 +0100)]
Merge branch 'stable-2.5' into devel-2.5
* stable-2.5:
jqueue: Fix epylint errors introduced in
37d76f1e4
jqueue: Fix deadlock between job queue and dependency manager
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 22 Dec 2011 15:00:53 +0000 (16:00 +0100)]
test/lockperf: Use a list of integers, not booleans
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 22 Dec 2011 13:16:57 +0000 (14:16 +0100)]
Add lock performance utility
I had an idea for improving locking performance. To see if it worked I
wrote this tool. Unfortunately the idea didn't quite work (broke
unittests left and right), but the tool is still handy for evaluating
future changes to the “SharedLock” class.
It is not installed or run at build/test time. In its current form it is
intended for manual use. Example output:
---
Total number of acquisitions: 32642
Per-thread acquisitions:
Thread 0: 6536 (20.0%)
Thread 1: 6488 (19.9%)
Thread 2: 6536 (20.0%)
Thread 3: 6529 (20.0%)
Thread 4: 6553 (20.1%)
Benchmark CPU time: 5.010s
Average time per lock acquisition: 0.15348ms
Process:
User time: 4.160s
System time: 1.030s
Total time: 5.190s
---
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 19 Dec 2011 15:25:24 +0000 (16:25 +0100)]
jqueue: Factorize checking job processor's result
This allows for more unittesting.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 19 Dec 2011 15:26:08 +0000 (16:26 +0100)]
jqueue unittest: Rename simple fake-job class
Also add a parameter for priority, to be used in an upcoming
patch.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Tue, 13 Dec 2011 15:01:24 +0000 (16:01 +0100)]
noded: Fix /etc/hosts group ownership
When Ganeti was compiled with the option of running as a different
user/group, the group ownership of /etc/hosts was set to gnt-daemons. Now
permissions for /etc/hosts are preserved (or set correctly).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Tue, 13 Dec 2011 16:17:10 +0000 (17:17 +0100)]
unit tests: Add tests for uid and gid handling in utils.WriteFile
These tests need fakeroot. If it's not present, they are not run (they
don't fail).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Tue, 13 Dec 2011 14:51:02 +0000 (15:51 +0100)]
unit tests: Add tests for file mode handling in utils.WriteFile
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Tue, 13 Dec 2011 14:49:38 +0000 (15:49 +0100)]
utils.WriteFile: Add new parameter to preserve file permissions
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Tue, 13 Dec 2011 16:16:42 +0000 (17:16 +0100)]
unit tests: added fakeroot support
Tests ending in "-runasroot" or "-runasroot.py" are run through fakeroot,
so activity that needs root privileges can be tested.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 21 Dec 2011 16:01:08 +0000 (17:01 +0100)]
jqueue: Fix epylint errors introduced in
37d76f1e4
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 21 Dec 2011 13:48:13 +0000 (14:48 +0100)]
Switch blockdev_getmirrorstatus_multi to per-node bodies
Even for a small 4-node node-group, this reduces the payload of this
RPC call by half for each node; for a big node group, previous testing
(using a different patch) showed a general reduction in traffic over
the entire cluster verify from 9MB to ~2MB.
We only need to change the RPC call, not its return value, since that
was already in an "optimised" form.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 21 Dec 2011 13:11:32 +0000 (14:11 +0100)]
Change internal RPC client body values
Currently, all RPC payloads sent by the client to the remote node
daemons must be identical, due to how the data is passed
internally. This is deficient in both use (from the programmer's point
of view) and from the network traffic (cluster verify/disk data
gathering has a total payload which is O(n²) in size of the nodes
being queried, instead of O(n)).
This patch changes the RPC internals so that we always pass
dictionaries indexed by target node name. For the default use case,
when the payload is identical, we only serialise the payload once, so
the extra overhead is just a dict with the node names and values all
pointing to the same object. For different payloads, we will encode
the body multiple times, but hopefully the bodies will be smaller.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 21 Dec 2011 12:58:29 +0000 (13:58 +0100)]
Extend RPC definition change with another parameter
This will be used in a later patch for parameter pre-processing. For
now we just add the parameter as 'None' and add a consistency check
for the parameter definitions.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 21 Dec 2011 15:43:28 +0000 (16:43 +0100)]
hv_kvm: Make string containing regex a raw string
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 21 Dec 2011 14:50:51 +0000 (15:50 +0100)]
hv_kvm: Style fixes
Change a few lines to match rest of code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 21 Dec 2011 14:47:13 +0000 (15:47 +0100)]
test/hv_kvm: Code style, additional tests
Change code style to match rest of Ganeti code and add
two tests checking for single-line messages.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 21 Dec 2011 14:45:59 +0000 (15:45 +0100)]
hv_kvm.QmpMessage: Use dict.get
No need to look up the key twice.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 21 Dec 2011 14:29:32 +0000 (15:29 +0100)]
serializer: Remove JSON indentation and dict key sorting
Serializing to JSON using “simplejson” is significantly slower when
indentation and/or sorting of dictionary keys is used. In simplejson 1.x
the difference isn't that big, but with simplejson 2.x the difference
can be up to a factor of 7.5. The reason is that the latter no longer
uses C functions when sorting or indentation is used.
With this patch we revert everything to simplejson's defaults, which
should provide us with the best performance available.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 19 Dec 2011 15:26:55 +0000 (16:26 +0100)]
jqueue: Fix deadlock between job queue and dependency manager
When an opcode is about to be processed its dependencies are
evaluated using “_JobDependencyManager.CheckAndRegister”. Due
to its nature that function requires a lock on the manager's
internal structures. All of this happens while the job queue
lock is held in shared mode (required for the job processor).
When a job has been processed any pending dependencies are re-added
to the job workerpool. Before this patch that would require
the manager's lock and then, for adding the jobs, the job queue
lock. Since this is in reverse order it will lead to deadlocks.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 14 Dec 2011 11:11:46 +0000 (12:11 +0100)]
Fix unittests for daemon_util with disabled confd
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Andrea Spadaccini [Mon, 19 Dec 2011 15:46:39 +0000 (15:46 +0000)]
Fix parameters for QueryJob in JobExecutor
When gnt-job submit is used with the --each option, it sends as
parameters for QueryJobs a list of lists, each containing a job ID.
While this works now, it is not the intended format for QueryJobs.
This patch fixes this behavior, by making it send a list of job IDs.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 19 Dec 2011 14:21:04 +0000 (15:21 +0100)]
locking: Add “__repr__” to SharedLock and PipeCondition
These help when debugging.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>