Luca Bigliardi [Thu, 24 Jun 2010 10:02:23 +0000 (11:02 +0100)]
VerifyNode: add usermode helper reply
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Luca Bigliardi [Wed, 16 Jun 2010 16:59:20 +0000 (17:59 +0100)]
BaseDRBD: provide a way to query usermode_helper parameter
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Mon, 5 Jul 2010 14:31:29 +0000 (16:31 +0200)]
Fix a broken commandline switch option
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 5 Jul 2010 13:37:43 +0000 (15:37 +0200)]
Add a QA option to disable reboots during burnin
Since we have seen cases where (repeated) reboots are not supported
(e.g. Xen 3.4+), we need to be able to control this in the QA
configuration.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Sat, 3 Jul 2010 07:34:47 +0000 (08:34 +0100)]
Check pycurl module at configure time
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 5 Jul 2010 13:37:43 +0000 (15:37 +0200)]
Add a QA option to disable reboots during burnin
Since we have seen cases where (repeated) reboots are not supported
(e.g. Xen 3.4+), we need to be able to control this in the QA
configuration.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 5 Jul 2010 11:28:36 +0000 (13:28 +0200)]
OpCreateInstance: do not require hv/be/os params
It is perfectly legal to create an instance using only defaults
(although beparams will be most times passed in), so let's relax the
requirement for these three parameters.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 2 Jul 2010 15:20:51 +0000 (17:20 +0200)]
Fix ganeti-rapi version string
This was "broken" for almost a year :)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 2 Jul 2010 15:19:22 +0000 (17:19 +0200)]
Silence the check-dirs check
The big shell fragment is just noise, for the common case where it
doesn't fail.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 1 Jul 2010 17:03:03 +0000 (19:03 +0200)]
Remove _CheckBooleanOpField
This is no longer used, and we can remove it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 1 Jul 2010 17:01:48 +0000 (19:01 +0200)]
Rework the "type" system
This patch merges the _OP_REQP and _OP_DEFS class attributes into a
_OP_PARAMS list, which holds both. The associated unittest checks that
all opcode attributes are declared and checked, and that no LU uses the
old fields (could be removed later).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 1 Jul 2010 17:00:41 +0000 (19:00 +0200)]
Make _CheckDiskTemplate a valid type checker
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 1 Jul 2010 17:00:06 +0000 (19:00 +0200)]
A few more type definitions
This is to simplify the type declarations in the actual LUs.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Thu, 1 Jul 2010 14:25:41 +0000 (15:25 +0100)]
Merge branch 'stable-2.1'
* stable-2.1:
Bump up version for 2.1.5 release
RapiClient: fix multi-authentication in Python 2.6
Remove rapi-user and rapi-pass from qa-sample.json
qa: fix gnt-instance modify -t drbd
qa: shutdown instance before trying disk convert
Fix check in gnt-instance modify -t
Document optional ctypes dependency
Update NEWS for the 2.1.5 release
Pass force variant option at instance creation
BatchCreate: get force_variant from specs not opts
BatchCreate: set a default for force_variant
Conflicts:
INSTALL
- merge
NEWS
- merge
configure.ac
- keep 2.2 version
lib/rapi/client.py
- keep curl version
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 30 Jun 2010 10:48:25 +0000 (11:48 +0100)]
Bump up version for 2.1.5 release
Also update the release date and the NEWS file.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Thu, 1 Jul 2010 13:36:52 +0000 (14:36 +0100)]
RapiClient: fix multi-authentication in Python 2.6
In Python 2.6 the urllib2.HTTPBasicAuthHandler has a "retried" count for
failed authentications. The handler fails after 5 of them. To solve this
we reset the handler's "retried" member variable to 0 after every
successful request. This is a bit ugly, but makes the client work again
for more than 5 requests under all versions of Python.
Note that the digest authentication handler has a reset_retry_count()
method to do this, but the method is not defined for the basic
authentication handler, so we must reset the variable itself.
This member variable is unused in 2.4 and 2.5, so the change doesn't
affect the client under older Python versions.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Thu, 1 Jul 2010 10:17:34 +0000 (11:17 +0100)]
Remove rapi-user and rapi-pass from qa-sample.json
After commit
725ec2f10019c35bafeb1aabfce6f14174bf4f46 they are unused.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 1 Jul 2010 11:45:49 +0000 (13:45 +0200)]
utils.OwnIpAddress: Change try/except for Python 2.4
Python 2.4 doesn't support “except” and “finally” in the same block.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 1 Jul 2010 11:38:59 +0000 (13:38 +0200)]
RAPI client: Switch to pycURL
Currently the RAPI client uses the urllib2 and httplib modules from
Python's standard library. They're used with pyOpenSSL in a very fragile
way, and there are known issues when receiving large responses from a RAPI
server.
By switching to PycURL we leverage the power and stability of the
widely-used curl library (libcurl). This brings us much more flexibility
than before, and timeouts were easily implemented (something that would
have involved a lot of work with the built-in modules).
There's one small drawback: Programs using libcurl have to call
curl_global_init(3) (available as pycurl.global_init) while exactly one
thread is running (e.g. before other threads) and are supposed to call
curl_global_cleanup(3) (available as pycurl.global_cleanup) upon exiting.
See the manpages for details. A decorator is provided to simplify this.
Unittests for the new code are provided, increasing the test coverage of
the RAPI client from 74% to 89%.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 30 Jun 2010 15:45:45 +0000 (17:45 +0200)]
baserlib: Use boolean type for boolean variables
This does not yet fix all issues in the RAPI interface which were
introduced with the type system. More testing is needed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 1 Jul 2010 10:01:00 +0000 (11:01 +0100)]
qa: fix gnt-instance modify -t drbd
We need to pass the secondary node name, not a dict, which is an invalid
value.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Thu, 1 Jul 2010 11:29:12 +0000 (13:29 +0200)]
Fixing Makefile.am to reflect the document move and adding of cluster merger
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 29 Jun 2010 13:58:16 +0000 (15:58 +0200)]
Adding a user document for the use of cluster-merge
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Wed, 30 Jun 2010 09:20:26 +0000 (11:20 +0200)]
Move cluster merger design doc to design-2.1.rst
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 30 Jun 2010 17:26:06 +0000 (18:26 +0100)]
qa: shutdown instance before trying disk convert
Because we have to. :)
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 30 Jun 2010 17:19:30 +0000 (18:19 +0100)]
Fix check in gnt-instance modify -t
Currently when changing the instance disks the instance status (which
must be down) is only checked if the target status is among the mirrored
ones. This is incorrect: the instance must be down either way, even if
the desired final status is non-mirrored.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 30 Jun 2010 12:40:45 +0000 (14:40 +0200)]
Modify gnt-instance rename to support --no-check-name
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 30 Jun 2010 12:37:24 +0000 (14:37 +0200)]
Adding check_name option to the opcode and luxi call for instance rename
This will allow instance rename without dns check as it does for instance
add.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Manuel Franceschini [Wed, 30 Jun 2010 13:35:34 +0000 (15:35 +0200)]
Enable utils.OwnIpAddress to work with IPv6 addresses
Currently a TcpPing using the target address as source is used to
determine wheter or not an IP address belongs to a node. This does not
work for IPv6 anymore (I'm not sure why) as binding to IPv6 localhost as
source for the TCP connection in TcpPing still results in a successful
connection, even in case the target address is non-local.
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 20 Apr 2010 15:51:34 +0000 (17:51 +0200)]
Barebones LXC hypervisor
This needs lots of work, but it can successfully launch an LXC-based
instance. See the docstring for the limitations/work to be done.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Luca Bigliardi [Wed, 30 Jun 2010 12:43:19 +0000 (13:43 +0100)]
Document optional ctypes dependency
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Manuel Franceschini [Tue, 29 Jun 2010 10:43:22 +0000 (12:43 +0200)]
DRBD IPv6 support
Support IPv6 configuration for 'drbdsetup show' parser and add unittests
concerning that case. Renames some data files to use consistent names
clarifying their usage.
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Manuel Franceschini [Wed, 30 Jun 2010 11:41:20 +0000 (13:41 +0200)]
IPv6 support for utils.TcpPing()
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Manuel Franceschini [Thu, 24 Jun 2010 11:57:49 +0000 (13:57 +0200)]
Add function to retrieve family of an ip address
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Manuel Franceschini [Thu, 24 Jun 2010 09:49:54 +0000 (11:49 +0200)]
Introduce utils.IsValidIP{4,6}()
This patch introduces functions to check for valid IPv4 and IPv6
addresses and converts IsValidIP() to return True if it is either a IPv4
or a IPv6 address.
For now we do not change the functional behavior and replace IsValidIP
with IsValidIP4. This might change in the future.
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Manuel Franceschini [Tue, 29 Jun 2010 07:43:06 +0000 (09:43 +0200)]
User assertFalse instead of assert_(not ...)
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Manuel Franceschini [Wed, 30 Jun 2010 10:51:30 +0000 (12:51 +0200)]
Fix osparams bug when gnt-cluster modify'ing
Commit
625ac1135834c27a2e78fd61bf2df2ce29ad2966 introduced 'osparams'
slot in OPSetClusterParams which was not passed when invoked via
gnt-cluster modify.
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Manuel Franceschini [Tue, 29 Jun 2010 08:26:06 +0000 (10:26 +0200)]
Introduce IPv6 constants
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Manuel Franceschini [Mon, 28 Jun 2010 16:58:16 +0000 (18:58 +0200)]
Replace '0.0.0.0' with constant
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Manuel Franceschini [Thu, 24 Jun 2010 09:01:38 +0000 (11:01 +0200)]
Rename some constants to facilitate IPv6 support
Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Wed, 30 Jun 2010 08:40:39 +0000 (09:40 +0100)]
Add missing pylint disable for "except:"
Why it's needed here but not a few lines above is a mistery that only
pylint understands.
Also fix an indentation error in another disable, for the same function.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 29 Jun 2010 12:56:08 +0000 (14:56 +0200)]
Adding design-doc for privilege separation work done on Ganeti 2.2
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Tue, 29 Jun 2010 15:59:50 +0000 (16:59 +0100)]
Update NEWS for the 2.1.5 release
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Tue, 29 Jun 2010 15:31:40 +0000 (16:31 +0100)]
Pass force variant option at instance creation
This was supposed to be done in
"
06073e857e3b518c7195d57306ae01793240c0c2" but by mistake the field was
added to batch create rather than the normal instance create.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Tue, 18 May 2010 11:13:34 +0000 (12:13 +0100)]
masterd: use AsyncTerminatedMessageStream for luxi
Each luxi connection now creates an asyncore MasterClientHandler (which
is an AsyncTerminatedMessageStream subclass, sending each message to a
client worker). This makes it harder to DOS the master daemon by just
creating luxi connections, as each of them will use memory and file
descriptors, but not a dedicated thread.
Each connection will only handle one message at a time.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Tue, 29 Jun 2010 12:07:21 +0000 (13:07 +0100)]
Fix a few uncommon pylint errors in compat.py
- It's ok if the optional modules functools, roman and hashlib are not there.
- It's ok to reference any and all before defining them. We're doing all
this exactly because, under python 2.4, they are undefined.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Tue, 29 Jun 2010 11:20:20 +0000 (12:20 +0100)]
BatchCreate: get force_variant from specs not opts
There is no such option on the command line, and for batch creation it's
better to get the option from the json file (with a default of false, as
we fixed it in the previous patch).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 25 Jun 2010 15:44:35 +0000 (17:44 +0200)]
AsyncTerminatedMessageStream: limit message count
Currently the message stream can process any number of messages in
parallel (if they get dispatched to different threads or processes).
In order to limit their number we only handle messages and read from
the socket if we're under a certain limit of unanswered ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 25 Jun 2010 15:43:21 +0000 (17:43 +0200)]
AsyncTerminatedMessageStream: send_message
This function adds the ability for a AsyncTerminatedMessageStream to
have a thread-safe message delivery function.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 28 Jun 2010 13:47:12 +0000 (15:47 +0200)]
Add test script for cfgupgrade
This should catch cases where we update the configuration version, but
forget to adjust cfgupgrade accordingly.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Mon, 28 Jun 2010 13:49:46 +0000 (15:49 +0200)]
cfgupgrade: Add option to override config verification
This is needed in a new test script for cfgupgrade where it's better
to only use the minimum parts necessary for testing cfgupgrade.
Including full configuration data to ensure verification doesn't fail
would make maintenance difficult.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Mon, 28 Jun 2010 13:50:24 +0000 (15:50 +0200)]
cfgupgrade: Fix bug when checking configuration directory
In the condition for checking the configuration directory, one “or” should have
been an “and”. This bug was in cfgupgrade since commit
95e4a8142 (June 2008).
Found thanks to a test script.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Mon, 28 Jun 2010 12:34:04 +0000 (14:34 +0200)]
cfgupgrade: Add support for Ganeti 2.2
Since configuration changes are now usually done in the configuration
writer class, cfgupgrade's only task is to change the config version.
This means we can easily support upgrading from Ganeti 2.0 and 2.1 to
2.2.
This patch also brings a couple of small cleanups.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Mon, 28 Jun 2010 12:09:48 +0000 (14:09 +0200)]
Remove dead code from cfgupgrade
Three constants were no longer user after commit
11c31f5cb5, which
removed support for upgrading from Ganeti 1.2.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Tue, 29 Jun 2010 09:54:17 +0000 (10:54 +0100)]
BatchCreate: set a default for force_variant
Currently if the json file doesn't contain a "force_variant" parameter
BatchCreate stracktraces. We add a default so passing it explicitely is
not needed.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Guido Trotter [Mon, 28 Jun 2010 15:39:13 +0000 (16:39 +0100)]
Better specify old object slots removal policy
The comments seem to imply we can remove deprecated slots after 2.1 is
released. This is not quite true, if we still want to support upgrading
straight from 2.0. So let's specify better that they can be removed only
after upgrading from 2.0 can be deprecated.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 28 Jun 2010 13:14:43 +0000 (15:14 +0200)]
Fix a type declaration error in LUTestAllocator
Also, improve logging of type errors, to ease debugging.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Mon, 28 Jun 2010 12:48:29 +0000 (13:48 +0100)]
Fix _UpgradeConfig docstring
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 25 Jun 2010 16:35:01 +0000 (18:35 +0200)]
jqueue: remove the _big_jqueue_lock module global
By using ssynchronized in the new way, we can remove the module-global
_big_jqueue_lock and revert back to an internal _lock inside the jqueue.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Fri, 25 Jun 2010 16:09:19 +0000 (18:09 +0200)]
ssynchronized: act on a class member
The ssynchronized decorator takes the lock to act on in input.
With this change we allow a string to be passed, and if so we assume the
function it protects is a class method, and we act on the member of the
class itself named as the string we got.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 15 Jun 2010 16:39:58 +0000 (17:39 +0100)]
Share the jqueue lock on job-local changes
We can share the jqueue lock when we do per-job updates. These only
conflict with updates/checks on the same job from another thread (eg.
CancelJob, ArchiveJob, which keep the lock unshared, since they are less
frequent).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 9 Jun 2010 16:24:15 +0000 (17:24 +0100)]
_OpExecCallbacks abstract _AppendFeedback
Move some code to a decorated function rather than explicitely
acquiring/releasing the lock in AppendFeedback.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 17 Jun 2010 10:49:25 +0000 (11:49 +0100)]
Remove utils.LockedMethod
All users of this function were converted to another method, hence we
can remove it.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Fri, 4 Jun 2010 16:33:42 +0000 (17:33 +0100)]
jqueue: convert to a SharedLock()
Remove the jqueue _lock member and convert to a _big_jqueue_lock
sharedlock. This allows smooth transition from the old single lock to a
more granular approach.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 24 Jun 2010 14:41:58 +0000 (16:41 +0200)]
MarkUnfinishedOps: update job file on disk
Every time we call MarkUnfinishedOps we do it in a try/finally block
that updates the job file. With this patch we move the try/finally
inside. CancelJobUnlocked is removed, because it just becomes a wrapper
over MarkUnfinishedOps with two constant values.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 24 Jun 2010 15:51:54 +0000 (17:51 +0200)]
Remove spurious empty line
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 23 Jun 2010 22:17:42 +0000 (00:17 +0200)]
Remove pred from compat.any/all
This makes it compatible with the python builtin, and we can even use
the builtin when running under the right version of python. The all and
any functions are renamed to _all and _any, so that they can be tested,
and (non)existing unittests are updated (translation: there are no unit
tests, so none are updated).
This patch also moves the code deciding which partial to use next to
the definition of the _partial function, rather than after TryToRoman.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 28 Jun 2010 10:32:11 +0000 (12:32 +0200)]
Two more fixes for OS params and opcode defaults
If the OS is not using API v20, the parameter verification should be
entirely skipped.
The second change is a simple typo.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 25 Jun 2010 15:19:11 +0000 (17:19 +0200)]
Fix breakage due to OS parameters
I was using wrong python installation path (thanks Guido!), so I was not
actually testing the new backend.py module. Two immediate things bugs
are fixed, and after these burnin passes again…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Apollon Oikonomopoulos [Wed, 23 Jun 2010 13:47:36 +0000 (16:47 +0300)]
Introduce constants.DTS_MAY_ADOPT
DTS_MAY_ADOPT include disk templates that may use disk adoption and will be
used in all respective checks.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 23 Jun 2010 17:51:37 +0000 (19:51 +0200)]
Convert TestDelay repeat to the type system
Turns out with the current implementation of the type system a variable
can both be required and not required. Interesting (but perhaps worth a
further look).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 23 Jun 2010 15:41:53 +0000 (17:41 +0200)]
Rename some constants and relax pylint rules
For the type system, we want a slightly relaxed rule for constant
naming, so we update the pylint rule. But the old _TPInt and _TNEString
were not clear enough, so we expand them.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 20 Jun 2010 06:37:21 +0000 (08:37 +0200)]
Fix burnin's export test
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 18 Jun 2010 15:45:15 +0000 (17:45 +0200)]
Introduce a micro type system for opcodes
Currently, we have one structual validation for opcode attributes: the
_OP_REQP, which checks that a given attribute is not 'None', and the
rest of the checks are done at runtime. This means our type system has
two types: None versus Not-None.
We have been hit many times by small, trivial bugs in this area, and
only a huge amount of unittest and/or hand-written checks would ensure
that we cover all possibilities. This patch attempts to redress the
needs for manual checks by introducing a micro-type system for the
validation of the opcode attributes. What we lose, from the start, are
the custom error messages (e.g. "Invalid reboot mode, choose one of …",
or "The disk index must be a positive integer"). What we gain is the
ability to express easily things as:
- this parameter must be None or an int
- this parameter must be a non-empty list
- this parameter must be either none or a list of dictionaries with keys
from the list of valid hypervisors and the values dictionaries with
keys strings and values either None or strings; furthermore, the list
must be non-empty
These examples show that we have a composable (as opposed to just a few
static types) system, and that we can nest it a few times (just for
sanity; we could nest it up to stack depth).
We also gain lots of ))))))), which is not that nice :)
The current patch moves the existing _OP_REQP to the new framework, but
if accepted, a lot more validations should move to it. In the end, we
definitely should declare a type for all the opcode parameters
(eventually moving _OP_REQP directly to opcodes.py and validating in the
load/init case, and build __slots__ from it).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 18 Jun 2010 12:40:17 +0000 (14:40 +0200)]
Some more CheckPrereq/CheckArguments cleanup
For a few LUs, a few tests in, or even the whole CheckPrereq, can be
moved to CheckArguments, as they don't touch state and only do a 'type'
validation.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 18 Jun 2010 10:18:26 +0000 (12:18 +0200)]
LU.CheckPrereq: do not require implementation
Currently, the base class LogicalUnit's CheckPrereq will raise
NotImplementedError, which means that the child LUs have to implement
it. However, many LUs don't actually have a need for this function
(hence the many "pass" statements as the only body).
By changing the base class behaviour, we can simplify many LUs.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 18 Jun 2010 09:58:12 +0000 (11:58 +0200)]
Remove the obsolete EvacuateNode OpCode/LU
All code has been switched to the new-style LU… time for cleanup.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 18 Jun 2010 09:55:52 +0000 (11:55 +0200)]
RAPI: switch evacuate node to the new model
This patch removes the last use of the old-style OpEvacuateNode. It also
fixes the dry-run mode for this RAPI resource - the dry-run parameter
was not used at all before.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 18 Jun 2010 09:36:14 +0000 (11:36 +0200)]
Abstract export mode validity check
The export mode is checked in two places with the exact same code…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Fri, 18 Jun 2010 09:22:30 +0000 (11:22 +0200)]
Cleanup LU.ExpandNames versus CheckArguments
When LogicalUnit.CheckArguments was introduced, not all code dealing
with static argument checking was moved to it; many of these checks were
left in ExpandNames. With time, most of them migrated, and this patch
does the final cleanups.
The patch is straightforward, with the exception of LURebootInstance,
where an old style ParameterError exception is converted to the new
OpPrereqError with ECODE_INVAL.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 18 Jun 2010 06:17:00 +0000 (08:17 +0200)]
Move opcode attribute defaults to data structures
LUExportInstance had two opcode fields set to default via both
_CheckBooleanOpField and getattr(…, False).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 15 Jun 2010 22:40:44 +0000 (00:40 +0200)]
Add OS verification support to cluster verify
For this, we needed to extend the NodeImage class with a few extra
variables, and we do a trick in the node verification where we pick the
first node that returned valid OS data as the reference node, and then
we compare all other nodes against it.
The checks added are:
- consistency of DiagnoseOS responses
- multiple paths for an OS
- inconsistent OS between a reference node and the current node
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 16 Jun 2010 02:56:48 +0000 (04:56 +0200)]
Simplify gnt-os diagnose output
Currently, we always list the api/variants, even if these are empty.
This patch changes so that we make clear distiction for empty values
("[no variants]" versus "[variants: ]"), and we only list variants and
parameters when the OS API indicates they should be supported.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 14 Jun 2010 17:42:15 +0000 (19:42 +0200)]
Add a new gnt-os info command
This can be used to show the actual OS parameters and supported
variants, in a global manner (rather than per-node as gnt-os diagnose).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 14 Jun 2010 23:45:20 +0000 (01:45 +0200)]
LUDiagnoseOS: add more fields, cleanup
This patch exports all the way from backend a new field ‘api_version’
which holds the list of support API versions, and exposes the (already
computed) ‘parameters’ field.
The patch also reworks (again) the field calculation in its Exec()
method. All callers of LUDiagnoseOS pass in the 'valid' and 'variants'
parameters, thus having the special casing of whether to compute or not
the validity seems overkill. We move to a model where we always compute
these across-nodes arguments, in order to simplify the code, and we also
change the parameters set to be intersection of all node's values (which
means a change in description will drop the parameter from the list of
parameters).
Additionally, we update scripts/gnt-os, which was broken for multi-dir
OSes since the introduction of variants…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 13 Jun 2010 06:18:52 +0000 (08:18 +0200)]
Add support for OS parameters during import/export
Nothing special here, just copy/adjust the beparams code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 13 Jun 2010 05:57:15 +0000 (07:57 +0200)]
Add support for modifying instance OS parameters
We move the instance OS rename checks earlier, as we need to run the
validation against the new OS, if it has changed.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 13 Jun 2010 00:48:34 +0000 (02:48 +0200)]
Add support for modifying cluster OS parameters
We use _GetUpdatedParams in order to support removal too, and then
validate the OS parameters if the OS exists.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 13 Jun 2010 22:01:53 +0000 (00:01 +0200)]
_GetUpdatedParams: enhance value removal options
This patch adds controls for whether we recognize
constants.VALUE_DEFAULT or not as a default value, and also adds
dash-prefixes as another way for parameter removal.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 12 Jun 2010 07:11:50 +0000 (09:11 +0200)]
Add support for OS parameters during instance add
This is not yet complete, as it lacks proper support for instance
import.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 12 Jun 2010 06:49:29 +0000 (08:49 +0200)]
Show OS parameters in cluster/instance info
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 12 Jun 2010 02:17:20 +0000 (04:17 +0200)]
Add OS parameters to cluster and instance objects
The patch also modifies the instance RPC calls to fill the osparameters
correctly with the cluster defaults, and exports the OS parameters in
the instance/OS environment.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 12 Jun 2010 02:13:29 +0000 (04:13 +0200)]
Introduce an RPC call for OS parameters validation
While we only support the 'parameters' check today, the RPC call is
generic enough that will be able to support other checks in the future.
The backend function will both validate the parameters list (so as to
make sure we don't pass in extra parameters that the OS validation
doesn't care about) and the parameter values, via the OS verify script.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 12 Jun 2010 02:02:06 +0000 (04:02 +0200)]
Add reading of OS parameters from disk
The patch also modifies the internal methods in LUDiagnoseOS and gnt-os
to deal with the format change of call_os_diagnose.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sat, 12 Jun 2010 01:55:59 +0000 (03:55 +0200)]
Add os api v20 and related fields to the OS object
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 16 Jun 2010 02:22:51 +0000 (04:22 +0200)]
Silence a pylint warning
The OS parameters code will bump the number of lines over 10K, and thus
we need to silence this (no, we don't want any other module to become
this big…, so we use a targeted silence only).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 23 Jun 2010 04:29:57 +0000 (06:29 +0200)]
Update the 2.2 design doc with OS parameters
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Tue, 22 Jun 2010 09:46:05 +0000 (11:46 +0200)]
Remove job object condition
We don't need it anymore, since nobody waits on it.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 15 Jun 2010 16:39:30 +0000 (17:39 +0100)]
Parallelize WaitForJobChanges
As for QueryJobs we rely on file updates rather than condition
notification to acquire job changes. In order to do that we use the
pyinotify module to watch files. This might make the client a bit slower
(pending planned improvements, such as subscription-based
WaitForJobChanges) but detaches it from the job execution.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 22 Jun 2010 09:02:10 +0000 (11:02 +0200)]
Update the job file on feedback
This is needed to convert waitforjobchanges to use inotify and the
on-disk version and decouple it from the job queue lock. No replication
to remote nodes is done, to keep the operation fast.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>