Guido Trotter [Fri, 21 May 2010 10:24:54 +0000 (11:24 +0100)]
Mainloop: handle SIGINT as well (and terminate)
This is needed if daemons are in the foreground, and get ctrl+c-ed by
the user. Also add unittests to make sure the correct signals terminate
the mainloop.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Fri, 21 May 2010 11:45:56 +0000 (12:45 +0100)]
SingleFileEventHandler: update comments
The comments in the SingleFileEventHandler are still confd-specific.
Update them to make them generic for any single-file monitoring.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 20 May 2010 17:40:21 +0000 (19:40 +0200)]
Allow control of import/export compression method
For exports to/imports from the same machine, compression will
not be used anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 20 May 2010 17:17:32 +0000 (19:17 +0200)]
Put common import/export daemon options into object
The X509 key name and CA are passed from cmdlib all the way to
the backend import/export daemon. With the addition of an option
to choose the compression method, another parameter would have
to be passed all the way. By moving these options to a separate
object, adding new ones will become much easier.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 18 May 2010 16:46:40 +0000 (18:46 +0200)]
import-export daemon: Allow changing compression method
For example, exports on the same node shouldn't be compressed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Fri, 21 May 2010 11:17:57 +0000 (12:17 +0100)]
Merge branch 'devel-2.1'
* devel-2.1:
Add asyncnotifier unittests
Make ConfdInotifyEventHandler a library function
Comment on AsyncNotifier upstream's availability
Remove errors.ConfdFatalError
TestMainloop: test scheduler priorities as well
Fix race condition in mainloop unittest
TestMainloop.testReRun
Add AsyncUDPSocket tests
Add initial mainloop unittests
devel/review: make the range argument optional
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 21 May 2010 10:31:37 +0000 (11:31 +0100)]
Add asyncnotifier unittests
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 20 May 2010 10:45:35 +0000 (11:45 +0100)]
Make ConfdInotifyEventHandler a library function
Cut&Paste, plus the following changes:
- The class is renamed to SingleFileEventHandler
- The monitored filename must be passed in and doesn't default to the
ganeti cluster config file
- A small docstring is added to the class
- Pylint disables for except: and method names are added
This makes it possible to write unittests for this class.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 20 May 2010 10:33:35 +0000 (11:33 +0100)]
Comment on AsyncNotifier upstream's availability
Since we contributed AsyncNotifier back to the upstream pyinotify
project, we'll be able, one day, to remove the ganeti version of that
code. For now we still need it to support older distributions, but
having a note about when we'll be able to remove it is nice.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 20 May 2010 10:20:25 +0000 (11:20 +0100)]
Remove errors.ConfdFatalError
This exception is caught, but never thrown. It became useless when we
moved confd from on/off to enabled/disabled, but always running on all
nodes. Removing its definition and the code catching it can do no harm.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 19 May 2010 21:54:34 +0000 (22:54 +0100)]
TestMainloop: test scheduler priorities as well
By using enterabs we can schedule events at the same time, which will
then be sorted by priority.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 19 May 2010 21:52:03 +0000 (22:52 +0100)]
Fix race condition in mainloop unittest
Currently, in testDeferredCancel if the self._CancelEvent are entered
more than 0.3 seconds after the _SendSig have been entered, the test
could fail. This is unlikely but may happen. To avoid it we use
enterabs, to use absolute times.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 19 May 2010 16:22:47 +0000 (17:22 +0100)]
TestMainloop.testReRun
Currently we never do that, but it's nice to know we can restart the
mainloop, after termination.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 19 May 2010 15:39:41 +0000 (16:39 +0100)]
Add AsyncUDPSocket tests
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 19 May 2010 14:09:30 +0000 (15:09 +0100)]
Add initial mainloop unittests
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Mon, 17 May 2010 12:30:46 +0000 (13:30 +0100)]
devel/review: make the range argument optional
Rather than calling devel/review with from..to target we can just use
target, and assume target..HEAD as the range. This makes it a lot
easier, as now you just have to say something like "devel/review
devel-2.1" if you're already on the branch you want to commit, and want
to commit it to devel-2.1.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 17 May 2010 11:49:19 +0000 (13:49 +0200)]
Add tool to move instances between clusters
This is a first version of the instance move tool and it supports moving
1..N instances from one cluster to another. When moving a single instance,
the instance can be renamed, allowing for moves within the same cluster
(not really useful in practice, but good for testing). Documentation is
updated to describe this new tool and its usage.
The “move-instance” tool uses the workerpool to support parallel moves of
instances. Supporting them was simple as threads were required anyway due
to the synchronous RAPI client.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 4 May 2010 15:38:21 +0000 (17:38 +0200)]
RAPI changes for instance moves
Two new resources are added:
- /2/instances/$name/prepare-export
- /2/instances/$name/export
The documentation for the existing resource for creating instances is updated
for remote imports. The RAPI client is extended for the new resources.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 16:59:05 +0000 (18:59 +0200)]
Implement opcode changes for remote-import
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 13 May 2010 14:57:24 +0000 (16:57 +0200)]
Implement opcode changes for remote-export
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 30 Apr 2010 13:38:32 +0000 (15:38 +0200)]
Add opcode to prepare export
To prepare a remote export, the X509 key and certificate need to be generated.
A handshake value is also returned for an easier check whether both clusters
share the same cluster domain secret.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 18 May 2010 13:51:01 +0000 (14:51 +0100)]
Merge branch 'devel-2.1'
* devel-2.1:
Abstract the LUXI eom into a constant
KVM: vhost net acceleration support
Conflicts:
lib/luxi.py
- trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 18 May 2010 13:32:33 +0000 (15:32 +0200)]
Fix distcheck broken by check-cert-expired unittest
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 18 May 2010 12:40:09 +0000 (14:40 +0200)]
ganeti-cleaner: Remove expired X509 certs
Importing/exporting an instance to a remote machine creates X509
certificates which expire after some time. They need to be removed from
the nodes as they become useless.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 18 May 2010 09:37:15 +0000 (10:37 +0100)]
Abstract the LUXI eom into a constant
Currently the EOM terminator is hardcoded on the server side, and is
customizable in the Transport object (with the default being the same as
the value found in the server), but not in the luxi client.
With this patch we move the value to constants, and remove the "fake"
customizability, which would just break client/server communication. If
we ever need to have a luxi transport with a different terminator it's
easy enough to add it back.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Mon, 17 May 2010 12:05:13 +0000 (13:05 +0100)]
KVM: vhost net acceleration support
This will only work on patched or newer (>= 2.6.34) kernels and with a
patched version of qemu-kvm.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 18 May 2010 10:57:05 +0000 (12:57 +0200)]
Merge branch 'devel-2.1'
* devel-2.1:
ganeti-cleaner: Write log file with removed files
RAPI client: Handle urllib2.HTTPError and raise GanetiApiError
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 13:48:15 +0000 (15:48 +0200)]
ganeti-cleaner: Write log file with removed files
Logfiles can be useful for debugging.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 13 May 2010 14:57:38 +0000 (16:57 +0200)]
Add function to read cluster domain secret
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 12:40:55 +0000 (14:40 +0200)]
LUExportInstance: Remove instance only if export was successful
Until now, the instance was always removed (if asked for by the
user). In case of export errors however, it shouldn't.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 11:49:41 +0000 (13:49 +0200)]
RAPI client: Handle urllib2.HTTPError and raise GanetiApiError
This allows users of the RAPI client to catch GanetiApiError for all HTTP
errors.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 11:00:07 +0000 (13:00 +0200)]
Merge branch 'devel-2.1'
* devel-2.1:
RAPI: /2/{nodes,instances}/$name should return 404 for unknown items
ganeti-masterd: Improve error logging for client requests
Return disk_template from LUQueryInstanceData
RAPI client: Rename Get{Node,Instance}Info, add new GetInstanceInfo
RAPI client: Log request to be made
Add missing documentation for RAPI instance creation mode
Add checks for master IP in cluster verify
Remove unused import from daemon.py
utils.IgnoreSignals
AsyncUDPSocket.handle_error
Add a forgotten comment about overriding a method
ganeti-noded: add the --no-mlock option
Describe more ganeti-noded options in the manpage
Conflicts:
daemons/ganeti-masterd: Trivial
test/ganeti.backend_unittest.py: Trivial
test/ganeti.utils_unittest.py: Trivial
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 17:08:21 +0000 (19:08 +0200)]
RAPI: /2/{nodes,instances}/$name should return 404 for unknown items
Currently they return a 500 Server Error, not really useful
for detecting nonexistent items.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 17:07:24 +0000 (19:07 +0200)]
ganeti-masterd: Improve error logging for client requests
Ganeti errors should also be logged with a backtrace.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 16:26:30 +0000 (18:26 +0200)]
Return disk_template from LUQueryInstanceData
Inter-cluster instance moves need the disk template. As they run
LUQueryInstanceData to get an instance's details, the disk template
must be returned.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 16:25:12 +0000 (18:25 +0200)]
RAPI client: Rename Get{Node,Instance}Info, add new GetInstanceInfo
GetInstanceInfo should return the resource /2/instances/$name/info,
but so far it returns /2/instances/$name. The same applies to
GetNodeInfo, which returns /2/nodes/$name. Both names are stripped
of their “Info” suffix and a new method, GetInstanceInfo, is added
for /2/instances/$name/info. There's no equivalent for nodes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 16:12:42 +0000 (18:12 +0200)]
RAPI client: Log request to be made
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 10:33:01 +0000 (12:33 +0200)]
Add missing documentation for RAPI instance creation mode
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 17 May 2010 09:54:34 +0000 (11:54 +0200)]
Add checks for master IP in cluster verify
This also updates a comment in the unittest for utils.py. We unittest
the new function for two things: correct reporting on real case (for
localhost), and correct reporting with a mocked-out TcpPing that returns
false.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 15:54:25 +0000 (16:54 +0100)]
RetryOnSignal: handle socket error as well
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 16:01:52 +0000 (17:01 +0100)]
Remove unused import from daemon.py
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>
Guido Trotter [Fri, 14 May 2010 13:44:23 +0000 (14:44 +0100)]
utils.IgnoreSignals
Remove duplicate code between a couple of asyncore related function by
having a function in charge of handling EINTR errors. Unittests included.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 13:52:58 +0000 (14:52 +0100)]
AsyncUDPSocket.handle_error
By overriding the default asyncore handle_error (which closes the
socket) with our own version, which logs what happened but tries to
proceed, we can get rid of a couple of try/except blocks. The resulting
churn is deindentation of the internal code.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 13:46:36 +0000 (14:46 +0100)]
Add a forgotten comment about overriding a method
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 07:45:53 +0000 (08:45 +0100)]
ganeti-noded: add the --no-mlock option
While mlock on noded is definitely good in most situations, there are
some - namely my laptop - where it has no benefit, and uses precious
non-swappable memory. To avoid this we make it optional, with a new
--no-mlock option. Note that only the main node daemon and its http
children are affected: the powercycle node child still uses mlock, which
doesn't harm, since it's a short lived process happening just before
node reboot anyway. The manpage is updated.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>
Guido Trotter [Fri, 14 May 2010 07:42:08 +0000 (08:42 +0100)]
Describe more ganeti-noded options in the manpage
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>
Luca Bigliardi [Thu, 13 May 2010 16:30:12 +0000 (17:30 +0100)]
Merge branch 'devel-2.1'
Conflicts:
daemons/ganeti-noded
lib/daemon.py
lib/rapi/baserlib.py
lib/rapi/rlib2.py
lib/utils.py
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Thu, 13 May 2010 15:49:35 +0000 (16:49 +0100)]
Fix some pylint warnings
Disable warnings for:
- except Exception,
- use of __errno_location,
- redeclaration of handleError()
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Luca Bigliardi [Thu, 13 May 2010 15:33:04 +0000 (16:33 +0100)]
Fix logging string format
Fix this pylint warning:
[W6501, Mlockall] Specify string format arguments as logging function parameters
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Thu, 13 May 2010 15:14:08 +0000 (16:14 +0100)]
Fix Filehandler / FileHandler typo
Fix typo spotted by pylint:
E1101:2095:LogFileHandler.handleError: Module 'logging' has no 'Filehandler' member
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 18:06:35 +0000 (20:06 +0200)]
QA: Test instance creation/deletion via RAPI client
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 17:56:20 +0000 (19:56 +0200)]
RAPI client: Implement instance creation
Currently this only supports the new instance creation request data
format version 1, but support for the old version can be easily
implemented.
Most arguments are optional and documented in the RAPI documentation.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 17:45:10 +0000 (19:45 +0200)]
RAPI: Add new request data format for instance creation
As mentioned in commit
d975f482d, the current way of creating an
instance via RAPI is not very flexible. With this patch, a new
instance creation request data format is introduced and documented.
Support can be detected by checking the list of features returned
by the /2/features resource for the value "instance-create-reqv1".
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 17:42:22 +0000 (19:42 +0200)]
Mention RAPI client in documentation
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 17:41:03 +0000 (19:41 +0200)]
rapi.baserlib: Add function to check variable type
Also add a separate function to retrieve body parameters. Having
it separate (independent of a class instance) will make it easier
to unittest users of this function.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 17:37:11 +0000 (19:37 +0200)]
Add new /2/features RAPI resource
The /2/features RAPI resource can be used to detect optional
features implemented by the RAPI server. This will be used
to recognize servers implementing a new request format for
instance creation requests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Luca Bigliardi [Thu, 13 May 2010 12:58:31 +0000 (13:58 +0100)]
Use console_logging in node daemon
Node daemon logs will be printed on system console if writing on the log file
fails.
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Wed, 12 May 2010 11:01:31 +0000 (12:01 +0100)]
Daemons conditionally setup console logging
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Wed, 12 May 2010 10:44:37 +0000 (11:44 +0100)]
Use LogFileHandler conditionally in SetupLogging
Add a parameter to SetupLogging to use LogFileHandler (default is off)
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Wed, 5 May 2010 16:09:55 +0000 (17:09 +0100)]
Introduce LogFileHandler class
Add a log handler class which logs to /dev/console in case of logging errors.
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Wed, 12 May 2010 10:35:51 +0000 (11:35 +0100)]
Add /dev/console constant
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Thu, 6 May 2010 15:16:49 +0000 (16:16 +0100)]
Lock PowercycleNode child in memory
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Thu, 6 May 2010 15:16:01 +0000 (16:16 +0100)]
Lock node daemon children in memory
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Thu, 6 May 2010 14:58:52 +0000 (15:58 +0100)]
Lock node daemon in memory
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Tue, 4 May 2010 13:55:21 +0000 (14:55 +0100)]
Introduce Mlockall()
Add Mlockall() utility to lock current process' virtual adress space into RAM.
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Wed, 12 May 2010 15:02:36 +0000 (16:02 +0100)]
Node daemon availability improvements proposal
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 12:12:00 +0000 (14:12 +0200)]
Fix wrong variable name in commit
d975f482
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 12 May 2010 11:30:27 +0000 (13:30 +0200)]
RAPI: Add initial support for instance creation request version
The way the resource /2/instances expects its request data (e.g.
instance name, disks, NICs) to be formatted in a dict is not
very extensible. HV and BE parameters are interleaved with all
other values. In commit
495cfdf0 a new parameter “mode” was added
which can be misunderstood as the instance creation mode, but is
actually a mode for the network interface.
Short of bumping the whole RAPI version (currently 2), which would
involve many, many other changes and potentially break clients,
we can not change the request format easily as it needs to stay
backwards and future-compatible as far as possible. This is not an
internal API, but one used by several external applications.
By introducing a data format version per request, we can still
support the old request format, and thereby not break existing users,
while we can add a more flexible request format which will, for
example, support multiple NICs and move HV/BE parameters to separate
containers, avoiding conflicts.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Wed, 12 May 2010 11:22:47 +0000 (12:22 +0100)]
Merge branch 'devel-2.1'
* devel-2.1:
Convert some ReadFile calls to ReadOneLineFile
utils.ReadOneLineFile()
Remove oneline= parameter from utils.ReadFile
install.rst: update os version and add KVM notes
INSTALL: update hypervisor requirements
RAPI client: Fix SSL error reporting for real
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 12 May 2010 10:09:33 +0000 (11:09 +0100)]
Convert some ReadFile calls to ReadOneLineFile
For passwords we require strict oneliners, we're a bit more lax with pid
and uid files.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 11 May 2010 16:00:22 +0000 (17:00 +0100)]
utils.ReadOneLineFile()
Read the first non-empty file line. When strict is set, abort if more
than one line is non-empty. Some unittests inspired by the reverted ones
from commit
b774bb106cc28d008e790ad2666eb64c76866fa0, and some new ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 11 May 2010 15:32:03 +0000 (16:32 +0100)]
Remove oneline= parameter from utils.ReadFile
This partially reverts commit
b774bb106cc28d008e790ad2666eb64c76866fa0.
Unittests unrelated to that particular functionality but introduced in
that commit are left untouched. Since the temporary directory is now
only used to check for a non-existing file, it is removed as well, and
a convenient very-very-unlikely-to-exist file is used instead.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 12 May 2010 08:40:15 +0000 (09:40 +0100)]
install.rst: update os version and add KVM notes
The 0.7 version was just an example, but since we're on 0.9 now,
updating the number cannot be wrong. The new admonition helps KVM users
to configure debootstrap correctly so that instances can be shutdown
cleanly.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 12 May 2010 08:30:03 +0000 (09:30 +0100)]
INSTALL: update hypervisor requirements
Xen: Just specify 3.0 or above as a requirement, as versions above 3.1
have been tested and work fine with Ganeti. Also note that the Xen
dependency is optional, if one wants to work with KVM.
KVM: list as an alternative, specify that 72 (the one shipped with
lenny) is the bare minimum, but newer qemu-kvm is recommended.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 12 May 2010 09:28:38 +0000 (11:28 +0200)]
RAPI client: Fix SSL error reporting for real
My previous patch, commit
857705e8, helped in one particular
situation where the exception didn't contain any arguments
(pyOpenSSL reporting a WantReadError, which shouldn't occur with a
blocking socket anyway). With this patch, more common and uncommon
errors should be easy to recognize.
SSL errors without any of these patches:
“ganeti.rapi.client.CertificateError: SSL issue:
[('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE',
'certificate verify failed')]”
SSL errors with both patches:
“ganeti.rapi.client.CertificateError: SSL issue:
[('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE',
'certificate verify failed')] (<OpenSSL.SSL.Error
instance at 0x…>)”
WantReadError without any of these two patches:
“ganeti.rapi.client.CertificateError: SSL issue: ”
WantReadError with both patches:
“ganeti.rapi.client.CertificateError: SSL issue:
(<OpenSSL.SSL.WantReadError instance at 0x…>)”
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 11 May 2010 17:59:21 +0000 (19:59 +0200)]
Merge branch 'devel-2.1'
* devel-2.1: (29 commits)
RAPI client: Improve SSL error reporting
RAPI client: Don't check node role in client
RAPI client: Update ReplaceInstanceDisks
RAPI client: Fix behaviour of “allocatable” storage flag
RAPI client: Encode boolean and None query values
RAPI client: No longer check storage type locally
RAPI client: Add constant for RAPI version
RAPI QA: Use RAPI client
QA: Restore RAPI cert after “gnt-cluster renew-crypto”
QA: Add function create temporary backup file
RAPI client: Don't assume job IDs to be numeric
Add RAPI client utility module with RAPI PollJob function
RAPI client: Include HTTP error code in GanetiApiError exception
Support different HTTP error codes in RAPI client unittest mock
RAPI client: Fix error message for unsupported methods in unittest
RAPI client: Allow waiting for job changes
RAPI: Allow waiting for job changes
RAPI client: Rename “DeleteJob” to “CancelJob”
RAPI client: Various code style changes
RAPI client: Add missing unittest for RepairNodeStorageUnits
...
Conflicts:
qa/qa_cluster.py: Trivial
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>
Michael Hanselmann [Tue, 11 May 2010 16:58:40 +0000 (18:58 +0200)]
RAPI client: Improve SSL error reporting
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Tue, 11 May 2010 15:55:45 +0000 (17:55 +0200)]
RAPI client: Don't check node role in client
Only the server knows which node roles can be set via RAPI.
Constants are provided for convenience.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Tue, 11 May 2010 15:50:50 +0000 (17:50 +0200)]
RAPI client: Update ReplaceInstanceDisks
- The RAPI client shouldn't check the replacement mode as only
the server knows which ones are valid (constants are still
provided for convenience)
- Disks shouldn't be a list of names, but of indexes
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Tue, 11 May 2010 14:22:29 +0000 (16:22 +0200)]
RAPI client: Fix behaviour of “allocatable” storage flag
When modifying a storage unit, the “allocatable” flag should default
to “no modification”. This replicates the behaviour of the command
line interface.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Tue, 11 May 2010 16:19:12 +0000 (18:19 +0200)]
RAPI client: Encode boolean and None query values
Boolean values must be passed as 0 or 1. None should be an empty
value ("").
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Tue, 11 May 2010 14:14:30 +0000 (16:14 +0200)]
RAPI client: No longer check storage type locally
Only the RAPI server (actually masterd) knows which storage types
are valid. The exception can no longer be raised as the type is
only checked in the job.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Tue, 11 May 2010 14:08:40 +0000 (16:08 +0200)]
RAPI client: Add constant for RAPI version
This reverts
a60e3cb0a partially by moving the RAPI version into a constant.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Mon, 10 May 2010 15:26:32 +0000 (17:26 +0200)]
RAPI QA: Use RAPI client
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 10 May 2010 15:21:45 +0000 (17:21 +0200)]
QA: Restore RAPI cert after “gnt-cluster renew-crypto”
The RAPI tests will depend on the certificate.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 10 May 2010 15:21:59 +0000 (17:21 +0200)]
QA: Add function create temporary backup file
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 10 May 2010 14:10:30 +0000 (16:10 +0200)]
RAPI client: Don't assume job IDs to be numeric
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Mon, 10 May 2010 14:09:37 +0000 (16:09 +0200)]
Add RAPI client utility module with RAPI PollJob function
The RAPI client module shouldn't depend on any Ganeti module, yet it's
useful to have some Ganeti-specific code, like a PollJob function for
RAPI.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Tue, 11 May 2010 12:45:56 +0000 (14:45 +0200)]
RAPI client: Include HTTP error code in GanetiApiError exception
Having the HTTP error code allows users of the class to act differently based
on the error code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Tue, 11 May 2010 12:47:15 +0000 (14:47 +0200)]
Support different HTTP error codes in RAPI client unittest mock
This allows testing error conditions (e.g. if a job wasn't found).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Fri, 7 May 2010 18:54:45 +0000 (20:54 +0200)]
RAPI client: Fix error message for unsupported methods in unittest
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Fri, 7 May 2010 19:04:33 +0000 (21:04 +0200)]
RAPI client: Allow waiting for job changes
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 4 May 2010 17:02:54 +0000 (19:02 +0200)]
RAPI: Allow waiting for job changes
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 7 May 2010 18:29:18 +0000 (20:29 +0200)]
RAPI client: Rename “DeleteJob” to “CancelJob”
Jobs can't be deleted, but cancelled (even though the HTTP method
“DELETE” is used).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Fri, 7 May 2010 17:56:59 +0000 (19:56 +0200)]
RAPI client: Various code style changes
- Replace hardcoded values with constants
- Code formatting
- Exception messages without periods and fixed string formatting
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Fri, 7 May 2010 17:40:59 +0000 (19:40 +0200)]
RAPI client: Add missing unittest for RepairNodeStorageUnits
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Fri, 7 May 2010 17:40:43 +0000 (19:40 +0200)]
RAPI client: Always pass _SendRequest args as positional
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Fri, 7 May 2010 17:28:49 +0000 (19:28 +0200)]
RAPI client: Simplify URL construction
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Fri, 7 May 2010 17:22:31 +0000 (19:22 +0200)]
RAPI client: Instantiate JSON encoder only once
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>
Michael Hanselmann [Fri, 7 May 2010 17:17:07 +0000 (19:17 +0200)]
RAPI client: Always return job ID
Even removing tags returns a job ID.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: David Knowles <dknowles@google.com>