Tom Limoncelli [Wed, 2 Jun 2010 15:06:37 +0000 (11:06 -0400)]
ganeti-watcher should attempt to fix ganeti-rapi
Update ganeti-watcher so that it tests the master's RAPI port with a
simple test (in this case GetVersion). If it fails, make one attempt
at restarting ganeti-rapi and retest.
- daemons/ganeti-watcher: Test rapi and make one attempt at restarting it.
- lib/utils.py: add StopDaemon() function.
Signed-off-by: Tom Limoncelli <tlim@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 2 Jun 2010 17:30:04 +0000 (18:30 +0100)]
TestAsyncUDPSocket: remove dead code and add test
- _ThreadedClient was added on the idea of making this unittest
concurrent, which was actually never done (we could test everything
without it, so well)
- handle_write() was never called without filling the send queue, and
this caused me trouble now that I learned to look at coverage
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 2 Jun 2010 17:19:52 +0000 (18:19 +0100)]
TestAsyncUDPSocket: test for oversized sends
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>
Guido Trotter [Wed, 2 Jun 2010 16:35:33 +0000 (17:35 +0100)]
Document the check-man change
Since this affects developers' systems, document it in NEWS and
devnotes.rst
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 2 Jun 2010 11:04:07 +0000 (12:04 +0100)]
Update NEWS for Ganeti 2.1.3
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 2 Jun 2010 14:48:54 +0000 (16:48 +0200)]
Second attempt at fixing check-man
I was wrong, actually LANG-vs-LC_ALL only fixed one case, by mistake. To
get proper UTF-8 encoding, we need to enforce any UTF-8 locale. We
choose the 'default' of en_US.UTF-8.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 2 Jun 2010 13:51:55 +0000 (15:51 +0200)]
Fix check-man for newer man-db
Again, check-man :)
Commit
5fa1642226 removed LC_ALL=C, since that breaks the check.
However, with no LANG/LC_* variables, man-db is still broken.
We import the new lintian behaviour, i.e. LANG=C (which seems to differ
from LC_ALL=C, even with empty environment). I'm not sure of the
difference, though.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Balazs Lecz [Wed, 26 May 2010 15:52:27 +0000 (16:52 +0100)]
Add RemoveDir utility function
Backported from master,
72087dcd5b06c0127e2ec3bf8c80f7f54da3fb01
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Tue, 1 Jun 2010 17:23:28 +0000 (18:23 +0100)]
Merge remote branch 'origin/devel-2.1'
* origin/devel-2.1:
Explicitely return None from IgnoreSignals
AsyncUDPSocket: fix IgnoreSignals usage and test
Add KVM chroot feature
Fix and Improve TryToRoman unittest
Conflicts:
test/ganeti.daemon_unittest.py
- trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Tue, 1 Jun 2010 17:14:54 +0000 (18:14 +0100)]
Explicitely return None from IgnoreSignals
Same result, but what happens is clearer.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Tue, 1 Jun 2010 16:56:52 +0000 (17:56 +0100)]
AsyncUDPSocket: fix IgnoreSignals usage and test
This bug was found in the asyncore master patch series, but actually
applies to 2.1 for AsyncUDPSocket as well.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Balazs Lecz [Wed, 26 May 2010 15:53:02 +0000 (16:53 +0100)]
Add KVM chroot feature
This patch adds a new boolean hypervisor parameter to the KVM hypervisor,
named 'use_chroot'.
If it's turned on for an instance, than KVM is started in "chroot mode":
Ganeti creates an empty directory for the instance and passes the path
of this dir to KVM via the -chroot flag.
KVM changes its root to this directory after starting up.
It also adds a "quarantine" feature for moving any unexpected files to
a separate directory for later analysis.
This has been backported from master,
commit
84c08e4ee04cadbaf7d7be8bc1c3b9023918e276
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 26 May 2010 18:58:40 +0000 (20:58 +0200)]
utils: Add function to check whether process handles a signal
This will be used to avoid a race condition between starting a program (dd
for import/export) and sending signals to it.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Tue, 1 Jun 2010 10:45:30 +0000 (11:45 +0100)]
Fix and Improve TryToRoman unittest
1) Don't break when the roman module is not found
2) Test that not finding the roman module doesn't make TryToRoman fail
(currently that is the case)
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 31 May 2010 16:25:57 +0000 (18:25 +0200)]
move-instance: Use error message instead of multiple state variables
Until now, move-instance used different status variables: “success”,
“abort” and “error_message”. With this patch, everything is changed
to use “error_message” only. This simplifies the code a bit.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Mon, 31 May 2010 15:53:30 +0000 (17:53 +0200)]
Distribute cluster domain secret
The cluster domain secret file was not distributed to other nodes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 1 Jun 2010 10:32:44 +0000 (11:32 +0100)]
Merge branch 'devel-2.1'
* devel-2.1:
Convert gnt-instance list and info to use roman
gnt-cluster info --roman
FormatUidPool: provide optional roman conversion
gnt-node: remove latinfriendlyfields
Move roman conversion to compat
Add a new opcode timestamp field
Fix IgnoreSignals on socket.error
RAPI client should convert urllib2.URLError to GanetiApiError
KVM: Migration bandwidth and downtime control
Make utils.EnsureDirs() ignore umask
Fix two race conditions in reboot instance
Support for latin friendly output in node list
Conflicts:
man/gnt-instance.sgml
- trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 1 Jun 2010 09:23:25 +0000 (10:23 +0100)]
Convert gnt-instance list and info to use roman
Finally gnt-instance has roman support as well.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 27 May 2010 10:28:04 +0000 (11:28 +0100)]
gnt-cluster info --roman
Convert to roman (if so the user wishes) the following:
- cluster candidate size
- uid pool
- any integer be or hv parameter
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 27 May 2010 10:26:16 +0000 (11:26 +0100)]
FormatUidPool: provide optional roman conversion
The convert= option of compat.tryToRoman is used to do optional
conversion without duplicating formatting code.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 27 May 2010 10:25:01 +0000 (11:25 +0100)]
gnt-node: remove latinfriendlyfields
Rather than relying on a static list of fields, we opportunistically
convert all integers.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 27 May 2010 10:21:17 +0000 (11:21 +0100)]
Move roman conversion to compat
The new TryToRoman function provides optional easy to use roman
conversion. Nunc cum demonstrationi unitati.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Mon, 31 May 2010 16:45:46 +0000 (18:45 +0200)]
ssconf: error out when writing oversized files
Since we impose a maximum limit when reading ssconf files, let's error
out when trying to write them too big, so we don't pretend everything is
ok, and make mistakes when we actually read partial files.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 1 Jun 2010 07:48:04 +0000 (09:48 +0200)]
Add a new opcode timestamp field
Since the current start_timestamp opcode attribute refers to the inital
start time, before locks are acquired, it's not useful to determine the
actual execution order of two opcodes/jobs competing for the same lock.
This patch adds a new field, exec_timestamp, that is updated when the
opcode moves from OP_STATUS_WAITLOCK to OP_STATUS_RUNNING, thus allowing
a clear view of the execution history. The new field is visible in the
job output via the 'opexec' field.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Mon, 31 May 2010 09:36:14 +0000 (11:36 +0200)]
Fix IgnoreSignals on socket.error
Some confusion arose handling EINTR on this function: in python 2.6
socket.error is an IOError, and thus:
- It's an EnvironmentError
- It has an .errno member
In 2.4 and 2.5 it's not, and so its errno variable must be extracted
from the args tuple. This patch fixes both the function, and the
unittests.
This is a cherry-pick of master commit
965d0e5ba37f3e88aa38230177ad1c66814bf927 with the portions not relevant
to 2.1 removed (changes to the RetryOnSignals function).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Mon, 31 May 2010 09:36:14 +0000 (11:36 +0200)]
Fix {Ignore, RetryOn}Signals on socket.error
Some confusion arose handling EINTR on those functions: in python 2.6
socket.error is an IOError, and thus:
- It's an EnvironmentError
- It has an .errno member
In 2.4 and 2.5 it's not, and so its errno variable must be extracted
from the args tuple. This patch fixes both functions, and the unittests.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Fri, 28 May 2010 18:48:17 +0000 (19:48 +0100)]
Move hash functions to the compat module
Since the hash functions' changed their module name between python 2.4
and 2.6, and we have to do an try/import/except trick, we'll do it just
once, for both hash functions, and in compat.py. This also fixes a use
of md5 in the utils unittests which didn't use the trick before, and
generated a deprecation warning under 2.6.
In compat we keep both a ganeti-wide non-version-specific version to be
used by other ganeti modules, and a python-version specific that can be
passed to python modules which expect a hash function for their input
but call it differently under different versions of python (hmac, for
example).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 26 May 2010 13:12:19 +0000 (14:12 +0100)]
reraise exceptions in async tests' error handlers
This makes sure that any unforeseen error raises an exception rather
then just increasing a counter. It makes unittest debugging a lot
easier.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 27 May 2010 15:19:36 +0000 (16:19 +0100)]
design-2.2: job queue lock analysis/remediation
This builds up on the "Master core scalability design doc" detailing the
critical situations in the job queue and proposing how to fix them. The
bulleted point list at the beginning is changed to subparagraph, as the
job queue part is quite longer and more detailed, then the remediation
section is updated removing the generic "we'll fix it somehow" paragraph
to propose a real solution.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 18 May 2010 15:38:25 +0000 (16:38 +0100)]
Master core scalability design doc
This initial design still lacks information about the job queue lock
contention decrease.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Tom Limoncelli [Mon, 31 May 2010 17:09:00 +0000 (13:09 -0400)]
RAPI client should convert urllib2.URLError to GanetiApiError
Signed-off-by: Tom Limoncelli <tlim@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Apollon Oikonomopoulos [Mon, 31 May 2010 11:28:47 +0000 (14:28 +0300)]
KVM: Migration bandwidth and downtime control
Introduce 2 new hypervisor options, migration_bandwidth and migration_downtime
and implement KVM migration bandwidth and downtime control.
migration_bandwidth controls KVM's maximal bandwidth during migration, in
MiB/s. Default value is 32 MiB/s, same as KVM's internal default. This option
is a global hypervisor option.
migration_downtime sets the amount of time (in ms) a KVM instance is allowed to
freeze while copying memory pages. This is useful when migrating busy guests,
as KVM's internal default of 30ms is too low for the page-copying algorithm to
converge. This is a per-instance option, with a default of 30ms, same as KVM's
internal default.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Balazs Lecz [Fri, 28 May 2010 12:31:21 +0000 (13:31 +0100)]
Make utils.EnsureDirs() ignore umask
EnsureDirs() should create directories with the exact mode requested
in the arguments, but it currently applies the umask.
This patch makes it independent from the umask.
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Fri, 21 May 2010 16:26:00 +0000 (18:26 +0200)]
import/export daemon: Move some I/O processing code to module
The code parsing the child process' output is moved to a separate
class in the impexpd module. As more programs are added, it'll
become more complex and should be separated.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Fri, 21 May 2010 14:07:34 +0000 (16:07 +0200)]
import/export daemon: Move command building into separate module
The import/export daemon code is already large. Moving some code
to a separate module will make it smaller and easier to test.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Fri, 21 May 2010 11:50:45 +0000 (13:50 +0200)]
import/export daemon: Move command building into class
Instead of passing around many variables for building the executed
command, they're now kept as instance variables.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Balazs Lecz [Wed, 26 May 2010 15:53:02 +0000 (16:53 +0100)]
Add KVM chroot feature
This patch adds a new boolean hypervisor parameter to the KVM hypervisor,
named 'use_chroot'.
If it's turned on for an instance, than KVM is started in "chroot mode":
Ganeti creates an empty directory for the instance and passes the path
of this dir to KVM via the -chroot flag.
KVM changes its root to this directory after starting up.
It also adds a "quarantine" feature for moving any unexpected files to
a separate directory for later analysis.
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Balazs Lecz [Wed, 26 May 2010 15:52:27 +0000 (16:52 +0100)]
Add RemoveDir utility function
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 27 May 2010 11:37:55 +0000 (13:37 +0200)]
Fix two race conditions in reboot instance
If the instance crashes between backend.InstanceReboot checks the list
of running instances and the execution of hv_xen.RebootInstance,
ini_info will be None. And if the instance doesn't reboot fast enough,
new_info will be None. Both cases lead to “TypeError: unsubscriptable
object”. Too bad pylint doesn't detect such cases.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Tue, 25 May 2010 15:30:52 +0000 (16:30 +0100)]
Support for latin friendly output in node list
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 25 May 2010 11:20:41 +0000 (12:20 +0100)]
Merge branch 'devel-2.1'
* devel-2.1:
Test for errors during inotify callback
SingleFileEventHandler: Remove try/except blocks
ErrorLoggingAsyncNotifier
daemon.GanetiBaseAsyncoreDispatcher
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Fri, 21 May 2010 13:54:54 +0000 (14:54 +0100)]
Test for errors during inotify callback
- Create a new _MyErrorLoggingAsyncNotifier class which registers
error counts, rather than logging them
- Add an additional ERR notifier to test with
- Check that no error was returned, for tests that weren't supposed to
- Add a new test case for a callback that's supposed to raise an
exception
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Fri, 21 May 2010 13:28:40 +0000 (14:28 +0100)]
SingleFileEventHandler: Remove try/except blocks
Since now we use the SingleFileEventHandler together with an error
handling asyncore dispatcher, we don't need the internal try/except
anymore.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Fri, 21 May 2010 13:27:04 +0000 (14:27 +0100)]
ErrorLoggingAsyncNotifier
This mixes AsyncNotifier with GanetiBaseAsyncoreDispatcher to provide an
AsyncNotifier which will log errors, rather than bail out.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 13 May 2010 17:32:25 +0000 (18:32 +0100)]
daemon.GanetiBaseAsyncoreDispatcher
Abstract a few common functionalities between all ganeti asyncore
dispatchers:
- Handle errors by logging them, and then continue
- By default check sockets only for readability
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Mon, 24 May 2010 10:05:53 +0000 (11:05 +0100)]
Merge branch 'devel-2.1'
* devel-2.1:
TestSingleFileEventHandler: abstract notifier type
Mainloop: handle SIGINT as well (and terminate)
SingleFileEventHandler: update comments
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Guido Trotter [Fri, 21 May 2010 14:12:05 +0000 (15:12 +0100)]
TestSingleFileEventHandler: abstract notifier type
Rather than hardcode that we have two notifiers, and notifier 0 is the
terminating one, we abstract this with class level constants. This makes
it easier to add more, with different features.
The only real change is that now the callback class takes as input the
whole test object, rather than just the notified array, to have access
to those constants.
The rest is just replacing of hardcoded 0s and 1s with
self.NOTIFIER_TERM and self.NOTIFIER_NORM, and of notifier_count with
len(self.NOTIFIERS).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Guido Trotter [Fri, 21 May 2010 10:24:54 +0000 (11:24 +0100)]
Mainloop: handle SIGINT as well (and terminate)
This is needed if daemons are in the foreground, and get ctrl+c-ed by
the user. Also add unittests to make sure the correct signals terminate
the mainloop.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Fri, 21 May 2010 11:45:56 +0000 (12:45 +0100)]
SingleFileEventHandler: update comments
The comments in the SingleFileEventHandler are still confd-specific.
Update them to make them generic for any single-file monitoring.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 20 May 2010 17:40:21 +0000 (19:40 +0200)]
Allow control of import/export compression method
For exports to/imports from the same machine, compression will
not be used anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 20 May 2010 17:17:32 +0000 (19:17 +0200)]
Put common import/export daemon options into object
The X509 key name and CA are passed from cmdlib all the way to
the backend import/export daemon. With the addition of an option
to choose the compression method, another parameter would have
to be passed all the way. By moving these options to a separate
object, adding new ones will become much easier.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 18 May 2010 16:46:40 +0000 (18:46 +0200)]
import-export daemon: Allow changing compression method
For example, exports on the same node shouldn't be compressed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Fri, 21 May 2010 11:17:57 +0000 (12:17 +0100)]
Merge branch 'devel-2.1'
* devel-2.1:
Add asyncnotifier unittests
Make ConfdInotifyEventHandler a library function
Comment on AsyncNotifier upstream's availability
Remove errors.ConfdFatalError
TestMainloop: test scheduler priorities as well
Fix race condition in mainloop unittest
TestMainloop.testReRun
Add AsyncUDPSocket tests
Add initial mainloop unittests
devel/review: make the range argument optional
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 21 May 2010 10:31:37 +0000 (11:31 +0100)]
Add asyncnotifier unittests
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 20 May 2010 10:45:35 +0000 (11:45 +0100)]
Make ConfdInotifyEventHandler a library function
Cut&Paste, plus the following changes:
- The class is renamed to SingleFileEventHandler
- The monitored filename must be passed in and doesn't default to the
ganeti cluster config file
- A small docstring is added to the class
- Pylint disables for except: and method names are added
This makes it possible to write unittests for this class.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 20 May 2010 10:33:35 +0000 (11:33 +0100)]
Comment on AsyncNotifier upstream's availability
Since we contributed AsyncNotifier back to the upstream pyinotify
project, we'll be able, one day, to remove the ganeti version of that
code. For now we still need it to support older distributions, but
having a note about when we'll be able to remove it is nice.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Thu, 20 May 2010 10:20:25 +0000 (11:20 +0100)]
Remove errors.ConfdFatalError
This exception is caught, but never thrown. It became useless when we
moved confd from on/off to enabled/disabled, but always running on all
nodes. Removing its definition and the code catching it can do no harm.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Wed, 19 May 2010 21:54:34 +0000 (22:54 +0100)]
TestMainloop: test scheduler priorities as well
By using enterabs we can schedule events at the same time, which will
then be sorted by priority.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 19 May 2010 21:52:03 +0000 (22:52 +0100)]
Fix race condition in mainloop unittest
Currently, in testDeferredCancel if the self._CancelEvent are entered
more than 0.3 seconds after the _SendSig have been entered, the test
could fail. This is unlikely but may happen. To avoid it we use
enterabs, to use absolute times.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 19 May 2010 16:22:47 +0000 (17:22 +0100)]
TestMainloop.testReRun
Currently we never do that, but it's nice to know we can restart the
mainloop, after termination.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 19 May 2010 15:39:41 +0000 (16:39 +0100)]
Add AsyncUDPSocket tests
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 19 May 2010 14:09:30 +0000 (15:09 +0100)]
Add initial mainloop unittests
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Mon, 17 May 2010 12:30:46 +0000 (13:30 +0100)]
devel/review: make the range argument optional
Rather than calling devel/review with from..to target we can just use
target, and assume target..HEAD as the range. This makes it a lot
easier, as now you just have to say something like "devel/review
devel-2.1" if you're already on the branch you want to commit, and want
to commit it to devel-2.1.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 17 May 2010 11:49:19 +0000 (13:49 +0200)]
Add tool to move instances between clusters
This is a first version of the instance move tool and it supports moving
1..N instances from one cluster to another. When moving a single instance,
the instance can be renamed, allowing for moves within the same cluster
(not really useful in practice, but good for testing). Documentation is
updated to describe this new tool and its usage.
The “move-instance” tool uses the workerpool to support parallel moves of
instances. Supporting them was simple as threads were required anyway due
to the synchronous RAPI client.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 4 May 2010 15:38:21 +0000 (17:38 +0200)]
RAPI changes for instance moves
Two new resources are added:
- /2/instances/$name/prepare-export
- /2/instances/$name/export
The documentation for the existing resource for creating instances is updated
for remote imports. The RAPI client is extended for the new resources.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 16:59:05 +0000 (18:59 +0200)]
Implement opcode changes for remote-import
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 13 May 2010 14:57:24 +0000 (16:57 +0200)]
Implement opcode changes for remote-export
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 30 Apr 2010 13:38:32 +0000 (15:38 +0200)]
Add opcode to prepare export
To prepare a remote export, the X509 key and certificate need to be generated.
A handshake value is also returned for an easier check whether both clusters
share the same cluster domain secret.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 18 May 2010 13:51:01 +0000 (14:51 +0100)]
Merge branch 'devel-2.1'
* devel-2.1:
Abstract the LUXI eom into a constant
KVM: vhost net acceleration support
Conflicts:
lib/luxi.py
- trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 18 May 2010 13:32:33 +0000 (15:32 +0200)]
Fix distcheck broken by check-cert-expired unittest
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 18 May 2010 12:40:09 +0000 (14:40 +0200)]
ganeti-cleaner: Remove expired X509 certs
Importing/exporting an instance to a remote machine creates X509
certificates which expire after some time. They need to be removed from
the nodes as they become useless.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Tue, 18 May 2010 09:37:15 +0000 (10:37 +0100)]
Abstract the LUXI eom into a constant
Currently the EOM terminator is hardcoded on the server side, and is
customizable in the Transport object (with the default being the same as
the value found in the server), but not in the luxi client.
With this patch we move the value to constants, and remove the "fake"
customizability, which would just break client/server communication. If
we ever need to have a luxi transport with a different terminator it's
easy enough to add it back.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Guido Trotter [Mon, 17 May 2010 12:05:13 +0000 (13:05 +0100)]
KVM: vhost net acceleration support
This will only work on patched or newer (>= 2.6.34) kernels and with a
patched version of qemu-kvm.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 18 May 2010 10:57:05 +0000 (12:57 +0200)]
Merge branch 'devel-2.1'
* devel-2.1:
ganeti-cleaner: Write log file with removed files
RAPI client: Handle urllib2.HTTPError and raise GanetiApiError
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 13:48:15 +0000 (15:48 +0200)]
ganeti-cleaner: Write log file with removed files
Logfiles can be useful for debugging.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 13 May 2010 14:57:38 +0000 (16:57 +0200)]
Add function to read cluster domain secret
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 12:40:55 +0000 (14:40 +0200)]
LUExportInstance: Remove instance only if export was successful
Until now, the instance was always removed (if asked for by the
user). In case of export errors however, it shouldn't.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 11:49:41 +0000 (13:49 +0200)]
RAPI client: Handle urllib2.HTTPError and raise GanetiApiError
This allows users of the RAPI client to catch GanetiApiError for all HTTP
errors.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 17 May 2010 11:00:07 +0000 (13:00 +0200)]
Merge branch 'devel-2.1'
* devel-2.1:
RAPI: /2/{nodes,instances}/$name should return 404 for unknown items
ganeti-masterd: Improve error logging for client requests
Return disk_template from LUQueryInstanceData
RAPI client: Rename Get{Node,Instance}Info, add new GetInstanceInfo
RAPI client: Log request to be made
Add missing documentation for RAPI instance creation mode
Add checks for master IP in cluster verify
Remove unused import from daemon.py
utils.IgnoreSignals
AsyncUDPSocket.handle_error
Add a forgotten comment about overriding a method
ganeti-noded: add the --no-mlock option
Describe more ganeti-noded options in the manpage
Conflicts:
daemons/ganeti-masterd: Trivial
test/ganeti.backend_unittest.py: Trivial
test/ganeti.utils_unittest.py: Trivial
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 17:08:21 +0000 (19:08 +0200)]
RAPI: /2/{nodes,instances}/$name should return 404 for unknown items
Currently they return a 500 Server Error, not really useful
for detecting nonexistent items.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 17:07:24 +0000 (19:07 +0200)]
ganeti-masterd: Improve error logging for client requests
Ganeti errors should also be logged with a backtrace.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 16:26:30 +0000 (18:26 +0200)]
Return disk_template from LUQueryInstanceData
Inter-cluster instance moves need the disk template. As they run
LUQueryInstanceData to get an instance's details, the disk template
must be returned.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 16:25:12 +0000 (18:25 +0200)]
RAPI client: Rename Get{Node,Instance}Info, add new GetInstanceInfo
GetInstanceInfo should return the resource /2/instances/$name/info,
but so far it returns /2/instances/$name. The same applies to
GetNodeInfo, which returns /2/nodes/$name. Both names are stripped
of their “Info” suffix and a new method, GetInstanceInfo, is added
for /2/instances/$name/info. There's no equivalent for nodes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 16:12:42 +0000 (18:12 +0200)]
RAPI client: Log request to be made
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 14 May 2010 10:33:01 +0000 (12:33 +0200)]
Add missing documentation for RAPI instance creation mode
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 17 May 2010 09:54:34 +0000 (11:54 +0200)]
Add checks for master IP in cluster verify
This also updates a comment in the unittest for utils.py. We unittest
the new function for two things: correct reporting on real case (for
localhost), and correct reporting with a mocked-out TcpPing that returns
false.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 15:54:25 +0000 (16:54 +0100)]
RetryOnSignal: handle socket error as well
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 16:01:52 +0000 (17:01 +0100)]
Remove unused import from daemon.py
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>
Guido Trotter [Fri, 14 May 2010 13:44:23 +0000 (14:44 +0100)]
utils.IgnoreSignals
Remove duplicate code between a couple of asyncore related function by
having a function in charge of handling EINTR errors. Unittests included.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 13:52:58 +0000 (14:52 +0100)]
AsyncUDPSocket.handle_error
By overriding the default asyncore handle_error (which closes the
socket) with our own version, which logs what happened but tries to
proceed, we can get rid of a couple of try/except blocks. The resulting
churn is deindentation of the internal code.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 13:46:36 +0000 (14:46 +0100)]
Add a forgotten comment about overriding a method
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 14 May 2010 07:45:53 +0000 (08:45 +0100)]
ganeti-noded: add the --no-mlock option
While mlock on noded is definitely good in most situations, there are
some - namely my laptop - where it has no benefit, and uses precious
non-swappable memory. To avoid this we make it optional, with a new
--no-mlock option. Note that only the main node daemon and its http
children are affected: the powercycle node child still uses mlock, which
doesn't harm, since it's a short lived process happening just before
node reboot anyway. The manpage is updated.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>
Guido Trotter [Fri, 14 May 2010 07:42:08 +0000 (08:42 +0100)]
Describe more ganeti-noded options in the manpage
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>
Luca Bigliardi [Thu, 13 May 2010 16:30:12 +0000 (17:30 +0100)]
Merge branch 'devel-2.1'
Conflicts:
daemons/ganeti-noded
lib/daemon.py
lib/rapi/baserlib.py
lib/rapi/rlib2.py
lib/utils.py
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Thu, 13 May 2010 15:49:35 +0000 (16:49 +0100)]
Fix some pylint warnings
Disable warnings for:
- except Exception,
- use of __errno_location,
- redeclaration of handleError()
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Luca Bigliardi [Thu, 13 May 2010 15:33:04 +0000 (16:33 +0100)]
Fix logging string format
Fix this pylint warning:
[W6501, Mlockall] Specify string format arguments as logging function parameters
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Luca Bigliardi [Thu, 13 May 2010 15:14:08 +0000 (16:14 +0100)]
Fix Filehandler / FileHandler typo
Fix typo spotted by pylint:
E1101:2095:LogFileHandler.handleError: Module 'logging' has no 'Filehandler' member
Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 18:06:35 +0000 (20:06 +0200)]
QA: Test instance creation/deletion via RAPI client
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 17:56:20 +0000 (19:56 +0200)]
RAPI client: Implement instance creation
Currently this only supports the new instance creation request data
format version 1, but support for the old version can be easily
implemented.
Most arguments are optional and documented in the RAPI documentation.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 12 May 2010 17:45:10 +0000 (19:45 +0200)]
RAPI: Add new request data format for instance creation
As mentioned in commit
d975f482d, the current way of creating an
instance via RAPI is not very flexible. With this patch, a new
instance creation request data format is introduced and documented.
Support can be detected by checking the list of features returned
by the /2/features resource for the value "instance-create-reqv1".
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>