Iustin Pop [Fri, 9 Jan 2009 12:52:28 +0000 (12:52 +0000)]
Silence warning on node list for offline nodes
The warning in node list is meant for nodes that return wrong
information, but for offline nodes this case is normal.
Reviewed-by: imsnah
Iustin Pop [Fri, 9 Jan 2009 12:52:17 +0000 (12:52 +0000)]
Rework the daemonization sequence
The current fork+close fds sequence has deficiencies which are hard to
work around:
- logging can start logging before we fork (e.g. if we need to emit
messages related to master checking), and thus use FDs which we
can't track nicely
- the queue locks the queue file, and again this fd needs to be kept
open which is hard from the main loop (and this error is currently
hidden by the fact that we don't log it)
Given the above, it's much simpler, in case we will fork later, to close
file descriptors right at the beginning of the program, and in Daemonize
only close/reopen the stdin/out/err fds.
In addition, we also close() the handlers we remove in SetupLogging so
that the cleanup is more thorough.
Reviewed-by: imsnah
Iustin Pop [Fri, 9 Jan 2009 12:22:51 +0000 (12:22 +0000)]
Cleanup replace-disks modes and options
In 1.2, due to the md+drbd7 legacy, we had a complex choice of replace
modes, and the new drbd8 modes where forced into this syntax, with some
complicated rules of transition from one mode to another (if REPLACE_ALL
but not new node passed, switch to REPLACE_SEC, etc.).
This patch cleans this situation by making a clear separation between
the two main modes:
- replace on current nodes (with the two sub-cases on primary and on
secondary)
- change to a new node (either via manually specified node or via
iallocator)
Reviewed-by: imsnah
Iustin Pop [Thu, 8 Jan 2009 16:39:47 +0000 (16:39 +0000)]
Fix cluster verify/node net test for offline nodes
For offline nodes, we shouldn't add them to the NV_NODELIST and
NV_NODENETTEST tests since they most likely won't succeed.
The patch makes gnt-cluster verify happy again in such cases.
Reviewed-by: imsnah
Iustin Pop [Thu, 8 Jan 2009 16:05:30 +0000 (16:05 +0000)]
rpc: Add a method for easy check of remote results
The patch adds a new method to the rpc.RpcResult class called
"RemoteFailMsg" which is useful for the RPC calls which return a
(status, payload) style result.
Reviewed-by: imsnah
Iustin Pop [Thu, 8 Jan 2009 14:16:44 +0000 (14:16 +0000)]
Add an instance_migratable rpc call
This is a forward-port of commit 1194 on the 1.2 branch:
This call will check whether an instance is up on its primary, and that
it has been started with symlinks. We currently have no on-secondary
checks, nor any hypervisor specific call.
Reviewed-by: iustinp
The difference from the original patch is that we don't include the
cmdlib changes, since those will come as a copy from the 1.2 cmdlib.py,
and not as individual patches.
Original-Author: ultrotter
Iustin Pop [Thu, 8 Jan 2009 12:03:31 +0000 (12:03 +0000)]
bdev: forward-port ReAttachNet/DisconnectNet
This is plain copy of the 1.2 ReAttachNet and DisconnectNet methods on
the DRBD8 device, with the logger to logging module changes and the
ReAttachNet method renamed to AttachNet.
These methods are not used anywhere right now, but will be used for
migration and a simpler disk-replace.
The code was originally committed on the 1.2 branch as revision numbers
1165 and 1204.
Originally-Reviewed-by: imsnah, ultrotter
Iustin Pop [Wed, 7 Jan 2009 17:02:37 +0000 (17:02 +0000)]
backend: Remove symlinks by disk name
This is a modified forward-port of commit 1184 on the 1.2 branch:
backend: Remove symlinks by disk name, not using a wildcard
Reviewed-by: ultrotter
The changes to the original patch are related to the docstring style and
iv_name to index switch.
Original-Author: imsnah
Iustin Pop [Wed, 7 Jan 2009 17:02:26 +0000 (17:02 +0000)]
Pass instance name to rpc call blockdev_close
This is an extract of commit 1166 on the 1.2 branch (Add a rpc call for
drbd network reconfiguration), but only the blockdev_close part.
The patch changes the blockdev_close call to take the instance so that
it can remove the symlinks of the instance.
Originally-Reviewed-by: imsnah
Iustin Pop [Wed, 7 Jan 2009 17:02:12 +0000 (17:02 +0000)]
Fix the _RemoveBlockDevLinks() function
This is a forward-port of commit 1163 on the 1.2 branch:
This fixes the removal of the instance symlinks (probably breakage from
the glob changes).
Reviewed-by: imsnah
Iustin Pop [Wed, 7 Jan 2009 17:01:58 +0000 (17:01 +0000)]
Remove instance's symlinks
This is a forward-port of commits 1150 and 1151 on the 1.2 branch:
Add _RemoveBlockDevLinks auxiliary function, called when an instance
fails to start and when it is shut down.
Reviewed-by: iustinp
and:
Fix cut&paste error when removing symlinks
It's just whitespace... isn't it? uhm... :) Anyway, fixing an error made
when reformatting the code for the new "safer" behaviour.
Reviewed-by: iustinp
Original-Author: ultrotter
Iustin Pop [Wed, 7 Jan 2009 17:01:46 +0000 (17:01 +0000)]
Catch BlockDeviceError when starting instance
This is a forward-port of commit 1149 on the 1.2 branch:
_GatherAndLinkBlockDevs used to raise the errors.BlockDeviceError
exception when it failed to create a block device, and with this patch
set it does so also when it fails to create a symlink to it.
With this patch we move the call to this function into a pre-existing
try-except block in the code, and catch the BlockDeviceError exception,
logging a message and returning a failure state if it happens.
Reviewed-by: iustinp
The changes are related to the new hypervisor and logging syntax.
Original-Author: ultrotter
Iustin Pop [Wed, 7 Jan 2009 17:01:36 +0000 (17:01 +0000)]
Create symlinks to intances' block devices
This is a forward-port of commit 1148 on the 1.2 branch:
Change the _GatherBlockDevs private function, called only one time by
StartInstance, to _GatherAndLinkBlockDevs, and make it transform the
device returned even more by calling the new _SimlinkBlockDev auxiliary
function.
This makes sure that every time an instance is started symlinks to its
block devices are created, and the instance is started off them, rather
than the underlying block devices.
Reviewed-by: iustinp
The changes we make to the patch is related to newer function signatures
in 2.0, and to the fact that iv_name is deprecated and we use instead
disk%d based on the disk index.
Original-Author: ultrotter
Iustin Pop [Wed, 7 Jan 2009 17:01:25 +0000 (17:01 +0000)]
Simplify hypervisor block_devices structure
This is a partial forward-port of commit 1136 on the 1.2 branch:
The hypervisor doesn't need to be passed the whole block device
structure, so we'll just give it the block device name on the local
node, and the name as seen by the instance. This will make it easier to
manipulate it later without messing with the block devices (eg. by
changing the system name to a symlink to the name itself).
Since the HVM hypervisor changes the "virtual" name a note is added
calling for a redesign that doesn't need this change, as different
hypervisors and emulation types will anyway have different names for
exported devices.
Reviewed-by: iustinp
The changes in this patch compared to the original are:
- we keep passing the original disk object, not for its iv_name, but
for it's physical_id which is needed by the file driver (this could
be fixed maybe)
- we don't use the iv_name anymore, since in 2.0 we already use the
index of the device
Original-Author: ultrotter
Iustin Pop [Wed, 7 Jan 2009 14:38:29 +0000 (14:38 +0000)]
_AssembleInstanceDisks: fix rpcresult handling
Commit 2117 changed _AssembleInstanceDisks to correctly parse the
failure status of the new RpcResult structure, but it didn't fix the
storing of only the result payload. Since RpcResult is not JSON
serializable, LUActivateInstanceDisks is failing.
Reviewed-by: ultrotter
Iustin Pop [Tue, 6 Jan 2009 09:57:53 +0000 (09:57 +0000)]
Fix some pylint-detected issues
Two bad indentation cases and a missing variable.
Reviewed-by: imsnah
Michael Hanselmann [Fri, 19 Dec 2008 19:31:17 +0000 (19:31 +0000)]
ganeti.bootstrap: Set permissions on newly uploaded files
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 19:31:04 +0000 (19:31 +0000)]
ganeti.cmdlib: Check remote API certificate on "gnt-cluster verify"
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 19:30:46 +0000 (19:30 +0000)]
ganeti.bootstrap: Upload remote API certificate to new nodes
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 19:30:31 +0000 (19:30 +0000)]
ganeti.bootstrap: Prepare for remote API certificate
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 19:30:17 +0000 (19:30 +0000)]
ganeti.bootstrap: Write SSL key to temporary file and set permissions
Previously, we set the permissions only after writing the key. This
gave other users on the system a small window during which they could
read the key.
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 19:30:05 +0000 (19:30 +0000)]
ganeti.bootstrap: Generate SSL certificate for remote API
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 19:29:50 +0000 (19:29 +0000)]
ganeti.bootstrap: Move SSL certificate generation into separate function
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 12:58:27 +0000 (12:58 +0000)]
ganeti-rapi: Implement HTTP authentication
Passwords are stored in "$localstatedir/lib/ganeti/rapi_users". User
options specify the access permissions of a user (see docstring for
ganeti.http.ReadPasswordFile), for which only "write" is supported
to grant write access. Every other user has read-only access.
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 12:58:10 +0000 (12:58 +0000)]
ganeti-rapi: Introduce per-request context
This will be used to evaluate access permissions to resources.
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 12:57:58 +0000 (12:57 +0000)]
ganeti.http: Function to read password file
Lines in the password file are of the following format:
<username> <password> [options]
Fields are separated by whitespace. Username and password are
mandatory, options are optional and separated by comma (",").
Empty lines and comments ("#") are ignored.
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 12:57:38 +0000 (12:57 +0000)]
ganeti.http: Add support for private data in HTTP requests
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 12:57:22 +0000 (12:57 +0000)]
ganeti.http: Add support for basic HTTP authentication
As per RFC2617.
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 19 Dec 2008 12:57:07 +0000 (12:57 +0000)]
ganeti.http: Prepare authentication for HTTP server
The authentication class will override PreHandleRequest.
Reviewed-by: amishchenko
Michael Hanselmann [Thu, 18 Dec 2008 16:39:04 +0000 (16:39 +0000)]
Job queue: Allow more than one file rename per RPC call
Reviewed-by: ultrotter
Michael Hanselmann [Thu, 18 Dec 2008 16:38:47 +0000 (16:38 +0000)]
ganeti.jqueue: Group job archivals to reduce number of RPC calls
Reducing the actual number of RPC calls will come in another patch.
Reviewed-by: ultrotter
Michael Hanselmann [Thu, 18 Dec 2008 16:38:32 +0000 (16:38 +0000)]
Prevent RPC timeout on auto-archiving jobs
With a large job queue, auto-archiving jobs can take a very long time,
causing timeouts on the luxi RPC layer. With this change, auto-
archive returns after half of the RPC timeout has passed. The user
will see how many jobs are left unchecked.
Reviewed-by: ultrotter
Michael Hanselmann [Thu, 18 Dec 2008 16:38:09 +0000 (16:38 +0000)]
jqueue: When auto-archiving jobs, calculate job status only once
This is done by passing the job object to _ArchiveJobUnlocked instead
of only the job ID. Also return whether job was actually archived.
Reviewed-by: ultrotter
Michael Hanselmann [Thu, 18 Dec 2008 16:23:26 +0000 (16:23 +0000)]
Use subdirectories for job queue archive
As it turned out, having many files in a single directory can be
very painful. With this patch, only 10'000 files are stored in a
directory for the job queue archive. With 10'000 directries, this
allows for up to 100 million jobs be archived without having large
numbers of files in a single directories. Not that it is realistic,
anyway.
Reviewed-by: ultrotter
Michael Hanselmann [Thu, 18 Dec 2008 16:23:05 +0000 (16:23 +0000)]
Add rename function automatically creating directories if needed
Unfortunately, os.makedirs in Python 2.4 is not safe against multiple
processes creating the same directory tree at the same time. This is
only fixed in Python 2.5 and up. Adding more checks in our code doesn't
make it any better.
Reviewed-by: iustinp
Michael Hanselmann [Thu, 18 Dec 2008 16:21:13 +0000 (16:21 +0000)]
ganeti.http: Don't pass poller object around
They're cheap to instantiate and doing this changes makes the code
a bit simpler.
Reviewed-by: ultrotter
Michael Hanselmann [Thu, 18 Dec 2008 13:45:41 +0000 (13:45 +0000)]
Rename http.HttpInternalError to HttpInternalServerError
All other exceptions are named after the error name in RFC2616 (HTTP/1.1).
Reviewed-by: amishchenko
Michael Hanselmann [Thu, 18 Dec 2008 13:45:24 +0000 (13:45 +0000)]
ganeti.http: Add more constants and errors
Reviewed-by: amishchenko
Michael Hanselmann [Thu, 18 Dec 2008 13:45:10 +0000 (13:45 +0000)]
ganeti.http: Ignore ENOTCONN when shutting down the connection
Reviewed-by: amishchenko
Michael Hanselmann [Thu, 18 Dec 2008 13:44:53 +0000 (13:44 +0000)]
Implement support for additional headers with HTTP errors
Reviewed-by: amishchenko
Michael Hanselmann [Wed, 17 Dec 2008 14:30:58 +0000 (14:30 +0000)]
Add simple unittests for ganeti.http
More complex unittests will need some refactoring in the HTTP code.
Reviewed-by: amishchenko
Michael Hanselmann [Wed, 17 Dec 2008 14:09:39 +0000 (14:09 +0000)]
ganeti.bootstrap: Whitespace fix
Reviewed-by: iustinp
Michael Hanselmann [Wed, 17 Dec 2008 13:18:35 +0000 (13:18 +0000)]
Add job queue size limit
A job queue with too many jobs can increase memory usage and/or make
the master daemon slow. The current limit is just an arbitrary number.
A "soft" limit for automatic job archival is prepared.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 17 Dec 2008 11:24:12 +0000 (11:24 +0000)]
utils.KillProcess: Use waitpid() to wait for child processes
Sometimes the proc filesystem doesn't reflect the current status of
a process. By calling waitpid(), we make sure to get the current
information, at least for child processes. The timeout is still
kept for child processes to make sure the proc filesystem is updated.
Reviewed-by: iustinp
Guido Trotter [Tue, 16 Dec 2008 16:24:46 +0000 (16:24 +0000)]
Release ganeti 2.0~alpha1
Reviewed-by: iustinp
Guido Trotter [Tue, 16 Dec 2008 16:24:35 +0000 (16:24 +0000)]
LUConnectConsole: fix primary_node online check
The primary node is part of the instance, not of the opcode.
Reviewed-by: iustinp
Guido Trotter [Tue, 16 Dec 2008 16:24:22 +0000 (16:24 +0000)]
_RunCmdPipe: handle EINTR in poller.poll()
poll() can be interrupted. rather than failing we retry until it
returns.
Reviewed-by: iustinp
Guido Trotter [Tue, 16 Dec 2008 16:24:08 +0000 (16:24 +0000)]
KVM: improve socat interface
Call socat with a full path specified at configure time, rather than
just by its name, and check for the binary to exist at hypervisor
verify.
Reviewed-by: iustinp
Guido Trotter [Tue, 16 Dec 2008 16:23:53 +0000 (16:23 +0000)]
KVM: use a different default kernel path
It makes sense for the default kvm kernel not to be called "xenU".
Reviewed-by: iustinp
Michael Hanselmann [Mon, 15 Dec 2008 10:06:24 +0000 (10:06 +0000)]
ganeti.http: Add three TODOs for improvements
Reviewed-by: iustinp
Michael Hanselmann [Mon, 15 Dec 2008 09:48:25 +0000 (09:48 +0000)]
ganeti.http: Explicitly initiate handshake
Otherwise it would be done on the first read/write operation, making
error handling more difficult (such as EOF during handshake).
Reviewed-by: iustinp
Michael Hanselmann [Mon, 15 Dec 2008 09:40:57 +0000 (09:40 +0000)]
ganeti.http: Implement handshake socket operation
Reviewed-by: iustinp
Michael Hanselmann [Mon, 15 Dec 2008 09:40:42 +0000 (09:40 +0000)]
ganeti.http: Handle SSL_ERROR_ZERO_RETURN
Also add a comment next to the place where the SSL connection is shut
down.
Reviewed-by: iustinp
Iustin Pop [Sun, 14 Dec 2008 12:05:30 +0000 (12:05 +0000)]
cleanup: ConfigWriter, initialize all attributes
We should initialized the _last_cluster_serial in the constructor too (just to
be consistent).
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:05:22 +0000 (12:05 +0000)]
cleanup: rapi v2 instance tags wrong attribute
This was changed in the past, but it seems this class was forgotten.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:05:12 +0000 (12:05 +0000)]
cleanup: http server, line too long
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:05:03 +0000 (12:05 +0000)]
cleanup: http client, line too long
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:04:54 +0000 (12:04 +0000)]
cleanup: xen hypervisor
Wrong indentation and uniformize one method signature.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:04:45 +0000 (12:04 +0000)]
cleanup: kvm code likes to redefine names
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:04:36 +0000 (12:04 +0000)]
lib/ssh.py: import the logging module
This only means most of our error paths in this module were not working
(and generating exceptions).
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:04:29 +0000 (12:04 +0000)]
SshRunner: add docstring for _BuildSshOptions
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:04:20 +0000 (12:04 +0000)]
Improve _autoconf.py comments
This adds a docstring to the _autoconf.py file detailing how it's
generated (the other comment it's not visible in pydoc/epydoc).
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:04:13 +0000 (12:04 +0000)]
cleanup: use _ for unused loop counter
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:04:05 +0000 (12:04 +0000)]
cleanup: WorkerPool, wrong variable name
Quoting Michael: "why is this even working?"
Reviewed-by: imsnah,amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:03:56 +0000 (12:03 +0000)]
cleanup: TcpPing, wrong variable name
The default value of 'False' wasn't initialized properly. It doesn't
require initialization, but it's cleaner this way.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:03:47 +0000 (12:03 +0000)]
cleanup: SetEtcHostsEntry unused var
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:03:38 +0000 (12:03 +0000)]
cleanup: fix IAllocator hypervisor usage
Two problems: the iallocator.hypervisor wasn't initialized to None in
the constructor, so pylint doesn't realize it's initialized later with
setattr.
Second, 'hypervisor' is a module, so we shouldn't use it as a variable.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:03:30 +0000 (12:03 +0000)]
cleanup: LUReplaceDisks unused vars
And a small whitespace fix.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:03:20 +0000 (12:03 +0000)]
cleanup: do not hide upper-scope name
hypervisor is a module, so we shouldn't use it as an argument.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:03:11 +0000 (12:03 +0000)]
cleanup: fix use of _CheckNodeOnline
A few cases of wrong variable name.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:03:03 +0000 (12:03 +0000)]
cleanup: LUAddNode, LUSetNodeParams unused variable
This is a leftover from the abstraction of AdjustCandidatePool, and it
also requires the config lock, so it's better to remove it.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:02:53 +0000 (12:02 +0000)]
cleanup: LURenameCluster wrong variable name
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:02:45 +0000 (12:02 +0000)]
cleanup: fix export NIC count the same way as disk
For safety, we use the same algorithm as in disk count.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:02:36 +0000 (12:02 +0000)]
cleanup: fix backend._RecursiveFindBD
_RecursiveFindBD takes a parameter that isn't used; moreover, nowhere in
the SVN history can I find a case that it has been used.
As such, remove this parameter and fix its callers.
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:02:27 +0000 (12:02 +0000)]
cleanup: more unused vars
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:02:18 +0000 (12:02 +0000)]
cleanup: sanitize a default parameter
Instead of relying that the usage of the parameter is ok with mutable
default parameters, let's just make it safer..
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:02:09 +0000 (12:02 +0000)]
cleanup: exceptions should derive from Exception
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:02:01 +0000 (12:02 +0000)]
cleanup: fix GatherMasterVotes
Remove unused vars
Reviewed-by: amishchenko
Iustin Pop [Sun, 14 Dec 2008 12:01:52 +0000 (12:01 +0000)]
cleanup: _InitSSHSetup doesn't need its argument
Reviewed-by: imsnah
Iustin Pop [Sun, 14 Dec 2008 12:01:41 +0000 (12:01 +0000)]
cleanup: fix 'variable unused' warning
In the iteration we don't care about the node names, so we change the
for loop to be over the values (and not itervalues).
Reviewed-by: amishchenko
Michael Hanselmann [Fri, 12 Dec 2008 16:50:41 +0000 (16:50 +0000)]
ganeti.http: Rename HttpBase._using_ssl to HttpBase.using_ssl
It'll be queried from other classes.
Reviewed-by: iustinp
Michael Hanselmann [Fri, 12 Dec 2008 16:50:27 +0000 (16:50 +0000)]
ganeti.http: Rename HttpSocketBase to HttpBase
It's more appropriate.
Reviewed-by: iustinp
Iustin Pop [Thu, 11 Dec 2008 17:13:30 +0000 (17:13 +0000)]
Fix epydoc format warnings
This patch should fix all outstanding epydoc parsing errors; as such, we
switch epydoc into verbose mode so that any new errors will be visible.
Reviewed-by: imsnah
Iustin Pop [Thu, 11 Dec 2008 14:58:01 +0000 (14:58 +0000)]
Switch epydoc to parse only
epydoc seems to be mightily confused by decorators and how they change
functions (it starts mixing the parameters of the decorated function
into the decorator itself); so we want it to parse only and not look at
the objects themselves.
Reviewed-by: ultrotter
Michael Hanselmann [Wed, 10 Dec 2008 12:11:47 +0000 (12:11 +0000)]
ganeti.backend: Improve compression check
Reviewed-by: iustinp
Michael Hanselmann [Wed, 10 Dec 2008 12:06:35 +0000 (12:06 +0000)]
ganeti.http: Docstring updates
Reviewed-by: iustinp
Michael Hanselmann [Tue, 9 Dec 2008 18:42:53 +0000 (18:42 +0000)]
ganeti.http: Remove _HttpClientError
This is a leftover from old code.
Reviewed-by: iustinp
Michael Hanselmann [Tue, 9 Dec 2008 17:35:53 +0000 (17:35 +0000)]
ganeti.http.server: Increase connection backlog to 1024
This solves a problem with many concurrent requests. By default, 1024
is the maximum backlog on Linux kernels. We limit the number of clients
through MAX_CHILDREN, too. The idea of just increasing the backlog is
taken from lighttpd.
Reviewed-by: amishchenko
Michael Hanselmann [Tue, 9 Dec 2008 13:24:26 +0000 (13:24 +0000)]
RPC: Compress file upload data
Adding compression to larger amounts of data is more efficient than
transferring it (len(nodes) - 1) times over the network without
compression. We were able to compress a 800KB config file to about
30 KB, which is about 40 KB with Base64 encoding (required due to
the way SimpleJson handles strings).
Reviewed-by: ultrotter
Iustin Pop [Tue, 9 Dec 2008 09:33:32 +0000 (09:33 +0000)]
Warn for instances living on offline nodes
The patch also changes the result to error for non-reachable secondary nodes
(as for primary nodes).
Reviewed-by: ultrotter
Iustin Pop [Mon, 8 Dec 2008 17:45:56 +0000 (17:45 +0000)]
Fix _AdjustCandidatePool
Currently the ConfigWriter.MaintainCandidatePool returns node names, and
_AdjustCandidatePool uses them as such, but then it passes these to
context.ReaddNode which in turn passes them to jqueue.JobQueue.AddNode which
uses them as objects.Node instances.
Since this is currently the only usage, we change return type from
ConfigWriter.MaintainCandidatePool to be objects and adjust the logging of
their names, so that the auto-adjusement works.
Reviewed-by: ultrotter
Iustin Pop [Mon, 8 Dec 2008 11:46:51 +0000 (11:46 +0000)]
gnt-node modify: add the offline attribute
This patch changes gnt-node modify and the associated opcode/lu to allow
modification of the node offline attribute.
Setting a node into offline mode automatically demotes it from the
master role.
Reviewed-by: ultrotter
Iustin Pop [Mon, 8 Dec 2008 09:10:48 +0000 (09:10 +0000)]
RPC: do not make calls to offline nodes
This patch changes the _MultNodeCall and _SingleNodeCall helpers to not
actually make calls to offline nodes, but instead generate fake
responses which have a parameter caller 'offline' set so that callers
can check for this value if they want (otherwise, it's just a failed RPC
call).
Reviewed-by: ultrotter
Guido Trotter [Sun, 7 Dec 2008 11:01:55 +0000 (11:01 +0000)]
chmod ganeti.initd before uploading it
When an upload is done to a node which doesn't have any version of
ganeti installed, this prevents a non-executable-initd error later in
the upload.
Reviewed-by: imsnah
Iustin Pop [Fri, 5 Dec 2008 11:41:33 +0000 (11:41 +0000)]
Make cluster verify understand offline nodes
This patch changes cluster verify to not alert on offline nodes, but
instead just show a note at the end with the number of such nodes.
It also removes warnings in verify-disks and hooks about failures to
make rpc calls to such nodes.
Reviewed-by: ultrotter
Iustin Pop [Fri, 5 Dec 2008 11:32:02 +0000 (11:32 +0000)]
cmdlib: check node stats in prereqs
This patch adds checks for offline nodes in most instance LUs so that we
can work with offline secondaries, but not with offline primaries. Some
cases (like grow disk, which needs both sides up) are not allowing
offline nodes at all.
Reviewed-by: ultrotter
Iustin Pop [Fri, 5 Dec 2008 11:20:20 +0000 (11:20 +0000)]
Add two utility functions to cmdlib
These will be used for parameter checking and node status checking.
Reviewed-by: ultrotter
Iustin Pop [Fri, 5 Dec 2008 11:14:19 +0000 (11:14 +0000)]
Add function to compute the master candidates
Since some nodes can be offline, we can't just take the length of the
node list as the maximum possible number of master candidates.
The patch adds an utility function to correctly compute this value and
replaces hardcoded computations with the use of this function. It then
adds utility functions to automate the maintenance of the node lists.
Reviewed-by: ultrotter
Iustin Pop [Fri, 5 Dec 2008 10:12:58 +0000 (10:12 +0000)]
http: use slicing instead of string modification
The combination of the current buffer splitting method and (4KB) buffer
size is very inefficient when writing big amounts of data. Just walking
over a 16 megabyte string using a 4K buffer takes (on a random computer)
1m06s, whereas using slices will decrease this to 0.080s, and slicing
with 32 KB size decreases this to 0.073s.
This means that uploading a big config file (it nears 1MB for big
clusters) will take more and more time per the number of nodes, since it
needs lots of slicing.
I happened upon this by accidentally setting all nodes as master
candidates, at which point just uploading the config file to all nodes
took 40s. Applying the patch decreases this to 15s (this probably can
still be optimized).
The patch also removes a duplicate constant (the one actually used is in
http/client.py), and changes the receive buffer size to use the same
constant.
Reviewed-by: imsnah
Iustin Pop [Fri, 5 Dec 2008 10:12:45 +0000 (10:12 +0000)]
Add the offline node list to ssconf
The patch also changes the various node list generation to be more
consistent.
Reviewed-by: imsnah