Michael Hanselmann [Fri, 4 Jul 2008 15:34:46 +0000 (15:34 +0000)]
Add generic worker pool implementation
Reviewed-by: ultrotter
Iustin Pop [Thu, 3 Jul 2008 12:06:46 +0000 (12:06 +0000)]
Reuse the luxi client in cli.SubmitOpCode
By a mistake, we don't reuse the luxi client. As such, we open and close
the connection at each poll cycle and spam the server logs.
Reviewed-by: ultrotter
Iustin Pop [Thu, 3 Jul 2008 12:06:35 +0000 (12:06 +0000)]
Add custom logging setup for daemons
It's better for daemons if:
- they log only to one log file
- the log level is included
- for debug runs, the filename/line number is included
This patch moves the custom formatter from the watcher to the logging
module and generalizes it; then it changes the master daemon to use this
function instead of the generic logging (which might be deprecated
anyway in the future).
Reviewed-by: imsnah
Iustin Pop [Thu, 3 Jul 2008 12:06:20 +0000 (12:06 +0000)]
Remove custom locking code from gnt-instance
The gnt-instance script doesn't run in the same process anymore, so we
can't and don't have to unlock.
Reviewed-by: ultrotter
Michael Hanselmann [Wed, 2 Jul 2008 11:58:39 +0000 (11:58 +0000)]
ganeti-masterd: Remove unused locking code
Reviewed-by: iustinp, ultrotter
Michael Hanselmann [Wed, 2 Jul 2008 11:58:20 +0000 (11:58 +0000)]
ganeti-masterd: Use logging module
Reviewed-by: ultrotter, iustinp
Guido Trotter [Tue, 1 Jul 2008 12:28:59 +0000 (12:28 +0000)]
Context: s/GLM/glm/
Make the GanetiLockManager instance of GanetiContext lowercase
Reviewed-by: imsnah
Michael Hanselmann [Tue, 1 Jul 2008 12:13:32 +0000 (12:13 +0000)]
Set locale when using docbook programs
At least docbook2man inserts a date formatted using the current
locale into its output.
Reviewed-by: iustinp
Iustin Pop [Tue, 1 Jul 2008 11:55:24 +0000 (11:55 +0000)]
Update .gitignore
Reviwed-by: imsnah
Iustin Pop [Tue, 1 Jul 2008 11:44:37 +0000 (11:44 +0000)]
Add a FirstFree function to utils.py
This function will return the first unused integer based on a list of
used integers (e.g. [0, 1, 3] will return 2).
Reviewed-by: imsnah
Guido Trotter [Tue, 1 Jul 2008 10:43:49 +0000 (10:43 +0000)]
Increase the thread size to 5
Now that we use the locking library to make sure running opcodes cannot
step on each other toes we can have a bigger thread size, and
potentially process many opcodes in a parallel manner.
Reviewed-by: iustinp
Guido Trotter [Tue, 1 Jul 2008 10:43:42 +0000 (10:43 +0000)]
Processor: acquire the BGL for LUs requiring it
If a LU required the BGL (all LUs do, right now, by default) we'll
acquire it in the Processor before starting them. For LUs that don't
we'll still acquire it, but in a shared fashion, so that they cannot run
together with LUs that do.
We'll also note down whether we own the BGL exclusively, and if we don't
and we try to chain a LU that does, we'll fail.
More work will need to be done, of course, to convert LUs not to require
the BGL, but this basic infrastructure should guarantee the coexistance
of the old and new world for the time being.
Reviewed-by: iustinp
Guido Trotter [Tue, 1 Jul 2008 10:43:32 +0000 (10:43 +0000)]
Processor: pass context in and use it.
The processor used to create a new ConfigWriter when it was initialized.
We now have one in the context, so we'll just recycle it. First of all
we'll pass the context in when creating a new Processor object, then
we'll just use context.cfg, which is granted to be initialized, wherever
we used self.cfg, and stop checking whether the config is already
initialized or not.
In the future the Processor will be able to use the context also to
acquire the BGL for LUs that require it, and to push the context down to
LUs that don't in order for them to manage their own locking.
Reviewed-by: iustinp
Guido Trotter [Tue, 1 Jul 2008 10:43:21 +0000 (10:43 +0000)]
Add REQ_BGL LogicalUnit run requirement
When logical units have REQ_BGL set (it is currently the default) they
need to be the only ganeti operation run on the cluster, and we'll
guarantee it at the master daemon level. Currently only one thread is
running at a time, so this requirement is never broken.
Reviewed-by: iustinp
Guido Trotter [Tue, 1 Jul 2008 10:43:11 +0000 (10:43 +0000)]
Burnin doesn't need a Processor
In 2.0 burnin submits job to the master daemon, so it doesn't need to
create an internal Processor anymore. Even if the processor is not used
anywhere in the burnin code it was still initialized as a leftover of
how burnin used to work. Fixing this.
Reviewed-by: iustinp
Iustin Pop [Tue, 1 Jul 2008 09:48:56 +0000 (09:48 +0000)]
Implement “gnt-job list -o +...”
This adds the same “-o +...” functionality in gnt-job as in the node and
instance scripts.
Reviewed-by: imsnah
Guido Trotter [Mon, 30 Jun 2008 16:11:48 +0000 (16:11 +0000)]
Fix sstore handling in Processor
- no need to keep the sstore as an object member, remove it
- don't reinitialize sstore only if self.cfg is None
This is not an issue, as the Processor is recycled for every opcode,
but in general we know that (a) we might need a different type of
sstore for different opcodes and (b) initializating them is cheap
- recreate sstore when chaining opcodes
Without this fix chaining an opcode which requires a writable sstore
to one which doesn't would fail. This doesn't happen today, but it's
better to fix it anyway
These changes are possible because nowadays all opcodes already require
a working cluster/configuration.
Reviewed-by: iustinp
Guido Trotter [Mon, 30 Jun 2008 16:11:01 +0000 (16:11 +0000)]
Remove duplicate code in hooks unittests
All the tests there used to creare a cfg, a sstore, an opcode and a LU.
Put all the duplicate code in the setUp function.
Reviewed-by: iustinp
Guido Trotter [Mon, 30 Jun 2008 12:37:48 +0000 (12:37 +0000)]
ganeti-masterd: init and distribute common context
This patch creates a new GanetiContext class, which is used to hold
context common to all ganeti worker threads. As for the
GanetiLockingManager class it is paramount that there is only one such
class throughout the execution of Ganeti, so the class checks for that,
and also forbids its own modification after it's been initialized. The
context for now contains a ConfigWriter and a GanetiLockingManager and
is created by the daemon and propagated to PoolWorker(s) and
JobRunner(s).
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:28:50 +0000 (14:28 +0000)]
AddNode: move the initial setup to boostrap
From the master node we can't start ssh and connect to the remote node,
nor we can do it from ganeti-noded as this ssh section will possibly ask
for key confirmation and password. So the code to copy the ganeti-noded
password and SSL key has been moved to bootstrap.py, and it's called by
gnt-node before the AddNode opcode.
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:28:37 +0000 (14:28 +0000)]
AddNode: Check for node existance
In the "new world" we'll need to setup ganeti-noded via ssh on the node
before calling the AddNode opcode. Before doing it we'll check that the
node is not already in the cluster, if --readd was not passed. This
guarantees we're not going to restart ganeti-noded on a running node.
This patch also incidentally fixes a non-style-guide conformant
docstring.
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:28:27 +0000 (14:28 +0000)]
LUAddNode: use node-verify to check node hostname
As we can't use ssh.VerifyNodeHostname directly, we'll set up a mini
node-verify to do checking between the master and the new node. In the
future networking checks, or more nodes, can be added as well.
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:28:17 +0000 (14:28 +0000)]
LUAddNode: use self.sstore, not a local ss
Since we're inside a LU we have access to self.sstore.
No need to use ss, which separate instantiation will disappear in a few
patches! ;)
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:28:06 +0000 (14:28 +0000)]
LUAddNode: upload files via rpc, not scp
We used to scp all the ssconf files, and the vnc password file to the
new node. With this patch we use the upload_file rpc, specifying just
the new node as a destination. All the files previously copied by scp
are already allowed by the backend.
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:27:56 +0000 (14:27 +0000)]
Allow VNC_PASSWORD_FILE to be rpc-uploaded
What could possibly go wrong?
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:27:46 +0000 (14:27 +0000)]
Change fping to TcpPing in two LUs
Two LUs are using RunCmd to call fping, in order to check for an IP
presence on the network. Substituting it with TcpPing will get rid of
it, which makes it not break in the new world order, where the master
cannot fork.
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:27:37 +0000 (14:27 +0000)]
raise QuitGanetiException in LeaveCluster
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:27:27 +0000 (14:27 +0000)]
ganeti-noded: Fix handling of QuitGanetiException
- s/GanetiQuitException/QuitGanetiException/
- Look for the arguments in err.args, not err itself
Reviewed-by: iustinp
Guido Trotter [Fri, 27 Jun 2008 14:27:18 +0000 (14:27 +0000)]
Simplify QuitGanetiException instantiation
Rather than packing all the arguments in a tuple, let's pass them
plainly. The superclass won't complain.
Reviewed-by: iustinp
Michael Hanselmann [Fri, 27 Jun 2008 09:02:22 +0000 (09:02 +0000)]
logger: Set formatter for stderr
Having a timestamp on log messages is very useful. The default
format string doesn't include a timestamp.
Reviewed-by: ultrotter
Guido Trotter [Thu, 26 Jun 2008 14:42:44 +0000 (14:42 +0000)]
When removing a node don't ssh to it
Even in 1.2 this behaviour is broken, as the rpc call will remove the
ssh keys before we get a chance to log in. Now the rpc takes care of
shutting down the node daemon as well, so we definitely can avoid this.
This makes the LURemoveNode operation work again with the threaded
master daemon.
Reviewed-by: iustinp
Guido Trotter [Thu, 26 Jun 2008 14:42:24 +0000 (14:42 +0000)]
ganeti-noded: quit on QuitGanetiException
Accoring to the usage documented in the QuitGanetiException docstring,
if we receive such an exception we'll set the global _EXIT_GANETI_NODED
variable to True, and then return either a valid value or an error
message to the user. This will be the last request we serve, though,
because the main loop will be interrupted and the daemon will terminate.
Reviewed-by: iustinp
Guido Trotter [Thu, 26 Jun 2008 14:42:14 +0000 (14:42 +0000)]
Add errors.QuitGanetiException
This exception does not signal an error but serves the purpose of making
the ganeti daemon shut down after handling a request. Currently it will
be used by ganeti-noded but in the future ganeti-masterd might make use
of it as well. Its usage is documented in the docstring.
Reviewed-by: iustinp
Guido Trotter [Thu, 26 Jun 2008 14:42:05 +0000 (14:42 +0000)]
ganeti-noded: serve not quite forever
Rather than calling httpd.serve_forever() in ganeti-noded we'll call
httpd.handle_request() but just while a global variable, which we'll
call _EXIT_GANETI_NODED, remains false.
Reviewed-by: iustinp
Guido Trotter [Thu, 26 Jun 2008 14:41:56 +0000 (14:41 +0000)]
Add missing empty line in SshKeyError's docstring
Reviewed-by: iustinp
Guido Trotter [Thu, 26 Jun 2008 14:41:47 +0000 (14:41 +0000)]
Remove spurious check during LUAddNode
There is no point in checking whether the cluster VNC password file
exists as a prerequisite for AddNode, considering the check happens on
the master node, not the target one. Removing this check.
Reviewed-by: iustinp
Guido Trotter [Thu, 26 Jun 2008 14:41:38 +0000 (14:41 +0000)]
Improve LURemoveNode BuildHooksEnv docstring
Reviewed-by: iustinp
Michael Hanselmann [Thu, 26 Jun 2008 09:41:36 +0000 (09:41 +0000)]
devel/upload: Add --no-restart option
If --no-restart is passed to devel/upload, it'll not run
"/etc/init.d/ganeti restart" (which kills processes), making
debugging on a terminal a bit easier.
Reviewed-by: iustinp, ultrotter
Michael Hanselmann [Wed, 25 Jun 2008 08:07:19 +0000 (08:07 +0000)]
Cleanup old DRBD 0.7.x code
Apparently there were still some leftovers. While removing an instance,
I got the message "unhandled exception 'module' object has no attribute
'LD_MD_R1'".
Reviewed-by: iustinp
Iustin Pop [Wed, 25 Jun 2008 06:45:19 +0000 (06:45 +0000)]
Cleanup LV status computation
Currently, when seeing if a LV is degraded or not (i.e. virtual volume),
we first attach to the device (which does an lvdisplay), then do a lvs
in order to display the lv_attr. This generates two external commands to
do (almost) the same thing.
This patch changes the Attach() method for LVs to call lvs and display
both the major/minor (needed for attach) and the lv_status (needed for
GetSyncStatus). Thus, later in GetSyncStatus, we don't need to run lvs
again, and instead just return the value computed in Attach().
Reviewed-by: imsnah
Iustin Pop [Tue, 24 Jun 2008 14:30:54 +0000 (14:30 +0000)]
Add a .gitignore file
This makes it easier to setup new git repositories, and makes it more
likely all people have the same ignore rules.
Reviewed-by: imsnah
Michael Hanselmann [Mon, 23 Jun 2008 17:22:06 +0000 (17:22 +0000)]
Add unittests for ganeti.serializer
Reviewed-by: iustinp
Michael Hanselmann [Mon, 23 Jun 2008 17:21:40 +0000 (17:21 +0000)]
Remove lib/Makefile.libcommon
Reviewed-by: iustinp
Iustin Pop [Mon, 23 Jun 2008 16:55:22 +0000 (16:55 +0000)]
Fix gnt-cluster “command” and “copyfile”
Since the disabling of forking in the master daemon, the two ssh-based
subcommands were not working anymore. However, there is no need at all
for the commands to be run from the master daemon (permissions to read
the cluster private ssh key notwithstanding), they can be run directly
from the command line utilities.
The patch removes the two opcodes OpRunClusterCommand and
OpClusterCopyFile (and their associated LUs) and changes the code in
‘gnt-cluster’ to query the list of nodes and run directly the SshRunner
over the list. As such, all forking is done from the gnt-cluster script,
and the commands are working again.
Reviewed-by: imsnah
Guido Trotter [Mon, 23 Jun 2008 15:00:16 +0000 (15:00 +0000)]
Handle any exception in ganeti-masterd
If an uncaught exception is thrown currently it destroys the calling
thread. This patch changes the behaviour to failing the current job,
logging a message, but trying to keep the daemon up.
Reviewed-by: imsnah
Michael Hanselmann [Mon, 23 Jun 2008 13:39:17 +0000 (13:39 +0000)]
cfgupgrade: Implement upgrading to Ganeti 2.0 configuration
Reviewed-by: iustinp
Michael Hanselmann [Mon, 23 Jun 2008 13:15:18 +0000 (13:15 +0000)]
Makefile.am: Don't create "--" directory
Automake automatically appends "--" to @mkdir_p@. In case you have
a directory named "--" in your source tree, you can remove it using
the command "rm -rf -- --".
Reviewed-by: iustinp
Michael Hanselmann [Mon, 23 Jun 2008 13:00:45 +0000 (13:00 +0000)]
objects: Remove config_version from cluster configuration
Reviewed-by: ultrotter
Michael Hanselmann [Mon, 23 Jun 2008 12:53:15 +0000 (12:53 +0000)]
cfgupgrade: Add main() function
Reviewed-by: iustinp
Michael Hanselmann [Mon, 23 Jun 2008 12:53:04 +0000 (12:53 +0000)]
cfgupgrade: Add logging module
Reviewed-by: iustinp
Guido Trotter [Mon, 23 Jun 2008 12:50:15 +0000 (12:50 +0000)]
Fix the zombie process unittest
The failure is because in high load, the parent gets to run before the
child has the chance to os._exit(), and therefore it is still running
when the parent does the check.
The fix removes the chance of this happening by waiting to receive a SIGCHLD
(but not calling wait()) before trying to test the pid.
Reviewed-by: imsnah
Michael Hanselmann [Mon, 23 Jun 2008 11:30:54 +0000 (11:30 +0000)]
Bump version to 2.0.0~alpha0
We decided to bump the major number to 2 a few weeks ago due to the huge number
of changes going into it.
Reviewed-by: iustinp
Michael Hanselmann [Mon, 23 Jun 2008 11:11:42 +0000 (11:11 +0000)]
Add functions to calculate version number to constants.py
In cfgupgrade, we need to extract parts of and build new version numbers.
Reviewed-by: iustinp
Michael Hanselmann [Mon, 23 Jun 2008 09:52:52 +0000 (09:52 +0000)]
utils.WriteFile: Remove optional check_abspath parameter
cfgupgrade will not work with relative paths at all, but rather get them
from constants.py.
Reviewed-by: iustinp
Iustin Pop [Sun, 22 Jun 2008 10:57:52 +0000 (10:57 +0000)]
Add a ‘tags’ field to instance and node listing
Currently there isn't any easy way to list all nodes or instance and
their tags; you have to query each node in turn, or list all the tags
via something like “gnt-cluster search-tags '.*'”. Of course, this is
not optimal.
The patch adds a new fields to “gnt-instance list” and “gnt-node list”
called ‘tags’, that will list the tags of the object in comma-separated
form. This field will be empty if there are no tags (when using a
separator this output can still be parsed by other scripts).
At opcode level, there is a new fields called ‘tags’ that returns a
(python) list of the object tags.
Reviewed-by: ultrotter
Iustin Pop [Sat, 21 Jun 2008 18:49:14 +0000 (18:49 +0000)]
Implement handling of luxi errors in cli.py
Currently the generic handling of ganeti errors in cli.py (GenericMain
and FormatError) only handles the core ganeti errors, and not the client
protocol errors (which live in a separate hierarchy).
This patch adds handling of luxi errors too, and also adds another luxi
error for the case when the master is not running. This gives us a nice:
gnta1:~# gnt-node list
Cannot communicate with the master daemon.
Is it running and listening on '/var/run/ganeti-master.sock'?
error message instead of a traceback.
Reviewed-by: amishchenko
Iustin Pop [Sat, 21 Jun 2008 11:27:22 +0000 (11:27 +0000)]
Remove twisted checks from configure.ac
Currently we don't use twisted, so we remove the twisted checks from the
configure stage.
Reviewed-by: amishchenko
Iustin Pop [Fri, 20 Jun 2008 11:04:27 +0000 (11:04 +0000)]
Add a rpc call for BlockDev.Close()
This patch adds rpc layer calls (in rpc.py and the equivalent in
ganeti-noded) to close a list of block devices, and the wrapper in
backend.py that takes a list of Disk objects, identifies them and
returns correctly formatted results.
The reason why this very basic call was missing until now from the rpc
layer is that we usually don't care about device closes (though we
should, and will do so in the future) as only drbd has a meaningful
Close() operation; right now we directly do Shutdown().
The patch is clean enough that it's actually independent of the live
migration implementation.
Reviewed-by: imsnah
Michael Hanselmann [Thu, 19 Jun 2008 14:06:28 +0000 (14:06 +0000)]
Check for docbook2{man,pdf,html}
docbook2{man,pdf,html} are mandatory. "configure" aborts if one
of them isn't found.
Reviewed-by: iustinp
Iustin Pop [Thu, 19 Jun 2008 13:37:08 +0000 (13:37 +0000)]
Small typo in gnt-instance manpage
Reviewed-by: manuel.franceschini
Michael Hanselmann [Thu, 19 Jun 2008 12:56:17 +0000 (12:56 +0000)]
Use a single Makefile.am instead of many
This change allows us to use cleaner dependencies between
directories. The build system is basically rewritten in large parts
and may contain bugs.
Reviewed-by: iustinp
Iustin Pop [Wed, 18 Jun 2008 15:09:08 +0000 (15:09 +0000)]
Fix bdev unittest when run under distcheck
The path to the filename for drbd8 proc data is not correctly computed
when using distcheck. The patch duplicates it from the other drbd tests.
Reviewed-by: ultrotter
Iustin Pop [Wed, 18 Jun 2008 15:08:53 +0000 (15:08 +0000)]
Rework the DRBD8 device status computation
Currently, compute the status of a drbd8 device in GetSyncStatus and
return only the values that we need (and fit in the framework of
GetSyncStatus). However, the full status details are useful (and needed)
in other places, so the patch attempts to improve this situation.
We abstract the status of a device outside in a separate class, that
knows how to parse contents from /proc/drbd and set easily accessible
attributes. We then simplify the GetSyncStatus to use this and return
the values that it needs, and add a separate method that returns the
full status object.
The move to a separate class cleans up a little bit the old
sync-progress computation from GetSyncStatus, but it's still many
regexes.
The patch also adds unittests for a few statuses, and modifies one
BaseDRBD call to accept a custom filename instead of '/proc/drbd' to
ease unittests.
Reviewed-by: imsnah
Michael Hanselmann [Wed, 18 Jun 2008 12:32:23 +0000 (12:32 +0000)]
ganeti-watcher: Replace custom exceptions with ganeti.error.*
Reviewed-by: iustinp
Michael Hanselmann [Wed, 18 Jun 2008 12:31:53 +0000 (12:31 +0000)]
ganeti-watcher: Don't write file if data didn't change
This is the safest way to detect changes and the amount of data
is small, so keeping a copy around is cheap enough.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 18 Jun 2008 12:31:34 +0000 (12:31 +0000)]
ganeti-watcher: Rename WatcherState.data to WatcherState._data
Cleanup: _data is private and should not be modified from outside
of this class.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 18 Jun 2008 12:31:16 +0000 (12:31 +0000)]
Don't log SystemExit exception in ganeti-watcher
Reviewed-by: iustinp
Michael Hanselmann [Wed, 18 Jun 2008 12:31:00 +0000 (12:31 +0000)]
Replace watcher state file atomically
- Lock it before renaming
- Code cleanup; close() automatically unlocks it
Reviewed-by: iustinp
Michael Hanselmann [Wed, 18 Jun 2008 12:30:44 +0000 (12:30 +0000)]
Write ganeti-watcher status file even if something failed
Reviewed-by: iustinp
Michael Hanselmann [Wed, 18 Jun 2008 12:30:11 +0000 (12:30 +0000)]
Add more parameters to utils.WriteFile
- Make closing file optional: Required by ganeti-watcher to keep
file open after writing it. Changes return value of utils.WriteFile
if "close" parameter evaluates to True.
- Pre- and post-write functions: Can be used to lock files. This
will be used by ganeti-watcher to lock the temporary file before
renaming.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 18 Jun 2008 12:29:52 +0000 (12:29 +0000)]
Use ganeti.serializer module in ganeti-watcher
Reviewed-by: ultrotter
Michael Hanselmann [Wed, 18 Jun 2008 12:29:37 +0000 (12:29 +0000)]
Replace custom logging code in watcher with logging module
- Log timestamp for all messages
- Write everything to logfile and optionally to stderr
- Log messages are no longer buffered, allowing a user to see progress
Reviewed-by: ultrotter
Michael Hanselmann [Wed, 18 Jun 2008 12:29:03 +0000 (12:29 +0000)]
Make sure serialized data ends with EOL character
Also fix the regular expression to not remove newlines. The simplejson
module puts whitespace at line endings when using indentation. Remove
unnecessary import of ConfigParser module.
Reviewed-by: ultrotter
Iustin Pop [Tue, 17 Jun 2008 15:08:24 +0000 (15:08 +0000)]
Allow disk object to set their own physical ID
Currently, the way to customize a DRBD disk from (node name 1, node name
2, port) to (ip1, port, ip2, port) is to use the ConfigWriter method
SetDiskID. However, since this needs a ConfigWriter object, it can be
run only on the master, and therefore disk object can't be passed to
more than one node unchanged. This, coupled with the rpc layer
limitation that all nodes in a multi-node call receive the same
arguments, prevent any kind of multi-node operation that has disks as an
argument.
This patch takes the SetDiskID method from ConfigWriter and ports it to
the disk object itself, and instead of the full node configuration it
uses a simple {node_name: replication_ip} mapping for all the nodes
involved in the disk tree (currently we only pass primary and secondary
node since we don't support nested drbd devices).
This allows us to send disks to both the primary and secondary nodes at
once and perform synchronized drbd activation on primary/secondary
nodes.
Note that while for the 1.2 branch this will not change old methods, it
is worth to investigate and possible replace all such calls on the
master to the nodes themselves for the 2.0 branch.
Reviewed-by: ultrotter
Iustin Pop [Tue, 17 Jun 2008 13:13:14 +0000 (13:13 +0000)]
Fix an error-handling case
There is a mistake in handling grow-disk for an invalid disk. This patch
fixes it.
Reviewed-by: imsnah
Iustin Pop [Tue, 17 Jun 2008 06:51:18 +0000 (06:51 +0000)]
Manpage updates for the new grow-disk command
The patch documents the steps needed to complete a user-visible grow
(i.e. not only grow-disk, but also filesystem resize is needed, etc.)
Reviewed-by: imsnah
Iustin Pop [Tue, 17 Jun 2008 06:51:05 +0000 (06:51 +0000)]
Implement gnt-instance grow-disk
This patch exposes at command line level the grow-disk operation.
Reviewed-by: imsnah
Iustin Pop [Tue, 17 Jun 2008 06:50:51 +0000 (06:50 +0000)]
Implement disk grow at LU level
This patch adds a new opcode and LU for growing an instance's disk.
The opcode allows growing only one disk at time, and will throw an error
if the operation fails midway (e.g. on the primary node after it has
been increased on the secondary node). As such, it might actually leave
different sized LVs on different nodes, but this will not create
problems.
Reviewed-by: imsnah
Iustin Pop [Tue, 17 Jun 2008 06:50:33 +0000 (06:50 +0000)]
Add method to update a disk object size
This patch adds a method that implements updating of a disk
(object.Disk) size, together with its children.
While this will not track the exact disk size, it allows at least an
approximate size to be recorded in the configuration (and queried).
Reviewed-by: imsnah
Iustin Pop [Tue, 17 Jun 2008 06:50:19 +0000 (06:50 +0000)]
Implement block device grow at the rpc layer
This simple patch exposes the block device grow operation at the rpc
layer. It does not increase the protocol version as it has been recently
changed by the live failover rpc call.
Reviewed-by: imsnah
Iustin Pop [Mon, 16 Jun 2008 16:06:43 +0000 (16:06 +0000)]
Expose block device grow in backend.py
This patch adds a wrapper over the block device grow operation that
converts the input and output parameters as needed for the rpc layer.
Reviewed-by: imsnah
Iustin Pop [Mon, 16 Jun 2008 16:01:03 +0000 (16:01 +0000)]
bdev: implement disk resize for lvm/drbd8
This patch implements disk resize at the bdev level for the LVM and
DRBD8 disk types. It is not implemented for DRBD7 and MD since the way
MD works with its underlaying devices makes it harder and this
combination is also deprecated.
The LVM resize operation is tried three times, with different allocation
policies:
- contiguous first, since this is best for allocation purposes (it
won't fragment too much the PV)
- cling, which is supported only by more recent LVM versions, will try
to place the new extents on the same PV as the rest of the LV
- and finally normal, which is the default
Reviewed-by: imsnah
Guido Trotter [Mon, 16 Jun 2008 14:32:22 +0000 (14:32 +0000)]
Move SetKey to WritableSimpleStore and use it
Before we used to be able to update SimpleStore by just calling SetKey, this
feature is now moved to an external class, which inherits from it. In this
patch the new WritableSimpleStore class is also put to use, in the LUs that
need it. Rather than making each LU instantiate it, we have a new LogicalUnit
flag REQ_WSSTORE which defaults to False, but when declared to be True asks the
LogicalUnit to be initialized with a writeable version of the SimpleStore.
LUMasterFailover and LURenameCluster are then changed to use it.
InitCluster is also changed to instantiate a WritableSimpleStore, rather
than a normal one.
Reviewed-by: imsnah
Iustin Pop [Mon, 16 Jun 2008 13:57:36 +0000 (13:57 +0000)]
Add migration support at the rpc layer
This patch adds the migration rpc call and its implementation in the
backend. The patch does not deal with the correct activation of disks.
Because of the new RPC, the protocol version is increased.
Reviewed-by: imsnah
Iustin Pop [Mon, 16 Jun 2008 10:37:26 +0000 (10:37 +0000)]
hypervisor: add live migration support
This is just the hypervisor-level migration (e.g. “xm migrate”) not the
whole node coordination work.
Reviewed-by: ultrotter
Guido Trotter [Sun, 15 Jun 2008 10:55:37 +0000 (10:55 +0000)]
Activate down instances' disks on replace-disks
When replacing disks or evacuating nodes with instances administratively
down ganeti fails because the instance disks are not active. This patch
activates them, performs the replacement, and shuts them down again.
Changing this also fixes the same issue on gnt-node evacuate.
Reviewed-by: iustinp
Guido Trotter [Sun, 15 Jun 2008 10:55:24 +0000 (10:55 +0000)]
FailoverInstance: change AddInstance with Update
We're not adding a new instance, just making configuration changes to
the one we're working on.
Reviewed-by: imsnah
Guido Trotter [Sun, 15 Jun 2008 10:55:09 +0000 (10:55 +0000)]
Burnin: Use iallocator in import/export
Currently the iallocator option is ignored by burnin at import/export
time even if passed in. With this patch it becomes used. The log message
used by the importer is also changed to reflect this.
This patch also improves import/export on the non-iallocator case:
- The secondary node is not passed anymore on non-mirrored templates
- On mirrored templates the secondary node is logged
Reviewed-by: imsnah
Iustin Pop [Sun, 15 Jun 2008 05:22:12 +0000 (05:22 +0000)]
Fix an error message in instance add
There is a mistake in the error message generated when we can't reach a
node for checking for available disk space. Without it, the error
message is:
Failure: prerequisites not met for this operation:
Cannot get current information from node '{u'gnte2.lab.k1024.org':
{'cpu_total': 1, 'memory_free': 480, 'vg_size': 131068, 'memory_total':
504, 'bootid': '
2176dd3b-2f96-42f0-8b6e-
2873ecaf5f9c', 'memory_dom0':
134, 'vg_free': 130172}, u'gnte1.lab.k1024.org': False}'
instead of the expected:
Failure: prerequisites not met for this operation:
Cannot get current information from node 'gnte2.lab.k1024.org'
Reviewed-by: imsnah
Michael Hanselmann [Fri, 13 Jun 2008 14:33:51 +0000 (14:33 +0000)]
Move warning flags from autogen.sh to configure.ac
Reviewed-by: iustinp
Michael Hanselmann [Fri, 13 Jun 2008 12:46:43 +0000 (12:46 +0000)]
Replace logging functions with calls to logging module
- Shorter code
- Reorder arguments to logger.SetupLogging calls to make more sense
Reviewed-by: iustinp
Guido Trotter [Fri, 13 Jun 2008 10:14:31 +0000 (10:14 +0000)]
Fail job on ganeti exceptions
When a Job raises a ganeti exception a message is printed but nothing is
reported in the job itself. It's better to update the job status, thus
notifying the client, possibly polling for the job result, of what went
wrong.
Reviewed-by: iustinp
Guido Trotter [Fri, 13 Jun 2008 10:14:09 +0000 (10:14 +0000)]
Fix a typo in jqueue.py
s/result/op_result/ (this code was never used, so this wasn't caught)
Reviewed-by: iustinp
Michael Hanselmann [Thu, 12 Jun 2008 13:47:32 +0000 (13:47 +0000)]
Don't use specific versions in autogen.sh
Not all distributions have the same version of aclocal, autoconf
or automake. Users can pass the name of a specific executables
via environment variables. Change configure.ac to require at
least autoconf 1.9.
Reviewed-by: iustinp
Michael Hanselmann [Thu, 12 Jun 2008 13:05:27 +0000 (13:05 +0000)]
Move InitCluster opcode into a single function
This allows us to initialize a new cluster. The code certainly contains
bugs and hooks aren't implemented yet.
Reviewed-by: iustinp
Michael Hanselmann [Thu, 12 Jun 2008 13:04:57 +0000 (13:04 +0000)]
Move cmdlib._HasValidVG to utils.CheckVolumeGroupSize
This is required for splitting the cluster initialization code.
Reviewed-by: iustinp
Michael Hanselmann [Thu, 12 Jun 2008 13:04:23 +0000 (13:04 +0000)]
Move {Set,Remove}EtcHostsEntry wrappers to utils.py
This is required for the split of the cluster initialization code.
Reviewed-by: iustinp, ultrotter
Michael Hanselmann [Thu, 12 Jun 2008 10:18:39 +0000 (10:18 +0000)]
Remove REQ_CLUSTER from opcode handling code
It's not needed anymore now that all opcodes require a cluster. Cluster
initialization was the only exception.
Reviewed-by: iustinp
Michael Hanselmann [Thu, 12 Jun 2008 10:06:35 +0000 (10:06 +0000)]
Remove unreachable code from cli.SubmitOpCode
Reviewed-by: iustinp
Michael Hanselmann [Thu, 12 Jun 2008 10:06:18 +0000 (10:06 +0000)]
Rename master socket to ganeti-master.sock
…/run/master.sock is not specific enough.
Reviewed-by: iustinp