Guido Trotter [Fri, 10 Oct 2008 16:04:07 +0000 (16:04 +0000)]
LUVerifyCluster: fix error from rpc call
When calling node_verify leads to an error _VerifyNodes tries to iterate
over a non-sequence. Catch the error before and avoid this from
happening.
Reviewed-by: iustinp
Michael Hanselmann [Fri, 10 Oct 2008 16:00:40 +0000 (16:00 +0000)]
Convert ganeti-noded to new HTTP server class
Reviewed-by: iustinp
Michael Hanselmann [Fri, 10 Oct 2008 16:00:25 +0000 (16:00 +0000)]
Add new HTTP server implementation
This patch adds another implementation of an HTTP server. It's
based on code of Python's BaseHTTPServer, from both version
2.4 and 3k. In the future we can write code to decide whether
we should fork for a request or not. Keep-alive is not supported.
Reviewed-by: iustinp
Michael Hanselmann [Fri, 10 Oct 2008 16:00:02 +0000 (16:00 +0000)]
Add daemon library with mainloop
This mainloop can be used in daemons like ganeti-noded.
Reviewed-by: iustinp
Iustin Pop [Fri, 10 Oct 2008 15:12:12 +0000 (15:12 +0000)]
Some updates on the job queue design doc
This clarifies the job storage and the reason for choosing it.
Reviewed-by: imsnah
René Nussbaumer [Fri, 10 Oct 2008 09:55:24 +0000 (09:55 +0000)]
Update design-2.0-job-queue to reflect changes
With change 1773 a new status WAITLOCK was introduced if a job/opcode is
waiting for a lock. This change updates the document about the job-queue
accordingly.
Reviewed-by: iustinp
Iustin Pop [Fri, 10 Oct 2008 09:55:02 +0000 (09:55 +0000)]
Convert rpc module to RpcRunner
This big patch changes the call model used in internode-rpc from
standalong function calls in the rpc module to via a RpcRunner class,
that holds all the methods. This can be used in the future to enable
smarter processing in the RPC layer itself (some quick examples are not
setting the DiskID from cmdlib code, but only once in each rpc call,
etc.).
There are a few RPC calls that are made outside of the LU code, and
these calls are left as staticmethods, so they can be used without a
class instance (which requires a ConfigWriter instance).
Reviewed-by: imsnah
Iustin Pop [Fri, 10 Oct 2008 09:52:47 +0000 (09:52 +0000)]
Cleanup in cmdlib for standalone function calls
This patch is a cleanup of the standalone functions in cmdlib. Many of
them too as argument a ConfigWriter instance, but some also took other
parameters from the lu (e.g. proc), and in the future, if we want to
also pass the RpcRunner, we would have to add yet another parameter.
One option is to make all these methods of top-level LogicalUnit class.
I took another approach, and made (almost) all these functions take as
first parameter the lu instance. It's like methods, just not declared
under LogicalUnit.
Reviewed-by: imsnah
Iustin Pop [Fri, 10 Oct 2008 09:51:34 +0000 (09:51 +0000)]
Small random fixes
Indentation in bootstrap was wrong and some names in cmdlib.py were not
right.
Reviewed-by: imsnah
Iustin Pop [Thu, 9 Oct 2008 11:29:32 +0000 (11:29 +0000)]
Move instance hypervisor check to ExpandNames
This check can be done earlier, in ExpandNames, and is needed here for
the hypervisor parameter check.
Reviewed-by: ultrotter
Alexander Schreiber [Wed, 8 Oct 2008 23:13:54 +0000 (23:13 +0000)]
Update documentation & man pages for changed hypervisor names.
Reviewed-by: imsnah
Oleksiy Mishchenko [Wed, 8 Oct 2008 18:48:19 +0000 (18:48 +0000)]
Ajust config unittest.
Reviewed-by: iustinp
Alexander Schreiber [Wed, 8 Oct 2008 17:31:17 +0000 (17:31 +0000)]
Update scripts and qa config for changed hypervisor names.
Reviewed-by: ultrotter
Alexander Schreiber [Wed, 8 Oct 2008 14:41:07 +0000 (14:41 +0000)]
Shorten variable names.
Reviewed-by: iustinp
Iustin Pop [Wed, 8 Oct 2008 14:31:43 +0000 (14:31 +0000)]
Sanitize the hypervisor names
Since in 2.0 the user will possibly have more interaction with the
hypervisor names, we sanitize them by removing the version numbers
(the version can be a prerequisite for the ganeti installation, we
shouldn't document it in variable names).
Reviewed-by: schreiberal
Iustin Pop [Wed, 8 Oct 2008 14:10:46 +0000 (14:10 +0000)]
Also export OS_API to the OS scripts
The idea is that if the OSes will support multiple version (e.g. both
1.2 and 2.0), then Ganeti should be able to talk to it using version
2.0, but then the script needs to be told nicely what version Ganeti is
using.
Reviewed-by: imsnah
Oleksiy Mishchenko [Wed, 8 Oct 2008 13:04:52 +0000 (13:04 +0000)]
Fix for gnt-cluster init.
Reviewed-by: iustinp
Iustin Pop [Wed, 8 Oct 2008 10:36:29 +0000 (10:36 +0000)]
Move the hypervisor attribute to the instances
This (big) patch moves the hypervisor type from the cluster to the
instance level; the cluster attribute remains as the default hypervisor,
and will be renamed accordingly in a next patch. The cluster also gains
the ‘enable_hypervisors’ attribute, and instances can be created with
any of the enabled ones (no provision yet for changing that attribute).
The many many changes in the rpc/backend layer are due to the fact that
all backend code read the hypervisor from the local copy of the config,
and now we have to send it (either in the instance object, or as a
separate parameter) for each function.
The node list by default will list the node free/total memory for the
default hypervisor, a new flag to it should exist to select another
hypervisor. Instance list has a new field, hypervisor, that shows the
instance hypervisor. Cluster verify runs for all enabled hypervisor
types.
The new FIXMEs are related to IAllocator, since now the node
total/free/used memory counts are wrong (we can't reliably compute the
free memory).
Reviewed-by: imsnah
Iustin Pop [Tue, 7 Oct 2008 16:09:53 +0000 (16:09 +0000)]
Updates to the security document
This patch changes formatting and the DRBD shared secret details, and
adds master daemon socket details to the security doc.
Reviewed-by: imsnah
Iustin Pop [Tue, 7 Oct 2008 16:08:55 +0000 (16:08 +0000)]
Move the SECURITY document to the doc/ dir
Reviewed-by: imsnah
Iustin Pop [Tue, 7 Oct 2008 15:37:28 +0000 (15:37 +0000)]
Fix formatting in design-2.0-os-interface
Reviewed-by: imsnah
Iustin Pop [Tue, 7 Oct 2008 15:23:18 +0000 (15:23 +0000)]
Small changes to the index design doc
This is just some additions of not-yet-mentioned docs.
Reviewed-by: ultrotter
Alexander Schreiber [Tue, 7 Oct 2008 13:16:32 +0000 (13:16 +0000)]
Change default instance reboot type to hard.
Merged r1777 from branches/ganeti/ganeti-1.2
Reviewed-by: imsnah
Alexander Schreiber [Tue, 7 Oct 2008 13:00:03 +0000 (13:00 +0000)]
Add new design docs to Makefile.am
Reviewed-by: imsnah
Iustin Pop [Tue, 7 Oct 2008 11:39:50 +0000 (11:39 +0000)]
rpc.call_instance_migrate: pass the whole instance
Currently the call_instance_migrate call only passes the instance name;
we need to pass the whole object for the hypervisor_type changes (all
the other individual instance rpc calls already pass the instance
object).
Reviewed-by: imsnah
Alexander Schreiber [Tue, 7 Oct 2008 10:22:04 +0000 (10:22 +0000)]
Slightly change the hypervisor parameter example.
Reviewed-by: iustinp
Alexander Schreiber [Tue, 7 Oct 2008 08:47:51 +0000 (08:47 +0000)]
Ganeti 2.0 cluster parameters design doc
Reviewed-by: ultrotter
Iustin Pop [Tue, 7 Oct 2008 08:03:19 +0000 (08:03 +0000)]
Implement job 'waiting' status
Background: when we have multiple jobs in the queue (more than just a
few), many of the jobs (up to the number of threads) will be in state
'running', although many of them could be actually blocked, waiting for
some locks. This is not good, as one cannot easily see what is
happening.
The patch extends the opcode/job possible statuses with another one,
waiting, which shows that the LU is in the acquire locks phase. The
mechanism for doing so is simple, we initialize (in the job queue) the
opcode with OP_STATUS_WAITLOCK, and when the processor is ready to give
control to the LU's Exec, it will call a notifier back into the
_JobQueueWorker that sets the opcode status to OP_STATUS_RUNNING (with
the proper queue locking). Because this mechanism does not save the job,
all opcodes on disk will be in status WAITLOCK and not RUNNING anymore,
so we also change the load sequence to consider WAITLOCK as RUNNING.
With the patch applied, creating in parallel (via burnin) five instances
on a five node cluster shows that only two are executing, while three
are waiting for locks.
Reviewed-by: imsnah
Guido Trotter [Tue, 7 Oct 2008 07:25:56 +0000 (07:25 +0000)]
OS Interface design doc
Reviewed-by: imsnah
Guido Trotter [Tue, 7 Oct 2008 07:25:38 +0000 (07:25 +0000)]
Add .. contents:: marker to design docs
Reviewed-by: imsnah
Iustin Pop [Mon, 6 Oct 2008 16:42:18 +0000 (16:42 +0000)]
Implement job auto-archiving
This patch adds a new luxi call that implements auto-archiving of jobs
older than a certain age (or -1 for all completed jobs), and the gnt-job
command that makes use of this (with 'all' for -1).
Reviewed-by: imsnah
Iustin Pop [Mon, 6 Oct 2008 15:59:32 +0000 (15:59 +0000)]
Add a simple timespec parsing function
This function will be used for auto-archiving jobs via the command line.
The function is pretty simple, we only support up to weeks since months
and higher are not 'precise' entities, and dealing with them would
require us to start using calendar functions.
Reviewed-by: imsnah
Iustin Pop [Mon, 6 Oct 2008 15:58:26 +0000 (15:58 +0000)]
backend.py change to get cluster name from master
Currently there are three function in backend that need the cluster name
in order to instantiate an SshRunner. The patch changes these to get the
cluster name from the master in the rpc call; once the multi-hypervisor
change is implemented, then very few places in which we need the SCR
remain in the backend.
Reviewed-by: killerfoxi, imsnah
Iustin Pop [Mon, 6 Oct 2008 15:56:31 +0000 (15:56 +0000)]
Disable re-reading of config file
Since the objects read from the config file are passed to the various
threads, it's unsafe to re-read the config file (and throw away
ConfigWriter._config_data). As such, we disable the re-reading of the
file (since now the master is the owner the file, it makes not sense to
re-read it), and any modifications to the file must be done offline,
otherwise they will be overwritten.
Reviewed-by: imsnah
Oleksiy Mishchenko [Mon, 6 Oct 2008 15:13:40 +0000 (15:13 +0000)]
RAPI Desing Doc
Reviewed-by: iustinp
Iustin Pop [Mon, 6 Oct 2008 14:11:02 +0000 (14:11 +0000)]
Start implementation of parallel burnin
This patch introduces a simple framework for executing jobs in parallel
in burnin (the ExecJobSet function) and the "--parallel" command line
flag.
The patch also changes the instance creation to run in parallel when the
above flag is given. Error handling/instance removal is currently flacky
with this options if there are errors in the instance creation.
We also modify burnin to reuse a single client.
Reviewed-by: imsnah
Iustin Pop [Mon, 6 Oct 2008 13:40:52 +0000 (13:40 +0000)]
Fix gnt-job list with empty timestamps
In case the job object doesn't have a timestamp (which is a separate
issue), the listing should not break. We fix this by changing the
FormatTimstamp function itself to return '?' in case the timestamp
doesn't look good (note that it still can break if non-integers are
returned, but this is unlikely).
Reviewed-by: imsnah
Iustin Pop [Mon, 6 Oct 2008 13:29:51 +0000 (13:29 +0000)]
Increase the number of threads to 25
Since our locks are not gathered nicely, we can have jobs that are
actually blocking on locks (parallel burnin shows this), so at least we
need to increase the number of threads above the usual number of jobs we
could have in a such a case.
Reviewed-by: imsnah
Alexander Schreiber [Mon, 6 Oct 2008 13:21:28 +0000 (13:21 +0000)]
Minor cleanups & typo fixes.
Reviewed-by: iustinp
Iustin Pop [Mon, 6 Oct 2008 13:16:23 +0000 (13:16 +0000)]
Fix SshRunner breakage from the changed API
More places actually use the SshRunner than just the gnt-cluster
commands.
Reviewed-by: ultrotter
Iustin Pop [Mon, 6 Oct 2008 11:48:59 +0000 (11:48 +0000)]
Change SshRunner usage
Currently the SshRunner uses a SimpleConfigReader instance, however this
is not best. We change it to use the cluster name directly (and its
constructor now takes this as parameter, instead of SCR), and its
callers are change to pass the name directly.
As a consequence, we can now remove the initialization of SCR in
gnt-cluster (copyfile and command), and instead we query the master for
the cluster name).
Reviewed-by: imsnah
Michael Hanselmann [Mon, 6 Oct 2008 11:01:49 +0000 (11:01 +0000)]
Update document describing cluster security
It may need further updates, but here's a start.
Reviewed-by: ultrotter
Iustin Pop [Sun, 5 Oct 2008 09:16:03 +0000 (09:16 +0000)]
Fix ssconf.GetMasterAndMyself
The ssconf migration left this out.
Reviwed-by: imsnah,ultrotter
Iustin Pop [Fri, 3 Oct 2008 14:32:30 +0000 (14:32 +0000)]
Fix a mistake in the gnt-backup man page
The actual location is /export, not /exports.
Reviewed-by: ultrotter
Michael Hanselmann [Thu, 2 Oct 2008 10:22:57 +0000 (10:22 +0000)]
Use docbook2* paths found during configure for actual build
docbook-wrapper had the names for the docbook2* programs hardcoded. This
patch changes Makefile.am and the wrapper script to pass them via
another argument.
Another issue where rapi.in was built before rapi-resources.sgml is
also fixed.
Reviewed-by: iustinp
Michael Hanselmann [Thu, 2 Oct 2008 10:22:02 +0000 (10:22 +0000)]
Remove references to Twisted framework
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:37:29 +0000 (17:37 +0000)]
Get rid of ssconf
Remove leftovers from ssconf.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:37:05 +0000 (17:37 +0000)]
Don't pass sstore to LUs anymore
sstore is no longer used in LUs.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:36:51 +0000 (17:36 +0000)]
Convert ganeti-master
Use simpleconfig instead of ssconf.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:36:20 +0000 (17:36 +0000)]
Convert ganeti-watcher
Use RPC calls instead of ssconf.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:36:09 +0000 (17:36 +0000)]
Convert ganeti-noded
Replace ssconf with utility functions.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:35:59 +0000 (17:35 +0000)]
Convert gnt-cluster
Replace ssconf with configuration.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:35:49 +0000 (17:35 +0000)]
Convert bootstrap.py
Replace ssconf with configuration.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:35:37 +0000 (17:35 +0000)]
Convert cmdlib.py
Replacing ssconf with configuration. Cluster rename is broken and stays
that way.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:35:21 +0000 (17:35 +0000)]
Convert ssh.py
Get rid of ssconf and convert to configuration instead.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:34:59 +0000 (17:34 +0000)]
Convert rpc.py
Replacing ssconf with utility functions.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:34:48 +0000 (17:34 +0000)]
Convert hypervisor
Replacing ssconf with configuration.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:34:32 +0000 (17:34 +0000)]
Convert mcpu.py
Replacing ssconf with configuration.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:34:18 +0000 (17:34 +0000)]
Convert config.py
The configuration version is now again in the configuration file.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:34:05 +0000 (17:34 +0000)]
Convert backend.py
Replacing ssconf with simpleconfig.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:33:52 +0000 (17:33 +0000)]
Add new query to get cluster config values
This can be used to retrieve certain cluster config values from
within clients.
OpDumpClusterConfig was not used anywhere, hence I'm just reusing
it. The way ConfigWriter.DumpConfig returned the configuration
was not thread-safe, anyway (no deepcopy).
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:33:33 +0000 (17:33 +0000)]
Move functions from ssconf.py elsewhere
These functions will be used to access config values instead of using
ssconf.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 17:33:18 +0000 (17:33 +0000)]
Add simple configuration reader/writer classes
This will be used to read the configuration file in the node daemon.
The write functionality is needed for master failover.
Reviewed-by: iustinp
Iustin Pop [Wed, 1 Oct 2008 09:27:27 +0000 (09:27 +0000)]
Fix the watcher with down nodes
The watcher didn't handle the down nodes, fix this by ignoring (in
secondary node reboot checks) any node that doesn't return a boot id.
Reviewed-by: imsnah
Iustin Pop [Wed, 1 Oct 2008 09:27:17 +0000 (09:27 +0000)]
Fix the watcher not restarting instance bug
The watcher was using conflicting attributes of the instance:
- it queried the admin_/oper_state, which are booleans
- but it compared those to the status (which is a text field)
The code was changed to query the aggregated 'status' field, as that
will also return indication of node problems, and we can use this only
one field for all decisions. We still ask for the admin_state field as
that is needed for the activate disks check (in secondary node restart).
The patch also touches the watcher in some other parts:
- log exceptions nicer
- convert a method to @staticmethod
- remove unused imports
Reviewed-by: imsnah
Iustin Pop [Wed, 1 Oct 2008 09:27:07 +0000 (09:27 +0000)]
Remove last use of utils.RunCmd from the watcher
The watcher has one last use of ganeti commands as opposed to sending
requests via luxi. The patch changes this to use the cli functions.
The patch also has two other changes:
- fix the docstring for OpVerifyDisks (found out while converting
this)
- enable stderr logging on the watcher when “-d” is passes
Reviewed-by: imsnah
Michael Hanselmann [Wed, 1 Oct 2008 09:24:18 +0000 (09:24 +0000)]
Fix unittests broken by revision 1727
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 09:03:22 +0000 (09:03 +0000)]
Add cluster options from ssconf to configuration
ssconf will become write-only from ganeti-masterd's point of view,
therefore all settings in there need to go into the main configuration
file.
Reviewed-by: iustinp
Michael Hanselmann [Wed, 1 Oct 2008 09:03:06 +0000 (09:03 +0000)]
Move instantiation of config into bootstrap.py
Future patches will add even more variables to the cluster config.
Adding more parameters wouldn't make the function easier to use and
it doesn't make sense to pass them to another function, as it's
only done once in bootstrap.py on cluster initialization.
Reviewed-by: iustinp
Iustin Pop [Wed, 1 Oct 2008 08:29:16 +0000 (08:29 +0000)]
Change the results from cli.PollJob
Curently PollJob accepts a generic job, but will return (history
artifact) only the first opcode result. This is wrong, as it doesn't
allow polling of a job with multiple results.
Its only caller (for now) is also changed, so no functional changes
should happen.
Reviewed-by: ultrotter, amishchenko
Michael Hanselmann [Tue, 30 Sep 2008 15:01:54 +0000 (15:01 +0000)]
Add list of build dependencies
Reviewed-by: iustinp
Michael Hanselmann [Tue, 30 Sep 2008 12:28:14 +0000 (12:28 +0000)]
Build HTML from RST input
This patch also adds the design documents to Makefile.am.
Reviewed-by: iustinp
Iustin Pop [Tue, 30 Sep 2008 12:22:13 +0000 (12:22 +0000)]
Fix ‘gnt-job info’ with no arguments
I didn't realize that my zip will break when no args are passed...
Reviewed-by: imsnah
Iustin Pop [Tue, 30 Sep 2008 12:13:26 +0000 (12:13 +0000)]
Add output of job/opcode timestamps
This patch adds posibility of selection of job/opcode timestamps in
gnt-job list and info.
The code handling the possible cases (None or a valid timestamps) are
ugly though...
Reviwed-by: imsnah
Iustin Pop [Tue, 30 Sep 2008 12:04:31 +0000 (12:04 +0000)]
Enhance the job-related timestamps
This patch adds start, stop, and received timestamp for jobs (and allows
querying of them), and allows querying of the opcode timestamps.
Reviewed-by: imsnah
Michael Hanselmann [Tue, 30 Sep 2008 11:36:46 +0000 (11:36 +0000)]
Small fixes for master daemon design document
It said CLI/RAPI will talk to master using HTTP, which isn't true. Add
a reference to job queue design document. Small typos.
Reviewed-by: iustinp
Alexander Schreiber [Tue, 30 Sep 2008 11:16:15 +0000 (11:16 +0000)]
Import design doc for commandline arguments
Reviewed-by: iustinp
Guido Trotter [Tue, 30 Sep 2008 10:21:21 +0000 (10:21 +0000)]
locking design: code path and declarations
Reviewed-by: iustinp
Guido Trotter [Tue, 30 Sep 2008 10:21:10 +0000 (10:21 +0000)]
locking design: explain use of async mode
Before we were discussing this possible future feature, and its
drawbacks, but not its usefulness. This patch corrects this.
Reviewed-by: iustinp
Guido Trotter [Tue, 30 Sep 2008 10:20:58 +0000 (10:20 +0000)]
locking design: talk about removing locks
Reviewed-by: iustinp
Guido Trotter [Tue, 30 Sep 2008 10:07:08 +0000 (10:07 +0000)]
Import (and update) granular locking design doc
Reviewed-by: iustinp
Iustin Pop [Tue, 30 Sep 2008 09:36:49 +0000 (09:36 +0000)]
Abstract the timestamp formatting into cli.py
Currently we format the timestamp inside the gnt-job info function. We
will need this more times in the future, so move it to cli.py as a
separate, exported function.
Reviewed-by: imsnah
Michael Hanselmann [Mon, 29 Sep 2008 17:55:49 +0000 (17:55 +0000)]
Add job queue design document
Reviewed-by: iustinp
Iustin Pop [Mon, 29 Sep 2008 16:18:30 +0000 (16:18 +0000)]
Add an 'index' of design documents
This will be an overview document, enumerating the changes without going
into details and pointing to the actual documents.
Reviewed-by: ultrotter
Iustin Pop [Mon, 29 Sep 2008 15:38:40 +0000 (15:38 +0000)]
Add opcode execution log in job info
This patch adds the job execution log in “gnt-job info” and also allows
its selection in “gnt-job list” (however here it's not very useful as
it's not easy to parse). It does this by adding a new field in the query
job call, named ‘oplog’.
With this, one can get a very clear examination of the job. What remains
to be added would be timestamps for start/stop of the processing for the
job itself and its opcodes.
Reviewed-by: imsnah
Iustin Pop [Mon, 29 Sep 2008 13:15:33 +0000 (13:15 +0000)]
Move a hardcoded constant to constants.py
For now we only use the ‘C’ protocol so we can put it in constants.py
instead of hardcoding it.
Reviewed-by: imsnah
Iustin Pop [Mon, 29 Sep 2008 13:15:20 +0000 (13:15 +0000)]
Enable the use of shared secrets
This patch enables the use of the shared secrets for DRBD8 disks, using
(hardcoded in constants.py) the md5 digest algorithm.
For making this more flexible, either we implement a cluster parameter
(once the new model is in place), or we can make it ./configure-time
selectable.
Reviewed-by: imsnah
Iustin Pop [Mon, 29 Sep 2008 13:15:08 +0000 (13:15 +0000)]
Extend DRBD disks with shared secret attribute
This patch, which is similar to r1679 (Extend DRBD disks with minors
attribute), extends the logical and physical id of the DRBD disks with a
shared secret attribute. This is generated at disk creation time and
saved in the config file.
The generation of the secret is done so that we don't have duplicates in
the configuration (otherwise the goal of preventing cross-connection
will not be reached), so we add to config.py more than just a simple
call to utils.GenerateSecret().
The patch does not yet enable the use of the secrets.
Reviewed-by: imsnah
Iustin Pop [Mon, 29 Sep 2008 13:09:53 +0000 (13:09 +0000)]
Add a info subcommand to gnt-job
Currently, it is hard to examine a job in detail; the output of ‘gnt-job
list’ is not easy to parse.
The patch adds a ‘gnt-job info’ command that is (vaguely) similar to
‘gnt-instance info’ in that it shows in a somewhat easy to understand
format the details of a job.
The result formatter is the most complicated part, since the results are
not standardized; the code attempts to format nicely the most common
result types (as taken from a random job list), via a generic algorithm.
Reviewed-by: imsnah
Iustin Pop [Mon, 29 Sep 2008 13:09:34 +0000 (13:09 +0000)]
Implement job summary in gnt-job list
It is not currently possibly to show a summary of the job in the output
of “gnt-job list”. The closes is listing the whole opcode(s), but that
is too verbose. Also, the default output (id, status) is not very
useful, unless one looks for (and knows about) an exact job ID.
The patch adds a “summary” description of a job composed of the list of
OP_ID of the individual opcodes. Moreover, if an opcode has a ‘logical’
target in a certain opcode field (e.g. start instance has the instance
name as the target), then it is included in the formatting also. It's
easier to explain via a sample output:
gnt-job list
ID Status Summary
1 error NODE_QUERY
2 success NODE_ADD(gnta2)
3 success CLUSTER_QUERY
4 success NODE_REMOVE(gnta2.example.com)
5 error NODE_QUERY
6 success NODE_ADD(gnta2)
7 success NODE_QUERY
8 success OS_DIAGNOSE
9 success INSTANCE_CREATE(instance1.example.com)
10 success INSTANCE_REMOVE(instance1.example.com)
11 error INSTANCE_CREATE(instance1.example.com)
12 success INSTANCE_CREATE(instance1.example.com)
13 success INSTANCE_SHUTDOWN(instance1.example.com)
14 success INSTANCE_ACTIVATE_DISKS(instance1.example.com)
15 error INSTANCE_CREATE(instance2.example.com)
16 error INSTANCE_CREATE(instance2.example.com)
17 success INSTANCE_CREATE(instance2.example.com)
18 success INSTANCE_ACTIVATE_DISKS(instance1.example.com)
19 success INSTANCE_ACTIVATE_DISKS(instance2.example.com)
20 success INSTANCE_SHUTDOWN(instance1.example.com)
21 success INSTANCE_SHUTDOWN(instance2.example.com)
This is done by a simple change to the opcode classes, which allows an
opcode to format itself. The additional function is small enough that it
can go in opcodes.py, where it could also be used by a client if needed.
Reviewed-by: imsnah
Iustin Pop [Mon, 29 Sep 2008 13:08:44 +0000 (13:08 +0000)]
Nicely sort the job list
Unless we decide to change the job identifiers to integer, we should at
least sort the list returned by _GetJobIDsUnlocked.
Reviewed-by: imsnah
Iustin Pop [Sun, 28 Sep 2008 14:44:52 +0000 (14:44 +0000)]
Move the pseudo-secret generation to utils.py
The bootstrap code needs a pseudo-secret and this is currently generated
inside the InitGanetiServerSetup function. Since more users will need
this, move it to utils.py
Reviewed-by: ultrotter
Iustin Pop [Sun, 28 Sep 2008 14:43:30 +0000 (14:43 +0000)]
Fix a bug related to static minors
When the node does not yet have any minors allocated, the first minor
(0) will not be entered in the ConfigWriter._temporary_drbds structure.
This does not happen for our current usage, since we always ask for two
minors (so the next call will not match this case), but it will be
triggered if we only ask for one minor, and then ask again before adding
the instance to the config file.
Reviewed-by: ultrotter
Iustin Pop [Sat, 27 Sep 2008 20:45:01 +0000 (20:45 +0000)]
Add checks for tcp/udp port collisions
In case the config file is manually modified, or in case of bugs, the
tcp/udp ports could be reused, which will create various problems
(instances not able to start, or drbd disks not able to communicate).
This patch extends the ConfigWriter.VerifyConfig() method (which is used
in cluster verify) to check for duplicates between:
- the ports used for DRBD disks
- the ports used for network console
- the ports marked as free in the config file
Also, if the cluster parameter ‘highest_used_port’ is actually lower
than the computed highest used port, this is also flagged as an error.
The output from gnt-cluster verify will show (output manually wrapped):
node1 # gnt-cluster verify
* Verifying global settings
- ERROR: tcp/udp port 11006 has duplicates: instance3.example.com/network port,
instance2.example.com/drbd disk sda
- ERROR: tcp/udp port 11017 has duplicates: instance3.example.com/drbd disk sda,
instance3.example.com/drbd disk sdb, cluster/port marked as free
- ERROR: Highest used port mismatch, saved 11010, computed 11017
* Gathering data (2 nodes)
...
Reviewed-by: ultrotter
Iustin Pop [Sat, 27 Sep 2008 15:58:51 +0000 (15:58 +0000)]
Update the cluster serial_no on certain operations
This patch adds update of the cluster serial number for:
- add/remove node (as the cluster's node list is changed)
- add/remove/rename instance (as the cluster's instance list is changed)
- change the volume group name
The rule for updating this attribute is when cluster-wide properties are
changed, but not individual node/instance ones.
There are other remaining cases to handle, pending on the ssconf
changes.
Reviewed-by: ultrotter
Iustin Pop [Sat, 27 Sep 2008 15:58:41 +0000 (15:58 +0000)]
Allow listing of the serial_no via gnt-* list
This patch adds listing of the serial_no attribute in gnt-instance and
gnt-node list, and updates to the manpages to reflect the change.
Reviewed-by: ultrotter
Iustin Pop [Sat, 27 Sep 2008 15:58:22 +0000 (15:58 +0000)]
Initialize and update the serial_no on objects
This patch add initialization of the serial_no on instance and nodes,
and update of the field whenever an object is updated in the generic
case, via ConfigWriter.Update(obj) and in the specific case of
instances' state being modified manually.
Reviewed-by: ultrotter
Iustin Pop [Sat, 27 Sep 2008 15:58:13 +0000 (15:58 +0000)]
Switch the global serial_no to the top object
Currently the serial_no that is incremented every time the configuration
file is written is located on the 'cluster' object in the configuration
structure. However, this is wrong as the cluster serial_no should be
incremented only when the cluster state is changed (for whatever
definition of “changed” we will use), not simply because the
configuration file is written.
This patch changes so that the ConfigWriter._BumpSerialNo affects the
top-level ConfigData object.
Reviewed-by: ultrotter
Iustin Pop [Sat, 27 Sep 2008 15:58:03 +0000 (15:58 +0000)]
Add serial_no attributes to objects
This patch adds the ‘serial_no’ attribute to the other top-level objects
(the configuration object itself, the nodes and the instances).
Reviewed-by: ultrotter
Iustin Pop [Sat, 27 Sep 2008 15:57:52 +0000 (15:57 +0000)]
Replace a cfg.AddInstance with UpdateInstance
This seems to be the last (deprecated) use of AddInstance in order to
update an instance.
The patch also removes a whitespace-at-eol case.
Reviewed-by: ultrotter