Iustin Pop [Thu, 31 Mar 2011 16:07:35 +0000 (18:07 +0200)]
Add a simple wrapper over utils.Retry
The new wrapper makes moving legacy code to utils.Retry or adding
retries in existing code simpler.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 31 Mar 2011 10:52:52 +0000 (12:52 +0200)]
Automatically enable hail if enabled and found
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 31 Mar 2011 10:49:07 +0000 (12:49 +0200)]
Expose whether htools was enabled to Python code
This exports whether htools was enabled at configure-time, and adds a
constant for our reference iallocator.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Thu, 31 Mar 2011 08:21:54 +0000 (10:21 +0200)]
test.ganeti.process_unittest: Fix race condition
There was a race condition on heavily loaded testsystem causing randomly
to fail the timeout unittests as the signal handler is not yet setup but
the timeout has already hit.
Therefore we introduce a workaround to wait until a program reached a
certain point (for example after signal handling setup) before we
actually go for the real run. The wait of course has a timeout as well,
but it's pretty high. If we hit the 20 seconds we have really big issues
anyway.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 31 Mar 2011 08:51:31 +0000 (10:51 +0200)]
Improve references to htools in the documentation
Was not sure about the bit in admin.rst, hope it's fine.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Wed, 30 Mar 2011 12:00:31 +0000 (14:00 +0200)]
Clarify the need for QuickCheck/Haskell tests
Expands the devnotes.rst doc and adds warnings in the Makefile.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 30 Mar 2011 11:32:26 +0000 (13:32 +0200)]
RAPI client: Remove support for version 0 instance creation requests
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 30 Mar 2011 10:52:02 +0000 (12:52 +0200)]
RAPI server: Drop support for instance creation format 0
Ganeti 2.1.3, released in June 2010, added support for a new, extensible
instance creation request format, called version 1. This patch removes
support for the old and undocumented version 0 format.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Simeon Miteff [Mon, 28 Mar 2011 20:47:36 +0000 (22:47 +0200)]
Improved GanetiRapiClient docstrings
- Added @rtype and/or @return where missing
- Fixed @param for Query() filter_ parameter (colon was missing)
Signed-off-by: Simeon Miteff <simeon.miteff@gmail.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Adeodato Simo [Fri, 25 Mar 2011 20:57:44 +0000 (20:57 +0000)]
Add design for inter-group instance moves (multi-reloc)
Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Adeodato Simo [Thu, 24 Mar 2011 21:28:47 +0000 (21:28 +0000)]
iallocator.rst: refactor for readability; minor improvements
This commits breaks down the "Input message" section of iallocator.rst into
two separate subsections: one detailing keys that are required in all
operation types; a second one detailing the "request" element, which is
different for each type of request.
Some other minor improvements are included as well:
- update input example to version 2, and add the "nodegroups" and
"enabled_hypervisors" top-level elements, and the "group" and
"hypervisor" attributes for nodes and allocation request, respectively.
- sort keys in the example dictionaries according to the order in earlier
sections, for easy comparison of documentation with its examples.
Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Mon, 28 Mar 2011 08:50:08 +0000 (10:50 +0200)]
Relax instance ERROR on admin_down on offline node
This fixes a issue, where an stopped instances is reported as ERROR
in cluster verify if it lives on a offline node. As the instances is
down this shouldn't happen.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 25 Mar 2011 13:22:24 +0000 (14:22 +0100)]
Implement submitting jobs from logical units
The design details can be seen in the design document
(doc/design-lu-generated-jobs.rst).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Adeodato Simo [Wed, 23 Mar 2011 16:49:24 +0000 (16:49 +0000)]
iallocator.rst: give pointers for alloc_policy semantics
Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Adeodato Simo [Wed, 23 Mar 2011 16:41:54 +0000 (16:41 +0000)]
Doc fix in iallocator.rst: multi-evac requires "evac_nodes"
The request argument for multi-evacuate mode is "evac_nodes", not "nodes"
(the example later in the file has the correct name already).
Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 24 Mar 2011 15:28:19 +0000 (16:28 +0100)]
watcher: improve logging a bit
Add some debug logging to detail why we don't run some steps.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Mon, 21 Mar 2011 15:47:47 +0000 (16:47 +0100)]
Remove old ensure-dirs (no longer needed)
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 18 Mar 2011 09:30:09 +0000 (10:30 +0100)]
Rewrite of ensure-dirs in python
I provided unittest to test the important pieces of the infrastructure.
The one remaining function (ResuriveEnsure) is not easy to unittest
but also not critical if it fails to operate correctly.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 23 Mar 2011 15:23:50 +0000 (16:23 +0100)]
hs-coverage: make a symlink hpc_index.html
This allows Apache to display the directory in a nicer way.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 23 Mar 2011 15:06:50 +0000 (16:06 +0100)]
Another attempt at fixing htools build without curl
OK, my previous small fix was not good. There is another issue: haddoc
(the documentation generator) needs to pass the same compiler options
(i.e. in our case, -DNO_CURL) to ghc. But in case of no curl, then it
shouldn't scan at all the RAPI library, as that is not used in our
builds.
Clearly, this is not a nice thing. So this patch changes from
including/excluding RAPI conditionally (in two places, the
ExtLoader.hs module and in hscan.hs), to always include RAPI, and
moves the curl/no curl login to RAPI itself, where it belongs.
Together with passing --optghc to haddock, this makes the builds
consistent both with and without RAPI. I also undo the removal of RAPI
from QC.hs.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 23 Mar 2011 14:42:09 +0000 (15:42 +0100)]
Fix some lint warnings in htools code
hlint gives more suggestions, but some make the code (IMHO) harder to
read.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 23 Mar 2011 12:25:22 +0000 (13:25 +0100)]
Fix lint errors in the htools code
These are just changes from hlint suggestions. Still compiles and
passes unittests.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 23 Mar 2011 16:01:13 +0000 (17:01 +0100)]
Add opcode summary to SubmitManyJobs errors
Requested-by: Iustin Pop <iustin@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 28 Feb 2011 17:04:45 +0000 (18:04 +0100)]
Add design for submitting jobs from LUs
This patch adds a design document describing how jobs can be submitted
from within LUs.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 23 Mar 2011 12:54:27 +0000 (13:54 +0100)]
Fix Haskell unittests without RAPI
Since we don't test for now the RAPI backend directly, we can simply
skip the import. Later we can make a conditional import if needed.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 1 Feb 2011 13:47:25 +0000 (14:47 +0100)]
Add import/export version 2 design document
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 24 Jan 2011 18:43:25 +0000 (19:43 +0100)]
Add design document for lighttpd as HTTP server
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 24 Jan 2011 18:42:45 +0000 (19:42 +0100)]
Add design document for X509 CA
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 23 Mar 2011 11:25:45 +0000 (12:25 +0100)]
Wrap long lines in configure.ac
- Use m4_normalize to make single-line strings while removing
unnecessary spaces
- Wrap lines longer than 80 characters
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 22 Mar 2011 17:58:55 +0000 (18:58 +0100)]
Update INSTALL and devnotes.rst with Haskell notes
This documents the needed libraries for Haskell development. It also
fixes a tiny typo in devnotes.rst.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 22 Mar 2011 17:18:36 +0000 (18:18 +0100)]
Revert and change the apidoc and coverage dirs
Based on Michael's suggestion, this patch partially reverts my
changes. The new directories are:
- doc/api/py
- doc/api/hs
- doc/coverage/py
- doc/coverage/hs
Basically the Python-specific output moves one level down (into py/)
compared to the original location, and the Haskell stuff goes into
hs/.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 22 Mar 2011 17:06:50 +0000 (18:06 +0100)]
Rename away htools/NEWS
Also add mention about it being obsolete.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 22 Mar 2011 17:04:52 +0000 (18:04 +0100)]
Cleanup the Haskell-related Makefile.am variables
This should be more readable now. I wanted to even use the nicer
_SOURCES, but _SOURCES is special in Automake (again), so _SRCS it is.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 22 Mar 2011 10:42:35 +0000 (11:42 +0100)]
Remove obsolete htools/Makefile
Only one target wasn't ported over (the TAGS one), as hasktags is not
available easily in distributions, so it doesn't make sense to enable
it for all developers.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 22 Mar 2011 10:35:22 +0000 (11:35 +0100)]
Move hlint rule to the main Makefile
Since we do have errors currently, this is not enabled from the main
'make lint' rule. That will get cleaned up later.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 22 Mar 2011 10:26:13 +0000 (11:26 +0100)]
Port the live-test htools rule to the main Makefile
This was a bit tricky, as the compilation from the top-dir changes the
paths in the .tix/.mix files.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 22 Mar 2011 09:47:59 +0000 (10:47 +0100)]
Enable htools apidoc generation and unify dir names
Previously, Python api doc was under doc/api (which didn't match with
the target rule, apidoc). After this patch, we have the following:
- make py-apidoc generates Python api doc under doc/py-apidoc
- make hs-apidoc generates Haskell api doc under doc/hs-apidoc
- make apidoc does both (if hs-apidoc enabled at configure time)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 21 Mar 2011 15:14:49 +0000 (16:14 +0100)]
Enable htools unittests and coverage reports
Since we have two sets of coverage data, we make the naming
consistent:
- doc/py-coverage for Python coverage
- doc/hs-coverage for Haskell coverage
We also need to exclude some more dirs from the list of checked
directories.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 21 Mar 2011 14:21:20 +0000 (15:21 +0100)]
Fix parallel compilation for htools
We do it via per-target-binary .o/.hi files. There are a few other
options, in the end this was chosen as different binaries can have
different options/components (e.g. hail won't use RAPI, etc.) and that
the unittests need different compilation options.
We add a wildcard *.o/*.hi to CLEANFILES, so that it removes all
per-target build artifacts.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 21 Mar 2011 12:51:21 +0000 (13:51 +0100)]
htools: add option to control the use of RAPI
RAPI needs the curl module, which is a binding to the libcurl C
library. This is more complex (can be compiled either against gnutls
or openssl, etc.) and it's not of much help when the tools are used on
the cluster itself, we disable it by default.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 21 Mar 2011 12:15:15 +0000 (13:15 +0100)]
htools: install hail in the iallocator dir
Instead of bindir. We do this by adding a new allocdir variable, and
redirecting hail to it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 21 Mar 2011 11:35:32 +0000 (12:35 +0100)]
Remove some obsolete htools files
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 21 Mar 2011 11:33:53 +0000 (12:33 +0100)]
Integrate htools's gitignore in top-level one
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 21 Mar 2011 11:25:07 +0000 (12:25 +0100)]
Makefile.am: add rules for htools compilation
This patch adds some (not very nice) rules for htools
compilation. Since automake only knows about some languages, and
doesn't have a real extension mechanism, I cheat and declare them as
"buildable scripts", instead of programs. Rationale:
automake support for dir_PROGRAMS is limited to a few languages; in
all cases, it assumes that translation from sources to binaries occurs
in two steps, compilation and linking; it's not easy to change its
assumption.
In our case, where “ghc --make” does both steps, it's hard to force it
into the automake model.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 21 Mar 2011 10:00:06 +0000 (11:00 +0100)]
configure.ac: add checks for Haskell compiler/libs
This patch adds an enable/disable option for htools (--enable-htools),
and associated tests for determining whether GHC (the compiler we use
for htools) and required libraries are present.
The method to do so is not very nice; usually, Haskell programs are
configured and compiled using cabal (http://www.haskell.org/cabal/, a
tool similar to Python's setuptools)), but that doesn't suit itself to
nice integration with autoconf/automake, so we test for the modules
presence manually.
In the end, we set a few variables:
- GHC: the path to the ghc compiler
- HTOOLS_MODULES: command line option for ghc to select the wanted
'parallel' module
- HTOOLS_NOCURL: set to -DNO_CURL if we don't want to enable curl (and
thus RAPI) support in htools
- HTOOLS: set to 'yes' if we should compile/install the htools
programs
- HTOOLS_APIDOC: set to yes if we should build/install the htools
API documentation
- WANT_HTOOLS, WANT_HTOOLSAPIDOC: two automake conditionals for later
use in Makefile.am
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 18 Mar 2011 14:08:30 +0000 (15:08 +0100)]
RAPI client: Tidy and test WaitForJobCompletion
- Use constants
- Don't sleep if no delay is given
- Mark function as deprecated: it uses polling instead of waiting for changes
(but the latter needs authentication); it can still be used
- Add unittests
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 18 Mar 2011 14:05:43 +0000 (15:05 +0100)]
RAPI client: Add job status constants
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 18 Mar 2011 12:50:32 +0000 (13:50 +0100)]
RAPI client: Job IDs are strings
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 18 Mar 2011 12:08:08 +0000 (13:08 +0100)]
Split BuildHooksEnv of LUs
Commit
dd7f677623 added another call to BuildHooksEnv to provide
post-phase status variables. Since BuildHooksEnv also built the node
lists, that meant they have to be built twice. First a rather strict
check was used, but it turned out to be more tricky. Commit
b423c51336
had to remove the strict check again.
With this patch the function is split in two parts, one generating the
actual environment variables, and another part returning the node lists.
The former is called twice.
Unittests are updated.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Apollon Oikonomopoulos [Fri, 18 Mar 2011 14:38:40 +0000 (16:38 +0200)]
cfgupgrade: really ignore hostname when told so
cfgupgrade provides a --ignore-hostname option, however consequently it
spawns config.ConfigWriter, which refuses to handle a foreign
configuration file by default.
config.ConfigWriter provides a relative keyword argument,
accept_foreign, so we tie this to the value of options.ignore_hostname.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 18 Mar 2011 10:22:39 +0000 (11:22 +0100)]
RAPI client: fix epydoc formatting
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Theo Van Dinter [Fri, 18 Mar 2011 09:39:18 +0000 (10:39 +0100)]
Add a helper function to the RAPI client
This adds a new method WaitForJobCompletion that can be used for
client who are not interested in the entire job log, just in its
completion status.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 17 Mar 2011 17:24:35 +0000 (18:24 +0100)]
Move htools-ganeti 2.3 design doc to doc/
I didn't know where to put in the TOC, so I added it after the ganeti
2.3 design doc.
Also, it seems that Sphinx is a bit limited in its list nesting
(compared to Pandoc), so I had to rework a bit the indentation (it
still doesn't generate good output, but…).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 17 Mar 2011 17:06:03 +0000 (18:06 +0100)]
Integrate htools man pages in the main tree
This moves the man pages to their right place, removes the static
footer and header, and adds Makefile rules for the new .1 man pages.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 17 Mar 2011 12:23:03 +0000 (13:23 +0100)]
Remove restrictive hook node list check
Commit
dd7f67762 added a restrictive check for the node lists returned
by BuildHooksEnv, leading to errors with some LUs, one of which was
fixed in commit
0dfa2c227. As it turns out, other LUs have similar
issues, some not easy to fix. This patch disables the restrictive check
until the BuildHooksEnv functions can be split into one part generating
the actual environment and one generating the node lists.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 17 Mar 2011 16:24:14 +0000 (17:24 +0100)]
Revert "Add temporary dir/file for htools import"
This reverts commit
8bc9e04a21457c4188475dc849d0d7d1e22ee929, not
needed anymore.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 17 Mar 2011 15:47:06 +0000 (16:47 +0100)]
Merge remote branch 'htools/master'
* htools/master: (605 commits)
Update NEWS file for version 0.3.1
Update copyright years in the man pages
Remove obsolete env vars from the manpages
Fix dist archive generation
luxi backend: show attribute names in errors
Update exit code information in the hbal man page
Fix broken hscolour.css symlink in tarball
Update NEWS file for the 0.3.0 release
Remove an obsolete TODO
Update live-test.sh w.r.t. simulation mode changes
Prevent allocation on unallocable node groups
Fix bug in simulation backend
Precompute allocation nodes
Change the AllocNodes type and functions
Extract node pair generation from tryAlloc
tryAlloc: restrict valid node pairs to same-group
Cluster.hs: add a new type alias
Rapi: read and use the vm_capable node flag
IAllocator: read and use the vm_capable node flag
IAllocator: replace fake policy with real one
...
Note: done via the following sequence of commands:
git checkout master && git pull
git remote add htools git://git.ganeti.org/htools.git
git remote update
git merge -s recursive -X subtree=htools htools/master
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 17 Mar 2011 10:06:20 +0000 (11:06 +0100)]
Add temporary dir/file for htools import
In order for “git merge -s recursive -X subtree=htools” to do its
magic, an (mostly empty) directory must exist in the target tree name
‘htools’. Hence we add a temp file to it before the merge, and remove
it after that.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 17 Mar 2011 10:52:23 +0000 (11:52 +0100)]
Fix hook node list when adding node
This broke QA (and everyone trying to add a node) by complaining about
different node lists.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 16 Mar 2011 17:35:18 +0000 (18:35 +0100)]
hooks: Provide variables with post-opcode values
When a hook is called, it is provided with a number of variables
describing the status of the instance/node/etc. before the operation.
Some opcodes provide extra variables to see modified values from hooks,
but that's not a generic solution.
This patch modifies the code calling hooks to generate the environment
once before and once after an opcode has been executed. Doing so should
be safe—I did not find any LU.BuildHooksEnv modifying LU instance
attributes. The values collected after running the opcode are prefixed
with “GANETI_POST_”, as opposed to “GANETI_” for pre-execution
variables. The latter are still provided for backwards compatibility.
Environment variable examples:
gnt-instance start $instance:
GANETI_INSTANCE_STATUS=down
GANETI_POST_INSTANCE_STATUS=up
gnt-instance modify -B memory=512 $instance:
GANETI_INSTANCE_BE_memory=768
GANETI_POST_INSTANCE_BE_memory=512
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 16 Mar 2011 12:59:32 +0000 (13:59 +0100)]
HooksMaster: Add more assertions for variable names
Also replace explicit loop with dict.update.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 15 Mar 2011 16:59:15 +0000 (17:59 +0100)]
mcpu: Tidy HooksMaster a bit
- Dictionary indentation
- Add empty lines for readability
- Simplify conditional code
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 15 Mar 2011 16:25:49 +0000 (17:25 +0100)]
cmdlib: Factorize running post-pase hook
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 11 Mar 2011 13:26:24 +0000 (14:26 +0100)]
QA: Add tests for queries via RAPI
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Tue, 15 Mar 2011 13:08:13 +0000 (14:08 +0100)]
qlang: Remove unused import
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 11 Mar 2011 13:25:24 +0000 (14:25 +0100)]
RAPI: Add support for querying resources
- Access is only permitted for authenticated clients (queries can return
sensitive data)
- Filters can be specified when sending a PUT request
- Updates RAPI client, documentation and tests
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Tue, 8 Mar 2011 13:14:11 +0000 (14:14 +0100)]
Add support for query resources in RAPI URIs
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 11 Mar 2011 13:26:06 +0000 (14:26 +0100)]
masterd: Simplify code for field queries
Instead of going via cmdlib and using special cases for different
resources, the list of fields is used directly.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 11 Mar 2011 13:26:17 +0000 (14:26 +0100)]
constants: Rename QR_OP_*, add QR_VIA_RAPI
Commit
28b71a76 added a list of resources which can be queried using
LUXI. Unfortunately the variable was named “QR_OP_LUXI”, which can be
confusing. This patch renames “QR_OP_QUERY” to “QR_VIA_OP”, “QR_OP_LUXI”
to “QR_VIA_LUXI” and adds a new variable, currently equal to
“QR_VIA_LUXI”, for remote API queries (mostly for documentation).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 11 Mar 2011 14:57:37 +0000 (15:57 +0100)]
qlang: Remove unused ReadSimpleFilter
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
René Nussbaumer [Tue, 15 Mar 2011 08:23:23 +0000 (09:23 +0100)]
TLReplaceDisks: Add check if disks are activated
Previously we failed later with a rather useless error message. This
patch fixes this and tells the user to activate-disks if replace-disks
is in the need of activated disks rather than abort with a cryptic error
message.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 8 Mar 2011 10:43:42 +0000 (11:43 +0100)]
LUOsDiagnose: Move legacy behaviour into filter
The behaviour of LUOsDiagnose needs special treatment. Commit
d22dfef7
changed it to not return hidden, blacklisted or invalid OSes if the
respective field is not requested. This behaviour needs to be preserved
for users of the old opcode, but since it provides an implicit
filtering, it should not be done for LUQuery (the generic way to query
for resources) and is therefore moved from the underlying implementation
to LUOsDiagnose. Users of the new opcode should pass an explicit filter.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 7 Mar 2011 13:48:46 +0000 (14:48 +0100)]
Convert OsDiagnose to query
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 11 Mar 2011 16:08:49 +0000 (17:08 +0100)]
qlang: Add some more documentation for filters
It's not perfect, but at least some more.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 11 Mar 2011 13:54:50 +0000 (14:54 +0100)]
query: Add conversion wrapper
Allows converting the value of a column before returning it. Useful for
sorting while still using one of the other generic functions for
retrieving the value.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Fri, 11 Mar 2011 16:56:23 +0000 (17:56 +0100)]
Fix epydoc warning about unknown reference
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 11 Mar 2011 16:29:41 +0000 (17:29 +0100)]
config: Wrap MatchNameComponent, reduce lock duration
- Remove duplication by merging two MatchNameComponent into a wrapper
- Reduce lock duration by getting list of names under lock and then
matching names without the lock
- Also, ExpandNodeName's docstring is fixed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 8 Mar 2011 13:13:50 +0000 (14:13 +0100)]
opcodes: Document OpQueryFields' parameters
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Fri, 11 Mar 2011 10:12:36 +0000 (11:12 +0100)]
Update NEWS file for version 0.3.1
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 11 Mar 2011 10:07:35 +0000 (11:07 +0100)]
Update copyright years in the man pages
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Fri, 11 Mar 2011 10:05:40 +0000 (11:05 +0100)]
Remove obsolete env vars from the manpages
Commit
16c2369c removed support for using environment variables for
selecting the text backend (since now it's just one option), but
didn't update the man pages.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 9 Mar 2011 16:23:10 +0000 (17:23 +0100)]
Instance failover: fix bug for INT_MIRROR cases
Patches
db366d9a and
aac4511a added support for EXT_MIRROR instances,
but inadvertently introduced a bug: for INT_MIRROR cases, we don't
need (actually we can't support) neither an iallocator nor a target
node.
To fix this, we move the iallocator/node checks in CheckPrereq (or
respectively in the tasklet CheckPrereq), where we have access to the
instance configuration, and additionally we check for and prevent
passing either of these two for INT_MIRROR instances.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Wed, 9 Mar 2011 09:50:57 +0000 (10:50 +0100)]
gnt-cluster epo: Adding --power-delay flag
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 9 Mar 2011 09:47:35 +0000 (10:47 +0100)]
gnt-node power: Adding --power-delay flag
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 9 Mar 2011 09:44:16 +0000 (10:44 +0100)]
cli.py: Adding POWER_DELAY_OPT
The command line option --power-delay sets the time waited between power
ons.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Tue, 8 Mar 2011 10:56:57 +0000 (11:56 +0100)]
OpOobCommand: Adding power on delay
This delays the invocation of the power on of the next node. So if you
power on a bunch of nodes it will not blow the fuse.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 8 Mar 2011 10:56:25 +0000 (11:56 +0100)]
OpOobCommand: Document all fields
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Tue, 8 Mar 2011 10:34:15 +0000 (11:34 +0100)]
gnt-cluster epo: Adding --shutdown-timeout
This adds the --shutdown-timeout flag to gnt-cluster epo to specify the
shutdown timeout for instance shutdown.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Apollon Oikonomopoulos [Mon, 28 Feb 2011 11:51:16 +0000 (13:51 +0200)]
Rename DTS_NET_MIRROR to DTS_INT_MIRROR
DTS_INT_MIRROR better contrasts DTS_EXT_MIRROR.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
[iustin@google.com: updated patch for changed context]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 28 Feb 2011 11:51:15 +0000 (13:51 +0200)]
Shared file support for tools/burnin
Make burnin work with -t sharedfile and update burnin's help.
Shared block storage support is *not* included, as currently there is no way to
provision shared block storage from within Ganeti.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 28 Feb 2011 11:51:14 +0000 (13:51 +0200)]
KVM: use cache=none for shared disk templates
Disable host cache for externally mirrored disks to avoid cache incoherency.
Without this, migrations between the same two nodes may end up in disk
corruption.
This is a runtime override of cluster defaults, mostly a workaround.
Unfortunately, we cannot use cmdlib._CheckHVParams, as we want to check the
hypervisor parameters *in conjunction* with the disk template, which is part of
the instance definition.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 28 Feb 2011 11:51:13 +0000 (13:51 +0200)]
Shared storage instance failover
Modify LUFailoverInstance to enable shared storage instances to failover.
Shared storage instance failover requires either a target node or an
iallocator to determine the target node. If none is given, the cluster default
iallocator is used.
The hook environment variables {OLD,NEW}_SECONDARY will be blank for shared
storage instances.
Locking behaviour is the same as for instance migration.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
[iustin@google.com: revert the DTS_NET_MIRROR specific changes]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 28 Feb 2011 11:51:12 +0000 (13:51 +0200)]
Shared storage node migration
Modify LUNodeMigrate to provide node migration for nodes with instances using
shared storage. gnt-node migrate has to be passed an iallocator for migration
of shared storage instances to be performed. When using a shared storage
backend, all cluster nodes are locked.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 28 Feb 2011 11:51:11 +0000 (13:51 +0200)]
Shared storage instance migration
Modify LUMigrateInstance and TLMigrateInstance to allow instance migrations for
instances with DTS_EXT_MIRROR disk templates.
Migrations of shared storage instances require either a target node, or an
iallocator to determine the target node. If none is given, the cluster default
iallocator is used.
Locking behaviour: If the iallocator is used, then initially all nodes are
locked and subsequently only the locks on the source node and the target node
selected by the iallocator are retained.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
[iustin@google.com: small changes in cmdlib.py]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 28 Feb 2011 11:51:10 +0000 (13:51 +0200)]
CLI changes to facilitate shared storage migration/failover
Add DST_NODE_OPT to cli.py to use for directly specifying the target node
during migration/failover.
gnt-instance failover/migrate also get passed an iallocator option.
gnt-node failover/migrate get only a target_node option.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 28 Feb 2011 11:51:09 +0000 (13:51 +0200)]
Migration and failover: add iallocator and target_node slots
Add iallocator and target_node slots to OpMigrateInstance and
OpFailoverInstance to facilitate shared-storage-backed instance mobility. Add
iallocator slot to OpMigrateNode (no explicit target_node in this case).
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 28 Feb 2011 11:51:08 +0000 (13:51 +0200)]
IAllocator changes to work with shared storage
Make cmdlib.IAllocator shared-storage-aware. IAllocator requires secondary
nodes only on DTS_NET_MIRROR disk templates and requires no secondaries for
DTS_EXT_MIRROR templates.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Fri, 4 Mar 2011 14:35:23 +0000 (16:35 +0200)]
Shared block storage support
This patch introduces basic shared block storage support.
It introduces a new storage backend, bdev.PersistentBlockDevice, to
use as a backend for shared block storage. The new bdev requires a new
BLOCKDEV_DRIVER_MANUAL constant with the value "manual" and uses it as
the first part of the block device unique_id.
A new disk template, DT_BLOCK is introduced as well and added to
DTS_EXT_MIRROR and DTS_MAY_ADOPT. Also added DTS_MUST_ADOPT constant
and use it to check for the presence of the adopt keyword during LU
invocation. We enforce the /dev/disk limitation upon adoption, but we
allow block devices to reside anywhere under /dev.
This is very basic support and includes no storage manipulation (provisioning,
resizing, renaming) which will have to be implemented through a "driver"
framework.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
[iustin@google.com: slight changes to bdev.py]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Fri, 4 Mar 2011 14:28:58 +0000 (16:28 +0200)]
Add bdev_sizes RPC call
The bdev_sizes multi-node RPC call returns the sizes of the requested
block devices on the desired nodes. Its intended use is to verify the
existence of a block device on a given node for shared block storage
support.
Block device paths are expected to lie under constants.BLOCKDEV_DIR
("/dev/disk" by default), where persistent symlinks for block devices
are assumed to exist.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
[iustin@google.com: small changes in backend.py]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 7 Mar 2011 19:01:08 +0000 (20:01 +0100)]
QA: Improve tests for gnt-os
- Test OS lists via command line and RAPI
- Test “gnt-os diagnose” and “gnt-os info”
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>