Michael Hanselmann [Fri, 12 Oct 2012 10:30:49 +0000 (12:30 +0200)]
errors: Document arguments to QueryFilterParseError
Also fix one small mistake in the docstring for QuitGanetiException.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 12 Oct 2012 09:11:16 +0000 (11:11 +0200)]
Fix distcheck after commit 7a962ec
I only ran “make check” and forgot about “distcheck”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 11:10:56 +0000 (13:10 +0200)]
Makefile/check-local: Ensure design document exists
This way a missing design document is detected.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 11:06:45 +0000 (13:06 +0200)]
Add design document for version 2.7
- Move design document for batch instance creation into it
- Sort list of hidden files in index.rst
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Thu, 11 Oct 2012 01:34:45 +0000 (03:34 +0200)]
QA: TestAssignNodesIncludingSplit can be disabled
The test can now be enabled/disabled with the "group-rwops" flag.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Tue, 9 Oct 2012 15:38:27 +0000 (17:38 +0200)]
QA: never put a master node offline
Before, it could happen that the master node was chosen to be set offline
in the "instance-remove-drbd-offline" test. Also changed a finally clause
to make sure that an exception while making the node online doesn't break
undoing other things.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Bernardo Dal Seno [Mon, 8 Oct 2012 13:43:16 +0000 (15:43 +0200)]
Add option to disable iallocator use in QA
By default some QA tests use iallocators. It is now possible to disable
iallocators in QA, so that QA doesn't force having iallocators set up.
The affected tests have now alternatives pathways.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Sébastien Bocahu [Wed, 19 Sep 2012 13:42:04 +0000 (15:42 +0200)]
Add support for cpu_cap and cpu_weight Xen params
This patch adds support for Xen's CPU scheduler 'cpu_cap' and
'cpu_weight' parameters.
Ganeti default values (cap: 0=unlimited, weight: 256) are Xen defaults.
cpu_cap is not validated correctly because of actual Ganeti limitation
on checks. I'd need check methods to get beparams or vcpus to get moved
to hvparams.
Signed-off-by: Sébastien Bocahu <sebastien.bocahu@nuxit.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Sun, 7 Oct 2012 21:26:43 +0000 (23:26 +0200)]
Enable conversion of errors constants
These will be needed for the serialisation of errors.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 23:58:34 +0000 (01:58 +0200)]
LUClusterVerifyGroup: Localize virtual file paths
The check for file consistency didn't properly handle virtual paths
in case of a virtual cluster. This didn't cause any breakage as in
a standard virtual cluster setup with only one node all files are
visible for every node.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Fri, 5 Oct 2012 01:06:40 +0000 (02:06 +0100)]
Cleanup haddock documentation a bit
While grepping for htools imports in the non-htools subdirectory, I
saw that our haddock prologue and title are very very old and refer to
the old htools-only state. Let's cleanup a bit…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 01:01:22 +0000 (02:01 +0100)]
Cleanup network timeouts and htools imports
This patch removes the last HTools module imports from non-htools code
(the HTools.Types module), but it requires an associated cleanup:
using luxi-specific constants for luxi timeouts (the only effect is
that one timeout decreases from 15 to 10, the default value in the
python code), and moving of the (now) RAPI specific constants to
RAPI.hs (which allows simplifying their type/usage).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 00:43:24 +0000 (01:43 +0100)]
Cleanup/expand the filter/query tests
This patch cleans up duplicate code in Test.Ganeti.Query.Filter and
then adds a test for names consistency with Python's code behaviour
(stable ordering for simple filters and otherwise niceSort'ed
ordering).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 23:32:06 +0000 (00:32 +0100)]
Add a helper for query field checks
… and also use it to simplify 'needsLiveData'. Additionally, add an
explicit export list to Ganeti.Query.Types, since otherwise we'd
(re)export all imported symbols.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Iustin Pop [Thu, 4 Oct 2012 01:30:36 +0000 (02:30 +0100)]
Enable query socket usage in gnt-node/gnt-group
This switches gnt-node/gnt-group (and their equivalent RAPI resources)
to go over the query socket.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 01:22:14 +0000 (02:22 +0100)]
Add support for classic queries
This patch adds support for classic-style queries (before query2) to
the query socket server. The patch is rather trivial, since as in
Python we just piggy-back on the query2 implementation.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Iustin Pop [Thu, 4 Oct 2012 01:08:56 +0000 (02:08 +0100)]
Add a makeSimpleFilter function
And associated unittests. This will be needed for classic-style
queries.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Wed, 3 Oct 2012 23:51:00 +0000 (00:51 +0100)]
Fix the node powered field
When initially implementing the node query, I thought the 'powered'
field is a representation of the run-time powered status, which would
make its query complex.
In reality, it's a simple config query, which we can support
easily. We also add a small helper, so that we don't hardcode the
RSUnavail case in many places.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Iustin Pop [Wed, 3 Oct 2012 23:27:59 +0000 (00:27 +0100)]
Simplify a bit queryFields
We don't add a type class for fully-generic handling, but we do
abstract the duplicate part.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Iustin Pop [Tue, 2 Oct 2012 23:45:12 +0000 (00:45 +0100)]
Add function for getting a group's merged disk params
… and use it in the Query implementation, removing the last
non-correct query field for Groups.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 00:05:20 +0000 (01:05 +0100)]
Switch ordering of names on query to niceSort
This makes the "all" names queries consistent with the Python
results. The change requires updating the unittests, at which point a
duplicate error message is simplified.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 2 Oct 2012 23:43:46 +0000 (00:43 +0100)]
Use the new name filtering behaviour in query
We do this not quite generically, which means we have to add
another layer in the call chain, and rename the current query
function, plus add special-case code for each query type. Hopefully we
will be able to improve on this in the future.
A (good) side effect of this patch is that we get the desired
ordering when names are requested, matching the Python code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Iustin Pop [Tue, 2 Oct 2012 23:39:11 +0000 (00:39 +0100)]
Add functionality for checking validity of names
This replicates in the Haskell Query2 implementation the behaviour of
the Python code: if a "simple" filter is passed (one that contains
only Or aggregators and EQ binary ops on the name field), then an
failure is flagged if the given values are not known.
Our implementation is pretty straightforward, with a few details:
- we ignore any NumericValues passed, since that inconsistency will be
flagged by the filter compiler
- we return an the non-normalized names from the getRequestedNames
function, and not the fully-normalized ones; this will be done later
in individual query functions
- we test a few of the desired behaviours of the above-mentioned
function
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 09:09:33 +0000 (11:09 +0200)]
vcluster: Don't virtualize /etc/hosts path
/etc/hosts is a bit special as it's a system-wide file and the virtual
cluster/node root doesn't apply. The modification of /etc/hosts should
be disabled in virtual clusters. If it isn't, however, the vcluster
functions would raise an exception complaining about a path outside of
the virtual node root. This patch adds a whitelist to exclude
/etc/hosts.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 08:53:00 +0000 (10:53 +0200)]
vcluster-setup: Stop hardcoding /etc/hosts
… use a constant instead.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 08:51:24 +0000 (10:51 +0200)]
qa_utils: Stop hardcoding path to /etc/hosts
- Simplified shell command for adding hosts, too
- The exception handling is also improved by re-raising any exceptions
instead of swallowing qa_error.Error
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 08:51:03 +0000 (10:51 +0200)]
cli: Stop hardcoding /etc/hosts path
There is a constant for this purpose.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 08:42:38 +0000 (10:42 +0200)]
Move constant for /etc/hosts to pathutils
Needed for coming patches.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 09:52:39 +0000 (11:52 +0200)]
bash_completion: Fix mistake introduced in merge 6a74f58
This part of the code was duplicated by accident.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 09:29:55 +0000 (11:29 +0200)]
Merge branch 'devel-2.6'
* devel-2.6:
verify-disks: Explicitely state nothing has to be done
Add list of design documents implemented in version 2.6
Better list of replace-disks arguments + typos fixed
jqueue: Look at archived jobs when watching
Show old primary/secondary node on disk replacement
gnt-instance reinstall: Don't always exit with success
LUClusterVerify: Ignore /proc/drbd if DRBD is disabled
Fixed typos in devnotes.rst
Always_failover doesn't require --allow-failover anymore
bash_completion: Enable extglob while parsing file
rpc: Remove duplicated logic, fix unittests
Annotate disk params on instance_start
cmdlib: Handle locking.ALL_SET correctly when copying locks
Conflicts:
Makefile.am: Sorted file lists
autotools/build-bash-completion: Compact output
lib/rpc_defs.py: instance_start changed on both sides
qa/qa_instance.py: Modified on both sides
test/ganeti.cmdlib_unittest.py: Tests added on both sides
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 09:16:29 +0000 (11:16 +0200)]
Merge branch 'stable-2.6' into devel-2.6
* stable-2.6:
verify-disks: Explicitely state nothing has to be done
Add list of design documents implemented in version 2.6
Better list of replace-disks arguments + typos fixed
jqueue: Look at archived jobs when watching
Show old primary/secondary node on disk replacement
gnt-instance reinstall: Don't always exit with success
LUClusterVerify: Ignore /proc/drbd if DRBD is disabled
Fixed typos in devnotes.rst
Always_failover doesn't require --allow-failover anymore
bash_completion: Enable extglob while parsing file
rpc: Remove duplicated logic, fix unittests
Annotate disk params on instance_start
cmdlib: Handle locking.ALL_SET correctly when copying locks
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 08:58:28 +0000 (10:58 +0200)]
gnt-job list: Add option to include archived jobs
This provides a convenience option to include archived jobs in the
output list. It's equivalent to using “-o +archived”, but tab completion
is nicer.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 08:44:38 +0000 (10:44 +0200)]
gnt-job: List archived jobs if requested
If requested via a filter or by including the “archived” output,
archived jobs will be loaded and shown. This is significantly slower
than just listing normal jobs, therefore by default they are not loaded
at all.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 08:44:12 +0000 (10:44 +0200)]
query: Report data type for unary operators
All data kinds (used to restrict the data collected) referenced in a
filter can be requested once it's been “compiled”. However, the kinds
of fields used in boolean expressions (e.g. ["?", "xyz"]) were not
recorded. This patch changes the code accordingly and provides a unit
test update.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 08:12:07 +0000 (10:12 +0200)]
jqueue: Add new in-memory attribute for archived jobs
This attribute is set to True for jobs which were restored from an
archived file. A new filter will act on this field.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 07:54:40 +0000 (09:54 +0200)]
jqueue: Correct docstring
The description was not accurate.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 11 Oct 2012 09:01:01 +0000 (11:01 +0200)]
Add htools/ganeti-confd to .gitignore
Sorry, this should have went in
b0dcdc10, but I forgot it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 05:21:07 +0000 (07:21 +0200)]
verify-disks: Explicitely state nothing has to be done
Example output:
$ gnt-cluster verify-disks
Submitted jobs 4327
Waiting for job 4327 ...
No disks need to be activated.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 11:15:47 +0000 (13:15 +0200)]
Add basic unit tests for "gnt-cluster epo"
This patch adds some unit tests for “gnt-cluster epo”. Not everything is
covered, but at least the bug fixed in the previous patch is.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 4 Oct 2012 02:44:23 +0000 (03:44 +0100)]
Fixup node disk free/total queries
The disk free/total values are optional ones, wrapped in a Maybe, so
we shouldn't directly serialise them. In order to simplify the
embedded extraction, we add a small helper function.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Dato Simó [Wed, 10 Oct 2012 15:14:56 +0000 (16:14 +0100)]
(htest) Text.hs: fix typo in comment
Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 10 Oct 2012 11:45:07 +0000 (13:45 +0200)]
Fix pylint breakage due to unused var in gnt_cluster
The usage of that variable was removed in
45a36f36, but accidentally
the enumerate() was left in.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 02:09:57 +0000 (03:09 +0100)]
Fixup hypervisor queries in node query
We need to only query the default (first enabled) hypervisor, not all
hypervisors. For this, we need to add a manual check to ensure that we
don't have a corrupt config (there's no "NonEmptyList" type…).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 02:08:56 +0000 (03:08 +0100)]
Fixup node live field names
The String parameter to 'nodeLiveFieldExtract' is the query2 field
name, not the RPC-layer field name. Grrr for not having a real data
type for this.
Furthermore, we add some safety check that we don't return JSNull via
rsNormal…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Iustin Pop [Wed, 10 Oct 2012 05:12:43 +0000 (07:12 +0200)]
Improve Haskell configure options and detection
This patch cleans up the Haskell library detection and defaults.
First, it makes the base compiler/libraries required, per the email discussion.
It then adds two new small autoconf macros, on to check for a required
Haskell library and one to do custom action based on test results. We
use these macros to cleanup and simplify a bit the module detection:
- rapi, confd, and split query are auto detected and enabled if _all_
required libraries are present
- unittests are enabled if _all_ required libraries are present
The patch also updates the documentation regarding required libraries.
After this patch, base Ganeti fully buildable on Debian Squeeze/Ubuntu
Lucid.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 10 Oct 2012 10:17:58 +0000 (12:17 +0200)]
Make regex-pcre an optional dependency
This patch makes regex-pcre optional, allowing its disable via a
preprocessor define NO_REGEX_PCRE. This define will be added to
config.ac/Makefile.am in the next patch.
The patch also changes multi-line strings into string concatenation,
due to limitations with the CPP language extension.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 11:16:30 +0000 (13:16 +0200)]
cluster epo: Fix bug where IndexError is raised
Updating the “node_query_list” variable would fail if no arguments were
passed and the “--all” option wasn't specified. A follow-up patch will
add unit tests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 09:44:13 +0000 (11:44 +0200)]
Add list of design documents implemented in version 2.6
Each version should have its dedicated list.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 2 Oct 2012 23:47:33 +0000 (00:47 +0100)]
Try to auto-enable htools-rapi and split query
We try to automatically enable the htools-rapi and split query (if
confd and htools-rapi are enabled) options. This is our intended
default configuration, and allows easier test of the new code
path. Further cleanups for checking whether confd can be enabled will
come later.
The move block is due to the fact that we first have to check for
htools-rapi, and only then we can auto-enable the feature.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 9 Oct 2012 10:38:50 +0000 (12:38 +0200)]
Fix usage of errors.ResolverError
This exception is documented to have three arguments, but in one case
we raise it with a simple string argument. Let's fix that.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 8 Oct 2012 15:44:01 +0000 (17:44 +0200)]
Remove unused/deprecated error classes
It seems a few of the error classes are no longer used:
- LVMError, deprecated in
8c5533a5 (before ganeti 1.2.2!)
- ConfdRequestError, deprecated in
b0dcdc10
- SshKeyError, introduced in the initial open source commit but never used (⁈)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Mon, 8 Oct 2012 15:22:30 +0000 (17:22 +0200)]
Typos and grammar fixes
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 4 Oct 2012 22:32:50 +0000 (23:32 +0100)]
Add an Utils.NiceSort() equivalent
This patch adds a NiceSort equivalent and the corresponding unittest
(partially copied from Python unittest). The difference between the
Python version and this one is that this implementation doesn't use
regular expressions, and as such it doesn't have the 8-groups
limitation.
The key-based version is separate from the non-key one (since we don't
have default arguments in Haskell), and is tested less in its absolute
properties but only that it is identical to the non-key version under
some transformations (the non-key version is much more tested).
This will be needed later in query name sorting.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 8 Oct 2012 11:09:29 +0000 (13:09 +0200)]
backend: Use utils.IsBelowDir instead of local code
utils.IsBelowDir is actually tested and doesn't allow writes to
“…/queue*”, like the old code here did.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 8 Oct 2012 10:16:42 +0000 (12:16 +0200)]
jstore: Nicer error message on non-numeric file content
An error like “invalid literal for int() with base 10” can be quite
confusing.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Fri, 5 Oct 2012 21:16:59 +0000 (23:16 +0200)]
Fixed QA for replace-disks
Bug introduced in commit
cc4b14f07: Part of a statement was missing. This
makes pylint-qa fail.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 21:25:01 +0000 (23:25 +0200)]
Merge ganeti-master-cleaner back into ganeti-cleaner
As I wrote during/after the review on commit 2958c56, “ganeti-cleaner:
Separate queue cleaning code”, while I appreciated the permission
separation, I didn't like too much the file-based approach:
- it is a very simple script, and lots of the code is duplicated
between the two; I wouldn't like to see "ganeti-vmcapable-cleaner",
"ganeti-master-candidate-cleaner", etc. in the future
- ganeti-master-cleaner "pollutes" the namespace, creating
tab-completion conflicts with ganeti-masterd
This patch simply merges the master-cleaner back into cleaner, while
keeping the separate user permissions scheme, separate log files, etc.
Additionally, it fixes two bugs in the unit-test (not run with set -u
and wrong path in the master-cleaner log files test; yay for even
worse safety than Python?).
And finally, since we have now support for --help-completion, it adds
bash completion support for this script :) (needs to be applied on top
of my argument support patch series).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 20:45:49 +0000 (22:45 +0200)]
Improve the confd man page
This was a very very thin man page; this patch expands it a bit, but
it could use more help still.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 20:23:09 +0000 (22:23 +0200)]
Split the OptComplNumeric into integer/float ones
This allows the command completion to treat them as separate option
types; this is not useful in build-bash-completion right now, but
might become so in the future.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 20:17:54 +0000 (22:17 +0200)]
Enable bash completion for Haskell daemons too
This requires a few tiny changes to build-bash-completion and to
Makefile rules, but is straightforward.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 20:08:17 +0000 (22:08 +0200)]
Define the actual arguments that are supported by programs
This defines the arguments supported and then modifies the
--help-completion output to include them too.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 19:34:10 +0000 (21:34 +0200)]
Support the new option types in build-bash-completion
Beside parsing the arguments as well, we also change the serialisation
format for choices/suggests, to make the Python code simpler.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 19:31:21 +0000 (21:31 +0200)]
Add support for arguments in --help-completion
This extends the current option-only support with support for
arguments too. They are serialised without a without the prefix (so
just with the type) and the min/max ranges (max="none" for unlimited).
A few new option types have been added as well.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 23:26:39 +0000 (00:26 +0100)]
A few cleanups in Makefile.am
A lot of the lists in Makefile.am were not sorted properly (or at
all); let's sort them for more sanity.
Additionally, check-local used to spew this big shell block, even
though it does emit nice messages when failing, so we don't need to
show the code; let's silence it (@).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 01:46:59 +0000 (02:46 +0100)]
Add missing ipolicy field
This was missing so far…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 2 Oct 2012 21:57:51 +0000 (22:57 +0100)]
Fixup TAGS generation with newer GHC
Newer GHC refuses to allow "-O" with interactive mode, so let's filter
that out. Furthermore, sometimes you don't have a clean tree exactly
when you need to look up something/update the tags, so let's filter
out the "-Werror" too.
And finally, we need to pass the actual exact flags (including
nocurl, parallel, etc.) that we use for building, so let's add those
too.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 21:59:14 +0000 (22:59 +0100)]
Remove qualified import of Utils.hs into its test module
This is a leftover from the times when we had a single, huge test
module; nowadays it's only an annoyance.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 21:39:29 +0000 (22:39 +0100)]
Rename Ganeti/HTools/Utils.hs to Ganeti/Utils.hs
This is, I believe, the last non-htools specific file that still lived
in the htools directory; it's already widely used in non-htools code,
so let's move it before we add more functionality to this module.
All changes are related to the name change, imports fixup, etc.; there
are no other changes in this patch.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 5 Oct 2012 16:36:42 +0000 (18:36 +0200)]
bdev: Remove unnecessary empty line
My local pylint didn't complain.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Bernardo Dal Seno [Fri, 28 Sep 2012 19:22:22 +0000 (21:22 +0200)]
Better list of replace-disks arguments + typos fixed
The man page and the bultin-in help for gnt-instance replace-disks were
inconsistent. Also fixed some typos in man pages.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 23:46:08 +0000 (01:46 +0200)]
Check fingerprint of file with allowed file storage paths
This makes differences show up in “gnt-cluster verify”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Fri, 5 Oct 2012 04:07:27 +0000 (06:07 +0200)]
Document the support for default iallocator
Updated the man pages for the commands that accept an iallocator: "." means
the default iallocator.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Thu, 4 Oct 2012 16:19:45 +0000 (18:19 +0200)]
Instance QA uses default iallocator
QA for recreate-disk and replace-disks now run also with the default
iallocator.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Wed, 19 Sep 2012 16:51:38 +0000 (18:51 +0200)]
Support for the default iallocator in replace-disks
"gnt-instance replace-disks" now behaves like the other commands, and uses
the default iallocator when "." is passed as the iallocator parameter.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Fri, 5 Oct 2012 04:16:30 +0000 (06:16 +0200)]
Explicitly ask for the default iallocator in commands
Now "gnt-instance recreate-disks" uses the default iallocator when "." is
specified as the iallocator. For uniformity, the same behavior applies to
these commands:
gnt-node evacuate
gnt-instance migrate
gnt-instance add
"." is used instead of "default" becuse the latter could be a valid name for
an iallocator.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 14:58:26 +0000 (16:58 +0200)]
bdev: Add functions to verify file storage paths
- LoadAllowedFileStoragePaths: Loads a list of allowed file storage
paths from a file
- CheckFileStoragePath: Checks a path against the list of allowed paths
The unit test for “utils.IsBelowDir” is updated with cases which weren't
tested before.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 5 Oct 2012 02:36:42 +0000 (04:36 +0200)]
jqueue: Look at archived jobs when watching
First: This enables the use of “gnt-job watch $id” for archived jobs.
Now, the reason for actually making this work is that during
sufficiently large group or node evacuations jobs are archived before
the client gets to poll for their output. This led to situations where
the jobs would finish successfully, but the client reported an error
because it couldn't see the job anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
(cherry picked from commit
045694691691af1b10d1d055bdac55cf205e47de)
Michael Hanselmann [Fri, 5 Oct 2012 02:36:42 +0000 (04:36 +0200)]
jqueue: Look at archived jobs when watching
First: This enables the use of “gnt-job watch $id” for archived jobs.
Now, the reason for actually making this work is that during
sufficiently large group or node evacuations jobs are archived before
the client gets to poll for their output. This led to situations where
the jobs would finish successfully, but the client reported an error
because it couldn't see the job anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Bernardo Dal Seno [Tue, 4 Sep 2012 15:04:48 +0000 (16:04 +0100)]
Design doc for partitioned Ganeti
Partitioned Ganeti allows to assign node resources in an exclusive manner,
in order to limit interference between instances running on the same nodes.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Fri, 5 Oct 2012 00:52:37 +0000 (02:52 +0200)]
backend: Check for shared storage also
If normal file storage was disabled but shared storage enabled,
“_TransformFileStorageDir” would still throw an exception.
in “opcodes._CheckStorageType” there's also a check, but I wasn't quite
sure what the correct way of handling it was, so I added a TODO comment.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 17:58:45 +0000 (19:58 +0200)]
utils.FilterEmptyLinesAndComments: Return list
We don't use generators often and lists are easier to re-use.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 17:37:50 +0000 (19:37 +0200)]
Improve disk wipe unit test
- Don't hardcode node name in some places
- Don't define functions inside functions
- Simplify code for testing with and without offset, this is now in two
separate tests
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 02:59:29 +0000 (04:59 +0200)]
Wipe added space when growing disks
This patch adds code to wipe newly added disk space when growing disks
using “gnt-instance grow-disk”. “New disk space” is defined as the delta
between the old block device size (not necessarily equal to the amount
recorded in the configuration) and the new recorded size. Extra caution
is taken to avoid overwriting existing data. Unit tests are included.
A small typo in gnt-cluster(8) (“it's”) is also fixed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 18:27:45 +0000 (20:27 +0200)]
Add unit tests for cmdlib._WipeDisks
This is in preparation for adding disk wipe on growing disks.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 13:55:57 +0000 (15:55 +0200)]
Add unit test for FilterEmptyLinesAndComments
I somehow forgot this in the previous patch.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 18:26:53 +0000 (20:26 +0200)]
cmdlib._WipeDisks: Code formatting
- LogInfo takes *args, no need to replace values right away
- Don't overwrite wipe_chunk_size right after it's been set
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 01:50:47 +0000 (03:50 +0200)]
Factorize removing comments and empty lines from string
This will also be used for verifying the file storage directory.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 01:53:04 +0000 (03:53 +0200)]
ssconf: Fix mistake made in commit ee501db
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 01:30:09 +0000 (03:30 +0200)]
Move a function from backend to ssconf
The “WriteSsconfFiles” function is used to write ssconf files. By moving
it we can avoid importing backend into bootstrap. The latter is imported
by CLI programs and backend doesn't have much to do with them.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 00:15:55 +0000 (02:15 +0200)]
Show old primary/secondary node on disk replacement
People unfamiliar with Ganeti's internals might be confused with the
different hostnames showing up later in the process.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 02:00:08 +0000 (04:00 +0200)]
cmdlib: Change wording of messages during disk wipe
Error messages don't need to say “please” and it's already obvious some
investigation is needed. LogWarning already logs the message using
“logging.error” internally.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 02:12:43 +0000 (04:12 +0200)]
Remove constant for disk wipe block size
It is dangerous to have this block size as a global constant as that
could give the impression of it being easily changed. Doing so without
further adjustments to how “dd” is called will lead to disks not being
wiped properly.
Two additional parameter checks are added to “backend.BlockdevWipe”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 01:41:11 +0000 (03:41 +0200)]
gnt-instance reinstall: Don't always exit with success
If one or more jobs failed the exit status should be set accordingly.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Tue, 2 Oct 2012 18:41:19 +0000 (20:41 +0200)]
LUClusterVerify: Ignore /proc/drbd if DRBD is disabled
This fixes issue 190. The problem was that the check for DRBD was
enabled if LVM storage is used and didn't depend at all on whether DRBD
is enabled.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit
3d8ae32740475f35ac2e7a4937f5469700202e5b)
Michael Hanselmann [Tue, 2 Oct 2012 18:41:19 +0000 (20:41 +0200)]
LUClusterVerify: Ignore /proc/drbd if DRBD is disabled
This fixes issue 190. The problem was that the check for DRBD was
enabled if LVM storage is used and didn't depend at all on whether DRBD
is enabled.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 2 Oct 2012 05:31:33 +0000 (07:31 +0200)]
cli: Exit with status 0 for --help
This patch is somewhat longer than was anticipated. Before, commands like
“gnt-instance --help” would exit with the status code 1 (failure). With
this patch, those commands exits with 0 (success) while unknown commands
still print the usage screen and exit with status 1.
To achieve this goal without too much ugly code some refactoring on
cli._ParseArgs was necessary. Printing the version and usage screen was
separated. At the same time this allows for some unit testing.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Fri, 28 Sep 2012 13:00:35 +0000 (15:00 +0200)]
Remove the python confd server side code
In 2.7 we will only support the Haskell version, if enabled.
Since the original hconfd enabling was a bit hack-ish (copying over
the actual installed ganeti-confd, Python version), the Makefile.am
changes are a bit more involved than just the removal of the Python
code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Fri, 28 Sep 2012 12:48:18 +0000 (14:48 +0200)]
Remove option to select the Python version of confd
This will need some more configure.ac checks for required libraries,
which will come later with the Haskell dependencies reorganisation.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 1 Oct 2012 05:50:36 +0000 (07:50 +0200)]
Add first version of virtual cluster documentation
- Add document describing virtual clusters
- Non-root clusters are not yet described, will be done in another
patch
- Change title of design document to avoid confusion
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Tue, 2 Oct 2012 02:56:53 +0000 (04:56 +0200)]
INSTALL: Typo, missing dependency
libghc-utf8-string-dev was missing.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>