Iustin Pop [Wed, 10 Oct 2012 11:45:07 +0000 (13:45 +0200)]
Fix pylint breakage due to unused var in gnt_cluster
The usage of that variable was removed in
45a36f36, but accidentally
the enumerate() was left in.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 02:09:57 +0000 (03:09 +0100)]
Fixup hypervisor queries in node query
We need to only query the default (first enabled) hypervisor, not all
hypervisors. For this, we need to add a manual check to ensure that we
don't have a corrupt config (there's no "NonEmptyList" type…).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 02:08:56 +0000 (03:08 +0100)]
Fixup node live field names
The String parameter to 'nodeLiveFieldExtract' is the query2 field
name, not the RPC-layer field name. Grrr for not having a real data
type for this.
Furthermore, we add some safety check that we don't return JSNull via
rsNormal…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Iustin Pop [Wed, 10 Oct 2012 05:12:43 +0000 (07:12 +0200)]
Improve Haskell configure options and detection
This patch cleans up the Haskell library detection and defaults.
First, it makes the base compiler/libraries required, per the email discussion.
It then adds two new small autoconf macros, on to check for a required
Haskell library and one to do custom action based on test results. We
use these macros to cleanup and simplify a bit the module detection:
- rapi, confd, and split query are auto detected and enabled if _all_
required libraries are present
- unittests are enabled if _all_ required libraries are present
The patch also updates the documentation regarding required libraries.
After this patch, base Ganeti fully buildable on Debian Squeeze/Ubuntu
Lucid.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 10 Oct 2012 10:17:58 +0000 (12:17 +0200)]
Make regex-pcre an optional dependency
This patch makes regex-pcre optional, allowing its disable via a
preprocessor define NO_REGEX_PCRE. This define will be added to
config.ac/Makefile.am in the next patch.
The patch also changes multi-line strings into string concatenation,
due to limitations with the CPP language extension.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 11:16:30 +0000 (13:16 +0200)]
cluster epo: Fix bug where IndexError is raised
Updating the “node_query_list” variable would fail if no arguments were
passed and the “--all” option wasn't specified. A follow-up patch will
add unit tests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 2 Oct 2012 23:47:33 +0000 (00:47 +0100)]
Try to auto-enable htools-rapi and split query
We try to automatically enable the htools-rapi and split query (if
confd and htools-rapi are enabled) options. This is our intended
default configuration, and allows easier test of the new code
path. Further cleanups for checking whether confd can be enabled will
come later.
The move block is due to the fact that we first have to check for
htools-rapi, and only then we can auto-enable the feature.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 9 Oct 2012 10:38:50 +0000 (12:38 +0200)]
Fix usage of errors.ResolverError
This exception is documented to have three arguments, but in one case
we raise it with a simple string argument. Let's fix that.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 8 Oct 2012 15:44:01 +0000 (17:44 +0200)]
Remove unused/deprecated error classes
It seems a few of the error classes are no longer used:
- LVMError, deprecated in
8c5533a5 (before ganeti 1.2.2!)
- ConfdRequestError, deprecated in
b0dcdc10
- SshKeyError, introduced in the initial open source commit but never used (⁈)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Helga Velroyen [Mon, 8 Oct 2012 15:22:30 +0000 (17:22 +0200)]
Typos and grammar fixes
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 4 Oct 2012 22:32:50 +0000 (23:32 +0100)]
Add an Utils.NiceSort() equivalent
This patch adds a NiceSort equivalent and the corresponding unittest
(partially copied from Python unittest). The difference between the
Python version and this one is that this implementation doesn't use
regular expressions, and as such it doesn't have the 8-groups
limitation.
The key-based version is separate from the non-key one (since we don't
have default arguments in Haskell), and is tested less in its absolute
properties but only that it is identical to the non-key version under
some transformations (the non-key version is much more tested).
This will be needed later in query name sorting.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 8 Oct 2012 11:09:29 +0000 (13:09 +0200)]
backend: Use utils.IsBelowDir instead of local code
utils.IsBelowDir is actually tested and doesn't allow writes to
“…/queue*”, like the old code here did.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 8 Oct 2012 10:16:42 +0000 (12:16 +0200)]
jstore: Nicer error message on non-numeric file content
An error like “invalid literal for int() with base 10” can be quite
confusing.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Fri, 5 Oct 2012 21:16:59 +0000 (23:16 +0200)]
Fixed QA for replace-disks
Bug introduced in commit
cc4b14f07: Part of a statement was missing. This
makes pylint-qa fail.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 21:25:01 +0000 (23:25 +0200)]
Merge ganeti-master-cleaner back into ganeti-cleaner
As I wrote during/after the review on commit 2958c56, “ganeti-cleaner:
Separate queue cleaning code”, while I appreciated the permission
separation, I didn't like too much the file-based approach:
- it is a very simple script, and lots of the code is duplicated
between the two; I wouldn't like to see "ganeti-vmcapable-cleaner",
"ganeti-master-candidate-cleaner", etc. in the future
- ganeti-master-cleaner "pollutes" the namespace, creating
tab-completion conflicts with ganeti-masterd
This patch simply merges the master-cleaner back into cleaner, while
keeping the separate user permissions scheme, separate log files, etc.
Additionally, it fixes two bugs in the unit-test (not run with set -u
and wrong path in the master-cleaner log files test; yay for even
worse safety than Python?).
And finally, since we have now support for --help-completion, it adds
bash completion support for this script :) (needs to be applied on top
of my argument support patch series).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 20:45:49 +0000 (22:45 +0200)]
Improve the confd man page
This was a very very thin man page; this patch expands it a bit, but
it could use more help still.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 20:23:09 +0000 (22:23 +0200)]
Split the OptComplNumeric into integer/float ones
This allows the command completion to treat them as separate option
types; this is not useful in build-bash-completion right now, but
might become so in the future.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 20:17:54 +0000 (22:17 +0200)]
Enable bash completion for Haskell daemons too
This requires a few tiny changes to build-bash-completion and to
Makefile rules, but is straightforward.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 20:08:17 +0000 (22:08 +0200)]
Define the actual arguments that are supported by programs
This defines the arguments supported and then modifies the
--help-completion output to include them too.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 19:34:10 +0000 (21:34 +0200)]
Support the new option types in build-bash-completion
Beside parsing the arguments as well, we also change the serialisation
format for choices/suggests, to make the Python code simpler.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 5 Oct 2012 19:31:21 +0000 (21:31 +0200)]
Add support for arguments in --help-completion
This extends the current option-only support with support for
arguments too. They are serialised without a without the prefix (so
just with the type) and the min/max ranges (max="none" for unlimited).
A few new option types have been added as well.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 23:26:39 +0000 (00:26 +0100)]
A few cleanups in Makefile.am
A lot of the lists in Makefile.am were not sorted properly (or at
all); let's sort them for more sanity.
Additionally, check-local used to spew this big shell block, even
though it does emit nice messages when failing, so we don't need to
show the code; let's silence it (@).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 01:46:59 +0000 (02:46 +0100)]
Add missing ipolicy field
This was missing so far…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 2 Oct 2012 21:57:51 +0000 (22:57 +0100)]
Fixup TAGS generation with newer GHC
Newer GHC refuses to allow "-O" with interactive mode, so let's filter
that out. Furthermore, sometimes you don't have a clean tree exactly
when you need to look up something/update the tags, so let's filter
out the "-Werror" too.
And finally, we need to pass the actual exact flags (including
nocurl, parallel, etc.) that we use for building, so let's add those
too.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 21:59:14 +0000 (22:59 +0100)]
Remove qualified import of Utils.hs into its test module
This is a leftover from the times when we had a single, huge test
module; nowadays it's only an annoyance.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Thu, 4 Oct 2012 21:39:29 +0000 (22:39 +0100)]
Rename Ganeti/HTools/Utils.hs to Ganeti/Utils.hs
This is, I believe, the last non-htools specific file that still lived
in the htools directory; it's already widely used in non-htools code,
so let's move it before we add more functionality to this module.
All changes are related to the name change, imports fixup, etc.; there
are no other changes in this patch.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 5 Oct 2012 16:36:42 +0000 (18:36 +0200)]
bdev: Remove unnecessary empty line
My local pylint didn't complain.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 23:46:08 +0000 (01:46 +0200)]
Check fingerprint of file with allowed file storage paths
This makes differences show up in “gnt-cluster verify”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Fri, 5 Oct 2012 04:07:27 +0000 (06:07 +0200)]
Document the support for default iallocator
Updated the man pages for the commands that accept an iallocator: "." means
the default iallocator.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Thu, 4 Oct 2012 16:19:45 +0000 (18:19 +0200)]
Instance QA uses default iallocator
QA for recreate-disk and replace-disks now run also with the default
iallocator.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Wed, 19 Sep 2012 16:51:38 +0000 (18:51 +0200)]
Support for the default iallocator in replace-disks
"gnt-instance replace-disks" now behaves like the other commands, and uses
the default iallocator when "." is passed as the iallocator parameter.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Fri, 5 Oct 2012 04:16:30 +0000 (06:16 +0200)]
Explicitly ask for the default iallocator in commands
Now "gnt-instance recreate-disks" uses the default iallocator when "." is
specified as the iallocator. For uniformity, the same behavior applies to
these commands:
gnt-node evacuate
gnt-instance migrate
gnt-instance add
"." is used instead of "default" becuse the latter could be a valid name for
an iallocator.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 14:58:26 +0000 (16:58 +0200)]
bdev: Add functions to verify file storage paths
- LoadAllowedFileStoragePaths: Loads a list of allowed file storage
paths from a file
- CheckFileStoragePath: Checks a path against the list of allowed paths
The unit test for “utils.IsBelowDir” is updated with cases which weren't
tested before.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 5 Oct 2012 02:36:42 +0000 (04:36 +0200)]
jqueue: Look at archived jobs when watching
First: This enables the use of “gnt-job watch $id” for archived jobs.
Now, the reason for actually making this work is that during
sufficiently large group or node evacuations jobs are archived before
the client gets to poll for their output. This led to situations where
the jobs would finish successfully, but the client reported an error
because it couldn't see the job anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Bernardo Dal Seno [Tue, 4 Sep 2012 15:04:48 +0000 (16:04 +0100)]
Design doc for partitioned Ganeti
Partitioned Ganeti allows to assign node resources in an exclusive manner,
in order to limit interference between instances running on the same nodes.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Fri, 5 Oct 2012 00:52:37 +0000 (02:52 +0200)]
backend: Check for shared storage also
If normal file storage was disabled but shared storage enabled,
“_TransformFileStorageDir” would still throw an exception.
in “opcodes._CheckStorageType” there's also a check, but I wasn't quite
sure what the correct way of handling it was, so I added a TODO comment.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 17:58:45 +0000 (19:58 +0200)]
utils.FilterEmptyLinesAndComments: Return list
We don't use generators often and lists are easier to re-use.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 17:37:50 +0000 (19:37 +0200)]
Improve disk wipe unit test
- Don't hardcode node name in some places
- Don't define functions inside functions
- Simplify code for testing with and without offset, this is now in two
separate tests
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 02:59:29 +0000 (04:59 +0200)]
Wipe added space when growing disks
This patch adds code to wipe newly added disk space when growing disks
using “gnt-instance grow-disk”. “New disk space” is defined as the delta
between the old block device size (not necessarily equal to the amount
recorded in the configuration) and the new recorded size. Extra caution
is taken to avoid overwriting existing data. Unit tests are included.
A small typo in gnt-cluster(8) (“it's”) is also fixed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 18:27:45 +0000 (20:27 +0200)]
Add unit tests for cmdlib._WipeDisks
This is in preparation for adding disk wipe on growing disks.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 13:55:57 +0000 (15:55 +0200)]
Add unit test for FilterEmptyLinesAndComments
I somehow forgot this in the previous patch.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 18:26:53 +0000 (20:26 +0200)]
cmdlib._WipeDisks: Code formatting
- LogInfo takes *args, no need to replace values right away
- Don't overwrite wipe_chunk_size right after it's been set
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 01:50:47 +0000 (03:50 +0200)]
Factorize removing comments and empty lines from string
This will also be used for verifying the file storage directory.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 01:53:04 +0000 (03:53 +0200)]
ssconf: Fix mistake made in commit ee501db
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 4 Oct 2012 01:30:09 +0000 (03:30 +0200)]
Move a function from backend to ssconf
The “WriteSsconfFiles” function is used to write ssconf files. By moving
it we can avoid importing backend into bootstrap. The latter is imported
by CLI programs and backend doesn't have much to do with them.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 02:00:08 +0000 (04:00 +0200)]
cmdlib: Change wording of messages during disk wipe
Error messages don't need to say “please” and it's already obvious some
investigation is needed. LogWarning already logs the message using
“logging.error” internally.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 3 Oct 2012 02:12:43 +0000 (04:12 +0200)]
Remove constant for disk wipe block size
It is dangerous to have this block size as a global constant as that
could give the impression of it being easily changed. Doing so without
further adjustments to how “dd” is called will lead to disks not being
wiped properly.
Two additional parameter checks are added to “backend.BlockdevWipe”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Tue, 2 Oct 2012 18:41:19 +0000 (20:41 +0200)]
LUClusterVerify: Ignore /proc/drbd if DRBD is disabled
This fixes issue 190. The problem was that the check for DRBD was
enabled if LVM storage is used and didn't depend at all on whether DRBD
is enabled.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 2 Oct 2012 05:31:33 +0000 (07:31 +0200)]
cli: Exit with status 0 for --help
This patch is somewhat longer than was anticipated. Before, commands like
“gnt-instance --help” would exit with the status code 1 (failure). With
this patch, those commands exits with 0 (success) while unknown commands
still print the usage screen and exit with status 1.
To achieve this goal without too much ugly code some refactoring on
cli._ParseArgs was necessary. Printing the version and usage screen was
separated. At the same time this allows for some unit testing.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Fri, 28 Sep 2012 13:00:35 +0000 (15:00 +0200)]
Remove the python confd server side code
In 2.7 we will only support the Haskell version, if enabled.
Since the original hconfd enabling was a bit hack-ish (copying over
the actual installed ganeti-confd, Python version), the Makefile.am
changes are a bit more involved than just the removal of the Python
code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Fri, 28 Sep 2012 12:48:18 +0000 (14:48 +0200)]
Remove option to select the Python version of confd
This will need some more configure.ac checks for required libraries,
which will come later with the Haskell dependencies reorganisation.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 1 Oct 2012 05:50:36 +0000 (07:50 +0200)]
Add first version of virtual cluster documentation
- Add document describing virtual clusters
- Non-root clusters are not yet described, will be done in another
patch
- Change title of design document to avoid confusion
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Tue, 2 Oct 2012 02:56:53 +0000 (04:56 +0200)]
INSTALL: Typo, missing dependency
libghc-utf8-string-dev was missing.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Bernardo Dal Seno [Mon, 1 Oct 2012 15:41:03 +0000 (17:41 +0200)]
Always distribute ganeti.utils.io_unittest-runasroot.py
Tests that depend on fakeroot were not distributed in the tarball created
by "make dist" when fakeroot was missing. Now all tests are distributed,
and a warning is printed when a test requires fakeroot but fakeroot is
missing.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 1 Oct 2012 05:49:37 +0000 (07:49 +0200)]
vcluster-setup: Watcher, init script
- Generate a wrapper script for ganeti-watcher
- Provide command line option to specify init script
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Thu, 27 Sep 2012 15:56:58 +0000 (17:56 +0200)]
vcluster-setup: Add more parameters to "gnt-cluster init"
This patch also adds two new parameters to the vcluster-setup script to
not configure /etc/hosts and networking. Build-time variables are now
used for paths.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 28 Sep 2012 13:31:59 +0000 (15:31 +0200)]
Pylint cleanup
The previous commits introduced some pylint errors.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 21 Sep 2012 04:07:12 +0000 (06:07 +0200)]
Add support for GANETI_ROOTDIR in Haskell code
unsafePerformIO is required to go from the IO monad to pure code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 28 Sep 2012 12:03:41 +0000 (14:03 +0200)]
Modify batch-create to use MultiAlloc
This is the CLI part of the OpInstanceMultiAlloc change. There is some
room for improvement especially with user handling, but it works for now
and the code got simplified significantly.
QA tests still needs to be written.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 28 Sep 2012 09:22:08 +0000 (11:22 +0200)]
Make validation error message more useful
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Fri, 28 Sep 2012 09:17:08 +0000 (11:17 +0200)]
Provide a custom Validate for OpInstanceMultiAlloc
This is needed, as we're not strictly separating validation from
populating the defaults. We do this all in one function. If we now
receive a non complete OpInstanceCreate opcode in instances, we would
run into AttributeErrors on those.
So we run the validation + recursively on all instances opcodes.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 27 Sep 2012 15:23:38 +0000 (17:23 +0200)]
ganeti-cleaner: Separate queue cleaning code
This code does not need to run as root, therefore it's better to split
it out. It is now run with the same permissions as the master daemon.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 26 Sep 2012 16:41:10 +0000 (18:41 +0200)]
Stop hardcoding root user
Some parts of the code still use a hardcoded user name: root. This patch
replaces all with a constant specified at build time. The end goal is to
make it possible to run a Ganeti cluster without any special privileges
(of course this will prevent some functionality from working).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 27 Sep 2012 14:10:58 +0000 (16:10 +0200)]
Remove explicit recursion from sanitizeDictResults
Since ERpcError is still not a monad, a simple foldr will have to
do for now.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 27 Sep 2012 15:19:35 +0000 (17:19 +0200)]
Pretty printing JSValue
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 27 Sep 2012 15:27:18 +0000 (17:27 +0200)]
Merge parseHttpResponse with rpcResultParse
parseHttpResponse is such a simple function that we can actually
do the entire unpacking in rpcResultParse.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 27 Sep 2012 13:45:05 +0000 (15:45 +0200)]
Simplify and generalize rpcResultFill
For now ERpcError is still not a monad, but we can still simplify
rpcResultFill implementation for different calls simply by abstracting
it into two helpers.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 27 Sep 2012 12:46:52 +0000 (14:46 +0200)]
Remove monadic results wherever possible
There is no need (at least right now) for some of the resulting
types to be in additional monad, so let us just have more
"normal" functions.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 27 Sep 2012 10:58:12 +0000 (12:58 +0200)]
Realign instances of typeclasses
As per Iustin's suggestion, this patch aligns parts of typeclass
instance on "=".
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 27 Sep 2012 08:56:36 +0000 (10:56 +0200)]
Rename RpcError -> String function
As per Iustin's suggestion, show is not a good solution - so instead
the name is now explainRpcError.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 27 Sep 2012 14:15:27 +0000 (16:15 +0200)]
Cleanup of build-time shell variable replacements
Instead of having a different set in (almost) every shell script, this
inserts the most commonly used variables at build time. This way the
code for injecting a root directory for virtual clusters also is just
needed once.
I chose inserting the variables using “sed” at build time over a runtime
script as the latter would need to be located from within the script and
the path would be different depending on the environment (unittest vs.
actual installation).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 27 Sep 2012 15:55:50 +0000 (17:55 +0200)]
ensure-dirs: Don't hardcode ssconf file group
Otherwise chown(2) will fail when noded doesn't run as root.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 27 Sep 2012 16:04:31 +0000 (18:04 +0200)]
Fix distcheck after commit
29386d3ee
Missing backslash.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Thu, 20 Sep 2012 20:09:59 +0000 (22:09 +0200)]
InstanceRecreateDisks: new assertion for node locks
Make sure that all the nodes where a disk operation is peformed are locked.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Thu, 27 Sep 2012 01:35:57 +0000 (03:35 +0200)]
Fix missing locks in InstanceRecreateDisks
In case of in-place recreate-disks operation, all node locks were released
before starting the actual execution.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 20 Sep 2012 16:53:01 +0000 (18:53 +0200)]
Add vcluster-setup utility
This utility can be used to configure an environment for a virtual
cluster. It sets up entries in /etc/hosts, creates the necessary
directory structure, and generates helper scripts.
Documentation for virtual clusters will come in a later patch.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 27 Sep 2012 14:42:33 +0000 (16:42 +0200)]
ensure-dirs: Don't convert list to tuple
Tuples are data structures, not containers.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 27 Sep 2012 14:28:29 +0000 (16:28 +0200)]
daemon-util: Use function to determine if confd is enabled
… instead of comparing with two different values in two placse.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Michael Hanselmann [Thu, 27 Sep 2012 12:52:38 +0000 (14:52 +0200)]
cfgshell: Remove pylint disable line
Commit
326830963 removed the use of a private member function. Also
replace '' with "" in one place.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
René Nussbaumer [Wed, 26 Sep 2012 12:47:41 +0000 (14:47 +0200)]
Adding RAPI client for instance multi allocation
It was very easy to refactor CreateInstance to repurpose some part of it
for the instance multi allocation request. So we do this.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
René Nussbaumer [Thu, 27 Sep 2012 08:06:25 +0000 (10:06 +0200)]
Populate the instance dict with opcode infos
This is needed so we can load it as an opcode. However, this information
should not be available on the client, so we inject that info when we
receive the request.
Provided unittest to verify behaviour.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Tue, 25 Sep 2012 13:35:40 +0000 (15:35 +0200)]
Revert unintentional change of daemon log file names
Commit
3329f4dea6 unintentionally changed the filenames of all daemon
log files. This patch reverts part of those changes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 26 Sep 2012 17:05:05 +0000 (19:05 +0200)]
Fix bad wording in 2.3 design document
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Tue, 18 Sep 2012 08:20:56 +0000 (10:20 +0200)]
InstanceInfo RPC call
This patch implements single instance info call - somewhat similar to
all_instances_info, except we give a specific instance name.
Current implementation of reading the InstanceInfo value is somewhat
counter-intuitive because when we query a node on thich there is
no instance with such name, this is not considered an error - but
it does produce an empty answer. Therefore the resulting type is
optional.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Tue, 18 Sep 2012 08:20:06 +0000 (10:20 +0200)]
instance_info now returns vcpus
instance_info and all_instances_info calls had different per-instance
return types, this commit fixes the mismatch.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Mon, 24 Sep 2012 11:51:10 +0000 (13:51 +0200)]
Change RpcResult typeclass
For storage_list call, the result type depends on the call parameters.
Therefore, we have to add call as an argument for rpcResultFill - and
by extension, to the typeclass.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Fri, 21 Sep 2012 09:11:03 +0000 (11:11 +0200)]
Version RPC call
Implementation of node version query.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Mon, 24 Sep 2012 09:09:05 +0000 (11:09 +0200)]
Node query now uses live data
Added support for NodeInfo RPC call as source of data for node query.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Wed, 26 Sep 2012 15:21:26 +0000 (17:21 +0200)]
Node query now collects live fields
We make use of the parameter added in the previous patch and add
option to add live parameters to the query.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Wed, 26 Sep 2012 13:00:41 +0000 (15:00 +0200)]
Add live parameter to query
The tests we currently have assume, that all the data required for
running the query is available - once we add live data, this will no
longer be the case.
This patch adds boolean parameter to query function, which tells it
whether to ignore live parameters gathering.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Mon, 24 Sep 2012 13:48:18 +0000 (15:48 +0200)]
Conversion from RPC error to query status
Simple conversion for queries that use live fields gathered from
RPC calls.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Mon, 24 Sep 2012 13:45:57 +0000 (15:45 +0200)]
Json helper for node query
This is a helper function that gets the JSON encoding of the first
element of the list (if possible).
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Fri, 21 Sep 2012 08:44:07 +0000 (10:44 +0200)]
Create ResulT for monad transformations
This patch creates ResultT, a monad transformation used later in
RPC <-> query integration.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 20 Sep 2012 15:16:53 +0000 (17:16 +0200)]
Tests for NodeGroup query
This adds tests similar to those used for node query. For now the
prop_queryGroup_noUnknown is disabled and commented out, as it is
fasifiable with ndparams and ipolicy. It may be removed or fixed
later on.
Also, prop_queryGrooup_types has one less property checked - it is
not the case that number of result rows should be equal to number
of nodes.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 20 Sep 2012 13:16:21 +0000 (15:16 +0200)]
NodeGroup query in Haskell
Implementation of nodegroup queries in Haskell. This is not yet
complete as we are missing merged disk parameters and option
want_diskparams is not implemented.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Thu, 20 Sep 2012 13:15:11 +0000 (15:15 +0200)]
Config support for NodeGroup query
For some results in nodegroup queries we need agregation similar to
one that is done for nodes.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Fri, 21 Sep 2012 09:25:31 +0000 (11:25 +0200)]
Fix RPC call and result (de)serialization
Previous version of RPC calls implementation in Haskell did not take
into account that the actual result type for queries is a list, not
a dictionary.
This patch aims at fixing the problem "for now" - it is not a pretty
solution, but it does work. Note that parsing of the result is now
split into two parts - first, we check if server's aswer is positive,
then if it is, we procede with decoding the actual result.
Values and order of some fields in the result type were changed to
reflect actual order of arguments from server responses.
AllInstancesInfo call was particularly tricky, because it returns a
dictionary where keys are instance names - and the response from
a given node is correct if all the instances were deserialized, not
just some.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Tue, 25 Sep 2012 15:18:07 +0000 (17:18 +0200)]
Add alias for Either RpcError a
This was repeated a lot, so we should abstract this into a type.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Fri, 21 Sep 2012 09:20:34 +0000 (11:20 +0200)]
Cleanup for RPC errors
Curl error messages are self contained and don't need "error" prefix;
added RpcResult Error which will be used once proper deserialization
of rpc response is in place.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Agata Murawska [Fri, 21 Sep 2012 08:44:41 +0000 (10:44 +0200)]
Change cluster's hypervisors type
We now have Hypervisor type in Objects, might just as well use it.
Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>