Add a default `.ghci' file
This options file for ghci preseeds the correct include paths, so thatinteractive sessions don't need to always pass these args.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Convert query path from string errors to GanetiException
This patch converts all the call paths from 'Result' (which containsjust string errors) to 'ErrorResult', which holdsGanetiException-encoded errors. We can now return properOpPrereq/OpExec errors to the clients of the luxi/query socket....
Add exception utility functions
In Python, formatError also returns the exit code, but I find thatsplitting them leads to clearer code.
Add an Errors module mirroring the Python one
As described in the module doc string, while writing this it dawnedupon me that we're mixing all errors together into a single hierarchy(well, type on the Haskell side), which is not good. Some errors areused purely within noded, some in the CLI frontends, etc. so these...
Abstract Luxi template functionality
These are almost generic, so let's change the signatures a bit a makethem fully so.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
Abstract a few types in THH.hs
This 'simple' way of defining objects will be used also for errors, solet's make it less Luxi-specific.
Add missing empty line
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove dumb-allocator code from devel/upload
The 'dumb-allocator' has been removed almost two years ago (commit6f547f96, “Remove dumb-allocator”), let's remove this special casingfor it from devel/upload.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Fix devel/upload restart of daemons
While running with a wrong --prefix/--sysconfdir, I saw thatdevel/upload actually uses a hardcoded path for the init script, eventhough it installs it in the correct place.
Signed-off-by: Iustin Pop <iustin@google.com>...
Add small design for Linux HA integration
This documents the status (or wanted status) of some example ocf modulesI've written for Ganeti. They are far from perfect, but they should beshipped as a starting point for other people who want to run this to...
Replace LIBDIR in .in files
LIBDIR
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Improve devel/upload command line handling
To people not used to it, it was completely non-obvious why"./devel/upload" didn't do anything.
Improve logging of AssertionErrors
Currently, when we have an assertion error raised from cmdlib, it looks like this:
[cluster] root@node4:~# gnt-instance grow-disk instance1 0 1G Failure: command execution error:
This is very very confusing. This patch adds a bit of traceback...
tools.prepare_node_join: Fix pep8 errors
Pep8 didn't agree with the indentation.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add initial implementation of prepare-node-join
This is a new tool as per the design document “design-ssh-setup”. Itreceives a JSON data structure on its standard input and configures theSSH daemon and root's SSH keys accordingly. Unit tests are included....
ssh.GetUserFiles: RSA support, unit tests
This patch changes “ssh.GetUserFiles” to support two different kinds ofSSH keys, RSA and DSA. Before it would always use DSA. Newly writtenunit tests are included.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Fix typo in walkthrough document
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Find coverage during configure, issue with Debian package
- Debian Squeeze and up have a package named “python-coverage”, but it doesn't use the same binary name as upstream (“coverage”).- Said package includes a patch to use symlinks instead of file copies...
Remove multiple uses of '.&&.' with conjoin
This is just a bit of cleanup. The (.&&.) operator is internally just:
a .&& b = conjoin [a, b]
so let's replace 'a .&&. b .&&. c .&&. d' directly with 'conjoin [a,b, c, d]'.
Improve message for (==?) operator
After seeing how nice HUnit formats the error message on failed'assertEqual', I think we can do better with ==?. Currently it says(on one line): "Expected equality, but 1 /= 2".
This patch changes the code to format it similar to HUnit:...
Add a new 'really-all' make target
This is used when one wants to build all binaries, including thosethat are used only for testing. A handy shortcut to make sure allbinaries can be built.
Annotated inequality operator for unit test properties
This includes: * The operator (/=?), which checks for inequality and prints an error message if it encounters equality. (Basically the negation of the (==?) operator). * Application of this operator in the test property...
Fix setting of 'failN1' flag for corner case
This patch includes:
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
A few unittests improvements
Small simplifications of other unit tests using the (==?)operator when possible, and typo fixes.
Corrected some commandlines in walkthrough
BasicTypes.hs: fix docstring for eitherToResult
eitherToResult now converts from `Either a b` to `GenericResult`, notnecessarily from `Either String` only. Also, fix a typo.
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add note about running individual test cases
This was asked a few times offline, so let's document it.
Add a small QA check in instance renames for tags update
We also need to change the signature for _GetInstanceInfo, since wedon't have access to the instance dictionary in the rename instancetest.
Update blockdev's "info" at instance rename
Currently, we set "info" metadata on block devices at device creationtime, but we never update it, leading to stale data in case ofinstance renames. This would not be a big problem in case of regularrenames (assuming this is a rare operation), but importing instances...
LVM: remove old tags when adding new ones
This patch adds a small helper function to clear an LV's tags, andcalls it at SetInfo time. We need this to be able to correctly trackinstance renames, once we will call SetInfo at such times.
Add a small bdev helper function
I wanted to write that snippet the third time, which is too much :)
Add a small note about tab completion and generated files
This can indeed be annoying, so let's document it.
Update for SSH design: Cluster name, ignore node daemon
While writing the initial version of this design I misunderstood what“setup-ssh” does. It doesn't actually start the node daemon and justconfigures SSH, nothing else.
This patch adds the cluster name as a field to the JSON structure and...
Compare significant fields only for simple SSH keys
For simple SSH keys, that is those without options such as“command="…"”, only the first two parts need to be compared. The thirdfield is a free-form comment.
This patch changes the comparison used in...
Add instance uptime to monitoring info
This can be used to detect a reboot between two checks.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
test/*.py: Replace '' with ""
There might be more, but at least replace all these low-hanging fruits.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add cluster monitoring agent design document
This design addresses the lack of a uniform way to query ganeti nodesfor real time information that can be used by monitoring.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove custom OpResult type/monad
Since we now have the GeneralResult as a multi-purpose monad, we canremove the custom OpResult monad, and just use 'GeneralResultFailMode' as our type. This allows removal of a few bits ofspecialised infrastructure, relying instead on the generic one....
Merge branch 'devel-2.6'
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
ensure-dirs: Don't accept arguments
Before they would just be silently ignored.
ensure-dirs: Fix program name on usage screen
No string replacements are used, so doubling of the percent sign is notnecessary.
Before: Usage: %ensure-dirs [--full-run]After: Usage: ensure-dirs [--full-run]
Convert man page highlighting to standard RST
Instead of using the sphinx-specific highlight extension, which is notparsed by Pandoc, let's switch to the standard RST directive, whichwill be picked up and will result in slightly nicer man pages whenconverted to HTML (in man output it remains the same)....
Convert two more 'sh' highlight styles to 'shell-example'
This is straightforward; only one shell example remained, but as weprocess the man pages via pandoc and not sphinx, it's not actuallyused, we leave it as such for now.
Small improvements to the move-instance doc
While reading the docs, I saw that this is not converted to the shelllexer, and that a few other small improvements can be done.
Use exitErr instead of explicit error message and exitWith
Furthermore, a few messages have their capitalisation changed (fixed).
Signed-off-by: Helga Velroyen <helgav@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix running of Haskell tests
Commit 21a5e56c forgot to rename a variable used in a conditional (ofcourse shell didn't complain about unused vars), so the AM_CONDITIONALwas always false.
Generalise the Result type
Currently, our error monad—Result—has a plain string error type. Thisis not good, as we don't have structured errors, we can't pass backproper error information to Python code, etc.
To solve this, we generalise this type as 'GenericResult a', and make...
Merge branch 'devel-2.6' into master
cli: Fix small typo
s/it/if/
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Dato Simó <dato@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add hbal-excl-tags.data to Makefile.am, missed in 0397694
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Group.hs: add 'allTags'; adjust loaders and test data for it
This commit adds a Group.allTags field to store the tags of node groups,and teaches each loader backend in HTools to populate it (additionally, theIAllocator class in lib/cmdlib.py now includes tags for groups too). Test...
Instance.hs: rename 'tags' to 'exclTags', provide 'allTags'
The mergeData function in Loader.hs included a step to filter an instance'stags to include only the exclusion tags (as specified via the commandline,or cluster-level tags). Later on, code in Node.hs assumed Instance.tags to...
htools-excl.test: add test case for exclusion tags in hbal
In preparation for future modifications in the exclusion tags field, add atest that verifies that exclusion tags are being honored: in a test clusterwith two instances of the same exclusion group in each node, hbal should...
Fix small typo in NEWS file
Sorry, I didn't see this before pushing 5a7cb9d3…
Remove support for PUT in noded
This takes care of a FIXME; 2.6 already uses the new method, so we'regood during upgrades.
Ignore empty/comment lines in OS variants file
Per a conversation on ganeti@googlegroups.com:
- gnt-os diagnose ; gnt-os list take in consideration blank lines in /etc/ganeti/instance-image/variants.list that could be confusing.
Let's fix this and also let's ignore comment lines....
gnt-job cancel: Confirmation and selection of jobs
New parameters, “--pending”, “--queued” and “--waiting”, are added toselect all jobs in the respective state. If one of those options is usedand “--force” is not given, the user is asked to confirm the operation....
Update NEWS for file storage paths
Mention that the file is something new and should be written bycfgupgrade.
Replace custom algorithm in constants unittest
There is no need for the “_IsUniqueSequence” function anymore, it caneasily be replaced by utils.FindDuplicates. Also, pass the message as akeyword parameter and use the more commonly used assert* functions....
Design for remote command execution via RPC
This is a first design for executing commands via RPC.
Makefile: Verify version of security.rst document
This document should be kept up-to-date.
Add new constant for pending job status
This constant contains the job status' “queued”, “waiting” and“cancelled”.
Conflicts: NEWS: Trivial lib/tools/ensure_dirs.py: constant moved to pathutils...
ensure-dirs: Fix permissions on master socket
A socket shouldn't have its executable bit set.
Update security document for version 2.6
Quite some things were out of date. Some formatting was also updated.
Add a very simple test rpc program
This only supports test delay for now, is not built by default (onlyon demand), and is also not installed anywhere.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
confd: add the test_delay RPC call
Also add some more haddock structure to the module. The RPC callitself is rather trivial.
Add htools program code to hlint call
Currently only the libraries are added, but not the actual main binary(usually short) code.
Add design for changing node SSH setup
The goal is to remove the dependency on Paramiko.
confd: fix RpcVersion call
By accident, we sent the node object as call data in this rpc call,instead the version request (i.e. nothing). This is due to the factthat the 'call' data comes second, not first in the function argument.
Cleanup HTools.Types/BasicTypes imports
Before we reorganised the source tree, the 'Result' type was exportedfrom HTools/Types.hs. This changed during the reorg, but at that timewe didn't change the exports; instead, we kept re-exporting it fromthe old module for compatibility....
Merge branch 'stable-2.6' into devel-2.6
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Update NEWS and bump version to 2.6.1
This is a small bug-fix only release.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
errors: Document arguments to QueryFilterParseError
Also fix one small mistake in the docstring for QuitGanetiException.
Fix distcheck after commit 7a962ec
I only ran “make check” and forgot about “distcheck”.
Makefile/check-local: Ensure design document exists
This way a missing design document is detected.
Add design document for version 2.7
- Move design document for batch instance creation into it- Sort list of hidden files in index.rst
QA: TestAssignNodesIncludingSplit can be disabled
The test can now be enabled/disabled with the "group-rwops" flag.
QA: never put a master node offline
Before, it could happen that the master node was chosen to be set offlinein the "instance-remove-drbd-offline" test. Also changed a finally clauseto make sure that an exception while making the node online doesn't break...
Add option to disable iallocator use in QA
By default some QA tests use iallocators. It is now possible to disableiallocators in QA, so that QA doesn't force having iallocators set up.The affected tests have now alternatives pathways.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Add support for cpu_cap and cpu_weight Xen params
This patch adds support for Xen's CPU scheduler 'cpu_cap' and'cpu_weight' parameters.
Ganeti default values (cap: 0=unlimited, weight: 256) are Xen defaults.
cpu_cap is not validated correctly because of actual Ganeti limitation...
Text.hs: update field lists in parseData comments
The comments in parseData had become out of date with the implementationsof load{Group,Node,Inst}. This commit updates the field list in comments tomatch the implementations.
Signed-off-by: Dato Simó <dato@google.com>...
Enable conversion of errors constants
These will be needed for the serialisation of errors.
LUClusterVerifyGroup: Localize virtual file paths
The check for file consistency didn't properly handle virtual pathsin case of a virtual cluster. This didn't cause any breakage as ina standard virtual cluster setup with only one node all files arevisible for every node....
Cleanup haddock documentation a bit
While grepping for htools imports in the non-htools subdirectory, Isaw that our haddock prologue and title are very very old and refer tothe old htools-only state. Let's cleanup a bit…
Cleanup network timeouts and htools imports
This patch removes the last HTools module imports from non-htools code(the HTools.Types module), but it requires an associated cleanup:using luxi-specific constants for luxi timeouts (the only effect isthat one timeout decreases from 15 to 10, the default value in the...
Cleanup/expand the filter/query tests
This patch cleans up duplicate code in Test.Ganeti.Query.Filter andthen adds a test for names consistency with Python's code behaviour(stable ordering for simple filters and otherwise niceSort'edordering).
Add a helper for query field checks
… and also use it to simplify 'needsLiveData'. Additionally, add anexplicit export list to Ganeti.Query.Types, since otherwise we'd(re)export all imported symbols.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Enable query socket usage in gnt-node/gnt-group
This switches gnt-node/gnt-group (and their equivalent RAPI resources)to go over the query socket.
Add support for classic queries
This patch adds support for classic-style queries (before query2) tothe query socket server. The patch is rather trivial, since as inPython we just piggy-back on the query2 implementation.
Add a makeSimpleFilter function
And associated unittests. This will be needed for classic-stylequeries.
Fix the node powered field
When initially implementing the node query, I thought the 'powered'field is a representation of the run-time powered status, which wouldmake its query complex.
In reality, it's a simple config query, which we can supporteasily. We also add a small helper, so that we don't hardcode the...
Simplify a bit queryFields
We don't add a type class for fully-generic handling, but we doabstract the duplicate part.
Add function for getting a group's merged disk params
… and use it in the Query implementation, removing the lastnon-correct query field for Groups.
Switch ordering of names on query to niceSort
This makes the "all" names queries consistent with the Pythonresults. The change requires updating the unittests, at which point aduplicate error message is simplified.
Use the new name filtering behaviour in query
We do this not quite generically, which means we have to addanother layer in the call chain, and rename the current queryfunction, plus add special-case code for each query type. Hopefully wewill be able to improve on this in the future....
Add functionality for checking validity of names
This replicates in the Haskell Query2 implementation the behaviour ofthe Python code: if a "simple" filter is passed (one that containsonly Or aggregators and EQ binary ops on the name field), then anfailure is flagged if the given values are not known....
vcluster: Don't virtualize /etc/hosts path
/etc/hosts is a bit special as it's a system-wide file and the virtualcluster/node root doesn't apply. The modification of /etc/hosts shouldbe disabled in virtual clusters. If it isn't, however, the vclusterfunctions would raise an exception complaining about a path outside of...
vcluster-setup: Stop hardcoding /etc/hosts
… use a constant instead.
qa_utils: Stop hardcoding path to /etc/hosts
- Simplified shell command for adding hosts, too- The exception handling is also improved by re-raising any exceptions instead of swallowing qa_error.Error