Fix setting of 'failN1' flag for corner case
This patch includes:
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
A few unittests improvements
Small simplifications of other unit tests using the (==?)operator when possible, and typo fixes.
Corrected some commandlines in walkthrough
BasicTypes.hs: fix docstring for eitherToResult
eitherToResult now converts from `Either a b` to `GenericResult`, notnecessarily from `Either String` only. Also, fix a typo.
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add note about running individual test cases
This was asked a few times offline, so let's document it.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add a small QA check in instance renames for tags update
We also need to change the signature for _GetInstanceInfo, since wedon't have access to the instance dictionary in the rename instancetest.
Update blockdev's "info" at instance rename
Currently, we set "info" metadata on block devices at device creationtime, but we never update it, leading to stale data in case ofinstance renames. This would not be a big problem in case of regularrenames (assuming this is a rare operation), but importing instances...
LVM: remove old tags when adding new ones
This patch adds a small helper function to clear an LV's tags, andcalls it at SetInfo time. We need this to be able to correctly trackinstance renames, once we will call SetInfo at such times.
Signed-off-by: Iustin Pop <iustin@google.com>...
Add a small bdev helper function
I wanted to write that snippet the third time, which is too much :)
Add a small note about tab completion and generated files
This can indeed be annoying, so let's document it.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
Update for SSH design: Cluster name, ignore node daemon
While writing the initial version of this design I misunderstood what“setup-ssh” does. It doesn't actually start the node daemon and justconfigures SSH, nothing else.
This patch adds the cluster name as a field to the JSON structure and...
Compare significant fields only for simple SSH keys
For simple SSH keys, that is those without options such as“command="…"”, only the first two parts need to be compared. The thirdfield is a free-form comment.
This patch changes the comparison used in...
Add instance uptime to monitoring info
This can be used to detect a reboot between two checks.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
test/*.py: Replace '' with ""
There might be more, but at least replace all these low-hanging fruits.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add cluster monitoring agent design document
This design addresses the lack of a uniform way to query ganeti nodesfor real time information that can be used by monitoring.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove custom OpResult type/monad
Since we now have the GeneralResult as a multi-purpose monad, we canremove the custom OpResult monad, and just use 'GeneralResultFailMode' as our type. This allows removal of a few bits ofspecialised infrastructure, relying instead on the generic one....
Merge branch 'devel-2.6'
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
ensure-dirs: Don't accept arguments
Before they would just be silently ignored.
ensure-dirs: Fix program name on usage screen
No string replacements are used, so doubling of the percent sign is notnecessary.
Before: Usage: %ensure-dirs [--full-run]After: Usage: ensure-dirs [--full-run]
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Convert man page highlighting to standard RST
Instead of using the sphinx-specific highlight extension, which is notparsed by Pandoc, let's switch to the standard RST directive, whichwill be picked up and will result in slightly nicer man pages whenconverted to HTML (in man output it remains the same)....
Convert two more 'sh' highlight styles to 'shell-example'
This is straightforward; only one shell example remained, but as weprocess the man pages via pandoc and not sphinx, it's not actuallyused, we leave it as such for now.
Small improvements to the move-instance doc
While reading the docs, I saw that this is not converted to the shelllexer, and that a few other small improvements can be done.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Use exitErr instead of explicit error message and exitWith
Furthermore, a few messages have their capitalisation changed (fixed).
Signed-off-by: Helga Velroyen <helgav@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix running of Haskell tests
Commit 21a5e56c forgot to rename a variable used in a conditional (ofcourse shell didn't complain about unused vars), so the AM_CONDITIONALwas always false.
Generalise the Result type
Currently, our error monad—Result—has a plain string error type. Thisis not good, as we don't have structured errors, we can't pass backproper error information to Python code, etc.
To solve this, we generalise this type as 'GenericResult a', and make...
Merge branch 'devel-2.6' into master
cli: Fix small typo
s/it/if/
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Dato Simó <dato@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add hbal-excl-tags.data to Makefile.am, missed in 0397694
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Group.hs: add 'allTags'; adjust loaders and test data for it
This commit adds a Group.allTags field to store the tags of node groups,and teaches each loader backend in HTools to populate it (additionally, theIAllocator class in lib/cmdlib.py now includes tags for groups too). Test...
Instance.hs: rename 'tags' to 'exclTags', provide 'allTags'
The mergeData function in Loader.hs included a step to filter an instance'stags to include only the exclusion tags (as specified via the commandline,or cluster-level tags). Later on, code in Node.hs assumed Instance.tags to...
htools-excl.test: add test case for exclusion tags in hbal
In preparation for future modifications in the exclusion tags field, add atest that verifies that exclusion tags are being honored: in a test clusterwith two instances of the same exclusion group in each node, hbal should...
Fix small typo in NEWS file
Sorry, I didn't see this before pushing 5a7cb9d3…
Remove support for PUT in noded
This takes care of a FIXME; 2.6 already uses the new method, so we'regood during upgrades.
Ignore empty/comment lines in OS variants file
Per a conversation on ganeti@googlegroups.com:
- gnt-os diagnose ; gnt-os list take in consideration blank lines in /etc/ganeti/instance-image/variants.list that could be confusing.
Let's fix this and also let's ignore comment lines....
gnt-job cancel: Confirmation and selection of jobs
New parameters, “--pending”, “--queued” and “--waiting”, are added toselect all jobs in the respective state. If one of those options is usedand “--force” is not given, the user is asked to confirm the operation....
Update NEWS for file storage paths
Mention that the file is something new and should be written bycfgupgrade.
Replace custom algorithm in constants unittest
There is no need for the “_IsUniqueSequence” function anymore, it caneasily be replaced by utils.FindDuplicates. Also, pass the message as akeyword parameter and use the more commonly used assert* functions....
Design for remote command execution via RPC
This is a first design for executing commands via RPC.
Makefile: Verify version of security.rst document
This document should be kept up-to-date.
Add new constant for pending job status
This constant contains the job status' “queued”, “waiting” and“cancelled”.
Conflicts: NEWS: Trivial lib/tools/ensure_dirs.py: constant moved to pathutils...
ensure-dirs: Fix permissions on master socket
A socket shouldn't have its executable bit set.
Update security document for version 2.6
Quite some things were out of date. Some formatting was also updated.
Add a very simple test rpc program
This only supports test delay for now, is not built by default (onlyon demand), and is also not installed anywhere.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
confd: add the test_delay RPC call
Also add some more haddock structure to the module. The RPC callitself is rather trivial.
Add htools program code to hlint call
Currently only the libraries are added, but not the actual main binary(usually short) code.
Add design for changing node SSH setup
The goal is to remove the dependency on Paramiko.
confd: fix RpcVersion call
By accident, we sent the node object as call data in this rpc call,instead the version request (i.e. nothing). This is due to the factthat the 'call' data comes second, not first in the function argument.
Cleanup HTools.Types/BasicTypes imports
Before we reorganised the source tree, the 'Result' type was exportedfrom HTools/Types.hs. This changed during the reorg, but at that timewe didn't change the exports; instead, we kept re-exporting it fromthe old module for compatibility....
Merge branch 'stable-2.6' into devel-2.6
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Update NEWS and bump version to 2.6.1
This is a small bug-fix only release.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
errors: Document arguments to QueryFilterParseError
Also fix one small mistake in the docstring for QuitGanetiException.
Fix distcheck after commit 7a962ec
I only ran “make check” and forgot about “distcheck”.
Makefile/check-local: Ensure design document exists
This way a missing design document is detected.
Add design document for version 2.7
- Move design document for batch instance creation into it- Sort list of hidden files in index.rst
QA: TestAssignNodesIncludingSplit can be disabled
The test can now be enabled/disabled with the "group-rwops" flag.
QA: never put a master node offline
Before, it could happen that the master node was chosen to be set offlinein the "instance-remove-drbd-offline" test. Also changed a finally clauseto make sure that an exception while making the node online doesn't break...
Add option to disable iallocator use in QA
By default some QA tests use iallocators. It is now possible to disableiallocators in QA, so that QA doesn't force having iallocators set up.The affected tests have now alternatives pathways.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Add support for cpu_cap and cpu_weight Xen params
This patch adds support for Xen's CPU scheduler 'cpu_cap' and'cpu_weight' parameters.
Ganeti default values (cap: 0=unlimited, weight: 256) are Xen defaults.
cpu_cap is not validated correctly because of actual Ganeti limitation...
Text.hs: update field lists in parseData comments
The comments in parseData had become out of date with the implementationsof load{Group,Node,Inst}. This commit updates the field list in comments tomatch the implementations.
Signed-off-by: Dato Simó <dato@google.com>...
Enable conversion of errors constants
These will be needed for the serialisation of errors.
LUClusterVerifyGroup: Localize virtual file paths
The check for file consistency didn't properly handle virtual pathsin case of a virtual cluster. This didn't cause any breakage as ina standard virtual cluster setup with only one node all files arevisible for every node....
Cleanup haddock documentation a bit
While grepping for htools imports in the non-htools subdirectory, Isaw that our haddock prologue and title are very very old and refer tothe old htools-only state. Let's cleanup a bit…
Cleanup network timeouts and htools imports
This patch removes the last HTools module imports from non-htools code(the HTools.Types module), but it requires an associated cleanup:using luxi-specific constants for luxi timeouts (the only effect isthat one timeout decreases from 15 to 10, the default value in the...
Cleanup/expand the filter/query tests
This patch cleans up duplicate code in Test.Ganeti.Query.Filter andthen adds a test for names consistency with Python's code behaviour(stable ordering for simple filters and otherwise niceSort'edordering).
Add a helper for query field checks
… and also use it to simplify 'needsLiveData'. Additionally, add anexplicit export list to Ganeti.Query.Types, since otherwise we'd(re)export all imported symbols.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Enable query socket usage in gnt-node/gnt-group
This switches gnt-node/gnt-group (and their equivalent RAPI resources)to go over the query socket.
Add support for classic queries
This patch adds support for classic-style queries (before query2) tothe query socket server. The patch is rather trivial, since as inPython we just piggy-back on the query2 implementation.
Add a makeSimpleFilter function
And associated unittests. This will be needed for classic-stylequeries.
Fix the node powered field
When initially implementing the node query, I thought the 'powered'field is a representation of the run-time powered status, which wouldmake its query complex.
In reality, it's a simple config query, which we can supporteasily. We also add a small helper, so that we don't hardcode the...
Simplify a bit queryFields
We don't add a type class for fully-generic handling, but we doabstract the duplicate part.
Add function for getting a group's merged disk params
… and use it in the Query implementation, removing the lastnon-correct query field for Groups.
Switch ordering of names on query to niceSort
This makes the "all" names queries consistent with the Pythonresults. The change requires updating the unittests, at which point aduplicate error message is simplified.
Use the new name filtering behaviour in query
We do this not quite generically, which means we have to addanother layer in the call chain, and rename the current queryfunction, plus add special-case code for each query type. Hopefully wewill be able to improve on this in the future....
Add functionality for checking validity of names
This replicates in the Haskell Query2 implementation the behaviour ofthe Python code: if a "simple" filter is passed (one that containsonly Or aggregators and EQ binary ops on the name field), then anfailure is flagged if the given values are not known....
vcluster: Don't virtualize /etc/hosts path
/etc/hosts is a bit special as it's a system-wide file and the virtualcluster/node root doesn't apply. The modification of /etc/hosts shouldbe disabled in virtual clusters. If it isn't, however, the vclusterfunctions would raise an exception complaining about a path outside of...
vcluster-setup: Stop hardcoding /etc/hosts
… use a constant instead.
qa_utils: Stop hardcoding path to /etc/hosts
- Simplified shell command for adding hosts, too- The exception handling is also improved by re-raising any exceptions instead of swallowing qa_error.Error
cli: Stop hardcoding /etc/hosts path
There is a constant for this purpose.
Move constant for /etc/hosts to pathutils
Needed for coming patches.
bash_completion: Fix mistake introduced in merge 6a74f58
This part of the code was duplicated by accident.
gnt-job list: Add option to include archived jobs
This provides a convenience option to include archived jobs in theoutput list. It's equivalent to using “-o +archived”, but tab completionis nicer.
gnt-job: List archived jobs if requested
If requested via a filter or by including the “archived” output,archived jobs will be loaded and shown. This is significantly slowerthan just listing normal jobs, therefore by default they are not loadedat all....
query: Report data type for unary operators
All data kinds (used to restrict the data collected) referenced in afilter can be requested once it's been “compiled”. However, the kindsof fields used in boolean expressions (e.g. ["?", "xyz"]) were notrecorded. This patch changes the code accordingly and provides a unit...
jqueue: Add new in-memory attribute for archived jobs
This attribute is set to True for jobs which were restored from anarchived file. A new filter will act on this field.
jqueue: Correct docstring
The description was not accurate.
Add htools/ganeti-confd to .gitignore
Sorry, this should have went in b0dcdc10, but I forgot it.
verify-disks: Explicitely state nothing has to be done
Example output:$ gnt-cluster verify-disksSubmitted jobs 4327Waiting for job 4327 ...No disks need to be activated.
Add basic unit tests for "gnt-cluster epo"
This patch adds some unit tests for “gnt-cluster epo”. Not everything iscovered, but at least the bug fixed in the previous patch is.
Fixup node disk free/total queries
The disk free/total values are optional ones, wrapped in a Maybe, sowe shouldn't directly serialise them. In order to simplify theembedded extraction, we add a small helper function.
(htest) Text.hs: fix typo in comment
Fix pylint breakage due to unused var in gnt_cluster
The usage of that variable was removed in 45a36f36, but accidentallythe enumerate() was left in.
Fixup hypervisor queries in node query
We need to only query the default (first enabled) hypervisor, not allhypervisors. For this, we need to add a manual check to ensure that wedon't have a corrupt config (there's no "NonEmptyList" type…).
Fixup node live field names
The String parameter to 'nodeLiveFieldExtract' is the query2 fieldname, not the RPC-layer field name. Grrr for not having a real datatype for this.
Furthermore, we add some safety check that we don't return JSNull viarsNormal…...
Improve Haskell configure options and detection
This patch cleans up the Haskell library detection and defaults.
First, it makes the base compiler/libraries required, per the email discussion.
It then adds two new small autoconf macros, on to check for a required...
Make regex-pcre an optional dependency
This patch makes regex-pcre optional, allowing its disable via apreprocessor define NO_REGEX_PCRE. This define will be added toconfig.ac/Makefile.am in the next patch.
The patch also changes multi-line strings into string concatenation,...
cluster epo: Fix bug where IndexError is raised
Updating the “node_query_list” variable would fail if no arguments werepassed and the “--all” option wasn't specified. A follow-up patch willadd unit tests.
Add list of design documents implemented in version 2.6
Each version should have its dedicated list.