cmdlib: Adapt the rpc calls
The following (blockdev) RPC calls are not converted yet (as they arenot straight forward or need more research):
rpc: Adapt the callbacks to the new encoder type
We change the cmdlib side on the next patch.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
rpc: Adding new encoders for annotating disk parameters
rpc: Adding helper to annotate disk params
config: Adding convenience method for disk parameters
objects: Add helper function to fill disk parameters
QA: Decorate tests with instance checks
Start using “InstanceCheck” decorator where appropriate. Use direct callfor check in places where the instance status changes during the test.
The parameters of “TestInstanceImport” had to be re-ordered to have the...
QA: Infrastructure for hook script to check instance status
This script can be used to check if an instance is running or stopped atvarious points during a QA run. Environment variables are used to passthe most essential information.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
QA: Support passing keyword arguments to test functions
Until now this wasn't possible.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Allow clock skews in certificate verification
Currently we allow for up to NODE_MAX_CLOCK_SKEW time differencebetween nodes in some operations, but not everywhere: SSL certificateverification (import/export, both intra and inter-cluster) has a zerolimit (downwards), and a week upwards. This can cause even...
Improve formatting in the hooks documentation
The pyeval role generates literal (e.g. ``OP_CLUSTER…``) formatting,which differs from the other headers. This is confusing, so let's adda note that these are virtual operations, which makes the output more...
Add decoding of Luxi calls and unittests for LuxiOp
This patch adds a hand-coded decoder for LuxiCall arguments, as thedata-structure is not uniform enough for automated generation (evenfor the serialisation, we had to add hints for some fields,de-serialisation is even harder)....
Some unit tests improvements
In preparation for unit-testing the Luxi definitions, this improvesthe auto-generation of OpCodes: currently this uses unbounded plainStrings for names and fields, and this creates too big test sizes(e.g. when automatically generating job sets)....
Auto-define a LuxiReq data type
We currently auto-generate a LuxiOp data type, which holds the entireoperation (including parameters). However, having a data type just forthe method call would be useful, so let's change THH to alsodefineSADT for the Luxi constructors....
Replace hardcoded constants with Constants.hs names
Now that luxi.py constants are exported, we can use them for moreconsistency.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix Makefile dependencies for building Constants.hs
I was wondering why my changes to luxi.py did not trigger the rebuild…
Correct capitalisation of two Luxi calls
Two Luxi calls have inconsistent an name/value mapping (in the Pythoncode):
- REQ_AUTOARCHIVE_JOBS versus AutoArchiveJobs (versus AutoarchiveJobs)- REQ_QUEUE_SET_DRAIN_FLAG versus SetDrainFlag (no Queue)
While these are only a consistency issue, let's fix them so that the...
Add a new JSON function
And its associated unittests.
Add design document for query path splitting
Implement support for multi-module export
This add support for exporting constants from multiple modules (asopposed to hard-coding constants), and also makes the output morereadable by skipping things we know for sure we don't want to convert(as opposed to things we would like to but don't know how to...
Implement reverse mapping of values to names
This adds a bit of dumb mapping of values to names, while trying to besafe. This is the best we can do without resorting to parsing orinterpreting ASTs.
The difference in the output is:
-- | Converted from Python list or set ADMINST_ALL...
Add missing qa/__init__.py file
The file was there and listed in Makefile.am, but I forgot to add it tocommit a0c3e7264a.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
QA: Fix shutdown for reinstall/rename tests
The “RunTestIf” uses an AND association between the passed test names,so the instance would be be shut down if both rename and reinstall wereenabled. Since this is not necessarily the case, OR must be used.
QA: Enable use of OR conditions in test checks
Until now “TestRunIf” and “TestEnabled” could only handle AND. With thispatch a new class named “Either” is added to “qa_config” and allows OR.The name “Either” was chosen instead of “Or” as the latter is very close...
Further fixes for new-style exception handling
Commit 30d25dd8 moved the htools code to new-style exception handling,but the hconfd code hasn't been, which fails when compiling on newerGHC versions.
This patch does the rest of the move; however, the situation is not...
Add NEWS entry for paramiko/pycrypto warnings
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Fix grow-disk handling of invalid units
The reason why grow-disk was doing:
$ gnt-instance grow-disk instance3 0 -64Unhandled Ganeti error: Invalid format
Is because it does it's own ParseUnit call, and doesn't transform thatinto a nicer message.
Signed-off-by: Iustin Pop <iustin@google.com>...
Fix rapi.testutils unittest
Since we use a testutils.InputTestClient(), then the actual errorexpected is VerificationError, and not GanetiApiError (which is usedat real run-time).
Make setup-ssh behave more friendly
So while testing my previous patch I run setup-ssh manually withoutany arguments, and it did nothing (no complains, no messages,anything). That was very surprising, so let's make it behave better ifno nodes are passed....
Silence Paramiko deprecation warnings
This works around a problem in Paramiko, but which looks like it's aGaneti problem:
[cluster] root@node4:~# gnt-node add --readd node3/usr/lib/python2.6/dist-packages/Crypto/Util/randpool.py:40:RandomPool_DeprecationWarning: This application uses RandomPool, which...
Add more RAPI test utilities
This patch adds a mock RAPI client to test input values to methods. Allmethods either raise an exception if there was a problem or return None.Third-party code can use this to test their input values to the RAPIclient.
Add example script for using RAPI test utilities
This script shows a few examples on how to use the RAPI input testclient. It is also run at “make check” time to ensure it's notcompletely broken.
rapi.testutils.FakeCurl: Add header support
With this patch headers are constructed from the PycURL optionsand passed to the mock implementation.
Move _FakeCurl from tests/ganeti.rapi.client to ganeti.rapi.testutils
This is preparation for the mock system, where we need the same cURLmock.
Signed-off-by: René Nussbaumer <rn@google.com>Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
errors: Add exception for RAPI testing utilities
This exception is raised to abort before actually sending a LUXI call(there is no LUXI server involved in the test). The testing utilitiescatch the exception to report a success (i.e. the code didn't throw...
server.rapi: Factorize RAPI user loading
By moving the code loading RAPI users from a file into a standalonefunction it becomes easier to use the RAPI request handler intests.
luxi: Add list of all requests
This will be used in a unittest for RAPI testing utilities.
RemoteApiHandler: Add support for mocked LUXI client
This will be used for providing a type-checking RAPI client for tests.
Add absolute grow-disk mode at CLI level
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add grow-disk tests in QA
Man page update for grow-disk
Add 'absolute' grow-disk mode at OpCode/LU level
This also improves the log messages for the (default) relative mode("by %s to %s").
Ensure that the grow disk amount is positive
The CLI currently doesn't permit this, but a direct job submissionwould allow it.
Log all external commands execution
This logs all external commands in normal (non-debug) mode. This willleak the DRBD secrets, but in any case we do log failed commands, soit's not a significant change.
QA: Add default setting for tests
Commit 1010ec70b1 enabled all tests by default. In some scenarios wherethe configuration is static one may not want newly added tests to berun.
Update NEWS to mention that CertificateError change.
Signed-off-by: Chris Schrier <schrierc@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Change CertificateError to subclass GanetiApiError
Add opcode parameter descriptions
Fixes QA after commit cfdf561dade7.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Chris Schrier <schrierc@google.com>
Stop acquiring BGL for LUXI queries
Short description: This fixes an issue whereby masterd would becomeunresponsive on the LUXI socket, leading to client timeouts. While madeworse in 2.5, the underlying issue was already present in 2.4.
Longer description: Until now all LUXI queries would acquire the BGL...
Convert LUClusterConfigQuery to query2
The main intention of this patch is to make it possible to retrievecluster tags via query2. While at it I decided to convertLUClusterConfigQuery right away. Some of the values returned byLUClusterQuery are also included, but the conversion of LUClusterQuery...
Stop using locks in LUXI "QueryTags"
Also mark it as deprecated in NEWS as normal queries can be usedinstead.
Fix RAPI QA with exports via query2
Remove unused constants
These are not used anywhere in Python or Haskell.
Convert listing exports to query2
This solves one case where locks are acquired during LUXI queries.Pretty late into the transition I noticed that OpBackupQuery had a“use_locking” parameter for a long time, but didn't use it. Sincemost of the other changes were already and this allows exports to...
utils.algo: Use str.isdigit instead of regular expression
str.isdigit is about 4x faster than using a regular expression ("\d+").This is in the inner sorting code so speed matters.
$ python -m timeit -s 'import re; s = re.compile("^\d+$")' \'s.match(""); s.match("Hello World"); s.match("1234")'...
Include PycURL error code in GanetiApiError.
Signed-off-by: Chris Schrier <schrierc@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Drop objects.QueryRequest
It was only used in one place and wasn't really necessary.
gnt-os modify: Add "--submit" option
gnt-node: Add "--submit" and "--priority" to commands
gnt-instance: Add "--submit" and "--priority" to commands
gnt-group: Add "--submit" and "--priority" to commands
gnt-cluster modify: Add "--submit" option
gnt-backup: Add "--submit" to two commands
Document "--submit" in ganeti.7
Like “--priority” and “--dry-run”, the “--submit” option is availablefor many commands and can be documented in a central place. This patchalso fixes a small number of style issues.
Fix further QA failures due to query changes
Hopefully these will be the last ones…
Fix error in opcode result processing
LUXI queries are processed without callbacks (seeserver.masterd.ClientOps._Query). With commit 07923a3c the logic forchecking an opcode's result for jobs to submit was changed andsubsequently raised an exception (“'NoneType' object has no attribute...
Add "show" as alias for "info" to gnt commands
This patch adds support for "show" as an alias for "info" tognt-(cluster|instance|node|os). It already exists in gnt-job.
Signed-off-by: Alexander Schreiber <als@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Copy debug level, priority and set comment for LU-generated opcodes
Before this patch, a node evacuation submitted with high priority wouldonly compute the solution at that priority, but the actual evacuationran at normal priority.
Fix QA failures with "gnt-job list"
Jobs have no “name” field.
gnt-job list: Add options for commonly used filters
While “gnt-job list” would also accept filters on the command line (e.g.“'status == "error"'”, having shortcuts in the form of options comes inhandy.
Merge branch 'devel-2.5'
gnt-* {add,list,remove}-tags: Unify options
- Listing tags is a query, so neither “--priority” nor “--submit” make sense- Support both options for adding/removing tags- Also remove “--submit” from “gnt-node health”; it doesn't work and doesn't make sense for listing node health...
Merge branch 'stable-2.5' into devel-2.5
Bump version for 2.5.0 final release
Also update NEWS file.
Merge branch 'devel-2.4' into stable-2.5
configure.ac: Fix “too many arguments” error
If GHC_PKG_QUICKCHECK contains multiple values, the test would failwith “too many arguments”.
Fix extra whitespace
Sorry, didn't catch this before…
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>(cherry picked from commit 54b010cad1ea0a536ed037bf315a04dd1c079964)...
Further fixes concerning drbd port release
Commit 3b3b1bc does not entirely fix the bug introduced in commitf396ad8. It fixes consistency of config data in permanent storage, butdoes not ensure consistency in data held in runtime memory of masterd.
The bug of duplicate ports is still triggered when LUInstanceRemove()...
Fix a bug concerning TCP port release
Commit f396ad8 returns the TCP port used by DRBD disk back to theTCP/UDP port pool using AddTcpUdpPort().
However, AddTcpUdpPort() writes the config on every invocation,using _WriteConfig(). This causes two problems:...
Fix query unittests after converting jobs to query2
I missed these among some shelltest-related failures.
QA: Add tests for “gnt-job list”
gnt-job list: Switch to query2
This brings “gnt-job list” up to the same level as “gnt-instance list”with filters. Further updates will add more parameters for the mostcommon filters (e.g. only running jobs).
Also update the man page.
gnt-job info: Convert to query2
Otherwise detecting unavailable jobs is hard (“status” is None, is thisan error or just an unavailable job?).
Add job support to query2 via LUXI
This enables the use of filters through query2 when listing jobs.
jqueue: Cache prepared field list in _JobChangesChecker
… instead of re-calculating it on every file change.
NEWS: Deprecate LUXI calls replaced with query2
Adding the “luxi” namespace is necessary in “sphinx_ext”.
jqueue: Convert GetInfo to query2
This rather inefficient implementation (fields are evaluated on everycall to GetInfo) is not good for WaitForJobChanges and doesn't supportfilters, but that will be rectified in later patches.
query: Add definitions for job fields
qlang.MakeFilter: Enable use of different name field
Jobs don't have a “name” field, so we must be able to controlthe field used for simple filters.
Merge cli.FormatTimestamp and utils.FormatTime
… to some degree at least. Unittests are included.
constants: Don't hardcode priorities for LOCK_ATTEMPTS_TIMEOUT
Also include unittest for LOCK_ATTEMPTS_TIMEOUT.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
jqueue._QueuedOpCode: Change a docstring
There was a typo and it's not necessary to repeat the class name.
locking: Remove unused OldStyleQueryLocks
No longer used after commit 090377807.
ganeti.initd: Add “status” action
Eric Rostetter sent a patch adding a “status” action, but unfortunatelyhis code was apparently specific to Red Hat. I hope this implementationis more distribution-agnostic; after all “status_of_proc” is part ofLSB. Example output:...
Add whitelist for opcodes using BGL
Update IP pool management design doc
Update IP pool managenent design doc to be consistentwith the implementation.
LUOobCommand: acquire BGL in shared mode
Fixed a typo so that now LUOobCommand acquires the BLG in shared mode, asintended.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>