utils: Move X509-related code into separate file
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
utils: Move I/O-related code into separate file
utils: Move code related to file locking into separate file
utils: Move wrappers into separate file
utils: Use function to disable fork
Use a function instead of a variable written by another module.
utils: Change RemovePidFile to take path, not name
This avoids having to monkey-patch the utils module for unittests.Monkey patching is evil and caused a bug while moving code around.
Add unit tests for CheckAssignmentForSplitInstances
Signed-off-by: Adeodato Simo <dato@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Expose OpAssignGroupNodes over RAPI and RAPI client
Validate RAPI resource paths
To stay consistent, RAPI resources should only use a restricted set ofcharacters. No uppercase, no “_”, etc. This is enforced by this patch.
utils: Move hashing-related code into separate file
utils: Split algorithms into separate file
utils: Split Retry & co. into separate file
utils: Move text-related functions into separate file
utils: Move Mlockall into separate file
The unittest file is renamed to match the other utils.* tests.
Add unit test for ganeti.rapi.client's ModifyGroup
This was forgotten in f18fab7.
Signed-off-by: Adeodato Simo <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Generalize the OpCode-should-be-in-mcpu test
Currently, the unittest TestDispatchTable in mcpu unittest does ahard-coded approach to test whether an opcode should be included ornot in the mcpu.Processor dispatch table. This is not flexible, so wereplace it with two changes:...
Fix syntax error with Python 2.4
Use new console information in hypervisor abstraction
This makes use of the new way of returning console information from themaster daemon. Unittests are included.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Introduce verbose opcode result for console
With this patch OpConnectConsole will no longer just return a commandwith arguments, but rather a detailed description about how to connectto an instance's console. Unittests for some parts are included. Another...
Support query of node field 'powered'
This field is based on OOB support and is only available if there's oobsupport for that node.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
ganeti.query_unittest.py: add tests for group queries
ganeti.query_unittest.py: test lock fields too
Additionally, change TestQueryFields.testSomeFields() to handle lists offields shorter than 20 elements.
Merge branch 'devel-2.3'
Convert “gnt-debug locks” to query2
Locks can now be queried using “Query(what="lock", …)” over LUXI.
cfgupgrade: Check master name, clarify question
- Check hostname and abort if it doesn't match contents of “ssconf_master_node”, can be overridden using “--ignore-hostname” parameter.- Clarify confirmation question and don't mention instances anymore....
RAPI: Add resource to grow instance disk
baserlib: Add function for filling opcodes
This function makes use of the opcode parameters which now livedirectly in the opcode. A number of RAPI resources can now be simplified.
RAPI: Add resource to modify cluster
Improve opcode summary tests
Test full summary instead of just format.
Migrate code verifying opcode parameters to base class
This allows the function to be used in other places as well.An optional parameter is added to control whether defaultvalues should be set. Unittests are added, providing fullcoverage.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Improve tests for OP_ID
… by detecting duplicates.
cmdlib: Remove opcode parameters
Remove the parameter definitions and use those from the opcode classesinstead. Small style changes are also made (empty lines, wrapping).
opcodes: Add opcode parameter definitions
This is the first step for migrating them from cmdlib. A metaclass isused to define “__slots__” upon class creation time (not instantiation).
query2: Add new field status “offline”
This allows “gnt-node list” to show the difference between modes markedoffline and nodes with e.g. RPC errors (“(nodata)”). node1 is themaster, node2's node daemon crashed and node3 is marked offline:
$ gnt-node list -o name,offline,dtotal,dfree...
Add unittests for ht module
Conflicts: man/gnt-instance.sgml: Trivial merge in gnt-instance.rst...
cli: Change “<…>” in query output to “(…)”
This should reduce the amount of damage in case of accidential copy &paste.
jqueue: Fix cancelling while in waitlock in queue
Since the recent change to leave jobs in the “waitlock” status (commit5fd6b6947), cancelling a job while it's back in the queue would break.This patch handles these cases and adds a unittest.
Remove utils.FormatTimestampWithTZ
Long story short: time.strftime("%Z", time.localtime()) doesn't work,even though it's documented to be equivalent to time.strftime("%Z").
$ TZ=America/Sao_Paulo python -c 'import time; printtime.strftime("%Z"), time.strftime("%Z", time.localtime())'...
Ensure temp files from RunCmd tests are removed
utils: Timezone fixes and tests
- Update docstrings to explicitely mention Epoch- Fix timezone bug in FormatTimestampWithTZ, where it would use GMT/UTC when it should use the local timezone- Add unittests for time formatting functions
Move QueryFields to query module
Also replace “sorted” with “utils.NiceSort” now that it supports a keyfunction.
utils.NiceSort: Use sorted(), add keyfunc, unittests
This patch changes utils.NiceSort to use the built-in “sorted()” andgets rid of the intermediate list. Instead of wrapping the itemsourselves, a key function is used. The caller can specify another key...
Conflicts: man/ganeti-cleaner.sgml: Removed man/ganeti-confd.sgml: Removed man/ganeti-masterd.sgml: Removed...
jqueue: Keep jobs in “waitlock” while returning to queue
Iustin Pop reported that a job's file is updated many times while itwaits for locks held by other thread(s). After an investigation it wasconcluded that the reason was a design decision for job priorities to...
Improve jqueue unittests
- Verify job file updates- Ensure queue lock is released while executing opcode
cli: Add infrastructure for query2
A new function for formatting the query results is added,``FormatTable``. This was determined to be easier and safer thanmodifying the existing ``GenerateTable`` function while keepingbackwards compatibility for code not yet converted. The new code makes...
Add some unit tests for ConfigWriter.AddNodeGroup()
qlang: Add function to build simple filter
This will be used in clients to build the filters for query2.
query: Handle items missing timestamps
In upgraded configurations, some items might miss the “ctime” and/or“mtime” values and need to be handled specially.
query: Add definition for instance queries
Group operations: update ganeti.rapi.client with all node group operations
Group operations: expose add/remove/rename in RAPI
utils: Add function to find duplicates in sequence
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
locking: add nodegroup lock level
This also changes masterd to initialize the ganeti's manager with thecurrent list of nodegroup uuids, and updates unittests
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Querying node groups: RAPI support
This implements /groups and /groups/%s RAPI end points.
Signed-off-by: Adeodato Simo <dato@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add OpQuery opcode
Add node query definition
This includes a bunch of helper functions which can be helpful for otherqueries, too. Unittests are included.
Add utility class for definition-based data queries
While not immediately visible, using definition-based queries will allowfor a number of optimizations and simplifications. This patch justcontains the utility class used for executing queries.
Add simple query filter parser
This parser reads only the format described by the query2design document: either an empty filter or an OR operatorwith equality checks as operands.
RAPI: Implement OS parameters for instance reinstallation
Dictionaries are hard to encode into query strings, therefore the“/2/instances/[instance_name]/reinstall” resource is changed to acceptits parameters via the request body. The old query string parameters are...
Move “rapi_users” file into separate directory
This reduces the number of notifications in “ganeti-rapi”. Until now itwas notified for every change in …/lib/ganeti and had to check whetherthe users file was affected. A symlink is always created in cfgupgrade...
impexpd: Implement support for IPv6
Improve unittests for the utils module
This just a random collection of unittest improvements. Coverageincreases from 73% to 76%.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix breakage from previous commit e695efb
Wrong file name in Makefile.am and wrong naming (.unittest vs_unittest).
Add missing paramter and unittests for this case
While git-am'ing the blockdev_prefix patch, I modified the hv_xen.pyfile but forgot to amend the patch :(, so here it is.
Furthermore, to detect such inconsistencies in the future, a unittest isadded that cross-checks the default parameters in constants.py and the...
asyncnotifier tests: simplify the setup code
First change is to replace "range(len(self.NOTIFIERS)))" withself.NOTIFIERS, as range(len(range(N))) ≡ range(N).
Sencond change is switch from an explicit indexing to use of zip.Unittests still pass :)
Signed-off-by: Iustin Pop <iustin@google.com>...
Add consistency test for mcpu dispatch table
Adding convenience getter for ndparams
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix utils unittest TestRunCmd.testTimeoutKill
Currently the test executes "/bin/sh -c 'trap "" TERM …" via the shell,which means we have two shells involved (the parent one and the childone, which does the trap). For some weird reason, this works for older...
Adding oob_program in the same style as beparams
Temporarily remove the interactive tests
make check < /dev/null breaks these tests. Until we have change RunCmdto override stdin, we need to remove them.
Support timeouts in RunCmd
Further investigations have to be done for merging some of these bitstogether with import-export daemon which uses similiar logic.
Adding unittests for RunningTimeout
Switch man pages from SGML to RST
This patch changes the Makefile rules to build the man pages from theRST sources instead of the SGML ones.
Note: pandoc quotes by default the ‘@’ character in manpage output, sowe explicitely undo this before writing the .in file....
Add unittests for TemporaryReservationManager
And fix an error message.
Add support and checks for version in LUXI
A new constant, LUXI_VERSION, is used to verify the peer's version. Theversion is optional, so old(er) clients and servers talking to peers notsupporting it won't break. Example with mismatching library:
$ gnt-instance list...
Add test for modifiable locking levels
GanetiLockManager, remove default values
The nodes and instances parameters to the constructor are mandatoryanyway, as a value of None will fail when creating the LockSet. Ratherthan fixing this adding code lines, since we never used the defaultvalue, let's remove them and require that the parameters are passed....
Add a "safe" file wrapper over WriteFile
Add functions to read and compare file 'ID's
utils: Add function to find items in dictionary using regex
This basically extracts a small piece of code from ganeti-rapi and putsit into a utility function. RAPI resources are found using a dictionaryin which the keys can either be static strings or compiled regular...
Extract base class from SingleFileEventHandler
The base class can contain code useful to other inotify users.As it is “SingleFileEventHandler” can not be used in ganeti-rapi,therefore it'll use its own small inotify handler class basedon this base class....
http.auth.ReadPasswordFile: Don't read file directly
Reading the file before this function allows for better errorreporting.
jqueue: Fix bug when cancelling jobs
If a job was cancelled while it was waiting for locks, an assertionwould've failed. This patch fixes the problem and provides a unittest to check for this situation.
Use only one version of WritePidFile
This patch merges the pid file handling used for ganeti-* daemons andimpexp daemons. The latter version is used, since it's more reliable:uses locked pid files as opposed to checking 'live' processes.
Merge branch 'devel-2.2'
Add simple unittest for utils.CommaJoin
cli: Move parsing of --net option to separate function
This function will also be used in tools/move-instance.
Abstract OS name/variant functions
Currently, the computation of the 'pure' name or the variant ishardcoded and spread around the functions that need it. This is notnice, and in the future we'd spread it even more with more usage ofvariants/pure os names....
Move the uuid regexp to utils.py
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
jqueue: Use priority for acquiring locks
jqueue: Use timeout when acquiring locks
As already noted in the design document, an opcode's priority isincreased when the lock(s) can't be acquired within a certain amount oftime, except at the highest priority, where in such a case a blockingacquire is used....
mcpu: Adjust lock acquire strategy
The changes to job queue processing require some changes on this class'interface. LockAttemptTimeoutStrategy might move to another place, but that'llbe done in a later patch.
jqueue: Rename current_op to better reflect what it actually is
jqueue: Add unittest for _QueuedJob.CalcStatus
jqueue: Change model from per-job to per-opcode processing
In order to support priorities, the processing of jobs needs to bechanged. Instead of processing jobs as a whole, the code is changed toprocess one opcode at a time and then return to the queue. See the...
Always enable logging for unittests
By enabling all log levels, we ensure all calls are fully evaluated.There was one case in the workerpool where a call to “logging.debug”was wrong, but not caught in unittests because debug logging wasdisabled.
The optional environment variable “LOGTOSTDERR” can be set to...