Group operations: expose add/remove/rename in RAPI
Signed-off-by: Adeodato Simo <dato@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
utils: Add function to find duplicates in sequence
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
locking: add nodegroup lock level
This also changes masterd to initialize the ganeti's manager with thecurrent list of nodegroup uuids, and updates unittests
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Querying node groups: RAPI support
This implements /groups and /groups/%s RAPI end points.
Signed-off-by: Adeodato Simo <dato@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add OpQuery opcode
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add node query definition
This includes a bunch of helper functions which can be helpful for otherqueries, too. Unittests are included.
Add utility class for definition-based data queries
While not immediately visible, using definition-based queries will allowfor a number of optimizations and simplifications. This patch justcontains the utility class used for executing queries.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add simple query filter parser
This parser reads only the format described by the query2design document: either an empty filter or an OR operatorwith equality checks as operands.
RAPI: Implement OS parameters for instance reinstallation
Dictionaries are hard to encode into query strings, therefore the“/2/instances/[instance_name]/reinstall” resource is changed to acceptits parameters via the request body. The old query string parameters are...
Move “rapi_users” file into separate directory
This reduces the number of notifications in “ganeti-rapi”. Until now itwas notified for every change in …/lib/ganeti and had to check whetherthe users file was affected. A symlink is always created in cfgupgrade...
impexpd: Implement support for IPv6
Improve unittests for the utils module
This just a random collection of unittest improvements. Coverageincreases from 73% to 76%.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix breakage from previous commit e695efb
Wrong file name in Makefile.am and wrong naming (.unittest vs_unittest).
Add missing paramter and unittests for this case
While git-am'ing the blockdev_prefix patch, I modified the hv_xen.pyfile but forgot to amend the patch :(, so here it is.
Furthermore, to detect such inconsistencies in the future, a unittest isadded that cross-checks the default parameters in constants.py and the...
asyncnotifier tests: simplify the setup code
First change is to replace "range(len(self.NOTIFIERS)))" withself.NOTIFIERS, as range(len(range(N))) ≡ range(N).
Sencond change is switch from an explicit indexing to use of zip.Unittests still pass :)
Signed-off-by: Iustin Pop <iustin@google.com>...
Add consistency test for mcpu dispatch table
Adding convenience getter for ndparams
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix utils unittest TestRunCmd.testTimeoutKill
Currently the test executes "/bin/sh -c 'trap "" TERM …" via the shell,which means we have two shells involved (the parent one and the childone, which does the trap). For some weird reason, this works for older...
Adding oob_program in the same style as beparams
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Merge branch 'devel-2.3'
Temporarily remove the interactive tests
make check < /dev/null breaks these tests. Until we have change RunCmdto override stdin, we need to remove them.
Support timeouts in RunCmd
Further investigations have to be done for merging some of these bitstogether with import-export daemon which uses similiar logic.
Adding unittests for RunningTimeout
Switch man pages from SGML to RST
This patch changes the Makefile rules to build the man pages from theRST sources instead of the SGML ones.
Note: pandoc quotes by default the ‘@’ character in manpage output, sowe explicitely undo this before writing the .in file....
Add unittests for TemporaryReservationManager
And fix an error message.
Add support and checks for version in LUXI
A new constant, LUXI_VERSION, is used to verify the peer's version. Theversion is optional, so old(er) clients and servers talking to peers notsupporting it won't break. Example with mismatching library:
$ gnt-instance list...
Add test for modifiable locking levels
GanetiLockManager, remove default values
The nodes and instances parameters to the constructor are mandatoryanyway, as a value of None will fail when creating the LockSet. Ratherthan fixing this adding code lines, since we never used the defaultvalue, let's remove them and require that the parameters are passed....
Add a "safe" file wrapper over WriteFile
Add functions to read and compare file 'ID's
utils: Add function to find items in dictionary using regex
This basically extracts a small piece of code from ganeti-rapi and putsit into a utility function. RAPI resources are found using a dictionaryin which the keys can either be static strings or compiled regular...
Extract base class from SingleFileEventHandler
The base class can contain code useful to other inotify users.As it is “SingleFileEventHandler” can not be used in ganeti-rapi,therefore it'll use its own small inotify handler class basedon this base class....
http.auth.ReadPasswordFile: Don't read file directly
Reading the file before this function allows for better errorreporting.
jqueue: Fix bug when cancelling jobs
If a job was cancelled while it was waiting for locks, an assertionwould've failed. This patch fixes the problem and provides a unittest to check for this situation.
Use only one version of WritePidFile
This patch merges the pid file handling used for ganeti-* daemons andimpexp daemons. The latter version is used, since it's more reliable:uses locked pid files as opposed to checking 'live' processes.
Merge branch 'devel-2.2'
Add simple unittest for utils.CommaJoin
cli: Move parsing of --net option to separate function
This function will also be used in tools/move-instance.
Abstract OS name/variant functions
Currently, the computation of the 'pure' name or the variant ishardcoded and spread around the functions that need it. This is notnice, and in the future we'd spread it even more with more usage ofvariants/pure os names....
Move the uuid regexp to utils.py
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
jqueue: Use priority for acquiring locks
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
jqueue: Use timeout when acquiring locks
As already noted in the design document, an opcode's priority isincreased when the lock(s) can't be acquired within a certain amount oftime, except at the highest priority, where in such a case a blockingacquire is used....
mcpu: Adjust lock acquire strategy
The changes to job queue processing require some changes on this class'interface. LockAttemptTimeoutStrategy might move to another place, but that'llbe done in a later patch.
jqueue: Rename current_op to better reflect what it actually is
jqueue: Add unittest for _QueuedJob.CalcStatus
jqueue: Change model from per-job to per-opcode processing
In order to support priorities, the processing of jobs needs to bechanged. Instead of processing jobs as a whole, the code is changed toprocess one opcode at a time and then return to the queue. See the...
Always enable logging for unittests
By enabling all log levels, we ensure all calls are fully evaluated.There was one case in the workerpool where a call to “logging.debug”was wrong, but not caught in unittests because debug logging wasdisabled.
The optional environment variable “LOGTOSTDERR” can be set to...
Add small jqueue unittests
cli: Add option definition for priority
Add support for job priority to opcodes and job queue objects
This allows clients to submit opcodes with a priority. Except for beingtracked by the job queue, it is not yet used by any code.
Unittests for jqueue._QueuedOpCode and jqueue._QueuedJob are provided for...
Add job priority constants
locking: Implement priorities in SharedLock and LockSet
For proper support of job priorities, jobs' locks need to respectpriorities. Otherwise it could happen that a job with a lower prioritycould get a lock before a job with a higher priority (depending on...
The config now should also belong to confd group and readable by it
jqueue: Use separate function for encoding errors
Comes with unittest.
Adding a runtime configuration library
This is used to expand the users/group names just once atinitial call.
Show list of pending acquires in “gnt-debug locks”
This is accomplished by keeping a list of waiting threads insteadof just their number inside the lock-internal condition. A fewother tweaks to the output format are also made.
hansmi helped me with merging the conflict. Thanks
Conflicts: lib/workerpool.py
Adding a paramiko fingerprint format helper
And provide unittests for them
workerpool: Add support for task priority
To add job priorities, the worker pool underlying the job queue mustsupport priorities per task. This patch adds them to the worker pool.
Fix race condition in locking unittest
While writing unittests for the new lock monitor, I made a small mistake anddidn't synchronize acquiring locks properly. This patch takes care of thisissue by adding additional synchronization.
Add simple lock monitor
This patch adds an initial implementation of a lock monitor, accessiblefor the user through “gnt-debug locks”. It currently shows all resourcelocks: BGL, nodes and instances. Config and job queue locks could beshown too, but wouldn't be of much help. The current owner(s) and mode...
workerpool: Allow setting task name
With this patch, the task name is added to the thread name and will show up inlogs. Log messages from jobs will look like “pid=578/JobQueue14/Job13 mcpu:289DEBUG LU locks acquired/cluster/BGL/shared”.
Use one function to parse “--fields” option values
Make family argument in FormatAddress optional
By doing this we delegate the task of finding the correct address familyto the FormatAddress method.
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Support IPv6 node add
Fix some small newline style issues
RAPI: Allow modifying instance
rapi.client, http.client: Format url correctly when using IPv6
This patch moves the FormatAddress helper function from daemon.py tonetutils.py. This enables its use in http.client as well as inrapi.client. Furthermore this adds functionality to format IPv6...
Support for resolving hostnames to IPv6 addresses
This patch enables IPv6 name resolution by using socket.getaddrinfoinstead of socket.gethostbyname_ex.
It renames the HostInfo class to Hostname and unifies its use throughoutthe code. This is achieved by using static calls where no object is...
Always use address instead of hostname in rpc.Client
In light of the upcoming IPv6 support, this patch enables the rpc.Clientto always use a node's address to connect to it. This is necessary as wedo not want to rely on name resolution to connect to the correct IP...
Introduce new IPAddress classes
This patch unifies the netutils functions dealing with IP addresses tothree classes:- IPAddress: Common IP address functionality- IPv4Address: IPv4 specific functionality- IPv6address: IPv6-specific functionality
Furthermore it adds methods to check whether an address is a loopback...
errors: Function to check whether value is encoded error
RAPI client: Support renaming instances
Allow renaming instances via RAPI
RAPI client: Don't re-use PycURL object
With this patch, a new PycURL object will be created for each request.This should make the RAPI client safe for simultaneous calls frommultiple threads. Unittests are adjusted accordingly.
An unnecessary variable assignment is also removed from the unittest...
Fix bug in bdev when drbd version format is x.x.x.x
This patch fixes a bug reported in [0]. Newer drbd versions can haveanother digit beyond the regular major, minor and point release digits.We modify the regex used to match that with an optional parts which is...
Test instance NIC and disk parameter names
Add new parameter type “maybe string”
Before strict checking was implemented, NIC IP addresses could be setto “None”. Commit bd061c35 added more strict checking, includingenforcing the IP address to be a string. With this new type, itcan again be set to None....
Add check for RAPI paths to start with /2
During a discussion in July 2010 it was decided that we'll stabilize on /2. Seemessage ID <20100716180012.GA9423@google.com> for reference.
Ensure assertions are evaluated in tests
A lot of assertions are used in Ganeti's code. Some unittests even checkwhether AssertionError is raised in some cases. Explicitely ensuringassertions are evaluated makes sure those tests don't fail andassertions are checked....
RAPI client: The os argument for instance reinstalls is optional
Signed-off-by: David Knowles <dknowles@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
RAPI client: Support migrating instances
workerpool: Change signature of AddTask function to not use *args
By changing it to a normal parameter, which must be a sequence, we canstart using keyword parameters.
Before this patch all arguments to “AddTask(self, *args)” were passed asarguments to the worker's “RunTask” method. Priorities, which should be...
RAPI: Support migrating instances
RAPI: Add os params to instance creation v1
Since the RAPI QA suite doesn't seem to offer easy testing of failedcreations, I didn't add this to the QA. Pointers on how to do it arewelcome.
The patch also changes the 'os' argument to be required, since that is...
Update the RAPI client for the migration mode
See the discussion on the previous patch about this. Basically unless wewant to a add a new 'feature' marking for the live migration parameter,there is no simple way to handle this nicely in the client.
Given that the client was/is marked as experimental, this patch simply...
Convert RPC client to PycURL
Instead of using our custom HTTP client, using PycURL's multiinterface allows us to get rid of the HTTP client threadpool.The majority of the code is still in the ganeti.http.clientmodule.
A simple per-thread HTTP client pool gives cURL a chance to...
Implement lock names for debugging purposes
This patch adds lock names to SharedLocks and LockSets, that can be usedlater for displaying the actual locks being held/used in places where weonly have the lock, and not the entire context of the locking operation....
Add ParseCpuMask() utility function
Also adds a generic ParseError exception.
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add test for some aspects of job queue
This new opcode and gnt-debug sub-command test some aspects of thejob queue, including the status of a job. The bug fixed in commit2034c70d507 was identified using this test. A future patch willrun this test automatically from the QA scripts....
KVM hypervisor: Use utils.ShellWriter for network script
This patch converts hv_kvm to use utils.ShellWriter for writingthe network script. It also adds a few unittests (the firstfor any hypervisor modules).
Move ShellWriter class to utils
Also add unittest.
Rename test for utils.IgnoreProcessNotFound
Usually our tests are named “Test…”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>
jqueue: Factorize code waiting for job changes
By splitting the _WaitForJobChangesHelper class into multiple smallerclasses, we gain in several places:
- Simpler code, less interaction between functions and variables- Easy to unittest (close to 100% coverage)...
Merge remote branch 'origin/devel-2.1'
Conflicts: test/ganeti.rapi.client_unittest.py: Trivial test/ganeti.rapi.rlib2_unittest.py: Trivial...
RAPI client: Implement old instance creation request format
Commit 8a47b4478 implemented instance creation in the RAPI client,but it left out support for the old instance creation request format.This patch now implements the old format as good as possible. This...