History | View | Annotate | Download (48.2 kB)
Convert “gnt-debug locks” to query2
Locks can now be queried using “Query(what="lock", …)” over LUXI.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
locking: add nodegroup lock level
This also changes masterd to initialize the ganeti's manager with thecurrent list of nodegroup uuids, and updates unittests
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Merge branch 'devel-2.3'
Conflicts: NEWS: Trivial
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
locking: Clarify message for removed locks
Just being told that a lock doesn't exist can be confusing. One casewere this happens is when a job (e.g. instance modify) waits for a jobremoving the instance (e.g. export with remove).
Move locking.RunningTimeout to utils
As we need this functionality in other places than just locking it makessense to move it to utils rather than keeping it in locking
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
GanetiLockManager, remove default values
The nodes and instances parameters to the constructor are mandatoryanyway, as a value of None will fail when creating the LockSet. Ratherthan fixing this adding code lines, since we never used the defaultvalue, let's remove them and require that the parameters are passed....
locking: Implement priority in Ganeti lock manager
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
locking: Don't set default priority as keyword default
This allows users of these classes to simply pass None if they want to use thedefault value (the actual default is an internal constant), instead ofdynamically constructing the keyword arguments.
Merge branch 'devel-2.2'
locking.SharedLock: Update class docstring
This was already outdated when the initial version of SharedLock was addedin commit 162c1c1f1 (February 2008).
locking: Implement priorities in SharedLock and LockSet
For proper support of job priorities, jobs' locks need to respectpriorities. Otherwise it could happen that a job with a lower prioritycould get a lock before a job with a higher priority (depending on...
Show list of pending acquires in “gnt-debug locks”
This is accomplished by keeping a list of waiting threads insteadof just their number inside the lock-internal condition. A fewother tweaks to the output format are also made.
Add simple lock monitor
This patch adds an initial implementation of a lock monitor, accessiblefor the user through “gnt-debug locks”. It currently shows all resourcelocks: BGL, nodes and instances. Config and job queue locks could beshown too, but wouldn't be of much help. The current owner(s) and mode...
locking.LockSet: Use function to get member lock name
Implement lock names for debugging purposes
This patch adds lock names to SharedLocks and LockSets, that can be usedlater for displaying the actual locks being held/used in places where weonly have the lock, and not the entire context of the locking operation....
ssynchronized: act on a class member
The ssynchronized decorator takes the lock to act on in input.With this change we allow a string to be passed, and if so we assume thefunction it protects is a class method, and we act on the member of theclass itself named as the string we got....
Remove locking._CountingCondition
This class is unused and untested. We must have forgot it around.
_BaseCondition: allow saving/restoring state
SharedLock _acquire_restore and _release_save
If a shared lock is used inside a condition, we need to make sure thatit's reacquired in the same way as it was originally, after the wait.
Add separate module for backported language functionality
utils.py, where they were before, is already huge.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge remote branch 'origin/stable-2.1' into devel-2.1
Fix bug introduced in commit 413b747
While commit 413b747 fixed the issue of poll(2) returning toosoon, it didn't work when the poll(2) call should've beenblocking. This is now fixed and verified.
Fix locking bug causing high CPU usage
Iustin Pop noticed unusually high CPU usage with 2.1's masterdaemon, even with very simple opcodes like OP_TEST_DELAY. Asit turns out, we inadvertently passed seconds as millisecondsto a call to poll(2). Due to the way the loop around the call...
locking: add/fix @type information
This patch missing @type information for all public methods, modifiesone to conform to the rest, and removes some information from @paramwhen it's been expressed in @type.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix slots definitions
According to http://docs.python.org/reference/datamodel.html#slots
locking: Fix race condition in LockSet
This patch fixes a race condition when acquiring all locks ina LockSet instance. The list of lock names needs to be sortedto guarantee a consistent locking order, but the names were notsorted when acquiring all locks in the set....
locking: Append to list outside error handling block
locking: Don't fail in error handling if lock isn't owned
In case an exception was thrown while acquiring the lock, not necessarily allowned locks are also really acquired. Before this change, an exception could bemasked by another exception thrown here. There is no good clean-up strategy...
Add targeted pylint disables
This patch should have only:
- pylint disables- docstring changes- whitespace changes
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Merge branch 'devel-2.0' into devel-2.1
Add targetted pylint disables
This patch adds targeted pylint disables, where it makes sense (eitherdue to limitations in pylint or due to historical usage), and also a fewblanket ones in rapi where all the names are… “different”.
Signed-off-by: Iustin Pop <iustin@google.com>...
Clarifiy some more wide pylint disables
This removes/updates some module-wide pylint disables.
Remove quotes from CommaJoin and convert to it
This patch removes the quotes from CommaJoin and converts most of thecallers (that I could find) to it. Since CommaJoin does str(i) for i inparam, we can remove these, thus simplifying slightly a few calls....
locking: Convert pipe condition to new timeout class
locking.LockSet: Move timeout calculation to separate class
This class can also be used by mcpu.
locking, mcpu: Ensure timeout is always >= 0.0
locking.LockSet: Improve assertions
locking: Factorize LockSet.acquire
By moving the main code of LockSet.acquire to its own functionwe reduce the code complexity a bit and clarify the exceptionhandling.
This also fixes a case where a lock acquire timeout wasn'thandled correctly, leading to obscure error messages....
Code and docstring style fixes
Found using pylint and epydoc.
locking.LockSet: Implement acquire timeouts
The timeout passed to LockSet.acquire() is measured over all lock acquires. IfLockSet.acquire fails to acquire all requested locks within the specifiedamount of time, all locks are released again and the acquire fails....
More locking tests race conditions fixes
There were more race conditions. By adding a notify function toSharedLock.acquire we can prevent them.
SingleActionPipeCondition =~ s/Action/Notify/
With this patch we simplify usage on the SingleActionCondition (whichwasn't a condition at all) by making it a real condition. This way wecan just wait() on it, or notifyAll() as we would on a normal one. The...
Abstract "base" condition code in a separate class
Each condition has an underlying lock, the acquire and release methods,and a few helper methods to check that it's called in the proper way.
Abstract them to a separate class so we can have more than one without...
locking.SharedLock: Fix bug in delete function
SharedLock.__acquire_unlocked uses keyword parameters. Just passingthe timeout would set the “shared” parameter.
Rename LockSet.acquire parameter “blocking” to “timeout”
Also remove the “blocking” parameter from LockSet.remove andGanetiLockManager.remove. There's no point in implementing timeouts on removalunless we need them.
Change SharedLock to new pipe(2)-based condition
Add _PipeCondition class
_PipeCondition is a condition implemented using pipe(2) and poll(2).It allows the implementation of timeouts without using a busy-wait loopwith time.sleep.
Unlike Python's built-in threading.Condition class and to save filedescriptors and an internal queue, it can only be used to notify...
Add _SingleActionPipeCondition class
This class will be used as a basic block for pipe(2)-basedconditions. Upon initialization it creates a pipe and can benotified once (hence the “single action” in the name). Acallable helper class is used to wait for notifications....
SharedLock: implement timeouts
This patch greatly simplifies the SharedLock code and implementstimeouts for the acquire() and delete() functions. A wrapper aroundPython's threading.Condition class must be used to ensure threadsafety when check whether there are any waiters left....
locking: Acquire SharedLock in shared mode in separate function
This is for some more symetry withSharedLock.__exclusive_acquire.
locking: Add level name dict
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
locking: Don't swallow exceptions
This is an indentation bug.
Fix some typos
locking.LockSet: don't modify input arguments
Currently LockSet.acquire() sorts in place it's input argument if it's alist. This is not good, since callers might depend on a specificordering of the input data, and this is a 'hidden' modification.
We fix it by simply using a sorted copy, instead of sorting in place....
Re-wrap some lines to keep them under 80 chars
This non-code change rewraps some lines in locking.py to keep them under80 chars.
Reviewed-by: ultrotter
Fix epydoc format warnings
This patch should fix all outstanding epydoc parsing errors; as such, weswitch epydoc into verbose mode so that any new errors will be visible.
Reviewed-by: imsnah
LockSet: forbid add() on a partially owned set
This patch bans add() on a half-acquired set. This behavior waspreviously possible, but created a deadlock if someone tried to acquirethe set-lock in the meantime, and thus is now forbidden. ThetestAddRemove unit test is fixed for this new behavior, and includes a...
Fix typo in a locking.py comment
Add GanetiLockManager.is_owned function
This is a public version of the private function we already had.We don't just change the previous version because it had lots of usersin the library itself and in the testing code.
Fix LockSet._names() to work with the set-lock
If the set-lock is acquired, currently, the _names function will fail ona double acquire of a non-recursive lock. This patch fixes the behavior,and some lines of code added to the testAcquireSetLock test check that...
Add locking.ALL_SET constant and use it
Rather than specifying None in needed_locks every time, with a nicecomment saying to read what we mean rather than what we write, and thatNone actually means All, in our magic world, we'll hide this secretunder the ALL_SET constant in the locking module, which has value, you...
LockSet: allow lists with duplicate values
If a list with a duplicate value is passed to a lockset what the codenow does is to try to acquire the lock twice, generating adouble-acquire exception in the SharedLock code. This is definitely anissue. In order to solve it we can either forbit double values in a list...
Invert nodes/instances locking order
An implementation mistake from the original design caused nodes to belocked before instances, rather than after. This patch inverts the levelnumbering, changing also the relevant unittests and the recursivelocking function starting point....
Locking: remove LEVEL_CONFIG lockset
Since the ConfigWriter now handles its own locking it's not necessary tohave a specific level for the config in the Locking Manager anymore.This patch thus removes it, and all the unittest calls that used it, ordepended on it being present....
Locking: add ssynchronized decorator
This patch creates a new decorator function ssynchronized in the lockinglibrary, which takes as input a SharedLock, and synchronizes access tothe decorated functions using it. The usual SharedLock semantics apply,so it's possible to call more than one synchronized function at the same...
SharedLock: restrict assertion condition
When we release a shared lock if there are no exclusive waiter then the numberof shared waiters must be exactly equal to the ones scheduled to pass.
Reviewed-by: iustinp
Locking: fix one indentation error (codestyle)
Locking: fix superlong lines (codestyle)
Locking: remove an empty space at End Of Line
Increase SharedLock fairness
Previously if a shared thread was notified, together with the rest, and was notfast enough in waking up and acquiring the lock, another one could release it,decide there were no more sharers, and let an exclusive one in instead. With...
LockSet: handle empty case
A LockSet is mostly useful when it has some locks in it. On the other handthere are cases in which it must function even when empty. For example if acluster has no instances in it there's no reason why locking all of themshouldn't work anyway. This patch adds test code for that situation and...
LockSet: collapse two try/except into one
SharedLock: remove wrong assertion in code
r644 contained some cleanup code for LockSet. Among other things it removed asyntax error that allowed an assertion that previously wan't really checked totrigger. It turns out that even though the spirit of that assertion was correct...
Codestyle updates for locking code
LockSet: make acquire() able to get the whole set
This new functionality makes it possible to acquire a whole set, by passing"None" to the acquire() function as the list of elements. This will avoid newadditions to the set, and then acquire all the current elements. The list of...
LockSet: encapsulate acquire() in try-except
This patch adds a try/except area around most of the acquire() code (everythingafter the intial condition checks). Since the except: clause contains just a'raise' nothing really changes except the indentation of the code....
Make LockSet.__names() return a list, not a set
Previously the private version of the __names function returned directly a set.We'll keep this in the public interface but change the private version to alist in order to be able to sort() its result and then loop on it, even though...
LockSet: improve remove() api
Lockset's remove() function used to return a list of locks we failed to remove.Rather than doing this we'll return a list of removed locks, so it's moresimilar to how acquire() behaves. This patch also fixes the relevant unit tests....
LockSet: make acquire() return the set of names
In a LockSet acquire() returned True on success. This code changes that toreturn a set containing the names of the elements acquired. This is still atrue value if we acquired any lock but is slightly more useful (because if...
LockSet: invert try/for nesting in acquire()
This patch changes nothing to the functionality of a LockSet. Rather thantrying to do the whole for loop we try each of its steps. This opens the way tohandle differently a single failure.
Initial GanetiLockManager implementation
Includes some locking-related constants and explanations on how theLockManager should be used, the class itself and its test cases.
The class includes: - a basic constructor - functions to acquire and release lists of locks at the same level...
LockSet: make acquire() fail faster on wrong locks
This patch makes acquire() first look up all the locks in the dict and then tryto acquire them later. The advantage is that if a lockname is already wrongsince the beginning we won't need to first queue and acquire other locks to...
LockSet implementation and unit tests
A LockSet represents locking for a set of resources of the same type. A threadcan acquire multiple resources at the same time, and release some or all ofthem, but cannot acquire more resources incrementally at different times...
Add the delete() operation to SharedLock
This new operation lets a lock be cleanly deleted. The lock will be exclusivelyheld before deletion, and after it pending and future acquires will raise anexception. Other SharedLock operations are modify to deal with delete() and to...
Fix a couple of SharedLock docstrings
Use the actual class name rather than a spaced version of it.
Shared Lock implementation and unit tests.
Adding a locking.py file for the ganeti locking library. Its first component isthe implementation of a non-recursive blocking shared lock complete with atesting library.
Reviewed-by: imsnah, iustinp