Convert LUConnectConsole to concurrency
For ConnectConsole we just need to lock the instance we're connectingto. We make a few rpcs to its primary node, but node daemons can nowhandle multiple queries and nodes cannot be removed till they haveinstances on them anyway. Note that since we return the ssh command, and...
Add _ExpandAndLockInstance auxiliary function.
LUs that take an instance name as input and need to expand its name andlock it can use it to simplify their ExpandNames call. Possibly, and_ExpandAndLockNode will come as well.
Reviewed-by: iustinp
Convert two (simple) LUs to be concurrent
LUQueryClusterInfo and LUDumpClusterConfig can be made concurrent anddon't need to acquire any locks. In fact they don't interact with thecluster at all, but just with its configuration, which is thread-safe by...
Add missing empty line
Two top level definitions were separated only by one empty line.Fixing this.
Reviewed-by: imsnah
Put the poper RAPI baserlib
Make argument to CleanCacheUnlocked mandatory
Not passing the argument means it has the value None. Iterating Nonedoesn't work: >>> "123" in None Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: iterable argument required...
Split RAPI resources to pieces
Split conditions in worker pool
This patch splits the single threading.Condition object used in theworker pool for synchronization into three.
- worker_to_pool: Notified if a worker wants to notify the pool- pool_to_worker: Notified if the pool wants to notify a single...
Handle signals in node daemon
This also fixes a TODO added by ultrotter by killing the parentprocess when QuitGanetiException is raised.
Reviewed-by: ultrotter
Use new signal handler class in master daemon
Add signal handler class
This signal handler class abstracts some of the code previouslyused in other places. It also uninstalls its handler when Reset()is called or the class is destructed, thereby restoring theprevious behaviour.
Implement jobs resource in RAPI
Breath life in to RAPI for trunk
Fork ganeti-noded
Create a new ForkingHTTPServer in ganeti-noded by deriving both fromNodeDaemonHttpServer and ForkingMixin. This will allow us to processconcurrent requests.
Documentation updates
Migrate RAPI QA to trunk.
Add apidoc makefile target
The patch adds the apidoc target and the epydoc config file for it. Notethat this is for epydoc 3.0 and that it will put the docs into./doc/api/.
The patch also adds a new .gitignore rule for the auto-generated rapifragment....
Rename BaseJO to BaseOpCode
Since we don't have for now a job definition object anymore, we renamethis class to BaseOpCode. It's still useful (and not merged with OpCode)since it holds all the 'pure' logic (no custom field handling, etc.)whereas OpCode holds opcode specific data (OP_ID handling, etc)....
Sort the job list in _GetJobIDsUnlocked
Since the IDs are integers, we can simply sort them.
Fix previous patch using workerpool in masterd
The function to stop a worker pool is TerminateWorkers(), not Shutdown().
Fix a syntax error in gnt-backup
I broke gnt-backup in rev 1035, sorry :(
Use workerpool in master daemon
Reusing threads instead of starting one for each request is more efficient.
Further fixes to enable RAPI startup
Note that since RAPI itself doesn't use luxi.Client yet, nothing works,but at least it can startup now.
Add forgotten RAPI constant
This was forgot on the forward-porting of RAPI.
Improve cli.SubmitOpCode
Currently, the feedback_fn argument to SubmitOpCode is no longer used.We still need it in burnin, so we re-enable it by making the code callthat function with the msg argument in case feedback_fn is callable. Thepatch also modifies burnin to accept the new argument format (msg is not...
First version of user feedback fixes
This patch contains a raw version for fixing feedback_fn.
The new mechanism works as follows: - instead of a per-Processor feedback_fn, there's one for each ExecOpCode, so that feedback for different opcodes go via possibly...
Cache some jobs in memory
This patch adds a caching mechanisms to the JobStorage. Note that isdoes not make the memory cache authoritative.
The algorithm is: - all jobs loaded from disks are entered in the cache - all new jobs are entered in the cache...
Fix JobStorage._GetJobIDsUnlocked
The job ID returned must be an integer (and the regex enforces that),but we didn't convert it manually.
Change JobStorage to work with ids not filenames
Currently some of the functions in JobStorage work with filenames (whichis an implementation detail and should only be used when dealing withthe storage) and not with job IDs. We need to change this in order to...
Add experimental persistency to job queue
It's not perfect and it's not finished, but it's a start.
- Serial number is read only once, but written on each update- Jobs are kept only on disk (caching will be implemented)
Convert backend.py to the logging module
The patch also switches some of the exception logs to uselogging.exception (and therefore the log message will have a diferentformat).
(Note that this might not be a good choice in all cases, though)
Add PID to all logs
This patch (for trunk) adds the PID to all daemon logs.
Fix rapi documentation
As a nice side-effect this also fixes "make distcheck".
The way used to include the generated rapi-resources.sgml file onlyworks if it's built in the same directory. That's not the case during"make distcheck". The patch changes the sed script used for variables...
Fix backend.NodeVolumes handling of LVM output
This is the same fix as for GetVolumeList.
I've checked manually and all other places that call lvm commands arealready checking the output validity in terms of correct number offields.
Copy the rest of the Restful-API files to trunk
Fix backend.GetVolumeList handling of LVM output
Sometimes ‘lvs’ can spit error messages on stdout, even when one wantsto parse the output:...Inconsistent metadata copies found - updating to use version 2776...
So we need to validate the output to guard against such cases....
Use new HTTP server classes in ganeti-noded
Add generic HTTP server classes
Some of the code is adopted from the 1.2 branch(lib/rapi/RESTHTTPServer.py). This code can be used as a base for thevarious HTTP servers in Ganeti.
Make "gnt-job list" work again
"gnt-job list" was broken after my recent changes in the RPCbetween clients and the master. This patch makes it work again.
Initial copy of RAPI filebase to the trunk
Remove more old job queue code
Apparently I forgot to this code when removing the rest.
Move watcher's LockFile function to utils
Switch _QueuedOpCode to have their own lock
Right now, the queued opcode doesn't have a lock, and instead relies onthe parent QueuedJob's lock.
This is not good for logging feedback, so it's better to have a lock foreach queuedopcode.
Add a simple decorator for instance methods
This is just a simple, hardcoded decorator for object methods needingsynchronization on the _lock instance attribute.
jqueue: Log more information when running opcodes
Fix double-logging in daemons
Currently, in debug mode, both the logfile handler and the stderrhandler will log debug messages. Since the stderr is redirected to thesame logfile (to catch non-logged errors), it means log entries aredoubled.
The patch adds an extra parameter to the logger.SetupDaemon() function...
ganeti-noded logging improvements
The patch adds some more logging to the node daemon:
- log methods at beggining not only at the end- log method parameters (they are very verbose, but useful)
A separate change is to initialize the global variable in the global...
Fix utils.py unittest
Move the master socket in the ganeti run dir
... as it was intended from the beggining, but by mistake left in thetop run dir.
Reduce duplicate Attach() calls in bdev
Currently, the 'public' functions of bdev (FindDevice andAttachOrAssemble) will call the Attach() method right after classinstantiation.
But the constructor itself calls this function, and therefore we haveduplicate Attach() calls (which are not cheap at all)....
Convert bdev.py to the logging module
This does not enhance in any way the messages; it just switches to thenew module.
Convert utils.py to the logging module
The patch also logs all commands executed from RunCmd when we are atdebug level.
Remove the old locking functions
This removes (hopefully) all traces of the old locking functions anduses.
Remove old job queue code
Change masterd/client RPC protocol
- Introduce abstraction class on client side- Use constants for method names- Adopt legacy function SubmitOpCode to use it
Make luxi RPC more flexible
- Use constants for dict entries- Handle exceptions on server side- Rename client function to CallMethod to match server side naming
Instantiate new job queue in master daemon
Add very simple job queue
Add a more comment lines to testLockingConstants
This is to discourage even more whoever may think that this requirementis not really useful and can be lifted, and to at least know where it'sused before trying to break it.
Convert LUTestDelay to concurrent usage
In order to do so: - We set REQ_BGL to False - We implement ExpandNames
That's it, really.
Processor: Acquire locks before executing an LU
If we're running in a "new style" LU we may need some locks, as requiredby the ExpandNames function, to be able to run. We'll walk up the locklevels present in the needed_locks dictionary and acquire them, then run...
LogicalUnit: add ExpandNames function
New concurrent LUs will need to call ExpandNames so that any namespassed in by the user are canonicalized, and can be used by hooks,locking and other parts of the code. This was done in CheckPrereqbefore, but it's now splitted out, as it's needed for locking, which in...
Processor: Move LU execution to its own method
This makes the try...finally code simplier, and helps adding a morecomplex locking structure before the actual execution. It also fixes aconcurrency bug caused by the fact that write_count was read beforeacquiring the BGL, and thus spurious config update hooks run could have...
Add a new LockSet unittest
This test checks the LockSet behaviour when an empty list is passed.The current behaviour is expected, but since this is a corner case,we're safer to keep it under a check, and if we need a different onemonitor that everything is as we expect it to be....
constants: Add job and opcode status strings
workerpool: Don't notify if there was no task
Workers have to notify their pool if they finished a task to makethe WorkerPool.Quiesce function work. This is done in the finally:clause to notify even in case of an exception. However, beforewe notified on each run, even if there was no task, thereby creating...
Create all SUB_RUN_DIRS in ganeti-noded
Rather than just creating BDEV_CACHE_DIR we loop through theSUB_RUN_DIRS list and create all its childs.
Add a top level RUN_GANETI_DIR constant
This patch creates a base RUN_GANETI_DIR and then moves the other rundir constants to use that (even if just setting BDEV_CACHE_DIR as equalto it, rather than putting it deeper, for now).
Also we create a constant list of all the subdirs we need in RUN_DIR to...
symlinks: Add DISK_LINKS_DIR constant
The DISK_LINKS_DIR points to the RUN_DIR/ganeti/instance-disksdirectory, which will contain symlinks to the instances' disks. Theseprovide a stable name accross all nodes for them, and permitlive-migration to happen....
luxi: Use serializer module instead of simplejson
serializer.DumpJson: Control indentation by parameter
If the simplejson module supports indentation, it's always used. Thereare cases where we might not want to use it or enable it only fordebugging purposes, such as in RPC.
Add a missing import to cmdlib
cmdlib uses some constants from locking (ie. locking levels) but doesn'timport it. This patch fixes the issue.
Fix an error accessing the cfg
Since the context is passed to LogicalUnit, rather than the cfg, we canonly access the cfg as self.cfg, self.context.cfg, or context.cfg (inthe constructor). cfg is not valid anymore.
Add and remove instance/node locks
Whenever we add an instance or node to the cluster (i.e. to the configand whenever we remove them we should add/remove locks as well). In thefuture we may want to optimize this so that the configwriter does it, orit's handled at the context level, but till we're adding/removing...
Pass context to LUs
Rather than passing a ConfigWriter to the LUs we'll pass the wholecontext, from which a ConfigWriter can be extracted, but we can alsoaccess the GanetiLockManager. This also fixes the places where a FakeLUis created.
mocks: create a FakeContext object
This will be passed to FakeLUs
Fix a typo in LUTestDelay docstring
Locking: remove LEVEL_CONFIG lockset
Since the ConfigWriter now handles its own locking it's not necessary tohave a specific level for the config in the Locking Manager anymore.This patch thus removes it, and all the unittest calls that used it, ordepended on it being present....
ConfigWriter: synchronize access
Since we share the ConfigWriter we need somehow to make sure thataccessing it is properly synchronized. We'll do it using thelocking.ssynchronized decorator and a module-private shared lock.
This patch also renames a few functions, which were called inside the...
Locking: add ssynchronized decorator
This patch creates a new decorator function ssynchronized in the lockinglibrary, which takes as input a SharedLock, and synchronizes access tothe decorated functions using it. The usual SharedLock semantics apply,so it's possible to call more than one synchronized function at the same...
ConfigWriter: remove _ReleaseLock
Remove empty function _ReleaseLock and all its calls. Since we onlyhave one configwriter per cluster the locking needs to cover all thedata in the object, and not just the file contents. Locking inConfigWriter will be handled using the ganeti locking library....
Fix some issues with the watcher
This patch fixes two bugs: - the state file is not saved because we use the method for checking for udpated data - in two places 'Error' was used instead of 'Exception', which breaks error handling
Additionally:...
Add generic worker pool implementation
Reuse the luxi client in cli.SubmitOpCode
By a mistake, we don't reuse the luxi client. As such, we open and closethe connection at each poll cycle and spam the server logs.
Add custom logging setup for daemons
It's better for daemons if: - they log only to one log file - the log level is included - for debug runs, the filename/line number is included
This patch moves the custom formatter from the watcher to the logging...
Remove custom locking code from gnt-instance
The gnt-instance script doesn't run in the same process anymore, so wecan't and don't have to unlock.
ganeti-masterd: Remove unused locking code
Reviewed-by: iustinp, ultrotter
ganeti-masterd: Use logging module
Reviewed-by: ultrotter, iustinp
Context: s/GLM/glm/
Make the GanetiLockManager instance of GanetiContext lowercase
Set locale when using docbook programs
At least docbook2man inserts a date formatted using the currentlocale into its output.
Update .gitignore
Reviwed-by: imsnah
Add a FirstFree function to utils.py
This function will return the first unused integer based on a list ofused integers (e.g. [0, 1, 3] will return 2).
Increase the thread size to 5
Now that we use the locking library to make sure running opcodes cannotstep on each other toes we can have a bigger thread size, andpotentially process many opcodes in a parallel manner.
Processor: acquire the BGL for LUs requiring it
If a LU required the BGL (all LUs do, right now, by default) we'llacquire it in the Processor before starting them. For LUs that don'twe'll still acquire it, but in a shared fashion, so that they cannot run...
Processor: pass context in and use it.
The processor used to create a new ConfigWriter when it was initialized.We now have one in the context, so we'll just recycle it. First of allwe'll pass the context in when creating a new Processor object, thenwe'll just use context.cfg, which is granted to be initialized, wherever...
Add REQ_BGL LogicalUnit run requirement
When logical units have REQ_BGL set (it is currently the default) theyneed to be the only ganeti operation run on the cluster, and we'llguarantee it at the master daemon level. Currently only one thread isrunning at a time, so this requirement is never broken....
Burnin doesn't need a Processor
In 2.0 burnin submits job to the master daemon, so it doesn't need tocreate an internal Processor anymore. Even if the processor is not usedanywhere in the burnin code it was still initialized as a leftover ofhow burnin used to work. Fixing this....
Implement “gnt-job list -o +...”
This adds the same “-o +...” functionality in gnt-job as in the node andinstance scripts.
Fix sstore handling in Processor
- no need to keep the sstore as an object member, remove it- don't reinitialize sstore only if self.cfg is None This is not an issue, as the Processor is recycled for every opcode, but in general we know that (a) we might need a different type of...
Remove duplicate code in hooks unittests
All the tests there used to creare a cfg, a sstore, an opcode and a LU.Put all the duplicate code in the setUp function.