ganeti.daemon: Add timers to Mainloop
This is a fallout from my work on the HTTP client class. Repeatingtimers are prepared, but not yet implemented.
Reviewed-by: iustinp
Convert trunk to posix-compatibility
We change two functions to use RunCmd without shell, and the other(which needs a ssh command line) is changed to the '>... 2>&1' syntax.
Reviewed-by: imsnah
Remove zombie special case from IsProcessAlive
Based on the discussion on the mailing list, we remove the specialcasing of zombie processes.
Note we don't use kill -0 since that has a different meaning than "checkprocess is alive", so this function is still linux-specific....
Add cleanup of processes to utils.KillProcess
In case the process we want to signal is our own process, and the signalwe send is a deadly one, we should also cleanup after the process.
This patch adds a new parameter waitpid to this function that does this,...
Fix instance rename
As can be seen from the patch, we remove from the locking managerinst.name. However, we do this right after cfg.RenameInstance, whichchanges inst.name to the new name. So we need to remove old_name notthe new name.
The real question is why does the glm allow me to remove a not-existing...
Fix gnt-instance reinstall
Commit 1881 changed a rpc call but didn't fix all its users. This shouldfix it (but I can't test as HTTP is broken).
Implement working directory for utils.RunCmd
In order to replace usage of the shell just for 'cd %s; ...' we need tobe able to specify the working directory for childs.
The patch also changes the default working directory (when notspecified) to "/", as opposed to the current dir; since the daemos run...
ganeti.http: Move SSL socket creation into base class
The same code will be used by the HTTP client.
ganeti.daemon: Make Mainloop more flexible
While working on the HTTP client class, I wanted to use Mainloopbefore deciding to use threads instead.
- Add docstrings- Rename "quit" to "running", move it to a local variable- Support adding and removing I/O listeners while running...
ganeti.http: Add constant for "Unexpected EOF"
This is an expected error message and will be used in two places.
Add new HTTP client class
It is based on the WorkerPool class which is already used for the jobqueue and master daemon. Each request must be encapsulated in aninstance of HttpClientRequest, which will then be passed toHttpClientManager for processing. Upon completion, the request object...
ganeti.http: Use 411 Length Required in server code
Update LUGrowDisk _WaitForSync call
The _WaitForSync call changed in r1794 but the GrowDisk call wasn't updatedthen. This makes GrowDisk work in trunk.
ganeti.http: Add more constants
As a preparation for the new HTTP client class, add more constantsto ganeti.http.
Small documentation updates for workerpool.py
Documentation updates for jqueue.py
Yet another bug found while reviewing docs
The newer_than variable can be either None or an int, and we normalizeit to an integer previously and save it in the 'serial' variable, whichshould be used instead.
Documentation updates for utils.py
Update backend.py docstrings
This patch converts all of backend.py to epydoc formatting.
Fix another error handling case
The return from this error path is a dict, but the actual return value(on the non-error handling) is a list of dicts. Change accordingly.
Fix an error handling case
Found while reviewing documentation.
Reviewed-by: ultrotter
OSFromDisk remove superfluous empty line
Some documentation updates
This fixes a few doc issues and converts a few docstrings to epydoc.
Modify utils.RunCmd to write output to file
Currently we launch processes via the shell in a few places only toredirect standard output and error to a log file ("&> $file"). It isbetter to do such redirection from within RunCmd itself.
This patch splits RunCmd in two parts, the setup and the execution part,...
ganeti.http: Implement SSL certificates
Reviewed-by: killerfoxi
Document HttpServer.__init__
At the same time, simplify the interface a bit by not using a tuple.
Reviewed-by: killerfoxi, ultrotter
Fix gnt-* command with --submit
Currently we don't catch the JobSubmittedException (although the code toformat it nicely is there). The patch adds the exception name to thecatch list and makes --submit work.
Export the disk index in the import/export scripts
We want to export the disk index as some OSes will only want to exportthe first disk (or the second one, etc.), even if we have multipledisks.
The patch also updates the backend.ExportSnapshot docstring....
Fix leftover of hypervisor attribute rename
http library: Always fork before reading request
It turned out that clients not sending a full request will stopus from responding to further requests. This patch leverages thesituation a bit by always forking before handling the request,but we still have DoS situations....
daemon.py: Don't wake up every second
We don't have timers yet, hence waking up is not needed at all.
RAPI: Export beparams as dict. The patch also enables LUQueryInstances to accept beparams as valid field.
RAPI: Forgoten in r1923 instance add change.
LUCreateInstance: import multiple disks
Previously we used to handle only one disk. Now we'll import all theones present in the export, in order, on the instance's disks. Any diskwhich was present on the original instance but wasn't exported is justskipped in the target instance....
Convert ImportOSIntoInstance to OS API 10
- Change ImportOSIntoInstance not to get any "os_disk" and "swap_disk" arguments but to accept multiple target images to import, and to return a list of booleans with the result of each import- Change the relevant rpc call and the only caller to conform...
Convert ExportSnapshot to OS API 10
We pass the data via the environment rather than on the command line, asAPI 10 says. All the rest remains the same, and we export just one disk,as the master calls this function for every snapshotted disk.
LUExportInstance: snapshot all disks
Rather than just snapshotting the "sda" disk, we'll snapshot all of theinstance disks. If we can't snapshot a disk for any reason we'll log anerror and proceed anyway: in this case the resulting export will miss adisk. This also changes all the warning messages to self.LogWarning()....
Convert SnapshotBlockDevice's docstring to epydoc
Pass request headers in to RAPI handlers.
Move some LU logging to use proc.Log*
We change some messages which are obviously intended for the user to usethe proc.Log* functions instead of (only) the logging ones.
We also fix some wrong uses of feedback_fn.
Improve the mcpu.Processor logging routines
As discussed previously, many of the routinges in cmdlib.py are usinglogging functions as a carry-over from 1.2 (when these also showed themessage on stderr/to the user), instead of actually warning the user....
Fix whitespace-at-EOL
Please configure your editors to strip it, or enable your git hooks...
Set default hypervisor at cluster init
During cluster init, set the default hypervisor to be used for instances.Ensure that the default hypervisor belongs to the set enabled hypervisorsfor this cluster. Also fix a small bug with setting the default enabled...
Convert the job queue rpcs to address-based
The two main multi-node job queue RPC calls (jobqueue_update,jobqueue_rename) are converted to address-based calls, in order to speedup queue changes. For this, we need to change the _nodes attribute onthe jobqueue to be a dict {name: ip}, instead of a set....
Convert rpc.call_upload_file to use addresses
This patch allows rpc.call_upload_file to use addresses (if passed), andalso converts the ConfigWriter._DistributeConfig to pass them, sincethis is an often-done operation.
Convert non-static rpc calls to use addresses
This patch converts all non-staticmethods rpc calls to pass addresses tothe Client calls.
This is done via two new small helpers. These functions also deal withnodes that are not (yet) in the configuration file, and pass None for...
Add infrastructure for using addresses in rpc.py
The patch adds support for passing node addresses to NodeControllerobjects, which should speed up the connect sequence.
The patch also adds another very small optimization of computing thecontent-length header value in the parent Client class instead of each...
Improve conformance with the style guide for rpc.py
This patch renames some internal function to camel case, for compliancewith the style guide.
It also fixes the module docstring (how old was that?).
Use constants.VALUE_AUTO for ip comparison too
Forward-port-of: r1889, Reviewed-by: imsnah
LUCreateInstance: reuse mac address on import
When importing an instance with the same name as the exported one if themac address is "auto" we try to reuse the previous mac address. One canstill force generation of a new one with mac="generate".
Forward-port-of: r1887, Reviewed-by: iustinp...
LUCreateInstance: accept mac = VALUE_GENERATE
VALUE_GENERATE is handled exactly the same as VALUE_AUTO and the macaddress is generated randomly.
Forward-port-of: r1886, Reviewed-by: iustinp
Add VALUE_AUTO and VALUE_GENERATE constants
'auto' is used in multiple place in the code with a meaning similar to'default', 'generate' will be used to force generation of mac addresseswhen the default would be to reuse an old one.
Forward-port-of: r1884, Reviewed-by: iustinp...
Remove the logger.py module
Since now we use only one function from the logger module(SetupLogging), we move it to utils.py (which is already imported by allusers of this function), and we remove the module.
Convert http.py to use the logging module
Convert cmdlib.py to use the logging module
Note that many uses of logger.Error were used in 1.2 for theirside-effect of logging to stderr, where the user will see the messages,and not for having the entry in the log. As such, we need to go over andreview every use of logging.* and decide if it should use feedback_fn...
Convert hv_xen.py to use the logging module
Convert mcpu.py to use the logging module
Convert ssh.py to use the logging module
Convert rpc.py to use the logging module
Convert cli.py to logging
We also add two function for printing messages, so that scripts won'thave to import logger to get these. They are a simple extension over thelogger ones, as they accept the call style from logging: ToStdout("Message: %s", msg)...
Fix gnt-instance modify with beparams
The gnt-instance modify didn't work correctly w.r.t the be parameters.There was also a typo in the corresponding LU.
Remove old HTTP server code
All users of this code have been migrated to the new and shiny HttpServerclass.
It also fixes a typo in the ApacheLogfile unittests. It has not yet beendecided whether we should keep ApacheLogfile or not, hence leaving it in....
Remove --hypervisor-type from gnt-cluster.
We no longer use a single, cluster-wide hypervisor, but configure theactual to be used hypervisor on the instance level.
Fix gnt-cluster init to set cluster defaults.
Add constants for cluster defaults
Cleanup os_add/rename rpc for OS API 10
- remove now unused osdev and swapdev arguments from backend, noded, rpc, cmdlib- convert docstrings to epydoc
Temporarily explicitely break import/export
Since they're not converted to API 10 yet, we temporarily disable theimport/export functions.
AddOSToInstance: convert to api10
RunRenameInstance: convert to api10
Add new OSEnvironment function
This function calculates the basic environment for OS scripts in APIversion 10.
Bump up the OS API version
OSFromDisk: use script names from constants
Change OSFromDisk's docstring to epydoc
Plus update it with the real variable name
Add LDS_BLOCK for block device backed dev types
Create constants to replace os scripts names
ETag passing support.
Mainloop: handle sigterm
This fixes removals of the pid files and clean exit in the init script
Mainloop: init sigchld_handler as None
There is no point in initializing sigchld_handler as an empty dict, whenit will be overwritten by an instance of utils.SignalHandlerimmediately after.
Enable gnt-cluster modify to hv/beparams
This patch enables the cluster modify to change: - enabled hypervisor list - hvparams (per hypervisor) - beparams (only the default group)
Syntax: gnt-cluster modify -B vcpus=3 -H xen-pvm:no_initrd_path
Validation for parameters is somewhat missing - the individual...
rapi: Convert to new HTTP server class
Requests are no longer logged to a separate file.
Reviewed-by: amishchenko
Show the cluster parametrs in gnt-cluster info
This is just a raw update without any special formatting.
RAPI: Instance add/delete implementation.
Fix job queue behaviour when loading jobs
Currently, if loading a job fails, the job queue code raises anexception and prevents the proper processing of the jobs in the queue.We change this so that unparseable jobs are instead archived (if notalready)....
Prevent master failover if we have wrong data
If we don't actually know the current master (as determined via voting),we prevent the failover.
The patch also changes some messages (capitalization, typos).
Improvements to the master startup checks
In order to account for future improvements to master failover, we movethe actual data gathering capabilities from ganeti-masterd intobootstrap.py, and we leave only the verification into masterd.
The verification procedure is then changed to retry multiple times (up...
Add an interface for the drain flag changes/query
This adds the set/reset in the jqueue and luxi modules, and a way toquery it in OpQueryConfigValues, and also the comand line interface forit:$ gnt-cluster queue infoThe drain flag is unset$ gnt-cluster queue drain...
Add a rpc call for changing the drain flag
A new multi-node call is added that sets/resets the drain flag.
Implement the job queue drain flag
We add a (per-node) queue drain flag that blocks new job submission.There is not yet an interface to add/remove the flag (will come in nextpatches).
Implement transport of ganeti errors across luxi
This patch adds a generic method to identify the ganeti error given itsclass name, and implements this across the luxi protocol.
Change the JobQueueError parent class
Currently this is not derived from GenericError, but there's no reasonfor it to be so.
rapi: Don't throw class as an exception
rapi: Whitespace fixes
grow-disk: wait until resync is completed
The patch adds a new ‘--no-wait-for-sync’ parameter to grow-disk similarto the one in instance add, and changes the default to wait.
This is cleaner as at the moment when the command returns, we eitherhave a fully synced disk or there is an error....
Convert the KVM hypervisor to hvparams
I'm not sure which parameters it should use, I took all I saw in thecode. Update: ultrotter says the list is OK for now.
Some fixes related to auto_balance
Change the constant name to match the value (autobalance ->auto_balance).
Also add the auto_balance header so that gnt-instance can list it.
Fix auto_balance and memory modification
The secondary nodes should not be checked for free memory in case of nonauto-balanced instances.
This is a forward-port from rev 1567 on the 1.2 branch.
Use the auto_balance in cluster verify
This patch changes the cluster verify in two ways: - do not use instances marked with auto_balance=False in N+1 memory checks - report, if any, the number of instances with auto_balance=False
Forward port from rev 1562 on the 1.2 branch....
Fix ordering of instance/nodes
After the locking changes, and the partial locking of nodes, theinstance/node list is not ordered anymore. Reorder so that it behaves asbefore.
Remove old instance attributes
Change over to beparams
This big patch changes the master code to use the beparams. Errors mighthave crept in, but it passes a small burnin.