ganeti.rpc: Read SSL certificate and key only once per request
There's no need to read the SSL certificate and key for every nodein a request. Also add a TODO for better error reporting.
Reviewed-by: amishchenko
Documentation updates for mcpu.py
This is the only change needed to make mcpu epydoc-compliant.
Reviewed-by: ultrotter
LUCreateInstance: Fix import mac AUTO mode
Previously on import LUCreateInstance used to recycle the mac if the instancename was the same than the one used at export time. Now we do the same, butapply the setting separately for each nic.
Reviewed-by: iustinp
LUCreateInstance unlock all nodes mid-way
When creating a new instance, after saving the instance data to the config fileand creating the disks, but before waiting for sync and installing the OS, werelease the node locks, to allow for more instance creations to proceed in...
IAllocator: subtract down instances from free mem
Currently free_memory just reports the amount of free ram, as seen by thehypervisor. We adjust this amount by subtracting the memory for any instancewhich is down, and the difference for any instance which is configured to have...
Correct GetAllInstancesInfo rtype
GetAllInstancesInfo, in the backend, returns just a dict, not a dict of dicts.
IAllocator: use the right hypervisor
Since the hypervisor is instance dependent we'll get one on instance creation,and use the one in the instance config on relocation.
IAllocator: fill i_list in a more proper way
- reuse the previously called cluster_info, rather than calling it again- get all the instances from the config atomically, to prevent race conditions- use a list comprehension, for simplicity
Parallelize instance operations on the same node
With static minors we don't have a race condition anymore whenstarting/stopping/rebooting/reinstalling more than one instance on the samenode, so we'll drop node locking altogether.
Convert iallocator to the new _ComputeDiskSize
_ComputeDiskSize's API was changed for multidisk support in r2010, butiallocator's call to it were not fixed. Converting them now.
Documentation updates for cmdlib.py
This makes cmdlib.py not throw epydoc errors anymore.
Only update ssconf on cluster serial change
There is no need to update ssconf if the cluster serial number has notchanged.
Enable auto-unit formatting in script output
This patch enables by default the old 'human-readable' option, but in aslightly different model.
The option is now called "units" and takes either: - 'h' for automatic formatting - 'm', 'g' or 't' for mebi/gibi/tebibytes...
RAPI: Cancel a job
Reviewed-by: imsnah
Make cli.py use FieldSet for matching fields
This changes cli.py to FieldSet usage so that gnt-instance list willformat nicely the disk.size/*, and the count of disks/nics.
Move FieldSet class to utils.py
Since we can use the FieldSet class in cli.py to nicely format disksizes and such, we move it to utils.py and also move its associatedunittest. I didn't remove the cmdlib.py unittest file as that's not thegood direction :)...
Change disk index validation to FindDisk
This patch replaces the hand validation of the disk index with theinstance.FindDisk method (actually reverting to previous method, beforethe multi-disk, but now with indexes).
Change GrowDisk to work with multi-disk
This patch changes the instance.FindDisk method to take index arguments(instead of iv_names), and changes GrowDisk and list instancesaccordingly.
Use SSL for master/node RPC
This patch enables SSL between masterd and noded.
Get rid of node daemon password
With the new SSL client certificate stuff it's no longer needed.
ganeti.http: Add another class to contain SSL key and certificate
Otherwise we would read them for every request the HTTP clientmakes against a server and this is not needed.
Reuse HTTP client pool for RPC
ganeti-masterd: Add initialization and shutdown of RPC pool. It needsto be shutdown before forking.
ganeti.cli: Add decorator function to initialize and shutdown RPC pool.
ganeti.rpc: Add functions to initialize and shutdown RPC pool. Throw...
Write ssconf files when updating configuration
Add RPC call to update ssconf files
Change replace secondary to work with multi-disk
Also fix an error in the CheckPrereq.
ganeti.ssconf: Add function to write ssconf files
This function will be used to write ssconf files from the node daemon.By creating a lock file, we synchronize different child processes ofganeti-noded to not overwrite each other's changes. Also, external...
Convert replace-disks (same nodes) to multi-disk
This patch changes the drbd8 replace disk only (no secondary change) towork in with multi-disk. This mode of replaces works correctly withreplacing only a subset of disks.
Initial multi-disk/multi-nic support
This patch adds support for mult-disk/multi-nic in: - instance add - burnin
The start/stop/failover/cluster verify work as expected. Replace diskand grow disk are TODO.
There's also a change gnt-job to allow dictionaries to be listed in...
Add more disk/nic listing options in gnt-instance
This adds some more listing cases (useful for scripting/rapi): - disk.sizes for a list of all sizes - nic.(ips|macs|bridges)
Change Xen hypervisor to not use iv_name
Currently the iv_name is very linux-specific, and will break with themulti-disk changes.
The patch changes this to generate sdX names based on the disk index inthe disks structure, instead of relying on the iv_name....
ganeti.rpc: Use central functions for actual RPC calls
Before we had lots, lots and lots of code duplication. This patchchanges the code to use four central functions.
Make HttpClientManager threadsafe
This allows a single HttpClientManager to be used from more than onethread at the same time. We discussed having one HttpClientManagerper job queue thread. Assuming there should be one HTTP thread pernode, this would mean quadratic growth with the number of nodes. By...
HTTP server: Do not decode empty entity body
RAPI: Instance modify.
Split parameters filter to the separate function and reuse it in instance creation.
Allow querying of variable number of parameters
This patch adds support for querying in gnt-instance list of: - disk.count - nic.count - disk.size/$N - nic.(ip|mac|bridge)/$N
The patch also disables the exception raised when the header description...
Convert cmdlib.py to _FieldSet
This patch converts the current usage of _CheckOutputFields to theFieldSet class, but it doesn't start to use its variable matchingfeatures.
Add a FieldSet class for variable parameter sets
This patch adds a _FieldSet class that can be used for the new variableparameter sets: e.g. the sda_size will change to disk/0.size (orsimilar) and we need to both check validity and extract the index of the...
RAPI: Instance startup/shutdown resources
RAPI: Implement an instance reboot
Use utils.ReadFile to read SSL certificate
ganeti.rpc: Use utils.ReadFile instead of reading file directly
utils.SetupLogging: Remove previously setup handlers
If a logging function is called before the logging module is setupusing utils.SetupLogging, it calls logging.basicConfig, which addsa StreamHandler, on its own. If we leave it in, all log output will...
jqueue: Always use rpc.RpcRunner
"from ganeti.rpc import RpcRunner" does not conform to the style guide.
bootstrap: Always use rpc.RpcRunner
ganeti.rpc: Convert to ganeti.serializer
This is one of the last places where the simplejson module isused directly.
Convert RPC module to new HTTP client
Currently, HttpClientManager is instantiated for every RPC call. Thiswill be changed with another patch, as will the use of SSL. The “Run”method is no longer needed.
Fix utils.KillProcess
Rev 1978 introduced a breakage on the SIGKILL finall signal to theprocess, due to mistyped variable.
Reviewed-by: francis.perron
ganeti.http: Implement SSL for HTTP client
Implementing SSL for the HTTP client required more work than I expected.For correct error handling, quite a lot of code is needed. To avoidcode duplication, I moved a lot of the socket handling code into asingle function named _SocketOperation. It takes care of the polling...
ganeti.daemon: Add timers to Mainloop
This is a fallout from my work on the HTTP client class. Repeatingtimers are prepared, but not yet implemented.
Convert trunk to posix-compatibility
We change two functions to use RunCmd without shell, and the other(which needs a ssh command line) is changed to the '>... 2>&1' syntax.
Remove zombie special case from IsProcessAlive
Based on the discussion on the mailing list, we remove the specialcasing of zombie processes.
Note we don't use kill -0 since that has a different meaning than "checkprocess is alive", so this function is still linux-specific....
Add cleanup of processes to utils.KillProcess
In case the process we want to signal is our own process, and the signalwe send is a deadly one, we should also cleanup after the process.
This patch adds a new parameter waitpid to this function that does this,...
Fix instance rename
As can be seen from the patch, we remove from the locking managerinst.name. However, we do this right after cfg.RenameInstance, whichchanges inst.name to the new name. So we need to remove old_name notthe new name.
The real question is why does the glm allow me to remove a not-existing...
Fix gnt-instance reinstall
Commit 1881 changed a rpc call but didn't fix all its users. This shouldfix it (but I can't test as HTTP is broken).
Implement working directory for utils.RunCmd
In order to replace usage of the shell just for 'cd %s; ...' we need tobe able to specify the working directory for childs.
The patch also changes the default working directory (when notspecified) to "/", as opposed to the current dir; since the daemos run...
ganeti.http: Move SSL socket creation into base class
The same code will be used by the HTTP client.
ganeti.daemon: Make Mainloop more flexible
While working on the HTTP client class, I wanted to use Mainloopbefore deciding to use threads instead.
- Add docstrings- Rename "quit" to "running", move it to a local variable- Support adding and removing I/O listeners while running...
ganeti.http: Add constant for "Unexpected EOF"
This is an expected error message and will be used in two places.
Add new HTTP client class
It is based on the WorkerPool class which is already used for the jobqueue and master daemon. Each request must be encapsulated in aninstance of HttpClientRequest, which will then be passed toHttpClientManager for processing. Upon completion, the request object...
ganeti.http: Use 411 Length Required in server code
Update LUGrowDisk _WaitForSync call
The _WaitForSync call changed in r1794 but the GrowDisk call wasn't updatedthen. This makes GrowDisk work in trunk.
ganeti.http: Add more constants
As a preparation for the new HTTP client class, add more constantsto ganeti.http.
Small documentation updates for workerpool.py
Documentation updates for jqueue.py
Yet another bug found while reviewing docs
The newer_than variable can be either None or an int, and we normalizeit to an integer previously and save it in the 'serial' variable, whichshould be used instead.
Documentation updates for utils.py
Update backend.py docstrings
This patch converts all of backend.py to epydoc formatting.
Fix another error handling case
The return from this error path is a dict, but the actual return value(on the non-error handling) is a list of dicts. Change accordingly.
Fix an error handling case
Found while reviewing documentation.
OSFromDisk remove superfluous empty line
Some documentation updates
This fixes a few doc issues and converts a few docstrings to epydoc.
Modify utils.RunCmd to write output to file
Currently we launch processes via the shell in a few places only toredirect standard output and error to a log file ("&> $file"). It isbetter to do such redirection from within RunCmd itself.
This patch splits RunCmd in two parts, the setup and the execution part,...
ganeti.http: Implement SSL certificates
Reviewed-by: killerfoxi
Document HttpServer.__init__
At the same time, simplify the interface a bit by not using a tuple.
Reviewed-by: killerfoxi, ultrotter
Fix gnt-* command with --submit
Currently we don't catch the JobSubmittedException (although the code toformat it nicely is there). The patch adds the exception name to thecatch list and makes --submit work.
Export the disk index in the import/export scripts
We want to export the disk index as some OSes will only want to exportthe first disk (or the second one, etc.), even if we have multipledisks.
The patch also updates the backend.ExportSnapshot docstring....
Fix leftover of hypervisor attribute rename
http library: Always fork before reading request
It turned out that clients not sending a full request will stopus from responding to further requests. This patch leverages thesituation a bit by always forking before handling the request,but we still have DoS situations....
daemon.py: Don't wake up every second
We don't have timers yet, hence waking up is not needed at all.
RAPI: Export beparams as dict. The patch also enables LUQueryInstances to accept beparams as valid field.
RAPI: Forgoten in r1923 instance add change.
LUCreateInstance: import multiple disks
Previously we used to handle only one disk. Now we'll import all theones present in the export, in order, on the instance's disks. Any diskwhich was present on the original instance but wasn't exported is justskipped in the target instance....
Convert ImportOSIntoInstance to OS API 10
- Change ImportOSIntoInstance not to get any "os_disk" and "swap_disk" arguments but to accept multiple target images to import, and to return a list of booleans with the result of each import- Change the relevant rpc call and the only caller to conform...
Convert ExportSnapshot to OS API 10
We pass the data via the environment rather than on the command line, asAPI 10 says. All the rest remains the same, and we export just one disk,as the master calls this function for every snapshotted disk.
LUExportInstance: snapshot all disks
Rather than just snapshotting the "sda" disk, we'll snapshot all of theinstance disks. If we can't snapshot a disk for any reason we'll log anerror and proceed anyway: in this case the resulting export will miss adisk. This also changes all the warning messages to self.LogWarning()....
Convert SnapshotBlockDevice's docstring to epydoc
Pass request headers in to RAPI handlers.
Move some LU logging to use proc.Log*
We change some messages which are obviously intended for the user to usethe proc.Log* functions instead of (only) the logging ones.
We also fix some wrong uses of feedback_fn.
Improve the mcpu.Processor logging routines
As discussed previously, many of the routinges in cmdlib.py are usinglogging functions as a carry-over from 1.2 (when these also showed themessage on stderr/to the user), instead of actually warning the user....
Fix whitespace-at-EOL
Please configure your editors to strip it, or enable your git hooks...
Set default hypervisor at cluster init
During cluster init, set the default hypervisor to be used for instances.Ensure that the default hypervisor belongs to the set enabled hypervisorsfor this cluster. Also fix a small bug with setting the default enabled...
Convert the job queue rpcs to address-based
The two main multi-node job queue RPC calls (jobqueue_update,jobqueue_rename) are converted to address-based calls, in order to speedup queue changes. For this, we need to change the _nodes attribute onthe jobqueue to be a dict {name: ip}, instead of a set....
Convert rpc.call_upload_file to use addresses
This patch allows rpc.call_upload_file to use addresses (if passed), andalso converts the ConfigWriter._DistributeConfig to pass them, sincethis is an often-done operation.
Convert non-static rpc calls to use addresses
This patch converts all non-staticmethods rpc calls to pass addresses tothe Client calls.
This is done via two new small helpers. These functions also deal withnodes that are not (yet) in the configuration file, and pass None for...
Add infrastructure for using addresses in rpc.py
The patch adds support for passing node addresses to NodeControllerobjects, which should speed up the connect sequence.
The patch also adds another very small optimization of computing thecontent-length header value in the parent Client class instead of each...
Improve conformance with the style guide for rpc.py
This patch renames some internal function to camel case, for compliancewith the style guide.
It also fixes the module docstring (how old was that?).
Use constants.VALUE_AUTO for ip comparison too
Forward-port-of: r1889, Reviewed-by: imsnah
LUCreateInstance: reuse mac address on import
When importing an instance with the same name as the exported one if themac address is "auto" we try to reuse the previous mac address. One canstill force generation of a new one with mac="generate".
Forward-port-of: r1887, Reviewed-by: iustinp...
LUCreateInstance: accept mac = VALUE_GENERATE
VALUE_GENERATE is handled exactly the same as VALUE_AUTO and the macaddress is generated randomly.
Forward-port-of: r1886, Reviewed-by: iustinp
Add VALUE_AUTO and VALUE_GENERATE constants
'auto' is used in multiple place in the code with a meaning similar to'default', 'generate' will be used to force generation of mac addresseswhen the default would be to reuse an old one.
Forward-port-of: r1884, Reviewed-by: iustinp...