Fix logic bug in rev 2072
In revision 2072 "ConfigWriter: change cluster serial meaning" I misreadthe serial_no update logic: it was about updating the serial number onthe object itself, not on the cluster.
So we don't actually have at all cluster serial number increase when a...
jqueue: fix a bug in an error path
Dictionaries raise KeyError, and not ValueError when invalid keys arepasses to del.
Reviewed-by: imsnah
ConfigWriter: change cluster serial meaning
Currently, we increase the cluster serial number for instance additions,removals and renames. This is conforming with the REST paradigm, howeverit means that for each of these operations, we need to push ssconf...
instance import: adapt to multi-disk/nic world
This is mostly a copy from gnt-instance add; import works, but it's notoptimal - device count/parametrs should be able to be reused.
Reviewed-by: ultrotter
Fix gnt-backup export
This patch fixes a bug in disk calculation for gnt-backup export, whichcompletely broke one-disk instance export.
The patch also corrects some error messages and style issues.
Fix a message in LUExportInstance
We never verified the node name before, so this is most likely not anon-retrieve but a wrong name case.
Allow network-less instances
Currently there's no way to specify no NICs for an instance, even thoughthis is a supported configuration.
The patch adds a --no-nics option to gnt-instance add.
Reviewed-by: amishchenko
Fix gnt-job submit
The submit function was using a very old API. This patch brings it up todate.
Small change to job failure output
Currently, job failures are done by raising OpExecError(job result). Fora one-opcode job that failed, this is very non-intuitive:Failure: command execution error:[u'Disk size change not possible, use grow-disk']
This patch changes the output in two ways:...
Fix file-based block devices
We changed a while ago the protocol for opening block devices, butFileStorage was not changed. This patch makes it work again.
Fix instance creation
This patch fixes the diskless and drbd/file based instances. Sorry :(
ganeti-masterd: create RUN_GANETI_DIR as well
Since we're not sure ganeti-noded has started yet, we need to createRUN_GANETI_DIR before SOCKET_DIR as well, with the proper permissions.
convert run dir mode to constant
ganeti-noded used to create all directories under /var/run with anhard-coded mode. convert it to a constant.
jqueue: Log progress and load jobs one by one
By logging more information, a user can see how far it is in inspectingthe queue. This can be useful with a large number of jobs. Also, insteadof loading all jobs in one go, load only the list of job IDs and then...
jqueue: Shutdown workerpool in case of a problem
RAPI: Make calls safier
Reduce duplication of work in rpc.Client
This patch removes the duplicate serialization and calls toutils.GetNodeDaemonPort in rpc.Client, and instead moves them to callingfunctions (the _*NodeCall ones recently introduced).
Move the MASTER_SOCKET to SOCKET_DIR
Before it was in the abstract linux namespace, where unfortunately wecouldn't easily check from python the credentials of the connectingclients. Now we also have to remove the file on exit and when starting.
ganeti-masterd: create SOCKET_DIR
If SOCKET_DIR doesn't exist we create it in the master daemon, beforetrying to put a socket inside it.
Add SOCKET_DIR_MODE constant
We want the socket dir to have a restricted permission.
Add SOCKET_DIR constant
This new directory under /var/run will be used for file based unixsockets.
Add very basic online help to devel/upload
Implement support for multi devices changes
This big patch adds support for: - changing NIC/disks in the multi-device model - adding/removing NICs - adding/removing disks
The patch is big and not very nice; the error checking paths are notvery clear....
Slight change to the LU initialization code
This patch adds support for a separate LU.CheckArguments() method whichshould do syntactic checks without holding locks and without pollutingthe ExpandNames which is a lock-related function. See for example the...
Fix a bug in LUSetInstanceParams
The wrong names were reused in a copy-paste.
Show disk access mode in gnt-instance info
The mode parameter needs to be exported and shown in the info output.
Change _GenerateDiskTemplate iv_name generation
Currently the _GenerateDiskTemplate assumes it does initial creation ofdisks (i.e. it starts with index 0).
For dynamic disk adds, we need to pass an additional offset. This patchadds this offset and modifies its sole current caller....
Slighly improve multi-nic in gnt-instance info
Previously we had only one nic, so the NIC information was all on oneline. This patch changes it to:
- NICs: - nic/0: MAC: aa:00:00:e8:b2:ef, IP: None, bridge: None - nic/1: MAC: aa:00:00:53:ca:92, IP: None, bridge: xen-br0...
Pass ssconf values from master to node
Instead of parsing the configuration on the node, we pass the ssconfvalues from the master.
Reviewed-by: iustinp
ganeti.http: Don't reuse key and cert objects
Reusing the private key and certificate objects gave us problems. Thispatch changes the code to only cache the PEM data, but the objectsthemselves. For every socket, the private key and certificate objectsare created again....
Fix unittests broken by rev 2015
Ssconf files shouldn't be updated when running unittests.
ganeti.rpc: Read SSL certificate and key only once per request
There's no need to read the SSL certificate and key for every nodein a request. Also add a TODO for better error reporting.
Documentation updates for mcpu.py
This is the only change needed to make mcpu epydoc-compliant.
LUCreateInstance: Fix import mac AUTO mode
Previously on import LUCreateInstance used to recycle the mac if the instancename was the same than the one used at export time. Now we do the same, butapply the setting separately for each nic.
Burner: fix ImportExport
Update the call to CreateInstance used in ImportExport to the new API formultidisk/nic.
LUCreateInstance unlock all nodes mid-way
When creating a new instance, after saving the instance data to the config fileand creating the disks, but before waiting for sync and installing the OS, werelease the node locks, to allow for more instance creations to proceed in...
IAllocator: subtract down instances from free mem
Currently free_memory just reports the amount of free ram, as seen by thehypervisor. We adjust this amount by subtracting the memory for any instancewhich is down, and the difference for any instance which is configured to have...
Correct GetAllInstancesInfo rtype
GetAllInstancesInfo, in the backend, returns just a dict, not a dict of dicts.
IAllocator: use the right hypervisor
Since the hypervisor is instance dependent we'll get one on instance creation,and use the one in the instance config on relocation.
IAllocator: fill i_list in a more proper way
- reuse the previously called cluster_info, rather than calling it again- get all the instances from the config atomically, to prevent race conditions- use a list comprehension, for simplicity
Parallelize instance operations on the same node
With static minors we don't have a race condition anymore whenstarting/stopping/rebooting/reinstalling more than one instance on the samenode, so we'll drop node locking altogether.
Convert iallocator to the new _ComputeDiskSize
_ComputeDiskSize's API was changed for multidisk support in r2010, butiallocator's call to it were not fixed. Converting them now.
Documentation updates for cmdlib.py
This makes cmdlib.py not throw epydoc errors anymore.
Slight change to the apidoc build
Currently the temporary directory used for modules is under the currentdirectory; this patch changes it so a temp directory is used; so wecould build the apidocs with a R/O filesystem.
Only update ssconf on cluster serial change
There is no need to update ssconf if the cluster serial number has notchanged.
Enable auto-unit formatting in script output
This patch enables by default the old 'human-readable' option, but in aslightly different model.
The option is now called "units" and takes either: - 'h' for automatic formatting - 'm', 'g' or 't' for mebi/gibi/tebibytes...
Improvements to CLI output
This patch adds nicer formatting for some gnt-instance list operations.
RAPI: Cancel a job
Make cli.py use FieldSet for matching fields
This changes cli.py to FieldSet usage so that gnt-instance list willformat nicely the disk.size/*, and the count of disks/nics.
Move FieldSet class to utils.py
Since we can use the FieldSet class in cli.py to nicely format disksizes and such, we move it to utils.py and also move its associatedunittest. I didn't remove the cmdlib.py unittest file as that's not thegood direction :)...
Small burnin info message change
This looks beter than '0 disk'.
burnin: fix default growth parameter
Change disk index validation to FindDisk
This patch replaces the hand validation of the disk index with theinstance.FindDisk method (actually reverting to previous method, beforethe multi-disk, but now with indexes).
Change GrowDisk to work with multi-disk
This patch changes the instance.FindDisk method to take index arguments(instead of iv_names), and changes GrowDisk and list instancesaccordingly.
Use SSL for master/node RPC
This patch enables SSL between masterd and noded.
Get rid of node daemon password
With the new SSL client certificate stuff it's no longer needed.
ganeti.http: Add another class to contain SSL key and certificate
Otherwise we would read them for every request the HTTP clientmakes against a server and this is not needed.
ganeti-masterd: Remove PID file at the end
Removing the PID file should be the last thing done. This patch makessure it's also removed when master.server_cleanup() throws an exception.
Also initialize logging only after writing the PID file.
Reuse HTTP client pool for RPC
ganeti-masterd: Add initialization and shutdown of RPC pool. It needsto be shutdown before forking.
ganeti.cli: Add decorator function to initialize and shutdown RPC pool.
ganeti.rpc: Add functions to initialize and shutdown RPC pool. Throw...
Write ssconf files when updating configuration
Add RPC call to update ssconf files
Change replace secondary to work with multi-disk
Also fix an error in the CheckPrereq.
ganeti.ssconf: Add function to write ssconf files
This function will be used to write ssconf files from the node daemon.By creating a lock file, we synchronize different child processes ofganeti-noded to not overwrite each other's changes. Also, external...
Convert replace-disks (same nodes) to multi-disk
This patch changes the drbd8 replace disk only (no secondary change) towork in with multi-disk. This mode of replaces works correctly withreplacing only a subset of disks.
Initial multi-disk/multi-nic support
This patch adds support for mult-disk/multi-nic in: - instance add - burnin
The start/stop/failover/cluster verify work as expected. Replace diskand grow disk are TODO.
There's also a change gnt-job to allow dictionaries to be listed in...
Add more disk/nic listing options in gnt-instance
This adds some more listing cases (useful for scripting/rapi): - disk.sizes for a list of all sizes - nic.(ips|macs|bridges)
Change Xen hypervisor to not use iv_name
Currently the iv_name is very linux-specific, and will break with themulti-disk changes.
The patch changes this to generate sdX names based on the disk index inthe disks structure, instead of relying on the iv_name....
ganeti.rpc: Use central functions for actual RPC calls
Before we had lots, lots and lots of code duplication. This patchchanges the code to use four central functions.
Make HttpClientManager threadsafe
This allows a single HttpClientManager to be used from more than onethread at the same time. We discussed having one HttpClientManagerper job queue thread. Assuming there should be one HTTP thread pernode, this would mean quadratic growth with the number of nodes. By...
HTTP server: Do not decode empty entity body
RAPI: Instance modify.
Split parameters filter to the separate function and reuse it in instance creation.
Skip another testWaitingXblocksY
The waiting sharer blocks exclusive is another not-possible to testright now.
Sorry for missing it the first time; there are no other testWaiting...tests.
Allow querying of variable number of parameters
This patch adds support for querying in gnt-instance list of: - disk.count - nic.count - disk.size/$N - nic.(ip|mac|bridge)/$N
The patch also disables the exception raised when the header description...
Convert cmdlib.py to _FieldSet
This patch converts the current usage of _CheckOutputFields to theFieldSet class, but it doesn't start to use its variable matchingfeatures.
Add a FieldSet class for variable parameter sets
This patch adds a _FieldSet class that can be used for the new variableparameter sets: e.g. the sda_size will change to disk/0.size (orsimilar) and we need to both check validity and extract the index of the...
Skip the waitingExclusiveBlockSharer test
Until we have a way to properly test this, we'll skip the test.
More updates to the documentation
This patch removes the client-api.txt document (since it's obsoleted bythe documentation inside the design-2.0.rst document) and adds manyupdates to the latter.
Fix distcheck with the new dot/png rules
Adding a new filetype/make rule requires more work than the previouspatch. Sorry! Now pases distcheck.
Add a cluster architecture diagram
This adds a simple diagram for the cluster architecture and also adds adependency on graphviz (both doc and in configure).
Combine the 2.0 design documents into one
This patch combines all the design documents for 2.0 except thesecurity one into a single document, in order to ease reading and reduceduplication of information.
Future patches will start removing wrong pointers to old document names...
RAPI: Instance startup/shutdown resources
RAPI: Implement an instance reboot
Use utils.ReadFile to read SSL certificate
ganeti.rpc: Use utils.ReadFile instead of reading file directly
utils.SetupLogging: Remove previously setup handlers
If a logging function is called before the logging module is setupusing utils.SetupLogging, it calls logging.basicConfig, which addsa StreamHandler, on its own. If we leave it in, all log output will...
jqueue: Always use rpc.RpcRunner
"from ganeti.rpc import RpcRunner" does not conform to the style guide.
bootstrap: Always use rpc.RpcRunner
ganeti.rpc: Convert to ganeti.serializer
This is one of the last places where the simplejson module isused directly.
Convert RPC module to new HTTP client
Currently, HttpClientManager is instantiated for every RPC call. Thiswill be changed with another patch, as will the use of SSL. The “Run”method is no longer needed.
Convert the locking unittests to repetition-test
Currently the locking tests are using timeouts to ensure an event will'never happen'. However, this is suboptimal.
The patch converts all of these to instead: not wait, but sequence theoperations logically and expect that they execute as such. In case of...
Fix utils.KillProcess
Rev 1978 introduced a breakage on the SIGKILL finall signal to theprocess, due to mistyped variable.
Reviewed-by: francis.perron
ganeti.http: Implement SSL for HTTP client
Implementing SSL for the HTTP client required more work than I expected.For correct error handling, quite a lot of code is needed. To avoidcode duplication, I moved a lot of the socket handling code into asingle function named _SocketOperation. It takes care of the polling...
ganeti.daemon: Add timers to Mainloop
This is a fallout from my work on the HTTP client class. Repeatingtimers are prepared, but not yet implemented.
Convert trunk to posix-compatibility
We change two functions to use RunCmd without shell, and the other(which needs a ssh command line) is changed to the '>... 2>&1' syntax.
Abstract runtime creation of dirs into a function
Currently the dir creation in ganeti-noded is in the main function. Thisis not nice: we move it into a separate function and also add creationof the OS_LOG_DIR (with different permissions, but in the same way)....
Remove zombie special case from IsProcessAlive
Based on the discussion on the mailing list, we remove the specialcasing of zombie processes.
Note we don't use kill -0 since that has a different meaning than "checkprocess is alive", so this function is still linux-specific....
Add cleanup of processes to utils.KillProcess
In case the process we want to signal is our own process, and the signalwe send is a deadly one, we should also cleanup after the process.
This patch adds a new parameter waitpid to this function that does this,...
Fix instance rename
As can be seen from the patch, we remove from the locking managerinst.name. However, we do this right after cfg.RenameInstance, whichchanges inst.name to the new name. So we need to remove old_name notthe new name.
The real question is why does the glm allow me to remove a not-existing...
Make the makefile not recreate ganeti every time
Currently the makefile always recreates the ganeti symlink. Make it sothat it does only when it doesn't exist or it's not a symlink.
Fix gnt-instance reinstall
Commit 1881 changed a rpc call but didn't fix all its users. This shouldfix it (but I can't test as HTTP is broken).