Add a rpc call for changing the drain flag
A new multi-node call is added that sets/resets the drain flag.
Reviewed-by: imsnah
Implement the job queue drain flag
We add a (per-node) queue drain flag that blocks new job submission.There is not yet an interface to add/remove the flag (will come in nextpatches).
Implement transport of ganeti errors across luxi
This patch adds a generic method to identify the ganeti error given itsclass name, and implements this across the luxi protocol.
Change the JobQueueError parent class
Currently this is not derived from GenericError, but there's no reasonfor it to be so.
rapi: Don't throw class as an exception
Reviewed-by: ultrotter
rapi: Whitespace fixes
gnt-backup: update for cluster parameters
- add backend and hypervisor parameters- fix beparams validation/passing- pass hypervisor and hvparams- remove deprecated flags
Reviewed-by: iustinp
gnt-instance fix ValidateBeParams call
ValidateBeParams does not return, but its return value is assigned to avariable which is never used. Avoid this assignment.
gnt-instance remove deprecated flags
kernel, initrd, hvm_boot_order and vnc_bind_address are now hypervisorparameters and should not have their own flag. Moreover querying ofvnc_bind_address should of course pass through the hv/ namespace.
grow-disk: wait until resync is completed
The patch adds a new ‘--no-wait-for-sync’ parameter to grow-disk similarto the one in instance add, and changes the default to wait.
This is cleaner as at the moment when the command returns, we eitherhave a fully synced disk or there is an error....
Burnin OpGrowDisk
With this patch both the os and the swap disk are grown duringburnin. You can pass an increase size of 0 to skip this operation.
and:
burnin: don't try to grow diskless instances
When burnin is run on a diskless instance, it fails when trying...
Convert the KVM hypervisor to hvparams
I'm not sure which parameters it should use, I took all I saw in thecode. Update: ultrotter says the list is OK for now.
Some fixes related to auto_balance
Change the constant name to match the value (autobalance ->auto_balance).
Also add the auto_balance header so that gnt-instance can list it.
Fix auto_balance and memory modification
The secondary nodes should not be checked for free memory in case of nonauto-balanced instances.
This is a forward-port from rev 1567 on the 1.2 branch.
Use the auto_balance in cluster verify
This patch changes the cluster verify in two ways: - do not use instances marked with auto_balance=False in N+1 memory checks - report, if any, the number of instances with auto_balance=False
Forward port from rev 1562 on the 1.2 branch....
Fix ordering of instance/nodes
After the locking changes, and the partial locking of nodes, theinstance/node list is not ordered anymore. Reorder so that it behaves asbefore.
Remove old instance attributes
Change over to beparams
This big patch changes the master code to use the beparams. Errors mighthave crept in, but it passes a small burnin.
Change the backend to use the beparams
The backend.FinalizeExport function is changed to use the beparamsinstead of the instance attributes. Future enhancements should be donein order to export and import/reuse the whole be/hv params.
Change hypervisors to use the beparams
This patch changes the hypervisor functions to use the beparams dictinstead of the instance attributes.
Correctly fill the beparams too over RPC
As for the hvparams, we need to replace the sent instance beparams.
Add cli function for beparams validation
This is used to validate the parameters in both “gnt-instance add” and“gnt-backup import”.
Modify the beparams constants
Memory is renamed for easier use.
Fix a bug with instance creation and hvparameters
When creating an instance, we need to check not the opcode hvparams, butthe final, filled hvparams against validity. While we do this for theremote node calls (i.e. ValidateParams), we didn't do this for...
Allow instance info to only query the config file
This patch adds a new '-s' parameter to ‘gnt-instance info’ that makesit return only 'static' information. This is much faster, especially fordrbd instances.
This is a forward-port of rev 1570 on the ganeti-1.2 branch, resending...
Convert gnt-instance info to the hvparams model
Some informations are not printed nicely (e.g. “virtual CDROM: False”),but this is the first step.
Change gnt-instance modify to the hvparams model
Change gnt-instance list to the hvparams model
This is just a change of the various hvm_ and pvm parameters to the hvmodel. Parameters are queried via hv/$name or via the whole dict asreturned by hvparams.
Reviewed-by: ultrotter,imsnah
Switch instance hypervisor parameters to hvparams
This big patch changes instance create to the new hvparams structure.Old parameters are removed, so old jobs or old instances file will breakcurrent clusters.
Temporary fix for dual hvm/pvm instances
We have a problem with the current model of combining instance listsfrom multiple hypervisors: we don't allow duplicates, but "xm list" gives the same output for both pvm and hvm. This is a lack in the actualxen hypervisor implementation/split between pvm and hvm, but for now we...
Implement FillHV for instance-related rpc calls
We fill the instance hvparams with cluster defaults, and send a modifieddict over the wire to the node methods - they don't know anything aboutcluster defaults versus instance parameters.
Export the hypervisor.ValidateParameters over RPC
The newly-added node-specific ValidateParams hypervisor method isexported over RPC, using the semi-standard (success, message) returnvalue. Multi-node call, so that we call on both primary and secondary at...
Implement the new hypervisor API in base/xen
This patch adds the new methods CheckParameterSyntax and theValidateParameters in the base and xen hypervisors. It also changes thexen hypervisor to use the new hvparams field on instance.
Get rid of httperror module
This was a leftover from the early days of the remote API in Ganeti 1.2.
Implement parameter removal in SplitKeyVal
This patch adds paramter removal in SplitKeyVal, by prefixing avalue-less key with "-"; this is needed in resetting parameters back tocluster defaults, but care must be applied now that None can come fromthe parser....
Add the hv/be params object attributes
This patch adds the instance and cluster be/hvparams attributes, and theFill* methods on the cluster object. Nothing else is changes, exceptthat these attributes exist now on the objects.
Add constants for the HV/BE parameter names
Since we don't want the string values of the parameters (e.g.“kernel_path”) spread over the code, we introduce constants for these.
Reviewed-by: ultrotter,schreiberal
Fix a few rpc-related errors
This fixes: - whitespace change, double lines between methods - duplication of call_upload_file, introduced by mistake in rev 1795 and which went undetected because of the many changes in that ref (only diff -b shows it clearly)...
Add two new options types for CLI usage
For the new 2.0-style command line options, we need to parse strings ofthe type: ident:key=val[,...]and key=val[,...]
This patch adds two new option builders for these two, which return(ident, {key=val,}) and {key=val,} for the above two formats. It also...
Some small typos on design docs
Update the cluster parameters design doc
As discussed, we update the design doc with the three classes ofparameters and other changes.
Abstract checking own address into a function
Currently, we check if we have a given ip address (i.e. it's alive onone of our interfaces) but manually calling TcpPing(source=localhost).This works, but having it spread all over the code makes it hard to...
Mark call_node_leave_cluster as a static method
OS API: support for multiple versions in an OS
Allow multiple api versions in an OS. This is according to the OS APIchanges design doc, by which an OS can support multiple versions of theGaneti API and if one is supported by Ganeti it will work. Since up to...
LUVerifyCluster: fix error from rpc call
When calling node_verify leads to an error _VerifyNodes tries to iterateover a non-sequence. Catch the error before and avoid this fromhappening.
Convert ganeti-noded to new HTTP server class
Add new HTTP server implementation
This patch adds another implementation of an HTTP server. It'sbased on code of Python's BaseHTTPServer, from both version2.4 and 3k. In the future we can write code to decide whetherwe should fork for a request or not. Keep-alive is not supported....
Add daemon library with mainloop
This mainloop can be used in daemons like ganeti-noded.
Some updates on the job queue design doc
This clarifies the job storage and the reason for choosing it.
Update design-2.0-job-queue to reflect changes
With change 1773 a new status WAITLOCK was introduced if a job/opcode iswaiting for a lock. This change updates the document about the job-queueaccordingly.
Convert rpc module to RpcRunner
This big patch changes the call model used in internode-rpc fromstandalong function calls in the rpc module to via a RpcRunner class,that holds all the methods. This can be used in the future to enablesmarter processing in the RPC layer itself (some quick examples are not...
Cleanup in cmdlib for standalone function calls
This patch is a cleanup of the standalone functions in cmdlib. Many ofthem too as argument a ConfigWriter instance, but some also took otherparameters from the lu (e.g. proc), and in the future, if we want to...
Small random fixes
Indentation in bootstrap was wrong and some names in cmdlib.py were notright.
Move instance hypervisor check to ExpandNames
This check can be done earlier, in ExpandNames, and is needed here forthe hypervisor parameter check.
Update documentation & man pages for changed hypervisor names.
Ajust config unittest.
Update scripts and qa config for changed hypervisor names.
Shorten variable names.
Sanitize the hypervisor names
Since in 2.0 the user will possibly have more interaction with thehypervisor names, we sanitize them by removing the version numbers(the version can be a prerequisite for the ganeti installation, weshouldn't document it in variable names)....
Also export OS_API to the OS scripts
The idea is that if the OSes will support multiple version (e.g. both1.2 and 2.0), then Ganeti should be able to talk to it using version2.0, but then the script needs to be told nicely what version Ganeti isusing....
Fix for gnt-cluster init.
Move the hypervisor attribute to the instances
This (big) patch moves the hypervisor type from the cluster to theinstance level; the cluster attribute remains as the default hypervisor,and will be renamed accordingly in a next patch. The cluster also gains...
Updates to the security document
This patch changes formatting and the DRBD shared secret details, andadds master daemon socket details to the security doc.
Move the SECURITY document to the doc/ dir
Fix formatting in design-2.0-os-interface
Small changes to the index design doc
This is just some additions of not-yet-mentioned docs.
Change default instance reboot type to hard.
Merged r1777 from branches/ganeti/ganeti-1.2
Add new design docs to Makefile.am
rpc.call_instance_migrate: pass the whole instance
Currently the call_instance_migrate call only passes the instance name;we need to pass the whole object for the hypervisor_type changes (allthe other individual instance rpc calls already pass the instance...
Slightly change the hypervisor parameter example.
Ganeti 2.0 cluster parameters design doc
Implement job 'waiting' status
Background: when we have multiple jobs in the queue (more than just afew), many of the jobs (up to the number of threads) will be in state'running', although many of them could be actually blocked, waiting forsome locks. This is not good, as one cannot easily see what is...
OS Interface design doc
Add .. contents:: marker to design docs
Implement job auto-archiving
This patch adds a new luxi call that implements auto-archiving of jobsolder than a certain age (or -1 for all completed jobs), and the gnt-jobcommand that makes use of this (with 'all' for -1).
Add a simple timespec parsing function
This function will be used for auto-archiving jobs via the command line.The function is pretty simple, we only support up to weeks since monthsand higher are not 'precise' entities, and dealing with them wouldrequire us to start using calendar functions....
backend.py change to get cluster name from master
Currently there are three function in backend that need the cluster namein order to instantiate an SshRunner. The patch changes these to get thecluster name from the master in the rpc call; once the multi-hypervisor...
Disable re-reading of config file
Since the objects read from the config file are passed to the variousthreads, it's unsafe to re-read the config file (and throw awayConfigWriter._config_data). As such, we disable the re-reading of thefile (since now the master is the owner the file, it makes not sense to...
RAPI Desing Doc
Start implementation of parallel burnin
This patch introduces a simple framework for executing jobs in parallelin burnin (the ExecJobSet function) and the "--parallel" command lineflag.
The patch also changes the instance creation to run in parallel when the...
Fix gnt-job list with empty timestamps
In case the job object doesn't have a timestamp (which is a separateissue), the listing should not break. We fix this by changing theFormatTimstamp function itself to return '?' in case the timestampdoesn't look good (note that it still can break if non-integers are...
Increase the number of threads to 25
Since our locks are not gathered nicely, we can have jobs that areactually blocking on locks (parallel burnin shows this), so at least weneed to increase the number of threads above the usual number of jobs wecould have in a such a case....
Minor cleanups & typo fixes.
Fix SshRunner breakage from the changed API
More places actually use the SshRunner than just the gnt-clustercommands.
Change SshRunner usage
Currently the SshRunner uses a SimpleConfigReader instance, however thisis not best. We change it to use the cluster name directly (and itsconstructor now takes this as parameter, instead of SCR), and itscallers are change to pass the name directly....
Update document describing cluster security
It may need further updates, but here's a start.
Fix ssconf.GetMasterAndMyself
The ssconf migration left this out.
Reviwed-by: imsnah,ultrotter
Fix a mistake in the gnt-backup man page
The actual location is /export, not /exports.
Use docbook2* paths found during configure for actual build
docbook-wrapper had the names for the docbook2* programs hardcoded. Thispatch changes Makefile.am and the wrapper script to pass them viaanother argument.
Another issue where rapi.in was built before rapi-resources.sgml is...
Remove references to Twisted framework
Get rid of ssconf
Remove leftovers from ssconf.
Don't pass sstore to LUs anymore
sstore is no longer used in LUs.
Convert ganeti-master
Use simpleconfig instead of ssconf.
Convert ganeti-watcher
Use RPC calls instead of ssconf.
Convert ganeti-noded
Replace ssconf with utility functions.
Convert gnt-cluster
Replace ssconf with configuration.
Convert bootstrap.py
Convert cmdlib.py
Replacing ssconf with configuration. Cluster rename is broken and staysthat way.
Convert ssh.py
Get rid of ssconf and convert to configuration instead.
Convert rpc.py
Replacing ssconf with utility functions.