grow-disk: wait until resync is completed
The patch adds a new ‘--no-wait-for-sync’ parameter to grow-disk similarto the one in instance add, and changes the default to wait.
This is cleaner as at the moment when the command returns, we eitherhave a fully synced disk or there is an error....
Convert the KVM hypervisor to hvparams
I'm not sure which parameters it should use, I took all I saw in thecode. Update: ultrotter says the list is OK for now.
Reviewed-by: ultrotter
Some fixes related to auto_balance
Change the constant name to match the value (autobalance ->auto_balance).
Also add the auto_balance header so that gnt-instance can list it.
Fix auto_balance and memory modification
The secondary nodes should not be checked for free memory in case of nonauto-balanced instances.
This is a forward-port from rev 1567 on the 1.2 branch.
Use the auto_balance in cluster verify
This patch changes the cluster verify in two ways: - do not use instances marked with auto_balance=False in N+1 memory checks - report, if any, the number of instances with auto_balance=False
Forward port from rev 1562 on the 1.2 branch....
Fix ordering of instance/nodes
After the locking changes, and the partial locking of nodes, theinstance/node list is not ordered anymore. Reorder so that it behaves asbefore.
Remove old instance attributes
Change over to beparams
This big patch changes the master code to use the beparams. Errors mighthave crept in, but it passes a small burnin.
Change the backend to use the beparams
The backend.FinalizeExport function is changed to use the beparamsinstead of the instance attributes. Future enhancements should be donein order to export and import/reuse the whole be/hv params.
Change hypervisors to use the beparams
This patch changes the hypervisor functions to use the beparams dictinstead of the instance attributes.
Correctly fill the beparams too over RPC
As for the hvparams, we need to replace the sent instance beparams.
Add cli function for beparams validation
This is used to validate the parameters in both “gnt-instance add” and“gnt-backup import”.
Modify the beparams constants
Memory is renamed for easier use.
Fix a bug with instance creation and hvparameters
When creating an instance, we need to check not the opcode hvparams, butthe final, filled hvparams against validity. While we do this for theremote node calls (i.e. ValidateParams), we didn't do this for...
Allow instance info to only query the config file
This patch adds a new '-s' parameter to ‘gnt-instance info’ that makesit return only 'static' information. This is much faster, especially fordrbd instances.
This is a forward-port of rev 1570 on the ganeti-1.2 branch, resending...
Convert gnt-instance info to the hvparams model
Some informations are not printed nicely (e.g. “virtual CDROM: False”),but this is the first step.
Reviewed-by: imsnah
Change gnt-instance modify to the hvparams model
Change gnt-instance list to the hvparams model
This is just a change of the various hvm_ and pvm parameters to the hvmodel. Parameters are queried via hv/$name or via the whole dict asreturned by hvparams.
Reviewed-by: ultrotter,imsnah
Switch instance hypervisor parameters to hvparams
This big patch changes instance create to the new hvparams structure.Old parameters are removed, so old jobs or old instances file will breakcurrent clusters.
Temporary fix for dual hvm/pvm instances
We have a problem with the current model of combining instance listsfrom multiple hypervisors: we don't allow duplicates, but "xm list" gives the same output for both pvm and hvm. This is a lack in the actualxen hypervisor implementation/split between pvm and hvm, but for now we...
Implement FillHV for instance-related rpc calls
We fill the instance hvparams with cluster defaults, and send a modifieddict over the wire to the node methods - they don't know anything aboutcluster defaults versus instance parameters.
Export the hypervisor.ValidateParameters over RPC
The newly-added node-specific ValidateParams hypervisor method isexported over RPC, using the semi-standard (success, message) returnvalue. Multi-node call, so that we call on both primary and secondary at...
Implement the new hypervisor API in base/xen
This patch adds the new methods CheckParameterSyntax and theValidateParameters in the base and xen hypervisors. It also changes thexen hypervisor to use the new hvparams field on instance.
Get rid of httperror module
This was a leftover from the early days of the remote API in Ganeti 1.2.
Implement parameter removal in SplitKeyVal
This patch adds paramter removal in SplitKeyVal, by prefixing avalue-less key with "-"; this is needed in resetting parameters back tocluster defaults, but care must be applied now that None can come fromthe parser....
Add the hv/be params object attributes
This patch adds the instance and cluster be/hvparams attributes, and theFill* methods on the cluster object. Nothing else is changes, exceptthat these attributes exist now on the objects.
Add constants for the HV/BE parameter names
Since we don't want the string values of the parameters (e.g.“kernel_path”) spread over the code, we introduce constants for these.
Reviewed-by: ultrotter,schreiberal
Fix a few rpc-related errors
This fixes: - whitespace change, double lines between methods - duplication of call_upload_file, introduced by mistake in rev 1795 and which went undetected because of the many changes in that ref (only diff -b shows it clearly)...
Add two new options types for CLI usage
For the new 2.0-style command line options, we need to parse strings ofthe type: ident:key=val[,...]and key=val[,...]
This patch adds two new option builders for these two, which return(ident, {key=val,}) and {key=val,} for the above two formats. It also...
Abstract checking own address into a function
Currently, we check if we have a given ip address (i.e. it's alive onone of our interfaces) but manually calling TcpPing(source=localhost).This works, but having it spread all over the code makes it hard to...
Mark call_node_leave_cluster as a static method
Reviewed-by: iustinp
OS API: support for multiple versions in an OS
Allow multiple api versions in an OS. This is according to the OS APIchanges design doc, by which an OS can support multiple versions of theGaneti API and if one is supported by Ganeti it will work. Since up to...
LUVerifyCluster: fix error from rpc call
When calling node_verify leads to an error _VerifyNodes tries to iterateover a non-sequence. Catch the error before and avoid this fromhappening.
Add new HTTP server implementation
This patch adds another implementation of an HTTP server. It'sbased on code of Python's BaseHTTPServer, from both version2.4 and 3k. In the future we can write code to decide whetherwe should fork for a request or not. Keep-alive is not supported....
Add daemon library with mainloop
This mainloop can be used in daemons like ganeti-noded.
Convert rpc module to RpcRunner
This big patch changes the call model used in internode-rpc fromstandalong function calls in the rpc module to via a RpcRunner class,that holds all the methods. This can be used in the future to enablesmarter processing in the RPC layer itself (some quick examples are not...
Cleanup in cmdlib for standalone function calls
This patch is a cleanup of the standalone functions in cmdlib. Many ofthem too as argument a ConfigWriter instance, but some also took otherparameters from the lu (e.g. proc), and in the future, if we want to...
Small random fixes
Indentation in bootstrap was wrong and some names in cmdlib.py were notright.
Move instance hypervisor check to ExpandNames
This check can be done earlier, in ExpandNames, and is needed here forthe hypervisor parameter check.
Update scripts and qa config for changed hypervisor names.
Sanitize the hypervisor names
Since in 2.0 the user will possibly have more interaction with thehypervisor names, we sanitize them by removing the version numbers(the version can be a prerequisite for the ganeti installation, weshouldn't document it in variable names)....
Fix for gnt-cluster init.
Move the hypervisor attribute to the instances
This (big) patch moves the hypervisor type from the cluster to theinstance level; the cluster attribute remains as the default hypervisor,and will be renamed accordingly in a next patch. The cluster also gains...
rpc.call_instance_migrate: pass the whole instance
Currently the call_instance_migrate call only passes the instance name;we need to pass the whole object for the hypervisor_type changes (allthe other individual instance rpc calls already pass the instance...
Implement job 'waiting' status
Background: when we have multiple jobs in the queue (more than just afew), many of the jobs (up to the number of threads) will be in state'running', although many of them could be actually blocked, waiting forsome locks. This is not good, as one cannot easily see what is...
Implement job auto-archiving
This patch adds a new luxi call that implements auto-archiving of jobsolder than a certain age (or -1 for all completed jobs), and the gnt-jobcommand that makes use of this (with 'all' for -1).
Add a simple timespec parsing function
This function will be used for auto-archiving jobs via the command line.The function is pretty simple, we only support up to weeks since monthsand higher are not 'precise' entities, and dealing with them wouldrequire us to start using calendar functions....
backend.py change to get cluster name from master
Currently there are three function in backend that need the cluster namein order to instantiate an SshRunner. The patch changes these to get thecluster name from the master in the rpc call; once the multi-hypervisor...
Disable re-reading of config file
Since the objects read from the config file are passed to the variousthreads, it's unsafe to re-read the config file (and throw awayConfigWriter._config_data). As such, we disable the re-reading of thefile (since now the master is the owner the file, it makes not sense to...
Fix gnt-job list with empty timestamps
In case the job object doesn't have a timestamp (which is a separateissue), the listing should not break. We fix this by changing theFormatTimstamp function itself to return '?' in case the timestampdoesn't look good (note that it still can break if non-integers are...
Increase the number of threads to 25
Since our locks are not gathered nicely, we can have jobs that areactually blocking on locks (parallel burnin shows this), so at least weneed to increase the number of threads above the usual number of jobs wecould have in a such a case....
Fix SshRunner breakage from the changed API
More places actually use the SshRunner than just the gnt-clustercommands.
Change SshRunner usage
Currently the SshRunner uses a SimpleConfigReader instance, however thisis not best. We change it to use the cluster name directly (and itsconstructor now takes this as parameter, instead of SCR), and itscallers are change to pass the name directly....
Fix ssconf.GetMasterAndMyself
The ssconf migration left this out.
Reviwed-by: imsnah,ultrotter
Get rid of ssconf
Remove leftovers from ssconf.
Don't pass sstore to LUs anymore
sstore is no longer used in LUs.
Convert bootstrap.py
Replace ssconf with configuration.
Convert cmdlib.py
Replacing ssconf with configuration. Cluster rename is broken and staysthat way.
Convert ssh.py
Get rid of ssconf and convert to configuration instead.
Convert rpc.py
Replacing ssconf with utility functions.
Convert hypervisor
Replacing ssconf with configuration.
Convert mcpu.py
Convert config.py
The configuration version is now again in the configuration file.
Convert backend.py
Replacing ssconf with simpleconfig.
Add new query to get cluster config values
This can be used to retrieve certain cluster config values fromwithin clients.
OpDumpClusterConfig was not used anywhere, hence I'm just reusingit. The way ConfigWriter.DumpConfig returned the configurationwas not thread-safe, anyway (no deepcopy)....
Move functions from ssconf.py elsewhere
These functions will be used to access config values instead of usingssconf.
Add simple configuration reader/writer classes
This will be used to read the configuration file in the node daemon.The write functionality is needed for master failover.
Remove last use of utils.RunCmd from the watcher
The watcher has one last use of ganeti commands as opposed to sendingrequests via luxi. The patch changes this to use the cli functions.
The patch also has two other changes: - fix the docstring for OpVerifyDisks (found out while converting...
Add cluster options from ssconf to configuration
ssconf will become write-only from ganeti-masterd's point of view,therefore all settings in there need to go into the main configurationfile.
Move instantiation of config into bootstrap.py
Future patches will add even more variables to the cluster config.Adding more parameters wouldn't make the function easier to use andit doesn't make sense to pass them to another function, as it'sonly done once in bootstrap.py on cluster initialization....
Change the results from cli.PollJob
Curently PollJob accepts a generic job, but will return (historyartifact) only the first opcode result. This is wrong, as it doesn'tallow polling of a job with multiple results.
Its only caller (for now) is also changed, so no functional changes...
Enhance the job-related timestamps
This patch adds start, stop, and received timestamp for jobs (and allowsquerying of them), and allows querying of the opcode timestamps.
Abstract the timestamp formatting into cli.py
Currently we format the timestamp inside the gnt-job info function. Wewill need this more times in the future, so move it to cli.py as aseparate, exported function.
Add opcode execution log in job info
This patch adds the job execution log in “gnt-job info” and also allowsits selection in “gnt-job list” (however here it's not very useful asit's not easy to parse). It does this by adding a new field in the queryjob call, named ‘oplog’....
Move a hardcoded constant to constants.py
For now we only use the ‘C’ protocol so we can put it in constants.pyinstead of hardcoding it.
Enable the use of shared secrets
This patch enables the use of the shared secrets for DRBD8 disks, using(hardcoded in constants.py) the md5 digest algorithm.
For making this more flexible, either we implement a cluster parameter(once the new model is in place), or we can make it ./configure-time...
Extend DRBD disks with shared secret attribute
This patch, which is similar to r1679 (Extend DRBD disks with minorsattribute), extends the logical and physical id of the DRBD disks with ashared secret attribute. This is generated at disk creation time and...
Implement job summary in gnt-job list
It is not currently possibly to show a summary of the job in the outputof “gnt-job list”. The closes is listing the whole opcode(s), but thatis too verbose. Also, the default output (id, status) is not veryuseful, unless one looks for (and knows about) an exact job ID....
Nicely sort the job list
Unless we decide to change the job identifiers to integer, we should atleast sort the list returned by _GetJobIDsUnlocked.
Move the pseudo-secret generation to utils.py
The bootstrap code needs a pseudo-secret and this is currently generatedinside the InitGanetiServerSetup function. Since more users will needthis, move it to utils.py
Fix a bug related to static minors
When the node does not yet have any minors allocated, the first minor(0) will not be entered in the ConfigWriter._temporary_drbds structure.This does not happen for our current usage, since we always ask for twominors (so the next call will not match this case), but it will be...
Add checks for tcp/udp port collisions
In case the config file is manually modified, or in case of bugs, thetcp/udp ports could be reused, which will create various problems(instances not able to start, or drbd disks not able to communicate).
This patch extends the ConfigWriter.VerifyConfig() method (which is used...
Update the cluster serial_no on certain operations
This patch adds update of the cluster serial number for: - add/remove node (as the cluster's node list is changed) - add/remove/rename instance (as the cluster's instance list is changed) - change the volume group name...
Allow listing of the serial_no via gnt-* list
This patch adds listing of the serial_no attribute in gnt-instance andgnt-node list, and updates to the manpages to reflect the change.
Initialize and update the serial_no on objects
This patch add initialization of the serial_no on instance and nodes,and update of the field whenever an object is updated in the genericcase, via ConfigWriter.Update(obj) and in the specific case ofinstances' state being modified manually....
Switch the global serial_no to the top object
Currently the serial_no that is incremented every time the configurationfile is written is located on the 'cluster' object in the configurationstructure. However, this is wrong as the cluster serial_no should be...
Add serial_no attributes to objects
This patch adds the ‘serial_no’ attribute to the other top-level objects(the configuration object itself, the nodes and the instances).
Replace a cfg.AddInstance with UpdateInstance
This seems to be the last (deprecated) use of AddInstance in order toupdate an instance.
The patch also removes a whitespace-at-eol case.
Fix iallocator name
port forward of patch from revision 1690 with following message:
Patch on revision 1686 used the wrong field: ial.name, which is the instancename and not the iallocator name. self.op.iallocator is the right field.
Sorry for this inconvenience....
Fix a broken format string
This patch fixes a broken format string. It's expecting 3 parameters, but onlygets 2. This change will add the missing parameter. This is a forward-portof the fix in Ganeti 1.2
Switch config.py to logging
A couple of more modules are using the obsolete logger functions, configbeing one of them.
Switch to static minors for DRBD
With some todos remaining, this patch switches the DRBD devices to usethe passed minors, and the cmdlib code (add instance and replace disks)to request and assign minors to the DRBD disks.
Todos: - look at the disk RPC calls to see which can be optimized away, since...
Implement config support for drbd static minors
This patch adds support for allocating static minors.
Like for the LVM uuids, we add a new cache for the temporarily allocatedrequests, and the users of the new methods must manually clear thecache. If this doesn't happen, at worst we lose some minors....
Fix disk replace secondary with static minors
The code in 'updating instance configuration' section of the replacedisks with change secondary node was setting a wrong new logical_id forthe drbd devices (only set the new node, not the new minor). The patch...
Extend DRBD disks with minors attribute
This patch converts the DRBD disks to contain also a minor (per eachnode) attribute. This minor is not yet used and is always initializedwith None, so the patch does not have any real-world impact - except forautomatically upgrading config files (it adds the minors as None, None)....
Apply filter properly in LUQuery{Nodes, Instances}
Currently when not locking all nodes/instances are returned, regardlessif the user asked only for some of them. With this patch we return tothe previous behaviour: - if no names are specified return info on all current ones...
Remove auto_balance from burnin/cmdlib
There is no such feature in trunk yet.
Add utils.ReadFile function
It abstracts exception handling and is like a complement toutils.WriteFile.
GetAllInstancesInfo, change internal iterator name
GetAllInstancesInfo used "node" as an iterator name. Change it toinstance to make it less confusing.
Parallelize Tag operations
For now we lock the instance/node for adding/deleting tags from it, butwe could probably in the future do without, with more support from theconfig for atomic operations.