Convert mcpu.py to use the logging module
Reviewed-by: imsnah
Convert ssh.py to use the logging module
Convert rpc.py to use the logging module
Fix some pylint-detected issues on the scripts
Some names were wrong, and similar stuff detected by pylint.gnt-debug.GenericOpCode is still broken.
Convert the gnt scripts to ToStdout/err
Currently the gnt-* scripts are using a mix of print, logger.ToStd* andsys.stderr.write. We convert them all to using cli.ToStdout/err. Thisway, we can easily change the implementation for all at once.
Convert cli.py to logging
We also add two function for printing messages, so that scripts won'thave to import logger to get these. They are a simple extension over thelogger ones, as they accept the call style from logging: ToStdout("Message: %s", msg)...
Fix gnt-instance modify with beparams
The gnt-instance modify didn't work correctly w.r.t the be parameters.There was also a typo in the corresponding LU.
Update gnt-cluster man page for removal of --hypervisor-type.
We no longer use a single, cluster-wide hypervisor, but configure theactual to be used hypervisor on the instance level. The option has alreadybeen removed from gnt-cluster.
Remove old HTTP server code
All users of this code have been migrated to the new and shiny HttpServerclass.
It also fixes a typo in the ApacheLogfile unittests. It has not yet beendecided whether we should keep ApacheLogfile or not, hence leaving it in....
Remove --hypervisor-type from gnt-cluster.
We no longer use a single, cluster-wide hypervisor, but configure theactual to be used hypervisor on the instance level.
Fix gnt-cluster init to set cluster defaults.
Reviewed-by: iustinp
Add constants for cluster defaults
Cleanup os_add/rename rpc for OS API 10
- remove now unused osdev and swapdev arguments from backend, noded, rpc, cmdlib- convert docstrings to epydoc
Temporarily explicitely break import/export
Since they're not converted to API 10 yet, we temporarily disable theimport/export functions.
AddOSToInstance: convert to api10
RunRenameInstance: convert to api10
Add new OSEnvironment function
This function calculates the basic environment for OS scripts in APIversion 10.
Bump up the OS API version
OSFromDisk: use script names from constants
Change OSFromDisk's docstring to epydoc
Plus update it with the real variable name
Add LDS_BLOCK for block device backed dev types
Create constants to replace os scripts names
ETag passing support.
Mainloop: handle sigterm
This fixes removals of the pid files and clean exit in the init script
Mainloop: init sigchld_handler as None
There is no point in initializing sigchld_handler as an empty dict, whenit will be overwritten by an instance of utils.SignalHandlerimmediately after.
Fix remote API unittest
I broke it with rev 1864. This patch also removes whitespace atline endings.
Reviewed-by: amishchenko
Enable gnt-cluster modify to hv/beparams
This patch enables the cluster modify to change: - enabled hypervisor list - hvparams (per hypervisor) - beparams (only the default group)
Syntax: gnt-cluster modify -B vcpus=3 -H xen-pvm:no_initrd_path
Validation for parameters is somewhat missing - the individual...
rapi: Convert to new HTTP server class
Requests are no longer logged to a separate file.
design-2.0-cluster-parameters: escape final _
In rst a final _ is used for linking. Escaping it prevents an error atcompile time on design-2.0-cluster-parameters.rst
Show the cluster parametrs in gnt-cluster info
This is just a raw update without any special formatting.
RAPI: Instance add/delete implementation.
OS Design: invert new and old name
Internally in ganeti we already have renamed the instance, so it'sbetter to pass the new name in INSTANCE_NAME and the old one inOLD_INSTANCE_NAME rather than the old name in INSTANCE_NAME and the newone in NEW_INSTANCE_NAME as this reflects what the cluster thinks....
Really fix with the zombie test (hopefully)
So, instead of any timeouts for synchronization, we move to actualevents. We still have one (huge) timeout for pathological cases (just tocleanup), but we don't sleep at all in normal usage.
The patch moves the zombie/cleaned up child creation out from setUp(),...
Fix job queue behaviour when loading jobs
Currently, if loading a job fails, the job queue code raises anexception and prevents the proper processing of the jobs in the queue.We change this so that unparseable jobs are instead archived (if notalready)....
Prevent master failover if we have wrong data
If we don't actually know the current master (as determined via voting),we prevent the failover.
The patch also changes some messages (capitalization, typos).
Reviewed-by: ultrotter
Improvements to the master startup checks
In order to account for future improvements to master failover, we movethe actual data gathering capabilities from ganeti-masterd intobootstrap.py, and we leave only the verification into masterd.
The verification procedure is then changed to retry multiple times (up...
Provide example for backend parameters
Update the design about cluster parameters and providesome example of backend parameters. By now this includesall supported parameters.
Reviewed-by: iustinp, ultrotter
Add an interface for the drain flag changes/query
This adds the set/reset in the jqueue and luxi modules, and a way toquery it in OpQueryConfigValues, and also the comand line interface forit:$ gnt-cluster queue infoThe drain flag is unset$ gnt-cluster queue drain...
Adding batch-create to gnt-instance
This change is part of the integration of tools/batcher fromGaneti 1.2 into Ganeti 2.0 core code. It has a compatiblesubmission interface to the version from 1.2 with these differences:
Add a rpc call for changing the drain flag
A new multi-node call is added that sets/resets the drain flag.
Implement the job queue drain flag
We add a (per-node) queue drain flag that blocks new job submission.There is not yet an interface to add/remove the flag (will come in nextpatches).
Implement transport of ganeti errors across luxi
This patch adds a generic method to identify the ganeti error given itsclass name, and implements this across the luxi protocol.
Change the JobQueueError parent class
Currently this is not derived from GenericError, but there's no reasonfor it to be so.
rapi: Don't throw class as an exception
rapi: Whitespace fixes
gnt-backup: update for cluster parameters
- add backend and hypervisor parameters- fix beparams validation/passing- pass hypervisor and hvparams- remove deprecated flags
gnt-instance fix ValidateBeParams call
ValidateBeParams does not return, but its return value is assigned to avariable which is never used. Avoid this assignment.
gnt-instance remove deprecated flags
kernel, initrd, hvm_boot_order and vnc_bind_address are now hypervisorparameters and should not have their own flag. Moreover querying ofvnc_bind_address should of course pass through the hv/ namespace.
grow-disk: wait until resync is completed
The patch adds a new ‘--no-wait-for-sync’ parameter to grow-disk similarto the one in instance add, and changes the default to wait.
This is cleaner as at the moment when the command returns, we eitherhave a fully synced disk or there is an error....
Burnin OpGrowDisk
With this patch both the os and the swap disk are grown duringburnin. You can pass an increase size of 0 to skip this operation.
and:
burnin: don't try to grow diskless instances
When burnin is run on a diskless instance, it fails when trying...
Convert the KVM hypervisor to hvparams
I'm not sure which parameters it should use, I took all I saw in thecode. Update: ultrotter says the list is OK for now.
Some fixes related to auto_balance
Change the constant name to match the value (autobalance ->auto_balance).
Also add the auto_balance header so that gnt-instance can list it.
Fix auto_balance and memory modification
The secondary nodes should not be checked for free memory in case of nonauto-balanced instances.
This is a forward-port from rev 1567 on the 1.2 branch.
Use the auto_balance in cluster verify
This patch changes the cluster verify in two ways: - do not use instances marked with auto_balance=False in N+1 memory checks - report, if any, the number of instances with auto_balance=False
Forward port from rev 1562 on the 1.2 branch....
Fix ordering of instance/nodes
After the locking changes, and the partial locking of nodes, theinstance/node list is not ordered anymore. Reorder so that it behaves asbefore.
Remove old instance attributes
Change over to beparams
This big patch changes the master code to use the beparams. Errors mighthave crept in, but it passes a small burnin.
Change the backend to use the beparams
The backend.FinalizeExport function is changed to use the beparamsinstead of the instance attributes. Future enhancements should be donein order to export and import/reuse the whole be/hv params.
Change hypervisors to use the beparams
This patch changes the hypervisor functions to use the beparams dictinstead of the instance attributes.
Correctly fill the beparams too over RPC
As for the hvparams, we need to replace the sent instance beparams.
Add cli function for beparams validation
This is used to validate the parameters in both “gnt-instance add” and“gnt-backup import”.
Modify the beparams constants
Memory is renamed for easier use.
Fix a bug with instance creation and hvparameters
When creating an instance, we need to check not the opcode hvparams, butthe final, filled hvparams against validity. While we do this for theremote node calls (i.e. ValidateParams), we didn't do this for...
Allow instance info to only query the config file
This patch adds a new '-s' parameter to ‘gnt-instance info’ that makesit return only 'static' information. This is much faster, especially fordrbd instances.
This is a forward-port of rev 1570 on the ganeti-1.2 branch, resending...
Convert gnt-instance info to the hvparams model
Some informations are not printed nicely (e.g. “virtual CDROM: False”),but this is the first step.
Change gnt-instance modify to the hvparams model
Change gnt-instance list to the hvparams model
This is just a change of the various hvm_ and pvm parameters to the hvmodel. Parameters are queried via hv/$name or via the whole dict asreturned by hvparams.
Reviewed-by: ultrotter,imsnah
Switch instance hypervisor parameters to hvparams
This big patch changes instance create to the new hvparams structure.Old parameters are removed, so old jobs or old instances file will breakcurrent clusters.
Temporary fix for dual hvm/pvm instances
We have a problem with the current model of combining instance listsfrom multiple hypervisors: we don't allow duplicates, but "xm list" gives the same output for both pvm and hvm. This is a lack in the actualxen hypervisor implementation/split between pvm and hvm, but for now we...
Implement FillHV for instance-related rpc calls
We fill the instance hvparams with cluster defaults, and send a modifieddict over the wire to the node methods - they don't know anything aboutcluster defaults versus instance parameters.
Export the hypervisor.ValidateParameters over RPC
The newly-added node-specific ValidateParams hypervisor method isexported over RPC, using the semi-standard (success, message) returnvalue. Multi-node call, so that we call on both primary and secondary at...
Implement the new hypervisor API in base/xen
This patch adds the new methods CheckParameterSyntax and theValidateParameters in the base and xen hypervisors. It also changes thexen hypervisor to use the new hvparams field on instance.
Get rid of httperror module
This was a leftover from the early days of the remote API in Ganeti 1.2.
Implement parameter removal in SplitKeyVal
This patch adds paramter removal in SplitKeyVal, by prefixing avalue-less key with "-"; this is needed in resetting parameters back tocluster defaults, but care must be applied now that None can come fromthe parser....
Add the hv/be params object attributes
This patch adds the instance and cluster be/hvparams attributes, and theFill* methods on the cluster object. Nothing else is changes, exceptthat these attributes exist now on the objects.
Add constants for the HV/BE parameter names
Since we don't want the string values of the parameters (e.g.“kernel_path”) spread over the code, we introduce constants for these.
Reviewed-by: ultrotter,schreiberal
Fix a few rpc-related errors
This fixes: - whitespace change, double lines between methods - duplication of call_upload_file, introduced by mistake in rev 1795 and which went undetected because of the many changes in that ref (only diff -b shows it clearly)...
Add two new options types for CLI usage
For the new 2.0-style command line options, we need to parse strings ofthe type: ident:key=val[,...]and key=val[,...]
This patch adds two new option builders for these two, which return(ident, {key=val,}) and {key=val,} for the above two formats. It also...
Some small typos on design docs
Update the cluster parameters design doc
As discussed, we update the design doc with the three classes ofparameters and other changes.
Abstract checking own address into a function
Currently, we check if we have a given ip address (i.e. it's alive onone of our interfaces) but manually calling TcpPing(source=localhost).This works, but having it spread all over the code makes it hard to...
Mark call_node_leave_cluster as a static method
OS API: support for multiple versions in an OS
Allow multiple api versions in an OS. This is according to the OS APIchanges design doc, by which an OS can support multiple versions of theGaneti API and if one is supported by Ganeti it will work. Since up to...
LUVerifyCluster: fix error from rpc call
When calling node_verify leads to an error _VerifyNodes tries to iterateover a non-sequence. Catch the error before and avoid this fromhappening.
Convert ganeti-noded to new HTTP server class
Add new HTTP server implementation
This patch adds another implementation of an HTTP server. It'sbased on code of Python's BaseHTTPServer, from both version2.4 and 3k. In the future we can write code to decide whetherwe should fork for a request or not. Keep-alive is not supported....
Add daemon library with mainloop
This mainloop can be used in daemons like ganeti-noded.
Some updates on the job queue design doc
This clarifies the job storage and the reason for choosing it.
Update design-2.0-job-queue to reflect changes
With change 1773 a new status WAITLOCK was introduced if a job/opcode iswaiting for a lock. This change updates the document about the job-queueaccordingly.
Convert rpc module to RpcRunner
This big patch changes the call model used in internode-rpc fromstandalong function calls in the rpc module to via a RpcRunner class,that holds all the methods. This can be used in the future to enablesmarter processing in the RPC layer itself (some quick examples are not...
Cleanup in cmdlib for standalone function calls
This patch is a cleanup of the standalone functions in cmdlib. Many ofthem too as argument a ConfigWriter instance, but some also took otherparameters from the lu (e.g. proc), and in the future, if we want to...
Small random fixes
Indentation in bootstrap was wrong and some names in cmdlib.py were notright.
Move instance hypervisor check to ExpandNames
This check can be done earlier, in ExpandNames, and is needed here forthe hypervisor parameter check.
Update documentation & man pages for changed hypervisor names.
Ajust config unittest.
Update scripts and qa config for changed hypervisor names.
Shorten variable names.
Sanitize the hypervisor names
Since in 2.0 the user will possibly have more interaction with thehypervisor names, we sanitize them by removing the version numbers(the version can be a prerequisite for the ganeti installation, weshouldn't document it in variable names)....
Also export OS_API to the OS scripts
The idea is that if the OSes will support multiple version (e.g. both1.2 and 2.0), then Ganeti should be able to talk to it using version2.0, but then the script needs to be told nicely what version Ganeti isusing....
Fix for gnt-cluster init.