History | View | Annotate | Download (39.7 kB)
Unify the last “--hypervisor” (as list) option
This third form takes a list of name+options, and is used only ingnt-cluster. Who knew that we have three forms of ‘-H’?
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Unify the “--no-ip-check” option
Unify the “--net” option
Unify the “--disk-template” option
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Unify the “--no-nics” option
Unify the “--file-storage-dir” option
Unify the “--file-driver” option
Unify the “--iallocator” option
This patch also changes the gnt-debug's “--algorithm” intoIALLOCATOR_OPT, since it was the only non-standard use of this (due tohistorical reasons). Since it's just a debug option,backwards-compatibility is not critical here....
Unify the “--os-type” option
This was already unified inside gnt-instance, I just moved thatdefinition (and upper-cased it) into cli.py.
Unify the “--backend-parameters” option
This is not straight-forward. First, gnt-cluster had“--backend-parameters” wheres the rest of the script had just“--backend”. I settled on the full form, since one can always abbreviateoptions, but not vice-versa....
Unify the “--hypervisor” (params only) option
This unifies the two uses of --hypervisor-parameters with only options(a=b,…) form. The patch changes a little SetInstanceParms since thatexpected opts.hypervisor instead of opts.hvparams.
Note that there is another form of --hypervisor-parameters which also...
Unify the “--hypervisor” (with name) option
Unify the “--no-wait-for-sync” option
Add an error-simulation mode to cluster verify
One of the issues we have in ganeti is that it's very hard to test theerror-handling paths; QA and burnin only test the OK code-path, sinceit's hard to simulate errors.
LUVerifyCluster is special amongst the LUs in the fact that a) it has a...
Unify the multiple 'verbose' options
Currently multiple files define a 'verbose' option. This patch moves allthese definitions to a single VERBOSE_OPT option which is exported fromcli.py.
Rewrite the lib/cli.py export list
Right now, the export list is a mess, it's not sorted, and the exporteditems are in random places. This patches does grow the length of thelist, but at least it's more clear.
Signed-off-by: Iustin Pop <iustin@google.com>...
Add bash completion for --node node1[:node2]
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Luca Bigliardi <shammash@google.com>
Merge branch 'next' into branch-2.1
Bash completion: Implement dynamic option value completion
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Encode the actual exception raised by LU execution
Currently, the actual exception raised during an LU execution (one ofOpPrereqError, OpExecError, HooksError, etc.) is lost because thejqueue.py code simply sets that to a str(err), and the code in cli.py...
cli._Argument: Remove unused, optional “suggest” parameter
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add ArgHost class
It expands to the hostnames known by bash.
Re-introduce constants for most often used arguments
Make all options use cli_option
Change scripts to use new argument definitions
This can be used to generate the bash completion script automatically.In the future it may allow for better command line validation as well.
cli: Add new classes for argument definitions
cli: Merge ikv_option and keyval_option into cli_option
Merging them gets rid of two public functions in cli.py and optparse explicitlysupports more than one option type per class.
cli: Use ToStdout/ToStderr instead of print
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>(cherry picked from commit 03298ebe95f393fc77f8acce62e5aa0b95a2bdd0)
Remove references to utils.debug
Various modules set it to True when called in debugging mode, but theutils module supports no such global.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Merge commit 'origin/next' into branch-2.1
Conflicts: lib/cli.py: trivial extra empty line
Switch gnt-debug submit-job to JobExecutor
Currently gnt-debug submits jobs individually, but in 2.1 JobExecutoruses the optimized SubmitManyJobs luxi call and as such should be usedwhenever multiple jobs need to be submitted.
This patch converts gnt-debug submit-job to use it and also removes an...
Modify cli.JobExecutor to use SubmitManyJobs
This patch changes the generic "multiple job executor" to use the manyjobs submit model, which automatically makes all its users use the newmodel.
This makes, for example, startup/shutdown of a full cluster much more...
Fix some typos
Merge branch 'master' into branch-2.1
GenericMain, handle ParameterError from _ParseArgs
Before this case was not covered, and printed a stack trace.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
check_ident_key_val, handle no_ and - prefixes
If an ident member of an IdentKeyVal relationship starts with no_ or -,handle it the same way we do for a key. Some unittests are added tocheck that check_ident_key_val behaves as expected.
This patch also changes ForceDictType to, for now, fail on such an...
_SplitKeyVal with no data return an empty dict
If an empty string is passed to _SplitKeyVal, we should return {},rather than {'': True}. Also test for the correct behavior.
Implement dry-run mode at cli level (partially)
This patch adds support for the dry-run mode for all command lineoperations, and also makes use of this for commands using theSubmitOrSend function. For the ones not using it, the flag has noeffect (future patches)....
Fix handling of 'vcpus' in instance list
Currently running “gnt-instance list -o+vcpus” fails with a cryptic message: Unhandled Ganeti error: vcpus
This is due to multiple issues: - in some corner cases cmdlib.py raises an errors.ParameterError but this is not handled by cli.py...
Add a new CONFIRM_OPT option to cli.py
Today we are not very consistent as to what ‘--force’ represents:sometimes confirmation, sometimes forcing a possible dangerous option,etc.
This patch adds a new ‘--yes’ option that should be used for all simpleconfirmations of genre “yes, I really want to remove the instance”....
Fix gnt-job list argument handling
Currently QueryJob returns "None" when a wrong job ID is passed.Handle this in gnt-job list, by printing an error for each wrong job,and still giving output for all the jobs which actually do exist.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix some epydoc style issues
99% of the epydoc return tags are "@return:", but each of the modified fileshad one "@returns:" line. We fix this for consistency.
Reviewed-by: imsnah
Remove two fixed FIXME and convert one to TODO
The cli FIXME is not something broken, but rather some better handlingfeature we'd rather have, and the two backend FIXME are done (disks havetheir read only parameter set, and the error is raised and thus reaches...
Instance parameters: force typing
We want all the hv/be parameters to have a known type, rather than arandom mix of empty string, boolean values, and None, so we declare thetype of each variable and we enforce/convert it.
- Add some new constants for enforceable value types...
Fix some issues for lockless queries
This patch converts some more jobs with only queries into cheaper luxiqueries (no job created), and fixes some fallout from the locklessqueries changes.
Reviewed-by: ultrotter
Implement lockless query operations
This patch adds the framework for, and enables lockless OpQueryInstances. Thismeans that instances will be shown in ERROR_up or ERROR_down state, even thoughthis is not an error (but just an in-progress job).
The framework is implemented as follows:...
An attempt at fixing some encoding issues
This patch unifies the hardcoded re-encoding attempts into a singlefunction in utils.py. This function is used to take either an unicode orstr object and convert it to a ASCII-only str object which can be safely...
Rework the multi-instance gnt commands
This patch changes the multi-instance gnt-* commands (gnt-instancestart/stop, gnt-node evacuate/failover) such that the individualoperations are submitted in parallel, ideally improving the speed of theexecution....
Skip offline nodes in gnt-cluster commands
This patch makes gnt-cluster copyfile and command skip the offlinenodes.
Reviwed-by: ultrotter, imsnah
Add job queue size limit
A job queue with too many jobs can increase memory usage and/or makethe master daemon slow. The current limit is just an arbitrary number.A "soft" limit for automatic job archival is prepared.
Reviewed-by: iustinp
Fix epydoc format warnings
This patch should fix all outstanding epydoc parsing errors; as such, weswitch epydoc into verbose mode so that any new errors will be visible.
jqueue: Allow jobs waiting for locks to be canceled
- Add new "canceling" status- Notify clients when job is canceled- Give a return value from CancelJob- Handle it in the client library
Small change to job failure output
Currently, job failures are done by raising OpExecError(job result). Fora one-opcode job that failed, this is very non-intuitive:Failure: command execution error:[u'Disk size change not possible, use grow-disk']
This patch changes the output in two ways:...
Enable auto-unit formatting in script output
This patch enables by default the old 'human-readable' option, but in aslightly different model.
The option is now called "units" and takes either: - 'h' for automatic formatting - 'm', 'g' or 't' for mebi/gibi/tebibytes...
Make cli.py use FieldSet for matching fields
This changes cli.py to FieldSet usage so that gnt-instance list willformat nicely the disk.size/*, and the count of disks/nics.
Reuse HTTP client pool for RPC
ganeti-masterd: Add initialization and shutdown of RPC pool. It needsto be shutdown before forking.
ganeti.cli: Add decorator function to initialize and shutdown RPC pool.
ganeti.rpc: Add functions to initialize and shutdown RPC pool. Throw...
Allow querying of variable number of parameters
This patch adds support for querying in gnt-instance list of: - disk.count - nic.count - disk.size/$N - nic.(ip|mac|bridge)/$N
The patch also disables the exception raised when the header description...
Fix gnt-* command with --submit
Currently we don't catch the JobSubmittedException (although the code toformat it nicely is there). The patch adds the exception name to thecatch list and makes --submit work.
Remove the logger.py module
Since now we use only one function from the logger module(SetupLogging), we move it to utils.py (which is already imported by allusers of this function), and we remove the module.
Convert cli.py to logging
We also add two function for printing messages, so that scripts won'thave to import logger to get these. They are a simple extension over thelogger ones, as they accept the call style from logging: ToStdout("Message: %s", msg)...
Implement the job queue drain flag
We add a (per-node) queue drain flag that blocks new job submission.There is not yet an interface to add/remove the flag (will come in nextpatches).
Add cli function for beparams validation
This is used to validate the parameters in both “gnt-instance add” and“gnt-backup import”.
Implement parameter removal in SplitKeyVal
This patch adds paramter removal in SplitKeyVal, by prefixing avalue-less key with "-"; this is needed in resetting parameters back tocluster defaults, but care must be applied now that None can come fromthe parser....
Add two new options types for CLI usage
For the new 2.0-style command line options, we need to parse strings ofthe type: ident:key=val[,...]and key=val[,...]
This patch adds two new option builders for these two, which return(ident, {key=val,}) and {key=val,} for the above two formats. It also...
Implement job auto-archiving
This patch adds a new luxi call that implements auto-archiving of jobsolder than a certain age (or -1 for all completed jobs), and the gnt-jobcommand that makes use of this (with 'all' for -1).
Add a simple timespec parsing function
This function will be used for auto-archiving jobs via the command line.The function is pretty simple, we only support up to weeks since monthsand higher are not 'precise' entities, and dealing with them wouldrequire us to start using calendar functions....
Fix gnt-job list with empty timestamps
In case the job object doesn't have a timestamp (which is a separateissue), the listing should not break. We fix this by changing theFormatTimstamp function itself to return '?' in case the timestampdoesn't look good (note that it still can break if non-integers are...
Change the results from cli.PollJob
Curently PollJob accepts a generic job, but will return (historyartifact) only the first opcode result. This is wrong, as it doesn'tallow polling of a job with multiple results.
Its only caller (for now) is also changed, so no functional changes...
Abstract the timestamp formatting into cli.py
Currently we format the timestamp inside the gnt-job info function. Wewill need this more times in the future, so move it to cli.py as aseparate, exported function.
Rewrite the 'only submit job' handling in scripts
The "sys.exit(0)" was not nice as you couldn't differentiate it fromother exit codes. We change this to a specially defined exception forthis, so that multi-opcode commands can handle this nicely.
Fix error message when masterd is not listening
Reported by Iustin.
Make sure that client programs get all messages
This is a large patch, but I can't figure out how to split it withoutbreaking stuff. The old way of getting messages by always getting thelast one didn't bring all messages to the client if they were added...
cli: Use new RPC call instead of polling
This means commands will not take at least one second anymore.
jqueue: Change log message time format
See the comment in the patch.
Fix cli.PollJob
feedback_fn wasn't passed to it.
Implement job submission for scripts
This patch adds the infrastructure for executing a job in background,instead of foreground, via a new “--submit” option. The behaviour isthat the job ID is printed and the script will immediately exit.
The patch also converts gnt-node list to this model (yes, this will be a...
Split cli.SubmitOpCode in two parts
The current SubmitOpCode function is not flexible enough to be used forsubmitters that don't want to wait for the job finish.
The patch splits this in two, a SendJob function and a PollJob one, andthe old SubmitOpCode becomes a wrapper. Note that the new SendJob takes...
Make gnt-* commands fail nicely on non-masters
This patch adds a check that we are on the master after failing toconnect to the socket, and log nicely the master name.
Fix pylint-detected issues
This is mostly: - whitespace fix (space at EOL in some files, not all, broken indentation, etc) - variable names overriding others (one is a real bug in there) - too-long-lines - cleanup of most unused imports (not all)...
Unify SetupDaemon/SetupLogging
The 'old-style' info, error, debug logs do not make much sense. Thispatch unifies the SetupLogging and SetupDaemon functions. As a result,all the commands logs to a 'commands.log' file.
The patch also changes the log setup to keep going if there's an error...
Improve cli.SubmitOpCode
Currently, the feedback_fn argument to SubmitOpCode is no longer used.We still need it in burnin, so we re-enable it by making the code callthat function with the msg argument in case feedback_fn is callable. Thepatch also modifies burnin to accept the new argument format (msg is not...
First version of user feedback fixes
This patch contains a raw version for fixing feedback_fn.
The new mechanism works as follows: - instead of a per-Processor feedback_fn, there's one for each ExecOpCode, so that feedback for different opcodes go via possibly...
Make "gnt-job list" work again
"gnt-job list" was broken after my recent changes in the RPCbetween clients and the master. This patch makes it work again.
Change masterd/client RPC protocol
- Introduce abstraction class on client side- Use constants for method names- Adopt legacy function SubmitOpCode to use it
Reuse the luxi client in cli.SubmitOpCode
By a mistake, we don't reuse the luxi client. As such, we open and closethe connection at each poll cycle and spam the server logs.
Implement handling of luxi errors in cli.py
Currently the generic handling of ganeti errors in cli.py (GenericMainand FormatError) only handles the core ganeti errors, and not the clientprotocol errors (which live in a separate hierarchy).
This patch adds handling of luxi errors too, and also adds another luxi...
Replace logging functions with calls to logging module
- Shorter code- Reorder arguments to logger.SetupLogging calls to make more sense
Remove unreachable code from cli.SubmitOpCode
CLI: retry: remove command opts/args in "gnt-X"
This new version of the patch removes only the listing of the usage inthe "gnt-X" list, but keeps the strings in since we'll want to enhanceand use them in "gnt-X $cmd --help".
Revert "CLI: remove command opts/args in "gnt-X""
This reverts commit 976.
CLI: remove command opts/args in "gnt-X"
[Forward-port of the 1.2 branch patch]
This patch removes all the parameters and options from the output"gnt-X" (i.e. the subcommand list for command). This is done in order touniformize the output, currently only some parameters are shown and they...
Move the 'cmd' lock from cli.py to ganeti-masterd
This patch removes the lock and the lock options from cli.py and movesthem to the master.
Later during development we can remove it completely, but for now it'sgood to protect any other tool that uses the lock directly....
Convert cli.SubmitOpCode to use the master
This patch converts the cli.py SubmitOpCode method to use the unixprotocol and thus execute the opcodes via the master.
The patch allows a partial burnin to work with the master. Currently thequery opcodes, since they are executed via the SubmitOpCode, are...
Move some checks from cli.py to luxi.py
The idea of cli.py and luxi.py is that all protocol checks should be inluxi, and cli.py should just offer some helpful shortcuts for thecommand line scripts.
This patch removes the result checks from cli and adds some other checks...
Add submit function to lib/cli.py
This patch adds function that submit jobs or queries over the unix socketinterface to lib/cli.py. The will be used by the scripts instead of theSubmitOpCode function.
Improve the documentation of query output fields
The gnt-node and gnt-instance list commands have a customizable list ofoutput fields, but the list is not up to date (in the man page) and noteasily understandable from the ‘--help’ output.
This patch updates the man pages and adds the available fields and...