Generate query fields in man pages
This patch replaces the hand-written lists of fields, sometimesincomplete, wrong or inconsistent, with generated ones. What's not verynice yet is the representation for numbered lists (“nic.ip/0: IP addressof 1st network interface”, “nic.ip/1: IP address of 2nd network...
Add script to generate query fields documentation
- All lines matching "QUERY_FIELDS_${resource}" in the input will be replaced with a definition list describing the fields for $resource- The core code is kept in the Sphinx extension, so that it could be...
QUERY_FIELDS_${resource}
Add query field descriptions
- Make “doc” parameter to MakeField non-optional- Add descriptions for all fields
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
query: Add support for field description
Add function to format ordinals
See [1] for the rules.
[1] http://en.wikipedia.org/wiki/Names_of_numbers_in_English#Ordinal_numbers
Introducing gnt-cluster epo
This is a convenience command to do an automated EPO in the possible limits ofGaneti.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Move OOB_TIMEOUT_OPT to cli.py
OOB commands where we need to be able to specify the timeout areenhancing over the border of just gnt-node so we move it into cli.pyfor easy reuse in other cli parts.
epydoc.conf: Avoid newlines, rebuild if necessary
Using $(strip …) ensures there won't be any newlines in the value.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add constants for node roles
Add constants for instance status
They've been hardcoded for too long.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Merge remote branch 'devel-2.4'
query_unittest: Fix argument to set()
Commit e431074f introduced an uncatched bug. This patch fixes this. Theset is expecting a list or iteratable to work on, so it splitted theprovided instance name into a set of characters. This caused theexp_status never been set and therefore not catched in one assert rule...
query: Use aliases for legacy instance NIC fields
Merge branch 'devel-2.4'
Add unittest for cli.FormatResultError
Also make the parameter for verbosity mandatory.
Fix title of query field containing instance name
cmdlib: Fix pylint error
cmdlib: Use filters internally for queries
This is in preparation for implementing real query filters.
Add/rename label for LUXI in documentation
The HTTP server design doc will refer to the LUXI protocol,so conflicts need to be avoided.
Update news and bump version for 2.4.0 rc2
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Merge branch 'devel-2.4' into stable-2.4
Fix pylint warnings
- 1 80-char line infraction- 4 changes in how arguments are passed to logging functions- 3 pylint disable-msg's because cluster-merge needs to access ganeti config internals
Signed-off-by: Stephen Shirley <diamond@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>...
TestRapiInstanceRename use instance name
Currently the QA rename job wrongly passed the whole info dict to theclient.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Change the list formatting to a 'special' chars
And also enable verbose display via the, well, verbose option. Manpage and tests are updated, and the formatting is moved from 4 ifstatements to a data structure.
Signed-off-by: Iustin Pop <iustin@google.com>...
Add support for merging node groups
Signed-off-by: Stephen Shirley <diamond@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add option to rename groups on conflict
Fix minor docstring typo
Signed-off-by: Stephen Shirley <diamond@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add QA rapi test for instance reinstall
This tests at least the basic case, unfortunately there is no way tocheck all possibilities using the provided rapi client, as that will usethe new method unless the cluster doesn't support it.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
RAPI: remove required parameters for reinstall
Before c744425f354f1bef2d0d7d306e2d00c494d67d2b instance reinstallaccepted the "os" and "nostartup" optional query parameters. With thatcommit it was changed to allow "os" "start" and "osparams" via bodyrather than encoded in the URL. Unfortunately that commit introduced a...
NodeQuery: don't query non-vm_capable nodes
Because non-vm_capable nodes most likely don't have a hypervisorconfigured and/or storage, so the call will fail anyway.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
NodeQuery: mark live fields as UNAVAIL for non-vm_capable nodes
Since we don't have the data per design, UNAVAIL is appropriate here,while NODATA is not.
The patch also adds a comment: if we extend the live fields list tocontain other data in the future, we need to reevaluate this solution....
Fix HV/OS parameter validation on non-vm nodes
Currently, there is at least one LU that does wrong validation of HVparameters (against all nodes, LUClusterSetParams). It's possible tofix this case, but I went and modified the base functions to filterout non-vm_capable nodes so all callers are protected....
Fix LUClusterRepairDiskSizes and rpc result usage
This LU was introduced before the RPC result conversion from .data to.payload, and it has managed to keep the old-style usage (how? it'sthe only LU that does so). Fix by changing to payload, and add some...
Fix RPC mismatch in blockdev_getsize[s]
Commit 92fd2250 added consistency checks in the RPC layer, which brokethe call_blockdev_getsizes RPC call (declared with 's' at the end inrpc.py, without 's' in the node daemon).
The immediate fix is to correct the rpc function name, the long term...
Rephrase a misleading pydoc for netutils.TcpPing
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Stephen Shirley <diamond@google.com>
Remove superfluous redundant requirement
The condition is already covered by the previous requirement.
Don't remove master_candidate flag from merged nodes
Prevents lots of spurious warnings like:2011-02-10 17:00:22,776: CRITICAL Configuration data is not consistent:Not enough master candidates: actual 3, target 4
Signed-off-by: Stephen Shirley <diamond@google.com>...
Use a consistent ECID base
ECID was being calculated completely differently in__MergeNodeGroups() and _MergeConfig()
listrunner: convert from getopt to optparse
The “-A” (use agent) was not documented, and instead of adding manuallisting, I converted it to optparse like the other CLI tools.
Note that I cleaned up a bit the usage and help texts.
listrunner: fix agent usage
By delaying the agent key query until after the fork, we prevent theproblem of simultaneous access to the agent.
Tested that it works against 80 hosts in parallel without error; thecurrent version breaks already at 20 hosts....
Fix build error when using docutils 0.4
Docutils 0.4 doesn't provide the “Directive” class and Sphinx suppliesits own version which doesn't have the “assert_has_content” method.
Adding more details to gnt-node man-page
This adds some recently added flags to the man-page of gnt-node command.It also describes their purpose and handling.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove force_master support from LUOobCommand
As per discussion on the man-page1 update, this functionality should beremoved and replaced by just give the command to run if the user insistsof power cycle/power off the master and refuse to operate.
[1] http://groups.google.com/group/ganeti-devel/browse_thread/thread/95d4879a747cc295...
Add pylint disables to Sphinx extension
RAPI documentation: Use constants and assertions
This doesn't cover everything yet, but it's a good demonstration on howto use “pyeval” and “pyassert”.
errors: Add list of possible failure types
Sphinx extension: Allow evaluation of Python expressions
There are quite many hardcoded constants (e.g. “[…] one of ``file``,``lvm-pv`` or ``lvm-vg`` […]). By using constants it'll be easier toidentify these.
With such lists of values it's also easy to miss some when...
Sphinx extension: Don't use “from … import …”
Import with full name instead to reduce potential conflicts.
RAPI: Remove “__version__” for instance creation
If left in data, FillOpCode will complain. This fixes the QA breakageafter commit 526a662af.
RAPI: Use FillOpCode for adding node group
RAPI: Use FillOpCode for renaming groups
Make OpGroupRename consistent with OpInstanceRename
OpInstanceRename uses “instance_name” (like almost all other OpInstance*opcodes), not “old_name”, to specify the original name. OpGroupRename ismade consistent by renaming “old_name” to “group_name”....
RAPI: Clean up instance creation, use generated docs
- Use FillOpCode and unify parameter names between RAPI and opcode- Generate parameter documentation- Improve opcode parameter documentation
RAPI: Generate documentation for assigning nodes to groups
RAPI: Use FillOpCode for modifying node group
RAPI: Use FillOpCode for renaming instances
baserlib.FillOpCode: Allow parameter rename
Some RAPI parameters don't match the name of the underlyingopcode. With this patch they can be renamed while fillingthe opcode.
Revert "Disable the cluster-merge tool for the moment"
This reverts commit c0711f2cb989facd60430ab18c5b0e59a1f279ac.
Signed-off-by: Stephen Shirley <diamond@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix cluster-merging by not stopping noded
cli.RunWhileClusterStopped() stops noded on all of the nodes in theoriginal cluster. This prevents /etc/hosts updates on the master, andconfig redistribution doesn't reach the other nodes in the originalcluster. As all we want to do is merge while the master is stopped,...
Fix bug in iallocator data structures build
Commit a1cef11c fixed non-vm_capable nodes export, but brokeinadvertently offline nodes. The update of the dict only needs tohappen for online nodes, in the 'if' block.
Without this patch, offline nodes keep the data from the last node...
Fix html docs builds
This fixes the doc builds broken by commit b292b986, but is only anugly workaround due to our dependency chain.
opcodes.py imports (both directly and indirectly) constants.py, whichimports _autoconf.py. Hence one could say that all .py files have a...
Fix error msg for instances on offline nodes
Currently, for both primary and secondary offline nodes, we give thesame message:- ERROR: instance instance14: instance lives on offline node(s) node3- ERROR: instance instance15: instance lives on offline node(s) node3...
Minor reordering to match param order
cluster verify and instance disks on offline nodes
Currently, cluster-verify says:
- ERROR: instance instance14: couldn't retrieve status for disk/0 on node3: node offline- ERROR: instance instance14: instance lives on offline node(s) node3- ERROR: instance instance15: couldn't retrieve status for disk/0 on node3: node offline...
Cluster verify and N+1 warnings for offline nodes
Currently, cluster verify shows warnings N+1 warnings for offlinenodes having any redundant instances since the memory data that wehave for those nodes is zero, so any instance will trigger thewarning....
Handle gnt-instance shutdown --all for empty clusters
The current code gives:Failure: prerequisites not met for this operation:error type: wrong_input, error details:Selection filter does not match any instances
RAPI: Use FillOpCode for replacing disks
Generate parts of RAPI documentation from opcodes
This replaces hardcoded lists of parameters with automatically generatedones, making maintenance easier.
Add sphinx extension
For now this just registers a single new directive, “opcode_params”,which can be used to generate opcode parameter documentation.
Add opcode parameter descriptions
These will be used in automatically generated parts of the RAPIdocumentation. Not all opcodes are documented so far, but each addedopcode's parameters need to be fully documented (verified by anassertion).
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
opcodes: Prepare addition of parameter descriptions
For now they're all set to None, but a following patch will set descriptions.
ht: Add descriptive text representations
This patch adds text descriptions to all T* checks. Converting to astring (e.g. str(ht.TNone)) will produce a user-readable text. Thiswill be used for Opcode parameter documentation.
gnt-instance reinstall: display OS changes
Currently, when reinstalling an instance, especially when selecting anOS interactively, is hard to be sure one has chosen the correct OS.
This patch adds displaying the selected OS. I'm not entirely happy, asit's not very good English (e.g. keeping the same OS should be "but...
Allow passing timeout for oob helper in gnt_node.py
This allow to pass in --oob-timeout to define the timeout before theout-of-band helper is beeing aborted.
Use gnt-node add --force-join to add foreign nodes
Add --force-join option to gnt-node add
This is needed so cluster-merge can add nodes from other clusters.
Signed-off-by: Stephen Shirley <diamond@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...
Fix iterating over node groups
Current line tries to unpack dict incorrectly
RAPI: Use FillOpCode for modifying instance
RAPI: Use FillOpCode for migrating instance
RAPI: Use FillOpCode for exporting instance
Update NEWS file for the 2.4.0 rc1 release
Also bump up the version.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Disable the cluster-merge tool for the moment
Hopefully this can be fixed before the final 2.4 release…
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>Reviewed-by: Stephen Shirley <diamond@google.com>
Bump up intra-cluster import connect timeout
Currently, the export timeout is 10 times 20 seconds, but the importis only 30 seconds. I'm raising this to 60 seconds with two goals inmind:
- when debugging manually, this allows for easier synchronisation of...
Import-export: fix logging of daemon output
In case of failures, the recent daemon output is logged as %r on alist of unicode strings, which results in the (ugly):
Thu Feb 3 05:13:34 2011 snapshot/0 failed to send data: Exited with status 1 (recent output: [u' DUMP: Date of this level 0 dump: Thu Feb 3 05:13:18 2011', u' DUMP: Dumping /dev/mapper/6369a5f7-1e67-4d0d-a4f0-956b3649c6d7.disk0_data.snap-1 (an unlisted file system) to standard output', u' DUMP: Label: none', u' DUMP: Writing 10 Kilobyte records', u' DUMP: mapping (Pass I) [regular files]', u' DUMP: mapping (Pass II) [directories]', u' DUMP: estimated 54301 blocks.', u' DUMP: Volume 1 started with block 1 at: Thu Feb 3 05:13:19 2011', u' DUMP: dumping (Pass III) [directories]', u' DUMP: dumping (Pass IV) [regular files]', u'socat: E SSL_write(): Connection reset by peer', u"dd: dd: writing `standard output': Broken pipe", u' DUMP: Broken pipe', u' DUMP: The ENTIRE dump is aborted.'])...
Fix handling of ^C in the CLI scripts
This adds a message and nice handling of ^C, especially useful for``gnt-job watch``.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge branch 'devel-2.3' into devel-2.4
backend: Disable compression in export info file
The new import/export infrastructure in Ganeti 2.2 and up handlescompression differently. It no longer writes compressed files to thedestination. Unfortunately changing this behaviour would be non-trivial,...
Modify gnt-node power to support multiple nodes
This also has the sideeffect of presenting a table with the result
Make it possible to disable tty in qa_utils.GetCommandOutput
The tty can have strange side effect as it seems to replace \n with\r\n. In case of simply catting a file and compare it's output thiswill break. This patch adds the possibility to disable tty if needed....
Fix execution order in LUOobCommand causing wrongly setup node list
In commit bfceedbe a check was added to put the master at the end orskip it completely. While this functionality works, it was done at thewrong point because node_names was already processed to a node list...
utils.SetupLogging: Return function to reopen log file
This function can be used from a SIGHUP handler to reopen log files.Initial, simple unittests are included.
Reopen log files upon SIGHUP in daemons
utils.SetupLogging: Make program a mandatory argument
It's passed in by most users (daemons, CLI scripts) and for the others(burnin, watcher) it certainly doesn't hurt, especially when usingsyslog.
utils.log: Restrict I/O error handling coverage
The I/O error will occur while opening the file, not while addingand configuring the handler.
utils.log: Split formatter building into separate function
burner: Trivial code cleanup
- Use constant for exit value- Configure logging from main function, not from class' “__init__”
burnin: Reuse existing function for debug value
Instead of using its own, burnin can use cli.SetGenericOpcodeOpts.
Refactor _ConfirmOperation from gnt_instance.py to cli.py
This will allow us to use this functionality also in other cli-toolslike gnt-node power where we've to operate on multiple nodes.
Merge node groups from other cluster
Signed-off-by: Stephen Shirley <diamond@google.com>Reviewed-by: René Nussbaumer <rn@google.com>