History | View | Annotate | Download (38.4 kB)
utils.SetupLogging: Make program a mandatory argument
It's passed in by most users (daemons, CLI scripts) and for the others(burnin, watcher) it certainly doesn't hurt, especially when usingsyslog.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
burner: Trivial code cleanup
- Use constant for exit value- Configure logging from main function, not from class' “__init__”
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
burnin: Reuse existing function for debug value
Instead of using its own, burnin can use cli.SetGenericOpcodeOpts.
Rename OpStartupInstance and LUStartupInstance
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Rename OpQueryNodes and LUQueryNodes
Rename OpDiagnoseOS and LUDiagnoseOS
Rename OpMigrateInstance and LUMigrateInstance
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Rename OpMoveInstance and LUMoveInstance
Rename OpQueryInstances and LUQueryInstances
Rename OpRebootInstance and LURebootInstance
Rename OpReinstallInstance and LUReinstallInstance
Rename OpRemoveInstance and LURemoveInstance
Rename OpRenameInstance and LURenameInstance
Rename OpReplaceDisks and LUReplaceDisks
Rename OpSetInstanceParams and LUSetInstanceParams
Rename OpShutdownInstance and LUShutdownInstance
Rename OpActivateInstanceDisks and LUActivateInstanceDisks
Rename OpCreateInstance and LUCreateInstance
Rename OpDeactivateInstanceDisks and LUDeactivateInstanceDisks
Rename OpFailoverInstance and LUFailoverInstance
Rename OpGrowDisk and LUGrowDisk
Rename OpRemoveExport and LURemoveExport
Rename OpExportInstance and LUExportInstance
Allow burnin with hidden OSes
burnin is a cluster/testing feature, so it makes sense that a hidden OScan be used for it.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Change behaviour of OpDiagnoseOS w.r.t. 'valid'
This patch changes the behaviour of OpDiagnoseOS with regards to the'valid' field to be similar to the one for the hidden/blacklistedfields: unless this field is requested, invalid OSes are filtered out....
burning: fix handling of empty job sets
If we call burning with only existing instance, then it will fail tocreate any of them, and thus in the removal phase it won't have anythingto remove. Since calling luxi.SUBMIT_MULTIPLE_JOBS with an empty job set...
Fix burnin and live migration
This is breakage from the original 'live' parameter changes.
Merge branch 'devel-2.1'
Conflicts: qa/qa-sample.json (trivial) qa/qa_cluster.py (trivial)
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Luca Bigliardi <shammash@google.com>
QA, burnin: allow selection of reboot types
After some more investigation, only the soft reboot type fails for Xen3.4 (due to the reboot/uptime time counter). As such, it's better toallow selective testing, since we do want to test in general theseopcodes/the command line script....
Remove pred from compat.any/all
This makes it compatible with the python builtin, and we can even usethe builtin when running under the right version of python. The all andany functions are renamed to _all and _any, so that they can be tested,and (non)existing unittests are updated (translation: there are no unit...
Fix burnin's export test
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add support for OS parameters during instance add
This is not yet complete, as it lacks proper support for instanceimport.
Add separate module for backported language functionality
utils.py, where they were before, is already huge.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
burnin: skip migration based on hypervisor support
The patch adds selection of hypervisor in burnin (on multi-hypervisorclusters, we always used the default) and optional hypervisorparameters, and skips migration if the hypervisor doesn't support it.
Signed-off-by: Iustin Pop <iustin@google.com>...
Update docstrings in tools/ and enable epydoc
This patch updates the docstrings in tools/ (mostly in lvmstrap, whichis very very old code-base) and then enabled the tools in this directoryfor 'make apidoc' too.
burnin.ExecOrQueue: remove variable argument list
In order to later add an optional parameter we transform the variableops argument list in an explicit list.
burnin.ExecOrQueue: add post-process function
If a post-process function is passed to ExecOrQueue it is executed ifand only if the job is successful. This happens immediately if we'reproceding iteratively, and at the end, when we collect all job results,...
burnin: only remove instances we actually added
Currently burnin, if proceding in parallel, will remove all instanceswhich were passed, even if they failed to add. This is bad because itwill also remove instances which existed before burnin started. By...
Abstract the confd client creation
Most creation of confd clients will do the same steps: read MC file,parse it, read HMAC key, etc. We abstract this functionality so thatwe don't duplicate the code.
Fix burnin error when trying to grow a file volume
Abstract the growable disk types in a ganeti constants, and only rundisk grow, from burnin, on them.
burnin: implement basic confd testing
Just a few queries are checked, but this should give us confidence thatat least the basic confd framework is working properly.
Burnin: don't add/remove routed nics
Switch burnin to cli.JobExecutor
Burnin has a custom job executor, because of its need to retry some jobseries.
While we cannot replace all of it, at least the execution we can switchto cli.JobExecutor, to take advantage of the recently-introducedout-of-order waiting....
Improve burnin's Log function
This makes the Log function able to take multiple args for simplifiedmessage construction, similar to the ToStdout one.
Switch from os.path.join to utils.PathJoin
This passes a full burnin with lots of instances, and should be safe aswe mostly to join a known root (various constants) to a run-timevariable.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add debug mode to burnin
There are two entry points to job execution in burnin, ExecOp andExecOrQueue, and these are modified to call the new _SetDebug method onthe opcodes.
Add an early release lock/storage for disk replace
This patch adds an early_release parameter in the OpReplaceDisks andOpEvacuateNode opcodes, allowing earlier release of storage and moreimportantly of internal Ganeti locks.
The behaviour of the early release is that any locks and storage on all...
Further pylint disables, mostly for Unused args
Many of our functions have to follow a given API, and thus we have tokeep a given signature, but pylint doesn't understand this. Therefore,we silence this warning.
The patch does a few other cleanups.
Convert to static methods (where appropriate)
Many methods are simple pure functions, and not depending on the objectstate. We convert these to staticmethods.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Add targeted pylint disables
This patch should have only:
- pylint disables- docstring changes- whitespace changes
burnin: move decorators out of classes
Similar to commit c881c5, we move the decorators out of classes, suchthat they become simple functions instead of methods. This more clean,since only the wrapped functions need to be methods/have access to‘self’....
burnin: add --no-name-check support
This patch modifies burnin to accept the --no-name-check option and alsoadds --no-ip-check (which was always set to True before).
Remove quotes from CommaJoin and convert to it
This patch removes the quotes from CommaJoin and converts most of thecallers (that I could find) to it. Since CommaJoin does str(i) for i inparam, we can remove these, thus simplifying slightly a few calls....
burnin: skip instance moves on single node
If we have only one node, instance moves fail, because it tries to movethe instance to itself. Skipping the operation, because in that case itdoesn't make sense.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Update client os lists to name+variant format
List of OSes are displayed by gnt-os list, rapi, and gnt-instancereinstall --select-os, and checked by burnin. In all of these show thelist with name+variant, if the os has variants.
Merge remote branch 'origin/master' into mogu
Fix burnin's verbose mode
The timestamp need special formatting, which was done for the internalbuffer storage but not for the messages logged in verbose mode. Thispatch unifies the formatting for these two cases.
Unify the multiple 'verbose' options
Currently multiple files define a 'verbose' option. This patch moves allthese definitions to a single VERBOSE_OPT option which is exported fromcli.py.
Add more bash completion metadata for options
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Add burnin support for instance moves
This patch adds support for instance moves in burnin. This means thatnon-drbd instances finally get support for being moved.
Convert “burnin” to definition-based options
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add more auto-completion metadata
burnin: Don't pass any disk when replacing 2nd node
Replacing the secondary node always replaces it for all disks. Commit2945fd2d added an error message for this case.
burnin: move batch init/commit into a decorator
Many burnin steps initialize the batch queue at the beginning and commitit at the end of their operation. This patch moves this code to adecorator, in order to reduce redundant code.
burnin: move instance alive checks to a decorator
Many burn steps to a manual check of instance aliveness, via duplicatecode. This patch moves this code to a decorator.
burnin: Implement retryable operations
Some burnin steps are idempotent: e.g. reinstalling an instance (fromburning p.o.v.) can be done multiple times without any side-effects thatwould affect later burnin steps. As such, failing the whole burninprocess due a reinstall failure is undesirable....
burnin: fix removal errors hiding real errors
A long-standing bug in burnin makes errors during the removal phase(e.g. because an import has failed, or because the initial creation hasfailed) hide the original error.
This patch suppresses removal errors if we are already in ‘has_err’...
Burnin: fix rename
In rename, we must stop different names in the first and second phases,so we create two different opcodes for this purpose (instead of usingthe same one twice, which doesn't work).
Reviewed-by: imsnah
burnin: do not use drained nodes
This patch updates burnin not to use drained nodes (similar to thehandling of offline nodes).
Implement lockless query operations
This patch adds the framework for, and enables lockless OpQueryInstances. Thismeans that instances will be shown in ERROR_up or ERROR_down state, even thoughthis is not an error (but just an in-progress job).
The framework is implemented as follows:...
Make iallocator work with offline nodes
This patch changes the iallocator framework to work with and properlyexport to plugins offline nodes. It does this by only exporting thestatic configuration data for those nodes, and not attempting to parsethe runtime data....
Rework the execution model in burnin
This patch changes (significantly) the execution model in burnin: - for all runs, (almost) all instance mods in a single Burn* procedure are done as part of a job; so for example add disk, stop, remove disk, start are no longer done as separate jobs but as a single job...
Fix burnin problems when using http checks
The urllib2 module has very bad error handling. This patch changes to urllibwhich is simpler, and we derive a custom class from the FancyURLopener. Burningis no longer keeping sockets in CLOSE_WAIT state with this patch....
burnin: only call self.GrowDisks() if needed
In case we pass --disk-grow 0[,0..] then we should not call GrowDisks as itprints confusing log lines.
burnin: add option to not remove instances
This patch adds a burnin option to keep instances at the end, so thatdebugging after a burnin failure is easier.
Also, we reorder the command line parsing and client query so that onecan use ./tools/burnin --help even on non-ganeti machines....
burnin: update migration to latest log formatting
Reviewed-by: ultrotter
Forward port of the burnin migration
This is again a copy of the latest 1.2 burnin code related to migration.
burnin: redo the output formatting
Since we added many more tests in burnin, the output became almostunreadable. This patch changes the output to an indented one, so thatthe different phases and operations of burnin are more easilyunderstood.
Reviwed-by: ultrotter
burnin: move start_stop at the end
Traditionally the start/stop test was the last, so move it back to there(added as last option in commit 854).
Reviewed-by: amishchenko
burnin: introduce instance alive checks
This patch adds instance alive checks after most start operations. Thecheck is done in a custom way: - the instance is expected to have an http server up and running - and it should server the '/hostname.txt' resource containing the...
burnin: Add tests for add/remove disks and NICs
This patch adds testing of add/remove disks and NICs to the burnin.
burnin: Add activate/deactivate disks
burnin: use the new replace_disks constants
This patch updates burnin to the latest replace disks constant, andchanges the constants' values to be more accurate.
burnin: do not use offline nodes
This patch makes burnin skip the offline nodes in it's builtin nodeselection. It also removes an extra line.
burnin: add instance reinstall and reboot
These two operations were missing from burnin. The reboot is done withall valid modes (a new constant is added), and the reinstall is doneboth with and without specifying the OS (to account for the two codepaths in the LU)....
burnin: don't do export/import for file storage
This is currently not supported, so don't try to do export/import inthis case.
burnin: fix usage of diskless template
This allows burnin to work with diskless instances (since we cannot passright now no disks to it).
Burner: fix ImportExport
Update the call to CreateInstance used in ImportExport to the new API formultidisk/nic.
Reviewed-by: iustinp
Small burnin info message change
This looks beter than '0 disk'.
burnin: fix default growth parameter
Initial multi-disk/multi-nic support
This patch adds support for mult-disk/multi-nic in: - instance add - burnin
The start/stop/failover/cluster verify work as expected. Replace diskand grow disk are TODO.
There's also a change gnt-job to allow dictionaries to be listed in...
Remove the logger.py module
Since now we use only one function from the logger module(SetupLogging), we move it to utils.py (which is already imported by allusers of this function), and we remove the module.
grow-disk: wait until resync is completed
The patch adds a new ‘--no-wait-for-sync’ parameter to grow-disk similarto the one in instance add, and changes the default to wait.
This is cleaner as at the moment when the command returns, we eitherhave a fully synced disk or there is an error....
Burnin OpGrowDisk
With this patch both the os and the swap disk are grown duringburnin. You can pass an increase size of 0 to skip this operation.
and:
burnin: don't try to grow diskless instances
When burnin is run on a diskless instance, it fails when trying...
Change over to beparams
This big patch changes the master code to use the beparams. Errors mighthave crept in, but it passes a small burnin.
Start implementation of parallel burnin
This patch introduces a simple framework for executing jobs in parallelin burnin (the ExecJobSet function) and the "--parallel" command lineflag.
The patch also changes the instance creation to run in parallel when the...
Remove auto_balance from burnin/cmdlib
There is no such feature in trunk yet.
Make burnin aware of hvm device type flags.
Merged from branches/ganeti/ganeti-1.2 r1648Use static values for new hvm instance flags
Another burnin fix
This is a result of the log timestamp changes.
Unify SetupDaemon/SetupLogging
The 'old-style' info, error, debug logs do not make much sense. Thispatch unifies the SetupLogging and SetupDaemon functions. As a result,all the commands logs to a 'commands.log' file.
The patch also changes the log setup to keep going if there's an error...