Further pylint disables, mostly for Unused args
Many of our functions have to follow a given API, and thus we have tokeep a given signature, but pylint doesn't understand this. Therefore,we silence this warning.
The patch does a few other cleanups.
Signed-off-by: Iustin Pop <iustin@google.com>...
Convert to static methods (where appropriate)
Many methods are simple pure functions, and not depending on the objectstate. We convert these to staticmethods.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Remove more unused variables
This removes unused variables in the rest of the code (outside lib/).
Add targeted pylint disables
This patch should have only:
- pylint disables- docstring changes- whitespace changes
burnin: move decorators out of classes
Similar to commit c881c5, we move the decorators out of classes, suchthat they become simple functions instead of methods. This more clean,since only the wrapped functions need to be methods/have access to‘self’....
Fix use of the logging functions
The logging functions expand the arguments themselves, thus it's saferto let them do it rather than manual string formatting.
Also re-wraps one comment.
burnin: add --no-name-check support
This patch modifies burnin to accept the --no-name-check option and alsoadds --no-ip-check (which was always set to True before).
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Remove quotes from CommaJoin and convert to it
This patch removes the quotes from CommaJoin and converts most of thecallers (that I could find) to it. Since CommaJoin does str(i) for i inparam, we can remove these, thus simplifying slightly a few calls....
cfgupgrade: Implement upgrade to 2.1.0
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Make bootstrap._GenerateSelfSignedSslCert public
cfgupgrade: Remove Ganeti 1.2 support
This also fixes a few typos.
burnin: skip instance moves on single node
If we have only one node, instance moves fail, because it tries to movethe instance to itself. Skipping the operation, because in that case itdoesn't make sense.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Update client os lists to name+variant format
List of OSes are displayed by gnt-os list, rapi, and gnt-instancereinstall --select-os, and checked by burnin. In all of these show thelist with name+variant, if the os has variants.
Merge remote branch 'origin/master' into mogu
Fix burnin's verbose mode
The timestamp need special formatting, which was done for the internalbuffer storage but not for the messages logged in verbose mode. Thispatch unifies the formatting for these two cases.
Unify the multiple 'verbose' options
Currently multiple files define a 'verbose' option. This patch moves allthese definitions to a single VERBOSE_OPT option which is exported fromcli.py.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add more bash completion metadata for options
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Add burnin support for instance moves
This patch adds support for instance moves in burnin. This means thatnon-drbd instances finally get support for being moved.
Convert “burnin” to definition-based options
Add more auto-completion metadata
burnin: Don't pass any disk when replacing 2nd node
Replacing the secondary node always replaces it for all disks. Commit2945fd2d added an error message for this case.
Merge branch 'next' into branch-2.1
lvmstrap: Change diskinfo to use GenerateTable
This way the produced table is formatted nicely.
Signed-off-by: Stephen Shirley <diamond@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
burnin: move batch init/commit into a decorator
Many burnin steps initialize the batch queue at the beginning and commitit at the end of their operation. This patch moves this code to adecorator, in order to reduce redundant code.
burnin: move instance alive checks to a decorator
Many burn steps to a manual check of instance aliveness, via duplicatecode. This patch moves this code to a decorator.
burnin: Implement retryable operations
Some burnin steps are idempotent: e.g. reinstalling an instance (fromburning p.o.v.) can be done multiple times without any side-effects thatwould affect later burnin steps. As such, failing the whole burninprocess due a reinstall failure is undesirable....
burnin: fix removal errors hiding real errors
A long-standing bug in burnin makes errors during the removal phase(e.g. because an import has failed, or because the initial creation hasfailed) hide the original error.
This patch suppresses removal errors if we are already in ‘has_err’...
Fix some typos
Use ReadFile.splitlines() rather than readlines
A few places in the code open a file "manually" rather than using ourwrapper function, because they need an array with the lines. Combiningthe result of utils.ReadFile with splitlines() we get rid of theexceptions....
Change BEGR_DEFAULT to PP_DEFAULT
This way the same constant can represent the default profile also fornic, disk and OS parameters.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix serial_no field on instances
The instance objects did not get a serial_no field. This patch adds anew constants for the field name and uses it for all three cases(cluster, nodes, instances).
Reviewed-by: imsnah
Complete the cfgupgrade script for 2.0 migrations
This patch makes the cfgupgrade script to handle: - instance changes - disk changes - further cluster fixes - adds configuration checks at the end, in non-dry-run mode
Reviewed-by: ultrotter
First run at cfgupgrade for 2.0 upgrades
This patch makes cfgupgrade work on empty cluster (i.e. no instances),up to a point that the config file can be converted from 1.2 to 2.0.This is not yet complete, though.
Burnin: fix rename
In rename, we must stop different names in the first and second phases,so we create two different opcodes for this purpose (instead of usingthe same one twice, which doesn't work).
burnin: do not use drained nodes
This patch updates burnin not to use drained nodes (similar to thehandling of offline nodes).
Implement lockless query operations
This patch adds the framework for, and enables lockless OpQueryInstances. Thismeans that instances will be shown in ERROR_up or ERROR_down state, even thoughthis is not an error (but just an in-progress job).
The framework is implemented as follows:...
lvmstrap: allow removable devices too
For testing or just in case a device is exported by a bad driver withthe 'removable' flag set, this patch adds a flag to lvmstrap that allowsit to use these devices too.
Make iallocator work with offline nodes
This patch changes the iallocator framework to work with and properlyexport to plugins offline nodes. It does this by only exporting thestatic configuration data for those nodes, and not attempting to parsethe runtime data....
Rework the execution model in burnin
This patch changes (significantly) the execution model in burnin: - for all runs, (almost) all instance mods in a single Burn* procedure are done as part of a job; so for example add disk, stop, remove disk, start are no longer done as separate jobs but as a single job...
Fix burnin problems when using http checks
The urllib2 module has very bad error handling. This patch changes to urllibwhich is simpler, and we derive a custom class from the FancyURLopener. Burningis no longer keeping sockets in CLOSE_WAIT state with this patch....
burnin: only call self.GrowDisks() if needed
In case we pass --disk-grow 0[,0..] then we should not call GrowDisks as itprints confusing log lines.
burnin: add option to not remove instances
This patch adds a burnin option to keep instances at the end, so thatdebugging after a burnin failure is easier.
Also, we reorder the command line parsing and client query so that onecan use ./tools/burnin --help even on non-ganeti machines....
burnin: update migration to latest log formatting
Forward port of the burnin migration
This is again a copy of the latest 1.2 burnin code related to migration.
burnin: redo the output formatting
Since we added many more tests in burnin, the output became almostunreadable. This patch changes the output to an indented one, so thatthe different phases and operations of burnin are more easilyunderstood.
Reviwed-by: ultrotter
burnin: move start_stop at the end
Traditionally the start/stop test was the last, so move it back to there(added as last option in commit 854).
Reviewed-by: amishchenko
burnin: introduce instance alive checks
This patch adds instance alive checks after most start operations. Thecheck is done in a custom way: - the instance is expected to have an http server up and running - and it should server the '/hostname.txt' resource containing the...
burnin: Add tests for add/remove disks and NICs
This patch adds testing of add/remove disks and NICs to the burnin.
burnin: Add activate/deactivate disks
burnin: use the new replace_disks constants
This patch updates burnin to the latest replace disks constant, andchanges the constants' values to be more accurate.
burnin: do not use offline nodes
This patch makes burnin skip the offline nodes in it's builtin nodeselection. It also removes an extra line.
burnin: add instance reinstall and reboot
These two operations were missing from burnin. The reboot is done withall valid modes (a new constant is added), and the reinstall is doneboth with and without specifying the OS (to account for the two codepaths in the LU)....
burnin: don't do export/import for file storage
This is currently not supported, so don't try to do export/import inthis case.
burnin: fix usage of diskless template
This allows burnin to work with diskless instances (since we cannot passright now no disks to it).
Burner: fix ImportExport
Update the call to CreateInstance used in ImportExport to the new API formultidisk/nic.
Reviewed-by: iustinp
Small burnin info message change
This looks beter than '0 disk'.
burnin: fix default growth parameter
Initial multi-disk/multi-nic support
This patch adds support for mult-disk/multi-nic in: - instance add - burnin
The start/stop/failover/cluster verify work as expected. Replace diskand grow disk are TODO.
There's also a change gnt-job to allow dictionaries to be listed in...
Remove the logger.py module
Since now we use only one function from the logger module(SetupLogging), we move it to utils.py (which is already imported by allusers of this function), and we remove the module.
grow-disk: wait until resync is completed
The patch adds a new ‘--no-wait-for-sync’ parameter to grow-disk similarto the one in instance add, and changes the default to wait.
This is cleaner as at the moment when the command returns, we eitherhave a fully synced disk or there is an error....
Burnin OpGrowDisk
With this patch both the os and the swap disk are grown duringburnin. You can pass an increase size of 0 to skip this operation.
and:
burnin: don't try to grow diskless instances
When burnin is run on a diskless instance, it fails when trying...
Change over to beparams
This big patch changes the master code to use the beparams. Errors mighthave crept in, but it passes a small burnin.
Disable re-reading of config file
Since the objects read from the config file are passed to the variousthreads, it's unsafe to re-read the config file (and throw awayConfigWriter._config_data). As such, we disable the re-reading of thefile (since now the master is the owner the file, it makes not sense to...
Start implementation of parallel burnin
This patch introduces a simple framework for executing jobs in parallelin burnin (the ExecJobSet function) and the "--parallel" command lineflag.
The patch also changes the instance creation to run in parallel when the...
Remove auto_balance from burnin/cmdlib
There is no such feature in trunk yet.
Make burnin aware of hvm device type flags.
Merged from branches/ganeti/ganeti-1.2 r1648Use static values for new hvm instance flags
Another burnin fix
This is a result of the log timestamp changes.
Unify SetupDaemon/SetupLogging
The 'old-style' info, error, debug logs do not make much sense. Thispatch unifies the SetupLogging and SetupDaemon functions. As a result,all the commands logs to a 'commands.log' file.
The patch also changes the log setup to keep going if there's an error...
Improve cli.SubmitOpCode
Currently, the feedback_fn argument to SubmitOpCode is no longer used.We still need it in burnin, so we re-enable it by making the code callthat function with the msg argument in case feedback_fn is callable. Thepatch also modifies burnin to accept the new argument format (msg is not...
Burnin doesn't need a Processor
In 2.0 burnin submits job to the master daemon, so it doesn't need tocreate an internal Processor anymore. Even if the processor is not usedanywhere in the burnin code it was still initialized as a leftover ofhow burnin used to work. Fixing this....
cfgupgrade: Implement upgrading to Ganeti 2.0 configuration
cfgupgrade: Add main() function
cfgupgrade: Add logging module
Use a single Makefile.am instead of many
This change allows us to use cleaner dependencies betweendirectories. The build system is basically rewritten in large partsand may contain bugs.
Burnin: Use iallocator in import/export
Currently the iallocator option is ignored by burnin at import/exporttime even if passed in. With this patch it becomes used. The log messageused by the importer is also changed to reflect this.
This patch also improves import/export on the non-iallocator case:...
Replace logging functions with calls to logging module
- Shorter code- Reorder arguments to logger.SetupLogging calls to make more sense
LURemoveInstance: fix op.ignore_failures usage
Currently: the LURemoveInstance.Exec() method uses the ignore_failuresattribute of the OpRemoveInstance opcode, but it doesn't check for itsexistence. The patch adds this attribute to _OP_REQP and to all the...
Move the 'cmd' lock from cli.py to ganeti-masterd
This patch removes the lock and the lock options from cli.py and movesthem to the master.
Later during development we can remove it completely, but for now it'sgood to protect any other tool that uses the lock directly....
Convert cli.SubmitOpCode to use the master
This patch converts the cli.py SubmitOpCode method to use the unixprotocol and thus execute the opcodes via the master.
The patch allows a partial burnin to work with the master. Currently thequery opcodes, since they are executed via the SubmitOpCode, are...
Implement replace secondary via the iallocator
This patch implements secondary replace via the iallocator. The newopcode parameter 'iallocator' behaves like this: if passed, it willalways compute and assign a new secondary, behaving in effect as if the...
Implement memory size option for burnin
This simple patch adds memory size for the burnin instances, which helpstesting allocator algorithm response based on the available node memoryand instance memory size values.
Fix burnin ImportExport CreateInstance opcode
This patch adds the two attributes file_storage_dir and file_driverto the CreateInstance opcode in the ImportExport method, sinceotherwise it will fail with the file storage patches.
burnin: improve instance create message
When the allocator or a non-mirrored template are used the message shown byburnin at instance creation time was wrong. Fixing it.
burnin: remove created exports
Make burnin remove exports after importing the instance back.
Add instance rename to burnin
This patch adds the possibility to execute instance renaming whendoing a cluster burnin. The optional parameter --rename takesone 'spare' instance name.Given this option it renames all given instances to the spare oneand back to their original name....
Allow burnin to use an iallocator in instance creation
This patch adds iallocator support to burnin, currently only in instancecreation. This means that, depending on the external algorithm, not allnodes might get instances.
The patch also restricts node selection and iallocator options together....
Modify burnin to support file storage backend
This patch does two things:- adjusts burnin to work with the file storage backend patches- adds the possibilty to do a burnin for file based instances
Rework the results of OpDiagnoseOS opcode
Currently, the opcode DiagnoseOS is the only opcode that return astructure of objects.OS (which is a custom class, and not a simplepython object) and furthermore all the processing of OS validity acrossnodes is left to the clients of this opcode....
Remove support for Pickle configuration files
Add more explicit help for command line
Update copyright year, imports of cfgupgrade
Remove the option to create md/drbd7 instances
This patch removes the options that allow to create local_raid1 orremote_raid1 instances. It also modifies the documentation and removesthese disk templates from burnin and from qa.
Codestyle fixes: adding a few empty lines
Revert "Include the DRBD upgrade tool in the dist archive"
This reverts commit 604 as the upgrade tool should not be in the 1.3 branch.
Reviewed-by: schreiberal
Include the DRBD upgrade tool in the dist archive
This patch adds the DRBD upgrade tool in the archive and in theinstalled tools dir, and its associated README in the doc_DATA target.
Two small improvements to burnin
This tiny patch fixes the verbose option to actually work, and also whencreating instances it logs the secondary node too (even if this doesn'tapply for plain templates, it doesn't create an error).
Change the install directory for the tools
Currently, the tools are installed under $prefix/share/ganeti. Thisprevents installing other things in a nice way under share/ganeti (likearch-independent OS definitions), therefore we want the tools to liveunder share/ganeti/tools....
Allow use of 'diskless' disk template in burnin
Even if this doesn't have any practical use for actually creatinginstances, it can be used for very fast burnin and testing just theadd/start/stop/remove functionality.
This has also revealed a bug in export/import related to diskless...
Make instance start/stop skippable at burnin time
Even though burnin was born just to do that test it now contains a lot morethings one might try, so it makes sense to make instance start/stop optionaltoo.
This creates a burnin that at the bare minimum tests instance create and...