Change the debug CLI option to integer/count
This changes from boolean to integer/count (for a future differentiationbased on the actual debug level). All the uses of the code only testit's boolean status, so it still works as an integer value.
Signed-off-by: Iustin Pop <iustin@google.com>...
Add a generic 'debug_level' attribute to opcodes
Also automatically fix opcodes which have this missing in the LU initroutine.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix dumpers/loaders after slots cleanup
Commit 154b958 changed (correctly) the slots usage, but this brokedumpers/loaders since we relied directly on the own class slotsfield.
To compensate, we introduce a simple function for computing the slots...
Check built manpages for errors
One fix is necessary in gnt-cluster.sgml. Also adding “DELETE_ON_ERROR”target to remove output file if an error occurred while building it (inthis case the manpage).
This was reported by Iustin Pop in issue 87 and proposed check method...
Add an early release lock/storage for disk replace
This patch adds an early_release parameter in the OpReplaceDisks andOpEvacuateNode opcodes, allowing earlier release of storage and moreimportantly of internal Ganeti locks.
The behaviour of the early release is that any locks and storage on all...
Merge branch 'stable-2.1' into devel-2.1
TLReplaceDisks: Delay iallocator when evacuating node
When evacuating nodes, the iallocator was run for allinstances without taking planned changes into consideration.This patch delays part of CheckPrereq and running theiallocator for node evacuation....
Implement debug level across OS-related RPC calls
This doesn't implement the full functionality, we need to add the debuglevel to the opcodes too, but at least won't require changing the RPCcalls during the 2.1 series.
Second try to fix LUVerifyCluster
My previous patch, commit 785d142, fixed the case where a node is markedoffline. With this patch it'll also handle other failures correctly.
LUVerifyCluster: Fix bug with offline nodes
[…] * Other Notes - NOTICE: 1 offline node(s) found. * Hooks ResultsFailure: command execution error:iteration over non-sequence
Commit a0c9776a introduced an error simulation mode to LUVerifyCluster.Due to a small mistake, offline nodes weren't skipped when checking the...
utils: Fix retry delay calculator
Before this patch, it would always sleep for at leastthe time specified as the upper limit. Now it actuallylimits the sleep time.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Bump RPC protocol version to 30
Merge remote branch 'origin/stable-2.1' into devel-2.1
Bump version to 2.1.0~rc5
Makefile.am: Targets used directly should depend on BUILT_SOURCES
Merge branch 'devel-2.0' into devel-2.1
Conflicts: man/gnt-cluster.sgml (trivial)
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Make the snapshot decision based on disk type
… instead of disk size, which is not as reliable. This actuallysimplifies the code; but it still leaves the possibility of stackoverflows if the disk data structure is corrupted.
Three small typos in man pages
The credit goes again to Lintian.
Fix missing bridge for xen instances
Xen instances nic definitions miss the target bridge.
This bug was introduced in commit 503b97a9.
Signed-off-by: Alessandro Cincaglini <alessandro.ciancaglini@gmail.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...
Fix flipping MC flag bug
Currently unofflining or undraining an already functional mastercandidate node, can cause it to demote itself. In order to avoid that weonly trigger the self-promotion check if the node is not currently acandidate.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
ganeti-watcher: ensure confd is running as well
Ganeti-confd should be running on all 2.1 nodes.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add capability to use syslog for logging
This patch adds a configure-time parameter that will set the defaultsused by all programs, and command-line parameters in the daemons thatallow overriding it.
Syslog 'yes' enables syslog in addition to file-based logging, 'only'...
node daemon: allow working with broken queue dir
In case the queue dir cannot be create/initialized, currentlyganeti-noded exits. This means that a read-only filesystem or apermission error breaks all node daemon functionality, includingpowercycle. This is not good for the usual failure case for nodes....
utils.FileLock: handle init errors properly
If the open of the lock file fails (due to whatever reason), 'self'won't have the 'fd' attribute, and thus we fail in Close/__del__, whichwill ruin proper error reporting:
IOError: [Errno 30] Read-only file system: '/var/lib/ganeti/queue/lock'...
daemon-utils: remove usage of here-docs
In some versions of bash, here-docs and here-strings use temporaryfiles, which means daemon-util needs a writable temporary filesystem.Since echo is a bash builtin anyway, it's simple to switch to it andremove this dependency....
Fix typo in ganeti-os-interface(7)
The variable is called “IMPORT_INDEX”, not “IMPORT_IDX”.
locking: add/fix @type information
This patch missing @type information for all public methods, modifiesone to conform to the rest, and removes some information from @paramwhen it's been expressed in @type.
Fix slots definitions
According to http://docs.python.org/reference/datamodel.html#slots
Conflicts: lib/backend.py - trivial merge...
Fix the mocks.py for 2.0 unittests
The recent change to use LogWarning with multiple arguments inmpcu.py/HooksMaster broke the (simple) mock we have in the tests.
Add a crude disable for DRBD barriers
Ideally we want to/will have per-device DRBD controls of disk/metadataflushes. In the meantime, we want at least a disable of the barrierfunctionality for cases where one has battery-backed caches.
Background: DRBD has four mechanism of handling ordered disk-writes....
LURemoveNode safety in face of wrong node list
LURemoveNode runs under the BGL, which means we're guaranteed that thelist of nodes as retrieved in CheckPrereq is still valid inBuildHooksEnv. However, we can make Ganeti handle failures in case thelocking is broken (or the node list has been modified otherwise) easily,...
Fix an unsafe formatting bug
This might fix issue 84; in any case, the current situation is that wehave a potentially unsafe formatting, which should be fixed.
Ensure all int/float conversions are handled right
int()/float() can raise either ValueError (in case of int("a")), orTypeError (in case of int(None)). We had many bugs over time due tothis, and a recent one was just diagnosed, so we go over the codebase...
Bump version to 2.1.0~rc4
Update man page of gnt-instance
This patch updates the man page of gnt-instance to include the newlyadded tags filtering.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
KVM: fix pylint warning
Specify string format arguments as logging function parameters
Signed-off-by: Guido Trotter <ultrotter@google.com>
KVM: be more resilient on broken migration answers
Before, when doing kvm live migrations we use to accept an "unknownstatus" but to reject anything that didn't match our regexp. Since we'veseen "info migrate" return a completely empty answer, we'll be more...
Allow filtering for (node-)tags
This change introduces startup, shutdown, reboot, reinstall by usinginstance respectively node tags as a selection criteria.
Signed-off-by: René Nussbaumer <rn@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add unittests for cli.GenerateTable
cli: Fix bug when not using headers
Commit 9fe72672 added code to not write spaces at the end of each line.Unfortunately it didn't work properly when not printing headers—there wouldstill be spaces.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
daemon-util: Fix quoting issue
This patch fixes a quoting issue in daemon-util:
The reason was that the generated variables were not quoted properly and...
Bump version to 2.1.0~rc3
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Switch the SplitKeyVal function to accept escapes
This tiny patch switches the SplitKeyVal function (and thus the commandline options like -H, -B, etc.) to UnescapeAndSplit, thus allowing oneto use escaped commas in the values of parameters.
Fix long-standing race condition bug in locking unittest
Every once in a while we saw failures in the locking unittest, butweren't able to determine the cause. This patch fixes it by usingthreading.Event instead of plain threading.Condition to notify another...
confd client: copy the peers in UpdatePeerList
Since the peer list is shuffled by the client, we don't keep a referenceto the list which was passed in, but copy it internally.
testutils: Print name of test program before running it
While the name can be looked up in Makefile.am, thisis useful when an automated test just hangs and needsto be killed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Don't use hardcoded name for pylint
Look it up at configure time instead.
Partially revert "Makefile.am: Run pylint on all Python code"
This partially reverts commit e4e7c7df3a2356103c2442a91d628ca0470fb267,which made newer pylint versions fail.
build-bash-completion: Take care of pylint warnings
Add an UnescapeAndSplit function
In many cases, where we accept (usually from the command line) a list ofparameters, we remove the use of the separator as an component of any ofthe elements.
This patch adds a new function that can split strings of the form...
Small improvements for release script
- Print usage if no tree-ish (tag, branch or commit) was specified.- Use “gntrelease.XXXXXXXXXX” as temporary directory template. This makes it easier to find all of them.- Compute MD5 checksum. This doesn't cost a lot and...
Makefile.am: Run pylint on all Python code
That is, all Python code except unittests. In particular,autotool/build-bash-completion was missing from theprevious list.
build-bash-completion: Fix a few pylint warnings
check-python-code: Use “set -e” to abort on errors
Generate hmac file with a newline at the end
This makes it slightly easier to cut&paste its content.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
jqueue: Don't return negative number for unchecked jobs when archiving
When the queue was empty, the calculation for unchecked jobs whilearchiving would return -1. ``last_touched`` is set to 0, the job ID list(``all_job_ids``) is empty. Calculating ``len(all_job_ids) -...
cli.GenerateTable: Don't write EOL spaces
With this change, there won't be unnecessary space charactersat the end of lines.
Improve logging for workerpool tasks by providing repr
Before it would log something like “starting task(<ganeti.http.client._HttpClientPendingRequest object at 0x2aaaad176790>,)”,which isn't really useful for debugging. Now it'll log “[…]<ganeti.http.client._HttpClientPendingRequest...
workerpool: Simplify log messages
workerpool: Use worker name as thread name
This way it shows up in debug logs.
workerpool: Make worker ID alphanumeric
Having a proper name instead of just a number makes debuggingeasier.
locking: Fix race condition in LockSet
This patch fixes a race condition when acquiring all locks ina LockSet instance. The list of lock names needs to be sortedto guarantee a consistent locking order, but the names were notsorted when acquiring all locks in the set....
mcpu: Log lock status with sorted names
Reading and comparing sorted lists is easier when debugging locking problems.
locking: Append to list outside error handling block
locking: Don't fail in error handling if lock isn't owned
In case an exception was thrown while acquiring the lock, not necessarily allowned locks are also really acquired. Before this change, an exception could bemasked by another exception thrown here. There is no good clean-up strategy...
Provide example default files and install one for development
Merge branch 'devel-2.1' into stable-2.1
Normalize MAC addresses to all lower.
This change will normalize the MAC to all lower after validation.
Introduce a Luxi call for GetTags
This changes from submitting jobs to get the tags (in cli scripts) toqueries, which (since the tags query is a cheap one) should be muchfaster.
The tags queries are already done without locks (in the generic querypaths for instances/nodes/cluster), so this shouldn't break tags query...
LURenameCluster: run post hook on all nodes
Since the cluster name might be used for various purposes on nodes, weshould let all nodes "know" about a cluster rename by running the posthook on all nodes. This will make cluster rename slightlyslower/costlier, but it is not/shouldn't be an operation that is run...
Allow passing options to pylint in the lint rule
This allows automated builders to override the format, for example.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Fix unused imports or add silences where needed
In some cases pylint doesn't parse the import correctly, so we addsilences; but there are also many cases of unused imports, which wesimply remove.
bdev: Add a TODO and a pylint silence
A piece of old code in bdev.py uses a for loop over a single variablebecause we can 'break' out of the loop or exit on the 'else' path. Thisis not a nice usage of the for loop, it should be converted to astandard if...elif...else structure....
confd: add a TODO and a pylint disable
Two variables are used in a strange way in ExecConfd. Until that isclarified, add a TODO and a silence for the unused variable warning.
pylint: Temporarily disable W0201
There seems to be a bug in pylint relating to W0201 (“Attribute '%r'defined outside init”) being re-enabled by simple comments and notbeing able to be disabled again.
Until that is fixed, we disable this warning....
Disable R0922 in pylint
R0922, “Abstract class is only referenced 1 times”, cannot be disabledin the source code, and thuslib/http/auth.py:HttpServerRequestAuthentication gives this warning(it's actually also referenced from the tests, but we do not include...
Further pylint disables, mostly for Unused args
Many of our functions have to follow a given API, and thus we have tokeep a given signature, but pylint doesn't understand this. Therefore,we silence this warning.
The patch does a few other cleanups.
pylint: disable the similarities checker
This is a very slow checker, estimated to be O(n²) by its author; sousing it all the time is not good. It can be re-enabled on the commandline via “--disable-checker=”.
LUDiagnoseOS._DiagnoseByOS: remove unused arg
The node_list argument to _DiagnoseByOS is not used, and is obsoleted bythe fact that the rlist argument already has the valid nodes as keys(assuming RPC behaviour didn't change). Thus, we remove it and silence...
daemons: handle arguments correctly and uniformly
Of all daemons, only rapi did abort when given argument. None of ourdaemons use any arguments, but they accepted them blindly. This is avery bad experience for the user.
This patch adds checking and exiting in all daemons, in a uniform way....
hv_xen/_GetConfigFileDiskData: remove unused arg
The disk template is not needed, all that's used is the disk data. Assuch, remove this parameter from the function.
jqueue/_CheckRpcResult: log the whole operation
Currently only the rpc call, but not its description (which also showsthe argument) is logged. We change this to log failmsg too, and thisalso silences a warning.
Optparse extenders have to obey a given API
So we just silence the warning.
backend._OSOndiskAPIVersion: remove obsolete arg
The 'name' argument is not used anymore, probably since before 2.0.Since this is an internal function, we can just remove it (from itscaller too).
pylint cleanups: dangerous initializers
Plus a silence for a wrong "uninitialized var".
Convert to static methods (where appropriate)
Many methods are simple pure functions, and not depending on the objectstate. We convert these to staticmethods.
Remove more unused variables
This removes unused variables in the rest of the code (outside lib/).
Add targeted pylint disables
This patch should have only:
- pylint disables- docstring changes- whitespace changes
burnin: move decorators out of classes
Similar to commit c881c5, we move the decorators out of classes, suchthat they become simple functions instead of methods. This more clean,since only the wrapped functions need to be methods/have access to‘self’....
Rename an ConfdInotifyEventHandler init argument
'file' is a builtin keyword/type. Like many others, it should not beused as a variable/argument name.
No code is actually passing in this argument so renaming it is simple.
Introduce a makefile lint rule
This runs pylint over all the python files. Yes, it takes a long while,but it's the only way to properly analyze the source codes as only inthis way pylint can see all uses of the various modules/classes/etc.
Implement all hv functions in hv_chroot/hv_fake
The chroot and fake hypervisors were missing:
- the powercycle node functionality- proper handling of migration requests
The powercycle was just used as in the other hypervisors (use thestandard linux powercycle). The migration for chroot was disabled...
Update the _autoconf.py make rule for pylint
This adds targeted pylint disables, but since _autoconf.py isautogenerated we need to do this in Makefile.am (hence the separatepatch).
Disable TODO warnings in pylint
We don't get any useful out of this - a git grep is as effective, andthese only pollute the pylint output.
Add some stubs to bdev.FileStorage
This patch adds explicit errors (instead of notimplemented) inFileStorage (and the associated TODOs).
Fix indentation issues
Fix two bugs in ConfigWriter._EnsureUUID
Wrong argument name and wrong number of arguments in string formatting.