config: Add method to get members of nodes' groups
This will be used for locking during node evacuation.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.4'
node evac: don't call IAllocator if no instances
Currently we generate an empty list only for the '-n node' invocation,but for iallocator we still call the iallocator (which needs an RPCcall, etc.). By moving the computation of instances outside of the if...
gnt-debug: rename allocator to iallocator
I'm always confused by this strange difference, so let's rename thecommand to match what it tests.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
RPC/Backend: Make UploadFile uid and gid agnostic
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Resolve uid/gid upon mainloop run
GetEntResolver: Make it possible to resolve uid/gid to name
utils.algo: Add InvertDict to invert a dict
autotools: Add noded group
Fix a couple of style mistakes
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Cluster verify: accept a --node-group option
This will trigger a ClusterVerifyGroup operation only on the specifiedgroup, skipping other groups as well as cluster-wide verifications.
Signed-off-by: Adeodato Simo <dato@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>...
Cluster verify: check for nodes/instances with no group
Previously, all nodes and instances would always be visited/verified. Bydriving the verification by node group now, we will miss nodes andinstances that can't be reached from existing node groups, should that rare...
Cluster verify: fix LV checks for split instances
When sharding by group, if a mirrored instance is split (primary andsecondary) between two groups, its volumes will not be properly checked:the group of the primary will warn about a missing volume in the secondary,...
Cluster verify: make NV_NODELIST smaller
To cope with increasing cluster sizes, we now make nodes try to contact allother nodes in their group, and one node from every other group.
Cluster verify: verify hypervisor parameters only once
The list of all hypervisor parameters has to be computed inLUClusterVerifyGroup, since it needs to be passed to nodes asNV_HVPARAMS. However, it is better only to verify said parameters once,out of LUClusterVerifyConfig....
Split LUClusterVerify into LUClusterVerify{Config,Group}
With this change, LUClusterVerifyConfig becomes a "light" LU that onlyverifies the global config and other, master-only settings, and the bulk ofnode/instance verification is done by LUClusterVerifyGroup, which only acts...
Cluster verify: factor out error codes and functions
We move all error code definitions, plus the _Error and _ErrorIf helpers,to a private _VerifyErrors mix-in class that can be later shared by the newtwo cluster verify LUs.
(_Error and _ErrorIf code was moved around verbatim, except to disable...
Cluster verify: make "instance runs in wrong node" node-driven
Previously, the "instance should not be running in this node" error wascomputed by verifying, for each instance, whether any node other than itsprimary was running it. But this is not a well-suited approach if we were...
Verify an absent vm_capable node for files
If we're not verifying all nodes, adding a node outside the currentgroup for file checksums helps us making sure checksums are the same inall of the cluster.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Cluster verify: master must be present for _VerifyFiles
This commit prepares the call to _VerifyFiles for the case when the masternode is not one of the nodes that's being verified (which will be the casefor all node groups but one). We fix it by always passing master info and...
Cluster verify: don't assume we're verifying all nodes/instances
This commit fixes a few initial simple cases in which it was assumed thatwe're always working over the whole cluster. With this change, wedifferentiate between "nodes/instances to verify" and "checks that need...
Cluster verify: gather node/instance list in CheckPrereq
This commit introduces no behavior changes, and is only a minor refactoringthat aids with a cleaner division of future LUClusterVerify work. Thechange consists in:
- substitute the {node,instance}{list,info} structures previously created...
Merge remote branch 'origin/devel-2.4'
cli: Replace hardcoded disk templates with constants
mcpu: Add missing docstring to _ProcessResult
config: Add function to get instances in node group
This will be used for evacuating instances in a node group.
iallocator: Stricter check for multi-evac result
Check new secondary nodes' group like it's already done formulti-relocation requests.
cmdlib: Use ganeti.ht for checking iallocator result
ht: Add strict check for dictionaries
This allows checking specific dictionary items, unlike TDictor TDictOf.
cmdlib: Remove punctuation from error messages
gnt-debug: New iallocator mode
Add new iallocator mode to LUTestAllocator
cmdlib.IAllocator: Add multi-relocate support
Add constants for multi-relocation iallocator mode
Implement no_remember at RAPI level
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Implement no_remember at CLI level
Introduce instance start/stop no_remember attribute
This will allow stopping or starting an instance without changing theremembered state. While this seems counter-intuitive at first (it willcreate cluster verify errors), it can help in a few corner cases:...
cmdlib.IAllocator: Fewer temporary variables
Reduce the number of temporary variables and generate dictionaries inone go.
TLMigrateInstance: do not migrate to self
Check that the instance is not being migrated to its current primary nodeduring CheckPrereq. Otherwise migration is aborted because the instance isalready running and cleaned-up, which causes the running instance to be killed....
SharedLock: Implement downgrade from exclusive to shared mode
If a job needs to modify a resource and then wait for a result, it mustacquire the resource lock in exclusive mode. In some cases it would bepossible to only have a shared lock for waiting. Until now it was not...
gnt-debug: Use constants for iallocator direction
gnt-debug, opcodes: Use constants for iallocator
cmdlib.IAllocator: Use lookup table for mode-specific data
Preload the string-escape code in noded
This encoding, part of the standard Python installation, is used bythe pickle module (in turn used by subprocess when handlingfailures in program execution). Preloading it means that Python willcache it in memory so that even if the disk goes away or just the...
Abstract ignore_consistency opcode parameter
Two opcodes already use it and we need it for a third, time to add aconstant for it.
Fix a bug in LUInstanceMove
The opcode parameter ignore_consistency was used in the LU, but notactually declared in the OpCode. The patch adds it in the opcode andthe command line client.
ObQuote — Please, please, can I have static typing?
Signed-off-by: Iustin Pop <iustin@google.com>...
Try to prevent instance memory changes N+1 failures
There are multiple bugs with the code checking for N+1 failures in theinstance memory changes which needs significant changes, in themeantime we can at least:
- change the warning message into an error (--force will skip checks)...
Remove references to acquired_locks
These sneaked in from 2.4 during the merge, but this attribute isactually gone in the master branch.
Use floppy disk and a second CDROM on KVM
Hi all,this patch will add 3 new KVM parameters and a new option.
New Parameters: - floppy_image_path = "" -> Specify the floppy image to load asfloppy disk. - cdrom2_image_path = "" -> Specify a second cdrom image to load on...
jqueue: Update worker thread name to include opcode summary
With this patch, the worker thread name is updated to include a shortsummary of the opcode (basically its OP_ID). The base name of job queuethreads is shortened from “JobQueue” to “Jq”. Logs and the lock monitor...
Use the new dry-run mode in cmdlib
This will hopefully detect potential LVM (or any other storage, whenthey implement it) issues before committing changes just on somenodes.
Unfortunately due to the dry_run opcode handling, we can't integratethis into the usual handling (as we need to activate the disks before...
Implement grow dry-run at RPC level
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Implement dryrun mode for BlockDev.Grow()
This is always called with False from backend for now.
cmdlib: Sort nodes for OOB commands
Also reorder the methods to match all other LUs.
cmdlib: Use helper for expanding nodes for OOB commands
cmdlib: Expand instances using helper for repairing disks
Also change the way “share_locks” is filled.
Fix bug introduced in commit 0d5a0b96
When removing “acquired_locks” in commit 0d5a0b96, I didn't rememberthat it does not contain the Big Ganeti Lock.
Fix lock release in TLMigrateInstance
Commit 52f33103 introduced lock release factorization, replacing manuallock release using utility functions. However, it brokeTLMigrateInstance due to a typo (passing the Tasklet to ReleaseLocksinstead of the parent LU). We fix this by passing the LU to...
cmdlib: Remove acquired_locks attribute from LUs
The “acquired_locks” attribute in LUs is used to keep a list of acquiredlocks at each lock level. This information is already known in the lockmanager, which also happens to be the authoritative source. Removing the...
cmdlib: Use local alias for lock manager
Saves some typing and we'll use it more often in the future.
Make root_path an optional hypervisor parameter
This will allow us an easy migration to pv-grub, because a set root_pathconfused pv-grub.
opcodes: Add function for compact summary
Depending on the opcode and its parameters, the existing “Summary”function can give a rater long summary. For displaying the summary inlogs and in the lock monitor, it should be shorter. Hence this newfunction is added to just use the opcode ID with common prefixes...
Show locksets in lock monitor
When all locks contained in a set are acquired, the lockset's internallock is acquired with the same mode. With this patch the internal lockwill show up on the lock monitor, named e.g. “instances/[lockset]”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
locking: Make parameter to condition's wait() positional
It is always used in the locking code. Unittests are updated.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
SharedLock: Avoid acquires from sneaking in while notifying
In some rare cases new shared acquires could sneak in through thecondition cached in “__pending_shared” while the code was stillnotifying acquires. This was only working because such a condition...
Add 2 new variables to the OS scripts environment
Add INSTANCE_PRIMARY_NODE and INSTANCE_SECONDARY_NODES. These newvalues are useful for OS scripts that needs to know the nodes wherethe instance lives.. or has lived.
Add --no-wait-for-sync when converting to drbd
Currently, when converting an instance from plain to DRBD, theinstance is blocked during the entire resync period. This patch addsthe --no-wait-for-sync so that the operation finishes as soon as theDRBD sync has started, without waiting for the entire sync. This makes...
Recreate instance disks: allow changing nodes
This patch introduces the option of changing an instance's nodes whendoing the disk recreation. The rationale is that currently if aninstance lives on a node that has gone down and is marked offline,it's not possible to re-create the disks and reinstall the instance on...
Fix instance failover/migration w.r.t TLMigrateInstance
Commit 1c6e5787 removed the iallocator and target_node keywordparameters from TLMigrateInstance, but I didn't update their use inLUInstanceFailover and (not fully) in LUInstanceMigrate.
Rename instance: only show new name when different
It makes not sense to show messages like:Fri May 6 02:04:01 2011 - INFO: Resolved given name 'instance18' to'instance18'
So we'll skip the message if the resolved name is identical to therequested one....
Fix race condition in LUGroupAssignNodes
The original code would get all node information and their groupswithout before acquiring the necessary locks. With this patch the nodeinformation is only retrieved once all locks have been acquired. Groupsare locked optimistically and verified after acquiring the node locks....
Fix DTS_EXT_MIRROR migration
Commit faaabe3c fixed failover behaviour for DTS_INT_MIRROR instances, howeverit broke migration for DTS_EXT_MIRROR instances, by moving iallocator and nodechecks from LUInstanceMigrate to TLMigrateInstance. This has the side-effect...
Use node group locking for replacing disks
This is one of the first opcodes to make use of node group locking. Toget an instance's node groups, the instance's nodes need to be lookedat. Due to a previous design decision nodes are locked after the group,...
config: Add function to determine instance's groups
This will be used for locking only the necessary node group(s)for per-instance operations.
TLMigrateInstance: Fix live migration breakage
Commit 77fcff4 unintentionally incorporated code fromTLMigrateInstance.CheckPrereq into TLMigrateInstance._RunAllocator, presumablyduring a rebase from earlier versions of the patch to the 2.5 codebase. As a...
cmdlib: Update error messages, remove some punctuation
- Clarify some error messages- Remove unnecessary punctuation- Merge two if conditions in one place
cmdlib: Fix typo, s/nick/NIC/
A small optimisation in cluster verify
This removes (count of instances + count of nodes) lockacquires/releases.
A few docstring fixes
At least one generates an epydoc error :)
luxi: do not handle KeyboardInterrupt
With the current code, it's possible to mistake a ^C for a protocolerror:
node1# gnt-job info 221691[press ^C]Unhandled protocol error while talking to the master daemon:Error while deserializing response:
(and note empty error message)....
Handle EPIPE errors while writing to the terminal
This handles EPIPE errors in two places: ToStream (to catch loggingdone in GenericMain itself) and in GenericMain (to cover also plainprint statements).
Cluster verify: check for missing bridges
Currently cluster verify doesn't check for bridge information; theonly checks are done at instance create and failover/migratetime. This means a cluster that seems healthy will fail creation jobs.
This patch implements a simple verification that all nodes (in the...
cmdlib: Factorize lock releasing
There will be more lock releasing with upcoming changes, so this willcentralize the logic behind it (what locks to keep, which variables toupdate, etc.).
TLReplaceDisks: Use implicit loop for dictionary
Release unneeded locks while replacing disks
If an iallocator is used, “gnt-instance replace-disks” would acquire thelocks of all nodes (only the allocator will decide which node to use).Unfortunately the unneeded locks were not released during the operation,...
locking: Export “list_owned” from lock manager
This is analog to “is_owned” and will be used for assertions.
gnt-instance: Fix typo in error message
The iallocator parameter is “-I”, not “-i”.
mlock: fail gracefully if libc.so.6 cannot be loaded
This allows noded to continue instead of blowing up if the libc majornumber changes.
cmdlib: Drop SSH runner from LU base class
It is no longer used.
cmdlib.py: fix indentation in _VerifyNode
Signed-off-by: Adeodato Simo <dato@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
TLMigrateInstance: Fix confusing text
Commit d5cafd31 changed this error message, swapping thetext parts in the process.
LUInstanceRename: Amend comment about lock
Also add an assertion.
iallocator: Relocation nodes must be in same group
Quoting from iallocator.rst: “[…] ``relocate`` request is used when anexisting instance needs to be moved within its node group […]”.
Fix 'unused import' lint error
Sorry!
SetEtcHostsEntry: maintain existing ordering
Currently RemoveEtcHostsEntry keeps the ordering, but SetEtcHostsEntrynot, as it will always write the new entry at the end of file. Ipersonally dislike this as it "uglifies" my custom host files, so thispatch makes it update the record instead in-place so to say instead of...