History | View | Annotate | Download (13.1 kB)
Merge branch 'master' into next
Implement gnt-cluster check-disk-sizes
This patch adds a new opcode and lu for checking disk sizes. Currentlyit does only top-level disk verification, and also doesn't checkprimary/secondary node size mismatches (these two are added as TODOs inthe Exec() function of the LU)....
Fix pylint warnings
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix some typos
An attempt at fixing some encoding issues
This patch unifies the hardcoded re-encoding attempts into a singlefunction in utils.py. This function is used to take either an unicode orstr object and convert it to a ASCII-only str object which can be safely...
Forward port the live migration from 1.2 branch
This is forward port via copy (and not individual patches cherry-pick)of the latest code on the 1.2 branch related to the migration.
The changes compared to 1.2 are the fact that we don't need theIdentifyDisks step anymore (the drbd rpc calls are independent now), and...
Introduce a very simple LU to force config updates
This LU can be used to force a push of the config in case it's needed,for example after an upgrade to update the ssconf_release_version file.
Reviewed-by: imsnah
Make cluster verify understand offline nodes
This patch changes cluster verify to not alert on offline nodes, butinstead just show a note at the end with the number of such nodes.
It also removes warnings in verify-disks and hooks about failures tomake rpc calls to such nodes....
Convert rpc results to a custom type
For a long time we had the problem that both RPC-layer errors andresults from the remote node share the same "valuespace". This isbecause we shouldn't raise an exception when only one node failed(and lose the results from the other nodes)....
Add a gnt-node modify operation
This patch adds the OpCode, LogicalUnit and gnt-node command formodifying node parameters, more specifically the master candidate flagfor a node.
Documentation updates for mcpu.py
This is the only change needed to make mcpu epydoc-compliant.
Reviewed-by: ultrotter
Improve the mcpu.Processor logging routines
As discussed previously, many of the routinges in cmdlib.py are usinglogging functions as a carry-over from 1.2 (when these also showed themessage on stderr/to the user), instead of actually warning the user....
Convert mcpu.py to use the logging module
Convert rpc module to RpcRunner
This big patch changes the call model used in internode-rpc fromstandalong function calls in the rpc module to via a RpcRunner class,that holds all the methods. This can be used in the future to enablesmarter processing in the RPC layer itself (some quick examples are not...
Implement job 'waiting' status
Background: when we have multiple jobs in the queue (more than just afew), many of the jobs (up to the number of threads) will be in state'running', although many of them could be actually blocked, waiting forsome locks. This is not good, as one cannot easily see what is...
Don't pass sstore to LUs anymore
sstore is no longer used in LUs.
Reviewed-by: iustinp
Convert mcpu.py
Replacing ssconf with configuration.
Add new query to get cluster config values
This can be used to retrieve certain cluster config values fromwithin clients.
OpDumpClusterConfig was not used anywhere, hence I'm just reusingit. The way ConfigWriter.DumpConfig returned the configurationwas not thread-safe, anyway (no deepcopy)....
Implement adding/removal of locks by declaration
With this patch LUs can declare locks to be added when they start and/orremoved after they finish. For now locks can only be added in theacquired state, and removed if owned, and added locks default to be...
Use is_owned to determine whether to unlock
Now that is_owned is public we don't need to play games at the end of anLU. If we're still owning anything we just release it.
Processor: remove ChainOpCode
This function was incompatible with the new locking system, and itsusage has been removed from the code. For now LUs share code by callingcommon module-private functions in cmdlib.py, in the future they willuse tasklets (when those will be implemented)....
Fix issue when acquiring empty lock sets
By design if an empty list of locks is acquired from a set, no locks areacquired, and thus release() cannot be called on the set. On the otherhand if None is passed instead of the list, the whole set is acquired,...
Processor: lock all levels even if one is missing
If a locking level wasn't specified locking used to stop. This meansthat if one, for example, didn't specify anything at the LEVEL_INSTANCElevel, no locks at the LEVEL_NODE level were acquired either. With this...
ChainOpCode is still BGL-only
Prevent mistakes with an assert.
Fix pylint-detected issues
This is mostly: - whitespace fix (space at EOL in some files, not all, broken indentation, etc) - variable names overriding others (one is a real bug in there) - too-long-lines - cleanup of most unused imports (not all)...
Make sharing locks possible
LUs can declare which locks they need by populating theself.needed_locks dictionary, but those locks are always acquired asexclusive. Make it possible to acquire shared locks as well, bydeclaring a particular level as shared in the self.share_locks...
Add LogicalUnit.DeclareLocks
This additional LogicalUnit function is optional to implement, but letsyou change your locking needs for one level just before locking it, butafter the previous levels have been already locked. It is useful forexample to calculate what nodes to lock after locking an instance....
Rework master startup/shutdown/failover
This (big) patch reworks the master startup/shutdown and the fixes themaster failover.
What does the patch do?
For master start/stop: - remove the old ganeti-master script and its associated man page - moves the ip start/stop directly into the backend.(Start|Stop)Master...
Invert nodes/instances locking order
An implementation mistake from the original design caused nodes to belocked before instances, rather than after. This patch inverts the levelnumbering, changing also the relevant unittests and the recursivelocking function starting point....
First version of user feedback fixes
This patch contains a raw version for fixing feedback_fn.
The new mechanism works as follows: - instead of a per-Processor feedback_fn, there's one for each ExecOpCode, so that feedback for different opcodes go via possibly...
Processor: Acquire locks before executing an LU
If we're running in a "new style" LU we may need some locks, as requiredby the ExpandNames function, to be able to run. We'll walk up the locklevels present in the needed_locks dictionary and acquire them, then run...
LogicalUnit: add ExpandNames function
New concurrent LUs will need to call ExpandNames so that any namespassed in by the user are canonicalized, and can be used by hooks,locking and other parts of the code. This was done in CheckPrereqbefore, but it's now splitted out, as it's needed for locking, which in...
Processor: Move LU execution to its own method
This makes the try...finally code simplier, and helps adding a morecomplex locking structure before the actual execution. It also fixes aconcurrency bug caused by the fact that write_count was read beforeacquiring the BGL, and thus spurious config update hooks run could have...
Pass context to LUs
Rather than passing a ConfigWriter to the LUs we'll pass the wholecontext, from which a ConfigWriter can be extracted, but we can alsoaccess the GanetiLockManager. This also fixes the places where a FakeLUis created.
Context: s/GLM/glm/
Make the GanetiLockManager instance of GanetiContext lowercase
Processor: acquire the BGL for LUs requiring it
If a LU required the BGL (all LUs do, right now, by default) we'llacquire it in the Processor before starting them. For LUs that don'twe'll still acquire it, but in a shared fashion, so that they cannot run...
Processor: pass context in and use it.
The processor used to create a new ConfigWriter when it was initialized.We now have one in the context, so we'll just recycle it. First of allwe'll pass the context in when creating a new Processor object, thenwe'll just use context.cfg, which is granted to be initialized, wherever...
Fix sstore handling in Processor
- no need to keep the sstore as an object member, remove it- don't reinitialize sstore only if self.cfg is None This is not an issue, as the Processor is recycled for every opcode, but in general we know that (a) we might need a different type of...
Fix gnt-cluster “command” and “copyfile”
Since the disabling of forking in the master daemon, the two ssh-basedsubcommands were not working anymore. However, there is no need at allfor the commands to be run from the master daemon (permissions to readthe cluster private ssh key notwithstanding), they can be run directly...
Implement disk grow at LU level
This patch adds a new opcode and LU for growing an instance's disk.
The opcode allows growing only one disk at time, and will throw an errorif the operation fails midway (e.g. on the primary node after it hasbeen increased on the secondary node). As such, it might actually leave...
Move SetKey to WritableSimpleStore and use it
Before we used to be able to update SimpleStore by just calling SetKey, thisfeature is now moved to an external class, which inherits from it. In thispatch the new WritableSimpleStore class is also put to use, in the LUs that...
Move InitCluster opcode into a single function
This allows us to initialize a new cluster. The code certainly containsbugs and hooks aren't implemented yet.
Remove REQ_CLUSTER from opcode handling code
It's not needed anymore now that all opcodes require a cluster. Clusterinitialization was the only exception.
Add a LU Hooks notification function
Previously LUs could be failed by pre-hooks, and post-hooks just had effects bythemselves. This patch allows a LU to define the HooksCallBack function if itwants to know about its hooks' results and alter its results in response....
HooksMaster: Make RunPhase return the rpc output
Right now the hooks output is propagated from the nodes all the way up toHooksMaster.RunPhase, which uses it for debugging PRE hooks, but then silentlydiscards them. We'll now propagate it up to the Processor.ExecOpCode function,...
Add gnt-backup remove functionality
This patch also fixes the LUExportInstance Prereq docstring.
Allocator framework, 1st part: allocator input generation
In preparation for the introduction of automatic instance allocator,this patch adds an allocator simulation opcode, that based on the inputparameters, will return either the input message to the allocator...
parms->params Refactoring
- Substitute all occurences of name 'parms' with 'params'- Small codestyle fix
Map OpSetClusterParams to correponding LU
Change the order of config updates in some LUs
In the start and stop instance LUs, the configuration update is doneright at the end. This means that if, for example, the instance shutdownsucceeds, but the drive deactivation fails, the next run of the watcher...
Remove the add/remove mirror operations
These two operations are related to md/drbd7 code (remote_raid1). Removethem as part of the md/drbd7 removal.
Codestyle fixes: adding a few empty lines
Fixes small spell mistakes and comments
Add a test opcode that sleeps for a given duration
This can be used for testing purposes.
Reviewed-by: ultrotter,imsnah
Add the ‘gnt-cluster verify-disks’ command
This patch adds the OpVerifyDisks handling in mcpu.py and theverify-disks command in the gnt-cluster script, which for every instancecomputed by LUVerifyDisks submits a new OpActivateInstanceDisks request.
Soften the requirements for hooks execution
Currently, an unreachable node (or one that return undetermined failure)in the hooks pre-phase will abort the curren operation. This is notgood, as a down node could prevent many operation on the cluster.
This patch changes a RPC-level failure (and not a hook execution...
Changes related to logging
This patch modifes: - mcpu.Processor.LogWarning to have its 'hint' parameter as optional and only log it if not None - cmdlib._WaitForSync to not log directly to stdout/stderr but via the proc.Log(Info|Warning) methods...
Add better error logging functions for LUs
Currently, some LUs use logger.Error, others just feedback_fn, etc. Thispatch adds three functions to mcpu.Processor than can be used to logmessages to both the log and to the user.
These function will be used to enhance the output of replace-disks for...
Implement tag searching
This patch adds a search command for locating tags on all objects of thecluster using a regex pattern.
Reviewed-by: aat
Change the signature of some methods of mcpu.Processor
This patch moves the passing of the feedback_fn argument from the(Exec|Chain)OpCode to the initialization of the Processor instance.
Patch series for reboot feature, part 2
This patch series implements the reboot command for gnt-instance. Itsupports three types of reboot: soft (hypervisor reboot), hard (instanceconfig rebuild and reboot) and full (full instance shutdown and startupagain)....
Implement post-configuration-update hook
This patch adds a special hook: the post-configuration update hook. Thishook has only a post phase that runs after a top-level LU that modifiedthe configuration.
Since the hook is a post-phase one, no error checking is done on the...
Split the hooks env building in two parts
This patch moves some of the environment processing from _BuildEnv to anew _RunWrapper command which does the stringification and adds thesstore variables.
The reasoning is that the sstore can be fresher than before the...
Move hook execution decision to HooksMaster
Currently, the HooksMaster creation and execution decision is in theProcessor class. This is not optimal, so we change to always create ahooks master and instead make the decision inside that class, bycreating empty node lists for both pre and post if the lu doesn't...
Remove cfg and sstore parameters to HooksMaster
The HooksMaster class doesn't use the cfg parameter, and it's better touse it from the LU anyway (if needed). Let's remove it.
Also, the sstore of the LU can be fresher than the sstore we got at inittime, so use that instead and remove our own....
Remove the shebang from modules
Since modules are not directly executables, remove the shebang fromthem. This helps with lintian warnings.
Also make the autogenerated _autoconf.py contain two comment lines atthe beginning, like the other modules.
Change tags add/remove to process multiple tags
This patch changes the tags opcodes to work with multiple tags at onceinstead of only one. As such, the opcodes and some parameters arerenamed.
Implement cluster rename operation
This patch adds a new OpCode (and corresponding LU) that implements thecluster rename functionality.
This is done by shutting down the master role, making the needed sstoremodifications and distributing the changed files to all nodes, and then...
Implement instance rename operation
This patch adds support for instance rename operation at all remaininglayers: RPC, OpCode/LU and CLI.
Remove OpQueryNodeData and LUQueryNodeData
Now that LUQueryNodes supports all the functionality of LUQueryNodeData,let's migrate gnt-node.ShowNodeConfig to use it and remove all traces ofOpQueryNodeData and LUQueryNodeData.
Style changes for pep-8 and python-3000 compliance.
This changes the raising of exceptions from: raise Exception, valueto raise Exception(value)
as the first form will be removed in python-3000 and the second form ispreferred now.
The changes also involve a few cases of changing from raising standard...
Implement tag support for cluster, nodes and instances.
This is only the backend part, from the command line the tags can't beread/modified yet.
Move the cluster name from ConfigWriter to SimpleStore.
Reason: if left ConfigWriter, nodes don't know to which cluster they belong.This will bite us later when we'll revisit node join operation.
Cons: we lose the cluster name from the config file, which means a...
- Move --force option to cli.py- Implement “gnt-instance reinstall”- Fix two typos
- Implement “gnt-node volumes”- Create all --output options using a constant- Put node checking code from opcodes into a single function- Do the same for output fields
Big change/cleanup in relation to the master startup: - move the master node name from the ConfigWriter to SimpleStore (all nodes need this, and it was the only thing pulled in from the ConfigWriter on nodes) - fix mcpu.py and the testing w.r.t. this change; for testing, rename...
Initial commit.