History | View | Annotate | Download (185.6 kB)
Parallelize Tag operations
For now we lock the instance/node for adding/deleting tags from it, butwe could probably in the future do without, with more support from theconfig for atomic operations.
Reviewed-by: iustinp
Parallelize LUSetClusterParams (and add a FIXME)
Reviewed-by: imsnah
Parallelize LURemoveExport
Parallelize LURemoveInstance
Using the new add/remove infrastructure this becomes pretty easy! :)
Parallelize LUCreateInstance
Finally, instance create on different node, without iallocator, can runin parallel. Iallocator usage still needs all nodes to be locked,unfortunately. As a bonus most checks which could have been moved toExpandNames, before any locking is done....
Implement adding/removal of locks by declaration
With this patch LUs can declare locks to be added when they start and/orremoved after they finish. For now locks can only be added in theacquired state, and removed if owned, and added locks default to be...
Remove locking of instances in certain queries
This patch is similar to the node patch (rev 1650). We disable lockingof instance (and nodes) if we only query static information.
Reviewed-by: ultrotter
Optimize the OpQueryNodes for names only
Currently, OpQueryNodes is locking all nodes (in shared mode), whichwill also block the special case of querying only for the node names(this is needed for gnt-cluster command, for example). There is nological requirement to not give the administrator enough power if she/he...
Parallelize LUQueryInstanceData
Parallelize LUVerify{Cluster,Disks}
These are two easy querying LUs which require shared access to allnodes/instances.
Parallelize LUReplaceDisks
This is the most complex parallelization so far. We have to lock oneinstance (and its nodes) plus one more node if doing a remote replace,or all nodes if doing a remote replace with iallocator.
_LockInstancesNodes: support append mode
This will be used to lock the instance's nodes in addition to some more.
Parallelize LU{A,Dea}ctivateInstanceDisks
Now that they are not used in other opcodes by chaining,this can easily be done.
LUReplaceDisks: remove use of ChainOpCode
The calls to OpActivateInstanceDisks and OpDeactivateInstanceDisks hasbeen replaced by _StartInstanceDisks and _SafeShutdownInstanceDisksrespectively. This is the last usage of ChainOpCode.
Create new _SafeShutdownInstanceDisks function
This new function checks whether an instance is running, before shuttingdown its disks. This is what the Exec() of LUDeactivateInstanceDisksdid, so that is replaced by a call to this function.
Fix a typo in LogicalUnit.ExpandNames docstring
s/locking.LEVEL_INSTANCES/locking.LEVEL_INSTANCE/
Use constants.LOCKS_REPLACE instead of hardcoding
This constant replaces what we used to write in recalculate_locks, andrepresents the lock recalculation mode. It lives in constants.py becauseit's used only in cmdlib, and thus doesn't deal with the locking library...
Fix LUReplaceDisks with iallocator
self._RunAllocator() sets self.op.remote_node, but doesn't return thenew remote node. If we set it to the return value of the function webasically reset it to None, and iallocator is never run.
Fix LUGrowDisk
The rpc library returns a list, not a tuple, so we'll accept both.
Fix iallocator run
Parallelize LUExportInstance
Unfortunately for the first version we need to lock all nodes. The patchdiscusses why this is and discuss ways to improve this in the future.
Parallelize LUGrowDisk
LURebootInstance: lock only primary when possible
When rebooting an instance and we're not changing it's disks status (allthe cases except in a "full" reboot) we can lock just its primary node.
Add primary_only flag to _LockInstancesNodes
As the name says when the flag is on (the default is off) only theprimary nodes are locked, as opposed to all of them.
Add locking.ALL_SET constant and use it
Rather than specifying None in needed_locks every time, with a nicecomment saying to read what we mean rather than what we write, and thatNone actually means All, in our magic world, we'll hide this secretunder the ALL_SET constant in the locking module, which has value, you...
Prevent mistakes using _GetWantedNodes
All the users of _GetWantedNodes have been converted to be concurrentLUs, and thus cannot call this function with an empty list of nodesanymore. This patch makes this restriction a part of the functionitself. This prevents mistakes in new concurrent LUs, and creates more...
Paralleliza LUQueryNodeVolumes and LUQueryExports
Parallelize LUDiagnoseOS
LUQueryExports: make 'node' field mandatory
It turns out this fields was already mandatory. If it hadn't beed valid,in fact, a value of None would have been passed to _GetWantedNodes whichwould have thrown an exception.
s/Chain(OpQueryExports)/rpc.call_export_list(...)/
Parallel opcodes are not (yet?) supported for chaining. Turns outthough that chaining is used only four times in the code, and twice it'sfor querying exports. But what's the need to chain the full opcode, when...
Fix wrong indentation in LUQueryNodes
Merge r1607 from branches/ganeti/ganeti-1.2
Use a default vnc_bind_address if None is specified
merge r1568 from branches/ganeti/ganeti-1.2
Add more fields to gnt-instance list
merge r1548 from branches/ganeti/ganeti-1.2
Fix wrong wording of instance rename error message.
merge r1541 from branches/ganeti/ganeti-1.2
more information for VNC console port
merge r1539 from branches/ganeti/ganeti-1.2
Display VNC console port in gnt-instance info.
merge r1538 from branches/ganeti/ganeti-1.2
Check HVM device type on instance modify as well.
Check memory size before setting it
With this change when a user asks for a new memory size for an instance,the number is checked instead of just applied. The operation fails onlyif the instance would not be able to restart on its primary node, butgenerates warnings should it be impossible to failover the instance or...
Pass the force param to SetInstanceParms
It was already allowed in gnt-instance modify, but ignored.It will be used to force skipping parameter checks.
This is a forward-port from branches/ganeti-1.2
Original-Reviewed-by: imsnahReviewed-by: iustinp
Merge r1536 from branches/ganeti/ganeti-1.2
Add HVM device type flags 2/3
merge r997 from branches/ganeti/ganeti-1.2
Fix gnt-instance modify for HVM parameters
This patch makes gnt-instance modify work again for the advancedHVM parameters after it was broken by other changes.
Fix issue when acquiring empty lock sets
By design if an empty list of locks is acquired from a set, no locks areacquired, and thus release() cannot be called on the set. On the otherhand if None is passed instead of the list, the whole set is acquired,...
Parallelize LUQueryNodes
As for LUQueryInstances the first version just acquires a shared lock on allnodes. In the future further optimizations are possible, as outlined bycomments in the code.
Parallelize LUQueryInstances
This first version acquires a shared lock on all requested instances andtheir nodes. In the future it can be improved by acquiring less locks ifno dynamic fields have been asked, and/or by locking just primary nodes.
LURebootInstance: move arg check in ExpandNames
The check for the reboot type can be done without any locks held, sowe'll move it to ExpandNames. Plus, we note in a FIXME that if thereboot type is not full, we can probably just lock the primary node, and...
LUVerifyCluster: Return boolean indication success
Reviewed-by: schreiberal
Implement {Add,Readd,Remove}Node in GanetiContext
By doing this we've a central place which coordinates what needs to bedone when adding or removing nodes. Another patch will add calls intothe job queue.
Two log messages move to config.py.
When removing a node, node_leave_cluster is now called after it has...
Fix cluster destroy
With the recent startup/shutdown changes (and with the master daemon inplace), the cluster destroy needs some fixing.
This patch moves the finalization of the destroy out from cmdlib intobootstrap, so we can nicely shutdown the rapi and master daemons....
Parallelize LUFailoverInstance
Fix pylint-detected issues
This is mostly: - whitespace fix (space at EOL in some files, not all, broken indentation, etc) - variable names overriding others (one is a real bug in there) - too-long-lines - cleanup of most unused imports (not all)...
Parallelize {Startup,Shutdown,Reboot}Instance
Parallelize LUReinstallInstance
self.recalculate_locks[locking.LEVEL_NODE] could have any value andeverything would work anyway. We'll use the string 'replace' byconvention because in the future we might want an 'append' mode.
LogicalUnit._LockInstancesNodes helper function
This function is used to lock instances' primary and secondary nodesafter locking instances themselves.
Make sharing locks possible
LUs can declare which locks they need by populating theself.needed_locks dictionary, but those locks are always acquired asexclusive. Make it possible to acquire shared locks as well, bydeclaring a particular level as shared in the self.share_locks...
Add LogicalUnit.DeclareLocks
This additional LogicalUnit function is optional to implement, but letsyou change your locking needs for one level just before locking it, butafter the previous levels have been already locked. It is useful forexample to calculate what nodes to lock after locking an instance....
LURenameInstance, add/remove relevant locks
LURenameInstance forgot to remove the old lock name and add the new one,making it impossible for parallel LUs to act on the instance (without amaster daemon restart). This also fixes burning+rename with theparallelization of {Start,Stop}Instance....
Rework master startup/shutdown/failover
This (big) patch reworks the master startup/shutdown and the fixes themaster failover.
What does the patch do?
For master start/stop: - remove the old ganeti-master script and its associated man page - moves the ip start/stop directly into the backend.(Start|Stop)Master...
Add a new parameter to backend.(Start|Stop)Master
This patch adds a new, unused for now, parameter to the start and stopmaster operations in backend. The idea behind it is that we need to beable to control whether the IP (de)activation is coupled with daemon...
Convert set to a list in LUGetTags
The set triggers exception on a list-tags command and RAPI calls for tagssince it is not serializable by JSON.
Convert SetInstanceParams to concurrency
Grab a lock for the instance we're working on, and update its params.
Use Update in SetInstanceParams
When we set the instance params we're not adding a new instance, butjust updating an existing one, so why using AddInstance?
Convert LUConnectConsole to concurrency
For ConnectConsole we just need to lock the instance we're connectingto. We make a few rpcs to its primary node, but node daemons can nowhandle multiple queries and nodes cannot be removed till they haveinstances on them anyway. Note that since we return the ssh command, and...
Add _ExpandAndLockInstance auxiliary function.
LUs that take an instance name as input and need to expand its name andlock it can use it to simplify their ExpandNames call. Possibly, and_ExpandAndLockNode will come as well.
Convert two (simple) LUs to be concurrent
LUQueryClusterInfo and LUDumpClusterConfig can be made concurrent anddon't need to acquire any locks. In fact they don't interact with thecluster at all, but just with its configuration, which is thread-safe by...
Add missing empty line
Two top level definitions were separated only by one empty line.Fixing this.
Remove the old locking functions
This removes (hopefully) all traces of the old locking functions anduses.
Convert LUTestDelay to concurrent usage
In order to do so: - We set REQ_BGL to False - We implement ExpandNames
That's it, really.
LogicalUnit: add ExpandNames function
New concurrent LUs will need to call ExpandNames so that any namespassed in by the user are canonicalized, and can be used by hooks,locking and other parts of the code. This was done in CheckPrereqbefore, but it's now splitted out, as it's needed for locking, which in...
Add a missing import to cmdlib
cmdlib uses some constants from locking (ie. locking levels) but doesn'timport it. This patch fixes the issue.
Fix an error accessing the cfg
Since the context is passed to LogicalUnit, rather than the cfg, we canonly access the cfg as self.cfg, self.context.cfg, or context.cfg (inthe constructor). cfg is not valid anymore.
Add and remove instance/node locks
Whenever we add an instance or node to the cluster (i.e. to the configand whenever we remove them we should add/remove locks as well). In thefuture we may want to optimize this so that the configwriter does it, orit's handled at the context level, but till we're adding/removing...
Pass context to LUs
Rather than passing a ConfigWriter to the LUs we'll pass the wholecontext, from which a ConfigWriter can be extracted, but we can alsoaccess the GanetiLockManager. This also fixes the places where a FakeLUis created.
Fix a typo in LUTestDelay docstring
Add REQ_BGL LogicalUnit run requirement
When logical units have REQ_BGL set (it is currently the default) theyneed to be the only ganeti operation run on the cluster, and we'llguarantee it at the master daemon level. Currently only one thread isrunning at a time, so this requirement is never broken....
AddNode: move the initial setup to boostrap
From the master node we can't start ssh and connect to the remote node,nor we can do it from ganeti-noded as this ssh section will possibly askfor key confirmation and password. So the code to copy the ganeti-noded...
LUAddNode: use node-verify to check node hostname
As we can't use ssh.VerifyNodeHostname directly, we'll set up a mininode-verify to do checking between the master and the new node. In thefuture networking checks, or more nodes, can be added as well.
LUAddNode: use self.sstore, not a local ss
Since we're inside a LU we have access to self.sstore.No need to use ss, which separate instantiation will disappear in a fewpatches! ;)
LUAddNode: upload files via rpc, not scp
We used to scp all the ssconf files, and the vnc password file to thenew node. With this patch we use the upload_file rpc, specifying justthe new node as a destination. All the files previously copied by scpare already allowed by the backend....
Change fping to TcpPing in two LUs
Two LUs are using RunCmd to call fping, in order to check for an IPpresence on the network. Substituting it with TcpPing will get rid ofit, which makes it not break in the new world order, where the mastercannot fork....
When removing a node don't ssh to it
Even in 1.2 this behaviour is broken, as the rpc call will remove thessh keys before we get a chance to log in. Now the rpc takes care ofshutting down the node daemon as well, so we definitely can avoid this.
This makes the LURemoveNode operation work again with the threaded...
Remove spurious check during LUAddNode
There is no point in checking whether the cluster VNC password fileexists as a prerequisite for AddNode, considering the check happens onthe master node, not the target one. Removing this check.
Improve LURemoveNode BuildHooksEnv docstring
Cleanup old DRBD 0.7.x code
Apparently there were still some leftovers. While removing an instance,I got the message "unhandled exception 'module' object has no attribute'LD_MD_R1'".
Fix gnt-cluster “command” and “copyfile”
Since the disabling of forking in the master daemon, the two ssh-basedsubcommands were not working anymore. However, there is no need at allfor the commands to be run from the master daemon (permissions to readthe cluster private ssh key notwithstanding), they can be run directly...
Add a ‘tags’ field to instance and node listing
Currently there isn't any easy way to list all nodes or instance andtheir tags; you have to query each node in turn, or list all the tagsvia something like “gnt-cluster search-tags '.*'”. Of course, this is...
Fix an error-handling case
There is a mistake in handling grow-disk for an invalid disk. This patchfixes it.
Implement disk grow at LU level
This patch adds a new opcode and LU for growing an instance's disk.
The opcode allows growing only one disk at time, and will throw an errorif the operation fails midway (e.g. on the primary node after it hasbeen increased on the secondary node). As such, it might actually leave...
Move SetKey to WritableSimpleStore and use it
Before we used to be able to update SimpleStore by just calling SetKey, thisfeature is now moved to an external class, which inherits from it. In thispatch the new WritableSimpleStore class is also put to use, in the LUs that...
Activate down instances' disks on replace-disks
When replacing disks or evacuating nodes with instances administrativelydown ganeti fails because the instance disks are not active. This patchactivates them, performs the replacement, and shuts them down again....
FailoverInstance: change AddInstance with Update
We're not adding a new instance, just making configuration changes tothe one we're working on.
Fix an error message in instance add
There is a mistake in the error message generated when we can't reach anode for checking for available disk space. Without it, the errormessage is:Failure: prerequisites not met for this operation:Cannot get current information from node '{u'gnte2.lab.k1024.org':...
Move InitCluster opcode into a single function
This allows us to initialize a new cluster. The code certainly containsbugs and hooks aren't implemented yet.
Move cmdlib._HasValidVG to utils.CheckVolumeGroupSize
This is required for splitting the cluster initialization code.
Move {Set,Remove}EtcHostsEntry wrappers to utils.py
This is required for the split of the cluster initialization code.
Reviewed-by: iustinp, ultrotter
Remove REQ_CLUSTER from opcode handling code
It's not needed anymore now that all opcodes require a cluster. Clusterinitialization was the only exception.
Add check for node memory in instance creation
Currently the check for enough memory is done only on instance startcommand and failover command. But we also start an instance in instancecreate, therefore we need to check this instead of failing to start in...
Show cluster hypervisor for gnt-cluster info
Author: schreiberalReviewed-by: iustinp
Forward-port: make gnt-modify work with new HVM parameters
This fixes gnt-instance modify so it actually works with thenew HVM parameters for Ganeti 1.2
Forward-port: show only parameters relevant to the instance
This patch modifies the code for "gnt-instance info .." to only displayinstance parameters that actually apply to that instance, i.e. for PVMinstances no HVM parameters are shown and vice versa....
Forward-port: patch 2/4 extended HVM features for 1.2
This patch adds the commandline extensions and the code to storeand display the extended HVM features.