Synnefo » snf-ganeti » ganeti-local

Fix issue when acquiring empty lock sets

By design if an empty list of locks is acquired from a set, no locks are
acquired, and thus release() cannot be called on the set. On the other
hand if None is passed instead of the list, the whole set is acquired,...

Parallelize LUQueryInstances

This first version acquires a shared lock on all requested instances and
their nodes. In the future it can be improved by acquiring less locks if
no dynamic fields have been asked, and/or by locking just primary nodes.

Reviewed-by: imsnah

LUVerifyCluster: Return boolean indication success

Reviewed-by: schreiberal

Fix cluster destroy

With the recent startup/shutdown changes (and with the master daemon in
place), the cluster destroy needs some fixing.

This patch moves the finalization of the destroy out from cmdlib into
bootstrap, so we can nicely shutdown the rapi and master daemons....

Fix pylint-detected issues

This is mostly:
- whitespace fix (space at EOL in some files, not all, broken
indentation, etc)
- variable names overriding others (one is a real bug in there)
- too-long-lines
- cleanup of most unused imports (not all)...

Parallelize LUReinstallInstance

self.recalculate_locks[locking.LEVEL_NODE] could have any value and
everything would work anyway. We'll use the string 'replace' by
convention because in the future we might want an 'append' mode.

Reviewed-by: iustinp

Make sharing locks possible

LUs can declare which locks they need by populating the
self.needed_locks dictionary, but those locks are always acquired as
exclusive. Make it possible to acquire shared locks as well, by
declaring a particular level as shared in the self.share_locks...

LURenameInstance, add/remove relevant locks

LURenameInstance forgot to remove the old lock name and add the new one,
making it impossible for parallel LUs to act on the instance (without a
master daemon restart). This also fixes burning+rename with the
parallelization of {Start,Stop}Instance....

Add a new parameter to backend.(Start|Stop)Master

This patch adds a new, unused for now, parameter to the start and stop
master operations in backend. The idea behind it is that we need to be
able to control whether the IP (de)activation is coupled with daemon...

Convert SetInstanceParams to concurrency

Grab a lock for the instance we're working on, and update its params.

Reviewed-by: iustinp

Convert LUConnectConsole to concurrency

For ConnectConsole we just need to lock the instance we're connecting
to. We make a few rpcs to its primary node, but node daemons can now
handle multiple queries and nodes cannot be removed till they have
instances on them anyway. Note that since we return the ssh command, and...

Convert two (simple) LUs to be concurrent

LUQueryClusterInfo and LUDumpClusterConfig can be made concurrent and
don't need to acquire any locks. In fact they don't interact with the
cluster at all, but just with its configuration, which is thread-safe by...

Remove the old locking functions

This removes (hopefully) all traces of the old locking functions and
uses.

Reviewed-by: imsnah

LogicalUnit: add ExpandNames function

New concurrent LUs will need to call ExpandNames so that any names
passed in by the user are canonicalized, and can be used by hooks,
locking and other parts of the code. This was done in CheckPrereq
before, but it's now splitted out, as it's needed for locking, which in...

Fix an error accessing the cfg

Since the context is passed to LogicalUnit, rather than the cfg, we can
only access the cfg as self.cfg, self.context.cfg, or context.cfg (in
the constructor). cfg is not valid anymore.

Reviewed-by: iustinp

Pass context to LUs

Rather than passing a ConfigWriter to the LUs we'll pass the whole
context, from which a ConfigWriter can be extracted, but we can also
access the GanetiLockManager. This also fixes the places where a FakeLU
is created.

Reviewed-by: iustinp

Add REQ_BGL LogicalUnit run requirement

When logical units have REQ_BGL set (it is currently the default) they
need to be the only ganeti operation run on the cluster, and we'll
guarantee it at the master daemon level. Currently only one thread is
running at a time, so this requirement is never broken....

LUAddNode: use node-verify to check node hostname

As we can't use ssh.VerifyNodeHostname directly, we'll set up a mini
node-verify to do checking between the master and the new node. In the
future networking checks, or more nodes, can be added as well.

Reviewed-by: iustinp

LUAddNode: upload files via rpc, not scp

We used to scp all the ssconf files, and the vnc password file to the
new node. With this patch we use the upload_file rpc, specifying just
the new node as a destination. All the files previously copied by scp
are already allowed by the backend....

When removing a node don't ssh to it

Even in 1.2 this behaviour is broken, as the rpc call will remove the
ssh keys before we get a chance to log in. Now the rpc takes care of
shutting down the node daemon as well, so we definitely can avoid this.

This makes the LURemoveNode operation work again with the threaded...

Improve LURemoveNode BuildHooksEnv docstring

Reviewed-by: iustinp

Fix gnt-cluster “command” and “copyfile”

Since the disabling of forking in the master daemon, the two ssh-based
subcommands were not working anymore. However, there is no need at all
for the commands to be run from the master daemon (permissions to read
the cluster private ssh key notwithstanding), they can be run directly...

Fix an error-handling case

There is a mistake in handling grow-disk for an invalid disk. This patch
fixes it.

Reviewed-by: imsnah

Move SetKey to WritableSimpleStore and use it

Before we used to be able to update SimpleStore by just calling SetKey, this
feature is now moved to an external class, which inherits from it. In this
patch the new WritableSimpleStore class is also put to use, in the LUs that...

FailoverInstance: change AddInstance with Update

We're not adding a new instance, just making configuration changes to
the one we're working on.

Reviewed-by: imsnah

Move InitCluster opcode into a single function

This allows us to initialize a new cluster. The code certainly contains
bugs and hooks aren't implemented yet.

Reviewed-by: iustinp

Move {Set,Remove}EtcHostsEntry wrappers to utils.py

This is required for the split of the cluster initialization code.

Reviewed-by: iustinp, ultrotter

Add check for node memory in instance creation

Currently the check for enough memory is done only on instance start
command and failover command. But we also start an instance in instance
create, therefore we need to check this instead of failing to start in...

Forward-port: make gnt-modify work with new HVM parameters

This fixes gnt-instance modify so it actually works with the
new HVM parameters for Ganeti 1.2

Author: schreiberal
Reviewed-by: iustinp

Forward-port: patch 2/4 extended HVM features for 1.2

This patch adds the commandline extensions and the code to store
and display the extended HVM features.

Author: schreiberal
Reviewed-by: iustinp

LURemoveInstance: fix op.ignore_failures usage

Currently: the LURemoveInstance.Exec() method uses the ignore_failures
attribute of the OpRemoveInstance opcode, but it doesn't check for its
existence. The patch adds this attribute to _OP_REQP and to all the...

Forward-port changes made to readd in 1.2

qa_node.py: Fix typo in message
cmdlib.py: Don't add readded node to node list
ganeti-qa.py: Make sure readd isn't done for master node

Reviewed-by: iustinp

Export the number of cpus to iallocator scripts

Now that we have the number of cpus available from the hypervisors, we
can export this to the iallocator scripts.

Reviewed-by: ultrotter

Add cluster-verify hooks

Only post-hooks are run on cluster verify, and then their output is sent back
to the LU, which upon failure displays it to the user and changes the result of
the execution to a failure.

Reviewed-by: iustinp

Remove NoHooksLU.BuildHooksEnv

Since NoHooksLU defines HPATH as None, BuildHooksEnv will never be called (as
the LogicalUnit.BuildHooksEnv docstring correctly states). Removing the
function altogether, to avoid having dead code lying around, and to make sure...

Raise PrereqError when exporting file-based instance

This patch adds a check to LUExportInstance.CheckPrereq to raise an
error when an instance with file disks is exported.

Reviewed-by: ultrotter

Move iallocator script execution to ganeti-noded

Currently the iallocator execution takes place in the master, which is a
violation of the current architecture, and will create problems with a
threaded master daemon.

This patch moves the execution to the backend, similar to the hooks...

IAllocator: some more info exported

This patch adds the following information to the exported info:
- hypervisor type (in the main dict)
- total memory used by primary instances (in each node dict) (can be
computed from the node+instance dicts, but it's cheap to compute...

Style fixes for trunk

This small patch fixes:
- wrong indentation in two places
- use of 'os' variable that hides global scope os module

Reviewed-by: imsnah

Fix generalized relocate mode of IAllocator

The patch which generalized the IAllocator was half-true: it actually
put the selection of the node inside the IAllocator, so callers were not
able to specify replace primary node.

This patch does:
- split the arguments to the constructor in three sets: mode and name...

Generalize the replace_secondary mode in iallocator

Currently the replace_secondary mode is too restrictive. This patch
changes this to a general 'relocate' mode where the node(s) to be
changed are specified via a new key in the request dict ('nodes') so...

Move all iallocator functions into a class

This patch moves all the iallocator function into a separate class that
is then somewhat easier to use. It doesn't bring any new functionality.

The patch also changes the way the iallocator is called - the
OpTestAllocator opcode is no longer needed, and all its parameters...

Add --readd option to “gnt-node add”

This allows us to readd a node after it failed and required a
reinstallation or replacement.

Reviewed-by: iustinp

Reorder checks in instance create

This patch reorders the checks in the instance create prereq so that all
checks and normalisations that are not node-dependent are done before
the node dependent checks.

This is done so that, after the instance-related opcode parameters are...

Allocator framework, 1st part: allocator input generation

In preparation for the introduction of automatic instance allocator,
this patch adds an allocator simulation opcode, that based on the input
parameters, will return either the input message to the allocator...

Bugfix: wrong identifier in CheckPrereq message

Reviewed-by: iustinp

Verify: make skipping checks possible

Add a general way to skip some checks at cluster-verify time and make the N+1
memory redundancy check optional.

Reviewed-by: iustinp

Verify: save instance config

Save the instance config after we queried it in an instance_cfg dict. This can
be used later by any function that wants it, without reloading it from the
configuration module. It will be used for N+1 memory resilience checking....

Verify: add instance information to node_info

With this patch node_info is changed to store information about which primary
and secondary instances are configured on a node. This information is useful to
check memory and disk allocation. A list of non-redundant instances is also...

Rework the results of OpDiagnoseOS opcode

Currently, the opcode DiagnoseOS is the only opcode that return a
structure of objects.OS (which is a custom class, and not a simple
python object) and furthermore all the processing of OS validity across
nodes is left to the clients of this opcode....