Synnefo » snf-ganeti » ganeti-local

Restrict instance move to templates based on local files

Moving an instance is done by copying over the disks. Restrict
this to disk templates that are copyable. This avoids accidental
use on, e.g., the sharedfile template.

Signed-off-by: Klaus Aehlig <aehlig@google.com>...

Make _CreateDisk clean up partially created disks on failure

_CreateDisk used to just throw an exception if _CreateBlockDev failed
leaving the caller in the state that some disks were created, without
precise knowledge which. Usually, the clean up then overapproximated...

Fix instance policy checks for default back-end parameters

Policy violations of back-end parameters that used the cluster default
value were not reported in cluster-verify.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix policy check for disk templates

Instance disk template is checked against the policy, and diskless
instances aren't checked for the number of disks.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix LUTestAllocator with instance alloc

This is similar to commit 8775e62a; the addition of node_whitelist
broke this LU as well.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Remove early returns in network LUs

Remove any early returns in LUNetworkDisconnect/LUNetworkConnect
and replace them with if-else statements.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fix networks in LUInstanceSetParams()

Params passed in _CreateNewNic() are not yet evaluated and include
the value passed by user for the network. A lookup must be done first
in order to find the corresponding network UUID which should be stored
in the newly created NIC object....

Implement network locking in Instance queries

This is needed in case more info than each nic's network uuid is to
be returned. We need to lock networks to get valid data. For now
only the name is returned as an extra field. All other can be added
with trivial effort....

cmdlib changes to support nic.network as uuid

Refactor Instance related LUs to support nic.network as
a uuid. This removes all the unnecessary invocations to
LookupNetwork().

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>

If _UnlockedLookupNetwork() fails raise error

Make _UnlockedLookupNetwork() raise OpPrereqError (instead of returning
None) in case it does not find the requested network. Remove useless and
duplicate code such as:

if net_uuid is None:
raise...

This is a cherry-pick of commit 1cce2c4....

Remove network_type slot (Issue 363)

This slot was not used by Ganeti so the same info can be
provided via tags. In order not to break configuration data
we add a FromDict() method in Network config object that
removes the deprecated network_type (if found) and then invoke...

Make use of HooksDict() for networks

This can be used in hypervisor code as well. For consistency
export *NETWORK_NAME and not *NETWORK throughout the code.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Guido Trotter <ultrotter@google.com>

"exclusive_storage" cannot be changed on single nodes

There's never been support for a configuration where nodes in the same node
group have different values of the exclusive_storage flag. This patch
disables the possibility to change the flag for individual nodes....

_VerifyErrors()._Error() and _ErrorIf() are now consistent

_Error() didn't contain the logic for demoting errors to warnings and for
marking an operation as failed. Now _ErrorIf() is just a minimal wrapper
for _Error().

Unit tests included.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...

Run pre-migrate hooks on primary node too

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Reviewed-by: Guido Trotter <ultrotter@google.com>

cluster-verify checks uniformity of PV sizes

When exclusive_storage is set, cluster-verify complains if the size of PVs
is not uniform across a node group.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Moved checks within LUClusterVerifyGroup

Almost all instance-specific checks have been moved from the Exec method to
the _VerifyInstance method. This cleans up Exec, which was becoming too big
even for pylint…

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...

cluster-verify checks uniformity of exclusive_storage flag

The value of the flag should be the same for nodes within the same group.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add "use_locking" parameter to network query opcode

This was extracted from a patch by Dimitris Aragiorgis with the subject
“Add locking to _NetworkQuery”. The rest of the patch did no longer
apply and will require more work.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Add the gnt-storage client

Add a new client called 'gnt-storage'.
The client interacts with the ExtStorage interface, similarly to
the way gnt-os interacts with the OS interface.

For now, only two commands are supported: 'info' and 'diagnose'.

'diagnose' calculates the node status of each provider on each node,...

Merge branch 'stable-2.6' into devel-2.6

No conflicts, really trivial.

• stable-2.6:
  Final update of NEWS file for 2.6.2 and version bump
  Fix job completion with big job queues
  confd: reduce noise during normal config reload
  Change hbal handling of !auto_balance instances...

LUs raise an error if they cannot handle exclusive_storage

This happens when the exclusive_storage flag is set, and the requested
operation is incompatible with the flag or not yet supported.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add exclusive_storage parameter to node_info RPC

The flag is passed to the low-level functions that need it, but it's not
yet used. Also, RPCs get just a default value for now.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Call blockdev_create RPCs with the exclusive_storage flag

The flag is read from the configuration and passed to the RPC.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add exclusive_storage parameter to blockdev_create RPC

The flag is passed to the low-level functions that need it, but it's not
yet used.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Change network error text

This error is thrown if a network name already exists, not if the
network itself (e.g. “1.2.3.0/24”) is defined.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Multiple ExtStorage Providers and ext-params

Add support for passing parameters to the ext template (ext-params).
Take advantage of disk-params, that don't seem to make much sense in
this template (ExtStorage Providers are not predefined and we don't
know their needs) and use them to pass the ext-params dynamically to...

Implement the External Storage Interface

With this commit we introduce the External Storage Interface
to Ganeti, abbreviated: ExtStorage Interface.

The ExtStorage Interface provides Ganeti with the ability to interact
with externally connected shared storage pools, visible by all...

cmdlib: Additional opportunistic locking check

If opportunistic locking is requested in the opcode, but no iallocator
is used (not specified or no default), an error will be raised.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Fix network LUs use of error code in OpPrereqError

Currently, most network LUs simply use EINVAL instead of correct error
codes; this patch tries to adjust the obvious cases.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

cmdlib: Factorize network conflict detection

LUNetworkConnect and LUNetworkDisconnect had very similar code to detect
conflicts between instance's network interfaces and networks. This code
factorizes the common part and does some cleanup:

- Remove single-letter variable for lambda (“l”)...

Replace frozenset with compat.UniqueFrozenset

This is not a trivial s/frozenset/compat.UniqueFrozenset/, but rather
only replaces “frozenset” where appropriate. Most of the places are
“static” information that doesn't change after the module has been
loaded....

Allow shutting down offline instance

If an instance is offline we definitely shouldn't start it up.
But shutting it down, should it be up by mistake is not "that" bad.
Still, we only allow it with --force, as it still performs an action on
an instance we shouldn't touch. This should make everybody happy....

Fixes and improvements to comments

Some fixes, added more information in a few points, removed a stale (5+
year old) TODO comment.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

_DeclareLocksForMigration: Fix non-DRBD locking issue

When non-DRBD disks are used for an instance,
“lu.needed_locks[locking.LEVEL_NODE]” is set to “locking.ALL_SET” (which
is None). The assertion will then fail as None evaluates to False.

Reported by Constantinos Venetsanopoulos....

Fix two logging messages in TLReplaceDisks

Commit f0f8d060 (“Show old primary/secondary node on disk
replacement”) added two wrong uses of feedback_fn, which results in
log entries such like these:

"log": [
        [
          7, [1351258326, 466214], "message", "Replacing disk(s) 0 for instance 'instance1.example.com'"...

Very very very basic openvswitch support

This is a "better than nothing" support, just for kvm and just joining
the machine to the opevswitch bridge with the right command.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>...

Read watcher pause using RPC, not directly

The master daemon should not directly read files written by the node
daemon. This patch adds a new RPC to read the watcher pause file and
changes the master code to use it.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Add error code for temporary lack of resources

When an instance creation uses opportunistic locks, the iallocator might
not be able to find an allocation solution if not enough node locks (or
a suboptimal subset thereof) were acquired. As per the design document...

cmdlib: Node whitelist support for allocation request

Forward the node whitelist to the iallocator plugin.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUBackupExport: Use node allocation lock

Block instance allocations when all node locks will be acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUInstanceMultiAlloc: Use node allocation lock

Avoid conflicts between instance allocations.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUNodeSetParams: Use node allocation lock

Block instance allocations when all node locks will be acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUOobCommand: Use node allocation lock

If no node names are given, all node locks are acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Use new util function for mac_prefix validation

Use new NormalizeAndValidateThreeOctetMacPrefix() util function in
LUNetworkAdd/LUNetworkSetParams to validate network's MAC prefix.
Additionally, move the check in CheckArguments() in the case of
LUNetworkAdd....

LUClusterRepairDiskSizes: Use node allocation lock

This opcode acquires all node resource locks, which conflicts with
instance allocations.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

LUClusterVerifyGroup: Use node allocation lock

See comment in code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

LUInstanceChangeGroup: Acquire node allocation lock

Changing instances' groups shouldn't conflict with instance allocations.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Fix locking mistake introduced in commit 5cc1f88

The node resource locks were not set correctly on instance import.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

LUInstanceCreate: Acquire node allocation lock

Opportunistic locks are not yet used. This patch changes
LUInstanceCreate to acquire the node allocation lock to avoid conflicts
with other opcodes acquiring many node locks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...