Synnefo » snf-ganeti

Prevent ssconf values from having non-string values

For whatever reason, my test cluster managed to acquire
shared_file_storage_dir with a None value, instead of empty
string. This is not flagged in masterd itself, but the node daemon
will fail in writing the value to disk, as it calls len() on the...

Ignore parameter completion for bash completion

Named parameters are only handled by zsh completion for the time being.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Fix parts of shared storage migration

Commit faaabe3c fixed failover behaviour for DTS_INT_MIRROR instances, however
it broke migration for DTS_EXT_MIRROR instances, by moving iallocator and node
checks from LUInstanceMigrate to TLMigrateInstance. This has the side-effect...

Allow disk adoption during disk addition

It is now possible to allow adopting a disk during gnt-instance modify time, as
follows:

gnt-instance modify --disk add:adopt=/path/to/disk (blockdev)
or
gnt-instance modify --disk add:adopt=<lvname> (plain)

We do the same checks as during instance creation....

Add DTS_MIRRORED frozenset

Use DTS_MIRRORED to indicate mirrored disk templates that allow
migrations/failover.

DTS_MIRRORED is the union of DTS_EXT_MIRROR and DTS_NET_MIRROR.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>...

Shared block storage support

This patch introduces basic shared block storage support.

It introduces a new storage backend, bdev.PersistentBlockDevice, to
use as a backend for shared block storage. The new bdev requires a new
BLOCKDEV_DRIVER_MANUAL constant with the value "manual" and uses it as...

Migration and failover: add iallocator and target_node slots

Add iallocator and target_node slots to OpMigrateInstance and
OpFailoverInstance to facilitate shared-storage-backed instance mobility. Add
iallocator slot to OpMigrateNode (no explicit target_node in this case)....

Shared storage instance migration

Modify LUMigrateInstance and TLMigrateInstance to allow instance migrations for
instances with DTS_EXT_MIRROR disk templates.

Migrations of shared storage instances require either a target node, or an
iallocator to determine the target node. If none is given, the cluster default...

Shared storage instance failover

Modify LUFailoverInstance to enable shared storage instances to failover.
Shared storage instance failover requires either a target node or an
iallocator to determine the target node. If none is given, the cluster default...

Rename DTS_NET_MIRROR to DTS_INT_MIRROR

DTS_INT_MIRROR better contrasts DTS_EXT_MIRROR.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
[iustin@google.com: updated patch for changed context]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Core shared file storage support

This patch introduces core file storage support, consisting of the following:

A configure-time switch for enabling/disabling shared file storage
support and controlling the shared file storage location:
--with-shared-file-storage-dir=. Shared file storage configuration is then...

Treat empty oob_program param as default

There is currently no way to reset oob_program back to its default from
the cmdline, which causes problems for cluster-merge. This patch means
that the following now works:
gnt-cluster modify --node-parameters oob_program=...

Fix bug related to log opening failures

If opening the log file fails, then we shouldn't attempt to use that
variable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Merge branch 'devel-2.2' into devel-2.3

• devel-2.2:
  Fix LUClusterRepairDiskSizes and rpc result usage
  Fix RPC mismatch in blockdev_getsize[s]

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Fix RPC mismatch in blockdev_getsize[s]

Commit 92fd2250 added consistency checks in the RPC layer, which broke
the call_blockdev_getsizes RPC call (declared with 's' at the end in
rpc.py, without 's' in the node daemon).

The immediate fix is to correct the rpc function name, the long term...

Merge remote branch 'stable-2.4' into devel-2.4

• origin/stable-2.4:
  Fix typo in kvm-ifup script
  NEWS: Replace smartquotes, start lines with uppercase
  Update NEWS and release 2.4.0 rc3
  Fix potential data-loss bug in disk wipe routines

Signed-off-by: Guido Trotter <ultrotter@google.com>...

Merge branch 'devel-2.4' into stable-2.4

• devel-2.4:
  1-char comment typo fix
  Expand some acronyms, add to glossary
  query_unittest: Fix argument to set()

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

1-char comment typo fix

Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fix title of query field containing instance name

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Change the list formatting to a 'special' chars

And also enable verbose display via the, well, verbose option. Man
page and tests are updated, and the formatting is moved from 4 if
statements to a data structure.

Signed-off-by: Iustin Pop <iustin@google.com>...

RAPI: remove required parameters for reinstall

Before c744425f354f1bef2d0d7d306e2d00c494d67d2b instance reinstall
accepted the "os" and "nostartup" optional query parameters. With that
commit it was changed to allow "os" "start" and "osparams" via body
rather than encoded in the URL. Unfortunately that commit introduced a...

NodeQuery: mark live fields as UNAVAIL for non-vm_capable nodes

Since we don't have the data per design, UNAVAIL is appropriate here,
while NODATA is not.

The patch also adds a comment: if we extend the live fields list to
contain other data in the future, we need to reevaluate this solution....

Fix LUClusterRepairDiskSizes and rpc result usage

This LU was introduced before the RPC result conversion from .data to
.payload, and it has managed to keep the old-style usage (how? it's
the only LU that does so). Fix by changing to payload, and add some...

Fix bug in iallocator data structures build

Commit a1cef11c fixed non-vm_capable nodes export, but broke
inadvertently offline nodes. The update of the dict only needs to
happen for online nodes, in the 'if' block.

Without this patch, offline nodes keep the data from the last node...

cluster verify and instance disks on offline nodes

Currently, cluster-verify says:

- ERROR: instance instance14: couldn't retrieve status for disk/0 on node3: node offline
- ERROR: instance instance14: instance lives on offline node(s) node3
- ERROR: instance instance15: couldn't retrieve status for disk/0 on node3: node offline...

Handle gnt-instance shutdown --all for empty clusters

The current code gives:
Failure: prerequisites not met for this operation:
error type: wrong_input, error details:
Selection filter does not match any instances

Signed-off-by: Stephen Shirley <diamond@google.com>...

Bump up intra-cluster import connect timeout

Currently, the export timeout is 10 times 20 seconds, but the import
is only 30 seconds. I'm raising this to 60 seconds with two goals in
mind:

- when debugging manually, this allows for easier synchronisation of...

Fix handling of ^C in the CLI scripts

This adds a message and nice handling of ^C, especially useful for
``gnt-job watch``.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

backend: Disable compression in export info file

The new import/export infrastructure in Ganeti 2.2 and up handles
compression differently. It no longer writes compressed files to the
destination. Unfortunately changing this behaviour would be non-trivial,...

Reopen log files upon SIGHUP in daemons

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

utils.log: Restrict I/O error handling coverage

The I/O error will occur while opening the file, not while adding
and configuring the handler.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Enforce that new node groups have unique names

Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Minor grammar fix in QuitGanetiException docstring

Signed-off-by: Stephen Shirley <diamond@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Re-create instance disk symlinks on activate

This patch implements recreation of instance disk symlinks when the
activate-disks operation is run. Until now, it was not possible to
re-create these symlinks without stopping and starting or migrating an
instance as the RPC call where this is done was in instance startup...

Export console information as query field

This makes it possible to get the console information via a LUXI query.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

ConfigWriter: add checks for be/nd/nic params

This adds checking (in the configuration) for invalid be, nd and nic
params. The code is a bit tricky as nd params are at cluster,
nodegroup and node level, nicparams are at cluster and nic level,
whereas beparams are at cluster and instance level....

Prevent removal of last node group

- Add check in ConfigWriter to prevent last node group from being
removed
- Tidy up error message a bit

Signed-off-by: Stephen Shirley <diamond@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Merge branch 'devel-2.3' into devel-2.4

• devel-2.3:
  Wait for master to become available on initialization
  Start all daemons on cluster initialization
  Clarify job processing order in admin guide
  Improve option descriptions
  Remove two unused variables...

RAPI client: Wrap /2/redistribute-config resource

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Watcher: Fix endless repair tries for broken secondary

In cases where secondary was offline and not evacuated watcher tried
to activate-disks in an endless manner, but this is useless, as the
secondary is offline and therefore not responding to this approach....

gnt-cluster verify-disks: fix VG name

Recent multi-VG work already exports the missing LV names as vg/lv,
not simply lv. So the query and addition of the VG name in gnt-cluster
verify-disks is redundant, and even wrong for non-default-VG
instances.

Signed-off-by: Iustin Pop <iustin@google.com>...

Wait for master to become available on initialization

This is analogue to the existing check for a responsive node daemon.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Improve option descriptions

Also replace hardcoded “xenvg” with constant.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Show hidden/blacklisted OSes in cluster info

Since we can blacklist/hide non-existing OSes (for preseeding), we
cannot query easily the OSes themselves for this status. Hence we
export the entire lists in cluster info (which should be cheaper than
gnt-os diagnose)....

Rephrasing two error messages for auto promotion

Using auto_promote or auto-promote can lead to confusion on using the
user facing interfaces. While auto-promote is fine for CLI it's not for
RAPI and vice-versa. This patch should eliminate this confusion....

Fix bug in “gnt-node list-storage”

LVM PV storage units would always show as allocatable, even when they
weren't. For some reason I have not been able to determine, the function
parsing the attributes (“_GetAllocatable”) was not even called and the
list opcode simply returned the attribute string as the value (e.g....

Fix premature abort of LUOobCommand due to result.Raise

This is a bug I recognized while doing tests on gnt-node health. A left
over result.Raise line causes premature abort of LUOobCommand on the
first node failing the RPC call. This is not expected behaviour for...

Modify LUOobCommand to support multiple nodes

This will change the result of this LU to a query like result. A list of
tuples with information about the state of the data.

It also includes the modification to the commands calling this opcode.

Signed-off-by: René Nussbaumer <rn@google.com>...

Rename QRFS_* to RS_*

This patch renames QRFS_* to RS_* fields so they can be used in other
places (i.e. LUs) without confusion, as this was initially meant for
query operations.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix disk adoption breakage

Disk adoption is currently broken by 84d7e26b, which added multiple LVM
volume group support. This patch fixes the calls to rpc.call_vg_list,
which are multi-node calls but were handled as single-node calls in
84d7e26b.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>...