Synnefo » snf-ganeti » ganeti-local

cmdlib: Use local alias for lock manager

Saves some typing and we'll use it more often in the future.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Show locksets in lock monitor

When all locks contained in a set are acquired, the lockset's internal
lock is acquired with the same mode. With this patch the internal lock
will show up on the lock monitor, named e.g. “instances/[lockset]”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

SharedLock: Avoid acquires from sneaking in while notifying

In some rare cases new shared acquires could sneak in through the
condition cached in “__pending_shared” while the code was still
notifying acquires. This was only working because such a condition...

Fix DTS_EXT_MIRROR migration

Commit faaabe3c fixed failover behaviour for DTS_INT_MIRROR instances, however
it broke migration for DTS_EXT_MIRROR instances, by moving iallocator and node
checks from LUInstanceMigrate to TLMigrateInstance. This has the side-effect...

config: Add function to determine instance's groups

This will be used for locking only the necessary node group(s)
for per-instance operations.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

cmdlib: Update error messages, remove some punctuation

- Clarify some error messages
- Remove unnecessary punctuation
- Merge two if conditions in one place

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

cmdlib: Factorize lock releasing

There will be more lock releasing with upcoming changes, so this will
centralize the logic behind it (what locks to keep, which variables to
update, etc.).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

TLReplaceDisks: Use implicit loop for dictionary

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

locking: Export “list_owned” from lock manager

This is analog to “is_owned” and will be used for assertions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Merge branch 'devel-2.4'

• devel-2.4: (24 commits)
  mlock: fail gracefully if libc.so.6 cannot be loaded
  Allow creating the DRBD metadev in a different VG
  Make _GenerateDRBD8Branch accept different VG names
  Fix WriteFile with unicode data
  Replace disks: keep the meta device in the same VG...

cmdlib: Drop SSH runner from LU base class

It is no longer used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

TLMigrateInstance: Fix confusing text

Commit d5cafd31 changed this error message, swapping the
text parts in the process.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

iallocator: Relocation nodes must be in same group

Quoting from iallocator.rst: “[…] ``relocate`` request is used when an
existing instance needs to be moved within its node group […]”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

SetEtcHostsEntry: maintain existing ordering

Currently RemoveEtcHostsEntry keeps the ordering, but SetEtcHostsEntry
not, as it will always write the new entry at the end of file. I
personally dislike this as it "uglifies" my custom host files, so this
patch makes it update the record instead in-place so to say instead of...

Allow creating the DRBD metadev in a different VG

This is a simple change to allow specifying a different VG for the
meta device during the creation of instances and addition of disks via
gnt-instance modify.

Signed-off-by: Iustin Pop <iustin@google.com>...

Fix WriteFile with unicode data

Unicode is fun, indeed:

len(buffer("abc"))

3

len(buffer(u"abc"))

12

So we can't pass unicode data to buffer(), as the result will be to
write the in-memory (usually UTF-32) representation to disk.

Signed-off-by: Iustin Pop <iustin@google.com>...

Replace disks: keep the meta device in the same VG

This patch enhances the multi-VG support in replace disks, by keeping
the meta device in the same VG, as opposed to moving it to the data
device VG (note that we don't have a way to create the meta in a
different VG in the first place, but at least we correctly handle a...

Prevent readding of the master node

This breaks Ganeti in multiple ways. If we don't make the check in
gnt-node itself, then bootstrap.SetupNodeDaemon will restart the
master daemon, making the operation fail:

node1# gnt-node add --readd node1
  Cannot communicate with the master daemon....

Fix potential data-loss in utils.WriteFile

os.write can do incomplete writes, as long as at least some bytes have
been written (like write(2)):

os.write(fd, " " * 1300)

1300

os.write(fd, " " * 1300)

1300

os.write(fd, " " * 1300)

1300

os.write(fd, " " * 1300)...

RAPI: Add support for tagging node groups

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

gnt-group: Add commands for tagging groups

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Quote filename in gnt-instance.8

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

TLMigrateInstance: remove 10s sleeps

TLMigrateInstance._ExecMigration contains two 10-second sleeps between
individual migration steps.

Apart from prolonging the migration duration by 20s, the second sleep
causes FinalizeMigration to be called 10 seconds after the real...

gnt-instance info: automatically request locking

Commit dae661a4 added support for controlling the locking, but it
didn't modify the gnt-instance info code, which leads to this command
always showing:

Wed Apr 20 04:10:48 2011 - WARNING: Non-static data requested, locks...

Fix master IP activation in failover with no-voting

Thanks to net.for.hub@gmail.com for reporting this. The logic in
masterd.CheckMasterd did an early return in case of no_voting, hence
skipping the master IP activation. We just change the ifs to not
return but simply continue through the function....

Update manpages and other documents with editor settings

No rewrapping is done in this patch, just updates to the settings.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

gnt-node list: Query filter support

Update manpage, quote field names.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix bug in watcher

If “utils.RunParts” were to raise an exception, a log message was
written and the code continued to run. Due to the exception the
“results” variable would not be defined.

Also change the code to log a backtrace (getting an exception is rather...

cli: Add option to force names to be treated as filter

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

qlang: Add function to distinguish filters from names

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

qlang: Add parser for query filter language

With this parser, command line utilities will be able to provide filters
through query2 in a simplistic language. Example filters:

name  "node3.example.com" 
  master or (name  "node4.example.com")
  be/memory == 128 and name =~ /^web/i...

htools: make some error messages more explicit

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

QA: also run gnt-cluster repair-disk-sizes

So that we don't happen again to break this forever without realising
it.

The patch also replaces one ' with ".

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fix shared_file_storage_dir on upgrades

If the cluster was upgraded from 2.4 or earlier, this key won't exist
(it's only set to a correct value on cluster init), so we need to
properly set it to a null string (disabled).

Signed-off-by: Iustin Pop <iustin@google.com>...

Prevent ssconf values from having non-string values

For whatever reason, my test cluster managed to acquire
shared_file_storage_dir with a None value, instead of empty
string. This is not flagged in masterd itself, but the node daemon
will fail in writing the value to disk, as it calls len() on the...

Node operations: take into account auto_balance

This patch changes the add to secondary/remove from secondary code to
not deduct/add the instance's memory if the instance is not
auto_balanced.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Read auto_balance via Rapi

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Show the auto_balance flag in the instance listing

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

utils.WriteFile: Close file before renaming

Issue 154 (http://code.google.com/p/ganeti/issues/detail?id=154)
reported an “Operation not supported” error when writing instance
exports to a mounted CIFS filesystem. Experimentation showed the error
to only occur when using rename(2) on an opened file. Various references...

Nicer formatting for group query error

Before this patc the message would look like “Some groups do not exist:
[u'foo', u'bar']”, now it's “Some groups do not exist: foo, bar”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Some more changes to Makefile.am for htools

I duplicate the BINARY= rule in the ghc invocation in order to be able
to silence the if, which was confusing.

Additionally, a new target for running just the htools unit-tests is
provided.

Signed-off-by: Iustin Pop <iustin@google.com>...

Update version in README

Also add a check to Makefile's check-local target.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Merge branch 'devel-2.4'

• devel-2.4:
  LUInstanceQueryData: Don't acquire locks unless requested
  Increase the lock timeouts before we block-acquire
  daemon.py: move startup log message before prep_fn
  Display the actual memory values in N+1 failures...

OpCodes.hs: make allow_failover optional

And default to False, like in the Python codebase.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Two small Makefile fixes related to htools

First, fix hs-coverage on non-pristine tree, where the index.html file
already existed, and second, disallow compilation of htools binaries
if configure, for some reason, didn't enable them.

Signed-off-by: Iustin Pop <iustin@google.com>...

LUInstanceQueryData: Don't acquire locks unless requested

Until now LUInstanceQueryData always acquired locks for the instance(s)
and nodes involved. In combination with long-running operations this
prevented the use of “gnt-instance info”, even with the “--static”...

TLMigrateInstance: Merge failover code, allow fallback

As the code for failover for checking is almost identical it's an easy
task to switch it over to the TLMigrateInstance. This allows us to
fallback to failover if migrate fails prereq check for some reason....

utils: Add function generating regex for DNS name globbing

The intent of this function is to be able to provide a globbing operator
or query filters. One should be able to say, for example, something to
the effect of “gnt-instance shutdown '*.site'”.

Also rename a variable in MatchNameComponent....

cmdlib: Factorize computation of ancillary files

… and change the logic in _RedistributeAncillaryFiles. The virtually
same list of files will be used to verify the files' consistency.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

query: Add implementation of regex match operator

So far this operator was not implemented. This patch adds an additional
value preparation function to the function table for binary operators,
used to compile the regular expression. Unittests are included....