Synnefo » snf-ganeti

Removing all ssh setup code from the core

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Support IPv6 cluster init

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add new cluster parameter primary_ip_version

We expose the ip_version (4, 6) to the external interface and internally
we convert it to ip_family (AF_INET=2, AF_INET6=10). This makes the code
more concise as all functions deal with family rather than version....

Merge branch 'devel-2.2'

• devel-2.2:
  RAPI client: Support modifying instances
  RAPI: Allow modifying instance
  Small fixes for instance creation via RAPI documentation
  gnt-debug: Extend job queue tests
  jqueue: Mark opcodes following failed ones as failed, too...

Fix some small newline style issues

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

RAPI: Allow modifying instance

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

jqueue: Mark opcodes following failed ones as failed, too

When an opcode fails, the job queue would leave following opcodes as “queued”,
which can be quite confusing. With this patch, they're all marked as failed and
assertions are added to check this.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

rapi.client, http.client: Format url correctly when using IPv6

This patch moves the FormatAddress helper function from daemon.py to
netutils.py. This enables its use in http.client as well as in
rapi.client. Furthermore this adds functionality to format IPv6...

Support for resolving hostnames to IPv6 addresses

This patch enables IPv6 name resolution by using socket.getaddrinfo
instead of socket.gethostbyname_ex.

It renames the HostInfo class to Hostname and unifies its use throughout
the code. This is achieved by using static calls where no object is...

cluster init: Write ssconf before noded starts

This change is needed as we will need to read the primary ip version
cluster parameter before we start the node daemon. The reason is that we
need to know in advance if we bind to the IPv4 or IPv6 any address....

jqueue: More checks for cancelling queued job

We can also check when the lock status is updated. This will
improve job cancelling.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

jqueue: Add more debug output

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Re-add the 'live' parameter to migration opcodes

This patch reintroduces the live parameter, for backwards compatibility
at the Luxi level. This way, clients can work transparently with both
2.1 and 2.2, even though sub-optimally.

Signed-off-by: Iustin Pop <iustin@google.com>...

Revert "Add -usbdevice tablet to KVM when using vnc"

This reverts commit 5b062a58ac76b39c2dc6a7e1543affdf43dc7ee7.
This fixes issue 109. The mouse type can be set with the usb_mouse
hv parameter.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

cli.SubmitOpCode: Pass keyword parameter as keyword

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

Adjust message in case ~/.ssh is no directory

Use actual path, not something hardcoded.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

RAPI client: Support renaming instances

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

RAPI client: Don't re-use PycURL object

With this patch, a new PycURL object will be created for each request.
This should make the RAPI client safe for simultaneous calls from
multiple threads. Unittests are adjusted accordingly.

An unnecessary variable assignment is also removed from the unittest...

Allow instance NIC's IP address to be None

Also add some assertions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

cmdlib: Change expected type for source CA on remote import

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Change handling of non-Ganeti errors in jqueue

Currently, if a job execution raises a Ganeti-specific error (i.e.
subclass of GenericError), then we encode it as (error class, [error
args]). This matches the RAPI documentation.

However, if we get a non-Ganeti error, then we encode it as simply...

Add check for RAPI paths to start with /2

During a discussion in July 2010 it was decided that we'll stabilize on /2. See
message ID <20100716180012.GA9423@google.com> for reference.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

RAPI client: Support migrating instances

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

workerpool: Change signature of AddTask function to not use *args

By changing it to a normal parameter, which must be a sequence, we can
start using keyword parameters.

Before this patch all arguments to “AddTask(self, *args)” were passed as
arguments to the worker's “RunTask” method. Priorities, which should be...

workerpool: Move waiting for new tasks for a worker to the pool

This way fewer private variables of the pool are accesssed by the worker.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

workerpool: Additional check in BaseWorker.ShouldTerminate

Document that it should only be called from within RunTask and
add an assertion for this. This means we can no longer use a
method on the pool and hence remove WorkerPool.ShouldWorkerTerminate.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Rename masterfailover to master-failover

Most (all?) of our commands use dash-separator: replace-disks,
verify-disks, add-tags, etc. “gnt-cluster masterfailover” is an old
exception to this rule.

The patch replaces it with master-failover, add a compatiblity alias,...

Improve handling of lost jobs

Currently, if the cli.JobExecutor class is being used, and one of the
jobs is being archived before it can check its result, it will raise a
stracktrace as _ChooseJob is not prepared to handle this case.

This case makes JobExecutor work better with lost jobs (it still reports...

cmdlib: Return new name from rename operations

The new name is then displayed by the clients.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Manuel Franceschini <livewire@google.com>

Add modification of the reserved logical volumes

This doesn't allow addition/removal of individual volumes, only
wholesale replace of the entire list. It can be improved later, if we
ever get generic container parameters.

The man page changes replaces some tabs with spaces (hence the...

Introuce a new cluster parameter - reserved_lvs

This parameter, which is a list of regular expression patterns, will
make cluster verify ignore any such LVs. It will not prevent creation or
removal of such volumes by the backend code.

Signed-off-by: Iustin Pop <iustin@google.com>...

Move the UsesRPC decorator from cli to rpc

This is needed because not just the cli scripts need this decorator, but
the master daemon too (and it already duplicated the code once).

In cli.py we just leave a stub, so that we don't have to modify all the
scripts to import rpc.py....

Update the RAPI client for the migration mode

See the discussion on the previous patch about this. Basically unless we
want to a add a new 'feature' marking for the live migration parameter,
there is no simple way to handle this nicely in the client.

Given that the client was/is marked as experimental, this patch simply...

Rename migration type to migration mode

This is in preparation for the rename of the opcode 'live' parameter to
'mode'.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Merge branch 'devel-2.1' into master

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Always set commonName in X509 certificates

Due to the current switch of the RPC client to PycURL, a bug with newer
versions of libcurl surfaced. When the 'Subject' or 'Issuer' of
'server.pem' were empty, SSL handshake failed.

This patch changes the certificate generation functions such that they...

Change AddAuthorizedKey to also allow filehandles

This is required to use this function over paramiko
sftp file handles.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Introduce git reference/tag tracking for debugging

This patch adds a new vcs-version file that is generated via git (and
can be adapted if VCS is changed) and then embebbded as VCS_VERSION in
the constants module.

This means two things:
- local modifications without committing to git (or when using a tar.gz...

Convert RPC client to PycURL

Instead of using our custom HTTP client, using PycURL's multi
interface allows us to get rid of the HTTP client threadpool.
The majority of the code is still in the ganeti.http.client
module.

A simple per-thread HTTP client pool gives cURL a chance to...

Fix pylint complaints introduced in commit e58f87a958c

Due to a small mistake I missed three non-critical pylint complaints for
commit e58f87a958c. They're fixed with this patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>...

Add ParseCpuMask() utility function

Also adds a generic ParseError exception.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add test for some aspects of job queue

This new opcode and gnt-debug sub-command test some aspects of the
job queue, including the status of a job. The bug fixed in commit
2034c70d507 was identified using this test. A future patch will
run this test automatically from the QA scripts....

KVM hypervisor: Use utils.ShellWriter for network script

This patch converts hv_kvm to use utils.ShellWriter for writing
the network script. It also adds a few unittests (the first
for any hypervisor modules).

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

jqueue: Factorize code waiting for job changes

By splitting the _WaitForJobChangesHelper class into multiple smaller
classes, we gain in several places:

- Simpler code, less interaction between functions and variables
- Easy to unittest (close to 100% coverage)...

RAPI client: Implement old instance creation request format

Commit 8a47b4478 implemented instance creation in the RAPI client,
but it left out support for the old instance creation request format.
This patch now implements the old format as good as possible. This...

Use reserved documentation IPs and domains

Use RFC 5737 IP addresses and RFC 2606 domain names in all
unittests, docs, qa and docstrings.

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

jqueue: Setup inotify before checking for any job changes

Since the code waiting for job changes was modified to use inotify,
a race condition between checking for changes the first time and
setting up inotify occurs. If the job is modified after the check...

Add function to format all job log messages

Just calling utils.SafeEncode on the log message failed when it
wasn't of the type ELOG_MESSAGE and not a string. Now non-message
log entries are formatted using repr().

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Confd IPv6 support

This patch series basically adds a new parameter 'family' to the constructors
of daemon.AsyncUDPSocket and confd.client.ConfdUDPClient. This enables the
users of these two classes to support IPv6.

In ganeti-confd.ConfdAsyncUDPClient a method to check the address families of...

LXC: Fix GetInstanceInfo()

Don't try to get cgroups info if instance is not running.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>