Synnefo » snf-ganeti

Stop all daemons on cluster initialization

Addresses issue 145. When daemons are still running from a previous
cluster instance, starting the daemons again will fail.
“ConfigWriter.IsCluster” only looks at the configuration file, which
could already have been deleted....

bootstrap.RunNodeSetupCmd: Add IPv6 support

Commit 224ff0f modified the node SSH setup to use the system's SSH
client. Before that Paramiko was used. It's not entirely clear whehter
the latter ever supported IPv6 properly, but with this patch
“bootstrap.RunNodeSetupCmd” is changed to use it if configured. The code...

IP pool related objects, opcodes and constants

Config objects: * Introduce L{Network} with
- IPv4 network field (mandatory)
- IPv4 gateway, IPv6 (network/gateway), mac prefix, type (optional) * Modify existing config objects to support networks:
- Add new slot 'network' to L{NIC} config object...

Move a function from backend to ssconf

The “WriteSsconfFiles” function is used to write ssconf files. By moving
it we can avoid importing backend into bootstrap. The latter is imported
by CLI programs and backend doesn't have much to do with them.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

bootstrap.SetupNodeDaemon: Stop hardcoding bind address

Commit b43dcc5a11 added support for IPv6. To have the node daemon bind
to an IPv6 address on startup it changed the code to pass a static bind
address (0.0.0.0 or ::0). If a user had configured another bind address...

constants: Move most paths to separate module

This is inpreparation for the implementation of virtual clusters. Many
paths will change based on an environment variable and are no longer
constant and should no longer be in “constants.py”. Since “constants.py”...

Fix uses of OpPrereqError without code info

A while back, we did cleanup the code and ensured (manually) that use
of OpPrereqError includes an errors.ECODE_* field as second
argument. Since we cannot automate the check for this, it turns out
that more and more such usage has crept over the years, including in...

cmdlib: Remove all diskparams calculations not required anymore

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Replace single- with double-quotes

In at least two cases "%s" is replaced with str(), too.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add new disk_templates parameter to instance policy

This is a bit more complex patch, as it requires changing the
assumption that all keys in the policy dict points to values that are
themselves dicts. Right now we introduce an assumption that any
non-dicts are lists, we'll see in the future if this holds or whether...

InitCluster supports instance policy

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Remove BE_MEMORY from beparams but keep compatibility

Queries are already compatible (be/memory is an alias for be/maxmem) and
import/exports work. This patch patch fixes it for cluster init, modify
and instance add/start/modify.

Signed-off-by: Guido Trotter <ultrotter@google.com>...

Add --use-external-mip-script flag

- add a command line flag to gnt-cluster init and modify to change the
value of the cluster parameter use_external_mip_script;
- add two constants representing the paths of the default script and of
the external script;...

Use MasterNetworkParameters attributes for RPC

Instead of manually unpacking the return values of
cfg.GetMasterNetworkParameters, let it return an instance of
objects.MasterNetworkParameters and pass its attributes.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>...

Explicitly pass params to deactivate_master_ip

Make the master explicitly pass the parameters to the
deactivate_master_ip RPC, and change all the call flow to use the new
interface.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move bootstrap-related RPC to generated wrappers

With this patch, only 6 RPC are left as old-style code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add cluster netmask parameter

Add the master_netmask cluster parameter, that represents the netmask of
the master IP, encoded as a CIDR suffix.

This parameter can be set via the --master-netmask of gnt-cluster init
and gnt-cluster modify. The default behaviour is to be consistent with...

Implementation of TLS-protected SPICE connections

Added support for TLS-protected SPICE connections:

• added three new parameters to renew-crypto (--new-spice-certificate,
  --spice-certificate, --spice-ca-certificate) and their validation....

DeprecationWarning fixes for pylint

In version 0.21, pylint unified all the disable-* (and enable-*)
directives to disable (resp. enable). This leads to a lot of
DeprecationWarning being emitted even if one uses the recommended
version of pylint (0.21.1, as stated in devnotes.rst)....

remove bootstrap._InitSharedFileStorage

This function is a copy of bootstrap._InitFileStorage with the following
differences:
- check constants.ENABLE_SHARED_FILE_STORAGE and not
constants.ENABLE_FILE_STORAGE
- use different local variable names
- one different error string...

Shared file storage initialization code

Add shared file storage handling during cluster initialization.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

gnt-cluster master-failover: Undrain queue

- Move functions for drain status (tracked via file) from jqueue to jstore
- Undrain queue on master failover if necessary
- Add QA test

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Wait for master to become available on initialization

This is analogue to the existing check for a responsive node daemon.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Cleanup bootstrap.SetupNodeDaemon

- Code formatting
- Use ShellQuote for one argument
- Remove variables no longer used after commit 9294514d

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Adding ndparams to gnt-cluster init|modify and man page

This is reverting the revert and adaption to fix the issue which caused
the revert

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Revert "InitCluster support for ndparams"

This reverts commit fd7f5b411ca61be2e4eff638773ba6c0fac7eb32.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

ConfigWriter: prevent using a foreign config

If the configuration file doesn't denote this node as master, we prevent
startup. This would have detected our previous race condition more
easily, hence we add it as a permanent check.

Signed-off-by: Iustin Pop <iustin@google.com>...

Add prealloc_wipe_disks as a cluster-wide configuration variable

This is the first step for the support of wiping block devices prior
to creation of the instance.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

InitConfig: create nodegroups as well

This patch also ensures that the initial configuration has all the
needed UUIDs and that they are unique, by using a
TemporaryReservationManager inside InitConfit to generate them.

Signed-off-by: Guido Trotter <ultrotter@google.com>...

Switch to the RPC call to update /etc/hosts in LUAddNode and LURemoveNode

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Use family in backend.StartMaster

This patches changes the StartMaster method to consult the cluster
primary ip version when deciding whether to use arping or ndisc6 after
activating the master ip.

Signed-off-by: Manuel Franceschini <livewire@google.com>...

Add new cluster parameter primary_ip_version

We expose the ip_version (4, 6) to the external interface and internally
we convert it to ip_family (AF_INET=2, AF_INET6=10). This makes the code
more concise as all functions deal with family rather than version....

Support for resolving hostnames to IPv6 addresses

This patch enables IPv6 name resolution by using socket.getaddrinfo
instead of socket.gethostbyname_ex.

It renames the HostInfo class to Hostname and unifies its use throughout
the code. This is achieved by using static calls where no object is...

Introduce new IPAddress classes

This patch unifies the netutils functions dealing with IP addresses to
three classes:
- IPAddress: Common IP address functionality
- IPv4Address: IPv4 specific functionality
- IPv6address: IPv6-specific functionality

Furthermore it adds methods to check whether an address is a loopback...

Introduce lib/netutils.py

This patch moves network utility functions to a dedicated module.

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add a delay in master failover

I have seen some very seldom errors where (it seems) the address is
still live for a short while after removing it from the old master, thus
the new master will fail in startup/adding its own IP address.

To prevent against this, we add a delay/retry before we proceed, if the...

Introduce utils.IsValidIP{4,6}()

This patch introduces functions to check for valid IPv4 and IPv6
addresses and converts IsValidIP() to return True if it is either a IPv4
or a IPv6 address.

For now we do not change the functional behavior and replace IsValidIP...

Merge branch 'devel-2.1'

Conflicts:
doc/security.rst
trivial
lib/cli.py
trivial

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Merge branch 'devel-2.1'

• devel-2.1:
  utils: Add class to split string stream into lines
  Fix cluster behaviour with disabled file storage
  Update docstrings in tools/ and enable epydoc
  Forward-port the ganeti 2.0 cfgupgrade
  Add a new tool: sanitize-config...

Fix cfgupgrade with non-default DATA_DIR

Commit 43575108 added bootstrap.GenerateclusterCrypto and commit
7506a7f1 changed cfgupgrade to use it. However, this lost the
functionality of upgrading in non-default DATA_DIR.

To fix this, we enhance bootstrap.GenerateclusterCrypto to accept custom...

Add a new cluster parameter maintain_node_health

This will be used to conditionally enable the watcher node maintenance
feature.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Merge remote branch 'origin/devel-2.1'

• origin/devel-2.1:
  Rightname confd's HMAC key
  Rename SSL_CERT_FILE to NODED_CERT_FILE
  Clarify the error message for ':' in PV names

Conflicts:
lib/bootstrap.py: Trivial
lib/constants.py: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Rightname confd's HMAC key

Currently, the ganeti-confd's HMAC key is called “cluster HMAC key” or
simply “HMAC key” everywhere. With the implementation of inter-cluster
instance moves, another HMAC key will be introduced for signing critical
data. They can not be the same, so this patch clarifies the purpose of the...

bootstrap: Add new function to create cluster certs and keys

The code to generate cluster certificates, keys and secrets is currently
spread over several places. It makes sense to move it to a separate
function as we want to provide the user with the ability to automatically...

bootstrap: Wait for node daemon when adding new node

Until now this was only done for the master node, though
the problem originally fixed in 8f215968 also occurs for
other node daemons.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Generate hmac file with a newline at the end

This makes it slightly easier to cut&paste its content.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Introduce a wrapper for hostname resolving

Currently a few of the LU's CheckPrereq use utils.HostInfo which raises
a resolver error in case of failure. This is an exception from the
standard that CheckPrereq should raise an OpPrereqError if the error is
in the 'pre' phase (so that it can be retried)....

bootstrap: Convert to utils.Retry

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Make cluster initialization more reliable

There was a race condition between starting the node daemon
and sending requests to write the ssconf files. With this
patch, the initialization waits up to ten seconds for the
node daemon to become responsive.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Adding '--no-ssh-init' option to 'gnt-cluster init'.

Allows the initialization of a cluster without the creation or distribution
of SSH key pairs. Includes changes for LeaveCluster and RPC.

Signed-off-by: Ken Wehr <ksw@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>...