Synnefo » snf-ganeti

Add primary_ip_family to ssconf

Since this parameter will be used on all daemon startups, it needs to be
available on all nodes. This is achieved by querying it via ssconf. This
patch additionally adds a getter method to readily retrieve the primary
ip family from a ConfigWriter object....

cluster init: Write ssconf before noded starts

This change is needed as we will need to read the primary ip version
cluster parameter before we start the node daemon. The reason is that we
need to know in advance if we bind to the IPv4 or IPv6 any address....

Introduce lib/netutils.py

This patch moves network utility functions to a dedicated module.

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Generalize a recursive check on logical disks

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix _UpgradeConfig docstring

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add uid_pool to ssconf

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Distribute list of enabled hypervisors in ssconf

This can be used by nodes to know which hypervisors they are supposed to
support.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Implement verify checks for node/instance names

Since we index the nodes and instances by their name, we should have
checks that the dict key to object.name mapping is correct.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Switch from os.path.join to utils.PathJoin

This passes a full burnin with lots of instances, and should be safe as
we mostly to join a known root (various constants) to a run-time
variable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Remove many 'Unused variable' warnings

Note there are some cases left which need extra cleanup.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Re-add check for duplicate instance IP

This was originally implemented in 0ce8f948 and partially
rolled back in 9b65e0d4. Apart from re-adding the check,
this patch does some housekeeping by renaming the “_helper”
function to “_AddIpAddress”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>...

ConfigWriter: move _temporary_ids to reservation

In order to do this we need to pass a job id when reserving a resource.
We have one during _EnsureUUIDs because we passed it in from AddNode and
AddInstance. During config upgrade we use a fake job ID which we then...

ConfigWriter: simplify GenerateDRBDSecret

We can do this by adding a new TemporaryReservationManager

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Remove exceptions list from GenerateUniqueID

It's not used anywhere, so it's dead code.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

config.Add{Node,Instance}: get the ec id

This is ok because adding a node or instance cannot happen in a query.

We get the ec id from the LU and pass it to _EnsureUUID, which will
then for now not use it.

Signed-off-by: Guido Trotter <ultrotter@google.com>...

Add a configuration verify check for duplicate IPs

This patch adds a check that the cluster IP, the nodes primary (and
secondary, if enabled) IP and the instances NIC IPs are unique in the
cluster.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Change behaviour of ConfigWriter._WriteConfig

This patch changes the behaviour of _WriteConfig in case of
configuration errors:

- before, it used to abort the saving (even though the in-memory
configuration used by current jobs has already changed)
- now, we log it (both to the log and to the user) but continue, since...

Make bootstrap._GenerateSelfSignedSslCert public

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix master candidate removal

Currently during a master candidate removal, when it's possible to
promote another node, the removal operation fails because of a corrupt
config before it's even possible to do the promotion. Fixing this by
doing the promotion before, excluding the current node....

Add uuid on node/instance add and cluster init

This patch does a little bit of cleanup first, since we want to call
GenerateUniqueID without reacquiring the lock.

Note that we don't necessarily need to do this for the cluster, since at
first startup ConfigWriter will do it anyway. But it's better to...

Automatically cleanup _temporary_ids at save

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Change config upgrade to be explicit

Currently the config upgrade is done at each object instantiation, that
means that ganeti-noded will run UpgradeConfig on all objects received
remotely (instances, disks, nics). This is not so good, so this patch
changes it so that only the ConfigWriter runs this method at...

Use ReadFile/WriteFile in more places

This survived QA, burnin and unittests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Ssconf IP lists: change separator

Use " " instead of ",".

Signed-off-by: Luca Bigliardi <shammash@google.com>

Add master candidates IPs informations to ssconf

This will be used when querying confd, in order not to rely on DNS being
available.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Merge branch 'next' into branch-2.1

• next:
  Fix another issue with hypervisor_name change
  Update NEWS and version for 2.0.2 release
  Improve the description of node flags in man page
  Add enabled hypervisors to TestConfigRunner
  Add a few more checks to verify config...

Get rid of the default_hypervisor slot

Currently we have both a default_hypervisor and an enabled_hypervisors
list. The former is only settable at cluster init time, while the latter
can be changed with cluster modify.

This becomes cumbersome in a few ways: at cluster init time for example...

Merge branch 'next' into branch-2.1

• next:
  Create a new --no-voting option for masterfailover
  ganeti-masterd: allow non-interactive --no-voting
  Fix pylint warnings
  Add custom pylintrc
  bootstrap: Don't leak file descriptor when generating SSL certificate...

Fix some typos

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Convert write_ssconf_files to new style

The patch also adds logging of errors from the ConfigWriter in case the
RPC fails (although today we don't have failure modes).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix ConfigWriter._DistributeConfig error handling

Commit 1b54fc6c2fd92b614c029cc5312d02f9f1526253 added a typo (due to
copy-paste) in the error handling path of this function.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add a new ssconf file with the cluster tags

Since the cluster tags are/should be more-or-less static, add them as an
ssconf key, so that querying them is possible without creating a
job/requiring the masterd to be running.

Reviewed-by: imsnah

ConfigWriter.AddInstance check instance mac

There is a race condition in CreateInstance, since the mac address is
generated early and only added to the config (and thus really assured to
be unique) only at this point. Since it's possible that another instance...

Prevent race condition on MAC addresses

This patch adds a temporary set for MACs that have been requested but
are not yet in the configuration (as part of an instance NIC). The MACs
of an instance are automatically removed from this set when the instance...

ConfigWriter: handle the drained node flag

This patch changes the master candidate pool computations in
ConfigWriter to properly handle drained nodes. They are now excluded
from counting towards the reachable number of candidates.

The patch also adds verification of consistency for the node status....

Implement disk verify checks in config verify

This patch adds a simple check that the 'mode' attribute of top-level disks is
correct. It does not recurse over children.

The framework could be extended with other checks in the future.

Reviewed-by: imsnah

Introduce more configuration consistency checks

This patch enhances the duplicate DRBD minors checks (currently just a
few) and adds automatic checks of configuration consistency at
configuration file writing time.

In order to do so and show meaningful error messages, the...

Change the instance status attribute to boolean

Due to historic reasons, the “should run or not” attribute of an
instance was denoted by its “status” attribute having a string value of
either ‘up’ or ‘down’. Checking this is in code was done via hardcoding...

Make cluster-verify check the drbd minors space

This patch adds support for verification of drbd minors space in cluster
verify: minors which belong to running instances and should be online
but are not, and minors which do not belong to any instace but are in...

Fix gnt-os for offline nodes

We shouldn't query offline nodes in gnt-os. This patch adds an utility
function to ConfigWriter that returns the names of online nodes and uses
it in LUDiagnoseOS to query only the good nodes.

Reviewed-by: imsnah

Fix epydoc format warnings

This patch should fix all outstanding epydoc parsing errors; as such, we
switch epydoc into verbose mode so that any new errors will be visible.

Reviewed-by: imsnah

Add function to compute the master candidates

Since some nodes can be offline, we can't just take the length of the
node list as the maximum possible number of master candidates.

The patch adds an utility function to correctly compute this value and
replaces hardcoded computations with the use of this function. It then...

A few fixes related to master candidates

This patch:
- fixes cluster verify when all nodes are master candidates, but the
candidate_pool_size is higher
- warn when the master node is not marked as candidate
- disable setting master node to regular node...

Add the list of master candidates to ssconf

Reviewed-by: imsnah

Replicate the node list in ssconf

This patch adds node_list in the list of replicated values from
ConfigWriter.

Reviewed-by: imsnah

ConfigWriter: change cluster serial meaning

Currently, we increase the cluster serial number for instance additions,
removals and renames. This is conforming with the REST paradigm, however
it means that for each of these operations, we need to push ssconf...

Fix unittests broken by rev 2015

Ssconf files shouldn't be updated when running unittests.

Reviewed-by: ultrotter

Write ssconf files when updating configuration

Reviewed-by: ultrotter

Convert rpc.call_upload_file to use addresses

This patch allows rpc.call_upload_file to use addresses (if passed), and
also converts the ConfigWriter._DistributeConfig to pass them, since
this is an often-done operation.

Reviewed-by: imsnah

Disable re-reading of config file

Since the objects read from the config file are passed to the various
threads, it's unsafe to re-read the config file (and throw away
ConfigWriter._config_data). As such, we disable the re-reading of the
file (since now the master is the owner the file, it makes not sense to...