Synnefo » snf-ganeti

Remove ConfigWriter.InitConfig

It's been replaced by a simpler bootstrap.InitConfig function, which
does the same job, and is currently unused.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix some typos

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add a new ssconf file with the cluster tags

Since the cluster tags are/should be more-or-less static, add them as an
ssconf key, so that querying them is possible without creating a
job/requiring the masterd to be running.

Reviewed-by: imsnah

ConfigWriter.AddInstance check instance mac

There is a race condition in CreateInstance, since the mac address is
generated early and only added to the config (and thus really assured to
be unique) only at this point. Since it's possible that another instance...

Prevent race condition on MAC addresses

This patch adds a temporary set for MACs that have been requested but
are not yet in the configuration (as part of an instance NIC). The MACs
of an instance are automatically removed from this set when the instance...

ConfigWriter: handle the drained node flag

This patch changes the master candidate pool computations in
ConfigWriter to properly handle drained nodes. They are now excluded
from counting towards the reachable number of candidates.

The patch also adds verification of consistency for the node status....

Implement disk verify checks in config verify

This patch adds a simple check that the 'mode' attribute of top-level disks is
correct. It does not recurse over children.

The framework could be extended with other checks in the future.

Reviewed-by: imsnah

Introduce more configuration consistency checks

This patch enhances the duplicate DRBD minors checks (currently just a
few) and adds automatic checks of configuration consistency at
configuration file writing time.

In order to do so and show meaningful error messages, the...

Change the instance status attribute to boolean

Due to historic reasons, the “should run or not” attribute of an
instance was denoted by its “status” attribute having a string value of
either ‘up’ or ‘down’. Checking this is in code was done via hardcoding...

Make cluster-verify check the drbd minors space

This patch adds support for verification of drbd minors space in cluster
verify: minors which belong to running instances and should be online
but are not, and minors which do not belong to any instace but are in...

Fix gnt-os for offline nodes

We shouldn't query offline nodes in gnt-os. This patch adds an utility
function to ConfigWriter that returns the names of online nodes and uses
it in LUDiagnoseOS to query only the good nodes.

Reviewed-by: imsnah

Fix epydoc format warnings

This patch should fix all outstanding epydoc parsing errors; as such, we
switch epydoc into verbose mode so that any new errors will be visible.

Reviewed-by: imsnah

Add function to compute the master candidates

Since some nodes can be offline, we can't just take the length of the
node list as the maximum possible number of master candidates.

The patch adds an utility function to correctly compute this value and
replaces hardcoded computations with the use of this function. It then...

A few fixes related to master candidates

This patch:
- fixes cluster verify when all nodes are master candidates, but the
candidate_pool_size is higher
- warn when the master node is not marked as candidate
- disable setting master node to regular node...

Add the list of master candidates to ssconf

Reviewed-by: imsnah

Replicate the node list in ssconf

This patch adds node_list in the list of replicated values from
ConfigWriter.

Reviewed-by: imsnah

ConfigWriter: change cluster serial meaning

Currently, we increase the cluster serial number for instance additions,
removals and renames. This is conforming with the REST paradigm, however
it means that for each of these operations, we need to push ssconf...

Fix unittests broken by rev 2015

Ssconf files shouldn't be updated when running unittests.

Reviewed-by: ultrotter

Write ssconf files when updating configuration

Reviewed-by: ultrotter

Convert rpc.call_upload_file to use addresses

This patch allows rpc.call_upload_file to use addresses (if passed), and
also converts the ConfigWriter._DistributeConfig to pass them, since
this is an often-done operation.

Reviewed-by: imsnah

Disable re-reading of config file

Since the objects read from the config file are passed to the various
threads, it's unsafe to re-read the config file (and throw away
ConfigWriter._config_data). As such, we disable the re-reading of the
file (since now the master is the owner the file, it makes not sense to...

Add new query to get cluster config values

This can be used to retrieve certain cluster config values from
within clients.

OpDumpClusterConfig was not used anywhere, hence I'm just reusing
it. The way ConfigWriter.DumpConfig returned the configuration
was not thread-safe, anyway (no deepcopy)....

Add cluster options from ssconf to configuration

ssconf will become write-only from ganeti-masterd's point of view,
therefore all settings in there need to go into the main configuration
file.

Reviewed-by: iustinp

Extend DRBD disks with shared secret attribute

This patch, which is similar to r1679 (Extend DRBD disks with minors
attribute), extends the logical and physical id of the DRBD disks with a
shared secret attribute. This is generated at disk creation time and...

Add checks for tcp/udp port collisions

In case the config file is manually modified, or in case of bugs, the
tcp/udp ports could be reused, which will create various problems
(instances not able to start, or drbd disks not able to communicate).

This patch extends the ConfigWriter.VerifyConfig() method (which is used...

Initialize and update the serial_no on objects

This patch add initialization of the serial_no on instance and nodes,
and update of the field whenever an object is updated in the generic
case, via ConfigWriter.Update(obj) and in the specific case of
instances' state being modified manually....

Switch config.py to logging

A couple of more modules are using the obsolete logger functions, config
being one of them.

Reviewed-by: imsnah

Extend DRBD disks with minors attribute

This patch converts the DRBD disks to contain also a minor (per each
node) attribute. This minor is not yet used and is always initialized
with None, so the patch does not have any real-world impact - except for
automatically upgrading config files (it adds the minors as None, None)....

Add an atomic ConfigWrite.GetAllInstanceInfo()

In order to be able to query instance without locking them, we need the
same atomic query of multiple instances as for nodes.

Reviewed-by: ultrotter

Add a way to export all node information at once

The patch adds a new function to export all node information at once
(i.e. atomically with respect to the configuration lock).

Reviewed-by: ultrotter

Fix pylint-detected issues

This is mostly:
- whitespace fix (space at EOL in some files, not all, broken
indentation, etc)
- variable names overriding others (one is a real bug in there)
- too-long-lines
- cleanup of most unused imports (not all)...

ConfigWriter: remove _ReleaseLock

Remove empty function _ReleaseLock and all its calls. Since we only
have one configwriter per cluster the locking needs to cover all the
data in the object, and not just the file contents. Locking in
ConfigWriter will be handled using the ganeti locking library....

Wrap exception in _DistributeConfig code

nodelist.remove(X) could potentially raise a ValueError (even if the chance
that the current node is not in the list are pretty scarce, and its absence
should raise a red flag anyway). If this happens let things go on, as that's...

Only change instance status if not already so

This patch makes the _SetInstanceStatus only change the status of the
instance (in the config file) if it's not already in the desired state.
This skips the increase of the config serial no without actual data...

Abstract the json functions into a separate module

This simple patch adds a new module that holds the simplejson functions
for serialization/deserialization. This reduces the amount of redundant
code.

The patch also adds some normalizations to the json output:...

Fix SetVGName() to access object not dict

Reviewed-by: imsnah

Allow instance MAC address to be set.

Allow the MAC address of an instance to be specified optionally during
instance creation and later to be changed via instance modify.

Reviewed-by: iustinp

Rename utils.GetUUID to utils.NewUUID.

Reviewed-by: schreiberal

Abstract more strings values into constants

Currently, the disk types are defined using constants in the code.
Convert those into constants so that we can easily find them and check
their usage.

Note that we don't rename the values of the constants as they are used...

Use the kernel's ability to generate UUIDs.

This removes the dependency on either the uuid module or e2fsprogs' uuidgen.

Reviewed-by: iustinp

Revert a debugging 'raise' from rev 195

As far as I understand, this was just a debugging aid. Remove it so that
configuration errors are nicely handled.

Reviewed-by: imsnah

Remove requirement that host names are FQDN

We currently require that hostnames are FQDN not short names
(node1.example.com instead of node1). We can allow short names as long
as:
- we always resolve the names as returned by socket.gethostname()
- we rely on having a working resolver...

Style changes for pep-8 and python-3000 compliance.

This changes the raising of exceptions from:
raise Exception, value
to
raise Exception(value)

as the first form will be removed in python-3000 and the second form is
preferred now.

The changes also involve a few cases of changing from raising standard...

Re-stat the config file when we write it so that the next _OpenConfig()
won't reread it if not modified by someone else.

Change logival volume names to not be based on the instance's name, but
instead use an UUID prefix and a suffix denoting the disk iv_name
(sda/sdb) and possibly it's type (data/meta).

Reviewed-by: imsnah

Don't pass master_node

Reviewed-by: iustinp

The master_node is no longer stored in the config file (but in sstore)
and all users have been converted, let's remove it from the slots of the
cluster object.

Also let's move the tcpudp_port_pool to the cluster object.

Reviewed-by: imsnah

Big change/cleanup in relation to the master startup:
- move the master node name from the ConfigWriter to SimpleStore (all
nodes need this, and it was the only thing pulled in from the
ConfigWriter on nodes)
- fix mcpu.py and the testing w.r.t. this change; for testing, rename...