Synnefo » snf-ganeti

============================

Storage free space reporting

============================

.. contents:: :depth: 4

Background

==========

Currently Space reporting is broken for all storage types except drbd or

lvm (plain). This design looks at the root causes and proposes a way to

fix it.

Proposed changes

================

The changes below will streamline Ganeti to properly support

interaction with different storage types.

Configuration changes

---------------------

Add a new attribute "enabled_storage_methods" (type: list of strings) to the

cluster config which holds the types of storages, for example, "plain", "drbd",

or "ext". We consider the first one of the list as the default method.

For file storage, we'll report the storage space on the file storage dir,

which is currently limited to one directory. In the future, if we'll have

support for more directories, or for per-nodegroup directories this can be

changed.

Note that the abovementioned enabled_storage_methods are just "mechanisms"

parameters that define which storage methods the cluster can use. Further

filtering about what's allowed can go in the ipolicy, but these changes are

not covered in this design doc.

Since the ipolicy currently has a list of enabled storage types, we'll

use that to decide which storage type is the default, and to self-select

it for new instance creations, and reporting.

Enabling/disabling of storage types at ``./configure`` time will be

eventually removed.

RPC changes

-----------

The noded RPC call that reports node storage space will be changed to

accept a list of <method>,<key> string tuples. For each of them, it will

report the free amount of storage space found on storage <key> as known

by the requested storage type method. For example methods are ``lvm``,

``filesystem``, or ``rados``, and the key would be a volume group name, in

the case of lvm, a directory name for the filesystem and a rados pool name

for rados storage.

For now, we will implement only the storage reporting for non-shared storage,

that is ``filesystem`` and ``lvm``. For shared storage methods like ``rados``

and ``ext`` we will not implement a free space calculation, because it does

not make sense to query each node for the free space of a commonly used

storage.

Masterd will know (through a constant map) which storage type uses which

method for storage calculation (i.e. ``plain`` and ``drbd`` use ``lvm``,

``file`` uses ``filesystem``, etc) and query the one needed (or all of the

needed ones).

Note that for file and sharedfile the node knows which directories are

allowed and won't allow any other directory to be queried for security

reasons. The actual path still needs to be passed to distinguish the

two, as the method will be the same for both.

These calculations will be implemented in the node storage system

(currently lib/storage.py) but querying will still happen through the

``node info`` call, to avoid requiring an extra RPC each time.

Ganeti reporting

----------------

`gnt-node list`` can be queried for the different storage methods, if they

are enabled. By default, it will just report information about the default

storage method. Examples::

  > gnt-node list

  Node                       DTotal DFree MTotal MNode MFree Pinst Sinst

  mynode1                      3.6T  3.6T  64.0G 1023M 62.2G     1     0

  mynode2                      3.6T  3.6T  64.0G 1023M 62.0G     2     1

  mynode3                      3.6T  3.6T  64.0G 1023M 62.3G     0     2

  > gnt-node list -o dtotal/lvm,dfree/rados

  Node      DTotal (Lvm, myvg) DFree (Rados, myrados)

  mynode1                 3.6T                      -

  mynode2                 3.6T                      -

Note that for drbd, we only report the space of the vg and only if it was not

renamed to something different than the default volume group name. With this

design, there is also no possibility to ask about the meta volume group. We

restrict the design here to make the transition to storage pools easier (as it

is an interim state only). It is the administrator's responsibility to ensure

that there is enough space for the meta volume group.

When storage pools are implemented, we switch from referencing the storage

method to referencing the storage pool name. For that, of course, the pool

names need to be unique over all storage methods. For drbd, we will use the

default 'lvm' storage pool and possibly a second lvm-based storage pool for

the metavg. It will be possible to rename storage pools (thus also the default

lvm storage pool). There will be new functionality to ask about what storage

pools are available and of what type.

``gnt-cluster info`` will report which storage methods are enabled, i.e.

which ones are supported according to the cluster configuration. Example

output::

  > gnt-cluster info

  [...]

  Cluster parameters:

    - [...]

    - enabled storage methods: plain (default), drbd, lvm, rados

    - [...]

``gnt-node list-storage`` will not be affected by any changes, since this design

describes only free storage reporting for non-shared storage methods.

Allocator changes

-----------------

The iallocator protocol doesn't need to change: since we know which

storage type an instance has, we'll pass only the "free" value for that

storage type to the iallocator, when asking for an allocation to be

made. Note that for DRBD nowadays we ignore the case when vg and metavg

are different, and we only consider the main VG. Fixing this is outside

the scope of this design.

With this design, we ensure forward-compatibility with respect to storage

pools. For now, we'll report space for all available (non-shared) storage

types, in the future, for all available storage pools.

Rebalancing changes

-------------------

Hbal will not need changes, as it handles it already. We don't forecast

any changes needed to it.

Space reporting changes

-----------------------

Hspace will by default report by assuming the allocation will happen on

the default storage for the cluster/nodegroup. An option will be added

to manually specify a different storage.

Interactions with Partitioned Ganeti

------------------------------------

Also the design for :doc:`Partitioned Ganeti <design-partitioned>` deals

with reporting free space. Partitioned Ganeti has a different way to

report free space for LVM on nodes where the ``exclusive_storage`` flag

is set. That doesn't interact directly with this design, as the specific

of how the free space is computed is not in the scope of this design.

But the ``node info`` call contains the value of the

``exclusive_storage`` flag, which is currently only meaningful for the

LVM back-end. Additional flags like the ``external_storage`` flag

for lvm might be useful for other storage types as well. We therefore

extend the RPC call with <method>,<key> to <method>,<key>,<params> to

include any storage-method specific parameters in the RPC call.

The reporting of free spindles, also part of Partitioned Ganeti, is not

concerned with this design doc, as those are seen as a separate resource.

.. vim: set textwidth=72 :

.. Local Variables:

.. mode: rst

.. fill-column: 72

.. End: