Synnefo » snf-ganeti

===============================

Huge Pages Support for Ganeti

===============================

This is a design document about implementing support for huge pages in

Ganeti. (Please note that Ganeti works with Transparent Huge Pages i.e.

THP and any reference in this document to Huge Pages refers to explicit

Huge Pages).

Current State and Shortcomings:

-------------------------------

The Linux kernel allows using pages of larger size by setting aside a

portion of the memory. Using larger page size may enhance the

performance of applications that require a lot of memory by improving

page hits. To use huge pages, memory has to be reserved beforehand. This

portion of memory is subtracted from free memory and is considered as in

use. Currently Ganeti cannot take proper advantage of huge pages. On a

node, if huge pages are reserved and are available to fulfill the VM

request, Ganeti fails to recognize huge pages and considers the memory

reserved for huge pages as used memory.  This leads to failure of

launching VMs on a node where memory is available in the form of huge

pages rather than normal pages.

Proposed Changes:

-----------------

The following components will be changed in order for Ganeti to take

advantage of Huge Pages.

Hypervisor Parameters:

----------------------

Currently, It is possible to set or modify huge pages mount point at

cluster level via the hypervisor parameter ``mem_path`` as::

	$ gnt-cluster init \

	>--enabled-hypervisors=kvm -nic-parameters link=br100 \

	> -H kvm:mem_path=/mount/point/for/hugepages

This hypervisor parameter is inherited by all the instances as

default although it can be overriden at the instance level.

The following changes will be made to the inheritence behaviour.

-  The hypervisor parameter   ``mem_path`` and all other hypervisor

   parameters will be made available at the node group level (in

   addition to the cluster level), so that users can set defaults for

   the node group::

	$ gnt-group add/modify\

	> -H hv:parameter=value

   This changes the hypervisor inheritence level as::

     cluster -> group -> OS -> instance

-  Furthermore, the hypervisor parameter ``mem_path`` will be changeable

   only at the cluster or node group level and users must not be able to

   override this at OS or instance level. The following command must

   produce an error message that ``mem_path`` may only be set at either

   the cluster or the node group level::

	$ gnt-instance add -H kvm:mem_path=/mount/point/for/hugepages

Memory Pools:

-------------

Memory management of Ganeti will be improved by creating separate pools

for memory used by the node itself, memory used by the hypervisor and

the memory reserved for huge pages as:

- mtotal/xen (Xen memory)

- mfree/xen (Xen unused memory)

- mtotal/hp (Memory reserved for Huge Pages)

- mfree/hp (Memory available from unused huge pages)

- mpgsize/hp (Size of a huge page)

mfree and mtotal will be changed to mean "the total and free memory for

the default method in this cluster/nodegroup". Note that the default

method depends both on the default hypervisor and its parameters.

iAllocator Changes:

-------------------

If huge pages are set as default for a cluster of node group, then

iAllocator must consider the huge pages memory on the nodes, as a

parameter when trying to find the best node for the VM.

Note that the iallocator will also be changed to use the correct

parameter depending on the cluster/group.

hbal Changes:

-------------

The cluster balancer (hbal) will be changed to use the default  memory

pool and  recognize memory reserved for huge pages when trying to

rebalance the cluster.

.. vim: set textwidth=72 :

.. Local Variables:

.. mode: rst

.. fill-column: 72

.. End: