Statistics
| Branch: | Tag: | Revision:

root / doc / design-hugepages-support.rst @ 6d67e8bf

History | View | Annotate | Download (3.7 kB)

1
===============================
2
Huge Pages Support for Ganeti
3
===============================
4
This is a design document about implementing support for huge pages in
5
Ganeti. (Please note that Ganeti works with Transparent Huge Pages i.e.
6
THP and any reference in this document to Huge Pages refers to explicit
7
Huge Pages).
8

    
9
Current State and Shortcomings:
10
-------------------------------
11
The Linux kernel allows using pages of larger size by setting aside a
12
portion of the memory. Using larger page size may enhance the
13
performance of applications that require a lot of memory by improving
14
page hits. To use huge pages, memory has to be reserved beforehand. This
15
portion of memory is subtracted from free memory and is considered as in
16
use. Currently Ganeti cannot take proper advantage of huge pages. On a
17
node, if huge pages are reserved and are available to fulfill the VM
18
request, Ganeti fails to recognize huge pages and considers the memory
19
reserved for huge pages as used memory.  This leads to failure of
20
launching VMs on a node where memory is available in the form of huge
21
pages rather than normal pages.
22

    
23
Proposed Changes:
24
-----------------
25
The following components will be changed in order for Ganeti to take
26
advantage of Huge Pages.
27

    
28
Hypervisor Parameters:
29
----------------------
30
Currently, It is possible to set or modify huge pages mount point at
31
cluster level via the hypervisor parameter ``mem_path`` as::
32

    
33
	$ gnt-cluster init --no-lvm-storage --no-drbd-storage \
34
	>--enabled-hypervisors=kvm -nic-parameters link=br100 \
35
	> -H kvm:mem_path=/mount/point/for/hugepages
36

    
37
This hypervisor parameter is inherited by all the instances as
38
default although it can be overriden at the instance level.
39

    
40
The following changes will be made to the inheritence behaviour.
41

    
42
-  The hypervisor parameter   ``mem_path`` and all other hypervisor
43
   parameters will be made available at the node group level (in
44
   addition to the cluster level), so that users can set defaults for
45
   the node group::
46

    
47
	$ gnt-group add/modify\
48
	> -H hv:parameter=value
49

    
50
   This changes the hypervisor inheritence level as::
51

    
52
     cluster -> group -> OS -> instance
53

    
54
-  Furthermore, the hypervisor parameter ``mem_path`` will be changeable
55
   only at the cluster or node group level and users must not be able to
56
   override this at OS or instance level. The following command must
57
   produce an error message that ``mem_path`` may only be set at either
58
   the cluster or the node group level::
59

    
60
	$ gnt-instance add -H kvm:mem_path=/mount/point/for/hugepages
61

    
62
Memory Pools:
63
-------------
64
Memory management of Ganeti will be improved by creating separate pools
65
for memory used by the node itself, memory used by the hypervisor and
66
the memory reserved for huge pages as:
67
- mtotal/xen (Xen memory)
68
- mfree/xen (Xen unused memory)
69
- mtotal/hp (Memory reserved for Huge Pages)
70
- mfree/hp (Memory available from unused huge pages)
71
- mpgsize/hp (Size of a huge page)
72

    
73
mfree and mtotal will be changed to mean "the total and free memory for
74
the default method in this cluster/nodegroup". Note that the default
75
method depends both on the default hypervisor and its parameters.
76

    
77
iAllocator Changes:
78
-------------------
79
If huge pages are set as default for a cluster of node group, then
80
iAllocator must consider the huge pages memory on the nodes, as a
81
parameter when trying to find the best node for the VM.
82
Note that the iallocator will also be changed to use the correct
83
parameter depending on the cluster/group.
84

    
85
hbal Changes:
86
-------------
87
The cluster balancer (hbal) will be changed to use the default  memory
88
pool and  recognize memory reserved for huge pages when trying to
89
rebalance the cluster.
90

    
91
.. vim: set textwidth=72 :
92
.. Local Variables:
93
.. mode: rst
94
.. fill-column: 72
95
.. End: