1 Ganeti automatic instance allocation
2 ====================================
4 Documents Ganeti version 2.1
11 Currently in Ganeti the admin has to specify the exact locations for
12 an instance's node(s). This prevents a completely automatic node
13 evacuation, and is in general a nuisance.
15 The *iallocator* framework will enable automatic placement via
16 external scripts, which allows customization of the cluster layout per
17 the site's requirements.
22 There are two parts of the ganeti operation that are impacted by the
23 auto-allocation: how the cluster knows what the allocator algorithms
24 are and how the admin uses these in creating instances.
26 An allocation algorithm is just the filename of a program installed in
27 a defined list of directories.
32 At configure time, the list of the directories can be selected via the
33 ``--with-iallocator-search-path=LIST`` option, where *LIST* is a
34 comma-separated list of directories. If not given, this defaults to
35 ``$libdir/ganeti/iallocators``, i.e. for an installation under
36 ``/usr``, this will be ``/usr/lib/ganeti/iallocators``.
38 Ganeti will then search for allocator script in the configured list,
39 using the first one whose filename matches the one given by the user.
41 Command line interface changes
42 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
44 The node selection options in instanece add and instance replace disks
45 can be replace by the new ``--iallocator=NAME`` option (shortened to
46 ``-I``), which will cause the auto-assignement of nodes with the
47 passed iallocator. The selected node(s) will be show as part of the
53 The protocol for communication between Ganeti and an allocator script
54 will be the following:
56 #. ganeti launches the program with a single argument, a filename that
57 contains a JSON-encoded structure (the input message)
59 #. if the script finishes with exit code different from zero, it is
60 considered a general failure and the full output will be reported to
61 the users; this can be the case when the allocator can't parse the
64 #. if the allocator finishes with exit code zero, it is expected to
65 output (on its stdout) a JSON-encoded structure (the response)
70 The input message will be the JSON encoding of a dictionary containing
74 the version of the protocol; this document
81 the list of cluster tags
84 the list of enabled hypervisors
87 a dictionary containing the request data:
90 the request type; this can be either ``allocate``, ``relocate`` or
91 ``multi-evacuate``; the ``allocate`` request is used when a new
92 instance needs to be placed on the cluster, while the ``relocate``
93 request is used when an existing instance needs to be moved within
94 the cluster; the ``multi-evacuate`` protocol requests that the
95 script computes the optimal relocate solution for all secondary
96 instances of the given nodes
98 The following keys are needed in allocate/relocate mode:
101 the name of the instance; if the request is a realocation, then this
102 name will be found in the list of instances (see below), otherwise
103 is the FQDN of the new instance
106 how many nodes should the algorithm return; while this information
107 can be deduced from the instace's disk template, it's better if
108 this computation is left to Ganeti as then allocator scripts are
109 less sensitive to changes to the disk templates
112 the total disk space that will be used by this instance on the
113 (new) nodes; again, this information can be computed from the list
114 of instance disks and its template type, but Ganeti is better
117 If the request is an allocation, then there are extra fields in the
121 list of dictionaries holding the disk definitions for this
122 instance (in the order they are exported to the hypervisor):
125 either ``r`` or ``w`` denoting if the disk is read-only or
129 the size of this disk in mebibytes
132 a list of dictionaries holding the network interfaces for this
133 instance, containing:
136 the IP address that Ganeti know for this instance, or null
139 the MAC address for this interface
142 the bridge to which this interface will be connected
145 the number of VCPUs for the instance
148 the disk template for the instance
151 the memory size for the instance
154 the OS type for the instance
157 the list of the instance's tags
160 the hypervisor of this instance
163 If the request is of type relocate, then there is one more entry in
164 the request dictionary, named ``relocate_from``, and it contains a
165 list of nodes to move the instance away from; note that with Ganeti
166 2.0, this list will always contain a single node, the current
167 secondary of the instance.
169 The multi-evacuate mode has instead a single request argument:
172 the names of the nodes to be evacuated
175 a dictionary with the data for the cluster's node groups; it is keyed
176 on the group UUID, and the values are a dictionary with the following
182 the allocation policy of the node group
185 a dictionary with the data for the current existing instance on the
186 cluster, indexed by instance name; the contents are similar to the
187 instance definitions for the allocate mode, with the addition of:
190 if this instance is set to run (but not the actual status of the
194 list of nodes on which this instance is placed; the primary node
195 of the instance is always the first one
198 dictionary with the data for the nodes in the cluster, indexed by
199 the node name; the dict contains [*]_ :
202 the total disk size of this node (mebibytes)
205 the free disk space on the node
208 the total memory size
211 free memory on the node; note that currently this does not take
212 into account the instances which are down on the node
215 the physical number of CPUs present on the machine; depending on
216 the hypervisor, this might or might not be equal to how many CPUs
217 the node operating system sees;
220 the primary IP address of the node
223 the secondary IP address of the node (the one used for the DRBD
224 replication); note that this can be the same as the primary one
227 list with the tags of the node
230 a boolean flag denoting whether this node is a master candidate
233 a boolean flag denoting whether this node is being drained
236 a boolean flag denoting whether this node is offline
239 total memory required by primary instances
242 total memory required by running primary instances
245 the node group that this node belongs to
247 No allocations should be made on nodes having either the ``drained``
248 or ``offline`` flags set. More details about these of node status
249 flags is available in the manpage :manpage:`ganeti(7)`.
251 .. [*] Note that no run-time data is present for offline, drained or
252 non-vm_capable nodes; this means the tags total_memory,
253 reserved_memory, free_memory, total_disk, free_disk, total_cpus,
254 i_pri_memory and i_pri_up memory will be absent
260 The response message is much more simple than the input one. It is
261 also a dict having three keys:
264 a boolean value denoting if the allocation was successful or not
267 a string with information from the scripts; if the allocation fails,
268 this will be shown to the user
271 the output of the algorithm; even if the algorithm failed
272 (i.e. success is false), this must be returned as an empty list
274 for allocate/relocate, this is the list of node(s) for the instance;
275 note that the length of this list must equal the ``requested_nodes``
276 entry in the input message, otherwise Ganeti will consider the result
279 for multi-evacuation mode, this is a list of lists; each element of
280 the list is a list of instance name and the new secondary node
282 .. note:: Current Ganeti version accepts either ``result`` or ``nodes``
283 as a backwards-compatibility measure (older versions only supported
289 Input messages to scripts
290 ~~~~~~~~~~~~~~~~~~~~~~~~~
292 Input message, new instance allocation::
298 "name": "instance3.example.com",
317 "mac": "00:11:22:33:44:55",
322 "disk_template": "drbd",
324 "disk_space_total": 3328,
325 "os": "debootstrap+default"
327 "cluster_name": "cluster1.example.com",
329 "instance1.example.com": {
345 "mac": "aa:00:00:00:60:bf",
350 "disk_template": "plain",
355 "os": "debootstrap+default"
357 "instance2.example.com": {
373 "mac": "aa:00:00:55:f8:38",
378 "disk_template": "drbd",
384 "os": "debootstrap+default"
389 "node1.example.com": {
390 "total_disk": 858276,
391 "primary_ip": "198.51.100.1",
392 "secondary_ip": "192.0.2.1",
398 "node2.example.com": {
399 "total_disk": 858240,
400 "primary_ip": "198.51.100.2",
401 "secondary_ip": "192.0.2.2",
407 "node3.example.com.com": {
408 "total_disk": 572184,
409 "primary_ip": "198.51.100.3",
410 "secondary_ip": "192.0.2.3",
419 Input message, reallocation. Since only the request entry in the input
420 message is changed, we show only this changed entry::
428 "name": "instance2.example.com",
429 "disk_space_total": 832
433 Input message, node evacuation::
439 "type": "multi-evacuate"
445 Successful response message::
448 "info": "Allocation successful",
456 Failed response message::
459 "info": "Can't find a suitable node for position 2 (already selected: node2.example.com)",
464 Successful node evacuation message::
467 "info": "Request successful",
482 Command line messages
483 ~~~~~~~~~~~~~~~~~~~~~
486 # gnt-instance add -t plain -m 2g --os-size 1g --swap-size 512m --iallocator hail -o debootstrap+default instance3
487 Selected nodes for the instance: node1.example.com
488 * creating instance disks...
491 # gnt-instance add -t plain -m 3400m --os-size 1g --swap-size 512m --iallocator hail -o debootstrap+default instance4
492 Failure: prerequisites not met for this operation:
493 Can't compute nodes using iallocator 'hail': Can't find a suitable node for position 1 (already selected: )
495 # gnt-instance add -t drbd -m 1400m --os-size 1g --swap-size 512m --iallocator hail -o debootstrap+default instance5
496 Failure: prerequisites not met for this operation:
497 Can't compute nodes using iallocator 'hail': Can't find a suitable node for position 2 (already selected: node1.example.com)
499 Reference implementation
500 ~~~~~~~~~~~~~~~~~~~~~~~~
502 Ganeti's default iallocator is "hail" which is part of the separate
503 ganeti-htools project. In order to see its source code please clone
504 ``git://git.ganeti.org/htools.git``. Note that htools is implemented
505 using the Haskell programming language.
507 .. vim: set textwidth=72 :