1 Ganeti automatic instance allocation
2 ====================================
4 Documents Ganeti version 2.0
11 Currently in Ganeti the admin has to specify the exact locations for
12 an instance's node(s). This prevents a completely automatic node
13 evacuation, and is in general a nuisance.
15 The *iallocator* framework will enable automatic placement via
16 external scripts, which allows customization of the cluster layout per
17 the site's requirements.
22 There are two parts of the ganeti operation that are impacted by the
23 auto-allocation: how the cluster knows what the allocator algorithms
24 are and how the admin uses these in creating instances.
26 An allocation algorithm is just the filename of a program installed in
27 a defined list of directories.
32 At configure time, the list of the directories can be selected via the
33 ``--with-iallocator-search-path=LIST`` option, where *LIST* is a
34 comma-separated list of directories. If not given, this defaults to
35 ``$libdir/ganeti/iallocators``, i.e. for an installation under
36 ``/usr``, this will be ``/usr/lib/ganeti/iallocators``.
38 Ganeti will then search for allocator script in the configured list,
39 using the first one whose filename matches the one given by the user.
41 Command line interface changes
42 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
44 The node selection options in instanece add and instance replace disks
45 can be replace by the new ``--iallocator=NAME`` option (shortened to
46 ``-I``), which will cause the auto-assignement of nodes with the
47 passed iallocator. The selected node(s) will be show as part of the
53 The protocol for communication between Ganeti and an allocator script
54 will be the following:
56 #. ganeti launches the program with a single argument, a filename that
57 contains a JSON-encoded structure (the input message)
59 #. if the script finishes with exit code different from zero, it is
60 considered a general failure and the full output will be reported to
61 the users; this can be the case when the allocator can't parse the
64 #. if the allocator finishes with exit code zero, it is expected to
65 output (on its stdout) a JSON-encoded structure (the response)
70 The input message will be the JSON encoding of a dictionary containing
74 the version of the protocol; this document
81 the list of cluster tags
84 the list of enabled hypervisors
87 a dictionary containing the request data:
90 the request type; this can be either ``allocate`` or ``relocate``;
91 the ``allocate`` request is used when a new instance needs to be
92 placed on the cluster, while the ``relocate`` request is used when
93 an existing instance needs to be moved within the cluster
96 the name of the instance; if the request is a realocation, then
97 this name will be found in the list of instances (see below),
98 otherwise is the FQDN of the new instance
101 how many nodes should the algorithm return; while this information
102 can be deduced from the instace's disk template, it's better if
103 this computation is left to Ganeti as then allocator scripts are
104 less sensitive to changes to the disk templates
107 the total disk space that will be used by this instance on the
108 (new) nodes; again, this information can be computed from the list
109 of instance disks and its template type, but Ganeti is better
112 If the request is an allocation, then there are extra fields in the
116 list of dictionaries holding the disk definitions for this
117 instance (in the order they are exported to the hypervisor):
120 either ``r`` or ``w`` denoting if the disk is read-only or
124 the size of this disk in mebibytes
127 a list of dictionaries holding the network interfaces for this
128 instance, containing:
131 the IP address that Ganeti know for this instance, or null
134 the MAC address for this interface
137 the bridge to which this interface will be connected
140 the number of VCPUs for the instance
143 the disk template for the instance
146 the memory size for the instance
149 the OS type for the instance
152 the list of the instance's tags
155 the hypervisor of this instance
158 If the request is of type relocate, then there is one more entry in
159 the request dictionary, named ``relocate_from``, and it contains a
160 list of nodes to move the instance away from; note that with Ganeti
161 2.0, this list will always contain a single node, the current
162 secondary of the instance.
165 a dictionary with the data for the current existing instance on the
166 cluster, indexed by instance name; the contents are similar to the
167 instance definitions for the allocate mode, with the addition of:
170 if this instance is set to run (but not the actual status of the
174 list of nodes on which this instance is placed; the primary node
175 of the instance is always the first one
178 dictionary with the data for the nodes in the cluster, indexed by
179 the node name; the dict contains:
182 the total disk size of this node (mebibytes)
185 the free disk space on the node
188 the total memory size
191 free memory on the node; note that currently this does not take
192 into account the instances which are down on the node
195 the physical number of CPUs present on the machine; depending on
196 the hypervisor, this might or might not be equal to how many CPUs
197 the node operating system sees;
200 the primary IP address of the node
203 the secondary IP address of the node (the one used for the DRBD
204 replication); note that this can be the same as the primary one
207 list with the tags of the node
210 a boolean flag denoting whether this node is a master candidate
213 a boolean flag denoting whether this node is being drained
216 a boolean flag denoting whether this node is offline
219 total memory required by primary instances
222 total memory required by running primary instances
224 No allocations should be made on nodes having either the ``drained``
225 or ``offline`` flags set. More details about these of node status
226 flags is available in the manpage *ganeti(7)*.
232 The response message is much more simple than the input one. It is
233 also a dict having three keys:
236 a boolean value denoting if the allocation was successfull or not
239 a string with information from the scripts; if the allocation fails,
240 this will be shown to the user
243 the list of nodes computed by the algorithm; even if the algorithm
244 failed (i.e. success is false), this must be returned as an empty
245 list; also note that the length of this list must equal the
246 ``requested_nodes`` entry in the input message, otherwise Ganeti
247 will consider the result as failed
252 Input messages to scripts
253 ~~~~~~~~~~~~~~~~~~~~~~~~~
255 Input message, new instance allocation::
261 "name": "instance3.example.com",
280 "mac": "00:11:22:33:44:55",
285 "disk_template": "drbd",
287 "disk_space_total": 3328,
290 "cluster_name": "cluster1.example.com",
292 "instance1.example.com": {
308 "mac": "aa:00:00:00:60:bf",
313 "disk_template": "plain",
320 "instance2.example.com": {
336 "mac": "aa:00:00:55:f8:38",
341 "disk_template": "drbd",
352 "node1.example.com": {
353 "total_disk": 858276,
354 "primary_ip": "192.168.1.1",
355 "secondary_ip": "192.168.2.1",
361 "node2.example.com": {
362 "total_disk": 858240,
363 "primary_ip": "192.168.1.3",
364 "secondary_ip": "192.168.2.3",
370 "node3.example.com.com": {
371 "total_disk": 572184,
372 "primary_ip": "192.168.1.3",
373 "secondary_ip": "192.168.2.3",
382 Input message, reallocation. Since only the request entry in the input
383 message is changed, we show only this changed entry::
391 "name": "instance2.example.com",
392 "disk_space_total": 832
398 Successful response message::
401 "info": "Allocation successful",
409 Failed response message::
412 "info": "Can't find a suitable node for position 2 (already selected: node2.example.com)",
417 Command line messages
418 ~~~~~~~~~~~~~~~~~~~~~
421 # gnt-instance add -t plain -m 2g --os-size 1g --swap-size 512m --iallocator dumb-allocator -o etch-image instance3
422 Selected nodes for the instance: node1.example.com
423 * creating instance disks...
426 # gnt-instance add -t plain -m 3400m --os-size 1g --swap-size 512m --iallocator dumb-allocator -o etch-image instance4
427 Failure: prerequisites not met for this operation:
428 Can't compute nodes using iallocator 'dumb-allocator': Can't find a suitable node for position 1 (already selected: )
430 # gnt-instance add -t drbd -m 1400m --os-size 1g --swap-size 512m --iallocator dumb-allocator -o etch-image instance5
431 Failure: prerequisites not met for this operation:
432 Can't compute nodes using iallocator 'dumb-allocator': Can't find a suitable node for position 2 (already selected: node1.example.com)