1 <!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [
3 <article class="specification">
5 <title>Ganeti automatic instance allocation</title>
7 <para>Documents Ganeti version 1.2</para>
9 <title>Introduction</title>
11 <para>Currently in Ganeti the admin has to specify the exact
12 locations for an instance's node(s). This prevents a completely
13 automatic node evacuation, and is in general a nuisance.</para>
15 <para>The <acronym>iallocator</acronym> framework will enable
16 automatic placement via external scripts, which allows
17 customization of the cluster layout per the site's
23 <title>User-visible changes</title>
25 <para>There are two parts of the ganeti operation that are
26 impacted by the auto-allocation: how the cluster knows what the
27 allocator algorithms are and how the admin uses these in creating
30 <para>An allocation algorithm is just the filename of a program
31 installed in a defined list of directories.</para>
34 <title>Cluster configuration</title>
36 <para>At configure time, the list of the directories can be
38 <option>--with-iallocator-search-path=LIST</option> option,
39 where <userinput>LIST</userinput> is a comma-separated list of
40 directories. If not given, this defaults to
41 <constant>$libdir/ganeti/iallocators</constant>, i.e. for an
42 installation under <filename class="directory">/usr</filename>,
43 this will be <filename
44 class="directory">/usr/lib/ganeti/iallocators</filename>.</para>
46 <para>Ganeti will then search for allocator script in the
47 configured list, using the first one whose filename matches the
48 one given by the user.</para>
53 <title>Command line interface changes</title>
55 <para>The node selection options in instanece add and instance
56 replace disks can be replace by the new <option>--iallocator
57 <replaceable>NAME</replaceable></option> option, which will
58 cause the autoassignation. The selected node(s) will be show as
59 part of the command output.</para>
66 <title>IAllocator API</title>
68 <para>The protocol for communication between Ganeti and an
69 allocator script will be the following:</para>
73 <simpara>ganeti launches the program with a single argument, a
74 filename that contains a JSON-encoded structure (the input
78 <simpara>if the script finishes with exit code different from
79 zero, it is considered a general failure and the full output
80 will be reported to the users; this can be the case when the
81 allocator can't parse the input message;</simpara>
84 <simpara>if the allocator finishes with exit code zero, it is
85 expected to output (on its stdout) a JSON-encoded structure
86 (the response)</simpara>
91 <title>Input message</title>
93 <para>The input message will be the JSON encoding of a
94 dictionary containing the following:</para>
100 <simpara>the version of the protocol; this document
101 specifies version 1</simpara>
105 <term>cluster_name</term>
107 <simpara>the cluster name</simpara>
111 <term>cluster_tags</term>
113 <simpara>the list of cluster tags</simpara>
119 <simpara>a dictionary containing the request data:</simpara>
124 <simpara>the request type; this can be either
125 <literal>allocate</literal> or
126 <literal>relocate</literal>; the
127 <literal>allocate</literal> request is used when a
128 new instance needs to be placed on the cluster,
129 while the <literal>relocate</literal> request is
130 used when an existing instance needs to be moved
131 within the cluster</simpara>
137 <simpara>the name of the instance; if the request is
138 a realocation, then this name will be found in the
139 list of instances (see below), otherwise is the
140 <acronym>FQDN</acronym> of the new
145 <term>required_nodes</term>
147 <simpara>how many nodes should the algorithm return;
148 while this information can be deduced from the
149 instace's disk template, it's better if this
150 computation is left to Ganeti as then allocator
151 scripts are less sensitive to changes to the disk
156 <term>disk_space_total</term>
158 <simpara>the total disk space that will be used by
159 this instance on the (new) nodes; again, this
160 information can be computed from the list of
161 instance disks and its template type, but Ganeti is
162 better suited to compute it</simpara>
166 <simpara>If the request is an allocation, then there are
167 extra fields in the request dictionary:</simpara>
172 <simpara>list of dictionaries holding the disk
173 definitions for this instance (in the order they are
174 exported to the hypervisor):</simpara>
179 <simpara>either <literal>w</literal> or
180 <literal>w</literal> denoting if the disk is
181 read-only or writable; for Ganeti 1.2, this
182 will always be <literal>w</literal</simpara>
188 <simpara>the size of this disk in mebibyte</simpara>
197 <simpara>a list of dictionaries holding the network
198 interfaces for this instance, containing:</simpara>
203 <simpara>the IP address that Ganeti know for
204 this instance, or null</simpara>
210 <simpara>the MAC address for this interface</simpara>
216 <simpara>the bridge to which this interface
217 will be connected</simpara>
226 <simpara>the number of VCPUs for the instance</simpara>
230 <term>disk_template</term>
232 <simpara>the disk template for the instance</simpara>
238 <simpara>the memory size for the instance</simpara>
244 <simpara>the OS type for the instance</simpara>
250 <simpara>the list of the instance's tags</simpara>
254 <simpara>If the request is of type relocate, then there is
255 one more entry in the request dictionary, named
256 <varname>relocate_from</varname>, and it contains a list
257 of nodes to move the instance away from; note that with
258 Ganeti 1.2, this list will always contain a single node,
259 the current secondary of the instance.</simpara>
263 <term>instances</term>
265 <simpara>a dictionary with the data for the current
266 existing instance on the cluster, indexed by instance
267 name; the contents are similar to the instance definitions
268 for the allocate mode, with the addition of:</simpara>
271 <term>should_run</term>
273 <simpara>if this instance is set to run (but not the
274 actual status of the instance)</simpara>
280 <simpara>list of nodes on which this instance is
281 placed; the primary node of the instance is always
282 the first one</simpara>
291 <simpara>dictionary with the data for the nodes in the
292 cluster, indexed by the node name; the dict
296 <term>total_disk</term>
298 <simpara>the total disk size of this node
299 (mebibytes)</simpara>
303 <term>free_disk</term>
305 <simpara>the free disk space on the node</simpara>
309 <term>total_memory</term>
311 <simpara>the total memory size</simpara>
315 <term>free_memory</term>
317 <simpara>free memory on the node; note that
318 currently this does not take into account the
319 instances which are down on the node</simpara>
323 <term>total_cpus</term>
325 <simpara>the physical number of CPUs present on the
326 machine; depending on the hypervisor, this might or
327 might not be equal to how many CPUs the node
328 operating system sees;</simpara>
332 <term>primary_ip</term>
334 <simpara>the primary IP address of the
339 <term>secondary_ip</term>
341 <simpara>the secondary IP address of the node (the
342 one used for the DRBD replication); note that this
343 can be the same as the primary one</simpara>
349 <simpara>list with the tags of the node</simpara>
360 <title>Respone message</title>
362 <para>The response message is much more simple than the input
363 one. It is also a dict having three keys:</para>
368 <simpara>a boolean value denoting if the allocation was
369 successfull or not</simpara>
375 <simpara>a string with information from the scripts; if
376 the allocation fails, this will be shown to the
383 <simpara>the list of nodes computed by the algorithm; even
384 if the algorithm failed (i.e. success is false), this must
385 be returned as an empty list; also note that the length of
386 this list must equal the
387 <varname>requested_nodes</varname> entry in the input
388 message, otherwise Ganeti will consider the result as
397 <title>Examples</title>
399 <title>Input messages to scripts</title>
400 <simpara>Input message, new instance allocation</simpara>
406 "name": "instance3.example.com",
425 "mac": "00:11:22:33:44:55",
430 "disk_template": "drbd",
432 "disk_space_total": 3328,
435 "cluster_name": "cluster1.example.com",
437 "instance1.example.com": {
453 "mac": "aa:00:00:00:60:bf",
458 "disk_template": "plain",
465 "instance2.example.com": {
481 "mac": "aa:00:00:55:f8:38",
486 "disk_template": "drbd",
497 "node1.example.com": {
498 "total_disk": 858276,
499 "primary_ip": "192.168.1.1",
500 "secondary_ip": "192.168.2.1",
506 "node2.example.com": {
507 "total_disk": 858240,
508 "primary_ip": "192.168.1.3",
509 "secondary_ip": "192.168.2.3",
515 "node3.example.com.com": {
516 "total_disk": 572184,
517 "primary_ip": "192.168.1.3",
518 "secondary_ip": "192.168.2.3",
527 <simpara>Input message, reallocation. Since only the request
528 entry in the input message is changed, the following shows only
529 this entry:</simpara>
537 "name": "instance2.example.com",
538 "disk_space_total": 832
544 <title>Response messages</title>
545 <simpara>Successful response message:</simpara>
548 "info": "Allocation successful",
556 <simpara>Failed response message:</simpara>
559 "info": "Can't find a suitable node for position 2 (already selected: node2.example.com)",
567 <title>Command line messages</title>
569 # gnt-instance add -t plain -m 2g --os-size 1g --swap-size 512m --iallocator dumb-allocator -o etch-image instance3
570 Selected nodes for the instance: node1.example.com
571 * creating instance disks...
574 # gnt-instance add -t plain -m 3400m --os-size 1g --swap-size 512m --iallocator dumb-allocator -o etch-image instance4
575 Failure: prerequisites not met for this operation:
576 Can't compute nodes using iallocator 'dumb-allocator': Can't find a suitable node for position 1 (already selected: )
578 # gnt-instance add -t drbd -m 1400m --os-size 1g --swap-size 512m --iallocator dumb-allocator -o etch-image instance5
579 Failure: prerequisites not met for this operation:
580 Can't compute nodes using iallocator 'dumb-allocator': Can't find a suitable node for position 2 (already selected: node1.example.com)