Statistics
| Branch: | Tag: | Revision:

root / doc / iallocator.rst @ 7ed400f0

History | View | Annotate | Download (15.7 kB)

1
Ganeti automatic instance allocation
2
====================================
3

    
4
Documents Ganeti version 2.5
5

    
6
.. contents::
7

    
8
Introduction
9
------------
10

    
11
Currently in Ganeti the admin has to specify the exact locations for
12
an instance's node(s). This prevents a completely automatic node
13
evacuation, and is in general a nuisance.
14

    
15
The *iallocator* framework will enable automatic placement via
16
external scripts, which allows customization of the cluster layout per
17
the site's requirements.
18

    
19
User-visible changes
20
~~~~~~~~~~~~~~~~~~~~
21

    
22
There are two parts of the ganeti operation that are impacted by the
23
auto-allocation: how the cluster knows what the allocator algorithms
24
are and how the admin uses these in creating instances.
25

    
26
An allocation algorithm is just the filename of a program installed in
27
a defined list of directories.
28

    
29
Cluster configuration
30
~~~~~~~~~~~~~~~~~~~~~
31

    
32
At configure time, the list of the directories can be selected via the
33
``--with-iallocator-search-path=LIST`` option, where *LIST* is a
34
comma-separated list of directories. If not given, this defaults to
35
``$libdir/ganeti/iallocators``, i.e. for an installation under
36
``/usr``, this will be ``/usr/lib/ganeti/iallocators``.
37

    
38
Ganeti will then search for allocator script in the configured list,
39
using the first one whose filename matches the one given by the user.
40

    
41
Command line interface changes
42
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
43

    
44
The node selection options in instance add and instance replace disks
45
can be replace by the new ``--iallocator=NAME`` option (shortened to
46
``-I``), which will cause the auto-assignement of nodes with the
47
passed iallocator. The selected node(s) will be show as part of the
48
command output.
49

    
50
IAllocator API
51
--------------
52

    
53
The protocol for communication between Ganeti and an allocator script
54
will be the following:
55

    
56
#. ganeti launches the program with a single argument, a filename that
57
   contains a JSON-encoded structure (the input message)
58

    
59
#. if the script finishes with exit code different from zero, it is
60
   considered a general failure and the full output will be reported to
61
   the users; this can be the case when the allocator can't parse the
62
   input message
63

    
64
#. if the allocator finishes with exit code zero, it is expected to
65
   output (on its stdout) a JSON-encoded structure (the response)
66

    
67
Input message
68
~~~~~~~~~~~~~
69

    
70
The input message will be the JSON encoding of a dictionary containing
71
all the required information to perform the operation. We explain the
72
contents of this dictionary in two parts: common information that every
73
type of operation requires, and operation-specific information.
74

    
75
Common information
76
++++++++++++++++++
77

    
78
All input dictionaries to the IAllocator must carry the following keys:
79

    
80
version
81
  the version of the protocol; this document
82
  specifies version 2
83

    
84
cluster_name
85
  the cluster name
86

    
87
cluster_tags
88
  the list of cluster tags
89

    
90
enabled_hypervisors
91
  the list of enabled hypervisors
92

    
93
ipolicy
94
  the cluster-wide instance policy (for information; the per-node group
95
  values take precedence and should be used instead)
96

    
97
request
98
  a dictionary containing the details of the request; the keys vary
99
  depending on the type of operation that's being requested, as
100
  explained in `Operation-specific input`_ below.
101

    
102
nodegroups
103
  a dictionary with the data for the cluster's node groups; it is keyed
104
  on the group UUID, and the values are a dictionary with the following
105
  keys:
106

    
107
  name
108
    the node group name
109
  alloc_policy
110
    the allocation policy of the node group (consult the semantics of
111
    this attribute in the :manpage:`gnt-group(8)` manpage)
112
  ipolicy
113
    the instance policy of the node group
114

    
115
instances
116
  a dictionary with the data for the current existing instance on the
117
  cluster, indexed by instance name; the contents are similar to the
118
  instance definitions for the allocate mode, with the addition of:
119

    
120
  admin_state
121
    if this instance is set to run (but not the actual status of the
122
    instance)
123

    
124
  nodes
125
    list of nodes on which this instance is placed; the primary node
126
    of the instance is always the first one
127

    
128
nodes
129
  dictionary with the data for the nodes in the cluster, indexed by
130
  the node name; the dict contains [*]_ :
131

    
132
  total_disk
133
    the total disk size of this node (mebibytes)
134

    
135
  free_disk
136
    the free disk space on the node
137

    
138
  total_memory
139
    the total memory size
140

    
141
  free_memory
142
    free memory on the node; note that currently this does not take
143
    into account the instances which are down on the node
144

    
145
  total_cpus
146
    the physical number of CPUs present on the machine; depending on
147
    the hypervisor, this might or might not be equal to how many CPUs
148
    the node operating system sees;
149

    
150
  primary_ip
151
    the primary IP address of the node
152

    
153
  secondary_ip
154
    the secondary IP address of the node (the one used for the DRBD
155
    replication); note that this can be the same as the primary one
156

    
157
  tags
158
    list with the tags of the node
159

    
160
  master_candidate:
161
    a boolean flag denoting whether this node is a master candidate
162

    
163
  drained:
164
    a boolean flag denoting whether this node is being drained
165

    
166
  offline:
167
    a boolean flag denoting whether this node is offline
168

    
169
  i_pri_memory:
170
    total memory required by primary instances
171

    
172
  i_pri_up_memory:
173
    total memory required by running primary instances
174

    
175
  group:
176
    the node group that this node belongs to
177

    
178
  No allocations should be made on nodes having either the ``drained``
179
  or ``offline`` flags set. More details about these of node status
180
  flags is available in the manpage :manpage:`ganeti(7)`.
181

    
182
.. [*] Note that no run-time data is present for offline, drained or
183
   non-vm_capable nodes; this means the tags total_memory,
184
   reserved_memory, free_memory, total_disk, free_disk, total_cpus,
185
   i_pri_memory and i_pri_up memory will be absent
186

    
187
Operation-specific input
188
++++++++++++++++++++++++
189

    
190
All input dictionaries to the IAllocator carry, in the ``request``
191
dictionary, detailed information about the operation that's being
192
requested. The required keys vary depending on the type of operation, as
193
follows.
194

    
195
In all cases, it includes:
196

    
197
  type
198
    the request type; this can be either ``allocate``, ``relocate``,
199
    ``change-group`` or ``node-evacuate``. The
200
    ``allocate`` request is used when a new instance needs to be placed
201
    on the cluster. The ``relocate`` request is used when an existing
202
    instance needs to be moved within its node group.
203

    
204
    The ``multi-evacuate`` protocol used to request that the script
205
    computes the optimal relocate solution for all secondary instances
206
    of the given nodes. It is now deprecated and needs only be
207
    implemented if backwards compatibility with Ganeti 2.4 and lower is
208
    needed.
209

    
210
    The ``change-group`` request is used to relocate multiple instances
211
    across multiple node groups. ``node-evacuate`` evacuates instances
212
    off their node(s). These are described in a separate :ref:`design
213
    document <multi-reloc-detailed-design>`.
214

    
215
For both allocate and relocate mode, the following extra keys are needed
216
in the ``request`` dictionary:
217

    
218
  name
219
    the name of the instance; if the request is a realocation, then this
220
    name will be found in the list of instances (see below), otherwise
221
    is the FQDN of the new instance; type *string*
222

    
223
  required_nodes
224
    how many nodes should the algorithm return; while this information
225
    can be deduced from the instace's disk template, it's better if
226
    this computation is left to Ganeti as then allocator scripts are
227
    less sensitive to changes to the disk templates; type *integer*
228

    
229
  disk_space_total
230
    the total disk space that will be used by this instance on the
231
    (new) nodes; again, this information can be computed from the list
232
    of instance disks and its template type, but Ganeti is better
233
    suited to compute it; type *integer*
234

    
235
.. pyassert::
236

    
237
   constants.DISK_ACCESS_SET == set([constants.DISK_RDONLY,
238
     constants.DISK_RDWR])
239

    
240
Allocation needs, in addition:
241

    
242
  disks
243
    list of dictionaries holding the disk definitions for this
244
    instance (in the order they are exported to the hypervisor):
245

    
246
    mode
247
      either :pyeval:`constants.DISK_RDONLY` or
248
      :pyeval:`constants.DISK_RDWR` denoting if the disk is read-only or
249
      writable
250

    
251
    size
252
      the size of this disk in mebibytes
253

    
254
  nics
255
    a list of dictionaries holding the network interfaces for this
256
    instance, containing:
257

    
258
    ip
259
      the IP address that Ganeti know for this instance, or null
260

    
261
    mac
262
      the MAC address for this interface
263

    
264
    bridge
265
      the bridge to which this interface will be connected
266

    
267
  vcpus
268
    the number of VCPUs for the instance
269

    
270
  disk_template
271
    the disk template for the instance
272

    
273
  memory
274
   the memory size for the instance
275

    
276
  os
277
   the OS type for the instance
278

    
279
  tags
280
    the list of the instance's tags
281

    
282
  hypervisor
283
    the hypervisor of this instance
284

    
285
Relocation:
286

    
287
  relocate_from
288
     a list of nodes to move the instance away from (note that with
289
     Ganeti 2.0, this list will always contain a single node, the
290
     current secondary of the instance); type *list of strings*
291

    
292
As for ``node-evacuate``, it needs the following request arguments:
293

    
294
  instances
295
    a list of instance names to evacuate; type *list of strings*
296

    
297
  evac_mode
298
    specify which instances to evacuate; one of ``primary-only``,
299
    ``secondary-only``, ``all``, type *string*
300

    
301
``change-group`` needs the following request arguments:
302

    
303
  instances
304
    a list of instance names whose group to change; type
305
    *list of strings*
306

    
307
  target_groups
308
    must either be the empty list, or contain a list of group UUIDs that
309
    should be considered for relocating instances to; type
310
    *list of strings*
311

    
312
Response message
313
~~~~~~~~~~~~~~~~
314

    
315
The response message is much more simple than the input one. It is
316
also a dict having three keys:
317

    
318
success
319
  a boolean value denoting if the allocation was successful or not
320

    
321
info
322
  a string with information from the scripts; if the allocation fails,
323
  this will be shown to the user
324

    
325
result
326
  the output of the algorithm; even if the algorithm failed
327
  (i.e. success is false), this must be returned as an empty list
328

    
329
  for allocate/relocate, this is the list of node(s) for the instance;
330
  note that the length of this list must equal the ``requested_nodes``
331
  entry in the input message, otherwise Ganeti will consider the result
332
  as failed
333

    
334
  for the ``node-evacuate`` and ``change-group`` modes, this is a
335
  dictionary containing, among other information, a list of lists of
336
  serialized opcodes; see the :ref:`design document
337
  <multi-reloc-result>` for a detailed description
338

    
339
.. note:: Current Ganeti version accepts either ``result`` or ``nodes``
340
   as a backwards-compatibility measure (older versions only supported
341
   ``nodes``)
342

    
343
Examples
344
--------
345

    
346
Input messages to scripts
347
~~~~~~~~~~~~~~~~~~~~~~~~~
348

    
349
Input message, new instance allocation (common elements are listed this
350
time, but not included in further examples below)::
351

    
352
  {
353
    "version": 2,
354
    "cluster_name": "cluster1.example.com",
355
    "cluster_tags": [],
356
    "enabled_hypervisors": [
357
      "xen-pvm"
358
    ],
359
    "nodegroups": {
360
      "f4e06e0d-528a-4963-a5ad-10f3e114232d": {
361
        "name": "default",
362
        "alloc_policy": "preferred"
363
      }
364
    },
365
    "instances": {
366
      "instance1.example.com": {
367
        "tags": [],
368
        "should_run": false,
369
        "disks": [
370
          {
371
            "mode": "w",
372
            "size": 64
373
          },
374
          {
375
            "mode": "w",
376
            "size": 512
377
          }
378
        ],
379
        "nics": [
380
          {
381
            "ip": null,
382
            "mac": "aa:00:00:00:60:bf",
383
            "bridge": "xen-br0"
384
          }
385
        ],
386
        "vcpus": 1,
387
        "disk_template": "plain",
388
        "memory": 128,
389
        "nodes": [
390
          "nodee1.com"
391
        ],
392
        "os": "debootstrap+default"
393
      },
394
      "instance2.example.com": {
395
        "tags": [],
396
        "should_run": false,
397
        "disks": [
398
          {
399
            "mode": "w",
400
            "size": 512
401
          },
402
          {
403
            "mode": "w",
404
            "size": 256
405
          }
406
        ],
407
        "nics": [
408
          {
409
            "ip": null,
410
            "mac": "aa:00:00:55:f8:38",
411
            "bridge": "xen-br0"
412
          }
413
        ],
414
        "vcpus": 1,
415
        "disk_template": "drbd",
416
        "memory": 512,
417
        "nodes": [
418
          "node2.example.com",
419
          "node3.example.com"
420
        ],
421
        "os": "debootstrap+default"
422
      }
423
    },
424
    "nodes": {
425
      "node1.example.com": {
426
        "total_disk": 858276,
427
        "primary_ip": "198.51.100.1",
428
        "secondary_ip": "192.0.2.1",
429
        "tags": [],
430
        "group": "f4e06e0d-528a-4963-a5ad-10f3e114232d",
431
        "free_memory": 3505,
432
        "free_disk": 856740,
433
        "total_memory": 4095
434
      },
435
      "node2.example.com": {
436
        "total_disk": 858240,
437
        "primary_ip": "198.51.100.2",
438
        "secondary_ip": "192.0.2.2",
439
        "tags": ["test"],
440
        "group": "f4e06e0d-528a-4963-a5ad-10f3e114232d",
441
        "free_memory": 3505,
442
        "free_disk": 848320,
443
        "total_memory": 4095
444
      },
445
      "node3.example.com.com": {
446
        "total_disk": 572184,
447
        "primary_ip": "198.51.100.3",
448
        "secondary_ip": "192.0.2.3",
449
        "tags": [],
450
        "group": "f4e06e0d-528a-4963-a5ad-10f3e114232d",
451
        "free_memory": 3505,
452
        "free_disk": 570648,
453
        "total_memory": 4095
454
      }
455
    },
456
    "request": {
457
      "type": "allocate",
458
      "name": "instance3.example.com",
459
      "required_nodes": 2,
460
      "disk_space_total": 3328,
461
      "disks": [
462
        {
463
          "mode": "w",
464
          "size": 1024
465
        },
466
        {
467
          "mode": "w",
468
          "size": 2048
469
        }
470
      ],
471
      "nics": [
472
        {
473
          "ip": null,
474
          "mac": "00:11:22:33:44:55",
475
          "bridge": null
476
        }
477
      ],
478
      "vcpus": 1,
479
      "disk_template": "drbd",
480
      "memory": 2048,
481
      "os": "debootstrap+default",
482
      "tags": [
483
        "type:test",
484
        "owner:foo"
485
      ],
486
      hypervisor: "xen-pvm"
487
    }
488
  }
489

    
490
Input message, reallocation::
491

    
492
  {
493
    "version": 2,
494
    ...
495
    "request": {
496
      "type": "relocate",
497
      "name": "instance2.example.com",
498
      "required_nodes": 1,
499
      "disk_space_total": 832,
500
      "relocate_from": [
501
        "node3.example.com"
502
      ]
503
    }
504
  }
505

    
506

    
507
Response messages
508
~~~~~~~~~~~~~~~~~
509
Successful response message::
510

    
511
  {
512
    "success": true,
513
    "info": "Allocation successful",
514
    "result": [
515
      "node2.example.com",
516
      "node1.example.com"
517
    ]
518
  }
519

    
520
Failed response message::
521

    
522
  {
523
    "success": false,
524
    "info": "Can't find a suitable node for position 2 (already selected: node2.example.com)",
525
    "result": []
526
  }
527

    
528
Successful node evacuation message::
529

    
530
  {
531
    "success": true,
532
    "info": "Request successful",
533
    "result": [
534
      [
535
        "instance1",
536
        "node3"
537
      ],
538
      [
539
        "instance2",
540
        "node1"
541
      ]
542
    ]
543
  }
544

    
545

    
546
Command line messages
547
~~~~~~~~~~~~~~~~~~~~~
548
::
549

    
550
  # gnt-instance add -t plain -m 2g --os-size 1g --swap-size 512m --iallocator hail -o debootstrap+default instance3
551
  Selected nodes for the instance: node1.example.com
552
  * creating instance disks...
553
  [...]
554

    
555
  # gnt-instance add -t plain -m 3400m --os-size 1g --swap-size 512m --iallocator hail -o debootstrap+default instance4
556
  Failure: prerequisites not met for this operation:
557
  Can't compute nodes using iallocator 'hail': Can't find a suitable node for position 1 (already selected: )
558

    
559
  # gnt-instance add -t drbd -m 1400m --os-size 1g --swap-size 512m --iallocator hail -o debootstrap+default instance5
560
  Failure: prerequisites not met for this operation:
561
  Can't compute nodes using iallocator 'hail': Can't find a suitable node for position 2 (already selected: node1.example.com)
562

    
563
Reference implementation
564
~~~~~~~~~~~~~~~~~~~~~~~~
565

    
566
Ganeti's default iallocator is "hail" which is available when "htools"
567
components have been enabled at build time (see :doc:`install-quick` for
568
more details).
569

    
570
.. vim: set textwidth=72 :
571
.. Local Variables:
572
.. mode: rst
573
.. fill-column: 72
574
.. End: