Statistics
| Branch: | Tag: | Revision:

root / doc / iallocator.rst @ f0edfcf6

History | View | Annotate | Download (15.5 kB)

1
Ganeti automatic instance allocation
2
====================================
3

    
4
Documents Ganeti version 2.5
5

    
6
.. contents::
7

    
8
Introduction
9
------------
10

    
11
Currently in Ganeti the admin has to specify the exact locations for
12
an instance's node(s). This prevents a completely automatic node
13
evacuation, and is in general a nuisance.
14

    
15
The *iallocator* framework will enable automatic placement via
16
external scripts, which allows customization of the cluster layout per
17
the site's requirements.
18

    
19
User-visible changes
20
~~~~~~~~~~~~~~~~~~~~
21

    
22
There are two parts of the ganeti operation that are impacted by the
23
auto-allocation: how the cluster knows what the allocator algorithms
24
are and how the admin uses these in creating instances.
25

    
26
An allocation algorithm is just the filename of a program installed in
27
a defined list of directories.
28

    
29
Cluster configuration
30
~~~~~~~~~~~~~~~~~~~~~
31

    
32
At configure time, the list of the directories can be selected via the
33
``--with-iallocator-search-path=LIST`` option, where *LIST* is a
34
comma-separated list of directories. If not given, this defaults to
35
``$libdir/ganeti/iallocators``, i.e. for an installation under
36
``/usr``, this will be ``/usr/lib/ganeti/iallocators``.
37

    
38
Ganeti will then search for allocator script in the configured list,
39
using the first one whose filename matches the one given by the user.
40

    
41
Command line interface changes
42
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
43

    
44
The node selection options in instanece add and instance replace disks
45
can be replace by the new ``--iallocator=NAME`` option (shortened to
46
``-I``), which will cause the auto-assignement of nodes with the
47
passed iallocator. The selected node(s) will be show as part of the
48
command output.
49

    
50
IAllocator API
51
--------------
52

    
53
The protocol for communication between Ganeti and an allocator script
54
will be the following:
55

    
56
#. ganeti launches the program with a single argument, a filename that
57
   contains a JSON-encoded structure (the input message)
58

    
59
#. if the script finishes with exit code different from zero, it is
60
   considered a general failure and the full output will be reported to
61
   the users; this can be the case when the allocator can't parse the
62
   input message
63

    
64
#. if the allocator finishes with exit code zero, it is expected to
65
   output (on its stdout) a JSON-encoded structure (the response)
66

    
67
Input message
68
~~~~~~~~~~~~~
69

    
70
The input message will be the JSON encoding of a dictionary containing
71
all the required information to perform the operation. We explain the
72
contents of this dictionary in two parts: common information that every
73
type of operation requires, and operation-specific information.
74

    
75
Common information
76
++++++++++++++++++
77

    
78
All input dictionaries to the IAllocator must carry the following keys:
79

    
80
version
81
  the version of the protocol; this document
82
  specifies version 2
83

    
84
cluster_name
85
  the cluster name
86

    
87
cluster_tags
88
  the list of cluster tags
89

    
90
enabled_hypervisors
91
  the list of enabled hypervisors
92

    
93
request
94
  a dictionary containing the details of the request; the keys vary
95
  depending on the type of operation that's being requested, as
96
  explained in `Operation-specific input`_ below.
97

    
98
nodegroups
99
  a dictionary with the data for the cluster's node groups; it is keyed
100
  on the group UUID, and the values are a dictionary with the following
101
  keys:
102

    
103
  name
104
    the node group name
105
  alloc_policy
106
    the allocation policy of the node group (consult the semantics of
107
    this attribute in the :manpage:`gnt-group(8)` manpage)
108

    
109
instances
110
  a dictionary with the data for the current existing instance on the
111
  cluster, indexed by instance name; the contents are similar to the
112
  instance definitions for the allocate mode, with the addition of:
113

    
114
  admin_up
115
    if this instance is set to run (but not the actual status of the
116
    instance)
117

    
118
  nodes
119
    list of nodes on which this instance is placed; the primary node
120
    of the instance is always the first one
121

    
122
nodes
123
  dictionary with the data for the nodes in the cluster, indexed by
124
  the node name; the dict contains [*]_ :
125

    
126
  total_disk
127
    the total disk size of this node (mebibytes)
128

    
129
  free_disk
130
    the free disk space on the node
131

    
132
  total_memory
133
    the total memory size
134

    
135
  free_memory
136
    free memory on the node; note that currently this does not take
137
    into account the instances which are down on the node
138

    
139
  total_cpus
140
    the physical number of CPUs present on the machine; depending on
141
    the hypervisor, this might or might not be equal to how many CPUs
142
    the node operating system sees;
143

    
144
  primary_ip
145
    the primary IP address of the node
146

    
147
  secondary_ip
148
    the secondary IP address of the node (the one used for the DRBD
149
    replication); note that this can be the same as the primary one
150

    
151
  tags
152
    list with the tags of the node
153

    
154
  master_candidate:
155
    a boolean flag denoting whether this node is a master candidate
156

    
157
  drained:
158
    a boolean flag denoting whether this node is being drained
159

    
160
  offline:
161
    a boolean flag denoting whether this node is offline
162

    
163
  i_pri_memory:
164
    total memory required by primary instances
165

    
166
  i_pri_up_memory:
167
    total memory required by running primary instances
168

    
169
  group:
170
    the node group that this node belongs to
171

    
172
  No allocations should be made on nodes having either the ``drained``
173
  or ``offline`` flags set. More details about these of node status
174
  flags is available in the manpage :manpage:`ganeti(7)`.
175

    
176
.. [*] Note that no run-time data is present for offline, drained or
177
   non-vm_capable nodes; this means the tags total_memory,
178
   reserved_memory, free_memory, total_disk, free_disk, total_cpus,
179
   i_pri_memory and i_pri_up memory will be absent
180

    
181
Operation-specific input
182
++++++++++++++++++++++++
183

    
184
All input dictionaries to the IAllocator carry, in the ``request``
185
dictionary, detailed information about the operation that's being
186
requested. The required keys vary depending on the type of operation, as
187
follows.
188

    
189
In all cases, it includes:
190

    
191
  type
192
    the request type; this can be either ``allocate``, ``relocate``,
193
    ``change-group`` or ``node-evacuate``. The
194
    ``allocate`` request is used when a new instance needs to be placed
195
    on the cluster. The ``relocate`` request is used when an existing
196
    instance needs to be moved within its node group.
197

    
198
    The ``multi-evacuate`` protocol used to request that the script
199
    computes the optimal relocate solution for all secondary instances
200
    of the given nodes. It is now deprecated and needs only be
201
    implemented if backwards compatibility with Ganeti 2.4 and lower is
202
    needed.
203

    
204
    The ``change-group`` request is used to relocate multiple instances
205
    across multiple node groups. ``node-evacuate`` evacuates instances
206
    off their node(s). These are described in a separate :ref:`design
207
    document <multi-reloc-detailed-design>`.
208

    
209
For both allocate and relocate mode, the following extra keys are needed
210
in the ``request`` dictionary:
211

    
212
  name
213
    the name of the instance; if the request is a realocation, then this
214
    name will be found in the list of instances (see below), otherwise
215
    is the FQDN of the new instance; type *string*
216

    
217
  required_nodes
218
    how many nodes should the algorithm return; while this information
219
    can be deduced from the instace's disk template, it's better if
220
    this computation is left to Ganeti as then allocator scripts are
221
    less sensitive to changes to the disk templates; type *integer*
222

    
223
  disk_space_total
224
    the total disk space that will be used by this instance on the
225
    (new) nodes; again, this information can be computed from the list
226
    of instance disks and its template type, but Ganeti is better
227
    suited to compute it; type *integer*
228

    
229
.. pyassert::
230

    
231
   constants.DISK_ACCESS_SET == set([constants.DISK_RDONLY,
232
     constants.DISK_RDWR])
233

    
234
Allocation needs, in addition:
235

    
236
  disks
237
    list of dictionaries holding the disk definitions for this
238
    instance (in the order they are exported to the hypervisor):
239

    
240
    mode
241
      either :pyeval:`constants.DISK_RDONLY` or
242
      :pyeval:`constants.DISK_RDWR` denoting if the disk is read-only or
243
      writable
244

    
245
    size
246
      the size of this disk in mebibytes
247

    
248
  nics
249
    a list of dictionaries holding the network interfaces for this
250
    instance, containing:
251

    
252
    ip
253
      the IP address that Ganeti know for this instance, or null
254

    
255
    mac
256
      the MAC address for this interface
257

    
258
    bridge
259
      the bridge to which this interface will be connected
260

    
261
  vcpus
262
    the number of VCPUs for the instance
263

    
264
  disk_template
265
    the disk template for the instance
266

    
267
  memory
268
   the memory size for the instance
269

    
270
  os
271
   the OS type for the instance
272

    
273
  tags
274
    the list of the instance's tags
275

    
276
  hypervisor
277
    the hypervisor of this instance
278

    
279
Relocation:
280

    
281
  relocate_from
282
     a list of nodes to move the instance away from (note that with
283
     Ganeti 2.0, this list will always contain a single node, the
284
     current secondary of the instance); type *list of strings*
285

    
286
As for ``node-evacuate``, it needs the following request arguments:
287

    
288
  instances
289
    a list of instance names to evacuate; type *list of strings*
290

    
291
  evac_mode
292
    specify which instances to evacuate; one of ``primary-only``,
293
    ``secondary-only``, ``all``, type *string*
294

    
295
``change-group`` needs the following request arguments:
296

    
297
  instances
298
    a list of instance names whose group to change; type
299
    *list of strings*
300

    
301
  target_groups
302
    must either be the empty list, or contain a list of group UUIDs that
303
    should be considered for relocating instances to; type
304
    *list of strings*
305

    
306
Response message
307
~~~~~~~~~~~~~~~~
308

    
309
The response message is much more simple than the input one. It is
310
also a dict having three keys:
311

    
312
success
313
  a boolean value denoting if the allocation was successful or not
314

    
315
info
316
  a string with information from the scripts; if the allocation fails,
317
  this will be shown to the user
318

    
319
result
320
  the output of the algorithm; even if the algorithm failed
321
  (i.e. success is false), this must be returned as an empty list
322

    
323
  for allocate/relocate, this is the list of node(s) for the instance;
324
  note that the length of this list must equal the ``requested_nodes``
325
  entry in the input message, otherwise Ganeti will consider the result
326
  as failed
327

    
328
  for the ``node-evacuate`` and ``change-group`` modes, this is a
329
  dictionary containing, among other information, a list of lists of
330
  serialized opcodes; see the :ref:`design document
331
  <multi-reloc-result>` for a detailed description
332

    
333
.. note:: Current Ganeti version accepts either ``result`` or ``nodes``
334
   as a backwards-compatibility measure (older versions only supported
335
   ``nodes``)
336

    
337
Examples
338
--------
339

    
340
Input messages to scripts
341
~~~~~~~~~~~~~~~~~~~~~~~~~
342

    
343
Input message, new instance allocation (common elements are listed this
344
time, but not included in further examples below)::
345

    
346
  {
347
    "version": 2,
348
    "cluster_name": "cluster1.example.com",
349
    "cluster_tags": [],
350
    "enabled_hypervisors": [
351
      "xen-pvm"
352
    ],
353
    "nodegroups": {
354
      "f4e06e0d-528a-4963-a5ad-10f3e114232d": {
355
        "name": "default",
356
        "alloc_policy": "preferred"
357
      }
358
    },
359
    "instances": {
360
      "instance1.example.com": {
361
        "tags": [],
362
        "should_run": false,
363
        "disks": [
364
          {
365
            "mode": "w",
366
            "size": 64
367
          },
368
          {
369
            "mode": "w",
370
            "size": 512
371
          }
372
        ],
373
        "nics": [
374
          {
375
            "ip": null,
376
            "mac": "aa:00:00:00:60:bf",
377
            "bridge": "xen-br0"
378
          }
379
        ],
380
        "vcpus": 1,
381
        "disk_template": "plain",
382
        "memory": 128,
383
        "nodes": [
384
          "nodee1.com"
385
        ],
386
        "os": "debootstrap+default"
387
      },
388
      "instance2.example.com": {
389
        "tags": [],
390
        "should_run": false,
391
        "disks": [
392
          {
393
            "mode": "w",
394
            "size": 512
395
          },
396
          {
397
            "mode": "w",
398
            "size": 256
399
          }
400
        ],
401
        "nics": [
402
          {
403
            "ip": null,
404
            "mac": "aa:00:00:55:f8:38",
405
            "bridge": "xen-br0"
406
          }
407
        ],
408
        "vcpus": 1,
409
        "disk_template": "drbd",
410
        "memory": 512,
411
        "nodes": [
412
          "node2.example.com",
413
          "node3.example.com"
414
        ],
415
        "os": "debootstrap+default"
416
      }
417
    },
418
    "nodes": {
419
      "node1.example.com": {
420
        "total_disk": 858276,
421
        "primary_ip": "198.51.100.1",
422
        "secondary_ip": "192.0.2.1",
423
        "tags": [],
424
        "group": "f4e06e0d-528a-4963-a5ad-10f3e114232d",
425
        "free_memory": 3505,
426
        "free_disk": 856740,
427
        "total_memory": 4095
428
      },
429
      "node2.example.com": {
430
        "total_disk": 858240,
431
        "primary_ip": "198.51.100.2",
432
        "secondary_ip": "192.0.2.2",
433
        "tags": ["test"],
434
        "group": "f4e06e0d-528a-4963-a5ad-10f3e114232d",
435
        "free_memory": 3505,
436
        "free_disk": 848320,
437
        "total_memory": 4095
438
      },
439
      "node3.example.com.com": {
440
        "total_disk": 572184,
441
        "primary_ip": "198.51.100.3",
442
        "secondary_ip": "192.0.2.3",
443
        "tags": [],
444
        "group": "f4e06e0d-528a-4963-a5ad-10f3e114232d",
445
        "free_memory": 3505,
446
        "free_disk": 570648,
447
        "total_memory": 4095
448
      }
449
    },
450
    "request": {
451
      "type": "allocate",
452
      "name": "instance3.example.com",
453
      "required_nodes": 2,
454
      "disk_space_total": 3328,
455
      "disks": [
456
        {
457
          "mode": "w",
458
          "size": 1024
459
        },
460
        {
461
          "mode": "w",
462
          "size": 2048
463
        }
464
      ],
465
      "nics": [
466
        {
467
          "ip": null,
468
          "mac": "00:11:22:33:44:55",
469
          "bridge": null
470
        }
471
      ],
472
      "vcpus": 1,
473
      "disk_template": "drbd",
474
      "memory": 2048,
475
      "os": "debootstrap+default",
476
      "tags": [
477
        "type:test",
478
        "owner:foo"
479
      ],
480
      hypervisor: "xen-pvm"
481
    }
482
  }
483

    
484
Input message, reallocation::
485

    
486
  {
487
    "version": 2,
488
    ...
489
    "request": {
490
      "type": "relocate",
491
      "name": "instance2.example.com",
492
      "required_nodes": 1,
493
      "disk_space_total": 832,
494
      "relocate_from": [
495
        "node3.example.com"
496
      ]
497
    }
498
  }
499

    
500

    
501
Response messages
502
~~~~~~~~~~~~~~~~~
503
Successful response message::
504

    
505
  {
506
    "success": true,
507
    "info": "Allocation successful",
508
    "result": [
509
      "node2.example.com",
510
      "node1.example.com"
511
    ]
512
  }
513

    
514
Failed response message::
515

    
516
  {
517
    "success": false,
518
    "info": "Can't find a suitable node for position 2 (already selected: node2.example.com)",
519
    "result": []
520
  }
521

    
522
Successful node evacuation message::
523

    
524
  {
525
    "success": true,
526
    "info": "Request successful",
527
    "result": [
528
      [
529
        "instance1",
530
        "node3"
531
      ],
532
      [
533
        "instance2",
534
        "node1"
535
      ]
536
    ]
537
  }
538

    
539

    
540
Command line messages
541
~~~~~~~~~~~~~~~~~~~~~
542
::
543

    
544
  # gnt-instance add -t plain -m 2g --os-size 1g --swap-size 512m --iallocator hail -o debootstrap+default instance3
545
  Selected nodes for the instance: node1.example.com
546
  * creating instance disks...
547
  [...]
548

    
549
  # gnt-instance add -t plain -m 3400m --os-size 1g --swap-size 512m --iallocator hail -o debootstrap+default instance4
550
  Failure: prerequisites not met for this operation:
551
  Can't compute nodes using iallocator 'hail': Can't find a suitable node for position 1 (already selected: )
552

    
553
  # gnt-instance add -t drbd -m 1400m --os-size 1g --swap-size 512m --iallocator hail -o debootstrap+default instance5
554
  Failure: prerequisites not met for this operation:
555
  Can't compute nodes using iallocator 'hail': Can't find a suitable node for position 2 (already selected: node1.example.com)
556

    
557
Reference implementation
558
~~~~~~~~~~~~~~~~~~~~~~~~
559

    
560
Ganeti's default iallocator is "hail" which is available when "htools"
561
components have been enabled at build time (see :doc:`install-quick` for
562
more details).
563

    
564
.. vim: set textwidth=72 :
565
.. Local Variables:
566
.. mode: rst
567
.. fill-column: 72
568
.. End: