root / doc / design-bulk-create.rst @ 213ffbb1
History | View | Annotate | Download (3.2 kB)
1 | 4e27516b | René Nussbaumer | ================== |
---|---|---|---|
2 | 4e27516b | René Nussbaumer | Ganeti Bulk Create |
3 | 4e27516b | René Nussbaumer | ================== |
4 | 4e27516b | René Nussbaumer | |
5 | 4e27516b | René Nussbaumer | .. contents:: :depth: 4 |
6 | 4e27516b | René Nussbaumer | .. highlight:: python |
7 | 4e27516b | René Nussbaumer | |
8 | 4e27516b | René Nussbaumer | Current state and shortcomings |
9 | 4e27516b | René Nussbaumer | ============================== |
10 | 4e27516b | René Nussbaumer | |
11 | 4e27516b | René Nussbaumer | Creation of instances happens a lot. A fair load is done by just |
12 | 4e27516b | René Nussbaumer | creating instances and due to bad allocation shifting them around later |
13 | 4e27516b | René Nussbaumer | again. Additionally, if you turn up a new cluster you already know a |
14 | 4e27516b | René Nussbaumer | bunch of instances, which need to exists on the cluster. Doing this |
15 | 4e27516b | René Nussbaumer | one-by-one is not only cumbersome but might also fail, due to lack of |
16 | 4e27516b | René Nussbaumer | resources or lead to badly balanced clusters. |
17 | 4e27516b | René Nussbaumer | |
18 | 4e27516b | René Nussbaumer | Since the early Ganeti 2.0 alpha version there is a ``gnt-instance |
19 | 4e27516b | René Nussbaumer | batch-create`` command to allocate a bunch of instances based on a json |
20 | 4e27516b | René Nussbaumer | file. This feature, however, doesn't take any advantages of iallocator |
21 | 4e27516b | René Nussbaumer | and submits jobs in a serialized manner. |
22 | 4e27516b | René Nussbaumer | |
23 | 4e27516b | René Nussbaumer | |
24 | 4e27516b | René Nussbaumer | Proposed changes |
25 | 4e27516b | René Nussbaumer | ---------------- |
26 | 4e27516b | René Nussbaumer | |
27 | 4e27516b | René Nussbaumer | To overcome this shortcoming we would extend the current iallocator |
28 | 4e27516b | René Nussbaumer | interface to allow bulk requests. On the Ganeti side, a new opcode is |
29 | 4e27516b | René Nussbaumer | introduced to handle the bulk creation and returning the resulting |
30 | 4e27516b | René Nussbaumer | placement from the IAllocator_. |
31 | 4e27516b | René Nussbaumer | |
32 | 4e27516b | René Nussbaumer | |
33 | 4e27516b | René Nussbaumer | Problems |
34 | 4e27516b | René Nussbaumer | -------- |
35 | 4e27516b | René Nussbaumer | |
36 | 4e27516b | René Nussbaumer | Due to the design of chained jobs, we can guarantee, that with the state |
37 | 4e27516b | René Nussbaumer | at which the ``multi-alloc`` opcode is run, all of the instances will |
38 | 4e27516b | René Nussbaumer | fit (or all won't). But we can't guarantee that once the instance |
39 | 4e27516b | René Nussbaumer | creation requests were submit, no other jobs have sneaked in between. |
40 | 4e27516b | René Nussbaumer | This might still lead to failing jobs because the resources have changed |
41 | 4e27516b | René Nussbaumer | in the meantime. |
42 | 4e27516b | René Nussbaumer | |
43 | 4e27516b | René Nussbaumer | |
44 | 4e27516b | René Nussbaumer | Implementation |
45 | 4e27516b | René Nussbaumer | ============== |
46 | 4e27516b | René Nussbaumer | |
47 | 4e27516b | René Nussbaumer | |
48 | 4e27516b | René Nussbaumer | IAllocator |
49 | 4e27516b | René Nussbaumer | ---------- |
50 | 4e27516b | René Nussbaumer | |
51 | 4e27516b | René Nussbaumer | A new additional ``type`` will be added called ``multi-allocate`` to |
52 | 4e27516b | René Nussbaumer | distinguish between normal and bulk operation. For the bulk operation |
53 | 4e27516b | René Nussbaumer | the ``request`` will be a finite list of request dicts. |
54 | 4e27516b | René Nussbaumer | |
55 | 4e27516b | René Nussbaumer | If ``multi-allocate`` is declared, ``request`` must exist and is a list |
56 | 4e27516b | René Nussbaumer | of ``request`` dicts as described in :doc:`Operation specific input |
57 | 4e27516b | René Nussbaumer | <iallocator>`. The ``result`` then is a list of instance name and node |
58 | 4e27516b | René Nussbaumer | placements in the order of the ``request`` field. |
59 | 4e27516b | René Nussbaumer | |
60 | 4e27516b | René Nussbaumer | In addition, the old ``allocate`` request type will be deprecated and at |
61 | 4e27516b | René Nussbaumer | latest in Ganeti 2.8 incooperated into this new request. Current code |
62 | 4e27516b | René Nussbaumer | will need slight adaption to work with the new request. This needs |
63 | 4e27516b | René Nussbaumer | careful testing. |
64 | 4e27516b | René Nussbaumer | |
65 | 4e27516b | René Nussbaumer | |
66 | 4e27516b | René Nussbaumer | OpInstanceBulkAdd |
67 | 4e27516b | René Nussbaumer | ----------------- |
68 | 4e27516b | René Nussbaumer | |
69 | 4e27516b | René Nussbaumer | We add a new opcode ``OpInstanceBulkAdd``. It receives a list of |
70 | 4e27516b | René Nussbaumer | ``OpInstanceCreate`` on the ``instances`` field. This is done to make |
71 | 4e27516b | René Nussbaumer | sure, that these two loosely coupled opcodes do not get out of sync. On |
72 | 4e27516b | René Nussbaumer | the RAPI side, however, this just is a list of instance create |
73 | 4e27516b | René Nussbaumer | definitions. And the client is adapted accordingly. |
74 | 4e27516b | René Nussbaumer | |
75 | 4e27516b | René Nussbaumer | The opcode itself does some sanity checks on the instance creation |
76 | 4e27516b | René Nussbaumer | opcodes which includes: |
77 | 4e27516b | René Nussbaumer | |
78 | 4e27516b | René Nussbaumer | * ``mode`` is not set |
79 | 4e27516b | René Nussbaumer | * ``pnode`` and ``snodes`` is not set |
80 | 4e27516b | René Nussbaumer | * ``iallocator`` is not set |
81 | 4e27516b | René Nussbaumer | |
82 | 4e27516b | René Nussbaumer | Any of the above error will be aborted with ``OpPrereqError``. Once the |
83 | 4e27516b | René Nussbaumer | list has been verified it is handed to the ``iallocator`` as described |
84 | 4e27516b | René Nussbaumer | in IAllocator_. Upon success we then return the result of the |
85 | 4e27516b | René Nussbaumer | IAllocator_ call. |
86 | 4e27516b | René Nussbaumer | |
87 | 4e27516b | René Nussbaumer | At this point the current instance allocation would work with the |
88 | 4e27516b | René Nussbaumer | resources available on the cluster as perceived upon |
89 | 4e27516b | René Nussbaumer | ``OpInstanceBulkAdd`` invocation. However, there might be corner cases |
90 | 4e27516b | René Nussbaumer | where this is not true as described in Problems_. |
91 | 4e27516b | René Nussbaumer | |
92 | 4e27516b | René Nussbaumer | |
93 | 4e27516b | René Nussbaumer | .. vim: set textwidth=72 : |
94 | 4e27516b | René Nussbaumer | .. Local Variables: |
95 | 4e27516b | René Nussbaumer | .. mode: rst |
96 | 4e27516b | René Nussbaumer | .. fill-column: 72 |
97 | 4e27516b | René Nussbaumer | .. End: |