root / doc / design-2.1.rst @ 76bb661b
History | View | Annotate | Download (9.8 kB)
1 | 82a1c938 | Guido Trotter | ================= |
---|---|---|---|
2 | 82a1c938 | Guido Trotter | Ganeti 2.1 design |
3 | 82a1c938 | Guido Trotter | ================= |
4 | 82a1c938 | Guido Trotter | |
5 | 82a1c938 | Guido Trotter | This document describes the major changes in Ganeti 2.1 compared to |
6 | 82a1c938 | Guido Trotter | the 2.0 version. |
7 | 82a1c938 | Guido Trotter | |
8 | 82a1c938 | Guido Trotter | The 2.1 version will be a relatively small release. Its main aim is to avoid |
9 | 82a1c938 | Guido Trotter | changing too much of the core code, while addressing issues and adding new |
10 | 82a1c938 | Guido Trotter | features and improvements over 2.0, in a timely fashion. |
11 | 82a1c938 | Guido Trotter | |
12 | 82a1c938 | Guido Trotter | .. contents:: :depth: 3 |
13 | 82a1c938 | Guido Trotter | |
14 | 82a1c938 | Guido Trotter | Objective |
15 | 82a1c938 | Guido Trotter | ========= |
16 | 82a1c938 | Guido Trotter | |
17 | 82a1c938 | Guido Trotter | Ganeti 2.1 will add features to help further automatization of cluster |
18 | 82a1c938 | Guido Trotter | operations, further improbe scalability to even bigger clusters, and make it |
19 | 82a1c938 | Guido Trotter | easier to debug the Ganeti core. |
20 | 82a1c938 | Guido Trotter | |
21 | 82a1c938 | Guido Trotter | Background |
22 | 82a1c938 | Guido Trotter | ========== |
23 | 82a1c938 | Guido Trotter | |
24 | 82a1c938 | Guido Trotter | Overview |
25 | 82a1c938 | Guido Trotter | ======== |
26 | 82a1c938 | Guido Trotter | |
27 | 82a1c938 | Guido Trotter | Detailed design |
28 | 82a1c938 | Guido Trotter | =============== |
29 | 82a1c938 | Guido Trotter | |
30 | 82a1c938 | Guido Trotter | As for 2.0 we divide the 2.1 design into three areas: |
31 | 82a1c938 | Guido Trotter | |
32 | 587ff6fa | Guido Trotter | - core changes, which affect the master daemon/job queue/locking or all/most |
33 | 587ff6fa | Guido Trotter | logical units |
34 | 82a1c938 | Guido Trotter | - logical unit/feature changes |
35 | 82a1c938 | Guido Trotter | - external interface changes (eg. command line, os api, hooks, ...) |
36 | 82a1c938 | Guido Trotter | |
37 | 82a1c938 | Guido Trotter | Core changes |
38 | 82a1c938 | Guido Trotter | ------------ |
39 | 82a1c938 | Guido Trotter | |
40 | 82a1c938 | Guido Trotter | Feature changes |
41 | 82a1c938 | Guido Trotter | --------------- |
42 | 82a1c938 | Guido Trotter | |
43 | d1268971 | Guido Trotter | Redistribute Config |
44 | d1268971 | Guido Trotter | ~~~~~~~~~~~~~~~~~~~ |
45 | d1268971 | Guido Trotter | |
46 | d1268971 | Guido Trotter | Current State and shortcomings |
47 | d1268971 | Guido Trotter | ++++++++++++++++++++++++++++++ |
48 | d1268971 | Guido Trotter | Currently LURedistributeConfig triggers a copy of the updated configuration |
49 | d1268971 | Guido Trotter | file to all master candidates and of the ssconf files to all nodes. There are |
50 | d1268971 | Guido Trotter | other files which are maintained manually but which are important to keep in |
51 | d1268971 | Guido Trotter | sync. These are: |
52 | d1268971 | Guido Trotter | |
53 | d1268971 | Guido Trotter | - rapi SSL key certificate file (rapi.pem) (on master candidates) |
54 | d1268971 | Guido Trotter | - rapi user/password file rapi_users (on master candidates) |
55 | d1268971 | Guido Trotter | |
56 | d1268971 | Guido Trotter | Furthermore there are some files which are hypervisor specific but we may want |
57 | d1268971 | Guido Trotter | to keep in sync: |
58 | d1268971 | Guido Trotter | |
59 | d1268971 | Guido Trotter | - the xen-hvm hypervisor uses one shared file for all vnc passwords, and copies |
60 | d1268971 | Guido Trotter | the file once, during node add. This design is subject to revision to be able |
61 | d1268971 | Guido Trotter | to have different passwords for different groups of instances via the use of |
62 | d1268971 | Guido Trotter | hypervisor parameters, and to allow xen-hvm and kvm to use an equal system to |
63 | d1268971 | Guido Trotter | provide password-protected vnc sessions. In general, though, it would be |
64 | d1268971 | Guido Trotter | useful if the vnc password files were copied as well, to avoid unwanted vnc |
65 | d1268971 | Guido Trotter | password changes on instance failover/migrate. |
66 | d1268971 | Guido Trotter | |
67 | d1268971 | Guido Trotter | Optionally the admin may want to also ship files such as the global xend.conf |
68 | d1268971 | Guido Trotter | file, and the network scripts to all nodes. |
69 | d1268971 | Guido Trotter | |
70 | d1268971 | Guido Trotter | Proposed changes |
71 | d1268971 | Guido Trotter | ++++++++++++++++ |
72 | d1268971 | Guido Trotter | |
73 | d1268971 | Guido Trotter | RedistributeConfig will be changed to copy also the rapi files, and to call |
74 | d1268971 | Guido Trotter | every enabled hypervisor asking for a list of additional files to copy. We also |
75 | d1268971 | Guido Trotter | may want to add a global list of files on the cluster object, which will be |
76 | d1268971 | Guido Trotter | propagated as well, or a hook to calculate them. If we implement this feature |
77 | d1268971 | Guido Trotter | there should be a way to specify whether a file must be shipped to all nodes or |
78 | d1268971 | Guido Trotter | just master candidates. |
79 | d1268971 | Guido Trotter | |
80 | d1268971 | Guido Trotter | This code will be also shared (via tasklets or by other means, if tasklets are |
81 | d1268971 | Guido Trotter | not ready for 2.1) with the AddNode and SetNodeParams LUs (so that the relevant |
82 | d1268971 | Guido Trotter | files will be automatically shipped to new master candidates as they are set). |
83 | d1268971 | Guido Trotter | |
84 | 5b18ff3b | Guido Trotter | VNC Console Password |
85 | 5b18ff3b | Guido Trotter | ~~~~~~~~~~~~~~~~~~~~ |
86 | 5b18ff3b | Guido Trotter | |
87 | 5b18ff3b | Guido Trotter | Current State and shortcomings |
88 | 5b18ff3b | Guido Trotter | ++++++++++++++++++++++++++++++ |
89 | 5b18ff3b | Guido Trotter | |
90 | 5b18ff3b | Guido Trotter | Currently just the xen-hvm hypervisor supports setting a password to connect |
91 | 5b18ff3b | Guido Trotter | the the instances' VNC console, and has one common password stored in a file. |
92 | 5b18ff3b | Guido Trotter | |
93 | 5b18ff3b | Guido Trotter | This doesn't allow different passwords for different instances/groups of |
94 | 5b18ff3b | Guido Trotter | instances, and makes it necessary to remember to copy the file around the |
95 | 5b18ff3b | Guido Trotter | cluster when the password changes. |
96 | 5b18ff3b | Guido Trotter | |
97 | 5b18ff3b | Guido Trotter | Proposed changes |
98 | 5b18ff3b | Guido Trotter | ++++++++++++++++ |
99 | 5b18ff3b | Guido Trotter | |
100 | 5b18ff3b | Guido Trotter | We'll change the VNC password file to a vnc_password_file hypervisor parameter. |
101 | 5b18ff3b | Guido Trotter | This way it can have a cluster default, but also a different value for each |
102 | 5b18ff3b | Guido Trotter | instance. The VNC enabled hypervisors (xen and kvm) will publish all the |
103 | 5b18ff3b | Guido Trotter | password files in use through the cluster so that a redistribute-config will |
104 | 5b18ff3b | Guido Trotter | ship them to all nodes (see the Redistribute Config proposed changes above). |
105 | 5b18ff3b | Guido Trotter | |
106 | 5b18ff3b | Guido Trotter | The current VNC_PASSWORD_FILE constant will be removed, but its value will be |
107 | 5b18ff3b | Guido Trotter | used as the default HV_VNC_PASSWORD_FILE value, thus retaining backwards |
108 | 5b18ff3b | Guido Trotter | compatibility with 2.0. |
109 | 5b18ff3b | Guido Trotter | |
110 | 5b18ff3b | Guido Trotter | The code to export the list of VNC password files from the hypervisors to |
111 | 5b18ff3b | Guido Trotter | RedistributeConfig will be shared between the KVM and xen-hvm hypervisors. |
112 | 5b18ff3b | Guido Trotter | |
113 | 76bb661b | Guido Trotter | Disk/Net parameters |
114 | 76bb661b | Guido Trotter | ~~~~~~~~~~~~~~~~~~~ |
115 | 76bb661b | Guido Trotter | |
116 | 76bb661b | Guido Trotter | Current State and shortcomings |
117 | 76bb661b | Guido Trotter | ++++++++++++++++++++++++++++++ |
118 | 76bb661b | Guido Trotter | |
119 | 76bb661b | Guido Trotter | Currently disks and network interfaces have a few tweakable options and all the |
120 | 76bb661b | Guido Trotter | rest is left to a default we chose. We're finding that we need more and more to |
121 | 76bb661b | Guido Trotter | tweak some of these parameters, for example to disable barriers for DRBD |
122 | 76bb661b | Guido Trotter | devices, or allow striping for the LVM volumes. |
123 | 76bb661b | Guido Trotter | |
124 | 76bb661b | Guido Trotter | Moreover for many of these parameters it will be nice to have cluster-wide |
125 | 76bb661b | Guido Trotter | defaults, and then be able to change them per disk/interface. |
126 | 76bb661b | Guido Trotter | |
127 | 76bb661b | Guido Trotter | Proposed changes |
128 | 76bb661b | Guido Trotter | ++++++++++++++++ |
129 | 76bb661b | Guido Trotter | |
130 | 76bb661b | Guido Trotter | We will add new cluster level diskparams and netparams, which will contain all |
131 | 76bb661b | Guido Trotter | the tweakable parameters. All values which have a sensible cluster-wide default |
132 | 76bb661b | Guido Trotter | will go into this new structure while parameters which have unique values will not. |
133 | 76bb661b | Guido Trotter | |
134 | 76bb661b | Guido Trotter | Example of network parameters: |
135 | 76bb661b | Guido Trotter | - mode: bridge/route |
136 | 76bb661b | Guido Trotter | - link: for mode "bridge" the bridge to connect to, for mode route it can |
137 | 76bb661b | Guido Trotter | contain the routing table, or the destination interface |
138 | 76bb661b | Guido Trotter | |
139 | 76bb661b | Guido Trotter | Example of disk parameters: |
140 | 76bb661b | Guido Trotter | - stripe: lvm stripes |
141 | 76bb661b | Guido Trotter | - stripe_size: lvm stripe size |
142 | 76bb661b | Guido Trotter | - meta_flushes: drbd, enable/disable metadata "barriers" |
143 | 76bb661b | Guido Trotter | - data_flushes: drbd, enable/disable data "barriers" |
144 | 76bb661b | Guido Trotter | |
145 | 76bb661b | Guido Trotter | Some parameters are bound to be disk-type specific (drbd, vs lvm, vs files) or |
146 | 76bb661b | Guido Trotter | hypervisor specific (nic models for example), but for now they will all live in |
147 | 76bb661b | Guido Trotter | the same structure. Each component is supposed to validate only the parameters |
148 | 76bb661b | Guido Trotter | it knows about, and ganeti itself will make sure that no "globally unknown" |
149 | 76bb661b | Guido Trotter | parameters are added, and that no parameters have overridden meanings for |
150 | 76bb661b | Guido Trotter | different components. |
151 | 76bb661b | Guido Trotter | |
152 | 76bb661b | Guido Trotter | The parameters will be kept, as for the BEPARAMS into a "default" category, |
153 | 76bb661b | Guido Trotter | which will allow us to expand on by creating instance "classes" in the future. |
154 | 76bb661b | Guido Trotter | Instance classes is not a feature we plan implementing in 2.1, though. |
155 | 76bb661b | Guido Trotter | |
156 | 76bb661b | Guido Trotter | |
157 | 82a1c938 | Guido Trotter | External interface changes |
158 | 82a1c938 | Guido Trotter | -------------------------- |
159 | 82a1c938 | Guido Trotter | |
160 | b6cc971c | Guido Trotter | OS API |
161 | b6cc971c | Guido Trotter | ~~~~~~ |
162 | b6cc971c | Guido Trotter | |
163 | b6cc971c | Guido Trotter | The OS API of Ganeti 2.0 has been built with extensibility in mind. Since we |
164 | b6cc971c | Guido Trotter | pass everything as environment variables it's a lot easier to send new |
165 | b6cc971c | Guido Trotter | information to the OSes without breaking retrocompatibility. This section of |
166 | b6cc971c | Guido Trotter | the design outlines the proposed extensions to the API and their |
167 | b6cc971c | Guido Trotter | implementation. |
168 | b6cc971c | Guido Trotter | |
169 | b6cc971c | Guido Trotter | API Version Compatibility Handling |
170 | b6cc971c | Guido Trotter | ++++++++++++++++++++++++++++++++++ |
171 | b6cc971c | Guido Trotter | |
172 | b6cc971c | Guido Trotter | In 2.1 there will be a new OS API version (eg. 15), which should be mostly |
173 | b6cc971c | Guido Trotter | compatible with api 10, except for some new added variables. Since it's easy |
174 | b6cc971c | Guido Trotter | not to pass some variables we'll be able to handle Ganeti 2.0 OSes by just |
175 | b6cc971c | Guido Trotter | filtering out the newly added piece of information. We will still encourage |
176 | b6cc971c | Guido Trotter | OSes to declare support for the new API after checking that the new variables |
177 | b6cc971c | Guido Trotter | don't provide any conflict for them, and we will drop api 10 support after |
178 | b6cc971c | Guido Trotter | ganeti 2.1 has released. |
179 | b6cc971c | Guido Trotter | |
180 | b6cc971c | Guido Trotter | New Environment variables |
181 | b6cc971c | Guido Trotter | +++++++++++++++++++++++++ |
182 | b6cc971c | Guido Trotter | |
183 | b6cc971c | Guido Trotter | Some variables have never been added to the OS api but would definitely be |
184 | b6cc971c | Guido Trotter | useful for the OSes. We plan to add an INSTANCE_HYPERVISOR variable to allow |
185 | b6cc971c | Guido Trotter | the OS to make changes relevant to the virtualization the instance is going to |
186 | b6cc971c | Guido Trotter | use. Since this field is immutable for each instance, the os can tight the |
187 | b6cc971c | Guido Trotter | install without caring of making sure the instance can run under any |
188 | b6cc971c | Guido Trotter | virtualization technology. |
189 | b6cc971c | Guido Trotter | |
190 | b6cc971c | Guido Trotter | We also want the OS to know the particular hypervisor parameters, to be able to |
191 | b6cc971c | Guido Trotter | customize the install even more. Since the parameters can change, though, we |
192 | b6cc971c | Guido Trotter | will pass them only as an "FYI": if an OS ties some instance functionality to |
193 | b6cc971c | Guido Trotter | the value of a particular hypervisor parameter manual changes or a reinstall |
194 | b6cc971c | Guido Trotter | may be needed to adapt the instance to the new environment. This is not a |
195 | b6cc971c | Guido Trotter | regression as of today, because even if the OSes are left blind about this |
196 | b6cc971c | Guido Trotter | information, sometimes they still need to make compromises and cannot satisfy |
197 | b6cc971c | Guido Trotter | all possible parameter values. |
198 | b6cc971c | Guido Trotter | |
199 | b6cc971c | Guido Trotter | OS Parameters |
200 | b6cc971c | Guido Trotter | +++++++++++++ |
201 | b6cc971c | Guido Trotter | |
202 | b6cc971c | Guido Trotter | Currently we are assisting to some degree of "os proliferation" just to change |
203 | b6cc971c | Guido Trotter | a simple installation behavior. This means that the same OS gets installed on |
204 | b6cc971c | Guido Trotter | the cluster multiple times, with different names, to customize just one |
205 | b6cc971c | Guido Trotter | installation behavior. Usually such OSes try to share as much as possible |
206 | b6cc971c | Guido Trotter | through symlinks, but this still causes complications on the user side, |
207 | b6cc971c | Guido Trotter | especially when multiple parameters must be cross-matched. |
208 | b6cc971c | Guido Trotter | |
209 | b6cc971c | Guido Trotter | For example today if you want to install debian etch, lenny or squeeze you |
210 | b6cc971c | Guido Trotter | probably need to install the debootstrap OS multiple times, changing its |
211 | b6cc971c | Guido Trotter | configuration file, and calling it debootstrap-etch, debootstrap-lenny or |
212 | b6cc971c | Guido Trotter | debootstrap-squeeze. Furthermore if you have for example a "server" and a |
213 | b6cc971c | Guido Trotter | "development" environment which installs different packages/configuration files |
214 | b6cc971c | Guido Trotter | and must be available for all installs you'll probably end up with |
215 | b6cc971c | Guido Trotter | deboostrap-etch-server, debootstrap-etch-dev, debootrap-lenny-server, |
216 | b6cc971c | Guido Trotter | debootstrap-lenny-dev, etc. Crossing more than two parameters quickly becomes |
217 | b6cc971c | Guido Trotter | not manageable. |
218 | b6cc971c | Guido Trotter | |
219 | b6cc971c | Guido Trotter | In order to avoid this we plan to make OSes more customizable, by allowing |
220 | b6cc971c | Guido Trotter | arbitrary flags to be passed to them. These will be special "OS parameters" |
221 | b6cc971c | Guido Trotter | which will be handled by Ganeti mostly as hypervisor or be parameters. This |
222 | b6cc971c | Guido Trotter | slightly complicates the interface, but allows one OS (for example |
223 | b6cc971c | Guido Trotter | "debootstrap" to be customizable and not require copies to perform different |
224 | b6cc971c | Guido Trotter | cations). |
225 | b6cc971c | Guido Trotter | |
226 | b6cc971c | Guido Trotter | Each OS will be able to declare which parameters it supports by listing them |
227 | b6cc971c | Guido Trotter | one per line in a special "parameters" file in the OS dir. The parameters can |
228 | b6cc971c | Guido Trotter | have a per-os cluster default, or be specified at instance creation time. They |
229 | b6cc971c | Guido Trotter | will then be passed to the OS scripts as: INSTANCE_OS_PARAMETER_<NAME> with |
230 | b6cc971c | Guido Trotter | their specified value. The only value checking that will be performed is that |
231 | b6cc971c | Guido Trotter | the os parameter value is a string, with only "normal" characters in it. |
232 | b6cc971c | Guido Trotter | |
233 | b6cc971c | Guido Trotter | It will be impossible to change parameters for an instance, except at reinstall |
234 | b6cc971c | Guido Trotter | time. Upon reinstall with a different OS the parameters will be by default |
235 | b6cc971c | Guido Trotter | discarded and reset to the default (or passed) values, unless a special |
236 | b6cc971c | Guido Trotter | --keep-known-os-parameters flag is passed. |