root / doc / design-os.rst @ 56c934da
History | View | Annotate | Download (21.1 kB)
1 | e5eaa80a | Michele Tartara | =============================== |
---|---|---|---|
2 | e5eaa80a | Michele Tartara | Ganeti OS installation redesign |
3 | e5eaa80a | Michele Tartara | =============================== |
4 | e5eaa80a | Michele Tartara | |
5 | e5eaa80a | Michele Tartara | .. contents:: :depth: 3 |
6 | e5eaa80a | Michele Tartara | |
7 | e5eaa80a | Michele Tartara | This is a design document detailing a new OS installation procedure, which is |
8 | e5eaa80a | Michele Tartara | more secure, able to provide more features and easier to use for many common |
9 | e5eaa80a | Michele Tartara | tasks w.r.t. the current one. |
10 | e5eaa80a | Michele Tartara | |
11 | e5eaa80a | Michele Tartara | Current state and shortcomings |
12 | e5eaa80a | Michele Tartara | ============================== |
13 | e5eaa80a | Michele Tartara | |
14 | e5eaa80a | Michele Tartara | As of Ganeti 2.10, each instance is associated with an OS definition. An OS |
15 | e5eaa80a | Michele Tartara | definition is a set of scripts (``create``, ``export``, ``import``, ``rename``) |
16 | e5eaa80a | Michele Tartara | that are executed with root privileges on the primary host of the instance to |
17 | e5eaa80a | Michele Tartara | perform all the OS-related functionality (setting up an operating system inside |
18 | e5eaa80a | Michele Tartara | the disks of the instance being created, exporting/importing the instance, |
19 | e5eaa80a | Michele Tartara | renaming it). |
20 | e5eaa80a | Michele Tartara | |
21 | 56c934da | Jose A. Lopes | These scripts receive through environment variables a fixed set of parameters |
22 | e5eaa80a | Michele Tartara | related to the instance (such as the hypervisor, the name of the instance, the |
23 | e5eaa80a | Michele Tartara | number of disks, and their location) and a set of user defined parameters. |
24 | e5eaa80a | Michele Tartara | These parameters are also written in the configuration file of Ganeti, to allow |
25 | e5eaa80a | Michele Tartara | future reinstalls of the instance, and in various log files, namely: |
26 | e5eaa80a | Michele Tartara | |
27 | e5eaa80a | Michele Tartara | * node daemon log file: contains DEBUG strings of the ``/os_validate``, |
28 | e5eaa80a | Michele Tartara | ``/instance_os_add`` and ``/instance_start`` RPC calls. |
29 | e5eaa80a | Michele Tartara | |
30 | e5eaa80a | Michele Tartara | * master daemon log file: DEBUG strings related to the same RPC calls are stored |
31 | e5eaa80a | Michele Tartara | here as well. |
32 | e5eaa80a | Michele Tartara | |
33 | e5eaa80a | Michele Tartara | * commands log: the CLI commands that create a new instance, including their |
34 | e5eaa80a | Michele Tartara | parameters, are logged here. |
35 | e5eaa80a | Michele Tartara | |
36 | e5eaa80a | Michele Tartara | * RAPI log: the RAPI commands that create a new instance, including their |
37 | e5eaa80a | Michele Tartara | parameters, are logged here. |
38 | e5eaa80a | Michele Tartara | |
39 | e5eaa80a | Michele Tartara | * job logs: the job files stored in the job queue, or in its archive, contain |
40 | e5eaa80a | Michele Tartara | the parameters. |
41 | e5eaa80a | Michele Tartara | |
42 | e5eaa80a | Michele Tartara | The current situation presents a number of shortcomings: |
43 | e5eaa80a | Michele Tartara | |
44 | e5eaa80a | Michele Tartara | * Having the installation scripts run as root on the nodes doesn't allow |
45 | e5eaa80a | Michele Tartara | user-defined OS scripts, as they would pose a huge security issue. |
46 | e5eaa80a | Michele Tartara | Furthermore, even a script without malicious intentions might end up |
47 | e5eaa80a | Michele Tartara | distrupting a node because of a bug in it. |
48 | e5eaa80a | Michele Tartara | |
49 | e5eaa80a | Michele Tartara | * Ganeti cannot be used to create instances starting from user provided disk |
50 | e5eaa80a | Michele Tartara | images: even in the (hypothetical) case where the scripts are completely |
51 | e5eaa80a | Michele Tartara | secure and run not by root but by an unprivileged user with only the power to |
52 | e5eaa80a | Michele Tartara | mount arbitrary files as disk images, this is a security issue. It has been |
53 | e5eaa80a | Michele Tartara | proven that a carefully crafted file system might exploit kernel |
54 | e5eaa80a | Michele Tartara | vulnerabilities to gain control of the system. Therefore, directly mounting |
55 | e5eaa80a | Michele Tartara | images on the Ganeti nodes is not an option. |
56 | e5eaa80a | Michele Tartara | |
57 | e5eaa80a | Michele Tartara | * There is no way to inject files into an existing disk image. A common use case |
58 | e5eaa80a | Michele Tartara | is for the system administrator to provide a standard image of the system, to |
59 | e5eaa80a | Michele Tartara | be later personalized with the network configuration, private keys identifying |
60 | e5eaa80a | Michele Tartara | the machine, ssh keys of the users and so on. A possible workaround would be |
61 | e5eaa80a | Michele Tartara | for the scripts to mount the image (only if this is trusted!) and to receive |
62 | e5eaa80a | Michele Tartara | the configurations and ssh keys as user defined OS parameters. Unfortunately, |
63 | e5eaa80a | Michele Tartara | this is also not an option for security sensitive material (such as the ssh |
64 | e5eaa80a | Michele Tartara | keys) because the OS parameters are stored in many places on the system, as |
65 | e5eaa80a | Michele Tartara | already described above. |
66 | e5eaa80a | Michele Tartara | |
67 | e5eaa80a | Michele Tartara | * Most other virtualization software simply work with instance images, not with |
68 | e5eaa80a | Michele Tartara | installation scripts. This difference makes the interaction of Ganeti with |
69 | e5eaa80a | Michele Tartara | other software difficult. |
70 | e5eaa80a | Michele Tartara | |
71 | e5eaa80a | Michele Tartara | Proposed changes |
72 | e5eaa80a | Michele Tartara | ================ |
73 | e5eaa80a | Michele Tartara | |
74 | e5eaa80a | Michele Tartara | In order to fix the shortcomings of the current state, we plan to introduce the |
75 | 56c934da | Jose A. Lopes | following changes. |
76 | 56c934da | Jose A. Lopes | |
77 | 56c934da | Jose A. Lopes | |
78 | 56c934da | Jose A. Lopes | OS parameters categories |
79 | 56c934da | Jose A. Lopes | ++++++++++++++++++++++++ |
80 | 56c934da | Jose A. Lopes | |
81 | 56c934da | Jose A. Lopes | Change the OS parameters to have three categories: |
82 | 56c934da | Jose A. Lopes | |
83 | 56c934da | Jose A. Lopes | * ``public``: the current behavior. The parameter is logged and stored freely. |
84 | 56c934da | Jose A. Lopes | |
85 | 56c934da | Jose A. Lopes | * ``private``: the parameter is saved inside the Ganeti configuration (to allow |
86 | 56c934da | Jose A. Lopes | for instance reinstall) but it is not shown in logs, job logs, or passed back |
87 | 56c934da | Jose A. Lopes | via RAPI. |
88 | 56c934da | Jose A. Lopes | |
89 | 56c934da | Jose A. Lopes | * ``secret``: the parameter is not saved inside the Ganeti configuration. |
90 | 56c934da | Jose A. Lopes | Reinstalls are impossible unless the data is passed again. The parameter will |
91 | 56c934da | Jose A. Lopes | not appear in any log file. When a functionality is performed jointly by |
92 | 56c934da | Jose A. Lopes | multiple daemons (such as MasterD and LuxiD), currently Ganeti sometimes |
93 | 56c934da | Jose A. Lopes | serializes jobs on disk and later reloads them. Secret parameters will not be |
94 | 56c934da | Jose A. Lopes | serialized to disk. They will be passed around as part of the LUXI calls |
95 | 56c934da | Jose A. Lopes | exchanged by the daemons, and only kept in memory, in order to reduce their |
96 | 56c934da | Jose A. Lopes | accessibility as much as possible. In case of failure of the master node, |
97 | 56c934da | Jose A. Lopes | these parameters will be lost and cannot be recovered because they are not |
98 | 56c934da | Jose A. Lopes | serialized. As a result, the job cannot be taken over by the new master. This |
99 | 56c934da | Jose A. Lopes | is an expected and accepted side effect of jobs with secret parameters: if |
100 | 56c934da | Jose A. Lopes | they fail, they'll have to be restarted manually. |
101 | 56c934da | Jose A. Lopes | |
102 | 56c934da | Jose A. Lopes | |
103 | 56c934da | Jose A. Lopes | Metadata |
104 | 56c934da | Jose A. Lopes | ++++++++ |
105 | 56c934da | Jose A. Lopes | |
106 | 56c934da | Jose A. Lopes | In order to allow metadata to be sent inside the instance, a communication |
107 | 56c934da | Jose A. Lopes | mechanism between the instance and the host will be created. This mechanism |
108 | 56c934da | Jose A. Lopes | will be bidirectional (e.g.: to allow the setup process going on inside the |
109 | 56c934da | Jose A. Lopes | instance to communicate its progress to the host). Each instance will have |
110 | 56c934da | Jose A. Lopes | access exclusively to its own metadata, and it will be only able to communicate |
111 | 56c934da | Jose A. Lopes | with its host over this channel. This is the approach followed the |
112 | 56c934da | Jose A. Lopes | ``cloud-init`` tool and more details will be provided in the `Communication |
113 | 56c934da | Jose A. Lopes | mechanism and metadata service`_ section. |
114 | 56c934da | Jose A. Lopes | |
115 | 56c934da | Jose A. Lopes | |
116 | 56c934da | Jose A. Lopes | Installation procedure |
117 | 56c934da | Jose A. Lopes | ++++++++++++++++++++++ |
118 | 56c934da | Jose A. Lopes | |
119 | 56c934da | Jose A. Lopes | A new installation procedure will be introduced, with which it will be possible |
120 | 56c934da | Jose A. Lopes | to use an installation medium and run the OS scripts in an optional virtualized |
121 | 56c934da | Jose A. Lopes | environment and with an optional personalization package. There will be two |
122 | 56c934da | Jose A. Lopes | sets of parameters, namely, installation parameters, which are used mainly for |
123 | 56c934da | Jose A. Lopes | installs and reinstalls, and execution parameters, which are used in all the |
124 | 56c934da | Jose A. Lopes | other runs that are not part of an installation procedure. |
125 | 56c934da | Jose A. Lopes | |
126 | 56c934da | Jose A. Lopes | This set of installation parameters will allow, e.g., to attach an installation |
127 | 56c934da | Jose A. Lopes | floppy/cdrom/network, change the boot device order, or specify a disk image to |
128 | 56c934da | Jose A. Lopes | be used. Through this set of parameters, the administrator will have to provide |
129 | 56c934da | Jose A. Lopes | the hypervisor a location for an installation medium for the instance (e.g., a |
130 | 56c934da | Jose A. Lopes | boot disk, a network image, etc). This medium will carry out the installation |
131 | 56c934da | Jose A. Lopes | of the instance onto the instance's disks and will then be responsible for |
132 | 56c934da | Jose A. Lopes | getting the parameters for configuring the instance, such as, network |
133 | 56c934da | Jose A. Lopes | interfaces, IP address, and hostname. These parameters are taken from the |
134 | 56c934da | Jose A. Lopes | metadata. The installation parameters will be stored in the configuration of |
135 | 56c934da | Jose A. Lopes | Ganeti and used in future reinstalls, but not during normal execution. |
136 | 56c934da | Jose A. Lopes | |
137 | 56c934da | Jose A. Lopes | The instance is reinstalled using the same installation parameters from the |
138 | 56c934da | Jose A. Lopes | first installation. However, it will be the administrator's responsibility to |
139 | 56c934da | Jose A. Lopes | ensure that the any installation media is still available at the proper location |
140 | 56c934da | Jose A. Lopes | when a reinstall occurs. |
141 | 56c934da | Jose A. Lopes | |
142 | 56c934da | Jose A. Lopes | The parameter ``--os-parameters`` can still be used to specify the OS |
143 | 56c934da | Jose A. Lopes | parameters. However, without OS scripts, Ganeti cannot do more than a syntactic |
144 | 56c934da | Jose A. Lopes | check to validate the supplied OS parameters string. As a result, this string |
145 | 56c934da | Jose A. Lopes | will be directly passed to the instance as part of the metadata. If the |
146 | 56c934da | Jose A. Lopes | installation procedure is running inside a virtualized environment, then Ganeti |
147 | 56c934da | Jose A. Lopes | will take these parameters from the metadata and pass them to the OS scripts as |
148 | 56c934da | Jose A. Lopes | environment variables. |
149 | 56c934da | Jose A. Lopes | |
150 | 56c934da | Jose A. Lopes | * Use a disk image: |
151 | 56c934da | Jose A. Lopes | |
152 | 56c934da | Jose A. Lopes | Currently, it is already possible to specify an installation medium, such as, |
153 | 56c934da | Jose A. Lopes | a cdrom, but not a disk image. Therefore, a new parameter ``--os-image`` will |
154 | 56c934da | Jose A. Lopes | be used to specify the location of a disk image which will be dumped to the |
155 | 56c934da | Jose A. Lopes | instance's first disk before the instance is started. The location of the |
156 | 56c934da | Jose A. Lopes | image can be a URL and, if this is the case, Ganeti will download this image. |
157 | 56c934da | Jose A. Lopes | |
158 | 56c934da | Jose A. Lopes | * Run OS scripts: |
159 | 56c934da | Jose A. Lopes | |
160 | 56c934da | Jose A. Lopes | The parameter ``--os-type`` (short version: ``-o``), is currently used to |
161 | 56c934da | Jose A. Lopes | specify the OS scripts. This parameter will still be used to specify the OS |
162 | 56c934da | Jose A. Lopes | scripts with the difference that these OS scripts may optionally run inside a |
163 | 56c934da | Jose A. Lopes | virtualized environment for safety reasons, depending on whether they are |
164 | 56c934da | Jose A. Lopes | trusted or not. For more details on trusted and untrusted OS scripts, refer |
165 | 56c934da | Jose A. Lopes | to the `Installation process in a virtualized environment`_ section. |
166 | 56c934da | Jose A. Lopes | |
167 | 56c934da | Jose A. Lopes | * Personalization package |
168 | 56c934da | Jose A. Lopes | |
169 | 56c934da | Jose A. Lopes | As part of the instance creation command, it will be possible to indicate a |
170 | 56c934da | Jose A. Lopes | URL for a "personalization package", which is an archive containing a set of |
171 | 56c934da | Jose A. Lopes | files meant to be overlayed on top of the OS file system at the end of the |
172 | 56c934da | Jose A. Lopes | setup process and before the VM is started for the first time in normal mode. |
173 | 56c934da | Jose A. Lopes | Ganeti will provide a mechanism for receiving and unpacking this archive |
174 | 56c934da | Jose A. Lopes | whether the installation is being performed inside the virtualized environment |
175 | 56c934da | Jose A. Lopes | or not. |
176 | 56c934da | Jose A. Lopes | |
177 | 56c934da | Jose A. Lopes | The archive will be in TAR-GZIP format (with extension ``.tar.gz`` or |
178 | 56c934da | Jose A. Lopes | ``.tgz``) and contain the files according to the directory structure that will |
179 | 56c934da | Jose A. Lopes | be recreated on the installation disk. Files contained in this archive will |
180 | 56c934da | Jose A. Lopes | overwrite files with the same path created during the installation procedure |
181 | 56c934da | Jose A. Lopes | (if any). The URL of the "personalization package" will have to specify an |
182 | 56c934da | Jose A. Lopes | extension to identify the file format (in order to allow for more formats to |
183 | 56c934da | Jose A. Lopes | be supported in the future). The URL will be stored as part of the |
184 | 56c934da | Jose A. Lopes | configuration of the instance (therefore, the URL should not contain |
185 | 56c934da | Jose A. Lopes | confidential information, but the files there available can). |
186 | 56c934da | Jose A. Lopes | |
187 | 56c934da | Jose A. Lopes | It is up to the system administrator to ensure that a package is actually |
188 | 56c934da | Jose A. Lopes | available at that URL at install and reinstall time. The contents of the |
189 | 56c934da | Jose A. Lopes | package are allowed to change. E.g.: a system administrator might create a |
190 | 56c934da | Jose A. Lopes | package containing the private keys of the instance being created. When the |
191 | 56c934da | Jose A. Lopes | instance is reinstalled, a new package with new keys can be made available |
192 | 56c934da | Jose A. Lopes | there, thus allowing instance reinstall without the need to store keys. A |
193 | 56c934da | Jose A. Lopes | username and a password can be specified together with the URL. If the URL is |
194 | 56c934da | Jose A. Lopes | a HTTP(S) URL, they will be used as basic access authentication credentials to |
195 | 56c934da | Jose A. Lopes | access that URL. The username and password will not be saved in the config, |
196 | 56c934da | Jose A. Lopes | and will have to be provided again in case a reinstall is requested. |
197 | 56c934da | Jose A. Lopes | |
198 | 56c934da | Jose A. Lopes | The downloaded personalization package will not be stored locally on the node |
199 | 56c934da | Jose A. Lopes | for longer than it is needed while unpacking it and adding its files to the |
200 | 56c934da | Jose A. Lopes | instance being created. The personalization package will be overlayed on top |
201 | 56c934da | Jose A. Lopes | of the instance filesystem after the scripts that created it have been |
202 | 56c934da | Jose A. Lopes | executed. In order for the files in the package to be automatically overlayed |
203 | 56c934da | Jose A. Lopes | on top of the instance filesystem, it is required that the appliance is |
204 | 56c934da | Jose A. Lopes | actually able to mount the instance's disks. As a result, this will not work |
205 | 56c934da | Jose A. Lopes | for every filesystem. |
206 | 56c934da | Jose A. Lopes | |
207 | 56c934da | Jose A. Lopes | * Combine a disk image, OS scripts, and a personalization package |
208 | 56c934da | Jose A. Lopes | |
209 | 56c934da | Jose A. Lopes | It will possible to combine a disk image, OS scripts, and a personalization |
210 | 56c934da | Jose A. Lopes | package, both with or without a virtualized environment. There is one |
211 | 56c934da | Jose A. Lopes | exception which is if there are untrusted OS scripts. At least, an |
212 | 56c934da | Jose A. Lopes | installation medium or OS scripts should be specified. |
213 | 56c934da | Jose A. Lopes | |
214 | 56c934da | Jose A. Lopes | The disk image of the actual virtual appliance, which bootstraps the virtual |
215 | 56c934da | Jose A. Lopes | environment used in the installation procedure, will be read only, so that a |
216 | 56c934da | Jose A. Lopes | pristine copy of the appliance can be started every time a new instance needs |
217 | 56c934da | Jose A. Lopes | to be created and to further increase security. The data the instance needs |
218 | 56c934da | Jose A. Lopes | to write at runtime will only be stored in RAM and disappear as soon as the |
219 | 56c934da | Jose A. Lopes | instance is stopped. |
220 | 56c934da | Jose A. Lopes | |
221 | 56c934da | Jose A. Lopes | The parameter ``--enable-safe-install=yes|no`` will be used to give the |
222 | 56c934da | Jose A. Lopes | administrator control over whether to use a virtualized environment for the |
223 | 56c934da | Jose A. Lopes | installation procedure. By default, a virtualized environment will be used. |
224 | 56c934da | Jose A. Lopes | Note that some feature combinations, such as, using untrusted scripts, will |
225 | 56c934da | Jose A. Lopes | require the virtualized environment. In this case, Ganeti will not allow |
226 | 56c934da | Jose A. Lopes | disabling the virtualized environment. |
227 | e5eaa80a | Michele Tartara | |
228 | e5eaa80a | Michele Tartara | Implementation |
229 | e5eaa80a | Michele Tartara | ============== |
230 | e5eaa80a | Michele Tartara | |
231 | e5eaa80a | Michele Tartara | The implementation of this design will happen as an ordered sequence of steps, |
232 | e5eaa80a | Michele Tartara | of increasing impact on the system and, in some cases, dependent on each other: |
233 | e5eaa80a | Michele Tartara | |
234 | e5eaa80a | Michele Tartara | #. Private and secret instance parameters |
235 | e5eaa80a | Michele Tartara | #. Communication mechanism between host and instance |
236 | e5eaa80a | Michele Tartara | #. Metadata service |
237 | e5eaa80a | Michele Tartara | #. Personalization package (inside a virtualization environment) |
238 | 56c934da | Jose A. Lopes | #. Instance creation via a disk image |
239 | 56c934da | Jose A. Lopes | #. Instance creation inside a virtualized environment |
240 | e5eaa80a | Michele Tartara | |
241 | e5eaa80a | Michele Tartara | Some of these steps need to be more deeply specified w.r.t. what is already |
242 | e5eaa80a | Michele Tartara | written in the `Proposed changes`_ Section. Extra details will be provided in |
243 | e5eaa80a | Michele Tartara | the following subsections. |
244 | e5eaa80a | Michele Tartara | |
245 | e5eaa80a | Michele Tartara | Communication mechanism and metadata service |
246 | e5eaa80a | Michele Tartara | ++++++++++++++++++++++++++++++++++++++++++++ |
247 | e5eaa80a | Michele Tartara | |
248 | e5eaa80a | Michele Tartara | The communication mechanism and the metadata service are described together |
249 | 56c934da | Jose A. Lopes | because they are deeply tied. The communication mechanism will be made more |
250 | 56c934da | Jose A. Lopes | generic because it can be used for other purposes in the future (like allowing |
251 | 56c934da | Jose A. Lopes | instances to explicitly send commands to Ganeti, or to let Ganeti control a |
252 | 56c934da | Jose A. Lopes | helper instance, like the one hereby introduced for performing OS installs |
253 | 56c934da | Jose A. Lopes | inside a safe environment). |
254 | 56c934da | Jose A. Lopes | |
255 | 56c934da | Jose A. Lopes | The communication mechanism will be enabled automatically during an installation |
256 | 56c934da | Jose A. Lopes | procedure that requires a virtualized environment, but for backwards |
257 | 56c934da | Jose A. Lopes | compatibility it will be disabled when the instance is running normally, unless |
258 | 56c934da | Jose A. Lopes | it is explicitly requested. Specifically, a new parameter |
259 | 56c934da | Jose A. Lopes | ``--communication=yes|no`` (short version: ``-C``) will be added to |
260 | e5eaa80a | Michele Tartara | ``gnt-instance add`` and ``gnt-instance modify``. It will determine whether the |
261 | 56c934da | Jose A. Lopes | instance has a communication channel set to interact with the host and receive |
262 | 56c934da | Jose A. Lopes | metadata. The value of this parameter will be saved as part of the configuration |
263 | 56c934da | Jose A. Lopes | of the instance. |
264 | e5eaa80a | Michele Tartara | |
265 | e5eaa80a | Michele Tartara | When the communication mechanism is enabled, Ganeti will create a new network |
266 | e5eaa80a | Michele Tartara | interface inside the instance. This additional network interface will be the |
267 | e5eaa80a | Michele Tartara | last one in the instance, after all the user defined ones. On the host side, |
268 | e5eaa80a | Michele Tartara | this interface will only be accessible to the host itself, and not routed |
269 | e5eaa80a | Michele Tartara | outside the machine. |
270 | e5eaa80a | Michele Tartara | On this network interface, the instance will connect using the IP: |
271 | 36399e2a | Michele Tartara | 169.254.169.253 and netmask 255.255.255.0. |
272 | e5eaa80a | Michele Tartara | The host will be on the same network, with the IP address: 169.254.169.254. |
273 | e5eaa80a | Michele Tartara | |
274 | e5eaa80a | Michele Tartara | The way to create this interface depends on the specific hypervisor being used. |
275 | e5eaa80a | Michele Tartara | In KVM, it is possible to create a network interface inside the instance without |
276 | e5eaa80a | Michele Tartara | having a corresponding interface created on the host. Using a command like:: |
277 | e5eaa80a | Michele Tartara | |
278 | e5eaa80a | Michele Tartara | kvm -net nic -net \ |
279 | e5eaa80a | Michele Tartara | user,restrict=on,net=169.254.169.0/24,host=169.254.169.253, |
280 | e5eaa80a | Michele Tartara | guestfwd=tcp:169.254.169.254:80-tcp:127.0.0.1:8080 |
281 | e5eaa80a | Michele Tartara | |
282 | e5eaa80a | Michele Tartara | a network interface will be created inside the VM, part of the 169.254.169.0/24 |
283 | e5eaa80a | Michele Tartara | network, where the VM will have IP address .253 and the host port 8080 will be |
284 | e5eaa80a | Michele Tartara | reachable on port 80. |
285 | e5eaa80a | Michele Tartara | |
286 | e5eaa80a | Michele Tartara | In Xen, unfortunately, such a capability is not present, and an actual network |
287 | e5eaa80a | Michele Tartara | interface has to be created on the host (using the ``vif`` parameter of the Xen |
288 | e5eaa80a | Michele Tartara | configuration file). Each instance will have its corresponding ``vif`` network |
289 | e5eaa80a | Michele Tartara | interface on the host. These interfaces will not be connected to each other in |
290 | e5eaa80a | Michele Tartara | any way, and Ganeti will not configure them to allow traffic to be forwarded |
291 | e5eaa80a | Michele Tartara | beyond the host machine. The ``vif-route`` script of Xen might be helpful in |
292 | e5eaa80a | Michele Tartara | implementing this. |
293 | e5eaa80a | Michele Tartara | It will be the system administrator's responsibility to ensure that the extra |
294 | e5eaa80a | Michele Tartara | firewalling and routing rules specified on the host don't allow this |
295 | e5eaa80a | Michele Tartara | accidentally. |
296 | e5eaa80a | Michele Tartara | |
297 | e5eaa80a | Michele Tartara | The instance will be able to connect to 169.254.169.254:80, and issue GET |
298 | e5eaa80a | Michele Tartara | requests to an HTTP server that will provide the instance metadata. |
299 | e5eaa80a | Michele Tartara | |
300 | e5eaa80a | Michele Tartara | The choice of this IP address and port for accessing the metadata is done for |
301 | e5eaa80a | Michele Tartara | compatibility reasons with OpenStack's and Amazon EC2's ways of providing |
302 | e5eaa80a | Michele Tartara | metadata to the instance. The metadata will be provided by a single daemon, |
303 | e5eaa80a | Michele Tartara | which will determine what instance the request comes from and reply with the |
304 | e5eaa80a | Michele Tartara | metadata specific for that instance. |
305 | e5eaa80a | Michele Tartara | |
306 | e5eaa80a | Michele Tartara | Where possible, the metadata will be provided in a way compatible with Amazon |
307 | e5eaa80a | Michele Tartara | EC2, at:: |
308 | e5eaa80a | Michele Tartara | |
309 | e5eaa80a | Michele Tartara | http://169.254.169.254/<version>/meta-data/* |
310 | e5eaa80a | Michele Tartara | |
311 | e5eaa80a | Michele Tartara | If some metadata are Ganeti-specific and don't fit this structure, they will be |
312 | e5eaa80a | Michele Tartara | provided at:: |
313 | e5eaa80a | Michele Tartara | |
314 | e5eaa80a | Michele Tartara | http://169.254.169.254/ganeti/<version>/meta_data.json |
315 | e5eaa80a | Michele Tartara | |
316 | e5eaa80a | Michele Tartara | ``<version>`` is either a date in YYYY-MM-DD format, or ``latest`` to indicate |
317 | e5eaa80a | Michele Tartara | the most recent available protocol version. |
318 | e5eaa80a | Michele Tartara | |
319 | e5eaa80a | Michele Tartara | If needed in the future, this structure also allows us to support OpenStack's |
320 | e5eaa80a | Michele Tartara | metadata at:: |
321 | e5eaa80a | Michele Tartara | |
322 | e5eaa80a | Michele Tartara | http://169.254.169.254/openstack/<version>/meta_data.json |
323 | e5eaa80a | Michele Tartara | |
324 | e5eaa80a | Michele Tartara | A bi-directional, pipe-like communication channel will be provided. The instance |
325 | e5eaa80a | Michele Tartara | will be able to receive data from the host by a GET request at:: |
326 | e5eaa80a | Michele Tartara | |
327 | e5eaa80a | Michele Tartara | http://169.254.169.254/ganeti/<version>/read |
328 | e5eaa80a | Michele Tartara | |
329 | e5eaa80a | Michele Tartara | and to send data to the host by a POST request at:: |
330 | e5eaa80a | Michele Tartara | |
331 | e5eaa80a | Michele Tartara | http://169.254.169.254/ganeti/<version>/write |
332 | e5eaa80a | Michele Tartara | |
333 | e5eaa80a | Michele Tartara | As in a pipe, once the data are read, they will not be in the buffer anymore, so |
334 | e5eaa80a | Michele Tartara | subsequent GET requests to ``read`` will not return the same data twice. |
335 | e5eaa80a | Michele Tartara | Unlike a pipe, though, it will not be possible to perform blocking I/O |
336 | e5eaa80a | Michele Tartara | operations. |
337 | e5eaa80a | Michele Tartara | |
338 | e5eaa80a | Michele Tartara | The OS parameters will be accessible through a GET |
339 | e5eaa80a | Michele Tartara | request at:: |
340 | e5eaa80a | Michele Tartara | |
341 | e5eaa80a | Michele Tartara | http://169.254.169.254/ganeti/<version>/os/parameters.json |
342 | e5eaa80a | Michele Tartara | |
343 | e5eaa80a | Michele Tartara | as a JSON serialized dictionary having the parameter name as the key, and the |
344 | e5eaa80a | Michele Tartara | pair ``(<value>, <visibility>)`` as the value, where ``<value>`` is the |
345 | e5eaa80a | Michele Tartara | user-provided value of the parameter, and ``<visibility>`` is either ``public``, |
346 | e5eaa80a | Michele Tartara | ``private`` or ``secret``. |
347 | e5eaa80a | Michele Tartara | |
348 | 56c934da | Jose A. Lopes | The installation scripts to be run inside the virtualized environment will be |
349 | 56c934da | Jose A. Lopes | available at:: |
350 | e5eaa80a | Michele Tartara | |
351 | 56c934da | Jose A. Lopes | http://169.254.169.254/ganeti/<version>/os/scripts/<script_name> |
352 | e5eaa80a | Michele Tartara | |
353 | e5eaa80a | Michele Tartara | where ``<script_name>`` is the name of the script. |
354 | e5eaa80a | Michele Tartara | |
355 | e5eaa80a | Michele Tartara | |
356 | e5eaa80a | Michele Tartara | Rationale |
357 | e5eaa80a | Michele Tartara | --------- |
358 | e5eaa80a | Michele Tartara | |
359 | e5eaa80a | Michele Tartara | The choice of using a network interface for instance-host communication, as |
360 | e5eaa80a | Michele Tartara | opposed to VirtIO, XenBus or other methods, is due to the will of having a |
361 | e5eaa80a | Michele Tartara | generic, hypervisor-independent way of creating a communication channel, that |
362 | e5eaa80a | Michele Tartara | doesn't require unusual (para)virtualization drivers. |
363 | e5eaa80a | Michele Tartara | At the same time, a network interface was preferred over solutions involving |
364 | e5eaa80a | Michele Tartara | virtual floppy or USB devices because the latter tend to be detected and |
365 | e5eaa80a | Michele Tartara | configured by the guest operating systems, sometimes even in prominent positions |
366 | e5eaa80a | Michele Tartara | in the user interface, whereas it is fairly common to have an unconfigured |
367 | e5eaa80a | Michele Tartara | network interface in a system, usually without any negative side effects. |
368 | e5eaa80a | Michele Tartara | |
369 | e5eaa80a | Michele Tartara | |
370 | e5eaa80a | Michele Tartara | Installation process in a virtualized environment |
371 | e5eaa80a | Michele Tartara | +++++++++++++++++++++++++++++++++++++++++++++++++ |
372 | e5eaa80a | Michele Tartara | |
373 | e5eaa80a | Michele Tartara | In the new OS installation scenario, we distinguish between trusted and |
374 | e5eaa80a | Michele Tartara | untrusted code. |
375 | e5eaa80a | Michele Tartara | |
376 | e5eaa80a | Michele Tartara | The trusted installation code maintains the behavior of the current one and |
377 | e5eaa80a | Michele Tartara | requires no modifications, with the scripts running on the node the instance is |
378 | e5eaa80a | Michele Tartara | being created on. The untrusted code is stored in a subdirectory of the OS |
379 | e5eaa80a | Michele Tartara | definition called ``untrusted``. This directory contains scripts that are |
380 | e5eaa80a | Michele Tartara | equivalent to the already existing ones (``create``, ``export``, ``import``, |
381 | e5eaa80a | Michele Tartara | ``rename``) but that will be run inside an virtualized environment, to protect |
382 | e5eaa80a | Michele Tartara | the host from malicious tampering. |
383 | e5eaa80a | Michele Tartara | |
384 | e5eaa80a | Michele Tartara | The ``untrusted`` code is meant to either be untrusted itself, or to be trusted |
385 | e5eaa80a | Michele Tartara | code running operations that might be dangerous (such as mounting a |
386 | e5eaa80a | Michele Tartara | user-provided image). |
387 | e5eaa80a | Michele Tartara | |
388 | e5eaa80a | Michele Tartara | By default, all new OS definitions will have to be explicitly marked as trusted |
389 | e5eaa80a | Michele Tartara | by the cluster administrator (with a new ``gnt-os modify`` command) before they |
390 | e5eaa80a | Michele Tartara | can run code on the host. Otherwise, only the untrusted part of the code will be |
391 | e5eaa80a | Michele Tartara | allowed to run, inside the virtual appliance. For backwards compatibility |
392 | e5eaa80a | Michele Tartara | reasons, when upgrading an existing cluster, all the installed OSes will be |
393 | e5eaa80a | Michele Tartara | marked as trusted, so that they can keep running with no changes. |
394 | e5eaa80a | Michele Tartara | |
395 | e5eaa80a | Michele Tartara | In order to allow for the highest flexibility, if both a trusted and an |
396 | e5eaa80a | Michele Tartara | untrusted script are provided for the same operation (i.e. ``create``), both of |
397 | e5eaa80a | Michele Tartara | them will be executed at the same time, one on the host, and one inside the |
398 | e5eaa80a | Michele Tartara | installation appliance. They will be allowed to communicate with each other |
399 | e5eaa80a | Michele Tartara | through the already described communication mechanism, in order to orchestrate |
400 | e5eaa80a | Michele Tartara | their execution (e.g.: the untrusted code might execute the installation, while |
401 | e5eaa80a | Michele Tartara | the trusted one receives status updates from it and delivers them to a user |
402 | e5eaa80a | Michele Tartara | interface). |
403 | e5eaa80a | Michele Tartara | |
404 | e5eaa80a | Michele Tartara | The cluster administrator will have an option to completely disable scripts |
405 | e5eaa80a | Michele Tartara | running on the host, leaving only the ones running in the VM. |
406 | e5eaa80a | Michele Tartara | |
407 | e5eaa80a | Michele Tartara | Ganeti will provide a script to be run at install time that can be used to |
408 | e5eaa80a | Michele Tartara | create the virtualized environment that will perform the OS installation of new |
409 | e5eaa80a | Michele Tartara | instances. |
410 | e5eaa80a | Michele Tartara | This script will build a debootstrapped basic debian system including a software |
411 | e5eaa80a | Michele Tartara | that will read the metadata, setup the environment variables and launch the |
412 | e5eaa80a | Michele Tartara | installation scripts inside the virtualized environment. The script will also |
413 | e5eaa80a | Michele Tartara | provide hooks for personalization. |
414 | e5eaa80a | Michele Tartara | |
415 | e5eaa80a | Michele Tartara | It will also be possible to use other self-made virtualized environments, as |
416 | e5eaa80a | Michele Tartara | long as they connect to Ganeti over the described communication mechanism and |
417 | e5eaa80a | Michele Tartara | they know how to read and use the provided metadata to create a new instance. |
418 | e5eaa80a | Michele Tartara | |
419 | e5eaa80a | Michele Tartara | While performing an installation in the virtualized environment, a |
420 | e5eaa80a | Michele Tartara | personalizable timeout will be used to detect possible problems with the |
421 | e5eaa80a | Michele Tartara | installation process, and to kill the virtualized environment. The timeout will |
422 | e5eaa80a | Michele Tartara | be optional and set on a cluster basis by the administrator. If set, it will be |
423 | e5eaa80a | Michele Tartara | the total time allowed to setup an instance inside the appliance. It is mainly |
424 | e5eaa80a | Michele Tartara | meant as a safety measure to prevent an instance taken over by malicious scripts |
425 | e5eaa80a | Michele Tartara | to be available for a long time. |
426 | e5eaa80a | Michele Tartara | |
427 | e5eaa80a | Michele Tartara | .. vim: set textwidth=72 : |
428 | e5eaa80a | Michele Tartara | .. Local Variables: |
429 | e5eaa80a | Michele Tartara | .. mode: rst |
430 | e5eaa80a | Michele Tartara | .. fill-column: 72 |
431 | e5eaa80a | Michele Tartara | .. End: |