root / doc / install.rst @ 7ed400f0
History | View | Annotate | Download (26.4 kB)
1 | 28e15341 | Iustin Pop | Ganeti installation tutorial |
---|---|---|---|
2 | 28e15341 | Iustin Pop | ============================ |
3 | 28e15341 | Iustin Pop | |
4 | fd07c6b3 | Iustin Pop | Documents Ganeti version |version| |
5 | 28e15341 | Iustin Pop | |
6 | 28e15341 | Iustin Pop | .. contents:: |
7 | 28e15341 | Iustin Pop | |
8 | c71a1a3d | Iustin Pop | .. highlight:: text |
9 | c71a1a3d | Iustin Pop | |
10 | 28e15341 | Iustin Pop | Introduction |
11 | 28e15341 | Iustin Pop | ------------ |
12 | 28e15341 | Iustin Pop | |
13 | 28e15341 | Iustin Pop | Ganeti is a cluster virtualization management system based on Xen or |
14 | c71a1a3d | Iustin Pop | KVM. This document explains how to bootstrap a Ganeti node (Xen *dom0*, |
15 | c71a1a3d | Iustin Pop | the host Linux system for KVM), create a running cluster and install |
16 | c71a1a3d | Iustin Pop | virtual instances (Xen *domUs*, KVM guests). You need to repeat most of |
17 | c71a1a3d | Iustin Pop | the steps in this document for every node you want to install, but of |
18 | c71a1a3d | Iustin Pop | course we recommend creating some semi-automatic procedure if you plan |
19 | c71a1a3d | Iustin Pop | to deploy Ganeti on a medium/large scale. |
20 | 28e15341 | Iustin Pop | |
21 | 28e15341 | Iustin Pop | A basic Ganeti terminology glossary is provided in the introductory |
22 | c71a1a3d | Iustin Pop | section of the :doc:`admin`. Please refer to that document if you are |
23 | c71a1a3d | Iustin Pop | uncertain about the terms we are using. |
24 | 28e15341 | Iustin Pop | |
25 | c71a1a3d | Iustin Pop | Ganeti has been developed for Linux and should be distribution-agnostic. |
26 | 28e15341 | Iustin Pop | This documentation will use Debian Lenny as an example system but the |
27 | c71a1a3d | Iustin Pop | examples can be translated to any other distribution. You are expected |
28 | c71a1a3d | Iustin Pop | to be familiar with your distribution, its package management system, |
29 | c71a1a3d | Iustin Pop | and Xen or KVM before trying to use Ganeti. |
30 | 28e15341 | Iustin Pop | |
31 | 28e15341 | Iustin Pop | This document is divided into two main sections: |
32 | 28e15341 | Iustin Pop | |
33 | 28e15341 | Iustin Pop | - Installation of the base system and base components |
34 | 28e15341 | Iustin Pop | |
35 | 28e15341 | Iustin Pop | - Configuration of the environment for Ganeti |
36 | 28e15341 | Iustin Pop | |
37 | 28e15341 | Iustin Pop | Each of these is divided into sub-sections. While a full Ganeti system |
38 | c71a1a3d | Iustin Pop | will need all of the steps specified, some are not strictly required for |
39 | c71a1a3d | Iustin Pop | every environment. Which ones they are, and why, is specified in the |
40 | c71a1a3d | Iustin Pop | corresponding sections. |
41 | 28e15341 | Iustin Pop | |
42 | 28e15341 | Iustin Pop | Installing the base system and base components |
43 | 28e15341 | Iustin Pop | ---------------------------------------------- |
44 | 28e15341 | Iustin Pop | |
45 | 28e15341 | Iustin Pop | Hardware requirements |
46 | 28e15341 | Iustin Pop | +++++++++++++++++++++ |
47 | 28e15341 | Iustin Pop | |
48 | c71a1a3d | Iustin Pop | Any system supported by your Linux distribution is fine. 64-bit systems |
49 | c71a1a3d | Iustin Pop | are better as they can support more memory. |
50 | 28e15341 | Iustin Pop | |
51 | c71a1a3d | Iustin Pop | Any disk drive recognized by Linux (``IDE``/``SCSI``/``SATA``/etc.) is |
52 | c71a1a3d | Iustin Pop | supported in Ganeti. Note that no shared storage (e.g. ``SAN``) is |
53 | c71a1a3d | Iustin Pop | needed to get high-availability features (but of course, one can be used |
54 | c71a1a3d | Iustin Pop | to store the images). It is highly recommended to use more than one disk |
55 | c71a1a3d | Iustin Pop | drive to improve speed. But Ganeti also works with one disk per machine. |
56 | 28e15341 | Iustin Pop | |
57 | 28e15341 | Iustin Pop | Installing the base system |
58 | 28e15341 | Iustin Pop | ++++++++++++++++++++++++++ |
59 | 28e15341 | Iustin Pop | |
60 | 28e15341 | Iustin Pop | **Mandatory** on all nodes. |
61 | 28e15341 | Iustin Pop | |
62 | 28e15341 | Iustin Pop | It is advised to start with a clean, minimal install of the operating |
63 | c71a1a3d | Iustin Pop | system. The only requirement you need to be aware of at this stage is to |
64 | c71a1a3d | Iustin Pop | partition leaving enough space for a big (**minimum** 20GiB) LVM volume |
65 | c71a1a3d | Iustin Pop | group which will then host your instance filesystems, if you want to use |
66 | c71a1a3d | Iustin Pop | all Ganeti features. The volume group name Ganeti uses (by default) is |
67 | c71a1a3d | Iustin Pop | ``xenvg``. |
68 | 28e15341 | Iustin Pop | |
69 | c71a1a3d | Iustin Pop | You can also use file-based storage only, without LVM, but this setup is |
70 | c71a1a3d | Iustin Pop | not detailed in this document. |
71 | 28e15341 | Iustin Pop | |
72 | 7ed400f0 | Stratos Psomadakis | If you choose to use RBD-based instances, there's no need for LVM |
73 | 7ed400f0 | Stratos Psomadakis | provisioning. However, this feature is experimental, and is not |
74 | 7ed400f0 | Stratos Psomadakis | recommended for production clusters. |
75 | 7ed400f0 | Stratos Psomadakis | |
76 | 28e15341 | Iustin Pop | While you can use an existing system, please note that the Ganeti |
77 | 28e15341 | Iustin Pop | installation is intrusive in terms of changes to the system |
78 | 28e15341 | Iustin Pop | configuration, and it's best to use a newly-installed system without |
79 | 28e15341 | Iustin Pop | important data on it. |
80 | 28e15341 | Iustin Pop | |
81 | 28e15341 | Iustin Pop | Also, for best results, it's advised that the nodes have as much as |
82 | 28e15341 | Iustin Pop | possible the same hardware and software configuration. This will make |
83 | 28e15341 | Iustin Pop | administration much easier. |
84 | 28e15341 | Iustin Pop | |
85 | 28e15341 | Iustin Pop | Hostname issues |
86 | 28e15341 | Iustin Pop | ~~~~~~~~~~~~~~~ |
87 | 28e15341 | Iustin Pop | |
88 | 28e15341 | Iustin Pop | Note that Ganeti requires the hostnames of the systems (i.e. what the |
89 | 28e15341 | Iustin Pop | ``hostname`` command outputs to be a fully-qualified name, not a short |
90 | 28e15341 | Iustin Pop | name. In other words, you should use *node1.example.com* as a hostname |
91 | 28e15341 | Iustin Pop | and not just *node1*. |
92 | 28e15341 | Iustin Pop | |
93 | 28e15341 | Iustin Pop | .. admonition:: Debian |
94 | 28e15341 | Iustin Pop | |
95 | 28e15341 | Iustin Pop | Debian Lenny and Etch configures the hostname differently than you |
96 | 28e15341 | Iustin Pop | need it for Ganeti. For example, this is what Etch puts in |
97 | 28e15341 | Iustin Pop | ``/etc/hosts`` in certain situations:: |
98 | 28e15341 | Iustin Pop | |
99 | 28e15341 | Iustin Pop | 127.0.0.1 localhost |
100 | 28e15341 | Iustin Pop | 127.0.1.1 node1.example.com node1 |
101 | 28e15341 | Iustin Pop | |
102 | 28e15341 | Iustin Pop | but for Ganeti you need to have:: |
103 | 28e15341 | Iustin Pop | |
104 | 28e15341 | Iustin Pop | 127.0.0.1 localhost |
105 | 926feaf1 | Manuel Franceschini | 192.0.2.1 node1.example.com node1 |
106 | 28e15341 | Iustin Pop | |
107 | 926feaf1 | Manuel Franceschini | replacing ``192.0.2.1`` with your node's address. Also, the file |
108 | 28e15341 | Iustin Pop | ``/etc/hostname`` which configures the hostname of the system |
109 | 28e15341 | Iustin Pop | should contain ``node1.example.com`` and not just ``node1`` (you |
110 | 28e15341 | Iustin Pop | need to run the command ``/etc/init.d/hostname.sh start`` after |
111 | 28e15341 | Iustin Pop | changing the file). |
112 | 28e15341 | Iustin Pop | |
113 | 1232284c | Guido Trotter | .. admonition:: Why a fully qualified host name |
114 | 1232284c | Guido Trotter | |
115 | 7faf5110 | Michael Hanselmann | Although most distributions use only the short name in the |
116 | 7faf5110 | Michael Hanselmann | /etc/hostname file, we still think Ganeti nodes should use the full |
117 | 7faf5110 | Michael Hanselmann | name. The reason for this is that calling 'hostname --fqdn' requires |
118 | 7faf5110 | Michael Hanselmann | the resolver library to work and is a 'guess' via heuristics at what |
119 | 7faf5110 | Michael Hanselmann | is your domain name. Since Ganeti can be used among other things to |
120 | 7faf5110 | Michael Hanselmann | host DNS servers, we don't want to depend on them as much as |
121 | 7faf5110 | Michael Hanselmann | possible, and we'd rather have the uname() syscall return the full |
122 | 7faf5110 | Michael Hanselmann | node name. |
123 | 7faf5110 | Michael Hanselmann | |
124 | 7faf5110 | Michael Hanselmann | We haven't ever found any breakage in using a full hostname on a |
125 | 7faf5110 | Michael Hanselmann | Linux system, and anyway we recommend to have only a minimal |
126 | 7faf5110 | Michael Hanselmann | installation on Ganeti nodes, and to use instances (or other |
127 | 7faf5110 | Michael Hanselmann | dedicated machines) to run the rest of your network services. By |
128 | 7faf5110 | Michael Hanselmann | doing this you can change the /etc/hostname file to contain an FQDN |
129 | 7faf5110 | Michael Hanselmann | without the fear of breaking anything unrelated. |
130 | 1232284c | Guido Trotter | |
131 | 1232284c | Guido Trotter | |
132 | 756d5ec3 | Guido Trotter | Installing The Hypervisor |
133 | 756d5ec3 | Guido Trotter | +++++++++++++++++++++++++ |
134 | 28e15341 | Iustin Pop | |
135 | 28e15341 | Iustin Pop | **Mandatory** on all nodes. |
136 | 28e15341 | Iustin Pop | |
137 | 756d5ec3 | Guido Trotter | While Ganeti is developed with the ability to modularly run on different |
138 | 7faf5110 | Michael Hanselmann | virtualization environments in mind the only two currently useable on a |
139 | 7faf5110 | Michael Hanselmann | live system are Xen and KVM. Supported Xen versions are: 3.0.3, 3.0.4 |
140 | 7faf5110 | Michael Hanselmann | and 3.1. Supported KVM version are 72 and above. |
141 | 28e15341 | Iustin Pop | |
142 | c71a1a3d | Iustin Pop | Please follow your distribution's recommended way to install and set up |
143 | c71a1a3d | Iustin Pop | Xen, or install Xen from the upstream source, if you wish, following |
144 | c71a1a3d | Iustin Pop | their manual. For KVM, make sure you have a KVM-enabled kernel and the |
145 | c71a1a3d | Iustin Pop | KVM tools. |
146 | 28e15341 | Iustin Pop | |
147 | 756d5ec3 | Guido Trotter | After installing Xen, you need to reboot into your new system. On some |
148 | 7faf5110 | Michael Hanselmann | distributions this might involve configuring GRUB appropriately, whereas |
149 | 7faf5110 | Michael Hanselmann | others will configure it automatically when you install the respective |
150 | 7faf5110 | Michael Hanselmann | kernels. For KVM no reboot should be necessary. |
151 | 28e15341 | Iustin Pop | |
152 | 756d5ec3 | Guido Trotter | .. admonition:: Xen on Debian |
153 | 28e15341 | Iustin Pop | |
154 | c71a1a3d | Iustin Pop | Under Lenny or Etch you can install the relevant ``xen-linux-system`` |
155 | c71a1a3d | Iustin Pop | package, which will pull in both the hypervisor and the relevant |
156 | c71a1a3d | Iustin Pop | kernel. Also, if you are installing a 32-bit Lenny/Etch, you should |
157 | c71a1a3d | Iustin Pop | install the ``libc6-xen`` package (run ``apt-get install |
158 | c71a1a3d | Iustin Pop | libc6-xen``). |
159 | 28e15341 | Iustin Pop | |
160 | 28e15341 | Iustin Pop | Xen settings |
161 | 28e15341 | Iustin Pop | ~~~~~~~~~~~~ |
162 | 28e15341 | Iustin Pop | |
163 | 28e15341 | Iustin Pop | It's recommended that dom0 is restricted to a low amount of memory |
164 | c71a1a3d | Iustin Pop | (512MiB or 1GiB is reasonable) and that memory ballooning is disabled in |
165 | c71a1a3d | Iustin Pop | the file ``/etc/xen/xend-config.sxp`` by setting the value |
166 | c71a1a3d | Iustin Pop | ``dom0-min-mem`` to 0, like this:: |
167 | 28e15341 | Iustin Pop | |
168 | 28e15341 | Iustin Pop | (dom0-min-mem 0) |
169 | 28e15341 | Iustin Pop | |
170 | 28e15341 | Iustin Pop | For optimum performance when running both CPU and I/O intensive |
171 | c71a1a3d | Iustin Pop | instances, it's also recommended that the dom0 is restricted to one CPU |
172 | c71a1a3d | Iustin Pop | only, for example by booting with the kernel parameter ``nosmp``. |
173 | 28e15341 | Iustin Pop | |
174 | 28e15341 | Iustin Pop | It is recommended that you disable xen's automatic save of virtual |
175 | 28e15341 | Iustin Pop | machines at system shutdown and subsequent restore of them at reboot. |
176 | 28e15341 | Iustin Pop | To obtain this make sure the variable ``XENDOMAINS_SAVE`` in the file |
177 | 28e15341 | Iustin Pop | ``/etc/default/xendomains`` is set to an empty value. |
178 | 28e15341 | Iustin Pop | |
179 | aeaa2ea2 | Guido Trotter | If you want to use live migration make sure you have, in the xen config |
180 | aeaa2ea2 | Guido Trotter | file, something that allows the nodes to migrate instances between each |
181 | aeaa2ea2 | Guido Trotter | other. For example:: |
182 | 8ab90d80 | Guido Trotter | |
183 | 8ab90d80 | Guido Trotter | (xend-relocation-server yes) |
184 | 8ab90d80 | Guido Trotter | (xend-relocation-port 8002) |
185 | 8ab90d80 | Guido Trotter | (xend-relocation-address '') |
186 | 926feaf1 | Manuel Franceschini | (xend-relocation-hosts-allow '^192\\.0\\.2\\.[0-9]+$') |
187 | 8ab90d80 | Guido Trotter | |
188 | e8a3bf18 | Iustin Pop | |
189 | 84d7362b | Andrea Spadaccini | The second line assumes that the hypervisor parameter |
190 | e8a3bf18 | Iustin Pop | ``migration_port`` is set 8002, otherwise modify it to match. The last |
191 | e8a3bf18 | Iustin Pop | line assumes that all your nodes have secondary IPs in the |
192 | 926feaf1 | Manuel Franceschini | 192.0.2.0/24 network, adjust it accordingly to your setup. |
193 | 8ab90d80 | Guido Trotter | |
194 | 28e15341 | Iustin Pop | .. admonition:: Debian |
195 | 28e15341 | Iustin Pop | |
196 | 28e15341 | Iustin Pop | Besides the ballooning change which you need to set in |
197 | 28e15341 | Iustin Pop | ``/etc/xen/xend-config.sxp``, you need to set the memory and nosmp |
198 | 28e15341 | Iustin Pop | parameters in the file ``/boot/grub/menu.lst``. You need to modify |
199 | 28e15341 | Iustin Pop | the variable ``xenhopt`` to add ``dom0_mem=1024M`` like this:: |
200 | 28e15341 | Iustin Pop | |
201 | 28e15341 | Iustin Pop | ## Xen hypervisor options to use with the default Xen boot option |
202 | 28e15341 | Iustin Pop | # xenhopt=dom0_mem=1024M |
203 | 28e15341 | Iustin Pop | |
204 | c71a1a3d | Iustin Pop | and the ``xenkopt`` needs to include the ``nosmp`` option like this:: |
205 | 28e15341 | Iustin Pop | |
206 | 28e15341 | Iustin Pop | ## Xen Linux kernel options to use with the default Xen boot option |
207 | 28e15341 | Iustin Pop | # xenkopt=nosmp |
208 | 28e15341 | Iustin Pop | |
209 | 28e15341 | Iustin Pop | Any existing parameters can be left in place: it's ok to have |
210 | 28e15341 | Iustin Pop | ``xenkopt=console=tty0 nosmp``, for example. After modifying the |
211 | 28e15341 | Iustin Pop | files, you need to run:: |
212 | 28e15341 | Iustin Pop | |
213 | 28e15341 | Iustin Pop | /sbin/update-grub |
214 | 28e15341 | Iustin Pop | |
215 | c71a1a3d | Iustin Pop | If you want to run HVM instances too with Ganeti and want VNC access to |
216 | c71a1a3d | Iustin Pop | the console of your instances, set the following two entries in |
217 | 28e15341 | Iustin Pop | ``/etc/xen/xend-config.sxp``:: |
218 | 28e15341 | Iustin Pop | |
219 | 28e15341 | Iustin Pop | (vnc-listen '0.0.0.0') (vncpasswd '') |
220 | 28e15341 | Iustin Pop | |
221 | 28e15341 | Iustin Pop | You need to restart the Xen daemon for these settings to take effect:: |
222 | 28e15341 | Iustin Pop | |
223 | 28e15341 | Iustin Pop | /etc/init.d/xend restart |
224 | 28e15341 | Iustin Pop | |
225 | 28e15341 | Iustin Pop | Selecting the instance kernel |
226 | 28e15341 | Iustin Pop | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
227 | 28e15341 | Iustin Pop | |
228 | 28e15341 | Iustin Pop | After you have installed Xen, you need to tell Ganeti exactly what |
229 | c71a1a3d | Iustin Pop | kernel to use for the instances it will create. This is done by creating |
230 | c71a1a3d | Iustin Pop | a symlink from your actual kernel to ``/boot/vmlinuz-2.6-xenU``, and one |
231 | c71a1a3d | Iustin Pop | from your initrd to ``/boot/initrd-2.6-xenU`` [#defkernel]_. Note that |
232 | c71a1a3d | Iustin Pop | if you don't use an initrd for the domU kernel, you don't need to create |
233 | c71a1a3d | Iustin Pop | the initrd symlink. |
234 | 28e15341 | Iustin Pop | |
235 | 28e15341 | Iustin Pop | .. admonition:: Debian |
236 | 28e15341 | Iustin Pop | |
237 | 28e15341 | Iustin Pop | After installation of the ``xen-linux-system`` package, you need to |
238 | 28e15341 | Iustin Pop | run (replace the exact version number with the one you have):: |
239 | 28e15341 | Iustin Pop | |
240 | 28e15341 | Iustin Pop | cd /boot |
241 | 28e15341 | Iustin Pop | ln -s vmlinuz-2.6.26-1-xen-amd64 vmlinuz-2.6-xenU |
242 | 28e15341 | Iustin Pop | ln -s initrd.img-2.6.26-1-xen-amd64 initrd-2.6-xenU |
243 | 28e15341 | Iustin Pop | |
244 | 28e15341 | Iustin Pop | Installing DRBD |
245 | 28e15341 | Iustin Pop | +++++++++++++++ |
246 | 28e15341 | Iustin Pop | |
247 | c71a1a3d | Iustin Pop | Recommended on all nodes: DRBD_ is required if you want to use the high |
248 | c71a1a3d | Iustin Pop | availability (HA) features of Ganeti, but optional if you don't require |
249 | c71a1a3d | Iustin Pop | them or only run Ganeti on single-node clusters. You can upgrade a |
250 | c71a1a3d | Iustin Pop | non-HA cluster to an HA one later, but you might need to export and |
251 | 28e15341 | Iustin Pop | re-import all your instances to take advantage of the new features. |
252 | 28e15341 | Iustin Pop | |
253 | 28e15341 | Iustin Pop | .. _DRBD: http://www.drbd.org/ |
254 | 28e15341 | Iustin Pop | |
255 | c71a1a3d | Iustin Pop | Supported DRBD versions: 8.0+. It's recommended to have at least version |
256 | c71a1a3d | Iustin Pop | 8.0.12. Note that for version 8.2 and newer it is needed to pass the |
257 | c71a1a3d | Iustin Pop | ``usermode_helper=/bin/true`` parameter to the module, either by |
258 | c71a1a3d | Iustin Pop | configuring ``/etc/modules`` or when inserting it manually. |
259 | 28e15341 | Iustin Pop | |
260 | 28e15341 | Iustin Pop | Now the bad news: unless your distribution already provides it |
261 | c71a1a3d | Iustin Pop | installing DRBD might involve recompiling your kernel or anyway fiddling |
262 | c71a1a3d | Iustin Pop | with it. Hopefully at least the Xen-ified kernel source to start from |
263 | c71a1a3d | Iustin Pop | will be provided (if you intend to use Xen). |
264 | 28e15341 | Iustin Pop | |
265 | 28e15341 | Iustin Pop | The good news is that you don't need to configure DRBD at all. Ganeti |
266 | c71a1a3d | Iustin Pop | will do it for you for every instance you set up. If you have the DRBD |
267 | c71a1a3d | Iustin Pop | utils installed and the module in your kernel you're fine. Please check |
268 | c71a1a3d | Iustin Pop | that your system is configured to load the module at every boot, and |
269 | c71a1a3d | Iustin Pop | that it passes the following option to the module: |
270 | c71a1a3d | Iustin Pop | ``minor_count=NUMBER``. We recommend that you use 128 as the value of |
271 | c71a1a3d | Iustin Pop | the minor_count - this will allow you to use up to 64 instances in total |
272 | c71a1a3d | Iustin Pop | per node (both primary and secondary, when using only one disk per |
273 | c71a1a3d | Iustin Pop | instance). You can increase the number up to 255 if you need more |
274 | c71a1a3d | Iustin Pop | instances on a node. |
275 | c71a1a3d | Iustin Pop | |
276 | 28e15341 | Iustin Pop | |
277 | 28e15341 | Iustin Pop | .. admonition:: Debian |
278 | 28e15341 | Iustin Pop | |
279 | c71a1a3d | Iustin Pop | On Debian, you can just install (build) the DRBD module with the |
280 | c71a1a3d | Iustin Pop | following commands, making sure you are running the target (Xen or |
281 | c71a1a3d | Iustin Pop | KVM) kernel:: |
282 | 28e15341 | Iustin Pop | |
283 | 28e15341 | Iustin Pop | apt-get install drbd8-source drbd8-utils |
284 | 28e15341 | Iustin Pop | m-a update |
285 | 28e15341 | Iustin Pop | m-a a-i drbd8 |
286 | c71a1a3d | Iustin Pop | echo drbd minor_count=128 usermode_helper=/bin/true >> /etc/modules |
287 | 28e15341 | Iustin Pop | depmod -a |
288 | c71a1a3d | Iustin Pop | modprobe drbd minor_count=128 usermode_helper=/bin/true |
289 | 28e15341 | Iustin Pop | |
290 | c71a1a3d | Iustin Pop | It is also recommended that you comment out the default resources in |
291 | c71a1a3d | Iustin Pop | the ``/etc/drbd.conf`` file, so that the init script doesn't try to |
292 | c71a1a3d | Iustin Pop | configure any drbd devices. You can do this by prefixing all |
293 | 28e15341 | Iustin Pop | *resource* lines in the file with the keyword *skip*, like this:: |
294 | 28e15341 | Iustin Pop | |
295 | 92c1ea55 | Iustin Pop | skip { |
296 | 92c1ea55 | Iustin Pop | resource r0 { |
297 | 92c1ea55 | Iustin Pop | ... |
298 | 92c1ea55 | Iustin Pop | } |
299 | 28e15341 | Iustin Pop | } |
300 | 28e15341 | Iustin Pop | |
301 | 92c1ea55 | Iustin Pop | skip { |
302 | 92c1ea55 | Iustin Pop | resource "r1" { |
303 | 92c1ea55 | Iustin Pop | ... |
304 | 92c1ea55 | Iustin Pop | } |
305 | 28e15341 | Iustin Pop | } |
306 | 28e15341 | Iustin Pop | |
307 | 7ed400f0 | Stratos Psomadakis | Installing RBD |
308 | 7ed400f0 | Stratos Psomadakis | +++++++++++++++ |
309 | 7ed400f0 | Stratos Psomadakis | |
310 | 7ed400f0 | Stratos Psomadakis | Recommended on all nodes: RBD_ is required if you want to create |
311 | 7ed400f0 | Stratos Psomadakis | instances with RBD disks residing inside a RADOS cluster (make use of |
312 | 7ed400f0 | Stratos Psomadakis | the rbd disk template). RBD-based instances can failover or migrate to |
313 | 7ed400f0 | Stratos Psomadakis | any other node in the ganeti cluster, enabling you to exploit of all |
314 | 7ed400f0 | Stratos Psomadakis | Ganeti's high availabilily (HA) features. |
315 | 7ed400f0 | Stratos Psomadakis | |
316 | 7ed400f0 | Stratos Psomadakis | .. attention:: |
317 | 7ed400f0 | Stratos Psomadakis | Be careful though: rbd is still experimental! For now it is |
318 | 7ed400f0 | Stratos Psomadakis | recommended only for testing purposes. No sensitive data should be |
319 | 7ed400f0 | Stratos Psomadakis | stored there. |
320 | 7ed400f0 | Stratos Psomadakis | |
321 | 7ed400f0 | Stratos Psomadakis | .. _RBD: http://ceph.newdream.net/ |
322 | 7ed400f0 | Stratos Psomadakis | |
323 | 7ed400f0 | Stratos Psomadakis | You will need the ``rbd`` and ``libceph`` kernel modules, the RBD/Ceph |
324 | 7ed400f0 | Stratos Psomadakis | userspace utils (ceph-common Debian package) and an appropriate |
325 | 7ed400f0 | Stratos Psomadakis | Ceph/RADOS configuration file on every VM-capable node. |
326 | 7ed400f0 | Stratos Psomadakis | |
327 | 7ed400f0 | Stratos Psomadakis | You will also need a working RADOS Cluster accessible by the above |
328 | 7ed400f0 | Stratos Psomadakis | nodes. |
329 | 7ed400f0 | Stratos Psomadakis | |
330 | 7ed400f0 | Stratos Psomadakis | RADOS Cluster |
331 | 7ed400f0 | Stratos Psomadakis | ~~~~~~~~~~~~~ |
332 | 7ed400f0 | Stratos Psomadakis | |
333 | 7ed400f0 | Stratos Psomadakis | You will need a working RADOS Cluster accesible by all VM-capable nodes |
334 | 7ed400f0 | Stratos Psomadakis | to use the RBD template. For more information on setting up a RADOS |
335 | 7ed400f0 | Stratos Psomadakis | Cluster, refer to the `official docs <http://ceph.newdream.net/>`_. |
336 | 7ed400f0 | Stratos Psomadakis | |
337 | 7ed400f0 | Stratos Psomadakis | If you want to use a pool for storing RBD disk images other than the |
338 | 7ed400f0 | Stratos Psomadakis | default (``rbd``), you should first create the pool in the RADOS |
339 | 7ed400f0 | Stratos Psomadakis | Cluster, and then set the corresponding rbd disk parameter named |
340 | 7ed400f0 | Stratos Psomadakis | ``pool``. |
341 | 7ed400f0 | Stratos Psomadakis | |
342 | 7ed400f0 | Stratos Psomadakis | Kernel Modules |
343 | 7ed400f0 | Stratos Psomadakis | ~~~~~~~~~~~~~~ |
344 | 7ed400f0 | Stratos Psomadakis | |
345 | 7ed400f0 | Stratos Psomadakis | Unless your distribution already provides it, you might need to compile |
346 | 7ed400f0 | Stratos Psomadakis | the ``rbd`` and ``libceph`` modules from source. You will need Linux |
347 | 7ed400f0 | Stratos Psomadakis | Kernel 3.2 or above for the kernel modules. Alternatively you will have |
348 | 7ed400f0 | Stratos Psomadakis | to build them as external modules (from Linux Kernel source 3.2 or |
349 | 7ed400f0 | Stratos Psomadakis | above), if you want to run a less recent kernel, or your kernel doesn't |
350 | 7ed400f0 | Stratos Psomadakis | include them. |
351 | 7ed400f0 | Stratos Psomadakis | |
352 | 7ed400f0 | Stratos Psomadakis | Userspace Utils |
353 | 7ed400f0 | Stratos Psomadakis | ~~~~~~~~~~~~~~~ |
354 | 7ed400f0 | Stratos Psomadakis | |
355 | 7ed400f0 | Stratos Psomadakis | The RBD template has been tested with ``ceph-common`` v0.38 and |
356 | 7ed400f0 | Stratos Psomadakis | above. We recommend using the latest version of ``ceph-common``. |
357 | 7ed400f0 | Stratos Psomadakis | |
358 | 7ed400f0 | Stratos Psomadakis | .. admonition:: Debian |
359 | 7ed400f0 | Stratos Psomadakis | |
360 | 7ed400f0 | Stratos Psomadakis | On Debian, you can just install the RBD/Ceph userspace utils with |
361 | 7ed400f0 | Stratos Psomadakis | the following command:: |
362 | 7ed400f0 | Stratos Psomadakis | |
363 | 7ed400f0 | Stratos Psomadakis | apt-get install ceph-common |
364 | 7ed400f0 | Stratos Psomadakis | |
365 | 7ed400f0 | Stratos Psomadakis | Configuration file |
366 | 7ed400f0 | Stratos Psomadakis | ~~~~~~~~~~~~~~~~~~ |
367 | 7ed400f0 | Stratos Psomadakis | |
368 | 7ed400f0 | Stratos Psomadakis | You should also provide an appropriate configuration file |
369 | 7ed400f0 | Stratos Psomadakis | (``ceph.conf``) in ``/etc/ceph``. For the rbd userspace utils, you'll |
370 | 7ed400f0 | Stratos Psomadakis | only need to specify the IP addresses of the RADOS Cluster monitors. |
371 | 7ed400f0 | Stratos Psomadakis | |
372 | 7ed400f0 | Stratos Psomadakis | .. admonition:: ceph.conf |
373 | 7ed400f0 | Stratos Psomadakis | |
374 | 7ed400f0 | Stratos Psomadakis | Sample configuration file:: |
375 | 7ed400f0 | Stratos Psomadakis | |
376 | 7ed400f0 | Stratos Psomadakis | [mon.a] |
377 | 7ed400f0 | Stratos Psomadakis | host = example_monitor_host1 |
378 | 7ed400f0 | Stratos Psomadakis | mon addr = 1.2.3.4:6789 |
379 | 7ed400f0 | Stratos Psomadakis | [mon.b] |
380 | 7ed400f0 | Stratos Psomadakis | host = example_monitor_host2 |
381 | 7ed400f0 | Stratos Psomadakis | mon addr = 1.2.3.5:6789 |
382 | 7ed400f0 | Stratos Psomadakis | [mon.c] |
383 | 7ed400f0 | Stratos Psomadakis | host = example_monitor_host3 |
384 | 7ed400f0 | Stratos Psomadakis | mon addr = 1.2.3.6:6789 |
385 | 7ed400f0 | Stratos Psomadakis | |
386 | 7ed400f0 | Stratos Psomadakis | For more information, please see the `Ceph Docs |
387 | 7ed400f0 | Stratos Psomadakis | <http://ceph.newdream.net/docs/latest/>`_ |
388 | 7ed400f0 | Stratos Psomadakis | |
389 | 28e15341 | Iustin Pop | Other required software |
390 | 28e15341 | Iustin Pop | +++++++++++++++++++++++ |
391 | 28e15341 | Iustin Pop | |
392 | cbf3d64b | Michael Hanselmann | See :doc:`install-quick`. |
393 | 28e15341 | Iustin Pop | |
394 | 28e15341 | Iustin Pop | Setting up the environment for Ganeti |
395 | 28e15341 | Iustin Pop | ------------------------------------- |
396 | 28e15341 | Iustin Pop | |
397 | 28e15341 | Iustin Pop | Configuring the network |
398 | 28e15341 | Iustin Pop | +++++++++++++++++++++++ |
399 | 28e15341 | Iustin Pop | |
400 | 28e15341 | Iustin Pop | **Mandatory** on all nodes. |
401 | 28e15341 | Iustin Pop | |
402 | 7faf5110 | Michael Hanselmann | You can run Ganeti either in "bridge mode" or in "routed mode". In |
403 | 7faf5110 | Michael Hanselmann | bridge mode, the default, the instances network interfaces will be |
404 | 7faf5110 | Michael Hanselmann | attached to a software bridge running in dom0. Xen by default creates |
405 | 7faf5110 | Michael Hanselmann | such a bridge at startup, but your distribution might have a different |
406 | 7faf5110 | Michael Hanselmann | way to do things, and you'll definitely need to manually set it up under |
407 | 7faf5110 | Michael Hanselmann | KVM. |
408 | 28e15341 | Iustin Pop | |
409 | c71a1a3d | Iustin Pop | Beware that the default name Ganeti uses is ``xen-br0`` (which was used |
410 | e721c742 | Guido Trotter | in Xen 2.0) while Xen 3.0 uses ``xenbr0`` by default. See the |
411 | e721c742 | Guido Trotter | `Initializing the cluster`_ section to learn how to choose a different |
412 | e721c742 | Guido Trotter | bridge, or not to use one at all and use "routed mode". |
413 | 28e15341 | Iustin Pop | |
414 | e721c742 | Guido Trotter | In order to use "routed mode" under Xen, you'll need to change the |
415 | 7faf5110 | Michael Hanselmann | relevant parameters in the Xen config file. Under KVM instead, no config |
416 | 7faf5110 | Michael Hanselmann | change is necessary, but you still need to set up your network |
417 | 7faf5110 | Michael Hanselmann | interfaces correctly. |
418 | 9f83899a | Guido Trotter | |
419 | 9f83899a | Guido Trotter | By default, under KVM, the "link" parameter you specify per-nic will |
420 | 7faf5110 | Michael Hanselmann | represent, if non-empty, a different routing table name or number to use |
421 | 7faf5110 | Michael Hanselmann | for your instances. This allows insulation between different instance |
422 | 7faf5110 | Michael Hanselmann | groups, and different routing policies between node traffic and instance |
423 | 7faf5110 | Michael Hanselmann | traffic. |
424 | 9f83899a | Guido Trotter | |
425 | 7faf5110 | Michael Hanselmann | You will need to configure your routing table basic routes and rules |
426 | 7faf5110 | Michael Hanselmann | outside of ganeti. The vif scripts will only add /32 routes to your |
427 | 7faf5110 | Michael Hanselmann | instances, through their interface, in the table you specified (under |
428 | 7faf5110 | Michael Hanselmann | KVM, and in the main table under Xen). |
429 | 9f83899a | Guido Trotter | |
430 | 9f83899a | Guido Trotter | .. admonition:: Bridging under Debian |
431 | 28e15341 | Iustin Pop | |
432 | 28e15341 | Iustin Pop | The recommended way to configure the Xen bridge is to edit your |
433 | 28e15341 | Iustin Pop | ``/etc/network/interfaces`` file and substitute your normal |
434 | 28e15341 | Iustin Pop | Ethernet stanza with the following snippet:: |
435 | 28e15341 | Iustin Pop | |
436 | 28e15341 | Iustin Pop | auto xen-br0 |
437 | 28e15341 | Iustin Pop | iface xen-br0 inet static |
438 | 28e15341 | Iustin Pop | address YOUR_IP_ADDRESS |
439 | 28e15341 | Iustin Pop | netmask YOUR_NETMASK |
440 | 28e15341 | Iustin Pop | network YOUR_NETWORK |
441 | 28e15341 | Iustin Pop | broadcast YOUR_BROADCAST_ADDRESS |
442 | 28e15341 | Iustin Pop | gateway YOUR_GATEWAY |
443 | 28e15341 | Iustin Pop | bridge_ports eth0 |
444 | 28e15341 | Iustin Pop | bridge_stp off |
445 | 28e15341 | Iustin Pop | bridge_fd 0 |
446 | 28e15341 | Iustin Pop | |
447 | 28e15341 | Iustin Pop | The following commands need to be executed on the local console: |
448 | 28e15341 | Iustin Pop | |
449 | 28e15341 | Iustin Pop | ifdown eth0 |
450 | 28e15341 | Iustin Pop | ifup xen-br0 |
451 | 28e15341 | Iustin Pop | |
452 | 28e15341 | Iustin Pop | To check if the bridge is setup, use the ``ip`` and ``brctl show`` |
453 | 28e15341 | Iustin Pop | commands:: |
454 | 28e15341 | Iustin Pop | |
455 | 28e15341 | Iustin Pop | # ip a show xen-br0 |
456 | 28e15341 | Iustin Pop | 9: xen-br0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc noqueue |
457 | 28e15341 | Iustin Pop | link/ether 00:20:fc:1e:d5:5d brd ff:ff:ff:ff:ff:ff |
458 | 28e15341 | Iustin Pop | inet 10.1.1.200/24 brd 10.1.1.255 scope global xen-br0 |
459 | 28e15341 | Iustin Pop | inet6 fe80::220:fcff:fe1e:d55d/64 scope link |
460 | 28e15341 | Iustin Pop | valid_lft forever preferred_lft forever |
461 | 28e15341 | Iustin Pop | |
462 | 28e15341 | Iustin Pop | # brctl show xen-br0 |
463 | 28e15341 | Iustin Pop | bridge name bridge id STP enabled interfaces |
464 | 28e15341 | Iustin Pop | xen-br0 8000.0020fc1ed55d no eth0 |
465 | 28e15341 | Iustin Pop | |
466 | c71a1a3d | Iustin Pop | .. _configure-lvm-label: |
467 | c71a1a3d | Iustin Pop | |
468 | 28e15341 | Iustin Pop | Configuring LVM |
469 | 28e15341 | Iustin Pop | +++++++++++++++ |
470 | 28e15341 | Iustin Pop | |
471 | 28e15341 | Iustin Pop | **Mandatory** on all nodes. |
472 | 28e15341 | Iustin Pop | |
473 | 28e15341 | Iustin Pop | The volume group is required to be at least 20GiB. |
474 | 28e15341 | Iustin Pop | |
475 | c71a1a3d | Iustin Pop | If you haven't configured your LVM volume group at install time you need |
476 | c71a1a3d | Iustin Pop | to do it before trying to initialize the Ganeti cluster. This is done by |
477 | c71a1a3d | Iustin Pop | formatting the devices/partitions you want to use for it and then adding |
478 | c71a1a3d | Iustin Pop | them to the relevant volume group:: |
479 | 28e15341 | Iustin Pop | |
480 | 28e15341 | Iustin Pop | pvcreate /dev/sda3 |
481 | 28e15341 | Iustin Pop | vgcreate xenvg /dev/sda3 |
482 | 28e15341 | Iustin Pop | |
483 | 28e15341 | Iustin Pop | or:: |
484 | 28e15341 | Iustin Pop | |
485 | 28e15341 | Iustin Pop | pvcreate /dev/sdb1 |
486 | 28e15341 | Iustin Pop | pvcreate /dev/sdc1 |
487 | 28e15341 | Iustin Pop | vgcreate xenvg /dev/sdb1 /dev/sdc1 |
488 | 28e15341 | Iustin Pop | |
489 | 28e15341 | Iustin Pop | If you want to add a device later you can do so with the *vgextend* |
490 | 28e15341 | Iustin Pop | command:: |
491 | 28e15341 | Iustin Pop | |
492 | 28e15341 | Iustin Pop | pvcreate /dev/sdd1 |
493 | 28e15341 | Iustin Pop | vgextend xenvg /dev/sdd1 |
494 | 28e15341 | Iustin Pop | |
495 | 28e15341 | Iustin Pop | Optional: it is recommended to configure LVM not to scan the DRBD |
496 | 28e15341 | Iustin Pop | devices for physical volumes. This can be accomplished by editing |
497 | c71a1a3d | Iustin Pop | ``/etc/lvm/lvm.conf`` and adding the ``/dev/drbd[0-9]+`` regular |
498 | c71a1a3d | Iustin Pop | expression to the ``filter`` variable, like this:: |
499 | 28e15341 | Iustin Pop | |
500 | 28e15341 | Iustin Pop | filter = ["r|/dev/cdrom|", "r|/dev/drbd[0-9]+|" ] |
501 | 28e15341 | Iustin Pop | |
502 | c71a1a3d | Iustin Pop | Note that with Ganeti a helper script is provided - ``lvmstrap`` which |
503 | c71a1a3d | Iustin Pop | will erase and configure as LVM any not in-use disk on your system. This |
504 | c71a1a3d | Iustin Pop | is dangerous and it's recommended to read its ``--help`` output if you |
505 | c71a1a3d | Iustin Pop | want to use it. |
506 | c71a1a3d | Iustin Pop | |
507 | 28e15341 | Iustin Pop | Installing Ganeti |
508 | 28e15341 | Iustin Pop | +++++++++++++++++ |
509 | 28e15341 | Iustin Pop | |
510 | 28e15341 | Iustin Pop | **Mandatory** on all nodes. |
511 | 28e15341 | Iustin Pop | |
512 | 28e15341 | Iustin Pop | It's now time to install the Ganeti software itself. Download the |
513 | 28e15341 | Iustin Pop | source from the project page at `<http://code.google.com/p/ganeti/>`_, |
514 | 28e15341 | Iustin Pop | and install it (replace 2.0.0 with the latest version):: |
515 | 28e15341 | Iustin Pop | |
516 | 28e15341 | Iustin Pop | tar xvzf ganeti-2.0.0.tar.gz |
517 | 28e15341 | Iustin Pop | cd ganeti-2.0.0 |
518 | 28e15341 | Iustin Pop | ./configure --localstatedir=/var --sysconfdir=/etc |
519 | 28e15341 | Iustin Pop | make |
520 | 28e15341 | Iustin Pop | make install |
521 | 28e15341 | Iustin Pop | mkdir /srv/ganeti/ /srv/ganeti/os /srv/ganeti/export |
522 | 28e15341 | Iustin Pop | |
523 | c71a1a3d | Iustin Pop | You also need to copy the file ``doc/examples/ganeti.initd`` from the |
524 | c71a1a3d | Iustin Pop | source archive to ``/etc/init.d/ganeti`` and register it with your |
525 | 28e15341 | Iustin Pop | distribution's startup scripts, for example in Debian:: |
526 | 28e15341 | Iustin Pop | |
527 | 28e15341 | Iustin Pop | update-rc.d ganeti defaults 20 80 |
528 | 28e15341 | Iustin Pop | |
529 | c71a1a3d | Iustin Pop | In order to automatically restart failed instances, you need to setup a |
530 | c71a1a3d | Iustin Pop | cron job run the *ganeti-watcher* command. A sample cron file is |
531 | c71a1a3d | Iustin Pop | provided in the source at ``doc/examples/ganeti.cron`` and you can copy |
532 | c71a1a3d | Iustin Pop | that (eventually altering the path) to ``/etc/cron.d/ganeti``. |
533 | c71a1a3d | Iustin Pop | |
534 | c71a1a3d | Iustin Pop | What gets installed |
535 | c71a1a3d | Iustin Pop | ~~~~~~~~~~~~~~~~~~~ |
536 | c71a1a3d | Iustin Pop | |
537 | c71a1a3d | Iustin Pop | The above ``make install`` invocation, or installing via your |
538 | c71a1a3d | Iustin Pop | distribution mechanisms, will install on the system: |
539 | c71a1a3d | Iustin Pop | |
540 | c71a1a3d | Iustin Pop | - a set of python libraries under the *ganeti* namespace (depending on |
541 | c71a1a3d | Iustin Pop | the python version this can be located in either |
542 | c71a1a3d | Iustin Pop | ``lib/python-$ver/site-packages`` or various other locations) |
543 | c71a1a3d | Iustin Pop | - a set of programs under ``/usr/local/sbin`` or ``/usr/sbin`` |
544 | c71a1a3d | Iustin Pop | - man pages for the above programs |
545 | c71a1a3d | Iustin Pop | - a set of tools under the ``lib/ganeti/tools`` directory |
546 | c71a1a3d | Iustin Pop | - an example iallocator script (see the admin guide for details) under |
547 | c71a1a3d | Iustin Pop | ``lib/ganeti/iallocators`` |
548 | c71a1a3d | Iustin Pop | - a cron job that is needed for cluster maintenance |
549 | c71a1a3d | Iustin Pop | - an init script for automatic startup of Ganeti daemons |
550 | c71a1a3d | Iustin Pop | - provided but not installed automatically by ``make install`` is a bash |
551 | c71a1a3d | Iustin Pop | completion script that hopefully will ease working with the many |
552 | c71a1a3d | Iustin Pop | cluster commands |
553 | 28e15341 | Iustin Pop | |
554 | 28e15341 | Iustin Pop | Installing the Operating System support packages |
555 | 28e15341 | Iustin Pop | ++++++++++++++++++++++++++++++++++++++++++++++++ |
556 | 28e15341 | Iustin Pop | |
557 | 28e15341 | Iustin Pop | **Mandatory** on all nodes. |
558 | 28e15341 | Iustin Pop | |
559 | 28e15341 | Iustin Pop | To be able to install instances you need to have an Operating System |
560 | 28e15341 | Iustin Pop | installation script. An example OS that works under Debian and can |
561 | 28e15341 | Iustin Pop | install Debian and Ubuntu instace OSes is provided on the project web |
562 | c71a1a3d | Iustin Pop | site. Download it from the project page and follow the instructions in |
563 | a425810f | Guido Trotter | the ``README`` file. Here is the installation procedure (replace 0.9 |
564 | c71a1a3d | Iustin Pop | with the latest version that is compatible with your ganeti version):: |
565 | 28e15341 | Iustin Pop | |
566 | 28e15341 | Iustin Pop | cd /usr/local/src/ |
567 | a425810f | Guido Trotter | wget http://ganeti.googlecode.com/files/ganeti-instance-debootstrap-0.9.tar.gz |
568 | a425810f | Guido Trotter | tar xzf ganeti-instance-debootstrap-0.9.tar.gz |
569 | a425810f | Guido Trotter | cd ganeti-instance-debootstrap-0.9 |
570 | 28e15341 | Iustin Pop | ./configure |
571 | 28e15341 | Iustin Pop | make |
572 | 28e15341 | Iustin Pop | make install |
573 | 28e15341 | Iustin Pop | |
574 | 28e15341 | Iustin Pop | In order to use this OS definition, you need to have internet access |
575 | 28e15341 | Iustin Pop | from your nodes and have the *debootstrap*, *dump* and *restore* |
576 | 28e15341 | Iustin Pop | commands installed on all nodes. Also, if the OS is configured to |
577 | 28e15341 | Iustin Pop | partition the instance's disk in |
578 | 28e15341 | Iustin Pop | ``/etc/default/ganeti-instance-debootstrap``, you will need *kpartx* |
579 | 28e15341 | Iustin Pop | installed. |
580 | 28e15341 | Iustin Pop | |
581 | 28e15341 | Iustin Pop | .. admonition:: Debian |
582 | 28e15341 | Iustin Pop | |
583 | 28e15341 | Iustin Pop | Use this command on all nodes to install the required packages:: |
584 | 28e15341 | Iustin Pop | |
585 | 28e15341 | Iustin Pop | apt-get install debootstrap dump kpartx |
586 | 28e15341 | Iustin Pop | |
587 | a425810f | Guido Trotter | .. admonition:: KVM |
588 | a425810f | Guido Trotter | |
589 | a425810f | Guido Trotter | In order for debootstrap instances to be able to shutdown cleanly |
590 | a425810f | Guido Trotter | they must install have basic acpi support inside the instance. Which |
591 | a425810f | Guido Trotter | packages are needed depend on the exact flavor of debian or ubuntu |
592 | a425810f | Guido Trotter | which you're installing, but the example defaults file has a |
593 | a425810f | Guido Trotter | commented out configuration line that works for debian lenny and |
594 | a425810f | Guido Trotter | squeeze:: |
595 | a425810f | Guido Trotter | |
596 | a425810f | Guido Trotter | EXTRA_PKGS="acpi-support-base,console-tools,udev" |
597 | a425810f | Guido Trotter | |
598 | a425810f | Guido Trotter | kbd can be used instead of console-tools, and more packages can be |
599 | a425810f | Guido Trotter | added, of course, if needed. |
600 | a425810f | Guido Trotter | |
601 | 28e15341 | Iustin Pop | Alternatively, you can create your own OS definitions. See the manpage |
602 | fd07c6b3 | Iustin Pop | :manpage:`ganeti-os-interface`. |
603 | 28e15341 | Iustin Pop | |
604 | 28e15341 | Iustin Pop | Initializing the cluster |
605 | 28e15341 | Iustin Pop | ++++++++++++++++++++++++ |
606 | 28e15341 | Iustin Pop | |
607 | c71a1a3d | Iustin Pop | **Mandatory** once per cluster, on the first node. |
608 | 28e15341 | Iustin Pop | |
609 | c71a1a3d | Iustin Pop | The last step is to initialize the cluster. After you have repeated the |
610 | 28e15341 | Iustin Pop | above process on all of your nodes, choose one as the master, and |
611 | 28e15341 | Iustin Pop | execute:: |
612 | 28e15341 | Iustin Pop | |
613 | 28e15341 | Iustin Pop | gnt-cluster init <CLUSTERNAME> |
614 | 28e15341 | Iustin Pop | |
615 | c71a1a3d | Iustin Pop | The *CLUSTERNAME* is a hostname, which must be resolvable (e.g. it must |
616 | c71a1a3d | Iustin Pop | exist in DNS or in ``/etc/hosts``) by all the nodes in the cluster. You |
617 | c71a1a3d | Iustin Pop | must choose a name different from any of the nodes names for a |
618 | c71a1a3d | Iustin Pop | multi-node cluster. In general the best choice is to have a unique name |
619 | c71a1a3d | Iustin Pop | for a cluster, even if it consists of only one machine, as you will be |
620 | c71a1a3d | Iustin Pop | able to expand it later without any problems. Please note that the |
621 | c71a1a3d | Iustin Pop | hostname used for this must resolve to an IP address reserved |
622 | 28e15341 | Iustin Pop | **exclusively** for this purpose, and cannot be the name of the first |
623 | 28e15341 | Iustin Pop | (master) node. |
624 | 28e15341 | Iustin Pop | |
625 | 7faf5110 | Michael Hanselmann | If you want to use a bridge which is not ``xen-br0``, or no bridge at |
626 | e721c742 | Guido Trotter | all, change it with the ``--nic-parameters`` option. For example to |
627 | e721c742 | Guido Trotter | bridge on br0 you can say:: |
628 | e721c742 | Guido Trotter | |
629 | e721c742 | Guido Trotter | --nic-parameters link=br0 |
630 | e721c742 | Guido Trotter | |
631 | e721c742 | Guido Trotter | Or to not bridge at all, and use a separate routing table:: |
632 | e721c742 | Guido Trotter | |
633 | e721c742 | Guido Trotter | --nic-parameters mode=routed,link=100 |
634 | 9f83899a | Guido Trotter | |
635 | e721c742 | Guido Trotter | If you don't have a xen-br0 interface you also have to specify a |
636 | e721c742 | Guido Trotter | different network interface which will get the cluster ip, on the master |
637 | e721c742 | Guido Trotter | node, by using the ``--master-netdev <device>`` option. |
638 | 28e15341 | Iustin Pop | |
639 | 28e15341 | Iustin Pop | You can use a different name than ``xenvg`` for the volume group (but |
640 | 28e15341 | Iustin Pop | note that the name must be identical on all nodes). In this case you |
641 | 240c769f | Andrea Spadaccini | need to specify it by passing the *--vg-name <VGNAME>* option to |
642 | 240c769f | Andrea Spadaccini | ``gnt-cluster init``. |
643 | 28e15341 | Iustin Pop | |
644 | c71a1a3d | Iustin Pop | To set up the cluster as an Xen HVM cluster, use the |
645 | 28e15341 | Iustin Pop | ``--enabled-hypervisors=xen-hvm`` option to enable the HVM hypervisor |
646 | c71a1a3d | Iustin Pop | (you can also add ``,xen-pvm`` to enable the PVM one too). You will also |
647 | c71a1a3d | Iustin Pop | need to create the VNC cluster password file |
648 | 28e15341 | Iustin Pop | ``/etc/ganeti/vnc-cluster-password`` which contains one line with the |
649 | 28e15341 | Iustin Pop | default VNC password for the cluster. |
650 | 28e15341 | Iustin Pop | |
651 | 28e15341 | Iustin Pop | To setup the cluster for KVM-only usage (KVM and Xen cannot be mixed), |
652 | 28e15341 | Iustin Pop | pass ``--enabled-hypervisors=kvm`` to the init command. |
653 | 28e15341 | Iustin Pop | |
654 | 28e15341 | Iustin Pop | You can also invoke the command with the ``--help`` option in order to |
655 | 28e15341 | Iustin Pop | see all the possibilities. |
656 | 28e15341 | Iustin Pop | |
657 | b8313b29 | Guido Trotter | Hypervisor/Network/Cluster parameters |
658 | b8313b29 | Guido Trotter | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
659 | b8313b29 | Guido Trotter | |
660 | b8313b29 | Guido Trotter | Please note that the default hypervisor/network/cluster parameters may |
661 | b8313b29 | Guido Trotter | not be the correct one for your environment. Carefully check them, and |
662 | b8313b29 | Guido Trotter | change them at cluster init time, or later with ``gnt-cluster modify``. |
663 | b8313b29 | Guido Trotter | |
664 | b8313b29 | Guido Trotter | Your instance types, networking environment, hypervisor type and version |
665 | b8313b29 | Guido Trotter | may all affect what kind of parameters should be used on your cluster. |
666 | b8313b29 | Guido Trotter | |
667 | b8313b29 | Guido Trotter | For example kvm instances are by default configured to use a host |
668 | b8313b29 | Guido Trotter | kernel, and to be reached via serial console, which works nice for linux |
669 | b8313b29 | Guido Trotter | paravirtualized instances. If you want fully virtualized instances you |
670 | b8313b29 | Guido Trotter | may want to handle their kernel inside the instance, and to use VNC. |
671 | b8313b29 | Guido Trotter | |
672 | 28e15341 | Iustin Pop | Joining the nodes to the cluster |
673 | 28e15341 | Iustin Pop | ++++++++++++++++++++++++++++++++ |
674 | 28e15341 | Iustin Pop | |
675 | 28e15341 | Iustin Pop | **Mandatory** for all the other nodes. |
676 | 28e15341 | Iustin Pop | |
677 | c71a1a3d | Iustin Pop | After you have initialized your cluster you need to join the other nodes |
678 | c71a1a3d | Iustin Pop | to it. You can do so by executing the following command on the master |
679 | c71a1a3d | Iustin Pop | node:: |
680 | 28e15341 | Iustin Pop | |
681 | 28e15341 | Iustin Pop | gnt-node add <NODENAME> |
682 | 28e15341 | Iustin Pop | |
683 | 28e15341 | Iustin Pop | Separate replication network |
684 | 28e15341 | Iustin Pop | ++++++++++++++++++++++++++++ |
685 | 28e15341 | Iustin Pop | |
686 | 28e15341 | Iustin Pop | **Optional** |
687 | 28e15341 | Iustin Pop | |
688 | 28e15341 | Iustin Pop | Ganeti uses DRBD to mirror the disk of the virtual instances between |
689 | 28e15341 | Iustin Pop | nodes. To use a dedicated network interface for this (in order to |
690 | 28e15341 | Iustin Pop | improve performance or to enhance security) you need to configure an |
691 | 28e15341 | Iustin Pop | additional interface for each node. Use the *-s* option with |
692 | 28e15341 | Iustin Pop | ``gnt-cluster init`` and ``gnt-node add`` to specify the IP address of |
693 | 28e15341 | Iustin Pop | this secondary interface to use for each node. Note that if you |
694 | c71a1a3d | Iustin Pop | specified this option at cluster setup time, you must afterwards use it |
695 | c71a1a3d | Iustin Pop | for every node add operation. |
696 | 28e15341 | Iustin Pop | |
697 | 28e15341 | Iustin Pop | Testing the setup |
698 | 28e15341 | Iustin Pop | +++++++++++++++++ |
699 | 28e15341 | Iustin Pop | |
700 | c71a1a3d | Iustin Pop | Execute the ``gnt-node list`` command to see all nodes in the cluster:: |
701 | 28e15341 | Iustin Pop | |
702 | 28e15341 | Iustin Pop | # gnt-node list |
703 | 28e15341 | Iustin Pop | Node DTotal DFree MTotal MNode MFree Pinst Sinst |
704 | 28e15341 | Iustin Pop | node1.example.com 197404 197404 2047 1896 125 0 0 |
705 | 28e15341 | Iustin Pop | |
706 | c71a1a3d | Iustin Pop | The above shows a couple of things: |
707 | 28e15341 | Iustin Pop | |
708 | c71a1a3d | Iustin Pop | - The various Ganeti daemons can talk to each other |
709 | c71a1a3d | Iustin Pop | - Ganeti can examine the storage of the node (DTotal/DFree) |
710 | c71a1a3d | Iustin Pop | - Ganeti can talk to the selected hypervisor (MTotal/MNode/MFree) |
711 | 28e15341 | Iustin Pop | |
712 | c71a1a3d | Iustin Pop | Cluster burnin |
713 | c71a1a3d | Iustin Pop | ~~~~~~~~~~~~~~ |
714 | 28e15341 | Iustin Pop | |
715 | c71a1a3d | Iustin Pop | With Ganeti a tool called :command:`burnin` is provided that can test |
716 | c71a1a3d | Iustin Pop | most of the Ganeti functionality. The tool is installed under the |
717 | c71a1a3d | Iustin Pop | ``lib/ganeti/tools`` directory (either under ``/usr`` or ``/usr/local`` |
718 | c71a1a3d | Iustin Pop | based on the installation method). See more details under |
719 | c71a1a3d | Iustin Pop | :ref:`burnin-label`. |
720 | 28e15341 | Iustin Pop | |
721 | c71a1a3d | Iustin Pop | Further steps |
722 | c71a1a3d | Iustin Pop | ------------- |
723 | 28e15341 | Iustin Pop | |
724 | c71a1a3d | Iustin Pop | You can now proceed either to the :doc:`admin`, or read the manpages of |
725 | c71a1a3d | Iustin Pop | the various commands (:manpage:`ganeti(7)`, :manpage:`gnt-cluster(8)`, |
726 | c71a1a3d | Iustin Pop | :manpage:`gnt-node(8)`, :manpage:`gnt-instance(8)`, |
727 | c71a1a3d | Iustin Pop | :manpage:`gnt-job(8)`). |
728 | 28e15341 | Iustin Pop | |
729 | c71a1a3d | Iustin Pop | .. rubric:: Footnotes |
730 | 28e15341 | Iustin Pop | |
731 | c71a1a3d | Iustin Pop | .. [#defkernel] The kernel and initrd paths can be changed at either |
732 | c71a1a3d | Iustin Pop | cluster level (which changes the default for all instances) or at |
733 | c71a1a3d | Iustin Pop | instance level. |
734 | 558fd122 | Michael Hanselmann | |
735 | 558fd122 | Michael Hanselmann | .. vim: set textwidth=72 : |
736 | c71a1a3d | Iustin Pop | .. Local Variables: |
737 | c71a1a3d | Iustin Pop | .. mode: rst |
738 | c71a1a3d | Iustin Pop | .. fill-column: 72 |
739 | c71a1a3d | Iustin Pop | .. End: |