4 |
4 |
<articleinfo>
|
5 |
5 |
<title>Ganeti installation tutorial</title>
|
6 |
6 |
</articleinfo>
|
7 |
|
<para>Documents Ganeti version 1.2</para>
|
|
7 |
<para>Documents Ganeti version 2.0</para>
|
8 |
8 |
|
9 |
9 |
<sect1>
|
10 |
10 |
<title>Introduction</title>
|
11 |
11 |
|
12 |
12 |
<para>
|
13 |
13 |
Ganeti is a cluster virtualization management system based on
|
14 |
|
Xen. This document explains how to bootstrap a Ganeti node (Xen
|
15 |
|
<literal>dom0</literal>), create a running cluster and install
|
16 |
|
virtual instance (Xen <literal>domU</literal>). You need to
|
17 |
|
repeat most of the steps in this document for every node you
|
18 |
|
want to install, but of course we recommend creating some
|
|
14 |
Xen or KVM. This document explains how to bootstrap a Ganeti
|
|
15 |
node (Xen <literal>dom0</literal>), create a running cluster and
|
|
16 |
install virtual instance (Xen <literal>domU</literal>). You
|
|
17 |
need to repeat most of the steps in this document for every node
|
|
18 |
you want to install, but of course we recommend creating some
|
19 |
19 |
semi-automatic procedure if you plan to deploy Ganeti on a
|
20 |
20 |
medium/large scale.
|
21 |
21 |
</para>
|
... | ... | |
29 |
29 |
|
30 |
30 |
<para>
|
31 |
31 |
Ganeti has been developed for Linux and is
|
32 |
|
distribution-agnostic. This documentation will use Debian Etch
|
|
32 |
distribution-agnostic. This documentation will use Debian Lenny
|
33 |
33 |
as an example system but the examples can easily be translated
|
34 |
34 |
to any other distribution. You are expected to be familiar with
|
35 |
|
your distribution, its package management system, and Xen before
|
36 |
|
trying to use Ganeti.
|
|
35 |
your distribution, its package management system, and Xen or KVM
|
|
36 |
before trying to use Ganeti.
|
37 |
37 |
</para>
|
38 |
38 |
|
39 |
39 |
<para>This document is divided into two main sections:
|
... | ... | |
90 |
90 |
at this stage is to partition leaving enough space for a big
|
91 |
91 |
(<emphasis role="strong">minimum
|
92 |
92 |
<constant>20GiB</constant></emphasis>) LVM volume group which
|
93 |
|
will then host your instance filesystems. The volume group
|
94 |
|
name Ganeti 1.2 uses (by default) is
|
95 |
|
<emphasis>xenvg</emphasis>.
|
|
93 |
will then host your instance filesystems, if you want to use
|
|
94 |
all Ganeti features. The volume group name Ganeti 2.0 uses (by
|
|
95 |
default) is <emphasis>xenvg</emphasis>.
|
|
96 |
</para>
|
|
97 |
|
|
98 |
<para>
|
|
99 |
You can also use file-based storage only, without LVM, but
|
|
100 |
this is not detailed in this document.
|
96 |
101 |
</para>
|
97 |
102 |
|
98 |
103 |
<para>
|
... | ... | |
122 |
127 |
<formalpara>
|
123 |
128 |
<title>Debian</title>
|
124 |
129 |
<para>
|
125 |
|
Note that Debian Etch configures the hostname differently
|
|
130 |
Note that Debian Lenny configures the hostname differently
|
126 |
131 |
than you need it for Ganeti. For example, this is what
|
127 |
132 |
Etch puts in <filename>/etc/hosts</filename> in certain
|
128 |
133 |
situations:
|
... | ... | |
158 |
163 |
|
159 |
164 |
<para>
|
160 |
165 |
While Ganeti is developed with the ability to modularly run on
|
161 |
|
different virtualization environments in mind the only one
|
162 |
|
currently useable on a live system is <ulink
|
163 |
|
url="http://xen.xensource.com/">Xen</ulink>. Supported
|
|
166 |
different virtualization environments in mind the only two
|
|
167 |
currently useable on a live system are <ulink
|
|
168 |
url="http://xen.xensource.com/">Xen</ulink> and KVM. Supported
|
164 |
169 |
versions are: <simplelist type="inline">
|
165 |
170 |
<member><literal>3.0.3</literal></member>
|
166 |
171 |
<member><literal>3.0.4</literal></member>
|
... | ... | |
170 |
175 |
<para>
|
171 |
176 |
Please follow your distribution's recommended way to install
|
172 |
177 |
and set up Xen, or install Xen from the upstream source, if
|
173 |
|
you wish, following their manual.
|
|
178 |
you wish, following their manual. For KVM, make sure you have
|
|
179 |
a KVM-enabled kernel and the KVM tools.
|
174 |
180 |
</para>
|
175 |
181 |
|
176 |
182 |
<para>
|
177 |
|
After installing Xen you need to reboot into your Xen-ified
|
178 |
|
dom0 system. On some distributions this might involve
|
|
183 |
After installing either hypervisor, you need to reboot into
|
|
184 |
your new system. On some distributions this might involve
|
179 |
185 |
configuring GRUB appropriately, whereas others will configure
|
180 |
|
it automatically when you install Xen from a package.
|
|
186 |
it automatically when you install the respective kernels.
|
181 |
187 |
</para>
|
182 |
188 |
|
183 |
189 |
<formalpara><title>Debian</title>
|
184 |
190 |
<para>
|
185 |
|
Under Debian Etch or Sarge+backports you can install the
|
186 |
|
relevant <literal>xen-linux-system</literal> package, which
|
187 |
|
will pull in both the hypervisor and the relevant
|
188 |
|
kernel. Also, if you are installing a 32-bit Etch, you should
|
189 |
|
install the <computeroutput>libc6-xen</computeroutput> package
|
190 |
|
(run <computeroutput>apt-get install
|
191 |
|
libc6-xen</computeroutput>).
|
|
191 |
Under Debian Lenny or Etch you can install the relevant
|
|
192 |
<literal>xen-linux-system</literal> package, which will pull
|
|
193 |
in both the hypervisor and the relevant kernel. Also, if you
|
|
194 |
are installing a 32-bit Lenny/Etch, you should install the
|
|
195 |
<computeroutput>libc6-xen</computeroutput> package (run
|
|
196 |
<computeroutput>apt-get install libc6-xen</computeroutput>).
|
192 |
197 |
</para>
|
193 |
198 |
</formalpara>
|
194 |
199 |
|
... | ... | |
197 |
202 |
|
198 |
203 |
<para>
|
199 |
204 |
It's recommended that dom0 is restricted to a low amount of
|
200 |
|
memory (<constant>512MiB</constant> is reasonable) and that
|
201 |
|
memory ballooning is disabled in the file
|
|
205 |
memory (<constant>512MiB</constant> or
|
|
206 |
<constant>1GiB</constant> is reasonable) and that memory
|
|
207 |
ballooning is disabled in the file
|
202 |
208 |
<filename>/etc/xen/xend-config.sxp</filename> by setting the
|
203 |
209 |
value <literal>dom0-min-mem</literal> to
|
204 |
210 |
<constant>0</constant>, like this:
|
... | ... | |
228 |
234 |
set the memory and nosmp parameters in the file
|
229 |
235 |
<filename>/boot/grub/menu.lst</filename>. You need to
|
230 |
236 |
modify the variable <literal>xenhopt</literal> to add
|
231 |
|
<userinput>dom0_mem=512M</userinput> like this:
|
|
237 |
<userinput>dom0_mem=1024M</userinput> like this:
|
232 |
238 |
<screen>
|
233 |
239 |
## Xen hypervisor options to use with the default Xen boot option
|
234 |
|
# xenhopt=dom0_mem=512M
|
|
240 |
# xenhopt=dom0_mem=1024M
|
235 |
241 |
</screen>
|
236 |
242 |
and the <literal>xenkopt</literal> needs to include the
|
237 |
243 |
<userinput>nosmp</userinput> option like this:
|
... | ... | |
250 |
256 |
</para>
|
251 |
257 |
</formalpara>
|
252 |
258 |
<para>
|
253 |
|
If you want to test the HVM support
|
254 |
|
with Ganeti and want VNC access to the console of your
|
255 |
|
instances, set the following two entries in
|
|
259 |
If you want to run HVM instances too with Ganeti and want
|
|
260 |
VNC access to the console of your instances, set the
|
|
261 |
following two entries in
|
256 |
262 |
<filename>/etc/xen/xend-config.sxp</filename>:
|
257 |
263 |
<screen>
|
258 |
264 |
(vnc-listen '0.0.0.0')
|
... | ... | |
315 |
321 |
|
316 |
322 |
<para>
|
317 |
323 |
Supported DRBD versions: <literal>8.0.x</literal>.
|
318 |
|
It's recommended to have at least version <literal>8.0.7</literal>.
|
|
324 |
It's recommended to have at least version <literal>8.0.12</literal>.
|
319 |
325 |
</para>
|
320 |
326 |
|
321 |
327 |
<para>
|
... | ... | |
346 |
352 |
</formalpara>
|
347 |
353 |
|
348 |
354 |
<screen>
|
349 |
|
apt-get install -t etch-backports drbd8-source drbd8-utils
|
|
355 |
apt-get install drbd8-source drbd8-utils
|
350 |
356 |
m-a update
|
351 |
357 |
m-a a-i drbd8
|
352 |
358 |
echo drbd minor_count=128 >> /etc/modules
|
... | ... | |
442 |
448 |
</formalpara>
|
443 |
449 |
<screen>
|
444 |
450 |
# apt-get install lvm2 ssh bridge-utils iproute iputils-arping \
|
445 |
|
python2.4 python-pyopenssl openssl python-pyparsing python-simplejson
|
|
451 |
python python-pyopenssl openssl python-pyparsing python-simplejson
|
446 |
452 |
</screen>
|
447 |
453 |
|
448 |
454 |
</sect2>
|
... | ... | |
628 |
634 |
|
629 |
635 |
<para>
|
630 |
636 |
To be able to install instances you need to have an Operating
|
631 |
|
System installation script. An example for Debian Etch is
|
|
637 |
System installation script. An example OS that works under
|
|
638 |
Debian and can install Debian and Ubuntu instace OSes is
|
632 |
639 |
provided on the project web site. Download it from <ulink
|
633 |
640 |
url="http://code.google.com/p/ganeti/"></ulink> and follow the
|
634 |
641 |
instructions in the <filename>README</filename> file. Here is
|
635 |
|
the installation procedure (replace <constant>0.2</constant>
|
|
642 |
the installation procedure (replace <constant>0.7</constant>
|
636 |
643 |
with the latest version that is compatible with your ganeti
|
637 |
644 |
version):
|
638 |
645 |
</para>
|
639 |
646 |
|
640 |
647 |
<screen>
|
641 |
|
cd /srv/ganeti/os
|
642 |
|
tar xvf ganeti-instance-debian-etch-0.4.tar
|
643 |
|
mv ganeti-instance-debian-etch-0.4 debian-etch
|
|
648 |
cd /usr/local/src/
|
|
649 |
wget http://ganeti.googlecode.com/files/ganeti-instance-debootstrap-0.7.tar.gz
|
|
650 |
tar xzf ganeti-instance-debootstrap-0.7.tar.gz
|
|
651 |
cd ganeti-instance-debootstrap-0.7
|
|
652 |
./configure
|
|
653 |
make
|
|
654 |
make install
|
644 |
655 |
</screen>
|
645 |
656 |
|
646 |
657 |
<para>
|
... | ... | |
652 |
663 |
</citerefentry> and <citerefentry>
|
653 |
664 |
<refentrytitle>restore</refentrytitle>
|
654 |
665 |
<manvolnum>8</manvolnum> </citerefentry> commands installed on
|
655 |
|
all nodes.
|
|
666 |
all nodes. Also, if the OS is configured to partition the
|
|
667 |
instance's disk in
|
|
668 |
<filename>/etc/default/ganeti-instance-debootstrap</filename>,
|
|
669 |
you will need <command>kpartx</command> installed.
|
656 |
670 |
</para>
|
657 |
671 |
<formalpara>
|
658 |
672 |
<title>Debian</title>
|
... | ... | |
660 |
674 |
Use this command on all nodes to install the required
|
661 |
675 |
packages:
|
662 |
676 |
|
663 |
|
<screen>apt-get install debootstrap dump</screen>
|
|
677 |
<screen>apt-get install debootstrap dump kpartx</screen>
|
664 |
678 |
</para>
|
665 |
679 |
</formalpara>
|
666 |
680 |
|
... | ... | |
682 |
696 |
node per cluster.</para>
|
683 |
697 |
|
684 |
698 |
|
685 |
|
<para>The last step is to initialize the cluster. After you've repeated
|
686 |
|
the above process on all of your nodes, choose one as the master, and execute:
|
|
699 |
<para>
|
|
700 |
The last step is to initialize the cluster. After you've
|
|
701 |
repeated the above process on all of your nodes, choose one as
|
|
702 |
the master, and execute:
|
687 |
703 |
</para>
|
688 |
704 |
|
689 |
705 |
<screen>
|
... | ... | |
699 |
715 |
choice is to have a unique name for a cluster, even if it
|
700 |
716 |
consists of only one machine, as you will be able to expand it
|
701 |
717 |
later without any problems. Please note that the hostname used
|
702 |
|
for this must resolve to an IP address reserved exclusively
|
703 |
|
for this purpose.
|
|
718 |
for this must resolve to an IP address reserved <emphasis
|
|
719 |
role="strong">exclusively</emphasis> for this purpose.
|
704 |
720 |
</para>
|
705 |
721 |
|
706 |
722 |
<para>
|
... | ... | |
723 |
739 |
|
724 |
740 |
<para>
|
725 |
741 |
To set up the cluster as an HVM cluster, use the
|
726 |
|
<option>--hypervisor=xen-hvm3.1</option> option to use
|
727 |
|
the Xen 3.1 HVM hypervisor. Note that with the
|
728 |
|
HVM support, you will only be able to create
|
729 |
|
HVM instances in a cluster set to this hypervisor type. Mixed
|
730 |
|
PVM/HVM clusters are not supported by the Ganeti 1.2
|
731 |
|
HVM support. You will also need to create the VNC
|
732 |
|
cluster password file
|
733 |
|
<filename>/etc/ganeti/vnc-cluster-password</filename>
|
|
742 |
<option>--enabled-hypervisors=xen-hvm</option> option to
|
|
743 |
enable the HVM hypervisor (you can also add
|
|
744 |
<userinput>,xen-pvm</userinput> to enable the PVM one
|
|
745 |
too). You will also need to create the VNC cluster password
|
|
746 |
file <filename>/etc/ganeti/vnc-cluster-password</filename>
|
734 |
747 |
which contains one line with the default VNC password for the
|
735 |
748 |
cluster.
|
736 |
749 |
</para>
|
737 |
750 |
|
738 |
751 |
<para>
|
|
752 |
To setup the cluster for KVM-only usage (KVM and Xen cannot be
|
|
753 |
mixed), pass <option>--enabled-hypervisors=kvm</option> to the
|
|
754 |
init command.
|
|
755 |
</para>
|
|
756 |
|
|
757 |
<para>
|
739 |
758 |
You can also invoke the command with the
|
740 |
759 |
<option>--help</option> option in order to see all the
|
741 |
760 |
possibilities.
|
... | ... | |
815 |
834 |
instance for you:
|
816 |
835 |
</para>
|
817 |
836 |
<screen>
|
818 |
|
gnt-instance add --node=node1 -o debian-etch -t plain inst1.example.com
|
|
837 |
gnt-instance add --node=node1 -o debootstrap -t plain inst1.example.com
|
819 |
838 |
* creating instance disks...
|
820 |
839 |
adding instance inst1.example.com to cluster config
|
821 |
840 |
Waiting for instance inst1.example.com to sync disks.
|
... | ... | |
844 |
863 |
</para>
|
845 |
864 |
|
846 |
865 |
<screen>
|
847 |
|
# gnt-instance add -t drbd -n node1:node2 -o debian-etch instance2
|
|
866 |
# gnt-instance add -t drbd -n node1:node2 -o debootstrap instance2
|
848 |
867 |
* creating instance disks...
|
849 |
868 |
adding instance instance2 to cluster config
|
850 |
869 |
Waiting for instance instance1 to sync disks.
|