Synnefo » snf-ganeti » ganeti-local

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [

]>

  <article class="specification">

  <articleinfo>

    <title>Ganeti node/cluster installation tutorial</title>

  </articleinfo>

  <para>Documents Ganeti version 1.2</para>

  <sect1>

    <title>Introduction</title>

    <para>

      Ganeti is a cluster virtualization management system. This

      document explains how to bootstrap a Ganeti node and create a

      running cluster. You need to repeat most of the steps in this

      document for every node you want to install, but of course we

      recommend creating some semi-automatic procedure if you plan to

      deploy Ganeti on a medium/large scale.

    </para>

    <para>This document is divided into two main sections:

      <itemizedlist>

        <listitem>

          <simpara>Installation of the core system and base

          components</simpara>

        </listitem>

        <listitem>

          <simpara>Configuration of the environment for

          Ganeti</simpara>

        </listitem>

      </itemizedlist>

    Each of these is divided into sub-sections. While a full Ganeti

    system will need all of the steps specified, some are not strictly

    required for every environment. Which ones they are, and why, is

    specified in the corresponding sections.

    </para>

    <para>

      While Ganeti itself is distribution-agnostic most of the

      examples in this document will be targeted at Debian or

      Debian-derived distributions. You are expected to be familiar

      with your distribution, its package management system, and Xen

      before trying to use Ganeti.

    </para>

    <para>

      A basic Ganeti terminology glossary is provided in the

      introductory section of the "admin guide". Please refer to that

      if you are uncertain about the terms we are using.

    </para>

  </sect1>

  <sect1>

    <title>Installing the system and base components</title>

    <sect2>

      <title>Installing the base system</title>

      <para><emphasis role="strong">Mandatory.</emphasis></para>

      <para>

        Please install your operating system as you would normally

        do. The only requirement you need to be aware of at this stage

        is to partition leaving enough space for a big LVM volume

        group which will then host your instance file systems. You can

        even create the volume group at installation time, of course:

        the default volume group name Ganeti 1.2 uses is "xenvg" but

        you may name it differently should you wish to, as long as the

        name is the same for all the nodes in the cluster.

      </para>

    </sect2>

    <sect2>

      <title>Installing Xen</title>

      <para>

        <emphasis role="strong">Mandatory:</emphasis> While Ganeti is

        developed with the ability to modularly run on different

        virtualization environments in mind the only one currently

        useable on a live system is <ulink

        url="http://xen.xensource.com/">Xen</ulink>.

      </para>

      <para>

        Please follow your distribution's recommended way to install

        and set up Xen, or install Xen from the upstream source, if

        you wish, following their manual.

      </para>

      <para>

        For example under Debian 4.0 or 3.1+backports you can install

        the relevant xen-linux-system package, which will pull in both

        the hypervisor and the relevant kernel. On Ubuntu (from Gutsy

        on) the package is called ubuntu-xen-server.

      </para>

      <para>

        After installing Xen you need to reboot into your xenified

        dom0 system. Again on some distributions this might involve

        configuring GRUB appropriately.

      </para>

    </sect2>

    <sect2>

      <title>Installing DRBD</title>

      <para>

        Recommended: <ulink url="http://www.drbd.org/">DRBD</ulink>

        is required if you want to use the high availability (HA)

        features of Ganeti, but optional if you don't require HA or

        only run Ganeti on single-node clusters. You can upgrade a

        non-HA cluster to an HA one later, but you might need to

        export and reimport all your instances to take advantage of

        the new features.

      </para>

      <para>

        Now the bad news: unless your distribution already provides it

        installing DRBD might involve recompiling your kernel or

        anyway fiddling with it. Hopefully at least the xenified

        kernel source to start from will be provided.

      </para>

      <para>

        Under Debian you can just install the drbd0.7-module-source

        and drbd0.7-utils packages, and your kernel source, and then

        run module-assistant to compile the drbd0.7 module. The

        following commands should do it:

      </para>

      <screen>

m-a update

m-a a-i drbd0.7

      </screen>

      <para>

        The good news is that you don't need to configure DRBD at all.

        Ganeti will do it for you for every instance you set up.  If

        you have the DRBD utils installed and the module in your

        kernel you're fine. Please check that your system is

        configured to load the module at every boot.

      </para>

    </sect2>

    <sect2>

      <title>Other required software</title>

      <para>Besides Xen and DRBD, you will need to install the

      following:</para>

      <itemizedlist>

        <listitem>

          <simpara><ulink url="http://sourceware.org/lvm2/">LVM

          version 2</ulink></simpara>

        </listitem>

        <listitem>

          <simpara><ulink

          url="http://www.openssl.org/">OpenSSL</ulink></simpara>

        </listitem>

        <listitem>

          <simpara><ulink

          url="http://www.openssh.com/portable.html">OpenSSH</ulink></simpara>

        </listitem>

        <listitem>

          <simpara><ulink url="http://bridge.sourceforge.net/">Bridge

          utilities</ulink></simpara>

        </listitem>

        <listitem>

          <simpara><ulink

          url="http://fping.sourceforge.net/">fping</ulink></simpara>

        </listitem>

        <listitem>

          <simpara><ulink

          url="http://developer.osdl.org/dev/iproute2">iproute2</ulink></simpara>

        </listitem>

        <listitem>

          <simpara><ulink

          url="ftp://ftp.inr.ac.ru/ip-routing/iputils-current.tar.gz">arping</ulink>

          (part of iputils package)</simpara>

        </listitem>

        <listitem>

          <simpara><ulink

          url="http://www.kernel.org/pub/linux/utils/raid/mdadm/">mdadm</ulink>

          (Linux Software Raid tools)</simpara>

        </listitem>

        <listitem>

          <simpara><ulink url="http://www.python.org">Python 2.4</ulink></simpara>

        </listitem>

        <listitem>

          <simpara><ulink url="http://twistedmatrix.com/">Python

          Twisted library</ulink> - the core library is

          enough</simpara>

        </listitem>

        <listitem>

          <simpara><ulink

          url="http://pyopenssl.sourceforge.net/">Python OpenSSL

          bindings</ulink></simpara>

        </listitem>

      </itemizedlist>

      <para>These programs are supplied as part of most Linux

      distributions, so usually they can be installed via apt or

      similar methods. Also many of them will already be installed on

      a standard machine. On Debian Etch you can use this command line

      to install all of them:</para>

      <screen>

# apt-get install lvm2 ssh bridge-utils iproute iputils-arping \

> fping python2.4 python-twisted-core python-pyopenssl openssl

      </screen>

    </sect2>

  </sect1>

  <sect1>

    <title>Setting up the environment for Ganeti</title>

    <sect2>

      <title>Configuring the network</title>

      <para>Ganeti relies on Xen running in "bridge mode", which means the

      instances network interfaces will be attached to a software bridge

      running in dom0. Xen by default creates such a bridge at startup, but

      your distribution might have a different way to do things.

      </para>

      <para>

      In Debian, in order to enable the default Xen behaviour, you

      have to edit <filename>/etc/xen/xend-config.sxp</filename> and

      replace <computeroutput>(network-script

      network-dummy)</computeroutput> with

      <computeroutput>(network-script

      network-bridge)</computeroutput>. The recommended Debian way to

      configure things, though, is to edit your

      <filename>/etc/network/interfaces</filename> file and substitute

      your normal ethernet stanza with something like:</para>

      <screen>

auto br0

iface br0 inet static

        address <replaceable>YOUR_IP_ADDRESS</replaceable>

        netmask <replaceable>YOUR_NETMASK</replaceable>

        network <replaceable>YOUR_NETWORK</replaceable>

        broadcast <replaceable>YOUR_BROADCAST_ADDRESS</replaceable>

        gateway <replaceable>YOUR_GATEWAY</replaceable>

        bridge_ports <replaceable>eth0</replaceable>

        bridge_stp off

        bridge_fd 0

      </screen>

    <para>

      Beware that the default name Ganeti uses is

      <hardware>xen-br0</hardware> (which was used in Xen 2.0)

      while Xen 3.0 uses <hardware>xenbr0</hardware> by

      default. The default bridge your cluster will use for new

      instances can be specified at cluster initialization time.

    </para>

    </sect2>

    <sect2>

      <title>Configuring LVM</title>

      <para>

        If you haven't configured your LVM volume group at install

        time you need to do it before trying to initialize the Ganeti

        cluster. This is done by formatting the devices/partitions you

        want to use for it and then adding them to the relevant volume

        group:

       </para>

       <screen>

pvcreate /dev/sda4

pvcreate /dev/sdb

pvcreate /dev/sdc1

vgcreate xenvg /dev/sda4 /dev/sdb /dev/sdc1

       </screen>

      <para>

	If you want to add a device later you can do so with the

	<citerefentry><refentrytitle>vgextend</refentrytitle>

	<manvolnum>8</manvolnum></citerefentry> command:

      </para>

      <screen>

pvcreate /dev/sdd

vgextend xenvg /dev/sdd

      </screen>

      <para>

        As said before you may choose a different name for the volume group,

        as long as you stick to the same name on all the nodes of a cluster.

      </para>

    </sect2>

    <sect2>

      <title>Installing Ganeti</title>

      <para>

        It's now time to install the Ganeti software itself. You can

        do it from source, with the usual steps (note that the

        <option>localstatedir</option> options must be set to

        <filename class="directory">/var</filename>):

      </para>

        <screen>

./configure --localstatedir=/var

make

make install

mkdir /srv/ganeti/ /srv/ganeti/os /srv/ganeti/export

        </screen>

      <para>

        You also need to copy from the source archive the file

        <filename>docs/examples/ganeti.initd</filename> to

        <filename>/etc/init.d/ganeti</filename> and register it into

        your distribution's startup scripts, for example in Debian:

      </para>

      <screen>update-rc.d ganeti defaults 20 80</screen>

    </sect2>

    <sect2>

      <title>Installing the Operating System support packages</title>

      <para>

        Another important component for Ganeti are the OS support

        packages, which let different operating systems be used as

        instances. You can grab a simple package that allows

        installing Debian Etch instances on the project web site

        (after download, untar it and follow the instructions in the

        <filename>README</filename> file).

      </para>

      <para>

        Alternatively, you can create your own OS definitions, see

        <citerefentry>

        <refentrytitle>ganeti-os-interface</refentrytitle>

        <manvolnum>8</manvolnum>

        </citerefentry>.

      </para>

    </sect2>

    <sect2>

      <title>Initializing the cluster</title>

      <para><emphasis role="strong">Mandatory:</emphasis> only on one

      node per cluster.</para>

      <para>The last step is to initialize the cluster. After you've repeated

        the above process or some semi-automatic form of it on all of your

        nodes choose one as the master, and execute:

      </para>

      <screen>

gnt-cluster init <replaceable>CLUSTERNAME</replaceable>

      </screen>

      <para>

        If the node's network interface which will be used for access

        from outside the cluster is not named

        <hardware>xen-br0</hardware>, you need to use the

        <option>--master-netdev=<replaceable>IFNAME</replaceable></option>

        option, replacing <replaceable>IFNAME</replaceable> with the

        correct one for your case (e.g. <hardware>xenbr0</hardware>,

        <hardware>eth0</hardware>, etc.). Usually this will be the

        same as the default bridge name (see below).

      </para>

      <para>

        Other options you can pass to <command>gnt-cluster

        init</command> include the default bridge name

        (<option>-b</option>), the cluster-wide name for the volume

        group (<option>-g</option>) and the secondary ip address for

        the initial node should you wish to keep the data replication

        network separate. Invoke it with <option>--help</option> to

        see all the possibilities.

      </para>

      <para>

        Note that the cluster name must exist in DNS. You must choose

        a name different from any of the nodes names for a multi-node

        cluster. In general the best choice is to have a unique name

        for a cluster, even if it consists of only one machine, as you

        will be able to expand it later without any problem.

      </para>

    </sect2>

    <sect2>

      <title>Joining the nodes to the cluster.</title>

      <para>

        <emphasis role="strong">Mandatory:</emphasis> for all the

        other nodes.

      </para>

      <para>

        If you have already initialized your cluster you need to join the other

        nodes to it. You can do so by executing the following command on the

        master node:

        <screen>

gnt-node add <replaceable>NODENAME</replaceable>

        </screen>

        The only option is <option>-s</option>, which sets the node's

        secondary ip address for replication purposes, if you are

        using a separate replication network.

      </para>

    </sect2>

  </sect1>

  <sect1>

    <title>This is it!</title>

    <para>

      Now you can follow the admin guide to use your new Ganeti

      cluster.

    </para>

  </sect1>

  </article>