Synnefo » snf-ganeti » ganeti-local

doc/install.html file. A glossary of terms can be found in the doc/admin.html

file.

SUBDIRS = man lib scripts daemons doc testing tools

For installation instructions, read the INSTALL and the doc/install.pdf

AC_CONFIG_FILES([Makefile man/Makefile doc/Makefile

  lib/Makefile scripts/Makefile daemons/Makefile

  doc/examples/Makefile])

docdir = $(datadir)/doc/$(PACKAGE)

SUBDIRS = examples

dist_doc_DATA = hooks.html hooks.pdf install.html install.pdf admin.html \

  admin.pdf

EXTRA_DIST = hooks.sgml install.sgml admin.sgml

%.html: %.sgml

	docbook2html --nochunks $<

%.pdf: %.sgml

	docbook2pdf $<

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [

]>

  <article class="specification">

  <articleinfo>

    <title>Ganeti administrator's guide</title>

  </articleinfo>

  <para>Documents Ganeti version 1.2</para>

  <sect1>

    <title>Introduction</title>

    <para>Ganeti is a virtualization cluster management software. You are

    expected to be a system administrator familiar with your Linux distribution

    and the Xen virtualization environment before using it.

    </para>

    <para>The various components of Ganeti all have man pages and interactive

    help. This manual though will help you getting familiar with the system by

    explaining the most common operations, grouped by related use.

    </para>

    <para>After a terminology glossary and a section on the prerequisites

    needed to use this manual, the rest of this document is divided in three

    main sections, which group different features of Ganeti:

      <itemizedlist>

        <listitem>

          <simpara>Instance Management</simpara>

        </listitem>

        <listitem>

          <simpara>High Availability Features</simpara>

        </listitem>

        <listitem>

          <simpara>Debugging Features</simpara>

        </listitem>

      </itemizedlist>

    </para>

    <sect2>

      <title>Ganeti terminology</title>

      <para>This section provides a small introduction to Ganeti terminology,

      which might be useful to read the rest of the document.

      <glosslist>

          <glossentry>

            <glossterm>Cluster</glossterm>

            <glossdef>

              <simpara>

                A set of machines (nodes) that cooperate to offer a

                coherent highly available virtualization service.

              </simpara>

            </glossdef>

          </glossentry>

          <glossentry>

            <glossterm>Node</glossterm>

            <glossdef>

              <simpara>

                A physical machine which is member of a cluster.

                Nodes are the basic cluster infrastructure, and are

                not fault tolerant.

              </simpara>

            </glossdef>

          </glossentry>

          <glossentry>

            <glossterm>Master node</glossterm>

            <glossdef>

              <simpara>

                The node which controls the Cluster, from which all

                Ganeti commands must be given.

              </simpara>

            </glossdef>

          </glossentry>

          <glossentry>

            <glossterm>Instance</glossterm>

            <glossdef>

              <simpara>

                A virtual machine which runs on a cluster. It can be a

                fault tolerant highly available entity.

              </simpara>

            </glossdef>

          </glossentry>

          <glossentry>

            <glossterm>Pool</glossterm>

            <glossdef>

              <simpara>

                A pool is a set of clusters sharing the same network.

              </simpara>

            </glossdef>

          </glossentry>

          <glossentry>

            <glossterm>Meta-Cluster</glossterm>

            <glossdef>

              <simpara>

                Anything that concerns more than one cluster.

              </simpara>

            </glossdef>

          </glossentry>

      </glosslist>

      </para>

    </sect2>

    <sect2>

      <title>Prerequisites</title>

      <para>

        You need to have your Ganeti cluster installed and configured

        before you try any of the commands in this document. Please

        follow the <emphasis>Ganeti installation tutorial</emphasis>

        for instructions on how to do that.

      </para>

    </sect2>

  </sect1>

  <sect1>

    <title>Managing Instances</title>

    <sect2>

      <title>Adding/Removing an instance</title>

      <para>

        Adding a new virtual instance to your Ganeti cluster is really

        easy. The command is:

      <synopsis>gnt-instance add -n <replaceable>TARGET_NODE</replaceable> -o <replaceable>OS_TYPE</replaceable> -t <replaceable>DISK_TEMPLATE</replaceable> <replaceable>INSTANCE_NAME</replaceable></synopsis>

        The instance name must be resolvable (e.g. exist in DNS) and

        of course map to an address in the same subnet as the cluster

        itself. Options you can give to this command include:

      <itemizedlist>

        <listitem>

          <simpara>The disk size (<option>-s</option>)</simpara>

        </listitem>

        <listitem>

          <simpara>The swap size (<option>--swap-size</option>)</simpara>

        </listitem>

        <listitem>

          <simpara>The memory size (<option>-m</option>)</simpara>

        </listitem>

        <listitem>

          <simpara>The number of virtual CPUs (<option>-p</option>)</simpara>

        </listitem>

        <listitem>

	  <simpara>The instance ip address (<option>-i</option>) (use

	  the value <literal>auto</literal> to make Ganeti record the

	  address from dns)</simpara>

        </listitem>

        <listitem>

	  <simpara>The bridge to connect the instance to

	  (<option>-b</option>), if you don't want to use the default

	  one</simpara>

        </listitem>

      </itemizedlist>

      </para>

      <para>There are four types of disk template you can choose from:</para>

      <variablelist>

        <varlistentry>

          <term>diskless</term>

	  <listitem><para>The instance has no disks. Only used for special

	  purpouse operating systems or for testing.</para></listitem>

        </varlistentry>

        <varlistentry>

          <term>plain</term>

	  <listitem><para>The instance will use LVM devices as backend for its

	  disks. No redundancy is provided.</para></listitem>

        </varlistentry>

        <varlistentry>

          <term>local_raid1</term>

	  <listitem><para>A local mirror is set between LVM devices to back the

	  instance. This provides some redundancy for the instance's

	  data.</para></listitem>

        </varlistentry>

        <varlistentry>

          <term>remote_raid1</term>

	  <listitem>

            <simpara><emphasis role="strong">Note:</emphasis> This is

            only valid for multi-node clusters.</simpara>

            <simpara>

                A mirror is set between the local node and a remote

                one, which must be specified with the --secondary-node

                option. Use this option to obtain a highly available

                instance that can be failed over to a remote node

                should the primary one fail.

	      </simpara>

            </listitem>

        </varlistentry>

      </variablelist>

      <para>

        For example if you want to create an highly available instance

        use the remote_raid1 disk template:

      <synopsis>gnt-instance add -n <replaceable>TARGET_NODE</replaceable> -o <replaceable>OS_TYPE</replaceable> -t remote_raid1 \

  --secondary-node=<replaceable>SECONDARY_NODE</replaceable> <replaceable>INSTANCE_NAME</replaceable></synopsis>

      <para>

        To know which operating systems your cluster supports you can use:

        <synopsis>gnt-os list</synopsis>

      </para>

      <para>

        Removing an instance is even easier than creating one. This

        operation is non-reversible and destroys all the contents of

        your instance. Use with care:

      <synopsis>gnt-instance remove <replaceable>INSTANCE_NAME</replaceable></synopsis>

      </para>

    </sect2>

    <sect2>

      <title>Starting/Stopping an instance</title>

      <para>

        Instances are automatically started at instance creation

        time. To manually start one which is currently stopped you can

        run:

      <synopsis>gnt-instance startup <replaceable>INSTANCE_NAME</replaceable></synopsis>

        While the command to stop one is:

      <synopsis>gnt-instance shutdown <replaceable>INSTANCE_NAME</replaceable></synopsis>

        The command to see all the instances configured and their

        status is:

      <synopsis>gnt-instance list</synopsis>

      </para>

      <para>

        Do not use the xen commands to stop instances. If you run for

        example xm shutdown or xm destroy on an instance Ganeti will

        automatically restart it (via the

        <citerefentry><refentrytitle>ganeti-watcher</refentrytitle>

        <manvolnum>8</manvolnum></citerefentry>)

      </para>

    </sect2>

    <sect2>

      <title>Exporting/Importing an instance</title>

      <para>

        You can create a snapshot of an instance disk and Ganeti

        configuration, which then you can backup, or import into

        another cluster.  The way to export an instance is:

      <synopsis>gnt-backup export -n <replaceable>TARGET_NODE</replaceable> <replaceable>INSTANCE_NAME</replaceable></synopsis>

        The target node can be any node in the cluster with enough

        space under <filename class="directory">/srv/ganeti</filename>

        to hold the instance image. Use the

        <option>--noshutdown</option> option to snapshot an instance

        without rebooting it. Any previous snapshot of the same

        instance existing cluster-wide under <filename

        class="directory">/srv/ganeti</filename> will be removed by

        this operation: if you want to keep them move them out of the

        Ganeti exports directory.

      </para>

      <para>

        Importing an instance is similar to creating a new one. The

        command is:

      <synopsis>gnt-backup import -n <replaceable>TARGET_NODE</replaceable> -t <replaceable>DISK_TEMPLATE</replaceable> --src-node=<replaceable>NODE</replaceable> --src-dir=DIR INSTANCE_NAME</synopsis>

        Most of the options available for the command

        <emphasis>gnt-instance add</emphasis> are supported here too.

      </para>

    </sect2>

  </sect1>

  <sect1>

    <title>High availability features</title>

    <note>

      <simpara>This section only applies to multi-node clusters.</simpara>

    </note>

    <sect2>

      <title>Failing over an instance</title>

      <para>

        If an instance is built in highly available mode you can at

        any time fail it over to its secondary node, even if the

        primary has somehow failed and it's not up anymore. Doing it

        is really easy, on the master node you can just run:

      <synopsis>gnt-instance failover <replaceable>INSTANCE_NAME</replaceable></synopsis>

        That's it. After the command completes the secondary node is

        now the primary, and vice versa.

      </para>

    </sect2>

    <sect2>

      <title>Replacing an instance disks</title>

      <para>

        So what if instead the secondary node for an instance has

        failed, or you plan to remove a node from your cluster, and

        you failed over all its instances, but it's still secondary

        for some? The solution here is to replace the instance disks,

        changing the secondary node:

      <synopsis>gnt-instance replace-disks -n <replaceable>NEW_SECONDARY</replaceable> <replaceable>INSTANCE_NAME</replaceable></synopsis>

        This process is a bit longer, but involves no instance

        downtime, and at the end of it the instance has changed its

        secondary node, to which it can if necessary be failed over.

      </para>

    </sect2>

    <sect2>

      <title>Failing over the master node</title>

      <para>

        This is all good as long as the Ganeti Master Node is

        up. Should it go down, or should you wish to decommission it,

        just run on any other node the command:

      <synopsis>gnt-cluster masterfailover</synopsis>

        and the node you ran it on is now the new master.

      </para>

    </sect2>

    <sect2>

      <title>Adding/Removing nodes</title>

      <para>

        And of course, now that you know how to move instances around,

        it's easy to free up a node, and then you can remove it from

        the cluster:

      <synopsis>

gnt-node remove <replaceable>NODE_NAME</replaceable>

      </synopsis>

        and maybe add a new one:

      <synopsis>

gnt-node add <optional><option>--secondary-ip=<replaceable>ADDRESS</replaceable></option></optional> <replaceable>NODE_NAME</replaceable>

      </synopsis>

      </para>

    </sect2>

  </sect1>

  <sect1>

    <title>Debugging Features</title>

    <para>

      At some point you might need to do some debugging operations on

      your cluster or on your instances. This section will help you

      with the most used debugging functionalities.

    </para>

    <sect2>

      <title>Accessing an instance's disks</title>

      <para>

        From an instance's primary node you have access to its

        disks. Never ever mount the underlying logical volume manually

        on a fault tolerant instance, or you risk breaking

        replication. The correct way to access them is to run the

        command:

      <synopsis> gnt-instance activate-disks <replaceable>INSTANCE_NAME</replaceable></synopsis>

        And then access the device that gets created.  After you've

        finished you can deactivate them with the deactivate-disks

        command, which works in the same way.

      </para>

    </sect2>

    <sect2>

      <title>Accessing an instance's console</title>

      <para>

        The command to access a running instance's console is:

      <synopsis>gnt-instance console <replaceable>INSTANCE_NAME</replaceable></synopsis>

        Use the console normally and then type

        <userinput>^]</userinput> when done, to exit.

      </para>

    </sect2>

    <sect2>

      <title>Instance Operating System Debugging</title>

      <para>

        Should you have any problems with operating systems support

        the command to ran to see a complete status for all your nodes

        is:

      <synopsis>gnt-os diagnose</synopsis>

      </para>

    </sect2>

    <sect2>

      <title>Cluster-wide debugging</title>

      <para>

        The gnt-cluster command offers several options to run tests or

        execute cluster-wide operations. For example:

      <screen>

gnt-cluster command

gnt-cluster copyfile

gnt-cluster verify

gnt-cluster getmaster

gnt-cluster version

      </screen>

        See the man page <citerefentry>

        <refentrytitle>gnt-cluster</refentrytitle>

        <manvolnum>8</manvolnum> </citerefentry> to know more about

        their usage.

      </para>

    </sect2>

  </sect1>

  </article>

PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

# restart failed instances

*/5 * * * * root /usr/local/sbin/ganeti-watcher

#! /bin/sh

# ganeti node daemon starter script

# based on skeleton from Debian GNU/Linux

PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

NODED=/usr/local/sbin/ganeti-noded

MASTER=/usr/local/sbin/ganeti-master

NAME=ganeti-noded

SCRIPTNAME=/etc/init.d/ganeti

DESC="Ganeti cluster"

test -f $NODED || exit 0

. /lib/lsb/init-functions

check_config() {

    for fname in /var/lib/ganeti/ssconf_node_pass /var/lib/ganeti/server.pem; do

        if ! [ -f "$fname" ]; then

            log_end_msg 0

            log_warning_msg "Config $fname not there, will not run."

            exit 0

        fi

    done

}

master_action() {

    log_action_begin_msg "ganeti-master"; $MASTER "$1"

    RC=$?

    case $RC in

        0)

            log_action_end_msg 0

            ;;

        11)

            log_action_end_msg 0 "not master"

            ;;

        *)

            log_action_end_msg 1 "exit code $RC"

            ;;

    esac

}

case "$1" in

    start)

        log_daemon_msg "Starting $DESC" "$NAME"

        check_config

        if start-stop-daemon --start --quiet --exec $NODED; then

            log_end_msg 0

        else

            log_end_msg 1

        fi

        master_action start

    ;;

    stop)

        log_daemon_msg "Stopping $DESC" "$NAME"

        if start-stop-daemon --stop --quiet --name $NAME; then

            log_end_msg 0

        else

            log_end_msg 1

        fi

        master_action stop

    ;;

    restart|force-reload)

        log_daemon_msg "Reloading $DESC"

        start-stop-daemon --stop --quiet --oknodo --retry 30 --name $NAME

        check_config

        start-stop-daemon --start --quiet --exec $NODED

        log_end_msg $?

        $MASTER stop

        master_action start

     ;;

    *)

        log_success_msg "Usage: $SCRIPTNAME {start|stop|force-reload|restart}"

        exit 1

    ;;

esac

exit 0

# vim: set sw=4 sts=4 et foldmethod=marker :

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [

]>

  <article class="specification">

  <articleinfo>

    <title>Ganeti customisation using hooks</title>

  </articleinfo>

  <para>Documents ganeti version 1.2</para>

  <section>

    <title>Introduction</title>

    <para>

      In order to allow customisation of operations, ganeti will run

      scripts under <filename

      class="directory">/etc/ganeti/hooks</filename> based on certain

      rules.

    </para>

      <para>This is similar to the <filename

      class="directory">/etc/network/</filename> structure present in

      Debian for network interface handling.</para>

    </section>

    <section>

      <title>Organisation</title>

      <para>For every operation, two sets of scripts are run:

      <itemizedlist>

          <listitem>

            <simpara>pre phase (for authorization/checking)</simpara>

          </listitem>

          <listitem>

            <simpara>post phase (for logging)</simpara>

          </listitem>

        </itemizedlist>

      </para>

      <para>Also, for each operation, the scripts are run on one or

      more nodes, depending on the operation type.</para>

      <para>Note that, even though we call them scripts, we are

      actually talking about any executable.</para>

      <section>

        <title><emphasis>pre</emphasis> scripts</title>

        <para>The <emphasis>pre</emphasis> scripts have a definite

        target: to check that the operation is allowed given the

        site-specific constraints. You could have, for example, a rule

        that says every new instance is required to exists in a

        database; to implement this, you could write a script that

        checks the new instance parameters against your

        database.</para>

        <para>The objective of these scripts should be their return

        code (zero or non-zero for success and failure). However, if

        they modify the environment in any way, they should be

        idempotent, as failed executions could be restarted and thus

        the script(s) run again with exactly the same

        parameters.</para>

      </section>

      <section>

        <title><emphasis>post</emphasis> scripts</title>

        <para>These scripts should do whatever you need as a reaction

        to the completion of an operation. Their return code is not

        checked (but logged), and they should not depend on the fact

        that the <emphasis>pre</emphasis> scripts have been

        run.</para>

      </section>

      <section>

        <title>Naming</title>

        <para>The allowed names for the scripts consist of (similar to

        <citerefentry> <refentrytitle>run-parts</refentrytitle>

        <manvolnum>8</manvolnum> </citerefentry>) upper and lower

        case, digits, underscores and hyphens. In other words, the

        regexp

        <computeroutput>^[a-zA-Z0-9_-]+$</computeroutput>. Also,

        non-executable scripts will be ignored.

        </para>

      </section>

      <section>

        <title>Order of execution</title>

        <para>On a single node, the scripts in a directory are run in

        lexicographic order (more exactly, the python string

        comparison order). It is advisable to implement the usual

        <emphasis>NN-name</emphasis> convention where

        <emphasis>NN</emphasis> is a two digit number.</para>

        <para>For an operation whose hooks are run on multiple nodes,

        there is no specific ordering of nodes with regard to hooks

        execution; you should assume that the scripts are run in

        parallel on the target nodes (keeping on each node the above

        specified ordering).  If you need any kind of inter-node

        synchronisation, you have to implement it yourself in the

        scripts.</para>

      </section>

      <section>

        <title>Execution environment</title>

        <para>The scripts will be run as follows:

          <itemizedlist>

          <listitem>

            <simpara>no command line arguments</simpara>

          </listitem>

            <listitem>

              <simpara>no controlling <acronym>tty</acronym></simpara>

            </listitem>

            <listitem>

              <simpara><varname>stdin</varname> is

              actually <filename>/dev/null</filename></simpara>

            </listitem>

            <listitem>

              <simpara><varname>stdout</varname> and

              <varname>stderr</varname> are directed to

              files</simpara>

            </listitem>

          <listitem>

            <simpara>the <varname>PATH</varname> is reset to

            <literal>/sbin:/bin:/usr/sbin:/usr/bin</literal></simpara>

          </listitem>

          <listitem>

            <simpara>the environment is cleared, and only

            ganeti-specific variables will be left</simpara>

          </listitem>

          </itemizedlist>

        </para>

      <para>All informations about the cluster is passed using

      environment variables. Different operations will have sligthly

      different environments, but most of the variables are

      common.</para>

    </section>

    <section>

      <title>Operation list</title>

      <table>

        <title>Operation list</title>

        <tgroup cols="7">

          <colspec>

          <colspec>

          <colspec>

          <colspec>

          <colspec>

          <colspec colname="prehooks">

          <colspec colname="posthooks">

          <spanspec namest="prehooks" nameend="posthooks"

            spanname="bothhooks">

          <thead>

            <row>

              <entry>Operation ID</entry>

              <entry>Directory prefix</entry>

              <entry>Description</entry>

              <entry>Command</entry>

              <entry>Supported env. variables</entry>

              <entry><emphasis>pre</emphasis> hooks</entry>

              <entry><emphasis>post</emphasis> hooks</entry>

            </row>

          </thead>

          <tbody>

            <row>

              <entry>OP_INIT_CLUSTER</entry>

              <entry><filename class="directory">cluster-init</filename></entry>

              <entry>Initialises the cluster</entry>

              <entry><computeroutput>gnt-cluster init</computeroutput></entry>

              <entry><constant>CLUSTER</constant>, <constant>MASTER</constant></entry>

              <entry spanname="bothhooks">master node, cluster name</entry>

            </row>

            <row>

              <entry>OP_MASTER_FAILOVER</entry>

              <entry><filename class="directory">master-failover</filename></entry>

              <entry>Changes the master</entry>

              <entry><computeroutput>gnt-cluster master-failover</computeroutput></entry>

              <entry><constant>OLD_MASTER</constant>, <constant>NEW_MASTER</constant></entry>

              <entry>the new master</entry>

              <entry>all nodes</entry>

            </row>

            <row>

              <entry>OP_ADD_NODE</entry>

              <entry><filename class="directory">node-add</filename></entry>

              <entry>Adds a new node to the cluster</entry>

              <entry><computeroutput>gnt-node add</computeroutput></entry>

              <entry><constant>NODE_NAME</constant>, <constant>NODE_PIP</constant>, <constant>NODE_SIP</constant></entry>

              <entry>all existing nodes</entry>

              <entry>all existing nodes plus the new node</entry>

            </row>

            <row>

              <entry>OP_REMOVE_NODE</entry>

              <entry><filename class="directory">node-remove</filename></entry>

              <entry>Removes a node from the cluster</entry>

              <entry><computeroutput>gnt-node remove</computeroutput></entry>

              <entry><constant>NODE_NAME</constant></entry>

              <entry spanname="bothhooks">all existing nodes except the removed node</entry>

            </row>

            <row>

              <entry>OP_INSTANCE_ADD</entry>

              <entry><filename class="directory">instance-add</filename></entry>

              <entry>Creates a new instance</entry>

              <entry><computeroutput>gnt-instance add</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant>, <constant>DISK_TEMPLATE</constant>, <constant>MEM_SIZE</constant>, <constant>DISK_SIZE</constant>, <constant>SWAP_SIZE</constant>, <constant>VCPUS</constant>, <constant>INSTANCE_IP</constant>, <constant>INSTANCE_ADD_MODE</constant>, <constant>SRC_NODE</constant>, <constant>SRC_PATH</constant>, <constant>SRC_IMAGE</constant></entry>

              <entry spanname="bothhooks" morerows="4">master node, primary and

                   secondary nodes</entry>

            </row>

            <row>

              <entry>OP_BACKUP_EXPORT</entry>

              <entry><filename class="directory">instance-export</filename></entry>

              <entry>Export the instance</entry>

              <entry><computeroutput>gnt-backup export</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>EXPORT_NODE</constant>, <constant>EXPORT_DO_SHUTDOWN</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_START</entry>

              <entry><filename class="directory">instance-start</filename></entry>

              <entry>Starts an instance</entry>

              <entry><computeroutput>gnt-instance start</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant>, <constant>FORCE</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_SHUTDOWN</entry>

              <entry><filename class="directory">instance-shutdown</filename></entry>

              <entry>Stops an instance</entry>

              <entry><computeroutput>gnt-instance shutdown</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_MODIFY</entry>

              <entry><filename class="directory">instance-modify</filename></entry>

              <entry>Modifies the instance parameters.</entry>

              <entry><computeroutput>gnt-instance modify</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>MEM_SIZE</constant>, <constant>VCPUS</constant>, <constant>INSTANCE_IP</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_FAILOVER</entry>

              <entry><filename class="directory">instance-failover</filename></entry>

              <entry>Failover an instance</entry>

              <entry><computeroutput>gnt-instance start</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant>, <constant>IGNORE_CONSISTENCY</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_REMOVE</entry>

              <entry><filename class="directory">instance-remove</filename></entry>

              <entry>Remove an instance</entry>

              <entry><computeroutput>gnt-instance remove</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_ADD_MDDRBD</entry>

              <entry><filename class="directory">mirror-add</filename></entry>

              <entry>Adds a mirror component</entry>

              <entry><computeroutput>gnt-instance add-mirror</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>NEW_SECONDARY</constant>, <constant>DISK_NAME</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_REMOVE_MDDRBD</entry>

              <entry><filename class="directory">mirror-remove</filename></entry>

              <entry>Removes a mirror component</entry>

              <entry><computeroutput>gnt-instance remove-mirror</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>OLD_SECONDARY</constant>, <constant>DISK_NAME</constant>, <constant>DISK_ID</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_REPLACE_DISKS</entry>

              <entry><filename class="directory">mirror-replace</filename></entry>

              <entry>Replace all mirror components</entry>

              <entry><computeroutput>gnt-instance replace-disks</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>OLD_SECONDARY</constant>, <constant>NEW_SECONDARY</constant></entry>

            </row>

          </tbody>

        </tgroup>

      </table>

    </section>

    <section>

      <title>Environment variables</title>

      <para>Note that all variables listed here are actually prefixed

      with <constant>GANETI_</constant> in order to provide a

      different namespace.</para>

      <section>

        <title>Common variables</title>

        <para>This is the list of environment variables supported by

        all operations:</para>

        <variablelist>

          <varlistentry>

            <term>HOOKS_VERSION</term>

            <listitem>

              <para>Documents the hooks interface version. In case this

            doesnt match what the script expects, it should not

            run. The documents conforms to the version

            <literal>1</literal>.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>HOOKS_PHASE</term>

            <listitem>

              <para>one of <constant>PRE</constant> or

              <constant>POST</constant> denoting which phase are we

              in.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>CLUSTER</term>

            <listitem>

              <para>the cluster name</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>MASTER</term>

            <listitem>

              <para>the master node</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>OP_ID</term>

            <listitem>

              <para>one of the <constant>OP_*</constant> values from

              the table of operations</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>OBJECT_TYPE</term>

            <listitem>

              <para>one of <simplelist type="inline">

                  <member><constant>INSTANCE</constant></member>

                  <member><constant>NODE</constant></member>

                  <member><constant>CLUSTER</constant></member>

                </simplelist>, showing the target of the operation.

             </para>

            </listitem>

          </varlistentry>

          <!-- commented out since it causes problems in our rpc

               multi-node optimised calls

          <varlistentry>

            <term>HOST_NAME</term>

            <listitem>

              <para>The name of the node the hook is run on as known by

            the cluster.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>HOST_TYPE</term>

            <listitem>

              <para>one of <simplelist type="inline">

                  <member><constant>MASTER</constant></member>

                  <member><constant>NODE</constant></member>

                </simplelist>, showing the role of this node in the cluster.

             </para>

            </listitem>

          </varlistentry>

          -->

        </variablelist>

      </section>

      <section>

        <title>Specialised variables</title>

        <para>This is the list of variables which are specific to one

        or more operations.</para>

        <variablelist>

          <varlistentry>

            <term>INSTANCE_NAME</term>

            <listitem>

              <para>The name of the instance which is the target of

              the operation.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_DISK_TYPE</term>

            <listitem>

              <para>The disk type for the instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_DISK_SIZE</term>

            <listitem>

              <para>The (OS) disk size for the instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_OS</term>

            <listitem>

              <para>The name of the instance OS.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_PRIMARY</term>

            <listitem>

              <para>The name of the node which is the primary for the

              instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_SECONDARIES</term>

            <listitem>

              <para>Space-separated list of secondary nodes for the

              instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>NODE_NAME</term>

            <listitem>

              <para>The target node of this operation (not the node on

              which the hook runs).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>NODE_PIP</term>

            <listitem>

              <para>The primary IP of the target node (the one over

              which inter-node communication is done).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>NODE_SIP</term>

            <listitem>

              <para>The secondary IP of the target node (the one over

              which drbd replication is done). This can be equal to

              the primary ip, in case the cluster is not

              dual-homed.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>OLD_MASTER</term>

            <term>NEW_MASTER</term>

            <listitem>

              <para>The old, respectively the new master for the

              master failover operation.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>FORCE</term>

            <listitem>

              <para>This is provided by some operations when the user

              gave this flag.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>IGNORE_CONSISTENCY</term>

            <listitem>

              <para>The user has specified this flag. It is used when

              failing over instances in case the primary node is

              down.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>MEM_SIZE, DISK_SIZE, SWAP_SIZE, VCPUS</term>

            <listitem>

              <para>The memory, disk, swap size and the number of

              processor selected for the instance (in

              <command>gnt-instance add</command> or

              <command>gnt-instance modify</command>).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_IP</term>

            <listitem>

              <para>If defined, the instance IP in the

              <command>gnt-instance add</command> and

              <command>gnt-instance set</command> commands. If not

              defined, it means that no IP has been defined.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>DISK_TEMPLATE</term>

            <listitem>

              <para>The disk template type when creating the instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_ADD_MODE</term>

            <listitem>

              <para>The mode of the create: either

              <constant>create</constant> for create from scratch or

              <constant>import</constant> for restoring from an

              exported image.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>SRC_NODE, SRC_PATH, SRC_IMAGE</term>

            <listitem>

              <para>In case the instance has been added by import,

              these variables are defined and point to the source

              node, source path (the directory containing the image

              and the config file) and the source disk image

              file.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>DISK_NAME</term>

            <listitem>

              <para>The disk name (either <filename>sda</filename> or

              <filename>sdb</filename>) in mirror operations

              (add/remove mirror).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>DISK_ID</term>

            <listitem>

              <para>The disk id for mirror remove operations. You can

              look this up using <command>gnt-instance

              info</command>.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>NEW_SECONDARY</term>

            <listitem>

              <para>The name of the node on which the new mirror

              componet is being added. This can be the name of the

              current secondary, if the new mirror is on the same

              secondary.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>OLD_SECONDARY</term>

            <listitem>

              <para>The name of the old secondary. This is used in

              both <command>replace-disks</command> and

              <command>remove-mirror</command>. Note that this can be

              equal to the new secondary (only

              <command>replace-disks</command> has both variables) if

              the secondary node hasn't actually changed).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>EXPORT_NODE</term>

            <listitem>

              <para>The node on which the exported image of the

              instance was done.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>EXPORT_DO_SHUTDOWN</term>

            <listitem>

              <para>This variable tells if the instance has been

              shutdown or not while doing the export. In the "was

              shutdown" case, it's likely that the filesystem is

              consistent, whereas in the "did not shutdown" case, the

              filesystem would need a check (journal replay or full

              fsck) in order to guarantee consistency.</para>

            </listitem>

          </varlistentry>

        </variablelist>

      </section>

    </section>

  </section>

  </article>

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [

]>

  <article class="specification">

  <articleinfo>

    <title>Ganeti installation tutorial</title>

  </articleinfo>

  <para>Documents Ganeti version 1.2</para>

  <sect1>

    <title>Introduction</title>

    <para>

      Ganeti is a cluster virtualization management system based on

      Xen. This document explains how to bootstrap a Ganeti node (Xen

      <literal>dom0</literal>), create a running cluster and install

      virtual instance (Xen <literal>domU</literal>).  You need to

      repeat most of the steps in this document for every node you

      want to install, but of course we recommend creating some

      semi-automatic procedure if you plan to deploy Ganeti on a

      medium/large scale.

    </para>

    <para>

      A basic Ganeti terminology glossary is provided in the

      introductory section of the <emphasis>Ganeti administrator's

      guide</emphasis>. Please refer to that document if you are

      uncertain about the terms we are using.

    </para>

    <para>

      Ganeti has been developed for Linux and is

      distribution-agnostic.  This documentation will use Debian Etch

      as an example system but the examples can easily be translated

      to any other distribution.  You are expected to be familiar with

      your distribution, its package management system, and Xen before

      trying to use Ganeti.

    </para>

    <para>This document is divided into two main sections:

      <itemizedlist>

        <listitem>

          <simpara>Installation of the base system and base

          components</simpara>

        </listitem>

        <listitem>

          <simpara>Configuration of the environment for

          Ganeti</simpara>

        </listitem>

      </itemizedlist>

    Each of these is divided into sub-sections. While a full Ganeti

    system will need all of the steps specified, some are not strictly

    required for every environment. Which ones they are, and why, is

    specified in the corresponding sections.

    </para>

  </sect1>

  <sect1>

    <title>Installing the base system and base components</title>

    <sect2>

      <title>Hardware requirements</title>

      <para>

         Any system supported by your Linux distribution is fine.

         64-bit systems are better as they can support more memory.

      </para>

      <para>

         Any disk drive recognized by Linux

         (<literal>IDE</literal>/<literal>SCSI</literal>/<literal>SATA</literal>/etc.)

         is supported in Ganeti. Note that no shared storage

         (e.g. <literal>SAN</literal>) is needed to get high-availability features. It is

         highly recommended to use more than one disk drive to improve

         speed. But Ganeti also works with one disk per machine.

      </para>

    <sect2>

      <title>Installing the base system</title>

      <para>

        <emphasis role="strong">Mandatory</emphasis> on all nodes.

      </para>

      <para>

        It is advised to start with a clean, minimal install of the

        operating system. The only requirement you need to be aware of

        at this stage is to partition leaving enough space for a big

        (<emphasis role="strong">minimum

        <constant>20GiB</constant></emphasis>) LVM volume group which

        will then host your instance filesystems. The volume group

        name Ganeti 1.2 uses (by default) is

        <emphasis>xenvg</emphasis>.

      </para>

      <para>

        While you can use an existing system, please note that the

        Ganeti installation is intrusive in terms of changes to the

        system configuration, and it's best to use a newly-installed

        system without important data on it.

      </para>

      <para>

        Also, for best results, it's advised that the nodes have as

        much as possible the same hardware and software

        configuration. This will make administration much easier.

      </para>

      <sect3>

        <title>Hostname issues</title>

        <para>

          Note that Ganeti requires the hostnames of the systems

          (i.e. what the <computeroutput>hostname</computeroutput>

          command outputs to be a fully-qualified name, not a short

          name. In other words, you should use

          <literal>node1.example.com</literal> as a hostname and not

          just <literal>node1</literal>.

        </para>

        <formalpara>

          <title>Debian</title>

          <para>

            Note that Debian Etch configures the hostname differently

            than you need it for Ganeti. For example, this is what

            Etch puts in <filename>/etc/hosts</filename> in certain

            situations:

<screen>

127.0.0.1       localhost

127.0.1.1       node1.example.com node1

</screen>

          but for Ganeti you need to have:

<screen>

127.0.0.1       localhost

192.168.1.1     node1.example.com node1

</screen>

            replacing <literal>192.168.1.1</literal> with your node's

            address. Also, the file <filename>/etc/hostname</filename>

            which configures the hostname of the system should contain

            <literal>node1.example.com</literal> and not just

            <literal>node1</literal> (you need to run the command

            <computeroutput>/etc/init.d/hostname.sh

            start</computeroutput> after changing the file).

          </para>

        </formalpara>

      </sect3>

    </sect2>

    <sect2>

      <title>Installing Xen</title>

      <para>

        <emphasis role="strong">Mandatory</emphasis> on all nodes.

      </para>

      <para>

        While Ganeti is developed with the ability to modularly run on

        different virtualization environments in mind the only one

        currently useable on a live system is <ulink

        url="http://xen.xensource.com/">Xen</ulink>. Supported

        versions are: <simplelist type="inline">

        <member><literal>3.0.3</literal></member>

        <member><literal>3.0.4</literal></member>

        <member><literal>3.1</literal></member> </simplelist>.

      </para>

      <para>

        Please follow your distribution's recommended way to install

        and set up Xen, or install Xen from the upstream source, if

        you wish, following their manual.

      </para>

      <para>

        After installing Xen you need to reboot into your Xen-ified

        dom0 system. On some distributions this might involve

        configuring GRUB appropriately, whereas others will configure

        it automatically when you install Xen from a package.

      </para>

      <formalpara><title>Debian</title>

      <para>

        Under Debian Etch or Sarge+backports you can install the

        relevant <literal>xen-linux-system</literal> package, which

        will pull in both the hypervisor and the relevant

        kernel. Also, if you are installing a 32-bit Etch, you should

        install the <computeroutput>libc6-xen</computeroutput> package

        (run <computeroutput>apt-get install

        libc6-xen</computeroutput>).

      </para>

      </formalpara>

      <sect3>

        <title>Xen settings</title>

        <para>

          It's recommended that dom0 is restricted to a low amount of

          memory (<constant>512MiB</constant> is reasonable) and that

          memory ballooning is disabled in the file

          <filename>/etc/xen/xend-config.sxp</filename> by setting the

          value <literal>dom0-min-mem</literal> to

          <constant>0</constant>, like this:

          <computeroutput>(dom0-min-mem 0)</computeroutput>

        </para>

        <para>

          For optimum performance when running both CPU and I/O

          intensive instances, it's also recommended that the dom0 is

          restricted to one CPU only, for example by booting with the

          kernel parameter <literal>nosmp</literal>.

        </para>

        <formalpara>

          <title>Debian</title>

          <para>

            Besides the ballooning change which you need to set in

            <filename>/etc/xen/xend-config.sxp</filename>, you need to

            set the memory and nosmp parameters in the file

            <filename>/boot/grub/menu.lst</filename>. You need to

            modify the variable <literal>xenhopt</literal> to add

            <userinput>dom0_mem=512M</userinput> like this:

<screen>

## Xen hypervisor options to use with the default Xen boot option

# xenhopt=dom0_mem=512M

</screen>

            and the <literal>xenkopt</literal> needs to include the

            <userinput>nosmp</userinput> option like this:

<screen>

## Xen Linux kernel options to use with the default Xen boot option

# xenkopt=nosmp

</screen>

          Any existing parameters can be left in place: it's ok to

          have <computeroutput>xenkopt=console=tty0

          nosmp</computeroutput>, for example. After modifying the

          files, you need to run:

<screen>

/sbin/update-grub

</screen>

          </para>

        </formalpara>

      </sect3>

      <sect3>

        <title>Selecting the instance kernel</title>

        <para>

          After you have installed Xen, you need to tell Ganeti

          exactly what kernel to use for the instances it will

          create. This is done by creating a

          <emphasis>symlink</emphasis> from your actual kernel to

          <filename>/boot/vmlinuz-2.6-xenU</filename>, and one from

          your initrd to

          <filename>/boot/initrd-2.6-xenU</filename>. Note that if you

          don't use an initrd for the <literal>domU</literal> kernel,

          you don't need to create the initrd symlink.

        </para>

        <formalpara>

          <title>Debian</title>

          <para>

            After installation of the

            <literal>xen-linux-system</literal> package, you need to

            run (replace the exact version number with the one you

            have):

            <screen>

cd /boot

ln -s vmlinuz-2.6.18-5-xen-686 vmlinuz-2.6-xenU

ln -s initrd.img-2.6.18-5-xen-686 initrd-2.6-xenU

            </screen>

          </para>

        </formalpara>

      </sect3>

    </sect2>

    <sect2>

      <title>Installing DRBD</title>

      <para>

        Recommended on all nodes: <ulink

        url="http://www.drbd.org/">DRBD</ulink> is required if you

        want to use the high availability (HA) features of Ganeti, but

        optional if you don't require HA or only run Ganeti on

        single-node clusters. You can upgrade a non-HA cluster to an

        HA one later, but you might need to export and re-import all

        your instances to take advantage of the new features.

      </para>

      <para>

        Supported DRBD version: the <literal>0.7</literal>

        series. It's recommended to have at least version

        <literal>0.7.24</literal> if you use <command>udev</command>

        since older versions have a bug related to device discovery

        which can be triggered in cases of hard drive failure.

      </para>