Synnefo » snf-ganeti » ganeti-local

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [

]>

  <article class="specification">

  <articleinfo>

    <title>Ganeti customisation using hooks</title>

  </articleinfo>

  <para>Documents ganeti version 1.2</para>

  <section>

    <title>Introduction</title>

    <para>

      In order to allow customisation of operations, ganeti will run

      scripts under <filename

      class="directory">/etc/ganeti/hooks</filename> based on certain

      rules.

    </para>

      <para>This is similar to the <filename

      class="directory">/etc/network/</filename> structure present in

      Debian for network interface handling.</para>

    </section>

    <section>

      <title>Organisation</title>

      <para>For every operation, two sets of scripts are run:

      <itemizedlist>

          <listitem>

            <simpara>pre phase (for authorization/checking)</simpara>

          </listitem>

          <listitem>

            <simpara>post phase (for logging)</simpara>

          </listitem>

        </itemizedlist>

      </para>

      <para>Also, for each operation, the scripts are run on one or

      more nodes, depending on the operation type.</para>

      <para>Note that, even though we call them scripts, we are

      actually talking about any executable.</para>

      <section>

        <title><emphasis>pre</emphasis> scripts</title>

        <para>The <emphasis>pre</emphasis> scripts have a definite

        target: to check that the operation is allowed given the

        site-specific constraints. You could have, for example, a rule

        that says every new instance is required to exists in a

        database; to implement this, you could write a script that

        checks the new instance parameters against your

        database.</para>

        <para>The objective of these scripts should be their return

        code (zero or non-zero for success and failure). However, if

        they modify the environment in any way, they should be

        idempotent, as failed executions could be restarted and thus

        the script(s) run again with exactly the same

        parameters.</para>

      <para>

        Note that if a node is unreachable at the time a hooks is run,

        this will not be interpreted as a deny for the execution. In

        other words, only an actual error returned from a script will

        cause abort, and not an unreachable node.

      </para>

      <para>

        Therefore, if you want to guarantee that a hook script is run

        and denies an action, it's best to put it on the master node.

      </para>

      </section>

      <section>

        <title><emphasis>post</emphasis> scripts</title>

        <para>These scripts should do whatever you need as a reaction

        to the completion of an operation. Their return code is not

        checked (but logged), and they should not depend on the fact

        that the <emphasis>pre</emphasis> scripts have been

        run.</para>

      </section>

      <section>

        <title>Naming</title>

        <para>The allowed names for the scripts consist of (similar to

        <citerefentry> <refentrytitle>run-parts</refentrytitle>

        <manvolnum>8</manvolnum> </citerefentry>) upper and lower

        case, digits, underscores and hyphens. In other words, the

        regexp

        <computeroutput>^[a-zA-Z0-9_-]+$</computeroutput>. Also,

        non-executable scripts will be ignored.

        </para>

      </section>

      <section>

        <title>Order of execution</title>

        <para>On a single node, the scripts in a directory are run in

        lexicographic order (more exactly, the python string

        comparison order). It is advisable to implement the usual

        <emphasis>NN-name</emphasis> convention where

        <emphasis>NN</emphasis> is a two digit number.</para>

        <para>For an operation whose hooks are run on multiple nodes,

        there is no specific ordering of nodes with regard to hooks

        execution; you should assume that the scripts are run in

        parallel on the target nodes (keeping on each node the above

        specified ordering).  If you need any kind of inter-node

        synchronisation, you have to implement it yourself in the

        scripts.</para>

      </section>

      <section>

        <title>Execution environment</title>

        <para>The scripts will be run as follows:

          <itemizedlist>

            <listitem>

              <simpara>no command line arguments</simpara>

            </listitem>

            <listitem>

              <simpara>no controlling <acronym>tty</acronym></simpara>

            </listitem>

            <listitem>

              <simpara><varname>stdin</varname> is

                actually <filename>/dev/null</filename></simpara>

            </listitem>

            <listitem>

              <simpara><varname>stdout</varname> and

                <varname>stderr</varname> are directed to

                files</simpara>

            </listitem>

            <listitem>

              <simpara>the <varname>PATH</varname> is reset to

                <literal>/sbin:/bin:/usr/sbin:/usr/bin</literal></simpara>

            </listitem>

            <listitem>

              <simpara>the environment is cleared, and only

                ganeti-specific variables will be left</simpara>

            </listitem>

          </itemizedlist>

        </para>

      <para>All informations about the cluster is passed using

      environment variables. Different operations will have sligthly

      different environments, but most of the variables are

      common.</para>

    </section>

    <section>

      <title>Operation list</title>

      <table>

        <title>Operation list</title>

        <tgroup cols="7">

          <colspec>

          <colspec>

          <colspec>

          <colspec>

          <colspec>

          <colspec colname="prehooks">

          <colspec colname="posthooks">

          <spanspec namest="prehooks" nameend="posthooks"

            spanname="bothhooks">

          <thead>

            <row>

              <entry>Operation ID</entry>

              <entry>Directory prefix</entry>

              <entry>Description</entry>

              <entry>Command</entry>

              <entry>Supported env. variables</entry>

              <entry><emphasis>pre</emphasis> hooks</entry>

              <entry><emphasis>post</emphasis> hooks</entry>

            </row>

          </thead>

          <tbody>

            <row>

              <entry>OP_INIT_CLUSTER</entry>

              <entry><filename class="directory">cluster-init</filename></entry>

              <entry>Initialises the cluster</entry>

              <entry><computeroutput>gnt-cluster init</computeroutput></entry>

              <entry><constant>CLUSTER</constant>, <constant>MASTER</constant></entry>

              <entry spanname="bothhooks">master node, cluster name</entry>

            </row>

            <row>

              <entry>OP_MASTER_FAILOVER</entry>

              <entry><filename class="directory">master-failover</filename></entry>

              <entry>Changes the master</entry>

              <entry><computeroutput>gnt-cluster master-failover</computeroutput></entry>

              <entry><constant>OLD_MASTER</constant>, <constant>NEW_MASTER</constant></entry>

              <entry>the new master</entry>

              <entry>all nodes</entry>

            </row>

            <row>

              <entry>OP_ADD_NODE</entry>

              <entry><filename class="directory">node-add</filename></entry>

              <entry>Adds a new node to the cluster</entry>

              <entry><computeroutput>gnt-node add</computeroutput></entry>

              <entry><constant>NODE_NAME</constant>, <constant>NODE_PIP</constant>, <constant>NODE_SIP</constant></entry>

              <entry>all existing nodes</entry>

              <entry>all existing nodes plus the new node</entry>

            </row>

            <row>

              <entry>OP_REMOVE_NODE</entry>

              <entry><filename class="directory">node-remove</filename></entry>

              <entry>Removes a node from the cluster</entry>

              <entry><computeroutput>gnt-node remove</computeroutput></entry>

              <entry><constant>NODE_NAME</constant></entry>

              <entry spanname="bothhooks">all existing nodes except the removed node</entry>

            </row>

            <row>

              <entry>OP_INSTANCE_ADD</entry>

              <entry><filename class="directory">instance-add</filename></entry>

              <entry>Creates a new instance</entry>

              <entry><computeroutput>gnt-instance add</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant>, <constant>DISK_TEMPLATE</constant>, <constant>MEM_SIZE</constant>, <constant>DISK_SIZE</constant>, <constant>SWAP_SIZE</constant>, <constant>VCPUS</constant>, <constant>INSTANCE_IP</constant>, <constant>INSTANCE_ADD_MODE</constant>, <constant>SRC_NODE</constant>, <constant>SRC_PATH</constant>, <constant>SRC_IMAGE</constant></entry>

              <entry spanname="bothhooks" morerows="4">master node, primary and

                   secondary nodes</entry>

            </row>

            <row>

              <entry>OP_BACKUP_EXPORT</entry>

              <entry><filename class="directory">instance-export</filename></entry>

              <entry>Export the instance</entry>

              <entry><computeroutput>gnt-backup export</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>EXPORT_NODE</constant>, <constant>EXPORT_DO_SHUTDOWN</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_START</entry>

              <entry><filename class="directory">instance-start</filename></entry>

              <entry>Starts an instance</entry>

              <entry><computeroutput>gnt-instance start</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant>, <constant>FORCE</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_SHUTDOWN</entry>

              <entry><filename class="directory">instance-shutdown</filename></entry>

              <entry>Stops an instance</entry>

              <entry><computeroutput>gnt-instance shutdown</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_MODIFY</entry>

              <entry><filename class="directory">instance-modify</filename></entry>

              <entry>Modifies the instance parameters.</entry>

              <entry><computeroutput>gnt-instance modify</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>MEM_SIZE</constant>, <constant>VCPUS</constant>, <constant>INSTANCE_IP</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_FAILOVER</entry>

              <entry><filename class="directory">instance-failover</filename></entry>

              <entry>Failover an instance</entry>

              <entry><computeroutput>gnt-instance start</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant>, <constant>IGNORE_CONSISTENCY</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_REMOVE</entry>

              <entry><filename class="directory">instance-remove</filename></entry>

              <entry>Remove an instance</entry>

              <entry><computeroutput>gnt-instance remove</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>INSTANCE_PRIMARY</constant>, <constant>INSTANCE_SECONDARIES</constant></entry>

              <entry spanname="bothhooks">master node</entry>

            </row>

            <row>

              <entry>OP_INSTANCE_ADD_MDDRBD</entry>

              <entry><filename class="directory">mirror-add</filename></entry>

              <entry>Adds a mirror component</entry>

              <entry><computeroutput>gnt-instance add-mirror</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>NEW_SECONDARY</constant>, <constant>DISK_NAME</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_REMOVE_MDDRBD</entry>

              <entry><filename class="directory">mirror-remove</filename></entry>

              <entry>Removes a mirror component</entry>

              <entry><computeroutput>gnt-instance remove-mirror</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>OLD_SECONDARY</constant>, <constant>DISK_NAME</constant>, <constant>DISK_ID</constant></entry>

            </row>

            <row>

              <entry>OP_INSTANCE_REPLACE_DISKS</entry>

              <entry><filename class="directory">mirror-replace</filename></entry>

              <entry>Replace all mirror components</entry>

              <entry><computeroutput>gnt-instance replace-disks</computeroutput></entry>

              <entry><constant>INSTANCE_NAME</constant>, <constant>OLD_SECONDARY</constant>, <constant>NEW_SECONDARY</constant></entry>

            </row>

          </tbody>

        </tgroup>

      </table>

    </section>

    <section>

      <title>Environment variables</title>

      <para>Note that all variables listed here are actually prefixed

      with <constant>GANETI_</constant> in order to provide a

      different namespace.</para>

      <section>

        <title>Common variables</title>

        <para>This is the list of environment variables supported by

        all operations:</para>

        <variablelist>

          <varlistentry>

            <term>HOOKS_VERSION</term>

            <listitem>

              <para>Documents the hooks interface version. In case this

            doesnt match what the script expects, it should not

            run. The documents conforms to the version

            <literal>1</literal>.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>HOOKS_PHASE</term>

            <listitem>

              <para>one of <constant>PRE</constant> or

              <constant>POST</constant> denoting which phase are we

              in.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>CLUSTER</term>

            <listitem>

              <para>the cluster name</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>MASTER</term>

            <listitem>

              <para>the master node</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>OP_ID</term>

            <listitem>

              <para>one of the <constant>OP_*</constant> values from

              the table of operations</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>OBJECT_TYPE</term>

            <listitem>

              <para>one of <simplelist type="inline">

                  <member><constant>INSTANCE</constant></member>

                  <member><constant>NODE</constant></member>

                  <member><constant>CLUSTER</constant></member>

                </simplelist>, showing the target of the operation.

             </para>

            </listitem>

          </varlistentry>

          <!-- commented out since it causes problems in our rpc

               multi-node optimised calls

          <varlistentry>

            <term>HOST_NAME</term>

            <listitem>

              <para>The name of the node the hook is run on as known by

            the cluster.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>HOST_TYPE</term>

            <listitem>

              <para>one of <simplelist type="inline">

                  <member><constant>MASTER</constant></member>

                  <member><constant>NODE</constant></member>

                </simplelist>, showing the role of this node in the cluster.

             </para>

            </listitem>

          </varlistentry>

          -->

        </variablelist>

      </section>

      <section>

        <title>Specialised variables</title>

        <para>This is the list of variables which are specific to one

        or more operations.</para>

        <variablelist>

          <varlistentry>

            <term>INSTANCE_NAME</term>

            <listitem>

              <para>The name of the instance which is the target of

              the operation.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_DISK_TYPE</term>

            <listitem>

              <para>The disk type for the instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_DISK_SIZE</term>

            <listitem>

              <para>The (OS) disk size for the instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_OS</term>

            <listitem>

              <para>The name of the instance OS.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_PRIMARY</term>

            <listitem>

              <para>The name of the node which is the primary for the

              instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_SECONDARIES</term>

            <listitem>

              <para>Space-separated list of secondary nodes for the

              instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>NODE_NAME</term>

            <listitem>

              <para>The target node of this operation (not the node on

              which the hook runs).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>NODE_PIP</term>

            <listitem>

              <para>The primary IP of the target node (the one over

              which inter-node communication is done).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>NODE_SIP</term>

            <listitem>

              <para>The secondary IP of the target node (the one over

              which drbd replication is done). This can be equal to

              the primary ip, in case the cluster is not

              dual-homed.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>OLD_MASTER</term>

            <term>NEW_MASTER</term>

            <listitem>

              <para>The old, respectively the new master for the

              master failover operation.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>FORCE</term>

            <listitem>

              <para>This is provided by some operations when the user

              gave this flag.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>IGNORE_CONSISTENCY</term>

            <listitem>

              <para>The user has specified this flag. It is used when

              failing over instances in case the primary node is

              down.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>MEM_SIZE, DISK_SIZE, SWAP_SIZE, VCPUS</term>

            <listitem>

              <para>The memory, disk, swap size and the number of

              processor selected for the instance (in

              <command>gnt-instance add</command> or

              <command>gnt-instance modify</command>).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_IP</term>

            <listitem>

              <para>If defined, the instance IP in the

              <command>gnt-instance add</command> and

              <command>gnt-instance set</command> commands. If not

              defined, it means that no IP has been defined.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>DISK_TEMPLATE</term>

            <listitem>

              <para>The disk template type when creating the instance.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>INSTANCE_ADD_MODE</term>

            <listitem>

              <para>The mode of the create: either

              <constant>create</constant> for create from scratch or

              <constant>import</constant> for restoring from an

              exported image.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>SRC_NODE, SRC_PATH, SRC_IMAGE</term>

            <listitem>

              <para>In case the instance has been added by import,

              these variables are defined and point to the source

              node, source path (the directory containing the image

              and the config file) and the source disk image

              file.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>DISK_NAME</term>

            <listitem>

              <para>The disk name (either <filename>sda</filename> or

              <filename>sdb</filename>) in mirror operations

              (add/remove mirror).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>DISK_ID</term>

            <listitem>

              <para>The disk id for mirror remove operations. You can

              look this up using <command>gnt-instance

              info</command>.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>NEW_SECONDARY</term>

            <listitem>

              <para>The name of the node on which the new mirror

              componet is being added. This can be the name of the

              current secondary, if the new mirror is on the same

              secondary.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>OLD_SECONDARY</term>

            <listitem>

              <para>The name of the old secondary. This is used in

              both <command>replace-disks</command> and

              <command>remove-mirror</command>. Note that this can be

              equal to the new secondary (only

              <command>replace-disks</command> has both variables) if

              the secondary node hasn't actually changed).</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>EXPORT_NODE</term>

            <listitem>

              <para>The node on which the exported image of the

              instance was done.</para>

            </listitem>

          </varlistentry>

          <varlistentry>

            <term>EXPORT_DO_SHUTDOWN</term>

            <listitem>

              <para>This variable tells if the instance has been

              shutdown or not while doing the export. In the "was

              shutdown" case, it's likely that the filesystem is

              consistent, whereas in the "did not shutdown" case, the

              filesystem would need a check (journal replay or full

              fsck) in order to guarantee consistency.</para>

            </listitem>

          </varlistentry>

        </variablelist>

      </section>

    </section>

  </section>

  </article>