1 <!doctype refentry PUBLIC "-//OASIS//DTD DocBook V4.1//EN" [
3 <!-- Fill in your name for FIRSTNAME and SURNAME. -->
4 <!-- Please adjust the date whenever revising the manpage. -->
5 <!ENTITY dhdate "<date>February 12, 2009</date>">
6 <!-- SECTION should be 1-8, maybe w/ subsection other parameters are
7 allowed: see man(7), man(1). -->
8 <!ENTITY dhsection "<manvolnum>8</manvolnum>">
9 <!ENTITY dhucpackage "<refentrytitle>gnt-cluster</refentrytitle>">
10 <!ENTITY dhpackage "gnt-cluster">
12 <!ENTITY debian "<productname>Debian</productname>">
13 <!ENTITY gnu "<acronym>GNU</acronym>">
14 <!ENTITY gpl "&gnu; <acronym>GPL</acronym>">
15 <!ENTITY footer SYSTEM "footer.sgml">
25 <holder>Google Inc.</holder>
33 <refmiscinfo>ganeti 2.0</refmiscinfo>
36 <refname>&dhpackage;</refname>
38 <refpurpose>ganeti administration, cluster-wide</refpurpose>
42 <command>&dhpackage; </command>
44 <arg choice="req">command</arg>
45 <arg>arguments...</arg>
49 <title>DESCRIPTION</title>
52 The <command>&dhpackage;</command> is used for cluster-wide
53 administration in the ganeti system.
58 <title>COMMANDS</title>
61 <title>ADD-TAGS</title>
64 <command>add-tags</command>
65 <arg choice="opt">--from <replaceable>file</replaceable></arg>
67 rep="repeat"><replaceable>tag</replaceable></arg>
71 Add tags to the cluster. If any of the tags contains invalid
72 characters, the entire operation will abort.
76 If the <option>--from</option> option is given, the list of
77 tags will be extended with the contents of that file (each
78 line becomes a tag). In this case, there is not need to pass
79 tags on the command line (if you do, both sources will be
80 used). A file name of - will be interpreted as stdin.
85 <title>COMMAND</title>
88 <command>command</command>
89 <arg>-n <replaceable>node</replaceable></arg>
90 <arg choice="req"><replaceable>command</replaceable></arg>
94 Executes a command on all nodes. If the option
95 <option>-n</option> is not given, the command will be executed
96 on all nodes, otherwise it will be executed only on the
97 node(s) specified. Use the option multiple times for running
98 it on multiple nodes, like:
101 # gnt-cluster command -n node1.example.com -n node2.example.com date
107 The command is executed serially on the selected nodes. If the
108 master node is present in the list, the command will be
109 executed last on the master. Regarding the other nodes, the
110 execution order is somewhat alphabetic, so that
111 node2.example.com will be earlier than node10.example.com but
112 after node1.example.com.
116 So given the node names node1, node2, node3, node10, node11,
117 with node3 being the master, the order will be: node1, node2,
118 node10, node11, node3.
122 The command is constructed by concatenating all other command
123 line arguments. For example, to list the contents of the
124 <filename class="directory">/etc</filename> directory on all
128 # gnt-cluster command ls -l /etc
131 and the command which will be executed will be
132 <computeroutput>"ls -l /etc"</computeroutput>
137 <title>COPYFILE</title>
140 <command>copyfile</command>
141 <arg>-n <replaceable>node</replaceable></arg>
142 <arg choice="req"><replaceable>file</replaceable></arg>
146 Copies a file to all or to some nodes. The argument specifies
147 the source file (on the current system), the
148 <option>-n</option> argument specifies the target node, or
149 nodes if the option is given multiple times. If
150 <option>-n</option> is not given at all, the file will be
155 # gnt-cluster -n node1.example.com -n node2.example.com copyfile /tmp/test
158 This will copy the file <filename>/tmp/test</filename> from
159 the current node to the two named nodes.
164 <title>DESTROY</title>
167 <command>destroy</command>
168 <arg choice="req">--yes-do-it</arg>
172 Remove all configuration files related to the cluster, so that
173 a <command>gnt-cluster init</command> can be done again
178 Since this is a dangerous command, you are required to pass
179 the argument <replaceable>--yes-do-it.</replaceable>
184 <title>GETMASTER</title>
187 <command>getmaster</command>
191 Displays the current master node.
199 <command>info</command>
203 Shows runtime cluster information: cluster name, architecture
204 (32 or 64 bit), master node, node list and instance list.
212 <command>init</command>
214 <arg>-s <replaceable>secondary_ip</replaceable></arg>
216 <arg>-g <replaceable>vg-name</replaceable></arg>
218 <arg>--master-netdev <replaceable>vg-name</replaceable></arg>
220 <arg>-m <replaceable>mac-prefix</replaceable></arg>
222 <arg>--no-lvm-storage</arg>
224 <arg>--no-etc-hosts</arg>
226 <arg>--no-ssh-init</arg>
228 <arg>--file-storage-dir <replaceable>dir</replaceable></arg>
230 <arg>--enabled-hypervisors <replaceable>hypervisors</replaceable></arg>
232 <arg>-t <replaceable>hypervisor name</replaceable></arg>
234 <arg>--hypervisor-parameters <replaceable>hypervisor</replaceable>:<replaceable>hv-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>hv-param</replaceable>=<replaceable>value</replaceable></arg></arg>
236 <arg>--backend-parameters <replaceable>be-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>be-param</replaceable>=<replaceable>value</replaceable></arg></arg>
238 <arg>--nic-parameters <replaceable>nic-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>nic-param</replaceable>=<replaceable>value</replaceable></arg></arg>
240 <arg choice="req"><replaceable>clustername</replaceable></arg>
244 This commands is only run once initially on the first node of
245 the cluster. It will initialize the cluster configuration and
246 setup ssh-keys and more.
250 Note that the <replaceable>clustername</replaceable> is not
251 any random name. It has to be resolvable to an IP address
252 using DNS, and it is best if you give the fully-qualified
253 domain name. This hostname must resolve to an IP address
254 reserved exclusively for this purpose.
258 The cluster can run in two modes: single-home or
259 dual-homed. In the first case, all traffic (both public
260 traffic, inter-node traffic and data replication traffic) goes
261 over the same interface. In the dual-homed case, the data
262 replication traffic goes over the second network. The
263 <option>-s</option> option here marks the cluster as
264 dual-homed and its parameter represents this node's address on
265 the second network. If you initialise the cluster with
266 <option>-s</option>, all nodes added must have a secondary IP
271 Note that for Ganeti it doesn't matter if the secondary
272 network is actually a separate physical network, or is done
273 using tunneling, etc. For performance reasons, it's
274 recommended to use a separate network, of course.
278 The <option>-g</option> option will let you specify a volume group
279 different than 'xenvg' for ganeti to use when creating instance disks.
280 This volume group must have the same name on all nodes. Once the
281 cluster is initialized this can be altered by using the
282 <command>modify</command> command. If you don't want to use lvm
283 storage at all use the <option>--no-lvm-storage</option> option.
284 Once the cluster is initialized you can change this setup with the
285 <command>modify</command> command.
289 The <option>--master-netdev</option> option is useful for specifying a
290 different interface on which the master will activate its IP address.
291 It's important that all nodes have this interface because you'll need
292 it for a master failover.
296 The <option>-m</option> option will let you specify a three byte prefix
297 under which the virtual MAC addresses of your instances will be
298 generated. The prefix must be specified in the format XX:XX:XX and the
303 The <option>--no-lvm-storage</option> option allows you to initialize
304 the cluster without lvm support. This means that only instances using
305 files as storage backend will be possible to create. Once the cluster
306 is initialized you can change this setup with the
307 <command>modify</command> command.
311 The <option>--no-etc-hosts</option> option allows you to initialize the
312 cluster without modifying the <filename>/etc/hosts</filename> file.
316 The <option>--no-ssh-init</option> option allows you to initialize the
317 cluster without creating or distributing SSH key pairs.
321 The <option>--file-storage-dir</option> option allows you
322 set the directory to use for storing the instance disk
323 files when using file storage as backend for instance disks.
327 The <option>--enabled-hypervisors</option> option allows you
328 to set the list of hypervisors that will be enabled for
329 this cluster. Instance hypervisors can only be chosen from
330 the list of enabled hypervisors, and the first entry of this list
331 will be used by default. Currently, the following hypervisors are
365 a simple chroot manager that starts chroot based on a
366 script at the root of the filesystem holding the
372 fake hypervisor for development/testing
380 Either a single hypervisor name or a comma-separated list of
381 hypervisor names can be specified. If this option is not
382 specified, only the xen-pvm hypervisor is enabled by default.
386 The <option>--hypervisor-parameters</option> option allows you
387 to set default hypervisor specific parameters for the
388 cluster. The format of this option is the name of the
389 hypervisor, followed by a colon and a comma-separated list of
390 key=value pairs. The keys available for each hypervisors are
391 detailed in the <citerefentry>
392 <refentrytitle>gnt-instance</refentrytitle>
393 <manvolnum>8</manvolnum> </citerefentry> man page, in the
394 <command>add</command> command plus the following parameters
395 which are only configurable globally (at cluster level):
399 <term>migration_port</term>
401 <simpara>Valid for the Xen PVM and KVM hypervisors.</simpara>
404 This options specifies the TCP port to use for
405 live-migration. For Xen, the same port should be
406 configured on all nodes in
407 the <filename>/etc/xen/xend-config.sxp</filename>
409 key <quote>xend-relocation-port</quote>.
418 The <option>--backend-parameters</option> option allows you to set
419 the default backend parameters for the cluster. The parameter
420 format is a comma-separated list of key=value pairs with the
421 following supported keys:
430 Number of VCPUs to set for an instance by default, must
431 be an integer, will be set to 1 if no specified.
439 Amount of memory to allocate for an instance by default,
440 can be either an integer or an integer followed by a
441 unit (M for mebibytes and G for gibibytes are
442 supported), will be set to 128M if not specified.
447 <term>auto_balance</term>
450 Value of the auto_balance flag for instances to use by
451 default, will be set to true if not specified.
459 The <option>--nic-parameters</option> option allows you to set
460 the default nic parameters for the cluster. The parameter
461 format is a comma-separated list of key=value pairs with the
462 following supported keys:
468 The default nic mode, 'routed' or 'bridged'.
476 In bridged mode the default NIC bridge. In routed mode it
477 represents an hypervisor-vif-script dependent value to allow
478 different instance groups. For example under the KVM default
479 network script it is interpreted as a routing table number or
490 <title>LIST-TAGS</title>
493 <command>list-tags</command>
496 <para>List the tags of the cluster.</para>
500 <title>MASTERFAILOVER</title>
503 <command>masterfailover</command>
504 <arg>--no-voting</arg>
508 Failover the master role to the current node.
512 The <option>--no-voting</option> option skips the remote node agreement
513 checks. This is dangerous, but necessary in some cases (for example
514 failing over the master role in a 2 node cluster with the original master
515 down). If the original master then comes up, it won't be able to start
516 its master daemon because it won't have enough votes, but so won't the
517 new master, if the master daemon ever needs a restart. You can pass
518 --no-voting to ganeti-masterd on the new master to solve this problem,
519 and gnt-cluster redist-conf to make sure the cluster is consistent again.
525 <title>MODIFY</title>
528 <command>modify</command>
530 <arg choice="opt">-g <replaceable>vg-name</replaceable></arg>
532 <arg choice="opt">--no-lvm-storage</arg>
534 <arg choice="opt">--enabled-hypervisors
535 <replaceable>hypervisors</replaceable></arg>
537 <arg choice="opt">--hypervisor-parameters <replaceable>hypervisor</replaceable>:<replaceable>hv-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>hv-param</replaceable>=<replaceable>value</replaceable></arg></arg>
539 <arg choice="opt">--backend-parameters <replaceable>be-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>be-param</replaceable>=<replaceable>value</replaceable></arg></arg>
541 <arg choice="opt">--nic-parameters <replaceable>nic-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>nic-param</replaceable>=<replaceable>value</replaceable></arg></arg>
543 <arg>-C <replaceable>candidate_pool_size</replaceable></arg>
548 Modify the options for the cluster.
552 The <option>-g</option>, <option>--no-lvm-storarge</option>,
553 <option>--enabled-hypervisors</option>,
554 <option>--hypervisor-parameters</option>,
555 <option>--backend-parameters</option> and
556 <option>--nic-parameters</option> and options are
557 described in the <command>init</command> command.
561 The <option>-C</option> options specifies the
562 <varname>candidate_pool_size</varname> cluster parameter. This
563 is the number of nodes that the master will try to keep as
564 <literal>master_candidates</literal>. For more details about
565 this role and other node roles, see the <citerefentry>
566 <refentrytitle>ganeti</refentrytitle><manvolnum>7</manvolnum>
567 </citerefentry>. If you increase the size, the master will
568 automatically promote as many nodes as required and possible
569 to reach the intended number.
577 <command>queue</command>
578 <arg choice="opt">drain</arg>
579 <arg choice="opt">undrain</arg>
580 <arg choice="opt">info</arg>
585 Change job queue properties.
589 The <option>drain</option> option sets the drain flag on the
590 job queue. No new jobs will be accepted, but jobs already in
591 the queue will be processed.
595 The <option>undrain</option> will unset the drain flag on the
596 job queue. New jobs will be accepted.
600 The <option>info</option> option shows the properties of the
606 <title>WATCHER</title>
609 <command>watcher</command>
611 <arg>pause <replaceable>duration</replaceable></arg>
618 Make the watcher pause or let it continue.
622 The <option>pause</option> option causes the watcher to pause for
623 <replaceable>duration</replaceable> seconds.
627 The <option>continue</option> option will let the watcher continue.
631 The <option>info</option> option shows whether the watcher is currently
637 <title>redist-conf</title>
639 <command>redist-conf</command>
644 This command forces a full push of configuration files from
645 the master node to the other nodes in the cluster. This is
646 normally not needed, but can be run if the
647 <command>verify</command> complains about configuration
652 The <option>--submit</option> option is used to send the job
653 to the master daemon but not wait for its completion. The job
654 ID will be shown so that it can be examined via
655 <command>gnt-job info</command>.
660 <title>REMOVE-TAGS</title>
663 <command>remove-tags</command>
664 <arg choice="opt">--from <replaceable>file</replaceable></arg>
666 rep="repeat"><replaceable>tag</replaceable></arg>
670 Remove tags from the cluster. If any of the tags are not
671 existing on the cluster, the entire operation will abort.
675 If the <option>--from</option> option is given, the list of
676 tags will be extended with the contents of that file (each
677 line becomes a tag). In this case, there is not need to pass
678 tags on the command line (if you do, both sources will be
679 used). A file name of - will be interpreted as stdin.
684 <title>RENAME</title>
687 <command>rename</command>
689 <arg choice="req"><replaceable>name</replaceable></arg>
693 Renames the cluster and in the process updates the master IP
694 address to the one the new name resolves to. At least one of
695 either the name or the IP address must be different, otherwise
696 the operation will be aborted.
700 Note that since this command can be dangerous (especially when
701 run over SSH), the command will require confirmation unless
702 run with the <option>-f</option> option.
707 <title>REPAIR-DISK-SIZES</title>
710 <command>repair-disk-sizes</command>
711 <arg rep="repeat">instance</arg>
715 This command checks that the recorded size of the given
716 instance's disks matches the actual size and updates any
717 mismatches found. This is needed if the Ganeti configuration
718 is no longer consistent with reality, as it will impact some
719 disk operations. If no arguments are given, all instances will
724 Note that only active disks can be checked by this command; in
725 case a disk cannot be activated it's advised to use
726 <command>gnt-instance activate-disks --ignore-size ...</command> to
727 force activation without regard to the
732 When the all disk sizes are consistent, the command will
733 return no output. Otherwise it will log details about the
734 inconsistencies in the configuration.
739 <title>SEARCH-TAGS</title>
742 <command>search-tags</command>
743 <arg choice="req"><replaceable>pattern</replaceable></arg>
747 Searches the tags on all objects in the cluster (the cluster
748 itself, the nodes and the instances) for a given pattern. The
749 pattern is interpreted as a regular expression and a search
750 will be done on it (i.e. the given pattern is not anchored to
751 the beggining of the string; if you want that, prefix the
752 pattern with <literal>^</literal>).
756 If no tags are matching the pattern, the exit code of the
757 command will be one. If there is at least one match, the exit
758 code will be zero. Each match is listed on one line, the
759 object and the tag separated by a space. The cluster will be
760 listed as <filename>/cluster</filename>, a node will be listed
762 <filename>/nodes/<replaceable>name</replaceable></filename>,
764 <filename>/instances/<replaceable>name</replaceable></filename>.
768 # gnt-cluster search-tags time
769 /cluster ctime:2007-09-01
770 /nodes/node1.example.com mtime:2007-10-04
775 <title>VERIFY</title>
778 <command>verify</command>
779 <arg choice="opt">--no-nplus1-mem</arg>
783 Verify correctness of cluster configuration. This is safe with
784 respect to running instances, and incurs no downtime of the
789 If the <option>--no-nplus1-mem</option> option is given, ganeti won't
790 check whether if it loses a node it can restart all the instances on
791 their secondaries (and report an error otherwise).
796 <title>VERIFY-DISKS</title>
799 <command>verify-disks</command>
803 The command checks which instances have degraded DRBD disks
804 and activates the disks of those instances.
808 This command is run from the <command>ganeti-watcher</command>
809 tool, which also has a different, complementary algorithm for
810 doing this check. Together, these two should ensure that DRBD
811 disks are kept consistent.
816 <title>VERSION</title>
819 <command>version</command>
823 Show the cluster version.
833 <!-- Keep this comment at the end of the file
838 sgml-minimize-attributes:nil
839 sgml-always-quote-attributes:t
842 sgml-parent-document:nil
843 sgml-default-dtd-file:nil
844 sgml-exposed-tags:nil
845 sgml-local-catalogs:nil
846 sgml-local-ecat-files:nil