1 <!doctype refentry PUBLIC "-//OASIS//DTD DocBook V4.1//EN" [
3 <!-- Fill in your name for FIRSTNAME and SURNAME. -->
4 <!-- Please adjust the date whenever revising the manpage. -->
5 <!ENTITY dhdate "<date>May 16, 2007</date>">
6 <!-- SECTION should be 1-8, maybe w/ subsection other parameters are
7 allowed: see man(7), man(1). -->
8 <!ENTITY dhsection "<manvolnum>8</manvolnum>">
9 <!ENTITY dhucpackage "<refentrytitle>gnt-instance</refentrytitle>">
10 <!ENTITY dhpackage "gnt-instance">
12 <!ENTITY debian "<productname>Debian</productname>">
13 <!ENTITY gnu "<acronym>GNU</acronym>">
14 <!ENTITY gpl "&gnu; <acronym>GPL</acronym>">
15 <!ENTITY footer SYSTEM "footer.sgml">
23 <holder>Google Inc.</holder>
31 <refmiscinfo>ganeti 1.2</refmiscinfo>
34 <refname>&dhpackage;</refname>
36 <refpurpose>ganeti instance administration</refpurpose>
40 <command>&dhpackage; </command>
42 <arg choice="req">command</arg>
43 <arg>arguments...</arg>
47 <title>DESCRIPTION</title>
50 The <command>&dhpackage;</command> is used for instance
51 administration in the ganeti system.
56 <title>COMMANDS</title>
59 <title>Creation/removal/querying</title>
64 <command>add</command>
65 <arg>-s <replaceable>disksize</replaceable></arg>
66 <arg>--swap-size <replaceable>disksize</replaceable></arg>
67 <arg>-m <replaceable>memsize</replaceable></arg>
69 <arg>-o <replaceable>os-type</replaceable></arg>
70 <arg>-b <replaceable>bridge</replaceable></arg>
71 <arg>--mac <replaceable>MAC-address</replaceable></arg>
72 <arg>--hvm-boot-order <replaceable>boot-order</replaceable></arg>
74 <arg>--kernel <group choice="req">
76 <arg><replaceable>kernel_path</replaceable></arg>
79 <arg>--initrd <group choice="req">
82 <arg><replaceable>initrd_path</replaceable></arg>
85 <arg>--file-storage-dir <replaceable>dir_path</replaceable></arg>
86 <arg>--file-driver <group choice="req">
91 <arg choice="req">-t<group>
99 <arg choice="req">-n <replaceable>node<optional>:secondary-node</optional></replaceable></arg>
100 <arg choice="req"><replaceable>instance</replaceable></arg>
104 Creates a new instance on the specified
105 host. <replaceable>instance</replaceable> must be in DNS and
106 resolve to a IP in the same network as the nodes in the
111 The <option>-s</option> option specifies the disk size for
112 the instance, in mebibytes (defaults to
113 <constant>20480MiB</constant> =
114 <constant>20GiB</constant>). You can also use one of the
115 suffixes <literal>m</literal>, <literal>g</literal> or
116 <literal>t</literal> to specificy the exact the units used;
117 these suffixes map to mebibytes, gibibytes and tebibytes.
121 The <option>--swap-size</option> option specifies the swap
122 disk size (in mebibytes) for the instance (the one presented
123 as <filename class="devicefile">/dev/sdb</filename>). The
124 default is <constant>4096MiB</constant>. As for the disk
125 size, you can specify other suffixes.
129 The <option>-m</option> option specifies the memory size for
130 the instance, in mebibytes (defaults to 128 MiB). Again, you
131 can use other suffixes (e.g. <userinput>2g</userinput>).
135 The <option>-o</option> options specifies the operating
136 system to be installed. The available operating systems can
137 be listed with <command>gnt-os list</command>.
141 The <option>-b</option> option specifies the bridge to which the
142 instance will be connected. (defaults to the cluster-wide default
143 bridge specified at cluster initialization time).
147 The <option>--mac</option> option specifies the MAC address
148 of the ethernet interface for the instance. If this option
149 is not specified, a new MAC address is generated randomly with
150 the configured MAC prefix. The randomly generated MAC
151 address is guaranteed to be unique among the instances of
156 The <option>--hvm-boot-order</option> option specifies the
157 boot device order for Xen HVM instances. The boot order is a
158 string of letters listing the boot devices, with valid
159 device letters being:
200 The option is only relevant for Xen HVM instances and
201 ignored by all other instances types.
205 The <option>--kernel</option> options allows the instance to
206 use a custom kernel (if a filename is passed) or to use the
207 default kernel (<filename>@CUSTOM_XEN_KERNEL@</filename>), if the
208 string <constant>default</constant> is passed.
212 The <option>--initrd</option> option is similar: it allows
213 the instance to use a custom initrd (if a filename is
214 passed) or to use the default initrd
215 (<filename>@CUSTOM_XEN_INITRD@</filename>), if the string
216 <constant>default</constant> is passed, or to disable the
217 use of an initrd, if the string <constant>none</constant> is
218 passed. Note that in the case the instance is set to use the
219 default initrd and it doesn't exist, it will be silently
220 ignored; if the instance is set to use a custom initrd and
221 it doesn't exist, this will be treated as an error and will
222 prevent the startup of the instance.
226 The <option>-t</option> options specifies the disk layout type for
227 the instance. The available choices are:
230 <term>diskless</term>
233 This creates an instance with no disks. Its useful for
234 testing only (or other special cases).
241 <para>Disk devices will be regular files.</para>
247 <para>Disk devices will be logical volumes.</para>
254 Disk devices will be drbd (version 8.x) on top of
263 The optional second value of the <option>--node</option> is used for
264 the remote raid template type and specifies the remote node.
268 If you do not want gnt-instance to wait for the disk mirror
269 to be synced, use the <option>--no-wait-for-sync</option>
274 The <option>--file-storage-dir</option> specifies the relative path
275 under the cluster-wide file storage directory to store file-based
276 disks. It is useful for having different subdirectories for
277 different instances. The full path of the directory where the disk
278 files are stored will consist of cluster-wide file storage directory
279 + optional subdirectory + instance name. Example:
280 /srv/ganeti/file-storage/mysubdir/instance1.example.com. This option
281 is only relevant for instances using the file storage backend.
285 The <option>--file-driver</option> specifies the driver to use for
286 file-based disks. Note that currently these drivers work with the
287 xen hypervisor only. This option is only relevant for instances using
288 the file storage backend. The available choices are:
293 <para>Kernel loopback driver.</para>
299 <para>blktap driver.</para>
306 The loop driver uses loopback devices to access the filesystem
307 within the file. However, running I/O intensive applications
308 in your instance using the loop driver might result in slowdowns.
309 Furthermore, if you use the loopback driver consider increasing
310 the maximum amount of loopback devices (on most systems it's 8)
311 using the max_loop param.
315 In order to be able to use the blktap driver you should check
316 if the 'blktapctrl' user space disk agent is running (usually
317 automatically started via xend). This user-level disk I/O
318 interface has the advantage of better performance. Especially
319 if you use a network file system (e.g. NFS) to store your instances
320 this is the recommended choice.
326 # gnt-instance add -t file -s 30g -m 512 -o debian-etch \
327 -n node1.example.com --file-storage-dir=mysubdir instance1.example.com
328 # gnt-instance add -t plain -s 30g -m 512 -o debian-etch \
329 -n node1.example.com instance1.example.com
330 # gnt-instance add -t drbd -s 30g -m 512 -o debian-etch \
331 -n node1.example.com:node2.example.com instance2.example.com
337 <title>REMOVE</title>
340 <command>remove</command>
341 <arg>--ignore-failures</arg>
342 <arg choice="req"><replaceable>instance</replaceable></arg>
346 Remove an instance. This will remove all data from the
347 instance and there is <emphasis>no way back</emphasis>. If
348 you are not sure if you use an instance again, use
349 <command>shutdown</command> first and leave it in the
350 shutdown state for a while.
355 The <option>--ignore-failures</option> option will cause the
356 removal to proceed even in the presence of errors during the
357 removal of the instance (e.g. during the shutdown or the
358 disk removal). If this option is not given, the command will
359 stop at the first error.
365 # gnt-instance remove instance1.example.com
374 <command>list</command>
375 <arg>--no-headers</arg>
376 <arg>--separator=<replaceable>SEPARATOR</replaceable></arg>
377 <arg>-o <replaceable>FIELD,...</replaceable></arg>
381 Shows the currently configured instances with memory usage,
382 disk usage, the node they are running on, and the CPU time,
383 counted in seconds, used by each instance since its latest
388 The <option>--no-headers</option> option will skip the
389 initial header line. The <option>--separator</option> option
390 takes an argument which denotes what will be used between
391 the output fields. Both these options are to help scripting.
395 The <option>-o</option> option takes a comma-separated list
396 of output fields. The available fields and their meaning
402 <simpara>the instance name</simpara>
408 <simpara>the OS of the instance</simpara>
414 <simpara>the primary node of the instance</simpara>
420 <simpara>comma-separated list of secondary nodes for the
421 instance; usually this will be just one node</simpara>
425 <term>admin_state</term>
427 <simpara>the desired state of the instance (either "yes"
428 or "no" denoting the instance should run or
433 <term>admin_ram</term>
435 <simpara>the desired memory for the instance</simpara>
439 <term>disk_template</term>
441 <simpara>the disk template of the instance</simpara>
445 <term>oper_state</term>
447 <simpara>the actual state of the instance; can be
448 one of the values "running", "stopped", "(node
455 <simpara>combined form of admin_state and oper_stat;
457 <computeroutput>ERROR_nodedown</computeroutput> if the
458 node of the instance is down,
459 <computeroutput>ERROR_down</computeroutput> if the
460 instance should run but is down,
461 <computeroutput>ERROR_up</computeroutput> if the
462 instance should be stopped but is actually running,
463 <computeroutput>ADMIN_down</computeroutput> if the
464 instance has been stopped (and is stopped) and
465 <computeroutput>running</computeroutput> if the
466 instance is set to be running (and is
471 <term>oper_ram</term>
473 <simpara>the actual memory usage of the instance as seen
474 by the hypervisor</simpara>
480 <simpara>the ip address ganeti recognizes as associated with
481 the instance interface</simpara>
487 <simpara>the instance interface MAC address</simpara>
493 <simpara>bridge the instance is connected to
498 <term>sda_size</term>
500 <simpara>the size of the instance's first disk</simpara>
504 <term>sdb_size</term>
506 <simpara>the size of the instance's second disk</simpara>
512 <simpara>the number of VCPUs allocated to the
520 There is a subtle grouping about the available output
521 fields: all fields except for <option>oper_state</option>
522 and <option>oper_ram</option> are configuration value and
523 not run-time values. So if you don't select any of the
524 <option>oper_*</option> fields, the query will be satisfied
525 instantly from the cluster configuration, without having to
526 ask the remote nodes for the data. This can be helpful for
527 big clusters when you only want some data and it makes sense
528 to specify a reduced set of output fields.
531 <para>The default output field list is:
532 <simplelist type="inline">
533 <member>name</member>
535 <member>pnode</member>
536 <member>admin_state</member>
537 <member>oper_state</member>
538 <member>oper_ram</member>
547 <command>info</command>
548 <arg rep="repeat"><replaceable>instance</replaceable></arg>
552 Show detailed information about the (given) instances. This
553 is different from <command>list</command> as it shows
554 detailed data about the instance's disks (especially useful
555 for remote raid templates).
560 <title>MODIFY</title>
563 <command>modify</command>
564 <arg choice="opt">-m <replaceable>memsize</replaceable></arg>
565 <arg choice="opt">-p <replaceable>vcpus</replaceable></arg>
566 <arg choice="opt">-i <replaceable>ip</replaceable></arg>
567 <arg choice="opt">-b <replaceable>bridge</replaceable></arg>
568 <arg choice="opt">--mac <replaceable>MAC-address</replaceable></arg>
569 <arg>--hvm-boot-order <replaceable>boot-order</replaceable></arg>
571 <arg>--kernel <group choice="req">
573 <arg><replaceable>kernel_path</replaceable></arg>
576 <arg>--initrd <group choice="req">
579 <arg><replaceable>initrd_path</replaceable></arg>
582 <arg choice="req"><replaceable>instance</replaceable></arg>
586 Modify the memory size, number of vcpus, ip address, MAC
587 address and/or bridge for an instance.
591 The memory size is given in MiB. Note that you need to give
592 at least one of the arguments, otherwise the command
597 The <option>--kernel</option>, <option>--initrd</option>
598 and <option>--hvm-boot-order</option>
599 options are described in the <command>add</command> command.
603 Additionally, the HVM boot order can be reset to the default
604 values by using <option>--hvm-boot-order=default</option>.
608 All the changes take effect at the next restart. If the
609 instance is running, there is no effect on the instance.
614 <title>REINSTALL</title>
617 <command>reinstall</command>
618 <arg choice="opt">-o <replaceable>os-type</replaceable></arg>
619 <arg choice="opt">-f <replaceable>force</replaceable></arg>
620 <arg choice="req"><replaceable>instance</replaceable></arg>
624 Reinstalls the operating system on the given instance. The instance
625 must be stopped when running this command. If the
626 <option>--os-type</option> is specified, the operating system is
632 <title>RENAME</title>
635 <command>rename</command>
636 <arg>--no-ip-check</arg>
637 <arg choice="req"><replaceable>instance</replaceable></arg>
638 <arg choice="req"><replaceable>new_name</replaceable></arg>
642 Renames the given instance. The instance must be stopped
643 when running this command. The requirements for the new name
644 are the same as for adding an instance: the new name must be
645 resolvable and the IP it resolves to must not be reachable
646 (in order to prevent duplicate IPs the next time the
647 instance is started). The IP test can be skipped if the
648 <option>--no-ip-check</option> option is passed.
655 <title>Starting/stopping/connecting to console</title>
658 <title>STARTUP</title>
661 <command>startup</command>
662 <arg>--extra=<replaceable>PARAMS</replaceable></arg>
666 <arg>--instance</arg>
669 <arg>--secondary</arg>
674 rep="repeat"><replaceable>name</replaceable></arg>
678 Starts one or more instances, depending on the following
679 options. The four available modes are:
682 <term><option>--instance</option></term>
684 <simpara>will start the instances given as arguments
685 (at least one argument required); this is the default
692 <simpara>will start the instances who have the given
693 node as either primary or secondary</simpara>
697 <term><option>--primary</option></term>
699 <simpara>will start all instances whose primary node
700 is in the list of nodes passed as arguments (at least
701 one node required)</simpara>
705 <term><option>--secondary</option></term>
707 <simpara>will start all instances whose secondary node
708 is in the list of nodes passed as arguments (at least
709 one node required)</simpara>
715 <simpara>will start all instances in the cluster (no
716 arguments accepted)</simpara>
723 Note that although you can pass more than one selection
724 option, the last one wins, so in order to guarantee the
725 desired result, don't pass more than one such option.
729 The <option>--extra</option> option is used to pass
730 additional argument to the instance's kernel for this start
731 only. Currently there is no way to specify a persistent set
732 of arguments (beside the one hardcoded). Note that this may
733 not apply to all virtualization types.
737 Use <option>--force</option> to start even if secondary disks are
744 # gnt-instance start instance1.example.com
745 # gnt-instance start --extra single test1.example.com
746 # gnt-instance start --node node1.example.com node2.example.com
747 # gnt-instance start --all
753 <title>SHUTDOWN</title>
756 <command>shutdown</command>
759 <arg>--instance</arg>
762 <arg>--secondary</arg>
768 rep="repeat"><replaceable>name</replaceable></arg>
772 Stops one or more instances. If the instance cannot be
773 cleanly stopped during a hardcoded interval (currently 2
774 minutes), it will forcibly stop the instance (equivalent to
775 switching off the power on a physical machine).
779 The <option>--instance</option>, <option>--node</option>,
780 <option>--primary</option>, <option>--secondary</option> and
781 <option>--all</option> options are similar as for the
782 <command>startup</command> command and they influence the
783 actual instances being shutdown.
789 # gnt-instance shutdown instance1.example.com
790 # gnt-instance shutdown --all
796 <title>REBOOT</title>
799 <command>reboot</command>
801 <arg>--extra=<replaceable>PARAMS</replaceable></arg>
803 <arg>--type=<replaceable>REBOOT-TYPE</replaceable></arg>
805 <arg>--ignore-secondaries</arg>
807 <arg>--force-multiple</arg>
810 <arg>--instance</arg>
813 <arg>--secondary</arg>
819 rep="repeat"><replaceable>name</replaceable></arg>
823 Reboots one or more instances. The type of reboot depends on
824 the value of <option>--type</option>. A soft reboot does a
825 hypervisor reboot, a hard reboot does a instance stop,
826 recreates the hypervisor config for the instance and
827 starts the instance. A full reboot does the equivalent
828 of <command>gnt-instance shutdown && gnt-instance
829 startup</command>. The default is soft reboot.
833 For the hard reboot the option
834 <option>--ignore-secondaries</option> ignores errors for the
835 secondary node while re-assembling the instance disks.
839 The <option>--instance</option>, <option>--node</option>,
840 <option>--primary</option>, <option>--secondary</option> and
841 <option>--all</option> options are similar as for the
842 <command>startup</command> command and they influence the
843 actual instances being rebooted.
847 Use the <option>--force-multiple</option> option to keep
848 gnt-instance from asking for confirmation when more than one
849 instance is affected.
855 # gnt-instance reboot instance1.example.com
856 # gnt-instance reboot --type=full instance1.example.com
862 <title>CONSOLE</title>
864 <command>console</command>
865 <arg choice="opt">--show-cmd</arg>
866 <arg choice="req"><replaceable>instance</replaceable></arg>
870 Connects to the console of the given instance. If the instance
871 is not up, an error is returned. Use the <option>--show-cmd</option>
872 option to display the command instead of executing it.
878 # gnt-instance console instance1.example.com
886 <title>Disk management</title>
889 <title>REPLACE-DISKS</title>
892 <command>replace-disks</command>
893 <arg choice="opt">--new-secondary <replaceable>NODE</replaceable></arg>
894 <arg choice="req"><replaceable>instance</replaceable></arg>
898 <command>replace-disks</command>
899 <arg choice="opt">-s</arg>
900 <arg choice="req">--new-secondary <replaceable>NODE</replaceable></arg>
901 <arg choice="req"><replaceable>instance</replaceable></arg>
905 <command>replace-disks</command>
907 <arg choice="req">-s</arg>
908 <arg choice="req">-p</arg>
910 <arg choice="req"><replaceable>instance</replaceable></arg>
914 This command is a generalized form for adding and replacing
919 The first form is usable with the
920 <literal>remote_raid1</literal> disk template. This will
921 replace the disks on both the primary and secondary node,
922 and optionally will change the secondary node to a new one
923 if you pass the <option>--new-secondary</option> option.
927 The second and third forms are usable with the
928 <literal>drbd</literal> disk template. The second form will
929 do a secondary replacement, but as opposed to the
930 <literal>remote_raid1</literal> will not replace the disks
931 on the primary, therefore it will execute faster. The third
932 form will replace the disks on either the primary
933 (<option>-p</option>) or the secondary (<option>-s</option>)
934 node of the instance only, without changing the node.
940 <title>ACTIVATE-DISKS</title>
943 <command>activate-disks</command>
944 <arg choice="req"><replaceable>instance</replaceable></arg>
947 Activates the block devices of the given instance. If
948 successful, the command will show the location and name of
951 node1.example.com:sda:/dev/md0
952 node1.example.com:sdb:/dev/md1
955 In this example, <emphasis>node1.example.com</emphasis> is
956 the name of the node on which the devices have been
957 activated. The <emphasis>sda</emphasis> and
958 <emphasis>sdb</emphasis> are the names of the block devices
959 inside the instance. <emphasis>/dev/md0</emphasis> and
960 <emphasis>/dev/md1</emphasis> are the names of the block
961 devices as visible on the node.
965 Note that it is safe to run this command while the instance
971 <title>DEACTIVATE-DISKS</title>
974 <command>deactivate-disks</command>
975 <arg choice="req"><replaceable>instance</replaceable></arg>
978 De-activates the block devices of the given instance. Note
979 that if you run this command for a remote raid instance
980 type, while it is running, it will not be able to shutdown
981 the block devices on the primary node, but it will shutdown
982 the block devices on the secondary nodes, thus breaking the
991 <title>Recovery</title>
994 <title>FAILOVER</title>
997 <command>failover</command>
999 <arg>--ignore-consistency</arg>
1000 <arg choice="req"><replaceable>instance</replaceable></arg>
1004 Failover will fail the instance over its secondary
1005 node. This works only for instances having a remote raid
1010 Normally the failover will check the consistency of the
1011 disks before failing over the instance. If you are trying to
1012 migrate instances off a dead node, this will fail. Use the
1013 <option>--ignore-consistency</option> option for this
1014 purpose. Note that this option can be dangerous as errors in
1015 shutting down the instance will be ignored, resulting in
1016 possibly having the instance running on two machines in
1017 parallel (on disconnected DRBD drives).
1023 # gnt-instance failover instance1.example.com
1034 <title>ADD-TAGS</title>
1037 <command>add-tags</command>
1038 <arg choice="opt">--from <replaceable>file</replaceable></arg>
1039 <arg choice="req"><replaceable>instancename</replaceable></arg>
1041 rep="repeat"><replaceable>tag</replaceable></arg>
1045 Add tags to the given instance. If any of the tags contains
1046 invalid characters, the entire operation will abort.
1049 If the <option>--from</option> option is given, the list of
1050 tags will be extended with the contents of that file (each
1051 line becomes a tag). In this case, there is not need to pass
1052 tags on the command line (if you do, both sources will be
1053 used). A file name of - will be interpreted as stdin.
1058 <title>LIST-TAGS</title>
1061 <command>list-tags</command>
1062 <arg choice="req"><replaceable>instancename</replaceable></arg>
1065 <para>List the tags of the given instance.</para>
1069 <title>REMOVE-TAGS</title>
1071 <command>remove-tags</command>
1072 <arg choice="opt">--from <replaceable>file</replaceable></arg>
1073 <arg choice="req"><replaceable>instancename</replaceable></arg>
1075 rep="repeat"><replaceable>tag</replaceable></arg>
1079 Remove tags from the given instance. If any of the tags are
1080 not existing on the node, the entire operation will abort.
1084 If the <option>--from</option> option is given, the list of
1085 tags will be extended with the contents of that file (each
1086 line becomes a tag). In this case, there is not need to pass
1087 tags on the command line (if you do, both sources will be
1088 used). A file name of - will be interpreted as stdin.
1100 <!-- Keep this comment at the end of the file
1105 sgml-minimize-attributes:nil
1106 sgml-always-quote-attributes:t
1109 sgml-parent-document:nil
1110 sgml-default-dtd-file:nil
1111 sgml-exposed-tags:nil
1112 sgml-local-catalogs:nil
1113 sgml-local-ecat-files:nil