1 gnt-cluster(8) Ganeti | Version @GANETI_VERSION@
2 ================================================
7 gnt-cluster - Ganeti administration, cluster-wide
12 **gnt-cluster** {command} [arguments...]
17 The **gnt-cluster** is used for cluster-wide administration in the
26 **add-tags** [--from *file*] {*tag*...}
28 Add tags to the cluster. If any of the tags contains invalid
29 characters, the entire operation will abort.
31 If the ``--from`` option is given, the list of tags will be
32 extended with the contents of that file (each line becomes a tag).
33 In this case, there is not need to pass tags on the command line
34 (if you do, both sources will be used). A file name of - will be
40 **command** [-n *node*] {*command*}
42 Executes a command on all nodes. If the option ``-n`` is not given,
43 the command will be executed on all nodes, otherwise it will be
44 executed only on the node(s) specified. Use the option multiple
45 times for running it on multiple nodes, like::
47 # gnt-cluster command -n node1.example.com -n node2.example.com date
49 The command is executed serially on the selected nodes. If the
50 master node is present in the list, the command will be executed
51 last on the master. Regarding the other nodes, the execution order
52 is somewhat alphabetic, so that node2.example.com will be earlier
53 than node10.example.com but after node1.example.com.
55 So given the node names node1, node2, node3, node10, node11, with
56 node3 being the master, the order will be: node1, node2, node10,
59 The command is constructed by concatenating all other command line
60 arguments. For example, to list the contents of the /etc directory
63 # gnt-cluster command ls -l /etc
65 and the command which will be executed will be ``ls -l /etc``.
70 **copyfile** [--use-replication-network] [-n *node*] {*file*}
72 Copies a file to all or to some nodes. The argument specifies the
73 source file (on the current system), the ``-n`` argument specifies
74 the target node, or nodes if the option is given multiple times. If
75 ``-n`` is not given at all, the file will be copied to all nodes.
76 Passing the ``--use-replication-network`` option will cause the
77 copy to be done over the replication network (only matters if the
78 primary/secondary IPs are different). Example::
80 # gnt-cluster -n node1.example.com -n node2.example.com copyfile /tmp/test
82 This will copy the file /tmp/test from the current node to the two
88 **destroy** {--yes-do-it}
90 Remove all configuration files related to the cluster, so that a
91 **gnt-cluster init** can be done again afterwards.
93 Since this is a dangerous command, you are required to pass the
94 argument *--yes-do-it.*
99 **epo** [--on] [--groups|--all] [--power-delay] *arguments*
101 Performs an emergency power-off on nodes given as arguments. If
102 ``--groups`` is given, arguments are node groups. If ``--all`` is
103 provided, the whole cluster will be shut down.
105 The ``--on`` flag recovers the cluster after an emergency power-off.
106 When powering on the cluster you can use ``--power-delay`` to define the
107 time in seconds (fractions allowed) waited between powering on
110 Please note that the master node will not be turned down or up
111 automatically. It will just be left in a state, where you can manully
112 perform the shutdown of that one node. If the master is in the list of
113 affected nodes and this is not a complete cluster emergency power-off
114 (e.g. using ``--all``), you're required to do a master failover to
115 another node not affected.
122 Displays the current master node.
129 Shows runtime cluster information: cluster name, architecture (32
130 or 64 bit), master node, node list and instance list.
132 Passing the ``--roman`` option gnt-cluster info will try to print
133 its integer fields in a latin friendly way. This allows further
134 diffusion of Ganeti among ancient cultures.
140 | [{-s|--secondary-ip} *secondary\_ip*]
141 | [--vg-name *vg-name*]
142 | [--master-netdev *interface-name*]
143 | [{-m|--mac-prefix} *mac-prefix*]
147 | [--file-storage-dir *dir*]
148 | [--enabled-hypervisors *hypervisors*]
149 | [-t *hypervisor name*]
150 | [{-H|--hypervisor-parameters} *hypervisor*:*hv-param*=*value*[,*hv-param*=*value*...]]
151 | [{-B|--backend-parameters} *be-param*=*value* [,*be-param*=*value*...]]
152 | [{-N|--nic-parameters} *nic-param*=*value* [,*nic-param*=*value*...]]
153 | [--maintain-node-health {yes \| no}]
154 | [--uid-pool *user-id pool definition*]
155 | [{-I|--default-iallocator} *default instance allocator*]
156 | [--primary-ip-version *version*]
157 | [--prealloc-wipe-disks {yes \| no}]
158 | [--node-parameters *ndparams*]
159 | [{-C|--candidate-pool-size} *candidate\_pool\_size*]
162 This commands is only run once initially on the first node of the
163 cluster. It will initialize the cluster configuration, setup the
164 ssh-keys, start the daemons on the master node, etc. in order to have
165 a working one-node cluster.
167 Note that the *clustername* is not any random name. It has to be
168 resolvable to an IP address using DNS, and it is best if you give the
169 fully-qualified domain name. This hostname must resolve to an IP
170 address reserved exclusively for this purpose, i.e. not already in
173 The cluster can run in two modes: single-home or dual-homed. In the
174 first case, all traffic (both public traffic, inter-node traffic and
175 data replication traffic) goes over the same interface. In the
176 dual-homed case, the data replication traffic goes over the second
177 network. The ``-s (--secondary-ip)`` option here marks the cluster as
178 dual-homed and its parameter represents this node's address on the
179 second network. If you initialise the cluster with ``-s``, all nodes
180 added must have a secondary IP as well.
182 Note that for Ganeti it doesn't matter if the secondary network is
183 actually a separate physical network, or is done using tunneling,
184 etc. For performance reasons, it's recommended to use a separate
187 The ``--vg-name`` option will let you specify a volume group
188 different than "xenvg" for Ganeti to use when creating instance
189 disks. This volume group must have the same name on all nodes. Once
190 the cluster is initialized this can be altered by using the
191 **modify** command. If you don't want to use lvm storage at all use
192 the ``--no-lvm-storage`` option. Once the cluster is initialized
193 you can change this setup with the **modify** command.
195 The ``--master-netdev`` option is useful for specifying a different
196 interface on which the master will activate its IP address. It's
197 important that all nodes have this interface because you'll need it
198 for a master failover.
200 The ``-m (--mac-prefix)`` option will let you specify a three byte
201 prefix under which the virtual MAC addresses of your instances will be
202 generated. The prefix must be specified in the format ``XX:XX:XX`` and
203 the default is ``aa:00:00``.
205 The ``--no-lvm-storage`` option allows you to initialize the
206 cluster without lvm support. This means that only instances using
207 files as storage backend will be possible to create. Once the
208 cluster is initialized you can change this setup with the
211 The ``--no-etc-hosts`` option allows you to initialize the cluster
212 without modifying the /etc/hosts file.
214 The ``--no-ssh-init`` option allows you to initialize the cluster
215 without creating or distributing SSH key pairs.
217 The ``--file-storage-dir`` option allows you set the directory to
218 use for storing the instance disk files when using file storage as
219 backend for instance disks.
221 The ``--prealloc-wipe-disks`` sets a cluster wide configuration
222 value for wiping disks prior to allocation. This increases security
223 on instance level as the instance can't access untouched data from
224 it's underlying storage.
226 The ``--enabled-hypervisors`` option allows you to set the list of
227 hypervisors that will be enabled for this cluster. Instance
228 hypervisors can only be chosen from the list of enabled
229 hypervisors, and the first entry of this list will be used by
230 default. Currently, the following hypervisors are available:
242 a simple chroot manager that starts chroot based on a script at the
243 root of the filesystem holding the chroot
246 fake hypervisor for development/testing
248 Either a single hypervisor name or a comma-separated list of
249 hypervisor names can be specified. If this option is not specified,
250 only the xen-pvm hypervisor is enabled by default.
252 The ``-H (--hypervisor-parameters)`` option allows you to set default
253 hypervisor specific parameters for the cluster. The format of this
254 option is the name of the hypervisor, followed by a colon and a
255 comma-separated list of key=value pairs. The keys available for each
256 hypervisors are detailed in the gnt-instance(8) man page, in the
257 **add** command plus the following parameters which are only
258 configurable globally (at cluster level):
261 Valid for the Xen PVM and KVM hypervisors.
263 This options specifies the TCP port to use for live-migration. For
264 Xen, the same port should be configured on all nodes in the
265 ``/etc/xen/xend-config.sxp`` file, under the key
266 "xend-relocation-port".
269 Valid for the KVM hypervisor.
271 This option specifies the maximum bandwidth that KVM will use for
272 instance live migrations. The value is in MiB/s.
274 This option is only effective with kvm versions >= 78 and qemu-kvm
277 The ``-B (--backend-parameters)`` option allows you to set the default
278 backend parameters for the cluster. The parameter format is a
279 comma-separated list of key=value pairs with the following supported
283 Number of VCPUs to set for an instance by default, must be an
284 integer, will be set to 1 if no specified.
287 Amount of memory to allocate for an instance by default, can be
288 either an integer or an integer followed by a unit (M for mebibytes
289 and G for gibibytes are supported), will be set to 128M if not
293 Value of the auto\_balance flag for instances to use by default,
294 will be set to true if not specified.
297 The ``-N (--nic-parameters)`` option allows you to set the default nic
298 parameters for the cluster. The parameter format is a comma-separated
299 list of key=value pairs with the following supported keys:
302 The default nic mode, 'routed' or 'bridged'.
305 In bridged mode the default NIC bridge. In routed mode it
306 represents an hypervisor-vif-script dependent value to allow
307 different instance groups. For example under the KVM default
308 network script it is interpreted as a routing table number or
311 The option ``--maintain-node-health`` allows one to enable/disable
312 automatic maintenance actions on nodes. Currently these include
313 automatic shutdown of instances and deactivation of DRBD devices on
314 offline nodes; in the future it might be extended to automatic
315 removal of unknown LVM volumes, etc.
317 The ``--uid-pool`` option initializes the user-id pool. The
318 *user-id pool definition* can contain a list of user-ids and/or a
319 list of user-id ranges. The parameter format is a comma-separated
320 list of numeric user-ids or user-id ranges. The ranges are defined
321 by a lower and higher boundary, separated by a dash. The boundaries
322 are inclusive. If the ``--uid-pool`` option is not supplied, the
323 user-id pool is initialized to an empty list. An empty list means
324 that the user-id pool feature is disabled.
326 The ``-I (--default-iallocator)`` option specifies the default
327 instance allocator. The instance allocator will be used for operations
328 like instance creation, instance and node migration, etc. when no
329 manual override is specified. If this option is not specified and
330 htools was not enabled at build time, the default instance allocator
331 will be blank, which means that relevant operations will require the
332 administrator to manually specify either an instance allocator, or a
333 set of nodes. If the option is not specified but htools was enabled,
334 the default iallocator will be **hail**(1) (assuming it can be found
335 on disk). The default iallocator can be changed later using the
338 The ``--primary-ip-version`` option specifies the IP version used
339 for the primary address. Possible values are 4 and 6 for IPv4 and
340 IPv6, respectively. This option is used when resolving node names
341 and the cluster name.
343 The ``--node-parameters`` option allows you to set default node
344 parameters for the cluster. Please see **ganeti**(7) for more
345 information about supported key=value pairs.
347 The ``-C (--candidate-pool-size)`` option specifies the
348 ``candidate_pool_size`` cluster parameter. This is the number of nodes
349 that the master will try to keep as master\_candidates. For more
350 details about this role and other node roles, see the ganeti(7).
357 List the tags of the cluster.
362 **master-failover** [--no-voting]
364 Failover the master role to the current node.
366 The ``--no-voting`` option skips the remote node agreement checks.
367 This is dangerous, but necessary in some cases (for example failing
368 over the master role in a 2 node cluster with the original master
369 down). If the original master then comes up, it won't be able to
370 start its master daemon because it won't have enough votes, but so
371 won't the new master, if the master daemon ever needs a restart.
372 You can pass ``--no-voting`` to **ganeti-masterd** on the new
373 master to solve this problem, and run **gnt-cluster redist-conf**
374 to make sure the cluster is consistent again.
381 Checks if the master daemon is alive.
383 If the master daemon is alive and can respond to a basic query (the
384 equivalent of **gnt-cluster info**), then the exit code of the
385 command will be 0. If the master daemon is not alive (either due to
386 a crash or because this is not the master node), the exit code will
393 | [--vg-name *vg-name*]
395 | [--enabled-hypervisors *hypervisors*]
396 | [{-H|--hypervisor-parameters} *hypervisor*:*hv-param*=*value*[,*hv-param*=*value*...]]
397 | [{-B|--backend-parameters} *be-param*=*value* [,*be-param*=*value*...]]
398 | [{-N|--nic-parameters} *nic-param*=*value* [,*nic-param*=*value*...]]
399 | [--uid-pool *user-id pool definition*]
400 | [--add-uids *user-id pool definition*]
401 | [--remove-uids *user-id pool definition*]
402 | [{-C|--candidate-pool-size} *candidate\_pool\_size*]
403 | [--maintain-node-health {yes \| no}]
404 | [--prealloc-wipe-disks {yes \| no}]
405 | [{-I|--default-iallocator} *default instance allocator*]
406 | [--reserved-lvs=*NAMES*]
407 | [--node-parameters *ndparams*]
408 | [--master-netdev *interface-name*]
410 Modify the options for the cluster.
412 The ``--vg-name``, ``--no-lvm-storarge``, ``--enabled-hypervisors``,
413 ``-H (--hypervisor-parameters)``, ``-B (--backend-parameters)``,
414 ``--nic-parameters``, ``-C (--candidate-pool-size)``,
415 ``--maintain-node-health``, ``--prealloc-wipe-disks``, ``--uid-pool``,
416 ``--node-parameters``, ``--master-netdev`` options are described in
417 the **init** command.
419 The ``--add-uids`` and ``--remove-uids`` options can be used to
420 modify the user-id pool by adding/removing a list of user-ids or
423 The option ``--reserved-lvs`` specifies a list (comma-separated) of
424 logical volume group names (regular expressions) that will be
425 ignored by the cluster verify operation. This is useful if the
426 volume group used for Ganeti is shared with the system for other
427 uses. Note that it's not recommended to create and mark as ignored
428 logical volume names which match Ganeti's own name format (starting
429 with UUID and then .diskN), as this option only skips the
430 verification, but not the actual use of the names given.
432 To remove all reserved logical volumes, pass in an empty argument
433 to the option, as in ``--reserved-lvs=`` or ``--reserved-lvs ''``.
435 The ``-I (--default-iallocator)`` is described in the **init**
436 command. To clear the default iallocator, just pass an empty string
442 **queue** {drain | undrain | info}
444 Change job queue properties.
446 The ``drain`` option sets the drain flag on the job queue. No new
447 jobs will be accepted, but jobs already in the queue will be
450 The ``undrain`` will unset the drain flag on the job queue. New
451 jobs will be accepted.
453 The ``info`` option shows the properties of the job queue.
458 **watcher** {pause *duration* | continue | info}
460 Make the watcher pause or let it continue.
462 The ``pause`` option causes the watcher to pause for *duration*
465 The ``continue`` option will let the watcher continue.
467 The ``info`` option shows whether the watcher is currently paused.
472 **redist-conf** [--submit]
474 This command forces a full push of configuration files from the
475 master node to the other nodes in the cluster. This is normally not
476 needed, but can be run if the **verify** complains about
477 configuration mismatches.
479 The ``--submit`` option is used to send the job to the master
480 daemon but not wait for its completion. The job ID will be shown so
481 that it can be examined via **gnt-job info**.
486 **remove-tags** [--from *file*] {*tag*...}
488 Remove tags from the cluster. If any of the tags are not existing
489 on the cluster, the entire operation will abort.
491 If the ``--from`` option is given, the list of tags to be removed will
492 be extended with the contents of that file (each line becomes a tag).
493 In this case, there is not need to pass tags on the command line (if
494 you do, tags from both sources will be removed). A file name of - will
495 be interpreted as stdin.
500 **rename** [-f] {*name*}
502 Renames the cluster and in the process updates the master IP
503 address to the one the new name resolves to. At least one of either
504 the name or the IP address must be different, otherwise the
505 operation will be aborted.
507 Note that since this command can be dangerous (especially when run
508 over SSH), the command will require confirmation unless run with
514 | **renew-crypto** [-f]
515 | [--new-cluster-certificate] [--new-confd-hmac-key]
516 | [--new-rapi-certificate] [--rapi-certificate *rapi-cert*]
517 | [--new-cluster-domain-secret] [--cluster-domain-secret *filename*]
519 This command will stop all Ganeti daemons in the cluster and start
520 them again once the new certificates and keys are replicated. The
521 options ``--new-cluster-certificate`` and ``--new-confd-hmac-key``
522 can be used to regenerate the cluster-internal SSL certificate
523 respective the HMAC key used by ganeti-confd(8).
525 To generate a new self-signed RAPI certificate (used by
526 ganeti-rapi(8)) specify ``--new-rapi-certificate``. If you want to
527 use your own certificate, e.g. one signed by a certificate
528 authority (CA), pass its filename to ``--rapi-certificate``.
530 ``--new-cluster-domain-secret`` generates a new, random cluster
531 domain secret. ``--cluster-domain-secret`` reads the secret from a
532 file. The cluster domain secret is used to sign information
533 exchanged between separate clusters via a third party.
538 **repair-disk-sizes** [instance...]
540 This command checks that the recorded size of the given instance's
541 disks matches the actual size and updates any mismatches found.
542 This is needed if the Ganeti configuration is no longer consistent
543 with reality, as it will impact some disk operations. If no
544 arguments are given, all instances will be checked.
546 Note that only active disks can be checked by this command; in case
547 a disk cannot be activated it's advised to use
548 **gnt-instance activate-disks --ignore-size ...** to force
549 activation without regard to the current size.
551 When the all disk sizes are consistent, the command will return no
552 output. Otherwise it will log details about the inconsistencies in
558 **search-tags** {*pattern*}
560 Searches the tags on all objects in the cluster (the cluster
561 itself, the nodes and the instances) for a given pattern. The
562 pattern is interpreted as a regular expression and a search will be
563 done on it (i.e. the given pattern is not anchored to the beggining
564 of the string; if you want that, prefix the pattern with ^).
566 If no tags are matching the pattern, the exit code of the command
567 will be one. If there is at least one match, the exit code will be
568 zero. Each match is listed on one line, the object and the tag
569 separated by a space. The cluster will be listed as /cluster, a
570 node will be listed as /nodes/*name*, and an instance as
571 /instances/*name*. Example:
575 # gnt-cluster search-tags time
576 /cluster ctime:2007-09-01
577 /nodes/node1.example.com mtime:2007-10-04
582 **verify** [--no-nplus1-mem] [--node-group *nodegroup*]
584 Verify correctness of cluster configuration. This is safe with
585 respect to running instances, and incurs no downtime of the
588 If the ``--no-nplus1-mem`` option is given, Ganeti won't check
589 whether if it loses a node it can restart all the instances on
590 their secondaries (and report an error otherwise).
592 With ``--node-group``, restrict the verification to those nodes and
593 instances that live in the named group. This will not verify global
594 settings, but will allow to perform verification of a group while other
595 operations are ongoing in other groups.
602 The command checks which instances have degraded DRBD disks and
603 activates the disks of those instances.
605 This command is run from the **ganeti-watcher** tool, which also
606 has a different, complementary algorithm for doing this check.
607 Together, these two should ensure that DRBD disks are kept
615 Show the cluster version.
617 .. vim: set textwidth=72 :