1 gnt-cluster(8) Ganeti | Version @GANETI_VERSION@
2 ================================================
7 gnt-cluster - Ganeti administration, cluster-wide
12 **gnt-cluster** {command} [arguments...]
17 The **gnt-cluster** is used for cluster-wide administration in the
26 **add-tags** [--from *file*] {*tag*...}
28 Add tags to the cluster. If any of the tags contains invalid
29 characters, the entire operation will abort.
31 If the ``--from`` option is given, the list of tags will be
32 extended with the contents of that file (each line becomes a tag).
33 In this case, there is not need to pass tags on the command line
34 (if you do, both sources will be used). A file name of - will be
40 **command** [-n *node*] {*command*}
42 Executes a command on all nodes. If the option ``-n`` is not given,
43 the command will be executed on all nodes, otherwise it will be
44 executed only on the node(s) specified. Use the option multiple
45 times for running it on multiple nodes, like::
47 # gnt-cluster command -n node1.example.com -n node2.example.com date
49 The command is executed serially on the selected nodes. If the
50 master node is present in the list, the command will be executed
51 last on the master. Regarding the other nodes, the execution order
52 is somewhat alphabetic, so that node2.example.com will be earlier
53 than node10.example.com but after node1.example.com.
55 So given the node names node1, node2, node3, node10, node11, with
56 node3 being the master, the order will be: node1, node2, node10,
59 The command is constructed by concatenating all other command line
60 arguments. For example, to list the contents of the /etc directory
63 # gnt-cluster command ls -l /etc
65 and the command which will be executed will be ``ls -l /etc``.
70 **copyfile** [--use-replication-network] [-n *node*] {*file*}
72 Copies a file to all or to some nodes. The argument specifies the
73 source file (on the current system), the ``-n`` argument specifies
74 the target node, or nodes if the option is given multiple times. If
75 ``-n`` is not given at all, the file will be copied to all nodes.
76 Passing the ``--use-replication-network`` option will cause the
77 copy to be done over the replication network (only matters if the
78 primary/secondary IPs are different). Example::
80 # gnt-cluster -n node1.example.com -n node2.example.com copyfile /tmp/test
82 This will copy the file /tmp/test from the current node to the two
88 **destroy** {--yes-do-it}
90 Remove all configuration files related to the cluster, so that a
91 **gnt-cluster init** can be done again afterwards.
93 Since this is a dangerous command, you are required to pass the
94 argument *--yes-do-it.*
101 Displays the current master node.
108 Shows runtime cluster information: cluster name, architecture (32
109 or 64 bit), master node, node list and instance list.
111 Passing the ``--roman`` option gnt-cluster info will try to print
112 its integer fields in a latin friendly way. This allows further
113 diffusion of Ganeti among ancient cultures.
119 | [-s *secondary\_ip*]
120 | [--vg-name *vg-name*]
121 | [--master-netdev *interface-name*]
126 | [--file-storage-dir *dir*]
127 | [--enabled-hypervisors *hypervisors*]
128 | [-t *hypervisor name*]
129 | [--hypervisor-parameters *hypervisor*:*hv-param*=*value*[,*hv-param*=*value*...]]
130 | [--backend-parameters *be-param*=*value* [,*be-param*=*value*...]]
131 | [--nic-parameters *nic-param*=*value* [,*nic-param*=*value*...]]
132 | [--maintain-node-health {yes \| no}]
133 | [--uid-pool *user-id pool definition*]
134 | [-I *default instance allocator*]
135 | [--primary-ip-version *version*]
136 | [--prealloc-wipe-disks {yes \| no}]
137 | [--node-parameters *ndparams*]
140 This commands is only run once initially on the first node of the
141 cluster. It will initialize the cluster configuration, setup the
142 ssh-keys, start the daemons on the master node, etc. in order to have
143 a working one-node cluster.
145 Note that the *clustername* is not any random name. It has to be
146 resolvable to an IP address using DNS, and it is best if you give the
147 fully-qualified domain name. This hostname must resolve to an IP
148 address reserved exclusively for this purpose, i.e. not already in
151 The cluster can run in two modes: single-home or dual-homed. In the
152 first case, all traffic (both public traffic, inter-node traffic
153 and data replication traffic) goes over the same interface. In the
154 dual-homed case, the data replication traffic goes over the second
155 network. The ``-s`` option here marks the cluster as dual-homed and
156 its parameter represents this node's address on the second network.
157 If you initialise the cluster with ``-s``, all nodes added must
158 have a secondary IP as well.
160 Note that for Ganeti it doesn't matter if the secondary network is
161 actually a separate physical network, or is done using tunneling,
162 etc. For performance reasons, it's recommended to use a separate
165 The ``--vg-name`` option will let you specify a volume group
166 different than "xenvg" for Ganeti to use when creating instance
167 disks. This volume group must have the same name on all nodes. Once
168 the cluster is initialized this can be altered by using the
169 **modify** command. If you don't want to use lvm storage at all use
170 the ``--no-lvm-storage`` option. Once the cluster is initialized
171 you can change this setup with the **modify** command.
173 The ``--master-netdev`` option is useful for specifying a different
174 interface on which the master will activate its IP address. It's
175 important that all nodes have this interface because you'll need it
176 for a master failover.
178 The ``-m`` option will let you specify a three byte prefix under
179 which the virtual MAC addresses of your instances will be
180 generated. The prefix must be specified in the format XX:XX:XX and
181 the default is aa:00:00.
183 The ``--no-lvm-storage`` option allows you to initialize the
184 cluster without lvm support. This means that only instances using
185 files as storage backend will be possible to create. Once the
186 cluster is initialized you can change this setup with the
189 The ``--no-etc-hosts`` option allows you to initialize the cluster
190 without modifying the /etc/hosts file.
192 The ``--no-ssh-init`` option allows you to initialize the cluster
193 without creating or distributing SSH key pairs.
195 The ``--file-storage-dir`` option allows you set the directory to
196 use for storing the instance disk files when using file storage as
197 backend for instance disks.
199 The ``--enabled-hypervisors`` option allows you to set the list of
200 hypervisors that will be enabled for this cluster. Instance
201 hypervisors can only be chosen from the list of enabled
202 hypervisors, and the first entry of this list will be used by
203 default. Currently, the following hypervisors are available:
205 The ``--prealloc-wipe-disks`` sets a cluster wide configuration
206 value for wiping disks prior to allocation. This increases security
207 on instance level as the instance can't access untouched data from
208 it's underlying storage.
224 a simple chroot manager that starts chroot based on a script at the
225 root of the filesystem holding the chroot
228 fake hypervisor for development/testing
231 Either a single hypervisor name or a comma-separated list of
232 hypervisor names can be specified. If this option is not specified,
233 only the xen-pvm hypervisor is enabled by default.
235 The ``--hypervisor-parameters`` option allows you to set default
236 hypervisor specific parameters for the cluster. The format of this
237 option is the name of the hypervisor, followed by a colon and a
238 comma-separated list of key=value pairs. The keys available for
239 each hypervisors are detailed in the gnt-instance(8) man page, in
240 the **add** command plus the following parameters which are only
241 configurable globally (at cluster level):
244 Valid for the Xen PVM and KVM hypervisors.
246 This options specifies the TCP port to use for live-migration. For
247 Xen, the same port should be configured on all nodes in the
248 ``/etc/xen/xend-config.sxp`` file, under the key
249 "xend-relocation-port".
252 Valid for the KVM hypervisor.
254 This option specifies the maximum bandwidth that KVM will use for
255 instance live migrations. The value is in MiB/s.
257 This option is only effective with kvm versions >= 78 and qemu-kvm
261 The ``--backend-parameters`` option allows you to set the default
262 backend parameters for the cluster. The parameter format is a
263 comma-separated list of key=value pairs with the following
267 Number of VCPUs to set for an instance by default, must be an
268 integer, will be set to 1 if no specified.
271 Amount of memory to allocate for an instance by default, can be
272 either an integer or an integer followed by a unit (M for mebibytes
273 and G for gibibytes are supported), will be set to 128M if not
277 Value of the auto\_balance flag for instances to use by default,
278 will be set to true if not specified.
281 The ``--nic-parameters`` option allows you to set the default nic
282 parameters for the cluster. The parameter format is a
283 comma-separated list of key=value pairs with the following
287 The default nic mode, 'routed' or 'bridged'.
290 In bridged mode the default NIC bridge. In routed mode it
291 represents an hypervisor-vif-script dependent value to allow
292 different instance groups. For example under the KVM default
293 network script it is interpreted as a routing table number or
297 The option ``--maintain-node-health`` allows to enable/disable
298 automatic maintenance actions on nodes. Currently these include
299 automatic shutdown of instances and deactivation of DRBD devices on
300 offline nodes; in the future it might be extended to automatic
301 removal of unknown LVM volumes, etc.
303 The ``--uid-pool`` option initializes the user-id pool. The
304 *user-id pool definition* can contain a list of user-ids and/or a
305 list of user-id ranges. The parameter format is a comma-separated
306 list of numeric user-ids or user-id ranges. The ranges are defined
307 by a lower and higher boundary, separated by a dash. The boundaries
308 are inclusive. If the ``--uid-pool`` option is not supplied, the
309 user-id pool is initialized to an empty list. An empty list means
310 that the user-id pool feature is disabled.
312 The ``-I (--default-iallocator)`` option specifies the default
313 instance allocator. The instance allocator will be used for
314 operations like instance creation, instance and node migration,
315 etc. when no manual override is specified. If this option is not
316 specified, the default instance allocator will be blank, which
317 means that relevant operations will require the administrator to
318 manually specify either an instance allocator, or a set of nodes.
319 The default iallocator can be changed later using the **modify**
322 The ``--primary-ip-version`` option specifies the IP version used
323 for the primary address. Possible values are 4 and 6 for IPv4 and
324 IPv6, respectively. This option is used when resolving node names
325 and the cluster name.
327 The ``--node-parameters`` option allows you to set default node
328 parameters for the cluster. Please see **ganeti**(7) for more
329 information about supported key=value pairs.
336 List the tags of the cluster.
341 **master-failover** [--no-voting]
343 Failover the master role to the current node.
345 The ``--no-voting`` option skips the remote node agreement checks.
346 This is dangerous, but necessary in some cases (for example failing
347 over the master role in a 2 node cluster with the original master
348 down). If the original master then comes up, it won't be able to
349 start its master daemon because it won't have enough votes, but so
350 won't the new master, if the master daemon ever needs a restart.
351 You can pass ``--no-voting`` to **ganeti-masterd** on the new
352 master to solve this problem, and run **gnt-cluster redist-conf**
353 to make sure the cluster is consistent again.
360 Checks if the master daemon is alive.
362 If the master daemon is alive and can respond to a basic query (the
363 equivalent of **gnt-cluster info**), then the exit code of the
364 command will be 0. If the master daemon is not alive (either due to
365 a crash or because this is not the master node), the exit code will
372 | [--vg-name *vg-name*]
374 | [--enabled-hypervisors *hypervisors*]
375 | [--hypervisor-parameters *hypervisor*:*hv-param*=*value*[,*hv-param*=*value*...]]
376 | [--backend-parameters *be-param*=*value* [,*be-param*=*value*...]]
377 | [--nic-parameters *nic-param*=*value* [,*nic-param*=*value*...]]
378 | [--uid-pool *user-id pool definition*]
379 | [--add-uids *user-id pool definition*]
380 | [--remove-uids *user-id pool definition*]
381 | [-C *candidate\_pool\_size*]
382 | [--maintain-node-health {yes \| no}]
383 | [--prealloc-wipe-disks {yes \| no}]
384 | [-I *default instance allocator*]
385 | [--reserved-lvs=*NAMES*]
386 | [--node-parameters *ndparams*]
387 | [--master-netdev *interface-name*]
389 Modify the options for the cluster.
391 The ``--vg-name``, ``--no-lvm-storarge``, ``--enabled-hypervisors``,
392 ``--hypervisor-parameters``, ``--backend-parameters``,
393 ``--nic-parameters``, ``--maintain-node-health``,
394 ``--prealloc-wipe-disks``, ``--uid-pool``, ``--node-parameters``,
395 ``--master-netdev`` options are described in the **init** command.
397 The ``-C`` option specifies the ``candidate_pool_size`` cluster
398 parameter. This is the number of nodes that the master will try to
399 keep as master\_candidates. For more details about this role and
400 other node roles, see the ganeti(7). If you increase the size, the
401 master will automatically promote as many nodes as required and
402 possible to reach the intended number.
404 The ``--add-uids`` and ``--remove-uids`` options can be used to
405 modify the user-id pool by adding/removing a list of user-ids or
408 The option ``--reserved-lvs`` specifies a list (comma-separated) of
409 logical volume group names (regular expressions) that will be
410 ignored by the cluster verify operation. This is useful if the
411 volume group used for Ganeti is shared with the system for other
412 uses. Note that it's not recommended to create and mark as ignored
413 logical volume names which match Ganeti's own name format (starting
414 with UUID and then .diskN), as this option only skips the
415 verification, but not the actual use of the names given.
417 To remove all reserved logical volumes, pass in an empty argument
418 to the option, as in ``--reserved-lvs=`` or ``--reserved-lvs ''``.
420 The ``-I`` is described in the **init** command. To clear the
421 default iallocator, just pass an empty string ('').
426 **queue** {drain | undrain | info}
428 Change job queue properties.
430 The ``drain`` option sets the drain flag on the job queue. No new
431 jobs will be accepted, but jobs already in the queue will be
434 The ``undrain`` will unset the drain flag on the job queue. New
435 jobs will be accepted.
437 The ``info`` option shows the properties of the job queue.
442 **watcher** {pause *duration* | continue | info}
444 Make the watcher pause or let it continue.
446 The ``pause`` option causes the watcher to pause for *duration*
449 The ``continue`` option will let the watcher continue.
451 The ``info`` option shows whether the watcher is currently paused.
456 **redist-conf** [--submit]
458 This command forces a full push of configuration files from the
459 master node to the other nodes in the cluster. This is normally not
460 needed, but can be run if the **verify** complains about
461 configuration mismatches.
463 The ``--submit`` option is used to send the job to the master
464 daemon but not wait for its completion. The job ID will be shown so
465 that it can be examined via **gnt-job info**.
470 **remove-tags** [--from *file*] {*tag*...}
472 Remove tags from the cluster. If any of the tags are not existing
473 on the cluster, the entire operation will abort.
475 If the ``--from`` option is given, the list of tags to be removed will
476 be extended with the contents of that file (each line becomes a tag).
477 In this case, there is not need to pass tags on the command line (if
478 you do, tags from both sources will be removed). A file name of - will
479 be interpreted as stdin.
484 **rename** [-f] {*name*}
486 Renames the cluster and in the process updates the master IP
487 address to the one the new name resolves to. At least one of either
488 the name or the IP address must be different, otherwise the
489 operation will be aborted.
491 Note that since this command can be dangerous (especially when run
492 over SSH), the command will require confirmation unless run with
498 | **renew-crypto** [-f]
499 | [--new-cluster-certificate] [--new-confd-hmac-key]
500 | [--new-rapi-certificate] [--rapi-certificate *rapi-cert*]
501 | [--new-cluster-domain-secret] [--cluster-domain-secret *filename*]
503 This command will stop all Ganeti daemons in the cluster and start
504 them again once the new certificates and keys are replicated. The
505 options ``--new-cluster-certificate`` and ``--new-confd-hmac-key``
506 can be used to regenerate the cluster-internal SSL certificate
507 respective the HMAC key used by ganeti-confd(8).
509 To generate a new self-signed RAPI certificate (used by
510 ganeti-rapi(8)) specify ``--new-rapi-certificate``. If you want to
511 use your own certificate, e.g. one signed by a certificate
512 authority (CA), pass its filename to ``--rapi-certificate``.
514 ``--new-cluster-domain-secret`` generates a new, random cluster
515 domain secret. ``--cluster-domain-secret`` reads the secret from a
516 file. The cluster domain secret is used to sign information
517 exchanged between separate clusters via a third party.
522 **repair-disk-sizes** [instance...]
524 This command checks that the recorded size of the given instance's
525 disks matches the actual size and updates any mismatches found.
526 This is needed if the Ganeti configuration is no longer consistent
527 with reality, as it will impact some disk operations. If no
528 arguments are given, all instances will be checked.
530 Note that only active disks can be checked by this command; in case
531 a disk cannot be activated it's advised to use
532 **gnt-instance activate-disks --ignore-size ...** to force
533 activation without regard to the current size.
535 When the all disk sizes are consistent, the command will return no
536 output. Otherwise it will log details about the inconsistencies in
542 **search-tags** {*pattern*}
544 Searches the tags on all objects in the cluster (the cluster
545 itself, the nodes and the instances) for a given pattern. The
546 pattern is interpreted as a regular expression and a search will be
547 done on it (i.e. the given pattern is not anchored to the beggining
548 of the string; if you want that, prefix the pattern with ^).
550 If no tags are matching the pattern, the exit code of the command
551 will be one. If there is at least one match, the exit code will be
552 zero. Each match is listed on one line, the object and the tag
553 separated by a space. The cluster will be listed as /cluster, a
554 node will be listed as /nodes/*name*, and an instance as
555 /instances/*name*. Example:
559 # gnt-cluster search-tags time
560 /cluster ctime:2007-09-01
561 /nodes/node1.example.com mtime:2007-10-04
566 **verify** [--no-nplus1-mem]
568 Verify correctness of cluster configuration. This is safe with
569 respect to running instances, and incurs no downtime of the
572 If the ``--no-nplus1-mem`` option is given, Ganeti won't check
573 whether if it loses a node it can restart all the instances on
574 their secondaries (and report an error otherwise).
581 The command checks which instances have degraded DRBD disks and
582 activates the disks of those instances.
584 This command is run from the **ganeti-watcher** tool, which also
585 has a different, complementary algorithm for doing this check.
586 Together, these two should ensure that DRBD disks are kept
594 Show the cluster version.