1 gnt-cluster(8) Ganeti | Version @GANETI_VERSION@
2 ================================================
7 gnt-cluster - Ganeti administration, cluster-wide
12 **gnt-cluster** {command} [arguments...]
17 The **gnt-cluster** is used for cluster-wide administration in the
26 **add-tags** [--from *file*] {*tag*...}
28 Add tags to the cluster. If any of the tags contains invalid
29 characters, the entire operation will abort.
31 If the ``--from`` option is given, the list of tags will be
32 extended with the contents of that file (each line becomes a tag).
33 In this case, there is not need to pass tags on the command line
34 (if you do, both sources will be used). A file name of - will be
40 **command** [-n *node*] {*command*}
42 Executes a command on all nodes. If the option ``-n`` is not given,
43 the command will be executed on all nodes, otherwise it will be
44 executed only on the node(s) specified. Use the option multiple
45 times for running it on multiple nodes, like::
47 # gnt-cluster command -n node1.example.com -n node2.example.com date
49 The command is executed serially on the selected nodes. If the
50 master node is present in the list, the command will be executed
51 last on the master. Regarding the other nodes, the execution order
52 is somewhat alphabetic, so that node2.example.com will be earlier
53 than node10.example.com but after node1.example.com.
55 So given the node names node1, node2, node3, node10, node11, with
56 node3 being the master, the order will be: node1, node2, node10,
59 The command is constructed by concatenating all other command line
60 arguments. For example, to list the contents of the /etc directory
63 # gnt-cluster command ls -l /etc
65 and the command which will be executed will be ``ls -l /etc``.
70 **copyfile** [--use-replication-network] [-n *node*] {*file*}
72 Copies a file to all or to some nodes. The argument specifies the
73 source file (on the current system), the ``-n`` argument specifies
74 the target node, or nodes if the option is given multiple times. If
75 ``-n`` is not given at all, the file will be copied to all nodes.
76 Passing the ``--use-replication-network`` option will cause the
77 copy to be done over the replication network (only matters if the
78 primary/secondary IPs are different). Example::
80 # gnt-cluster -n node1.example.com -n node2.example.com copyfile /tmp/test
82 This will copy the file /tmp/test from the current node to the two
88 **destroy** {--yes-do-it}
90 Remove all configuration files related to the cluster, so that a
91 **gnt-cluster init** can be done again afterwards.
93 Since this is a dangerous command, you are required to pass the
94 argument *--yes-do-it.*
101 Displays the current master node.
108 Shows runtime cluster information: cluster name, architecture (32
109 or 64 bit), master node, node list and instance list.
111 Passing the ``--roman`` option gnt-cluster info will try to print
112 its integer fields in a latin friendly way. This allows further
113 diffusion of Ganeti among ancient cultures.
119 | [-s *secondary\_ip*]
120 | [--vg-name *vg-name*]
121 | [--master-netdev *interface-name*]
126 | [--file-storage-dir *dir*]
127 | [--enabled-hypervisors *hypervisors*]
128 | [-t *hypervisor name*]
129 | [--hypervisor-parameters *hypervisor*:*hv-param*=*value*[,*hv-param*=*value*...]]
130 | [--backend-parameters *be-param*=*value* [,*be-param*=*value*...]]
131 | [--nic-parameters *nic-param*=*value* [,*nic-param*=*value*...]]
132 | [--maintain-node-health {yes \| no}]
133 | [--uid-pool *user-id pool definition*]
134 | [-I *default instance allocator*]
135 | [--primary-ip-version *version*]
136 | [--prealloc-wipe-disks {yes \| no}]
139 This commands is only run once initially on the first node of the
140 cluster. It will initialize the cluster configuration, setup the
141 ssh-keys, start the daemons on the master node, etc. in order to have
142 a working one-node cluster.
144 Note that the *clustername* is not any random name. It has to be
145 resolvable to an IP address using DNS, and it is best if you give the
146 fully-qualified domain name. This hostname must resolve to an IP
147 address reserved exclusively for this purpose, i.e. not already in
150 The cluster can run in two modes: single-home or dual-homed. In the
151 first case, all traffic (both public traffic, inter-node traffic
152 and data replication traffic) goes over the same interface. In the
153 dual-homed case, the data replication traffic goes over the second
154 network. The ``-s`` option here marks the cluster as dual-homed and
155 its parameter represents this node's address on the second network.
156 If you initialise the cluster with ``-s``, all nodes added must
157 have a secondary IP as well.
159 Note that for Ganeti it doesn't matter if the secondary network is
160 actually a separate physical network, or is done using tunneling,
161 etc. For performance reasons, it's recommended to use a separate
164 The ``--vg-name`` option will let you specify a volume group
165 different than "xenvg" for Ganeti to use when creating instance
166 disks. This volume group must have the same name on all nodes. Once
167 the cluster is initialized this can be altered by using the
168 **modify** command. If you don't want to use lvm storage at all use
169 the ``--no-lvm-storage`` option. Once the cluster is initialized
170 you can change this setup with the **modify** command.
172 The ``--master-netdev`` option is useful for specifying a different
173 interface on which the master will activate its IP address. It's
174 important that all nodes have this interface because you'll need it
175 for a master failover.
177 The ``-m`` option will let you specify a three byte prefix under
178 which the virtual MAC addresses of your instances will be
179 generated. The prefix must be specified in the format XX:XX:XX and
180 the default is aa:00:00.
182 The ``--no-lvm-storage`` option allows you to initialize the
183 cluster without lvm support. This means that only instances using
184 files as storage backend will be possible to create. Once the
185 cluster is initialized you can change this setup with the
188 The ``--no-etc-hosts`` option allows you to initialize the cluster
189 without modifying the /etc/hosts file.
191 The ``--no-ssh-init`` option allows you to initialize the cluster
192 without creating or distributing SSH key pairs.
194 The ``--file-storage-dir`` option allows you set the directory to
195 use for storing the instance disk files when using file storage as
196 backend for instance disks.
198 The ``--enabled-hypervisors`` option allows you to set the list of
199 hypervisors that will be enabled for this cluster. Instance
200 hypervisors can only be chosen from the list of enabled
201 hypervisors, and the first entry of this list will be used by
202 default. Currently, the following hypervisors are available:
204 The ``--prealloc-wipe-disks`` sets a cluster wide configuration
205 value for wiping disks prior to allocation. This increases security
206 on instance level as the instance can't access untouched data from
207 it's underlying storage.
223 a simple chroot manager that starts chroot based on a script at the
224 root of the filesystem holding the chroot
227 fake hypervisor for development/testing
230 Either a single hypervisor name or a comma-separated list of
231 hypervisor names can be specified. If this option is not specified,
232 only the xen-pvm hypervisor is enabled by default.
234 The ``--hypervisor-parameters`` option allows you to set default
235 hypervisor specific parameters for the cluster. The format of this
236 option is the name of the hypervisor, followed by a colon and a
237 comma-separated list of key=value pairs. The keys available for
238 each hypervisors are detailed in the gnt-instance(8) man page, in
239 the **add** command plus the following parameters which are only
240 configurable globally (at cluster level):
243 Valid for the Xen PVM and KVM hypervisors.
245 This options specifies the TCP port to use for live-migration. For
246 Xen, the same port should be configured on all nodes in the
247 ``/etc/xen/xend-config.sxp`` file, under the key
248 "xend-relocation-port".
251 Valid for the KVM hypervisor.
253 This option specifies the maximum bandwidth that KVM will use for
254 instance live migrations. The value is in MiB/s.
256 This option is only effective with kvm versions >= 78 and qemu-kvm
260 The ``--backend-parameters`` option allows you to set the default
261 backend parameters for the cluster. The parameter format is a
262 comma-separated list of key=value pairs with the following
266 Number of VCPUs to set for an instance by default, must be an
267 integer, will be set to 1 if no specified.
270 Amount of memory to allocate for an instance by default, can be
271 either an integer or an integer followed by a unit (M for mebibytes
272 and G for gibibytes are supported), will be set to 128M if not
276 Value of the auto\_balance flag for instances to use by default,
277 will be set to true if not specified.
280 The ``--nic-parameters`` option allows you to set the default nic
281 parameters for the cluster. The parameter format is a
282 comma-separated list of key=value pairs with the following
286 The default nic mode, 'routed' or 'bridged'.
289 In bridged mode the default NIC bridge. In routed mode it
290 represents an hypervisor-vif-script dependent value to allow
291 different instance groups. For example under the KVM default
292 network script it is interpreted as a routing table number or
296 The option ``--maintain-node-health`` allows to enable/disable
297 automatic maintenance actions on nodes. Currently these include
298 automatic shutdown of instances and deactivation of DRBD devices on
299 offline nodes; in the future it might be extended to automatic
300 removal of unknown LVM volumes, etc.
302 The ``--uid-pool`` option initializes the user-id pool. The
303 *user-id pool definition* can contain a list of user-ids and/or a
304 list of user-id ranges. The parameter format is a comma-separated
305 list of numeric user-ids or user-id ranges. The ranges are defined
306 by a lower and higher boundary, separated by a dash. The boundaries
307 are inclusive. If the ``--uid-pool`` option is not supplied, the
308 user-id pool is initialized to an empty list. An empty list means
309 that the user-id pool feature is disabled.
311 The ``-I (--default-iallocator)`` option specifies the default
312 instance allocator. The instance allocator will be used for
313 operations like instance creation, instance and node migration,
314 etc. when no manual override is specified. If this option is not
315 specified, the default instance allocator will be blank, which
316 means that relevant operations will require the administrator to
317 manually specify either an instance allocator, or a set of nodes.
318 The default iallocator can be changed later using the **modify**
321 The ``--primary-ip-version`` option specifies the IP version used
322 for the primary address. Possible values are 4 and 6 for IPv4 and
323 IPv6, respectively. This option is used when resolving node names
324 and the cluster name.
331 List the tags of the cluster.
336 **master-failover** [--no-voting]
338 Failover the master role to the current node.
340 The ``--no-voting`` option skips the remote node agreement checks.
341 This is dangerous, but necessary in some cases (for example failing
342 over the master role in a 2 node cluster with the original master
343 down). If the original master then comes up, it won't be able to
344 start its master daemon because it won't have enough votes, but so
345 won't the new master, if the master daemon ever needs a restart.
346 You can pass ``--no-voting`` to **ganeti-masterd** on the new
347 master to solve this problem, and run **gnt-cluster redist-conf**
348 to make sure the cluster is consistent again.
355 Checks if the master daemon is alive.
357 If the master daemon is alive and can respond to a basic query (the
358 equivalent of **gnt-cluster info**), then the exit code of the
359 command will be 0. If the master daemon is not alive (either due to
360 a crash or because this is not the master node), the exit code will
367 | [--vg-name *vg-name*]
369 | [--enabled-hypervisors *hypervisors*]
370 | [--hypervisor-parameters *hypervisor*:*hv-param*=*value*[,*hv-param*=*value*...]]
371 | [--backend-parameters *be-param*=*value* [,*be-param*=*value*...]]
372 | [--nic-parameters *nic-param*=*value* [,*nic-param*=*value*...]]
373 | [--uid-pool *user-id pool definition*]
374 | [--add-uids *user-id pool definition*]
375 | [--remove-uids *user-id pool definition*]
376 | [-C *candidate\_pool\_size*]
377 | [--maintain-node-health {yes \| no}]
378 | [--prealloc-wipe-disks {yes \| no}]
379 | [-I *default instance allocator*]
380 | [--reserved-lvs=*NAMES*]
382 Modify the options for the cluster.
384 The ``--vg-name``, ``--no-lvm-storarge``,
385 ``--enabled-hypervisors``, ``--hypervisor-parameters``,
386 ``--backend-parameters``, ``--nic-parameters``,
387 ``--maintain-node-health``, ``--prealloc-wipe-disks``,
388 ``--uid-pool`` options are described in the **init** command.
390 The ``-C`` option specifies the ``candidate_pool_size`` cluster
391 parameter. This is the number of nodes that the master will try to
392 keep as master\_candidates. For more details about this role and
393 other node roles, see the ganeti(7). If you increase the size, the
394 master will automatically promote as many nodes as required and
395 possible to reach the intended number.
397 The ``--add-uids`` and ``--remove-uids`` options can be used to
398 modify the user-id pool by adding/removing a list of user-ids or
401 The option ``--reserved-lvs`` specifies a list (comma-separated) of
402 logical volume group names (regular expressions) that will be
403 ignored by the cluster verify operation. This is useful if the
404 volume group used for Ganeti is shared with the system for other
405 uses. Note that it's not recommended to create and mark as ignored
406 logical volume names which match Ganeti's own name format (starting
407 with UUID and then .diskN), as this option only skips the
408 verification, but not the actual use of the names given.
410 To remove all reserved logical volumes, pass in an empty argument
411 to the option, as in ``--reserved-lvs=`` or ``--reserved-lvs ''``.
413 The ``-I`` is described in the **init** command. To clear the
414 default iallocator, just pass an empty string ('').
419 **queue** {drain | undrain | info}
421 Change job queue properties.
423 The ``drain`` option sets the drain flag on the job queue. No new
424 jobs will be accepted, but jobs already in the queue will be
427 The ``undrain`` will unset the drain flag on the job queue. New
428 jobs will be accepted.
430 The ``info`` option shows the properties of the job queue.
435 **watcher** {pause *duration* | continue | info}
437 Make the watcher pause or let it continue.
439 The ``pause`` option causes the watcher to pause for *duration*
442 The ``continue`` option will let the watcher continue.
444 The ``info`` option shows whether the watcher is currently paused.
449 **redist-conf** [--submit]
451 This command forces a full push of configuration files from the
452 master node to the other nodes in the cluster. This is normally not
453 needed, but can be run if the **verify** complains about
454 configuration mismatches.
456 The ``--submit`` option is used to send the job to the master
457 daemon but not wait for its completion. The job ID will be shown so
458 that it can be examined via **gnt-job info**.
463 **remove-tags** [--from *file*] {*tag*...}
465 Remove tags from the cluster. If any of the tags are not existing
466 on the cluster, the entire operation will abort.
468 If the ``--from`` option is given, the list of tags to be removed will
469 be extended with the contents of that file (each line becomes a tag).
470 In this case, there is not need to pass tags on the command line (if
471 you do, tags from both sources will be removed). A file name of - will
472 be interpreted as stdin.
477 **rename** [-f] {*name*}
479 Renames the cluster and in the process updates the master IP
480 address to the one the new name resolves to. At least one of either
481 the name or the IP address must be different, otherwise the
482 operation will be aborted.
484 Note that since this command can be dangerous (especially when run
485 over SSH), the command will require confirmation unless run with
491 | **renew-crypto** [-f]
492 | [--new-cluster-certificate] [--new-confd-hmac-key]
493 | [--new-rapi-certificate] [--rapi-certificate *rapi-cert*]
494 | [--new-cluster-domain-secret] [--cluster-domain-secret *filename*]
496 This command will stop all Ganeti daemons in the cluster and start
497 them again once the new certificates and keys are replicated. The
498 options ``--new-cluster-certificate`` and ``--new-confd-hmac-key``
499 can be used to regenerate the cluster-internal SSL certificate
500 respective the HMAC key used by ganeti-confd(8).
502 To generate a new self-signed RAPI certificate (used by
503 ganeti-rapi(8)) specify ``--new-rapi-certificate``. If you want to
504 use your own certificate, e.g. one signed by a certificate
505 authority (CA), pass its filename to ``--rapi-certificate``.
507 ``--new-cluster-domain-secret`` generates a new, random cluster
508 domain secret. ``--cluster-domain-secret`` reads the secret from a
509 file. The cluster domain secret is used to sign information
510 exchanged between separate clusters via a third party.
515 **repair-disk-sizes** [instance...]
517 This command checks that the recorded size of the given instance's
518 disks matches the actual size and updates any mismatches found.
519 This is needed if the Ganeti configuration is no longer consistent
520 with reality, as it will impact some disk operations. If no
521 arguments are given, all instances will be checked.
523 Note that only active disks can be checked by this command; in case
524 a disk cannot be activated it's advised to use
525 **gnt-instance activate-disks --ignore-size ...** to force
526 activation without regard to the current size.
528 When the all disk sizes are consistent, the command will return no
529 output. Otherwise it will log details about the inconsistencies in
535 **search-tags** {*pattern*}
537 Searches the tags on all objects in the cluster (the cluster
538 itself, the nodes and the instances) for a given pattern. The
539 pattern is interpreted as a regular expression and a search will be
540 done on it (i.e. the given pattern is not anchored to the beggining
541 of the string; if you want that, prefix the pattern with ^).
543 If no tags are matching the pattern, the exit code of the command
544 will be one. If there is at least one match, the exit code will be
545 zero. Each match is listed on one line, the object and the tag
546 separated by a space. The cluster will be listed as /cluster, a
547 node will be listed as /nodes/*name*, and an instance as
548 /instances/*name*. Example:
552 # gnt-cluster search-tags time
553 /cluster ctime:2007-09-01
554 /nodes/node1.example.com mtime:2007-10-04
559 **verify** [--no-nplus1-mem]
561 Verify correctness of cluster configuration. This is safe with
562 respect to running instances, and incurs no downtime of the
565 If the ``--no-nplus1-mem`` option is given, Ganeti won't check
566 whether if it loses a node it can restart all the instances on
567 their secondaries (and report an error otherwise).
574 The command checks which instances have degraded DRBD disks and
575 activates the disks of those instances.
577 This command is run from the **ganeti-watcher** tool, which also
578 has a different, complementary algorithm for doing this check.
579 Together, these two should ensure that DRBD disks are kept
587 Show the cluster version.