1 Ganeti customisation using hooks
2 ================================
4 Documents Ganeti version 2.6
12 In order to allow customisation of operations, ganeti runs scripts
13 under ``/etc/ganeti/hooks`` based on certain rules.
16 This is similar to the ``/etc/network/`` structure present in Debian
17 for network interface handling.
22 For every operation, two sets of scripts are run:
24 - pre phase (for authorization/checking)
25 - post phase (for logging)
27 Also, for each operation, the scripts are run on one or more nodes,
28 depending on the operation type.
30 Note that, even though we call them scripts, we are actually talking
36 The *pre* scripts have a definite target: to check that the operation
37 is allowed given the site-specific constraints. You could have, for
38 example, a rule that says every new instance is required to exists in
39 a database; to implement this, you could write a script that checks
40 the new instance parameters against your database.
42 The objective of these scripts should be their return code (zero or
43 non-zero for success and failure). However, if they modify the
44 environment in any way, they should be idempotent, as failed
45 executions could be restarted and thus the script(s) run again with
46 exactly the same parameters.
48 Note that if a node is unreachable at the time a hooks is run, this
49 will not be interpreted as a deny for the execution. In other words,
50 only an actual error returned from a script will cause abort, and not
53 Therefore, if you want to guarantee that a hook script is run and
54 denies an action, it's best to put it on the master node.
59 These scripts should do whatever you need as a reaction to the
60 completion of an operation. Their return code is not checked (but
61 logged), and they should not depend on the fact that the *pre* scripts
67 The allowed names for the scripts consist of (similar to *run-parts*)
68 upper and lower case, digits, underscores and hyphens. In other words,
69 the regexp ``^[a-zA-Z0-9_-]+$``. Also, non-executable scripts will be
76 On a single node, the scripts in a directory are run in lexicographic
77 order (more exactly, the python string comparison order). It is
78 advisable to implement the usual *NN-name* convention where *NN* is a
81 For an operation whose hooks are run on multiple nodes, there is no
82 specific ordering of nodes with regard to hooks execution; you should
83 assume that the scripts are run in parallel on the target nodes
84 (keeping on each node the above specified ordering). If you need any
85 kind of inter-node synchronisation, you have to implement it yourself
91 The scripts will be run as follows:
93 - no command line arguments
95 - no controlling *tty*
97 - stdin is actually */dev/null*
99 - stdout and stderr are directed to files
101 - PATH is reset to :pyeval:`constants.HOOKS_PATH`
103 - the environment is cleared, and only ganeti-specific variables will
107 All information about the cluster is passed using environment
108 variables. Different operations will have sligthly different
109 environments, but most of the variables are common.
120 Adds a node to the cluster.
123 :env. vars: NODE_NAME, NODE_PIP, NODE_SIP, MASTER_CAPABLE, VM_CAPABLE
124 :pre-execution: all existing nodes
125 :post-execution: all nodes plus the new node
131 Removes a node from the cluster. On the removed node the hooks are
132 called during the execution of the operation and not after its
135 :directory: node-remove
136 :env. vars: NODE_NAME
137 :pre-execution: all existing nodes except the removed node
138 :post-execution: all existing nodes
143 Changes a node's parameters.
145 :directory: node-modify
146 :env. vars: MASTER_CANDIDATE, OFFLINE, DRAINED, MASTER_CAPABLE, VM_CAPABLE
147 :pre-execution: master node, the target node
148 :post-execution: master node, the target node
153 Relocate secondary instances from a node.
155 :directory: node-evacuate
156 :env. vars: NEW_SECONDARY, NODE_NAME
157 :pre-execution: master node, target node
158 :post-execution: master node, target node
163 Relocate secondary instances from a node.
165 :directory: node-migrate
166 :env. vars: NODE_NAME
167 :pre-execution: master node
168 :post-execution: master node
171 Node group operations
172 ~~~~~~~~~~~~~~~~~~~~~
177 Adds a node group to the cluster.
179 :directory: group-add
180 :env. vars: GROUP_NAME
181 :pre-execution: master node
182 :post-execution: master node
187 Changes a node group's parameters.
189 :directory: group-modify
190 :env. vars: GROUP_NAME, NEW_ALLOC_POLICY
191 :pre-execution: master node
192 :post-execution: master node
197 Removes a node group from the cluster. Since the node group must be
198 empty for removal to succeed, the concept of "nodes in the group" does
199 not exist, and the hook is only executed in the master node.
201 :directory: group-remove
202 :env. vars: GROUP_NAME
203 :pre-execution: master node
204 :post-execution: master node
209 Renames a node group.
211 :directory: group-rename
212 :env. vars: OLD_NAME, NEW_NAME
213 :pre-execution: master node and all nodes in the group
214 :post-execution: master node and all nodes in the group
219 Evacuates a node group.
221 :directory: group-evacuate
222 :env. vars: GROUP_NAME, TARGET_GROUPS
223 :pre-execution: master node and all nodes in the group
224 :post-execution: master node and all nodes in the group
230 All instance operations take at least the following variables:
231 INSTANCE_NAME, INSTANCE_PRIMARY, INSTANCE_SECONDARY,
232 INSTANCE_OS_TYPE, INSTANCE_DISK_TEMPLATE, INSTANCE_MEMORY,
233 INSTANCE_DISK_SIZES, INSTANCE_VCPUS, INSTANCE_NIC_COUNT,
234 INSTANCE_NICn_IP, INSTANCE_NICn_BRIDGE, INSTANCE_NICn_MAC,
235 INSTANCE_DISK_COUNT, INSTANCE_DISKn_SIZE, INSTANCE_DISKn_MODE.
237 The INSTANCE_NICn_* and INSTANCE_DISKn_* variables represent the
238 properties of the *n* -th NIC and disk, and are zero-indexed.
244 Creates a new instance.
246 :directory: instance-add
247 :env. vars: ADD_MODE, SRC_NODE, SRC_PATH, SRC_IMAGES
248 :pre-execution: master node, primary and secondary nodes
249 :post-execution: master node, primary and secondary nodes
251 OP_INSTANCE_REINSTALL
252 +++++++++++++++++++++
254 Reinstalls an instance.
256 :directory: instance-reinstall
257 :env. vars: only the standard instance vars
258 :pre-execution: master node, primary and secondary nodes
259 :post-execution: master node, primary and secondary nodes
264 Exports the instance.
266 :directory: instance-export
267 :env. vars: EXPORT_MODE, EXPORT_NODE, EXPORT_DO_SHUTDOWN, REMOVE_INSTANCE
268 :pre-execution: master node, primary and secondary nodes
269 :post-execution: master node, primary and secondary nodes
276 :directory: instance-start
278 :pre-execution: master node, primary and secondary nodes
279 :post-execution: master node, primary and secondary nodes
286 :directory: instance-stop
288 :pre-execution: master node, primary and secondary nodes
289 :post-execution: master node, primary and secondary nodes
296 :directory: instance-reboot
297 :env. vars: IGNORE_SECONDARIES, REBOOT_TYPE, SHUTDOWN_TIMEOUT
298 :pre-execution: master node, primary and secondary nodes
299 :post-execution: master node, primary and secondary nodes
301 OP_INSTANCE_SET_PARAMS
302 ++++++++++++++++++++++
304 Modifies the instance parameters.
306 :directory: instance-modify
307 :env. vars: NEW_DISK_TEMPLATE, RUNTIME_MEMORY
308 :pre-execution: master node, primary and secondary nodes
309 :post-execution: master node, primary and secondary nodes
314 Failovers an instance. In the post phase INSTANCE_PRIMARY and
315 INSTANCE_SECONDARY refer to the nodes that were repectively primary
316 and secondary before failover.
318 :directory: instance-failover
319 :env. vars: IGNORE_CONSISTENCY, SHUTDOWN_TIMEOUT, OLD_PRIMARY, OLD_SECONDARY, NEW_PRIMARY, NEW_SECONDARY
320 :pre-execution: master node, secondary node
321 :post-execution: master node, primary and secondary nodes
326 Migrates an instance. In the post phase INSTANCE_PRIMARY and
327 INSTANCE_SECONDARY refer to the nodes that were repectively primary
328 and secondary before migration.
330 :directory: instance-migrate
331 :env. vars: MIGRATE_LIVE, MIGRATE_CLEANUP, OLD_PRIMARY, OLD_SECONDARY, NEW_PRIMARY, NEW_SECONDARY
332 :pre-execution: master node, secondary node
333 :post-execution: master node, primary and secondary nodes
341 :directory: instance-remove
342 :env. vars: SHUTDOWN_TIMEOUT
343 :pre-execution: master node
344 :post-execution: master node, primary and secondary nodes
346 OP_INSTANCE_REPLACE_DISKS
347 +++++++++++++++++++++++++
349 Replace an instance's disks.
351 :directory: mirror-replace
352 :env. vars: MODE, NEW_SECONDARY, OLD_SECONDARY
353 :pre-execution: master node, primary and secondary nodes
354 :post-execution: master node, primary and secondary nodes
356 OP_INSTANCE_GROW_DISK
357 +++++++++++++++++++++
359 Grows the disk of an instance.
361 :directory: disk-grow
362 :env. vars: DISK, AMOUNT
363 :pre-execution: master node, primary and secondary nodes
364 :post-execution: master node, primary and secondary nodes
371 :directory: instance-rename
372 :env. vars: INSTANCE_NEW_NAME
373 :pre-execution: master node, primary and secondary nodes
374 :post-execution: master node, primary and secondary nodes
379 Move an instance by data-copying.
381 :directory: instance-move
382 :env. vars: TARGET_NODE, SHUTDOWN_TIMEOUT
383 :pre-execution: master node, primary and target nodes
384 :post-execution: master node, primary and target nodes
386 OP_INSTANCE_RECREATE_DISKS
387 ++++++++++++++++++++++++++
389 Recreate an instance's missing disks.
391 :directory: instance-recreate-disks
392 :env. vars: only the standard instance vars
393 :pre-execution: master node, primary and secondary nodes
394 :post-execution: master node, primary and secondary nodes
396 OP_INSTANCE_REPLACE_DISKS
397 +++++++++++++++++++++++++
399 Replace the disks of an instance.
401 :directory: mirrors-replace
402 :env. vars: MODE, NEW_SECONDARY, OLD_SECONDARY
403 :pre-execution: master node, primary and new secondary nodes
404 :post-execution: master node, primary and new secondary nodes
406 OP_INSTANCE_CHANGE_GROUP
407 ++++++++++++++++++++++++
409 Moves an instance to another group.
411 :directory: instance-change-group
412 :env. vars: TARGET_GROUPS
413 :pre-execution: master node
414 :post-execution: master node
423 This hook is called via a special "empty" LU right after cluster
426 :directory: cluster-init
429 :post-execution: master node
434 The post phase of this hook is called during the execution of destroy
435 operation and not after its completion.
437 :directory: cluster-destroy
440 :post-execution: master node
442 OP_CLUSTER_VERIFY_GROUP
443 +++++++++++++++++++++++
445 Verifies all nodes in a group. This is a special LU with regard to
446 hooks, as the result of the opcode will be combined with the result of
447 post-execution hooks, in order to allow administrators to enhance the
448 cluster verification procedure.
450 :directory: cluster-verify
451 :env. vars: CLUSTER, MASTER, CLUSTER_TAGS, NODE_TAGS_<name>
453 :post-execution: all nodes in a group
460 :directory: cluster-rename
462 :pre-execution: master-node
463 :post-execution: master-node
465 OP_CLUSTER_SET_PARAMS
466 +++++++++++++++++++++
468 Modifies the cluster parameters.
470 :directory: cluster-modify
471 :env. vars: NEW_VG_NAME
472 :pre-execution: master node
473 :post-execution: master node
475 Virtual operation :pyeval:`constants.FAKE_OP_MASTER_TURNUP`
476 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
478 This doesn't correspond to an actual op-code, but it is called when the
479 master IP is activated.
481 :directory: master-ip-turnup
482 :env. vars: MASTER_NETDEV, MASTER_IP, MASTER_NETMASK, CLUSTER_IP_VERSION
483 :pre-execution: master node
484 :post-execution: master node
486 Virtual operation :pyeval:`constants.FAKE_OP_MASTER_TURNDOWN`
487 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
489 This doesn't correspond to an actual op-code, but it is called when the
490 master IP is deactivated.
492 :directory: master-ip-turndown
493 :env. vars: MASTER_NETDEV, MASTER_IP, MASTER_NETMASK, CLUSTER_IP_VERSION
494 :pre-execution: master node
495 :post-execution: master node
501 The following operations are no longer present or don't execute hooks
502 anymore in Ganeti 2.0:
506 - OP_INSTANCE_ADD_MDDRBD
507 - OP_INSTANCE_REMOVE_MDDRBD
510 Environment variables
511 ---------------------
513 Note that all variables listed here are actually prefixed with *GANETI_*
514 in order to provide a clear namespace. In addition, post-execution
515 scripts receive another set of variables, prefixed with *GANETI_POST_*,
516 representing the status after the opcode executed.
521 This is the list of environment variables supported by all operations:
524 Documents the hooks interface version. In case this doesnt match
525 what the script expects, it should not run. The documents conforms
529 One of *PRE* or *POST* denoting which phase are we in.
538 One of the *OP_* values from the list of operations.
541 One of ``INSTANCE``, ``NODE``, ``CLUSTER``.
544 The path to the Ganeti configuration directory (to read, for
545 example, the *ssconf* files).
548 Specialised variables
549 ~~~~~~~~~~~~~~~~~~~~~
551 This is the list of variables which are specific to one or more
555 IP version of the master IP (4 or 6)
558 The name of the instance which is the target of the operation.
560 INSTANCE_BE_x,y,z,...
561 Instance BE params. There is one variable per BE param. For instance, GANETI_INSTANCE_BE_auto_balance
563 INSTANCE_DISK_TEMPLATE
564 The disk type for the instance.
567 The new disk type for the instance.
570 The number of disks for the instance.
573 The size of disk *n* for the instance.
576 Either *rw* for a read-write disk or *ro* for a read-only one.
578 INSTANCE_HV_x,y,z,...
579 Instance hypervisor options. There is one variable per option. For instance, GANETI_INSTANCE_HV_use_bootloader
582 The instance hypervisor.
585 The number of NICs for the instance.
588 The bridge to which the *n* -th NIC of the instance is attached.
591 The IP (if any) of the *n* -th NIC of the instance.
594 The MAC address of the *n* -th NIC of the instance.
597 The mode of the *n* -th NIC of the instance.
600 The name of the instance OS.
603 The name of the node which is the primary for the instance. Note that
604 for migrations/failovers, you shouldn't rely on this variable since
605 the nodes change during the exectution, but on the
606 OLD_PRIMARY/NEW_PRIMARY values.
609 Space-separated list of secondary nodes for the instance. Note that
610 for migrations/failovers, you shouldn't rely on this variable since
611 the nodes change during the exectution, but on the
612 OLD_SECONDARY/NEW_SECONDARY values.
615 The memory size (in MiBs) of the instance.
618 The number of virtual CPUs for the instance.
621 The run status of the instance.
624 Whether a node is capable of being promoted to master.
627 Whether the node can host instances.
630 Network device of the master IP
636 Netmask of the master IP
639 A space-delimited list of the instance's tags.
642 The target node of this operation (not the node on which the hook
646 The primary IP of the target node (the one over which inter-node
647 communication is done).
650 The secondary IP of the target node (the one over which drbd
651 replication is done). This can be equal to the primary ip, in case
652 the cluster is not dual-homed.
655 This is provided by some operations when the user gave this flag.
658 The user has specified this flag. It is used when failing over
659 instances in case the primary node is down.
662 The mode of the instance create: either *create* for create from
663 scratch or *import* for restoring from an exported image.
665 SRC_NODE, SRC_PATH, SRC_IMAGE
666 In case the instance has been added by import, these variables are
667 defined and point to the source node, source path (the directory
668 containing the image and the config file) and the source disk image
672 The name of the node on which the new mirror component is being
673 added (for replace disk). This can be the name of the current
674 secondary, if the new mirror is on the same secondary. For
675 migrations/failovers, this is the old primary node.
678 The name of the old secondary in the replace-disks command. Note that
679 this can be equal to the new secondary if the secondary node hasn't
680 actually changed. For migrations/failovers, this is the new primary
683 OLD_PRIMARY, NEW_PRIMARY
684 For migrations/failovers, the old and respectively new primary
685 nodes. These two mirror the NEW_SECONDARY/OLD_SECONDARY variables
688 The instance export mode. Either "remote" or "local".
691 The node on which the exported image of the instance was done.
694 This variable tells if the instance has been shutdown or not while
695 doing the export. In the "was shutdown" case, it's likely that the
696 filesystem is consistent, whereas in the "did not shutdown" case,
697 the filesystem would need a check (journal replay or full fsck) in
698 order to guarantee consistency.
701 Whether the instance was removed from the node.
704 Amount of time to wait for the instance to shutdown.
707 Amount of time to wait before aborting the op.
710 Old/new name of the node group.
713 The name of the node group.
716 The new allocation policy for the node group.
719 The list of cluster tags, space separated.
722 The list of tags for node *<name>*, space separated.
727 The startup of an instance will pass this environment to the hook
730 GANETI_CLUSTER=cluster1.example.com
731 GANETI_DATA_DIR=/var/lib/ganeti
733 GANETI_HOOKS_PATH=instance-start
734 GANETI_HOOKS_PHASE=post
735 GANETI_HOOKS_VERSION=2
736 GANETI_INSTANCE_DISK0_MODE=rw
737 GANETI_INSTANCE_DISK0_SIZE=128
738 GANETI_INSTANCE_DISK_COUNT=1
739 GANETI_INSTANCE_DISK_TEMPLATE=drbd
740 GANETI_INSTANCE_MEMORY=128
741 GANETI_INSTANCE_NAME=instance2.example.com
742 GANETI_INSTANCE_NIC0_BRIDGE=xen-br0
743 GANETI_INSTANCE_NIC0_IP=
744 GANETI_INSTANCE_NIC0_MAC=aa:00:00:a5:91:58
745 GANETI_INSTANCE_NIC_COUNT=1
746 GANETI_INSTANCE_OS_TYPE=debootstrap
747 GANETI_INSTANCE_PRIMARY=node3.example.com
748 GANETI_INSTANCE_SECONDARY=node5.example.com
749 GANETI_INSTANCE_STATUS=down
750 GANETI_INSTANCE_VCPUS=1
751 GANETI_MASTER=node1.example.com
752 GANETI_OBJECT_TYPE=INSTANCE
753 GANETI_OP_CODE=OP_INSTANCE_STARTUP
754 GANETI_OP_TARGET=instance2.example.com
756 .. vim: set textwidth=72 :