Documents Ganeti version 1.2 Ganeti is a cluster virtualization management system. This document explains how to bootstrap a Ganeti node and create a running cluster. You need to repeat most of the steps in this document for every node you want to install, but of course we recommend creating some semi-automatic procedure if you plan to deploy Ganeti on a medium/large scale. This document is divided into two main sections: Installation of the core system and base components Configuration of the environment for Ganeti Each of these is divided into sub-sections. While a full Ganeti system will need all of the steps specified, some are not strictly required for every environment. Which ones they are, and why, is specified in the corresponding sections. While Ganeti itself is distribution-agnostic most of the examples in this document will be targeted at Debian or Debian-derived distributions. You are expected to be familiar with your distribution, its package management system, and Xen before trying to use Ganeti. A basic Ganeti terminology glossary is provided in the introductory section of the "admin guide". Please refer to that if you are uncertain about the terms we are using. Mandatory. Please install your operating system as you would normally do. The only requirement you need to be aware of at this stage is to partition leaving enough space for a big LVM volume group which will then host your instance file systems. You can even create the volume group at installation time, of course: the default volume group name Ganeti 1.2 uses is "xenvg" but you may name it differently should you wish to, as long as the name is the same for all the nodes in the cluster. Mandatory: While Ganeti is developed with the ability to modularly run on different virtualization environments in mind the only one currently useable on a live system is Xen. Please follow your distribution's recommended way to install and set up Xen, or install Xen from the upstream source, if you wish, following their manual. For example under Debian 4.0 or 3.1+backports you can install the relevant xen-linux-system package, which will pull in both the hypervisor and the relevant kernel. On Ubuntu (from Gutsy on) the package is called ubuntu-xen-server. After installing Xen you need to reboot into your xenified dom0 system. Again on some distributions this might involve configuring GRUB appropriately. Recommended: DRBD is required if you want to use the high availability (HA) features of Ganeti, but optional if you don't require HA or only run Ganeti on single-node clusters. You can upgrade a non-HA cluster to an HA one later, but you might need to export and reimport all your instances to take advantage of the new features. Now the bad news: unless your distribution already provides it installing DRBD might involve recompiling your kernel or anyway fiddling with it. Hopefully at least the xenified kernel source to start from will be provided. Under Debian you can just install the drbd0.7-module-source and drbd0.7-utils packages, and your kernel source, and then run module-assistant to compile the drbd0.7 module. The following commands should do it: m-a update m-a a-i drbd0.7 The good news is that you don't need to configure DRBD at all. Ganeti will do it for you for every instance you set up. If you have the DRBD utils installed and the module in your kernel you're fine. Please check that your system is configured to load the module at every boot. Ganeti relies on Xen running in "bridge mode", which means the instances network interfaces will be attached to a software bridge running in dom0. Xen by default creates such a bridge at startup, but your distribution might have a different way to do things. In Debian, in order to enable the default Xen behaviour, you have to edit /etc/xen/xend-config.sxp and replace (network-script network-dummy) with (network-script network-bridge). The recommended Debian way to configure things, though, is to edit your /etc/network/interfaces file and substitute your normal ethernet stanza with something like: auto br0 iface br0 inet static address YOUR_IP_ADDRESS netmask YOUR_NETMASK network YOUR_NETWORK broadcast YOUR_BROADCAST_ADDRESS gateway YOUR_GATEWAY bridge_ports eth0 bridge_stp off bridge_fd 0 Beware that the default name Ganeti uses is xen-br0 (which was used in Xen 2.0) while Xen 3.0 uses xenbr0 by default. The default bridge your cluster will use for new instances can be specified at cluster initialization time. If you haven't configured your LVM volume group at install time you need to do it before trying to initialize the Ganeti cluster. This is done by formatting the devices/partitions you want to use for it and then adding them to the relevant volume group: pvcreate /dev/sda4 pvcreate /dev/sdb pvcreate /dev/sdc1 vgcreate xenvg /dev/sda4 /dev/sdb /dev/sdc1 If you want to add a device later you can do so with the vgextend 8 command: pvcreate /dev/sdd vgextend xenvg /dev/sdd As said before you may choose a different name for the volume group, as long as you stick to the same name on all the nodes of a cluster. It's now time to install the Ganeti software itself if you haven't done it yet. You can do it from source, with the usual steps (note that the localstatedir options must be set to /var): ./configure --localstatedir=/var make make install or you can install the package relevant to your distribution, for example in Debian/Ubuntu: dpkg -i ganeti_VERSION_all.deb or, if you have a source repository that holds the Ganeti software: apt-get install ganeti Another important component for Ganeti is the OS support packages, which let different operating systems be used as instances. You can install them by installing the relevant ganeti-instance-OS package. Mandatory: only on one node per cluster. The last step is to initialize the cluster. After you've repeated the above process or some semi-automatic form of it on all of your nodes choose one as the master, and execute: gnt-cluster init CLUSTERNAME Options you can pass to gnt-cluster init include the default bridge name (-b), the cluster-wide name for the volume group (-g) and the secondary ip address for the initial node should you wish to keep the data replication network separate. Invoke it with --help to see all the possibilities. Note that the cluster name must exist in DNS. You must choose a name different from any of the nodes names for a multi-node cluster. In general the best choice is to have a unique name for a cluster, even if it consists of only one machine. Mandatory: for all the other nodes. If you have already initialized your cluster you need to join the other nodes to it. You can do so by executing the following command on the master node: gnt-node add NODENAME The only option is -s, which sets the node's secondary ip address for replication purposes, if you are using a separate replication network. Now you can follow the admin guide to use your new Ganeti cluster.