ganeti-instance-image ===================== This is a guest OS definition for Ganeti (http://code.google.com/p/ganeti). It will install a Linux-based image using either a tarball, filesystem dump, or a qemu-img disk image file. This definition also allows for manual creation of an instance by simply setting only the disks up and allowing you to boot via the install cd manually. The goal of this instance is to allow fast and flexible installation of instances without the need for external tools such as debootstrap. Installation ------------ In order to install this package from source, you need to determine what options ganeti itself has been configured with. If ganeti was built directly from source, then the only place it looks for OS definitions is ``/srv/ganeti/os``, and you need to install the OS under it:: ./configure --prefix=/usr --localstatedir=/var \ --sysconfdir=/etc \ --with-os-dir=/srv/ganeti/os make && make install If ganeti was installed from a package, its default OS path should already include /usr/share/ganeti/os, so you can just run:: ./configure -prefix=/usr --localstatedir=/var \ --sysconfdir=/etc make && make install Note that you need to repeat this procedure on all nodes of the cluster. The actual path that ganeti has been installed with can be determined by looking for a file named _autoconf.py under a ganeti directory in the python modules tree (e.g. ``/usr/lib/python2.4/site-packages/ganeti/_autoconf.py``). In this file, a variable named OS_SEARCH_PATH will list all the directories in which ganeti will look for OS definitions. Configuration of instance creation ---------------------------------- The kind of instance created can be customized via a settings file. This file may or may not be installed by default, as the instance creation will work without it. The creation scripts will look for it in ``$sysconfdir/default/ganeti-instance-image``, so if you have run configure with the parameter ``--sysconfdir=/etc``, the final filename will be ``/etc/default/ganeti-instance-image``. The following settings will be examined in this file: - CDINSTALL: If 'yes' only setup disks for a cd based install or manual installation via other means. It will not deploy any images or create any partitions. (default: no) - SWAP: Create a swap partition (tarball only) (default: yes) - SWAP_SIZE: Manually set the default swap partition size in MB (default: size of instance memory) - FILESYSTEM: Set which filesystem to format the disks as. Currently only supports ext3 or ext4. (default: ext3) - KERNEL_ARGS: Add additional kernel boot parameters to an instance. This currently only works on booting a kernel from inside. - IMAGE_NAME: Name for the image to use. Generally they will have names similar to: centos-5.4, debian-5.0, etc. The naming is free form depending on what you name the file itself. - IMAGE_TYPE: Create instance by either using a gzipped tarball, file system dump, or an image created by qemu-img. Accepts either 'tarball', 'dump', or 'qemu'. (default: dump). - IMAGE_DIR: Override default location for images. (default: ``$localstatedir/cache/ganeti-instance-image``) - ARCH: Define the architecture of the image to use. Accepts either 'x86' or 'x86_64'. - CUSTOMIZE_DIR: A directory containing customization script for the instance. (by default $sysconfdir/ganeti/instance-image/hooks) See "Customization of the instance" below. - IMAGE_DEBUG: Enable verbose output for instance scripts. Note that the settings file is important on the node that the instance is installed on, not the cluster master. This is indeed not a very good model of using this OS but currently the OS interface in ganeti is limiting. Creation of Deployment Images ----------------------------- There are three types that are supported for deploying images. * tarball * dump * qemu image Tarball ~~~~~~~ Tarball based images are quite simply a tarball of a working system. An good example use case for this is deploying a Gentoo instance using a stage4 tarball. The only requirement is that the tarball is gzipped instead of bzip2 for speed. If you wish use a kernel inside of the VM instead of externally, make sure that a working kernel, grub config are install in the tarball. Enable the 'grub' custom script to install the grub boot image during installation. Qemu Images ~~~~~~~~~~~ NOTE: qemu images will create a partition of the exact same size as it was originally created with. So if you create a 4GB image and created a new instance of 10G it will create a partition that is only 4GB and leave the rest as "free". To create a new qemu based disk image, you will need to able the ``CDINSTALL`` option and install the VM using the distro's provided installation medium. It is not recommended to build images on systems outside of ganeti (such as libvirt) as we have encountered issues with systems segfaulting. Once the instance has been created, boot the instance and point it to the install medium:: gnt-instance start -H cdrom_image_path=path/to/iso/ubuntu-9.10.iso, \ boot_order=cdrom instance-name Once the base image has been installed, ensure you have the acpid package installed so that ganeti can shutdown the VM properly. Once you are happy with your base image, shutdown the VM, activate the disks, and create the disk image using qemu-img. Its recommended you use qcow2 with compression to reduce the amount of disk space used:: # activate disks gnt-instance activate-disks instance-name # create image qemu-img convert -c -f host_device /dev/drbd1 \ -O qcow2 $IMAGE_DIR/ubuntu-9.10-x86_64.img Note: Older versions of qemu-img may not support the ``host_device`` format so use ``raw`` instead which should work in theory. Dump ~~~~ The last, and most efficient type of disk image is creating filesystem dumps using the dump command. The advantage with using dumps is that its much faster to deploy using it, and it also has built-in compression. The disadvantage is that you need to install grub manually which might be an issue on some operating systems. We currently fully support grub 1 and have partial support with grub2. After the new instance has booted, you will need to run ``update-grub`` and reboot the VM to get the new settings. We currently cannot run ``update-grub`` during the install because of an upstream grub2 issue. You will need to create images for both the boot and root partition (if you include a boot partition). Create a base image for an instance just like its described in Qemu Images. Make sure the instance is shutdown and then issue the following commands (assuming the activated disk is drbd1):: dump -0 -q -z9 -f ${IMAGE_DIR}/${IMAGE_NAME}-${ARCH}-boot.dump \ /dev/mapper/drbdq-1 dump -0 -q -z9 -f ${IMAGE_DIR}/${IMAGE_NAME}-${ARCH}-root.dump \ /dev/mapper/drbdq-3 Partition Layout ~~~~~~~~~~~~~~~~ Currently the partition layout is locked into a specific way in order to make it work more elegantly with ganeti. We might change this to be more flexible in the future, however you *must* use the following layout otherwise ganeti will not install the VM correctly. Currently the following partition layout is assumed: With swap:: /dev/$disk1 /boot /dev/$disk2 swap /dev/$disk3 / Without swap:: /dev/$disk1 /boot /dev/$disk2 / NOTE: If you have kernel_path set, /boot will not be created and all partition numbers will go up by one. For example: With swap:: /dev/$disk1 swap /dev/$disk2 / Without swap:: /dev/$disk1 / Image Naming ~~~~~~~~~~~~ The naming convention that is used is the following: tarball: $IMAGE_NAME-$ARCH.tar.gz dump: $IMAGE_NAME-$ARCH-boot.dump $IMAGE_NAME-$ARCH-root.dump qemu-img: $IMAGE_NAME-$ARCH.img Useful Scripts ~~~~~~~~~~~~~~ There are a set of useful scripts located in /usr/share/ganeti/os/image/tools that you are welcome to use. These scripts are all intended to be run on the master node:: mount-disks $instance_name umount-disks $instance_name This will mount or umount an instance to /tmp/${instance_name}_root ``make-dump $instance_name [ $IMAGE_DIR ]`` Create dump images for the given OS variant. You can override the default $IMAGE_DIR setting by giving it as a second argument. ``make-qemu-img $instance_name [ $IMAGE_DIR ]`` Create an qemu image for the given OS variant. Customization of the instance ----------------------------- If run-parts is in the os create script, and the CUSTOMIZE_DIR (by default $sysconfdir/ganeti/instance-image/hooks, /etc/ganeti/instance-image/hooks if you configured the os with --sysconfdir=/etc) directory exists any executable whose name matches the run-parts execution rules (quoting run-parts(8): the names must consist entirely of upper and lower case letters, digits, underscores, and hyphens) is executed to allow further personalization of the installation. The following environment variables are passed, in addition to the ones ganeti passes to the OS scripts: TARGET: directory in which the filesystem is mounted BLOCKDEV: ganeti block device ROOT_DEV: device in which the root (/) filesystem resides (the one mounted in TARGET) BOOT_DEV: device in which the boot (/boot) filesystem resides IMAGE_TYPE: type of image being used (tarball, qemu, dump) The scripts in CUSTOMIZE_DIR can exit with an error code to signal an error in the instance creation, should they fail. The scripts in CUSTOMIZE_DIR should not start any long-term processes or daemons using this directory, otherwise the installation will fail because it won't be able to umount the filesystem from the directory, and hand the instance back to Ganeti. Included Custom Scripts ----------------------- This OS definition includes three optional customization scripts that are disabled by default. They are not required but are useful. Grub ~~~~ When enabled, this can setup three things: - Install grub into the MBR - Setup serial access to grub - Add optional kernel boot parameters In general, the MBR will only be installed if you're not using a qemu image type, or the ``kernel_path`` parameter is empty or initiating an import. There is currently partial support for install a grub2 MBR (which Ubuntu Karmic and newer requires). If ``serial_console`` is ``True`` then this script will try to enable serial support for grub. Interfaces ~~~~~~~~~~ When enabled, it would try to setup networking for eth0 and enable DHCP. It assumes you already have a DHCP client installed on the guest OS. This currently supports the following operating systems: - Redhat (CentOS/Fedora) - Debian/Ubuntu - Gentoo - OpenSUSE If you need to set a static ip for instances, you can do that by creating several files in a manner described below. Subnets: Create a file in ``/etc/ganeti/instance-image/networks/subnets`` that has a useful name such as ``vlan42``. This file will describe subnet routing information such as the netmask and gateway. The following is an example: NETMASK=255.255.255.0 GATEWAY=192.168.1.1 Instance: Create a file in ``/etc/ganeti/instance-image/networks/instances`` and name it the FQDN of the instance. The file will describe the IP address for the instance and which subnet it resides on. For example, we could create a file named ``myinstance.osuosl.org`` and have the following in it: ADDRESS=192.168.1.100 SUBNET=vlan42 SSH ~~~ When enabled, it will clear out any generated ssh keys that the image may have so that each instance have *unique* host keys. Currently its disabled for Debian/Ubuntu since the keys won't be regenerated via the init script. We plan to fix this manually at some point in the future. # vi: set tw=80 ft=rst :