For optimum performance when running both CPU and I/O intensive
instances, it's also recommended that the dom0 is restricted to one CPU
-only, for example by booting with the kernel parameter ``maxcpus=1``.
+only. For example you can add ``dom0_max_vcpus=1,dom0_vcpus_pin`` to your
+kernels boot command line and set ``dom0-cpus`` in
+``/etc/xen/xend-config.sxp`` like this::
+
+ (dom0-cpus 1)
It is recommended that you disable xen's automatic save of virtual
machines at system shutdown and subsequent restore of them at reboot.
line assumes that all your nodes have secondary IPs in the
192.0.2.0/24 network, adjust it accordingly to your setup.
-.. admonition:: Debian
-
- Besides the ballooning change which you need to set in
- ``/etc/xen/xend-config.sxp``, you need to set the memory and nosmp
- parameters in the file ``/boot/grub/menu.lst``. You need to modify
- the variable ``xenhopt`` to add ``dom0_mem=1024M`` like this:
-
- .. code-block:: text
-
- ## Xen hypervisor options to use with the default Xen boot option
- # xenhopt=dom0_mem=1024M
-
- and the ``xenkopt`` needs to include the ``maxcpus`` option like
- this:
-
- .. code-block:: text
-
- ## Xen Linux kernel options to use with the default Xen boot option
- # xenkopt=maxcpus=1
-
- Any existing parameters can be left in place: it's ok to have
- ``xenkopt=console=tty0 maxcpus=1``, for example. After modifying the
- files, you need to run::
-
- $ /sbin/update-grub
-
If you want to run HVM instances too with Ganeti and want VNC access to
the console of your instances, set the following two entries in
``/etc/xen/xend-config.sxp``:
$ apt-get install drbd8-source drbd8-utils
$ m-a update
$ m-a a-i drbd8
+
+ Or on newer versions, if the kernel already has modules:
+
+ $ apt-get install drbd8-utils
+
+ Then to configure it for Ganeti::
+
$ echo drbd minor_count=128 usermode_helper=/bin/true >> /etc/modules
$ depmod -a
$ modprobe drbd minor_count=128 usermode_helper=/bin/true
- It is also recommended that you comment out the default resources in
- the ``/etc/drbd.conf`` file, so that the init script doesn't try to
+ It is also recommended that you comment out the default resources (if any)
+ in the ``/etc/drbd.conf`` file, so that the init script doesn't try to
configure any drbd devices. You can do this by prefixing all
*resource* lines in the file with the keyword *skip*, like this:
Other required software
+++++++++++++++++++++++
-See :doc:`install-quick`.
+Please install all software requirements mentioned in :doc:`install-quick`.
+If you want to build Ganeti from source, don't forget to follow the steps
+required for that as well.
Setting up the environment for Ganeti
-------------------------------------
**Mandatory** on all nodes.
-You can run Ganeti either in "bridged mode" or in "routed mode". In
-bridged mode, the default, the instances network interfaces will be
-attached to a software bridge running in dom0. Xen by default creates
-such a bridge at startup, but your distribution might have a different
-way to do things, and you'll definitely need to manually set it up under
-KVM.
+You can run Ganeti either in "bridged mode", "routed mode" or
+"openvswitch mode". In bridged mode, the default, the instances network
+interfaces will be attached to a software bridge running in dom0. Xen by
+default creates such a bridge at startup, but your distribution might
+have a different way to do things, and you'll definitely need to
+manually set it up under KVM.
Beware that the default name Ganeti uses is ``xen-br0`` (which was used
in Xen 2.0) while Xen 3.0 uses ``xenbr0`` by default. See the
instances, through their interface, in the table you specified (under
KVM, and in the main table under Xen).
+Also for "openvswitch mode" under Xen a custom network script is needed.
+Under KVM everything should work, but you'll need to configure your
+switches outside of Ganeti (as for bridges).
+
+.. admonition:: Bridging issues with certain kernels
+
+ Some kernel versions (e.g. 2.6.32) have an issue where the bridge
+ will automatically change its ``MAC`` address to the lower-numbered
+ slave on port addition and removal. This means that, depending on
+ the ``MAC`` address of the actual NIC on the node and the addresses
+ of the instances, it could be that starting, stopping or migrating
+ instances will lead to timeouts due to the address of the bridge
+ (and thus node itself) changing.
+
+ To prevent this, it's enough to set the bridge manually to a
+ specific ``MAC`` address, which will disable this automatic address
+ change. In Debian, this can be done as follows in the bridge
+ configuration snippet::
+
+ up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE
+
+ which will "set" the bridge address to the initial one, disallowing
+ changes.
+
.. admonition:: Bridging under Debian
The recommended way to configure the Xen bridge is to edit your
bridge_ports eth0
bridge_stp off
bridge_fd 0
+ # example for setting manually the bridge address to the eth0 NIC
+ up ip link set addr $(cat /sys/class/net/eth0/address) dev $IFACE
The following commands need to be executed on the local console::
source archive to ``/etc/init.d/ganeti`` and register it with your
distribution's startup scripts, for example in Debian::
+ $ chmod +x /etc/init.d/ganeti
$ update-rc.d ganeti defaults 20 80
In order to automatically restart failed instances, you need to setup a
installation script. An example OS that works under Debian and can
install Debian and Ubuntu instace OSes is provided on the project web
site. Download it from the project page and follow the instructions in
-the ``README`` file. Here is the installation procedure (replace 0.9
+the ``README`` file. Here is the installation procedure (replace 0.12
with the latest version that is compatible with your ganeti version)::
$ cd /usr/local/src/
- $ wget http://ganeti.googlecode.com/files/ganeti-instance-debootstrap-%0.9%.tar.gz
- $ tar xzf ganeti-instance-debootstrap-%0.9%.tar.gz
- $ cd ganeti-instance-debootstrap-%0.9%
- $ ./configure
+ $ wget http://ganeti.googlecode.com/files/ganeti-instance-debootstrap-%0.12%.tar.gz
+ $ tar xzf ganeti-instance-debootstrap-%0.12%.tar.gz
+ $ cd ganeti-instance-debootstrap-%0.12%
+ $ ./configure --with-os-dir=/srv/ganeti/os
$ make
$ make install
``kbd`` can be used instead of ``console-tools``, and more packages
can be added, of course, if needed.
+Please refer to the ``README`` file of ``ganeti-instance-debootstrap`` for
+further documentation.
+
Alternatively, you can create your own OS definitions. See the manpage
-:manpage:`ganeti-os-interface`.
+:manpage:`ganeti-os-interface(7)`.
Initializing the cluster
++++++++++++++++++++++++
**Mandatory** once per cluster, on the first node.
The last step is to initialize the cluster. After you have repeated the
-above process on all of your nodes, choose one as the master, and
-execute::
+above process on all of your nodes and choose one as the master. Make sure
+there is a SSH key pair on the master node (optionally generating one using
+``ssh-keygen``). Finally execute::
$ gnt-cluster init %CLUSTERNAME%
Your instance types, networking environment, hypervisor type and version
may all affect what kind of parameters should be used on your cluster.
-For example kvm instances are by default configured to use a host
-kernel, and to be reached via serial console, which works nice for Linux
-paravirtualized instances. If you want fully virtualized instances you
-may want to handle their kernel inside the instance, and to use VNC.
+.. admonition:: KVM
+
+ Instances are by default configured to use a host kernel, and to be
+ reached via serial console, which works nice for Linux paravirtualized
+ instances. If you want fully virtualized instances you may want to
+ handle their kernel inside the instance, and to use VNC.
+
+ Some versions of KVM have a bug that will make an instance hang when
+ configured to use the serial console (which is the default) unless a
+ connection is made to it within about 2 seconds of the instance's
+ startup. For such case it's recommended to disable the
+ ``serial_console`` option.
+
Joining the nodes to the cluster
++++++++++++++++++++++++++++++++
projects / ganeti-local / blobdiff