You can also use file-based storage only, without LVM, but this setup is
not detailed in this document.
+If you choose to use RBD-based instances, there's no need for LVM
+provisioning. However, this feature is experimental, and is not
+recommended for production clusters.
+
While you can use an existing system, please note that the Ganeti
installation is intrusive in terms of changes to the system
configuration, and it's best to use a newly-installed system without
but for Ganeti you need to have::
127.0.0.1 localhost
- 192.168.1.1 node1.example.com node1
+ 192.0.2.1 node1.example.com node1
- replacing ``192.168.1.1`` with your node's address. Also, the file
+ replacing ``192.0.2.1`` with your node's address. Also, the file
``/etc/hostname`` which configures the hostname of the system
should contain ``node1.example.com`` and not just ``node1`` (you
need to run the command ``/etc/init.d/hostname.sh start`` after
(xend-relocation-server yes)
(xend-relocation-port 8002)
(xend-relocation-address '')
- (xend-relocation-hosts-allow '^192\\.168\\.3\\.[0-9]+$')
+ (xend-relocation-hosts-allow '^192\\.0\\.2\\.[0-9]+$')
+
-The last line assumes that all your nodes have secondary IPs in the
-192.168.3.0/24 network, adjust it accordingly to your setup.
+The second line assumes that the hypervisor parameter
+``migration_port`` is set 8002, otherwise modify it to match. The last
+line assumes that all your nodes have secondary IPs in the
+192.0.2.0/24 network, adjust it accordingly to your setup.
.. admonition:: Debian
configure any drbd devices. You can do this by prefixing all
*resource* lines in the file with the keyword *skip*, like this::
- skip resource r0 {
- ...
+ skip {
+ resource r0 {
+ ...
+ }
}
- skip resource "r1" {
- ...
+ skip {
+ resource "r1" {
+ ...
+ }
}
-Other required software
-+++++++++++++++++++++++
+Installing RBD
++++++++++++++++
-Besides Xen and DRBD, you will need to install the following (on all
-nodes):
+Recommended on all nodes: RBD_ is required if you want to create
+instances with RBD disks residing inside a RADOS cluster (make use of
+the rbd disk template). RBD-based instances can failover or migrate to
+any other node in the ganeti cluster, enabling you to exploit of all
+Ganeti's high availabilily (HA) features.
-- LVM version 2, `<http://sourceware.org/lvm2/>`_
+.. attention::
+ Be careful though: rbd is still experimental! For now it is
+ recommended only for testing purposes. No sensitive data should be
+ stored there.
-- OpenSSL, `<http://www.openssl.org/>`_
+.. _RBD: http://ceph.newdream.net/
-- OpenSSH, `<http://www.openssh.com/portable.html>`_
+You will need the ``rbd`` and ``libceph`` kernel modules, the RBD/Ceph
+userspace utils (ceph-common Debian package) and an appropriate
+Ceph/RADOS configuration file on every VM-capable node.
-- bridge utilities, `<http://bridge.sourceforge.net/>`_
+You will also need a working RADOS Cluster accessible by the above
+nodes.
-- iproute2, `<http://developer.osdl.org/dev/iproute2>`_
+RADOS Cluster
+~~~~~~~~~~~~~
-- arping (part of iputils package),
- `<ftp://ftp.inr.ac.ru/ip-routing/iputils-current.tar.gz>`_
+You will need a working RADOS Cluster accesible by all VM-capable nodes
+to use the RBD template. For more information on setting up a RADOS
+Cluster, refer to the `official docs <http://ceph.newdream.net/>`_.
-- Python version 2.4 or 2.5, `<http://www.python.org>`_
+If you want to use a pool for storing RBD disk images other than the
+default (``rbd``), you should first create the pool in the RADOS
+Cluster, and then set the corresponding rbd disk parameter named
+``pool``.
-- Python OpenSSL bindings, `<http://pyopenssl.sourceforge.net/>`_
+Kernel Modules
+~~~~~~~~~~~~~~
-- simplejson Python module, `<http://www.undefined.org/python/#simplejson>`_
+Unless your distribution already provides it, you might need to compile
+the ``rbd`` and ``libceph`` modules from source. You will need Linux
+Kernel 3.2 or above for the kernel modules. Alternatively you will have
+to build them as external modules (from Linux Kernel source 3.2 or
+above), if you want to run a less recent kernel, or your kernel doesn't
+include them.
-- pyparsing Python module, `<http://pyparsing.wikispaces.com/>`_
+Userspace Utils
+~~~~~~~~~~~~~~~
-- pyinotify Python module, `<http://trac.dbzteam.org/pyinotify>`_
+The RBD template has been tested with ``ceph-common`` v0.38 and
+above. We recommend using the latest version of ``ceph-common``.
-These programs are supplied as part of most Linux distributions, so
-usually they can be installed via apt or similar methods. Also many of
-them will already be installed on a standard machine.
+.. admonition:: Debian
+ On Debian, you can just install the RBD/Ceph userspace utils with
+ the following command::
-.. admonition:: Debian
+ apt-get install ceph-common
+
+Configuration file
+~~~~~~~~~~~~~~~~~~
+
+You should also provide an appropriate configuration file
+(``ceph.conf``) in ``/etc/ceph``. For the rbd userspace utils, you'll
+only need to specify the IP addresses of the RADOS Cluster monitors.
+
+.. admonition:: ceph.conf
- You can use this command line to install all needed packages::
+ Sample configuration file::
- # apt-get install lvm2 ssh bridge-utils iproute iputils-arping \
- python python-pyopenssl openssl python-pyparsing \
- python-simplejson python-pyinotify
+ [mon.a]
+ host = example_monitor_host1
+ mon addr = 1.2.3.4:6789
+ [mon.b]
+ host = example_monitor_host2
+ mon addr = 1.2.3.5:6789
+ [mon.c]
+ host = example_monitor_host3
+ mon addr = 1.2.3.6:6789
+
+For more information, please see the `Ceph Docs
+<http://ceph.newdream.net/docs/latest/>`_
+
+Other required software
++++++++++++++++++++++++
+
+See :doc:`install-quick`.
Setting up the environment for Ganeti
-------------------------------------
KVM.
Beware that the default name Ganeti uses is ``xen-br0`` (which was used
-in Xen 2.0) while Xen 3.0 uses ``xenbr0`` by default. The default bridge
-your Ganeti cluster will use for new instances can be specified at
-cluster initialization time.
+in Xen 2.0) while Xen 3.0 uses ``xenbr0`` by default. See the
+`Initializing the cluster`_ section to learn how to choose a different
+bridge, or not to use one at all and use "routed mode".
-If you want to run in "routing mode" you need to specify that at cluster
-init time (using the --nicparam option), and then no bridge will be
-needed. In this mode instance traffic will be routed by dom0, instead of
-bridged.
-
-In order to use "routing mode" under Xen, you'll need to change the
+In order to use "routed mode" under Xen, you'll need to change the
relevant parameters in the Xen config file. Under KVM instead, no config
change is necessary, but you still need to set up your network
interfaces correctly.
installation script. An example OS that works under Debian and can
install Debian and Ubuntu instace OSes is provided on the project web
site. Download it from the project page and follow the instructions in
-the ``README`` file. Here is the installation procedure (replace 0.7
+the ``README`` file. Here is the installation procedure (replace 0.9
with the latest version that is compatible with your ganeti version)::
cd /usr/local/src/
- wget http://ganeti.googlecode.com/files/ganeti-instance-debootstrap-0.7.tar.gz
- tar xzf ganeti-instance-debootstrap-0.7.tar.gz
- cd ganeti-instance-debootstrap-0.7
+ wget http://ganeti.googlecode.com/files/ganeti-instance-debootstrap-0.9.tar.gz
+ tar xzf ganeti-instance-debootstrap-0.9.tar.gz
+ cd ganeti-instance-debootstrap-0.9
./configure
make
make install
apt-get install debootstrap dump kpartx
+.. admonition:: KVM
+
+ In order for debootstrap instances to be able to shutdown cleanly
+ they must install have basic acpi support inside the instance. Which
+ packages are needed depend on the exact flavor of debian or ubuntu
+ which you're installing, but the example defaults file has a
+ commented out configuration line that works for debian lenny and
+ squeeze::
+
+ EXTRA_PKGS="acpi-support-base,console-tools,udev"
+
+ kbd can be used instead of console-tools, and more packages can be
+ added, of course, if needed.
+
Alternatively, you can create your own OS definitions. See the manpage
:manpage:`ganeti-os-interface`.
(master) node.
If you want to use a bridge which is not ``xen-br0``, or no bridge at
-all, use ``--nicparams``.
+all, change it with the ``--nic-parameters`` option. For example to
+bridge on br0 you can say::
-If the bridge name you are using is not ``xen-br0``, use the *-b
-<BRIDGENAME>* option to specify the bridge name. In this case, you
-should also use the *--master-netdev <BRIDGENAME>* option with the same
-BRIDGENAME argument.
+ --nic-parameters link=br0
+
+Or to not bridge at all, and use a separate routing table::
+
+ --nic-parameters mode=routed,link=100
+
+If you don't have a xen-br0 interface you also have to specify a
+different network interface which will get the cluster ip, on the master
+node, by using the ``--master-netdev <device>`` option.
You can use a different name than ``xenvg`` for the volume group (but
note that the name must be identical on all nodes). In this case you
-need to specify it by passing the *-g <VGNAME>* option to ``gnt-cluster
-init``.
+need to specify it by passing the *--vg-name <VGNAME>* option to
+``gnt-cluster init``.
To set up the cluster as an Xen HVM cluster, use the
``--enabled-hypervisors=xen-hvm`` option to enable the HVM hypervisor
You can also invoke the command with the ``--help`` option in order to
see all the possibilities.
+Hypervisor/Network/Cluster parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Please note that the default hypervisor/network/cluster parameters may
+not be the correct one for your environment. Carefully check them, and
+change them at cluster init time, or later with ``gnt-cluster modify``.
+
+Your instance types, networking environment, hypervisor type and version
+may all affect what kind of parameters should be used on your cluster.
+
+For example kvm instances are by default configured to use a host
+kernel, and to be reached via serial console, which works nice for linux
+paravirtualized instances. If you want fully virtualized instances you
+may want to handle their kernel inside the instance, and to use VNC.
+
Joining the nodes to the cluster
++++++++++++++++++++++++++++++++
projects / ganeti-local / blobdiff