Synnefo

It describes how to install the whole Synnefo stack on two (2) physical nodes,

assumes the nodes run Debian Wheezy. After successful installation, you will

are "node1.example.com" and "node2.example.com" and their public IPs are "203.0.113.1" and

"203.0.113.2" respectively. It is important that the two machines are under the same domain name.

set up a private dns server, using dnsmasq for example. See node1 below for 

more information on how to do so.

| ``deb http://apt.dev.grnet.gr wheezy/``

| ``deb-src http://apt.dev.grnet.gr wheezy/``

Update your list of packages and continue with the installation:

.. code-block:: console

   # apt-get update

data inside this directory. By 'all data', we mean files, images, and Pithos

		* dnsmasq (DNS server)

To install gunicorn and gevent, run:

   # apt-get install gunicorn python-gevent

editting the file ``/etc/postgresql/9.1/main/postgresql.conf`` and change

Furthermore, edit ``/etc/postgresql/9.1/main/pg_hba.conf`` to allow node1 and

    host		all	all	203.0.113.1/32	md5

    host		all	all	203.0.113.2/32	md5

Make sure to substitute "203.0.113.1" and "203.0.113.2" with node1's and node2's

Node1 will host Cyclades. Cyclades should communicate with the other Synnefo 

Services and users over a secure channel. In order for the connection to be 

trusted, the keys provided to Apache below should be signed with a certificate.

and sign your keys with this. To do so on node1 run:

		# apt-get install openvpn

The previous will create a ``ca.crt`` file in the directory ``/etc/openvpn/easy-rsa/2.0/keys``.

Copy this file under ``/usr/local/share/ca-certificates/`` directory and run :

This will create a ``01.pem`` and a ``node1.example.com.key`` files in the 

``/etc/openvpn/easy-rsa/2.0/keys`` directory. Copy these in ``/etc/ssl/certs/``

and ``/etc/ssl/private/`` respectively and use them in the apache2 

configuration file below instead of the defaults.

As mentioned in the General Prerequisites section, there should be a directory 

called ``/srv/pithos`` visible by both nodes. We create and setup the ``data``

   # mkdir /srv/pithos

This directory must be shared via `NFS <https://en.wikipedia.org/wiki/Network_File_System>`_.

In order to do this, run:

.. code-block:: console

   # apt-get install rpcbind nfs-kernel-server

Now edit ``/etc/exports`` and add the following line:

.. code-block:: console

   /srv/pithos/ 203.0.113.2(rw,no_root_squash,sync,subtree_check)

Once done, run:

.. code-block:: console

   # /etc/init.d/nfs-kernel-server restart

If your machines are not under the same domain name you have to set up a dns server.

In order to set up a dns server using dnsmasq do the following:

   # apt-get install dnsmasq

Then edit your ``/etc/hosts/`` file as follows:

		203.0.113.1     node1.example.com

		203.0.113.2     node2.example.com

dnsmasq will serve any IPs/domains found in ``/etc/resolv.conf``.

There is a `"bug" in libevent 2.0.5 <http://sourceforge.net/p/levent/bugs/193/>`_

, where if you have multiple nameservers in your ``/etc/resolv.conf``, libevent

will round-robin against them. To avoid this, you must use a single nameserver

for all your needs. Edit your ``/etc/resolv.conf`` to include your dns server: 

   nameserver 203.0.113.1

Because of the aforementioned bug, you can't specify more than one DNS servers

in your ``/etc/resolv.conf``. In order for dnsmasq to serve domains not in 

``/etc/hosts``, edit ``/etc/dnsmasq.conf`` and change the line starting with 

``#resolv-file=`` to:

.. code-block:: console

   resolv-file=/etc/external-dns

Now create the file ``/etc/external-dns`` and specify any extra DNS servers you

want dnsmasq to query for domains, e.g., 8.8.8.8:

.. code-block:: console

   nameserver 8.8.8.8

In the ``/etc/dnsmasq.conf`` file, you can also specify the ``listen-address`` 

and the ``interface`` you would like dnsmasq to listen to.

Finally, restart dnsmasq:

.. code-block:: console

   # /etc/init.d/dnsmasq restart

    * dnsmasq (DNS server)

To install gunicorn and gevent, run:

   # apt-get install gunicorn python-gevent

``/usr/local/share/ca-certificate`` and run:

   # update-ca-certificates

   nameserver 203.0.113.1

to inform the node about the new DNS server.

As mentioned before, this should be the only ``nameserver`` entry in 

``/etc/resolv.conf``.

To install Astakos, grab the package from our repository (make sure  you made

the additions needed in your ``/etc/apt/sources.list`` file and updated, as 

described previously), by running:

Gunicorn setup

--------------

Copy the file ``/etc/gunicorn.d/synnefo.example`` to

``/etc/gunicorn.d/synnefo``, to make it a valid gunicorn configuration file:

.. code-block:: console

    # mv /etc/gunicorn.d/synnefo.example /etc/gunicorn.d/synnefo

.. warning:: Do NOT start the server yet, because it won't find the

    ``synnefo.settings`` module. Also, in case you are using ``/etc/hosts``

    instead of a DNS to get the hostnames, change ``--worker-class=gevent`` to

    ``--worker-class=sync``. We will start the server after successful

    installation of Astakos. If the server is running::

       # /etc/init.d/gunicorn stop

After Astakos is successfully installed, you will find the directory

After getting familiar with Synnefo, you will be able to customize the software

For the snf-webproject component (installed as an Astakos dependency), we

         'HOST': '203.0.113.1',

For Astakos specific configuration, edit the following options in

services). ``ASTAKOS_BASE_URL`` is the Astakos top-level URL. Appending an

Many of the ``Astakos`` operations require the server to notify service users 

and administrators via email. e.g. right after the signup process, the service 

sents an email to the registered email address containing an verification url. 

After the user verifies the email address, Astakos once again needs to 

notify administrators with a notice that a new account has just been verified.

More specifically Astakos sends emails in the following cases

- Feedback messages submited from Astakos contact view and Astakos feedback

installation.

Edit ``/etc/synnefo/00-snf-common-admins.conf``:

    EMAIL_HOST = "mysmtp.server.example.com"

    EMAIL_SUBJECT_PREFIX = "[example.com] "

    DEFAULT_FROM_EMAIL = "server@example.com"

    CONTACT_EMAIL = "server@example.com"

    SERVER_EMAIL = "server-errors@example.com"

Astakos, they are defined in a different configuration file than the one

previously used to set Astakos specific settings.

an email notification, the recipients list differs. Specifically, for

    ADMINS = (('Admin name', 'admin@example.com'),

              ('Admin2 name', 'admin2@example.com))

    MANAGERS = (('Manager name', 'manager@example.com'),)

    HELPDESK = (('Helpdesk user name', 'helpdesk@example.com'),)

         'HOST': '203.0.113.1',

for Astakos:

command will ask you to register the standard Synnefo components (Astakos,

Cyclades and Pithos) along with the services they provide. Note that you

have to register at least Astakos in order to have a usable authentication

settings. For example, the base URL for Astakos would be

   it registers the three components in Astakos and then in each host it

   to the Astakos host, where it finally imports it:

Notice that in this installation astakos and cyclades are in node1 and pithos is in node2.

your browser again. Try to sign in using your new credentials. If the Astakos

This package provides the standalone Pithos web client. The web client is the

web UI for Pithos and will be accessible by clicking "Pithos" on the Astakos

Gunicorn setup

--------------

Copy the file ``/etc/gunicorn.d/synnefo.example`` to

``/etc/gunicorn.d/synnefo``, to make it a valid gunicorn configuration file

(as happened for node1):

.. code-block:: console

    # cp /etc/gunicorn.d/synnefo.example /etc/gunicorn.d/synnefo

.. warning:: Do NOT start the server yet, because it won't find the

    ``synnefo.settings`` module. Also, in case you are using ``/etc/hosts``

    instead of a DNS to get the hostnames, change ``--worker-class=gevent`` to

    ``--worker-class=sync``. We will start the server after successful

    installation of Astakos. If the server is running::

       # /etc/init.d/gunicorn stop

after installation of Astakos. Here, you will not have to change anything that

The ``PITHOS_SERVICE_TOKEN`` is the token used for authentication with Astakos.

Pithos web UI with the Astakos web UI (through the top cloudbar):

The ``CLOUDBAR_LOCATION`` tells the client where to find the Astakos common

Pithos web client to get from Astakos all the information needed to fill its

own cloudbar. So we put our Astakos deployment urls there.

doesn't use a Django DB. It pools HTTP connections to Astakos and Pithos

Mount the NFS directory

-----------------------

First install the package nfs-common by running:

.. code-block:: console

   root@node2:~ # apt-get install nfs-common

now create the directory /srv/pithos/ and mount the remote directory to it:

.. code-block:: console

   root@node2:~ # mkdir /srv/pithos/

   root@node2:~ # mount -t nfs 203.0.113.1:/srv/pithos/ /srv/pithos/

Login, and you will see your profile page. Now, click the "Pithos" link on the

``https://node2.example.com/ui``

Kamaki

======

`Kamaki <http://www.synnefo.org/docs/kamaki/latest/index.html>`_ is an 

Openstack API client library and command line interface with custom extentions 

specific to Synnefo.

Kamaki Installation and Configuration

-------------------------------------

To install kamaki run:

.. code-block:: console

   # apt-get install kamaki

Now, visit 

 `https://node1.example.com/astakos/ui/`

log in and click on ``API access``. Scroll all the way to the bottom of the 

page, click on the orange ``Download your .kamakirc`` button and save the file

as ``.kamakirc`` in your home directory.

That's all, kamaki is now configured and you can start using it. For a list of

commands, see the `official documentantion <http://www.synnefo.org/docs/kamaki/latest/commands.html>`_.

Please refer to the `ganeti documentation <http://docs.ganeti.org/ganeti/2.8/html>`_ for all 

the gory details. A successful Ganeti installation concludes with a working

Ganeti Prerequisites

--------------------

You're gonna need the ``lvm2`` and ``vlan`` packages, so run:

   # apt-get install lvm2 vlan

Ganeti requires FQDN. To properly configure your nodes please

see `this <http://docs.ganeti.org/ganeti/2.6/html/install.html#hostname-issues>`_.

Ganeti requires an extra available IP and its FQDN e.g., ``203.0.113.100`` and 

``ganeti.node1.example.com``. Add this IP to your DNS server configuration, as 

explained above.

Also, Ganeti will need a volume group with the same name e.g., ``ganeti`` 

across all nodes, of at least 20GiB. To create the volume group, 

see `this <http://www.tldp.org/HOWTO/LVM-HOWTO/createvgs.html>`_.

Moreover, node1 and node2 must have the same dsa, rsa keys and authorised_keys

under ``/root/.ssh/`` for password-less root ssh between each other. To 

generate said keys, see `this <https://wiki.debian.org/SSH#Using_shared_keys>`_.

In the following sections, we assume that the public interface of all nodes is

``eth0`` and there are two extra interfaces ``eth1`` and ``eth2``, which can 

also be vlans on your primary interface e.g., ``eth0.1`` and ``eth0.2``  in 

case you don't have multiple physical interfaces. For information on how to 

create vlans, please see

`this <https://wiki.debian.org/NetworkConfiguration#Howto_use_vlan_.28dot1q.2C_802.1q.2C_trunk.29_.28Etch.2C_Lenny.29>`_.

Finally, setup two bridges on the host machines (e.g: br1/br2 on eth1/eth2 

respectively), as described `here <https://wiki.debian.org/BridgeNetworkConnections>`_.

Ganeti Installation and Initialization

--------------------------------------

We assume that Ganeti will use the KVM hypervisor. To install KVM, run on all 

Ganeti nodes:

   # apt-get install qemu-kvm

It's time to install Ganeti. To be able to use hotplug (which will be part of 

the official Ganeti 2.10), we recommend using our Ganeti package version:

`2.8.2+snapshot1+b64v1+hotplug5+ippoolfix+rapifix+netxen+lockfix2-1~wheezy`

Let's briefly explain each patch:

    * hotplug: hotplug devices (NICs and Disks) (ganeti 2.10)

    * b64v1: Save bitarray of network IP pools in config file, encoded in base64, instead of 0/1.

    * ippoolfix: Ability to give an externally reserved IP to an instance (e.g. gateway IP).  (ganeti 2.10)

    * rapifix: Extend RAPI το support 'depends' and 'shutdown_timeout' body arguments. (ganeti 2.9)

    * netxen: Network configuration for xen instances, exactly like in kvm instances. (ganeti 2.9)

    * lockfix2: Fixes for 2 locking issues:

      - Issue 622: Fix for opportunistic locking that caused an assertion error (Patch waiting in ganeti-devel list)

      - Issue 621: Fix for network locking issue that resulted in: [Lock 'XXXXXX' not found in set 'instance' (it may have been removed)]

    * snapshot: Add trivial 'snapshot' functionality that is unused by Synnefo or Ganeti.

To install Ganeti run:

.. code-block:: console

   # apt-get install snf-ganeti ganeti-htools ganeti-haskell

Ganeti will make use of drbd. To enable this and make the configuration 

permanent you have to do the following :

.. code-block:: console

   # modprobe drbd minor_count=255 usermode_helper=/bin/true

   # echo 'drbd minor_count=255 usermode_helper=/bin/true' >> /etc/modules

                    --no-etc-hosts --vg-name=ganeti --nic-parameters link=br1 \

                    --default-iallocator hail \

                    --hypervisor-parameters kvm:kernel_path=,vnc_bind_address=0.0.0.0 \

``br1`` will be the default interface for any newly created VMs.

you need the `snf-image <http://www.synnefo.org/docs/snf-image/latest/index.html>`_ OS

		directions about the certificates, in order to circumvent this you should edit the file

To do this, we need to set the corresponding variable in

To upload the file using kamaki, run:

.. code-block:: console

   # kamaki file upload debian_base-6.0-x86_64.diskdump pithos

definition installed on all VM-capable nodes, a Debian Squeeze Image on

Pithos and kamaki installed and configured. Make sure you also have the 

`metadata file <http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump.meta>`_ 

for this image.

To spawn a VM from a Pithos file, we need to know:

    1) The hashmap of the file

    2) The size of the file

If you uploaded the file with kamaki as described above, run:

.. code-block:: console

   # kamaki file info pithos:debian_base-6.0-x86_64.diskdump 

else, replace ``pithos`` and ``debian_base-6.0-x86_64.diskdump`` with the 

container and filename you used, when uploading the file.

The hashmap is the field ``x-object-hash``, while the size of the file is the

``content-length`` field, that ``kamaki file info`` command returns.

                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithosmap://<HashMap>/<Size>",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \

               should have the format ``pithosmap://<HashMap>/<size>``:

               * ``HashMap``: the map of the file

               * ``size``: the size of the file, same size as reported in 

                 ``ls -la filename``

needs of the system administrator.

In this section, we'll describe the simplest scenario, which will provide

access to the public Internet along with private networking capabilities for 

the VMs.

snf-network is a set of custom scripts, that perform all the necessary actions,

so that VMs have a working networking configuration.

nfdhcpd is an NFQUEUE based daemon, answering DHCP requests and running locally 

on every Ganeti node. Its leases file, gets automatically updated by 

snf-network and information provided by Ganeti.

   # apt-get install python-nfqueue=0.4+physindev-1~wheezy

variable to your DNS IP/s (the one running dnsmasq for instance or you can use

Google's DNS server ``8.8.8.8``). Restart the server on all nodes:

In order for nfdhcpd to receive the VMs requests, we have to mangle all DHCP 

traffic coming from the corresponding interfaces. To accomplish that run:

and append it to your ``/etc/rc.local``.

In the following section, we'll guide you through a very basic network setup.

This assumes the following:

    * Node1 has access to the public network via eth0.

    * Node1 will become a NAT server for the VMs.

    * All nodes have ``br1/br2`` dedicated for the VMs' public/private traffic.

    * VMs' public network is ``10.0.0.0/24`` with gateway ``10.0.0.1``.

Setting up the NAT server on node1

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To setup the NAT server on node1, run:

   # ip addr add 10.0.0.1/24 dev br1

   # iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

   # echo 1 > /proc/sys/net/ipv4/ip_forward

and append it to your ``/etc/rc.local``.

Testing the Public Networks

~~~~~~~~~~~~~~~~~~~~~~~~~~~

First add the network in Ganati:

   # gnt-network add --network=10.0.0.0/24 --gateway=10.0.0.1 --tags=nfdhcpd test-net-public

Then, provide connectivity mode and link to the network:

   # gnt-network connect test-net-public bridged br1

the Public Network. We will add a new VM, almost the same way we did it on the

previous testing section. However, now we'll also add one NIC, configured to be

managed from our previously defined network.

Fetch the Debian Old Base image locally (in all nodes), by running:

.. code-block:: console

   # wget http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump -O /var/lib/snf-image/debian_base-6.0-x86_64.diskdump

Also in all nodes, bring all ``br*`` interfaces up:

   # ifconfig br1 up

   # ifconfig br2 up

Finally, run on the GANETI-MASTER (node1):

.. code-block:: console

                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id=debian_base-6.0-x86_64,img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \

The following things should happen:

    * Ganeti creates a tap interface.

    * snf-network bridges the tap interface to ``br1`` and updates nfdhcpd state.

    * nfdhcpd serves 10.0.0.2 IP to the interface of ``testvm2``.

Now try to ping the outside world e.g., ``www.synnefo.org`` from inside the VM 

(connect to the VM using VNC as before).

In this section, we'll describe a basic network configuration, that will provide 

isolated private networks to the end-users. All private network traffic, will 

pass through ``br1`` and isolation will be guaranteed with a specific set of 

``ebtables`` rules.

We'll create two instances and connect them to the same Private Network. This 

means that the instances will have a second NIC connected to the ``br1``.

   # gnt-network add --network=192.168.1.0/24 --mac-prefix=aa:00:55 --tags=nfdhcpd,private-filtered test-net-prv-mac

   # gnt-network connect test-net-prv-mac bridged br1

                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id=debian_base-6.0-x86_64,img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \

                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id=debian_base-6.0-x86_64,img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \

                      --net 1:ip=pool,network=test-net-prv-mac -n node2 \

Above, we create two instances with the first NIC connected to the internet and

their second NIC connected to a MAC filtered private Network. Now, connect to the

    interface (``eth0``), which has been automatically assigned a "public" IP.

 b) ``eth1`` will have mac prefix ``aa:00:55``

 c) On testvm3  ping 192.168.1.2

The ``CYCLADES_SERVICE_TOKEN`` is the token used for authentication with Astakos.

Cyclades assuming that you have setup the :ref:`Rapi User <rapi-user>`

created NIC on a bridge. 

   $ snf-manage network-create --subnet=10.0.0.0/24 \

                               --gateway=10.0.0.1 \

                               --public --dhcp --flavor=CUSTOM \

                               --link=br1 --mode=bridged \

   # snf-manage reconcile-networks

You can use ``snf-manage reconcile-networks --fix-all`` to fix any

inconsistencies that may have arisen.

   # snf-manage network-list

   # snf-manage network-inspect <net_id>

   # gnt-network list

   # gnt-network info <network_name>

   # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536

Also, change the Synnefo setting in :file:`/etc/synnefo/20-snf-cyclades-app-api.conf`:

   DEFAULT_MAC_FILTERED_BRIDGE = 'br2'

browser and go to the Astakos home page. Login and then click 'Cyclades' on the