Synnefo

.. _quick-install-admin-guide:

Administrator's Quick Installation Guide

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is the Administrator's quick installation guide.

It describes how to install the whole synnefo stack on two (2) physical nodes,

with minimum configuration. It installs synnefo from Debian packages, and

assumes the nodes run Debian Squeeze. After successful installation, you will

have the following services running:

    * Identity Management (Astakos)

    * Object Storage Service (Pithos)

    * Compute Service (Cyclades)

    * Image Service (part of Cyclades)

    * Network Service (part of Cyclades)

and a single unified Web UI to manage them all.

The Volume Storage Service (Archipelago) and the Billing Service (Aquarium) are

not released yet.

If you just want to install the Object Storage Service (Pithos), follow the

guide and just stop after the "Testing of Pithos" section.

Installation of Synnefo / Introduction

======================================

We will install the services with the above list's order. The last three

services will be installed in a single step (at the end), because at the moment

they are contained in the same software component (Cyclades). Furthermore, we

will install all services in the first physical node, except Pithos which will

be installed in the second, due to a conflict between the snf-pithos-app and

snf-cyclades-app component (scheduled to be fixed in the next version).

For the rest of the documentation we will refer to the first physical node as

"node1" and the second as "node2". We will also assume that their domain names

are "node1.example.com" and "node2.example.com" and their IPs are "4.3.2.1" and

"4.3.2.2" respectively.

.. note:: It is import that the two machines are under the same domain name.

    If they are not, you can do this by editting the file ``/etc/hosts``

    on both machines, and add the following lines:

    .. code-block:: console

        4.3.2.1     node1.example.com

        4.3.2.2     node2.example.com

General Prerequisites

=====================

These are the general synnefo prerequisites, that you need on node1 and node2

and are related to all the services (Astakos, Pithos, Cyclades).

To be able to download all synnefo components you need to add the following

lines in your ``/etc/apt/sources.list`` file:

| ``deb http://apt2.dev.grnet.gr stable/``

| ``deb-src http://apt2.dev.grnet.gr stable/``

and import the repo's GPG key:

| ``curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -``

Also add the following line to enable the ``squeeze-backports`` repository,

which may provide more recent versions of certain packages. The repository

is deactivated by default and must be specified expicitly in ``apt-get``

operations:

| ``deb http://backports.debian.org/debian-backports squeeze-backports main``

You also need a shared directory visible by both nodes. Pithos will save all

data inside this directory. By 'all data', we mean files, images, and pithos

specific mapping data. If you plan to upload more than one basic image, this

directory should have at least 50GB of free space. During this guide, we will

assume that node1 acts as an NFS server and serves the directory ``/srv/pithos``

to node2 (be sure to set no_root_squash flag). Node2 has this directory

mounted under ``/srv/pithos``, too.

Before starting the synnefo installation, you will need basic third party

software to be installed and configured on the physical nodes. We will describe

each node's general prerequisites separately. Any additional configuration,

specific to a synnefo service for each node, will be described at the service's

section.

Finally, it is required for Cyclades and Ganeti nodes to have synchronized

system clocks (e.g. by running ntpd).

Node1

-----

General Synnefo dependencies

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    * apache (http server)

    * gunicorn (WSGI http server)

    * postgresql (database)

    * rabbitmq (message queue)

    * ntp (NTP daemon)

    * gevent

You can install apache2, progresql and ntp by running:

.. code-block:: console

   # apt-get install apache2 postgresql ntp

Make sure to install gunicorn >= v0.12.2. You can do this by installing from

the official debian backports:

.. code-block:: console

   # apt-get -t squeeze-backports install gunicorn

Also, make sure to install gevent >= 0.13.6. Again from the debian backports:

.. code-block:: console

   # apt-get -t squeeze-backports install python-gevent

On node1, we will create our databases, so you will also need the

python-psycopg2 package:

.. code-block:: console

   # apt-get install python-psycopg2

To install RabbitMQ>=2.8.4, use the RabbitMQ APT repository by adding the

following line to ``/etc/apt/sources.list``:

.. code-block:: console

    deb http://www.rabbitmq.com/debian testing main

Add RabbitMQ public key, to trusted key list:

.. code-block:: console

  # wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc

  # apt-key add rabbitmq-signing-key-public.asc

Finally, to install the package run:

.. code-block:: console

  # apt-get update

  # apt-get install rabbitmq-server

Database setup

~~~~~~~~~~~~~~

On node1, we create a database called ``snf_apps``, that will host all django

apps related tables. We also create the user ``synnefo`` and grant him all

privileges on the database. We do this by running:

.. code-block:: console

    root@node1:~ # su - postgres

    postgres@node1:~ $ psql

    postgres=# CREATE DATABASE snf_apps WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;

    postgres=# CREATE USER synnefo WITH PASSWORD 'example_passw0rd';

    postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_apps TO synnefo;

We also create the database ``snf_pithos`` needed by the Pithos backend and

grant the ``synnefo`` user all privileges on the database. This database could

be created on node2 instead, but we do it on node1 for simplicity. We will

create all needed databases on node1 and then node2 will connect to them.

.. code-block:: console

    postgres=# CREATE DATABASE snf_pithos WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;

    postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_pithos TO synnefo;

Configure the database to listen to all network interfaces. You can do this by

editting the file ``/etc/postgresql/8.4/main/postgresql.conf`` and change

``listen_addresses`` to ``'*'`` :

.. code-block:: console

    listen_addresses = '*'

Furthermore, edit ``/etc/postgresql/8.4/main/pg_hba.conf`` to allow node1 and

node2 to connect to the database. Add the following lines under ``#IPv4 local

connections:`` :

.. code-block:: console

    host		all	all	4.3.2.1/32	md5

    host		all	all	4.3.2.2/32	md5

Make sure to substitute "4.3.2.1" and "4.3.2.2" with node1's and node2's

actual IPs. Now, restart the server to apply the changes:

.. code-block:: console

   # /etc/init.d/postgresql restart

Gunicorn setup

~~~~~~~~~~~~~~

Create the file ``/etc/gunicorn.d/synnefo`` containing the following:

.. code-block:: console

    CONFIG = {

     'mode': 'django',

     'environment': {

       'DJANGO_SETTINGS_MODULE': 'synnefo.settings',

     },

     'working_dir': '/etc/synnefo',

     'user': 'www-data',

     'group': 'www-data',

     'args': (

       '--bind=127.0.0.1:8080',

       '--worker-class=gevent',

       '--workers=8',

       '--log-level=debug',

     ),

    }

.. warning:: Do NOT start the server yet, because it won't find the

    ``synnefo.settings`` module. Also, in case you are using ``/etc/hosts``

    instead of a DNS to get the hostnames, change ``--worker-class=gevent`` to

    ``--worker-class=sync``. We will start the server after successful

    installation of astakos. If the server is running::

       # /etc/init.d/gunicorn stop

Apache2 setup

~~~~~~~~~~~~~

Create the file ``/etc/apache2/sites-available/synnefo`` containing the

following:

.. code-block:: console

    <VirtualHost *:80>

        ServerName node1.example.com

        RewriteEngine On

        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]

        RewriteRule ^(.*)$ - [F,L]

        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

    </VirtualHost>

Create the file ``/etc/apache2/sites-available/synnefo-ssl`` containing the

following:

.. code-block:: console

    <IfModule mod_ssl.c>

    <VirtualHost _default_:443>

        ServerName node1.example.com

        Alias /static "/usr/share/synnefo/static"

        #  SetEnv no-gzip

        #  SetEnv dont-vary

       AllowEncodedSlashes On

       RequestHeader set X-Forwarded-Protocol "https"

    <Proxy * >

        Order allow,deny

        Allow from all

    </Proxy>

        SetEnv                proxy-sendchunked

        SSLProxyEngine        off

        ProxyErrorOverride    off

        ProxyPass        /static !

        ProxyPass        / http://localhost:8080/ retry=0

        ProxyPassReverse / http://localhost:8080/

        RewriteEngine On

        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]

        RewriteRule ^(.*)$ - [F,L]

        SSLEngine on

        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem

        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

    </VirtualHost>

    </IfModule>

Now enable sites and modules by running:

.. code-block:: console

   # a2enmod ssl

   # a2enmod rewrite

   # a2dissite default

   # a2ensite synnefo

   # a2ensite synnefo-ssl

   # a2enmod headers

   # a2enmod proxy_http

.. warning:: Do NOT start/restart the server yet. If the server is running::

       # /etc/init.d/apache2 stop

.. _rabbitmq-setup:

Message Queue setup

~~~~~~~~~~~~~~~~~~~

The message queue will run on node1, so we need to create the appropriate

rabbitmq user. The user is named ``synnefo`` and gets full privileges on all

exchanges:

.. code-block:: console

   # rabbitmqctl add_user synnefo "example_rabbitmq_passw0rd"

   # rabbitmqctl set_permissions synnefo ".*" ".*" ".*"

We do not need to initialize the exchanges. This will be done automatically,

during the Cyclades setup.

Pithos data directory setup

~~~~~~~~~~~~~~~~~~~~~~~~~~~

As mentioned in the General Prerequisites section, there is a directory called

``/srv/pithos`` visible by both nodes. We create and setup the ``data``

directory inside it:

.. code-block:: console

   # cd /srv/pithos

   # mkdir data

   # chown www-data:www-data data

   # chmod g+ws data

You are now ready with all general prerequisites concerning node1. Let's go to

node2.

Node2

-----

General Synnefo dependencies

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    * apache (http server)

    * gunicorn (WSGI http server)

    * postgresql (database)

    * ntp (NTP daemon)

    * gevent

You can install the above by running:

.. code-block:: console

   # apt-get install apache2 postgresql ntp

Make sure to install gunicorn >= v0.12.2. You can do this by installing from

the official debian backports:

.. code-block:: console

   # apt-get -t squeeze-backports install gunicorn

Also, make sure to install gevent >= 0.13.6. Again from the debian backports:

.. code-block:: console

   # apt-get -t squeeze-backports install python-gevent

Node2 will connect to the databases on node1, so you will also need the

python-psycopg2 package:

.. code-block:: console

   # apt-get install python-psycopg2

Database setup

~~~~~~~~~~~~~~

All databases have been created and setup on node1, so we do not need to take

any action here. From node2, we will just connect to them. When you get familiar

with the software you may choose to run different databases on different nodes,

for performance/scalability/redundancy reasons, but those kind of setups are out

of the purpose of this guide.

Gunicorn setup

~~~~~~~~~~~~~~

Create the file ``/etc/gunicorn.d/synnefo`` containing the following

(same contents as in node1; you can just copy/paste the file):

.. code-block:: console

    CONFIG = {

     'mode': 'django',

     'environment': {

      'DJANGO_SETTINGS_MODULE': 'synnefo.settings',

     },

     'working_dir': '/etc/synnefo',

     'user': 'www-data',

     'group': 'www-data',

     'args': (

       '--bind=127.0.0.1:8080',

       '--worker-class=gevent',

       '--workers=4',

       '--log-level=debug',

       '--timeout=43200'

     ),

    }

.. warning:: Do NOT start the server yet, because it won't find the

    ``synnefo.settings`` module. Also, in case you are using ``/etc/hosts``

    instead of a DNS to get the hostnames, change ``--worker-class=gevent`` to

    ``--worker-class=sync``. We will start the server after successful

    installation of astakos. If the server is running::

       # /etc/init.d/gunicorn stop

Apache2 setup

~~~~~~~~~~~~~

Create the file ``/etc/apache2/sites-available/synnefo`` containing the

following:

.. code-block:: console

    <VirtualHost *:80>

        ServerName node2.example.com

        RewriteEngine On

        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]

        RewriteRule ^(.*)$ - [F,L]

        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

    </VirtualHost>

Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``

containing the following:

.. code-block:: console

    <IfModule mod_ssl.c>

    <VirtualHost _default_:443>

        ServerName node2.example.com

        Alias /static "/usr/share/synnefo/static"

        SetEnv no-gzip

        SetEnv dont-vary

        AllowEncodedSlashes On

        RequestHeader set X-Forwarded-Protocol "https"

        <Proxy * >

            Order allow,deny

            Allow from all

        </Proxy>

        SetEnv                proxy-sendchunked

        SSLProxyEngine        off

        ProxyErrorOverride    off

        ProxyPass        /static !

        ProxyPass        / http://localhost:8080/ retry=0

        ProxyPassReverse / http://localhost:8080/

        SSLEngine on

        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem

        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

    </VirtualHost>

    </IfModule>

As in node1, enable sites and modules by running:

.. code-block:: console

   # a2enmod ssl

   # a2enmod rewrite

   # a2dissite default

   # a2ensite synnefo

   # a2ensite synnefo-ssl

   # a2enmod headers

   # a2enmod proxy_http

.. warning:: Do NOT start/restart the server yet. If the server is running::

       # /etc/init.d/apache2 stop

We are now ready with all general prerequisites for node2. Now that we have

finished with all general prerequisites for both nodes, we can start installing

the services. First, let's install Astakos on node1.

Installation of Astakos on node1

================================

To install astakos, grab the package from our repository (make sure  you made

the additions needed in your ``/etc/apt/sources.list`` file, as described

previously), by running:

.. code-block:: console

   # apt-get install snf-astakos-app snf-pithos-backend

After successful installation of snf-astakos-app, make sure that also

snf-webproject has been installed (marked as "Recommended" package). By default

Debian installs "Recommended" packages, but if you have changed your

configuration and the package didn't install automatically, you should

explicitly install it manually running:

.. code-block:: console

   # apt-get install snf-webproject

The reason snf-webproject is "Recommended" and not a hard dependency, is to give

the experienced administrator the ability to install Synnefo in a custom made

`Django <https://www.djangoproject.com/>`_ project. This corner case

concerns only very advanced users that know what they are doing and want to

experiment with synnefo.

.. _conf-astakos:

Configuration of Astakos

========================

Conf Files

----------

After astakos is successfully installed, you will find the directory

``/etc/synnefo`` and some configuration files inside it. The files contain

commented configuration options, which are the default options. While installing

new snf-* components, new configuration files will appear inside the directory.

In this guide (and for all services), we will edit only the minimum necessary

configuration options, to reflect our setup. Everything else will remain as is.

After getting familiar with synnefo, you will be able to customize the software

as you wish and fits your needs. Many options are available, to empower the

administrator with extensively customizable setups.

For the snf-webproject component (installed as an astakos dependency), we

need the following:

Edit ``/etc/synnefo/10-snf-webproject-database.conf``. You will need to

uncomment and edit the ``DATABASES`` block to reflect our database:

.. code-block:: console

    DATABASES = {

     'default': {

         # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'

         'ENGINE': 'postgresql_psycopg2',

         # ATTENTION: This *must* be the absolute path if using sqlite3.

         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name

         'NAME': 'snf_apps',

         'USER': 'synnefo',                      # Not used with sqlite3.

         'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.

         # Set to empty string for localhost. Not used with sqlite3.

         'HOST': '4.3.2.1',

         # Set to empty string for default. Not used with sqlite3.

         'PORT': '5432',

     }

    }

Edit ``/etc/synnefo/10-snf-webproject-deploy.conf``. Uncomment and edit

``SECRET_KEY``. This is a Django specific setting which is used to provide a

seed in secret-key hashing algorithms. Set this to a random string of your

choise and keep it private:

.. code-block:: console

    SECRET_KEY = 'sy6)mw6a7x%n)-example_secret_key#zzk4jo6f2=uqu!1o%)'

For astakos specific configuration, edit the following options in

``/etc/synnefo/20-snf-astakos-app-settings.conf`` :

.. code-block:: console

    ASTAKOS_DEFAULT_ADMIN_EMAIL = None

    ASTAKOS_COOKIE_DOMAIN = '.example.com'

    ASTAKOS_BASE_URL = 'https://node1.example.com'

The ``ASTAKOS_COOKIE_DOMAIN`` should be the base url of our domain (for all

services). ``ASTAKOS_BASE_URL`` is the astakos top-level URL.

``ASTAKOS_DEFAULT_ADMIN_EMAIL`` refers to the administrator's email.

Every time a new account is created a notification is sent to this email.

For this we need access to a running mail server, so we have disabled

it for now by setting its value to None. For more informations on this,

read the relative :ref:`section <mail-server>`.

.. note:: For the purpose of this guide, we don't enable recaptcha authentication.

    If you would like to enable it, you have to edit the following options:

    .. code-block:: console

        ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('

        ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('

        ASTAKOS_RECAPTCHA_USE_SSL = True

        ASTAKOS_RECAPTCHA_ENABLED = True

    For the ``ASTAKOS_RECAPTCHA_PUBLIC_KEY`` and ``ASTAKOS_RECAPTCHA_PRIVATE_KEY``

    go to https://www.google.com/recaptcha/admin/create and create your own pair.

Then edit ``/etc/synnefo/20-snf-astakos-app-cloudbar.conf`` :

.. code-block:: console

    CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'

    CLOUDBAR_SERVICES_URL = 'https://node1.example.com/ui/get_services'

    CLOUDBAR_MENU_URL = 'https://node1.example.com/ui/get_menu'

Those settings have to do with the black cloudbar endpoints and will be

described in more detail later on in this guide. For now, just edit the domain

to point at node1 which is where we have installed Astakos.

If you are an advanced user and want to use the Shibboleth Authentication

method, read the relative :ref:`section <shibboleth-auth>`.

.. note:: Because Cyclades and Astakos are running on the same machine

    in our example, we have to deactivate the CSRF verification. We can do so

    by adding to

    ``/etc/synnefo/99-local.conf``:

    .. code-block:: console

        MIDDLEWARE_CLASSES.remove('django.middleware.csrf.CsrfViewMiddleware')

        TEMPLATE_CONTEXT_PROCESSORS.remove('django.core.context_processors.csrf')

Enable Pooling

--------------

This section can be bypassed, but we strongly recommend you apply the following,

since they result in a significant performance boost.

Synnefo includes a pooling DBAPI driver for PostgreSQL, as a thin wrapper

around Psycopg2. This allows independent Django requests to reuse pooled DB

connections, with significant performance gains.

To use, first monkey-patch psycopg2. For Django, run this before the

``DATABASES`` setting in ``/etc/synnefo/10-snf-webproject-database.conf``:

.. code-block:: console

    from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2

    monkey_patch_psycopg2()

Since we are running with greenlets, we should modify psycopg2 behavior, so it

works properly in a greenlet context:

.. code-block:: console

    from synnefo.lib.db.psyco_gevent import make_psycopg_green

    make_psycopg_green()

Use the Psycopg2 driver as usual. For Django, this means using

``django.db.backends.postgresql_psycopg2`` without any modifications. To enable

connection pooling, pass a nonzero ``synnefo_poolsize`` option to the DBAPI

driver, through ``DATABASES.OPTIONS`` in Django.

All the above will result in an ``/etc/synnefo/10-snf-webproject-database.conf``

file that looks like this:

.. code-block:: console

    # Monkey-patch psycopg2

    from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2

    monkey_patch_psycopg2()

    # If running with greenlets

    from synnefo.lib.db.psyco_gevent import make_psycopg_green

    make_psycopg_green()

    DATABASES = {

     'default': {

         # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'

         'ENGINE': 'postgresql_psycopg2',

         'OPTIONS': {'synnefo_poolsize': 8},

         # ATTENTION: This *must* be the absolute path if using sqlite3.

         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name

         'NAME': 'snf_apps',

         'USER': 'synnefo',                      # Not used with sqlite3.

         'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.

         # Set to empty string for localhost. Not used with sqlite3.

         'HOST': '4.3.2.1',

         # Set to empty string for default. Not used with sqlite3.

         'PORT': '5432',

     }

    }

Database Initialization

-----------------------

After configuration is done, we initialize the database by running:

.. code-block:: console

    # snf-manage syncdb

At this example we don't need to create a django superuser, so we select

``[no]`` to the question. After a successful sync, we run the migration needed

for astakos:

.. code-block:: console

    # snf-manage migrate im

    # snf-manage migrate quotaholder_app

Then, we load the pre-defined user groups

.. code-block:: console

    # snf-manage loaddata groups

.. _services-reg:

Services Registration

---------------------

When the database is ready, we need to register the services. The following

command will ask you to register the standard Synnefo services (astakos,

cyclades, and pithos). Note that you have to register at least astakos in

order to have a usable authentication system. For each service, you will be

asked to provide the service URL (to appear in the Cloudbar) as well as its

API URL. Moreover, the command will automatically register the resource

definitions offered by the respective service.

.. code-block:: console

    # snf-register-services

Setting Default Base Quota for Resources

----------------------------------------

We now have to specify the limit on resources that each user can employ

(exempting resources offered by projects).

.. code-block:: console

    # snf-manage resource-modify --limit-interactive

Servers Initialization

----------------------

Finally, we initialize the servers on node1:

.. code-block:: console

    root@node1:~ # /etc/init.d/gunicorn restart

    root@node1:~ # /etc/init.d/apache2 restart

We have now finished the Astakos setup. Let's test it now.

Testing of Astakos

==================

Open your favorite browser and go to:

``http://node1.example.com/im``

If this redirects you to ``https://node1.example.com/ui/`` and you can see

the "welcome" door of Astakos, then you have successfully setup Astakos.

Let's create our first user. At the homepage click the "CREATE ACCOUNT" button

and fill all your data at the sign up form. Then click "SUBMIT". You should now

see a green box on the top, which informs you that you made a successful request

and the request has been sent to the administrators. So far so good, let's

assume that you created the user with username ``user@example.com``.

Now we need to activate that user. Return to a command prompt at node1 and run:

.. code-block:: console

    root@node1:~ # snf-manage user-list

This command should show you a list with only one user; the one we just created.

This user should have an id with a value of ``1``. It should also have an

"active" status with the value of ``0`` (inactive). Now run:

.. code-block:: console

    root@node1:~ # snf-manage user-update --set-active 1

This modifies the active value to ``1``, and actually activates the user.

When running in production, the activation is done automatically with different

types of moderation, that Astakos supports. You can see the moderation methods

(by invitation, whitelists, matching regexp, etc.) at the Astakos specific

documentation. In production, you can also manually activate a user, by sending

him/her an activation email. See how to do this at the :ref:`User

activation <user_activation>` section.

Now let's go back to the homepage. Open ``http://node1.example.com/ui/`` with

your browser again. Try to sign in using your new credentials. If the astakos

menu appears and you can see your profile, then you have successfully setup

Astakos.

Let's continue to install Pithos now.

Installation of Pithos on node2

===============================

To install Pithos, grab the packages from our repository (make sure  you made

the additions needed in your ``/etc/apt/sources.list`` file, as described

previously), by running:

.. code-block:: console

   # apt-get install snf-pithos-app snf-pithos-backend

After successful installation of snf-pithos-app, make sure that also

snf-webproject has been installed (marked as "Recommended" package). Refer to

the "Installation of Astakos on node1" section, if you don't remember why this

should happen. Now, install the pithos web interface:

.. code-block:: console

   # apt-get install snf-pithos-webclient

This package provides the standalone pithos web client. The web client is the

web UI for Pithos and will be accessible by clicking "pithos" on the Astakos

interface's cloudbar, at the top of the Astakos homepage.

.. _conf-pithos:

Configuration of Pithos

=======================

Conf Files

----------

After Pithos is successfully installed, you will find the directory

``/etc/synnefo`` and some configuration files inside it, as you did in node1

after installation of astakos. Here, you will not have to change anything that

has to do with snf-common or snf-webproject. Everything is set at node1. You

only need to change settings that have to do with Pithos. Specifically:

Edit ``/etc/synnefo/20-snf-pithos-app-settings.conf``. There you need to set

this options:

.. code-block:: console

   ASTAKOS_BASE_URL = 'https://node1.example.com/'

   PITHOS_BACKEND_DB_CONNECTION = 'postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos'

   PITHOS_BACKEND_BLOCK_PATH = '/srv/pithos/data'

   PITHOS_SERVICE_TOKEN = 'pithos_service_token22w=='

   # Set to False if astakos & pithos are on the same host

   #PITHOS_PROXY_USER_SERVICES = True

The ``PITHOS_BACKEND_DB_CONNECTION`` option tells to the Pithos app where to

find the Pithos backend database. Above we tell Pithos that its database is

``snf_pithos`` at node1 and to connect as user ``synnefo`` with password

``example_passw0rd``.  All those settings where setup during node1's "Database

setup" section.

The ``PITHOS_BACKEND_BLOCK_PATH`` option tells to the Pithos app where to find

the Pithos backend data. Above we tell Pithos to store its data under

``/srv/pithos/data``, which is visible by both nodes. We have already setup this

directory at node1's "Pithos data directory setup" section.

The ``ASTAKOS_BASE_URL`` option informs the Pithos app where Astakos is.

The Astakos service is used for user management (authentication, quotas, etc.)

The ``PITHOS_SERVICE_TOKEN`` should be the Pithos token returned by running on

the Astakos node (node1 in our case):

.. code-block:: console

   # snf-manage service-list

The token has been generated automatically during the :ref:`Pithos service

registration <services-reg>`.

Then we need to setup the web UI and connect it to astakos. To do so, edit

``/etc/synnefo/20-snf-pithos-webclient-settings.conf``:

.. code-block:: console

    PITHOS_UI_LOGIN_URL = "https://node1.example.com/ui/login?next="

    PITHOS_UI_FEEDBACK_URL = "https://node2.example.com/feedback"

The ``PITHOS_UI_LOGIN_URL`` option tells the client where to redirect you, if

you are not logged in. The ``PITHOS_UI_FEEDBACK_URL`` option points at the

Pithos feedback form. Astakos already provides a generic feedback form for all

services, so we use this one.

The ``PITHOS_UPDATE_MD5`` option by default disables the computation of the

object checksums. This results to improved performance during object uploading.

However, if compatibility with the OpenStack Object Storage API is important

then it should be changed to ``True``.

Then edit ``/etc/synnefo/20-snf-pithos-webclient-cloudbar.conf``, to connect the

Pithos web UI with the astakos web UI (through the top cloudbar):

.. code-block:: console

    CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'

    PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE = '3'

    CLOUDBAR_SERVICES_URL = 'https://node1.example.com/ui/get_services'

    CLOUDBAR_MENU_URL = 'https://node1.example.com/ui/get_menu'

The ``CLOUDBAR_LOCATION`` tells the client where to find the astakos common

cloudbar.

The ``PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE`` points to an already registered

Astakos service. You can see all :ref:`registered services <services-reg>` by

running on the Astakos node (node1):

.. code-block:: console

   # snf-manage service-list

The value of ``PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE`` should be the pithos

service's ``id`` as shown by the above command, in our case ``3``.

The ``CLOUDBAR_SERVICES_URL`` and ``CLOUDBAR_MENU_URL`` options are used by the

Pithos web client to get from astakos all the information needed to fill its

own cloudbar. So we put our astakos deployment urls there.

Pooling and Greenlets

---------------------

Pithos is pooling-ready without the need of further configuration, because it

doesn't use a Django DB. It pools HTTP connections to Astakos and pithos

backend objects for access to the Pithos DB.

However, as in Astakos, since we are running with Greenlets, it is also

recommended to modify psycopg2 behavior so it works properly in a greenlet

context. This means adding the following lines at the top of your

``/etc/synnefo/10-snf-webproject-database.conf`` file:

.. code-block:: console

    from synnefo.lib.db.psyco_gevent import make_psycopg_green

    make_psycopg_green()

Furthermore, add the ``--worker-class=gevent`` (or ``--worker-class=sync`` as

mentioned above, depending on your setup) argument on your

``/etc/gunicorn.d/synnefo`` configuration file. The file should look something

like this:

.. code-block:: console

    CONFIG = {

     'mode': 'django',

     'environment': {

       'DJANGO_SETTINGS_MODULE': 'synnefo.settings',

     },

     'working_dir': '/etc/synnefo',

     'user': 'www-data',

     'group': 'www-data',

     'args': (

       '--bind=127.0.0.1:8080',

       '--workers=4',

       '--worker-class=gevent',

       '--log-level=debug',

       '--timeout=43200'

     ),

    }

Stamp Database Revision

-----------------------

Pithos uses the alembic_ database migrations tool.

.. _alembic: http://alembic.readthedocs.org

After a sucessful installation, we should stamp it at the most recent

revision, so that future migrations know where to start upgrading in

the migration history.

First, find the most recent revision in the migration history:

.. code-block:: console

    root@node2:~ # pithos-migrate history

    2a309a9a3438 -> 27381099d477 (head), alter public add column url

    165ba3fbfe53 -> 2a309a9a3438, fix statistics negative population

    3dd56e750a3 -> 165ba3fbfe53, update account in paths

    230f8ce9c90f -> 3dd56e750a3, Fix latest_version

    8320b1c62d9 -> 230f8ce9c90f, alter nodes add column latest version

    None -> 8320b1c62d9, create index nodes.parent

Finally, we stamp it with the one found in the previous step:

.. code-block:: console

    root@node2:~ # pithos-migrate stamp 27381099d477

Servers Initialization

----------------------

After configuration is done, we initialize the servers on node2:

.. code-block:: console

    root@node2:~ # /etc/init.d/gunicorn restart

    root@node2:~ # /etc/init.d/apache2 restart

You have now finished the Pithos setup. Let's test it now.

Testing of Pithos

=================

Open your browser and go to the Astakos homepage:

``http://node1.example.com/im``

Login, and you will see your profile page. Now, click the "pithos" link on the

top black cloudbar. If everything was setup correctly, this will redirect you

to:

and you will see the blue interface of the Pithos application.  Click the

orange "Upload" button and upload your first file. If the file gets uploaded

successfully, then this is your first sign of a successful Pithos installation.

Go ahead and experiment with the interface to make sure everything works

correctly.

You can also use the Pithos clients to sync data from your Windows PC or MAC.

If you don't stumble on any problems, then you have successfully installed

Pithos, which you can use as a standalone File Storage Service.

If you would like to do more, such as:

    * Spawning VMs

    * Spawning VMs from Images stored on Pithos

    * Uploading your custom Images to Pithos

    * Spawning VMs from those custom Images

    * Registering existing Pithos files as Images

    * Connect VMs to the Internet

    * Create Private Networks

    * Add VMs to Private Networks

please continue with the rest of the guide.

Cyclades Prerequisites

======================

Before proceeding with the Cyclades installation, make sure you have

successfully set up Astakos and Pithos first, because Cyclades depends on

them. If you don't have a working Astakos and Pithos installation yet, please

return to the :ref:`top <quick-install-admin-guide>` of this guide.

Besides Astakos and Pithos, you will also need a number of additional working

prerequisites, before you start the Cyclades installation.

Ganeti

------

`Ganeti <http://code.google.com/p/ganeti/>`_ handles the low level VM management

for Cyclades, so Cyclades requires a working Ganeti installation at the backend.

Please refer to the

`ganeti documentation <http://docs.ganeti.org/ganeti/2.5/html>`_ for all the

gory details. A successful Ganeti installation concludes with a working

:ref:`GANETI-MASTER <GANETI_NODES>` and a number of :ref:`GANETI-NODEs

<GANETI_NODES>`.

The above Ganeti cluster can run on different physical machines than node1 and

node2 and can scale independently, according to your needs.

For the purpose of this guide, we will assume that the :ref:`GANETI-MASTER

<GANETI_NODES>` runs on node1 and is VM-capable. Also, node2 is a

:ref:`GANETI-NODE <GANETI_NODES>` and is Master-capable and VM-capable too.

We highly recommend that you read the official Ganeti documentation, if you are

not familiar with Ganeti.

Unfortunatelly, the current stable version of the stock Ganeti (v2.6.2) doesn't

support IP pool management. This feature will be available in Ganeti >= 2.7.

Synnefo depends on the IP pool functionality of Ganeti, so you have to use

GRNET provided packages until stable 2.7 is out. To do so:

.. code-block:: console

   # apt-get install snf-ganeti ganeti-htools

   # rmmod -f drbd && modprobe drbd minor_count=255 usermode_helper=/bin/true

You should have:

Ganeti >= 2.6.2+ippool11+hotplug5+extstorage3+rdbfix1+kvmfix2-1

We assume that Ganeti will use the KVM hypervisor. After installing Ganeti on

both nodes, choose a domain name that resolves to a valid floating IP (let's

say it's ``ganeti.node1.example.com``). Make sure node1 and node2 have same

dsa/rsa keys and authorised_keys for password-less root ssh between each other.

If not then skip passing --no-ssh-init but be aware that it will replace

/root/.ssh/* related files and you might lose access to master node. Also,

make sure there is an lvm volume group named ``ganeti`` that will host your

VMs' disks. Finally, setup a bridge interface on the host machines (e.g: br0).

Then run on node1:

.. code-block:: console

    root@node1:~ # gnt-cluster init --enabled-hypervisors=kvm --no-ssh-init \

                    --no-etc-hosts --vg-name=ganeti --nic-parameters link=br0 \

                    --master-netdev eth0 ganeti.node1.example.com

    root@node1:~ # gnt-cluster modify --default-iallocator hail

    root@node1:~ # gnt-cluster modify --hypervisor-parameters kvm:kernel_path=

    root@node1:~ # gnt-cluster modify --hypervisor-parameters kvm:vnc_bind_address=0.0.0.0

    root@node1:~ # gnt-node add --no-ssh-key-check --master-capable=yes \

                    --vm-capable=yes node2.example.com

    root@node1:~ # gnt-cluster modify --disk-parameters=drbd:metavg=ganeti

    root@node1:~ # gnt-group modify --disk-parameters=drbd:metavg=ganeti default

For any problems you may stumble upon installing Ganeti, please refer to the

`official documentation <http://docs.ganeti.org/ganeti/2.5/html>`_. Installation

of Ganeti is out of the scope of this guide.

.. _cyclades-install-snfimage:

snf-image

---------

Installation

~~~~~~~~~~~~

For :ref:`Cyclades <cyclades>` to be able to launch VMs from specified Images,

you need the :ref:`snf-image <snf-image>` OS Definition installed on *all*

VM-capable Ganeti nodes. This means we need :ref:`snf-image <snf-image>` on

node1 and node2. You can do this by running on *both* nodes:

.. code-block:: console

   # apt-get install snf-image snf-pithos-backend python-psycopg2

snf-image also needs the `snf-pithos-backend <snf-pithos-backend>`, to be able

to handle image files stored on Pithos. It also needs `python-psycopg2` to be

able to access the Pithos database. This is why, we also install them on *all*

VM-capable Ganeti nodes.

.. warning:: snf-image uses ``curl`` for handling URLs. This means that it will

    not  work out of the box if you try to use URLs served by servers which do

    not have a valid certificate. To circumvent this you should edit the file

    ``/etc/default/snf-image``. Change ``#CURL="curl"`` to ``CURL="curl -k"``.

After `snf-image` has been installed successfully, create the helper VM by

running on *both* nodes:

.. code-block:: console

   # snf-image-update-helper

This will create all the needed files under ``/var/lib/snf-image/helper/`` for

snf-image to run successfully, and it may take a few minutes depending on your

Internet connection.

Configuration

~~~~~~~~~~~~~

snf-image supports native access to Images stored on Pithos. This means that

it can talk directly to the Pithos backend, without the need of providing a

public URL. More details, are described in the next section. For now, the only

thing we need to do, is configure snf-image to access our Pithos backend.

To do this, we need to set the corresponding variables in

``/etc/default/snf-image``, to reflect our Pithos setup:

.. code-block:: console

    PITHOS_DB="postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos"

    PITHOS_DATA="/srv/pithos/data"

If you have installed your Ganeti cluster on different nodes than node1 and

node2 make sure that ``/srv/pithos/data`` is visible by all of them.

If you would like to use Images that are also/only stored locally, you need to

save them under ``IMAGE_DIR``, however this guide targets Images stored only on

Pithos.

Testing

~~~~~~~

You can test that snf-image is successfully installed by running on the

:ref:`GANETI-MASTER <GANETI_NODES>` (in our case node1):

.. code-block:: console

   # gnt-os diagnose

This should return ``valid`` for snf-image.

If you are interested to learn more about snf-image's internals (and even use

it alongside Ganeti without Synnefo), please see

`here <https://code.grnet.gr/projects/snf-image/wiki>`_ for information

concerning installation instructions, documentation on the design and

implementation, and supported Image formats.

.. _snf-image-images:

Actual Images for snf-image

---------------------------

Now that snf-image is installed successfully we need to provide it with some

Images. :ref:`snf-image <snf-image>` supports Images stored in ``extdump``,

``ntfsdump`` or ``diskdump`` format. We recommend the use of the ``diskdump``

format. For more information about snf-image Image formats see `here

<https://code.grnet.gr/projects/snf-image/wiki/Image_Format>`_.

:ref:`snf-image <snf-image>` also supports three (3) different locations for the

above Images to be stored:

    * Under a local folder (usually an NFS mount, configurable as ``IMAGE_DIR``

      in :file:`/etc/default/snf-image`)

    * On a remote host (accessible via public URL e.g: http://... or ftp://...)

    * On Pithos (accessible natively, not only by its public URL)

For the purpose of this guide, we will use the Debian Squeeze Base Image found

on the official `snf-image page

<https://code.grnet.gr/projects/snf-image/wiki#Sample-Images>`_. The image is

of type ``diskdump``. We will store it in our new Pithos installation.

To do so, do the following:

a) Download the Image from the official snf-image page.

b) Upload the Image to your Pithos installation, either using the Pithos Web

   UI or the command line client `kamaki

   <http://www.synnefo.org/docs/kamaki/latest/index.html>`_.

Once the Image is uploaded successfully, download the Image's metadata file

from the official snf-image page. You will need it, for spawning a VM from

Ganeti, in the next section.

Of course, you can repeat the procedure to upload more Images, available from

the `official snf-image page

<https://code.grnet.gr/projects/snf-image/wiki#Sample-Images>`_.

.. _ganeti-with-pithos-images:

Spawning a VM from a Pithos Image, using Ganeti

-----------------------------------------------

Now, it is time to test our installation so far. So, we have Astakos and

Pithos installed, we have a working Ganeti installation, the snf-image

definition installed on all VM-capable nodes and a Debian Squeeze Image on

Pithos. Make sure you also have the `metadata file

<https://pithos.okeanos.grnet.gr/public/gwqcv>`_ for this image.

Run on the :ref:`GANETI-MASTER's <GANETI_NODES>` (node1) command line:

.. code-block:: console

   # gnt-instance add -o snf-image+default --os-parameters \

                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://UUID/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \

                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \

                      testvm1

In the above command:

 * ``img_passwd``: the arbitrary root password of your new instance

 * ``img_format``: set to ``diskdump`` to reflect the type of the uploaded Image

 * ``img_id``: If you want to deploy an Image stored on Pithos (our case), this

               should have the format ``pithos://<UUID>/<container>/<filename>``:

               * ``username``: ``user@example.com`` (defined during Astakos sign up)

               * ``container``: ``pithos`` (default, if the Web UI was used)

               * ``filename``: the name of file (visible also from the Web UI)

 * ``img_properties``: taken from the metadata file. Used only the two mandatory

                       properties ``OSFAMILY`` and ``ROOT_PARTITION``. `Learn more

                       <https://code.grnet.gr/projects/snf-image/wiki/Image_Format#Image-Properties>`_

If the ``gnt-instance add`` command returns successfully, then run:

.. code-block:: console

   # gnt-instance info testvm1 | grep "console connection"

to find out where to connect using VNC. If you can connect successfully and can

login to your new instance using the root password ``my_vm_example_passw0rd``,

then everything works as expected and you have your new Debian Base VM up and

running.

If ``gnt-instance add`` fails, make sure that snf-image is correctly configured

to access the Pithos database and the Pithos backend data (newer versions

require UUID instead of a username). Another issue you may encounter is that in

relatively slow setups, you may need to raise the default HELPER_*_TIMEOUTS in

/etc/default/snf-image. Also, make sure you gave the correct ``img_id`` and

``img_properties``. If ``gnt-instance add`` succeeds but you cannot connect,

again find out what went wrong. Do *NOT* proceed to the next steps unless you

are sure everything works till this point.

If everything works, you have successfully connected Ganeti with Pithos. Let's

move on to networking now.

.. warning::

    You can bypass the networking sections and go straight to

    :ref:`Cyclades Ganeti tools <cyclades-gtools>`, if you do not want to setup

    the Cyclades Network Service, but only the Cyclades Compute Service

    (recommended for now).

Networking Setup Overview

-------------------------

This part is deployment-specific and must be customized based on the specific

needs of the system administrator. However, to do so, the administrator needs

to understand how each level handles Virtual Networks, to be able to setup the

backend appropriately, before installing Cyclades. To do so, please read the

:ref:`Network <networks>` section before proceeding.

Since synnefo 0.11 all network actions are managed with the snf-manage

network-* commands. This needs the underlying setup (Ganeti, nfdhcpd,

snf-network, bridges, vlans) to be already configured correctly. The only

actions needed in this point are:

a) Have Ganeti with IP pool management support installed.

b) Install :ref:`snf-network <snf-network>`, which provides a synnefo specific kvm-ifup script, etc.

c) Install :ref:`nfdhcpd <nfdhcpd>`, which serves DHCP requests of the VMs.

In order to test that everything is setup correctly before installing Cyclades,

we will make some testing actions in this section, and the actual setup will be

done afterwards with snf-manage commands.

.. _snf-network:

snf-network

~~~~~~~~~~~

snf-network includes `kvm-vif-bridge` script that is invoked every time

a tap (a VM's NIC) is created. Based on environment variables passed by

Ganeti it issues various commands depending on the network type the NIC is

connected to and sets up a corresponding dhcp lease.

Install snf-network on all Ganeti nodes:

.. code-block:: console

   # apt-get install snf-network

Then, in :file:`/etc/default/snf-network` set:

.. code-block:: console

   MAC_MASK=ff:ff:f0:00:00:00

.. _nfdhcpd:

nfdhcpd

~~~~~~~

Each NIC's IP is chosen by Ganeti (with IP pool management support).

`kvm-vif-bridge` script sets up dhcp leases and when the VM boots and

makes a dhcp request, iptables will mangle the packet and `nfdhcpd` will

create a dhcp response.

.. code-block:: console

   # apt-get install nfqueue-bindings-python=0.3+physindev-1

   # apt-get install nfdhcpd

Edit ``/etc/nfdhcpd/nfdhcpd.conf`` to reflect your network configuration. At

least, set the ``dhcp_queue`` variable to ``42`` and the ``nameservers``

variable to your DNS IP/s. Those IPs will be passed as the DNS IP/s of your new

VMs. Once you are finished, restart the server on all nodes:

.. code-block:: console

   # /etc/init.d/nfdhcpd restart

If you are using ``ferm``, then you need to run the following:

.. code-block:: console

   # echo "@include 'nfdhcpd.ferm';" >> /etc/ferm/ferm.conf

   # /etc/init.d/ferm restart

or make sure to run after boot:

.. code-block:: console

   # iptables -t mangle -A PREROUTING -p udp -m udp --dport 67 -j NFQUEUE --queue-num 42

and if you have IPv6 enabled:

.. code-block:: console

   # ip6tables -t mangle -A PREROUTING -p ipv6-icmp -m icmp6 --icmpv6-type 133 -j NFQUEUE --queue-num 43

   # ip6tables -t mangle -A PREROUTING -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j NFQUEUE --queue-num 44

You can check which clients are currently served by nfdhcpd by running:

.. code-block:: console

   # kill -SIGUSR1 `cat /var/run/nfdhcpd/nfdhcpd.pid`

When you run the above, then check ``/var/log/nfdhcpd/nfdhcpd.log``.

Public Network Setup

--------------------

To achieve basic networking the simplest way is to have a common bridge (e.g.

``br0``, on the same collision domain with the router) where all VMs will

connect to. Packets will be "forwarded" to the router and then to the Internet.

If you want a more advanced setup (ip-less routing and proxy-arp plese refer to

:ref:`Network <networks>` section).

Physical Host Setup

~~~~~~~~~~~~~~~~~~~

Assuming ``eth0`` on both hosts is the public interface (directly connected

to the router), run on every node:

.. code-block:: console

   # apt-get install vlan

   # brctl addbr br0

   # ip link set br0 up

   # vconfig add eth0 100

   # ip link set eth0.100 up

   # brctl addif br0 eth0.100

Testing a Public Network

~~~~~~~~~~~~~~~~~~~~~~~~

Let's assume, that you want to assign IPs from the ``5.6.7.0/27`` range to you

new VMs, with ``5.6.7.1`` as the router's gateway. In Ganeti you can add the

network by running:

.. code-block:: console

   # gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network-type=public --tags=nfdhcpd test-net-public

Then, connect the network to all your nodegroups. We assume that we only have

one nodegroup (``default``) in our Ganeti cluster:

.. code-block:: console

   # gnt-network connect test-net-public default bridged br0

Now, it is time to test that the backend infrastracture is correctly setup for

the Public Network. We will add a new VM, the same way we did it on the

previous testing section. However, now will also add one NIC, configured to be

managed from our previously defined network. Run on the GANETI-MASTER (node1):

.. code-block:: console

   # gnt-instance add -o snf-image+default --os-parameters \

                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://UUID/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \

                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \

                      --net 0:ip=pool,network=test-net-public \

                      testvm2

If the above returns successfully, connect to the new VM and run:

.. code-block:: console

   root@testvm2:~ # ip addr

   root@testvm2:~ # ip route

   root@testvm2:~ # cat /etc/resolv.conf

to check IP address (5.6.7.2), IP routes (default via 5.6.7.1) and DNS config

(nameserver option in nfdhcpd.conf). This shows correct configuration of

ganeti, snf-network and nfdhcpd.

Now ping the outside world. If this works too, then you have also configured

correctly your physical host and router.

Make sure everything works as expected, before proceeding with the Private

Networks setup.

.. _private-networks-setup:

Private Networks Setup

----------------------

Synnefo supports two types of private networks:

 - based on MAC filtering

 - based on physical VLANs

Both types provide Layer 2 isolation to the end-user.

For the first type a common bridge (e.g. ``prv0``) is needed while for the

second a range of bridges (e.g. ``prv1..prv100``) each bridged on a different

physical VLAN. To this end to assure isolation among end-users' private networks

each has to have different MAC prefix (for the filtering to take place) or to be

"connected" to a different bridge (VLAN actually).

Physical Host Setup

~~~~~~~~~~~~~~~~~~~

In order to create the necessary VLAN/bridges, one for MAC filtered private

networks and various (e.g. 20) for private networks based on physical VLANs,

run on every node:

Assuming ``eth0`` of both hosts are somehow (via cable/switch with VLANs

configured correctly) connected together, run on every node:

.. code-block:: console

   # modprobe 8021q

   # $iface=eth0

   # for prv in $(seq 0 20); do

        vlan=$prv

        bridge=prv$prv

        vconfig add $iface $vlan

        ifconfig $iface.$vlan up

        brctl addbr $bridge

        brctl setfd $bridge 0

        brctl addif $bridge $iface.$vlan

        ifconfig $bridge up

      done

The above will do the following :

 * provision 21 new bridges: ``prv0`` - ``prv20``

 * provision 21 new vlans: ``eth0.0`` - ``eth0.20``

 * add the corresponding vlan to the equivalent bridge

You can run ``brctl show`` on both nodes to see if everything was setup

correctly.