Statistics
| Branch: | Tag: | Revision:

root / docs / quick-install-admin-guide.rst @ 0c29b5ba

History | View | Annotate | Download (73.4 kB)

1 bdb83fd6 cven
.. _quick-install-admin-guide:
2 bdb83fd6 cven
3 bdb83fd6 cven
Administrator's Quick Installation Guide
4 a96ec00f Constantinos Venetsanopoulos
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5 a96ec00f Constantinos Venetsanopoulos
6 a96ec00f Constantinos Venetsanopoulos
This is the Administrator's quick installation guide.
7 a96ec00f Constantinos Venetsanopoulos
8 a96ec00f Constantinos Venetsanopoulos
It describes how to install the whole synnefo stack on two (2) physical nodes,
9 a96ec00f Constantinos Venetsanopoulos
with minimum configuration. It installs synnefo from Debian packages, and
10 a96ec00f Constantinos Venetsanopoulos
assumes the nodes run Debian Squeeze. After successful installation, you will
11 a96ec00f Constantinos Venetsanopoulos
have the following services running:
12 a96ec00f Constantinos Venetsanopoulos
13 a96ec00f Constantinos Venetsanopoulos
 * Identity Management (Astakos)
14 dad708b4 Antony Chazapis
 * Object Storage Service (Pithos+)
15 a96ec00f Constantinos Venetsanopoulos
 * Compute Service (Cyclades)
16 a96ec00f Constantinos Venetsanopoulos
 * Image Registry Service (Plankton)
17 a96ec00f Constantinos Venetsanopoulos
18 a96ec00f Constantinos Venetsanopoulos
and a single unified Web UI to manage them all.
19 a96ec00f Constantinos Venetsanopoulos
20 a96ec00f Constantinos Venetsanopoulos
The Volume Storage Service (Archipelago) and the Billing Service (Aquarium) are
21 a96ec00f Constantinos Venetsanopoulos
not released yet.
22 a96ec00f Constantinos Venetsanopoulos
23 dad708b4 Antony Chazapis
If you just want to install the Object Storage Service (Pithos+), follow the guide
24 a96ec00f Constantinos Venetsanopoulos
and just stop after the "Testing of Pithos+" section.
25 a96ec00f Constantinos Venetsanopoulos
26 a96ec00f Constantinos Venetsanopoulos
27 a96ec00f Constantinos Venetsanopoulos
Installation of Synnefo / Introduction
28 a96ec00f Constantinos Venetsanopoulos
======================================
29 a96ec00f Constantinos Venetsanopoulos
30 a96ec00f Constantinos Venetsanopoulos
We will install the services with the above list's order. Cyclades and Plankton
31 a96ec00f Constantinos Venetsanopoulos
will be installed in a single step (at the end), because at the moment they are
32 a96ec00f Constantinos Venetsanopoulos
contained in the same software component. Furthermore, we will install all
33 a96ec00f Constantinos Venetsanopoulos
services in the first physical node, except Pithos+ which will be installed in
34 a96ec00f Constantinos Venetsanopoulos
the second, due to a conflict between the snf-pithos-app and snf-cyclades-app
35 a96ec00f Constantinos Venetsanopoulos
component (scheduled to be fixed in the next version).
36 a96ec00f Constantinos Venetsanopoulos
37 a96ec00f Constantinos Venetsanopoulos
For the rest of the documentation we will refer to the first physical node as
38 a96ec00f Constantinos Venetsanopoulos
"node1" and the second as "node2". We will also assume that their domain names
39 a96ec00f Constantinos Venetsanopoulos
are "node1.example.com" and "node2.example.com" and their IPs are "4.3.2.1" and
40 a96ec00f Constantinos Venetsanopoulos
"4.3.2.2" respectively.
41 a96ec00f Constantinos Venetsanopoulos
42 04308336 Ilias Tsitsimpis
.. note:: It is import that the two machines are under the same domain name.
43 04308336 Ilias Tsitsimpis
    If they are not, you can do this by editting the file ``/etc/hosts``
44 04308336 Ilias Tsitsimpis
    on both machines, and add the following lines:
45 04308336 Ilias Tsitsimpis
46 04308336 Ilias Tsitsimpis
    .. code-block:: console
47 04308336 Ilias Tsitsimpis
48 04308336 Ilias Tsitsimpis
        4.3.2.1     node1.example.com
49 04308336 Ilias Tsitsimpis
        4.3.2.2     node2.example.com
50 04308336 Ilias Tsitsimpis
51 a96ec00f Constantinos Venetsanopoulos
52 a96ec00f Constantinos Venetsanopoulos
General Prerequisites
53 a96ec00f Constantinos Venetsanopoulos
=====================
54 a96ec00f Constantinos Venetsanopoulos
55 a96ec00f Constantinos Venetsanopoulos
These are the general synnefo prerequisites, that you need on node1 and node2
56 a96ec00f Constantinos Venetsanopoulos
and are related to all the services (Astakos, Pithos+, Cyclades, Plankton).
57 a96ec00f Constantinos Venetsanopoulos
58 a96ec00f Constantinos Venetsanopoulos
To be able to download all synnefo components you need to add the following
59 a96ec00f Constantinos Venetsanopoulos
lines in your ``/etc/apt/sources.list`` file:
60 a96ec00f Constantinos Venetsanopoulos
61 d109485a Constantinos Venetsanopoulos
| ``deb http://apt.dev.grnet.gr squeeze main``
62 d109485a Constantinos Venetsanopoulos
| ``deb-src http://apt.dev.grnet.gr squeeze main``
63 37e1b6cd Dimitris Aragiorgis
| ``deb http://apt.dev.grnet.gr squeeze-backports main``
64 a96ec00f Constantinos Venetsanopoulos
65 ec9862dd Stratos Psomadakis
and import the repo's GPG key:
66 ec9862dd Stratos Psomadakis
67 ec9862dd Stratos Psomadakis
| ``curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -``
68 ec9862dd Stratos Psomadakis
69 169f7d38 Vangelis Koukis
Also add the following line to enable the ``squeeze-backports`` repository,
70 169f7d38 Vangelis Koukis
which may provide more recent versions of certain packages. The repository
71 169f7d38 Vangelis Koukis
is deactivated by default and must be specified expicitly in ``apt-get``
72 169f7d38 Vangelis Koukis
operations:
73 169f7d38 Vangelis Koukis
74 169f7d38 Vangelis Koukis
| ``deb http://backports.debian.org/debian-backports squeeze-backports main``
75 169f7d38 Vangelis Koukis
76 a96ec00f Constantinos Venetsanopoulos
You also need a shared directory visible by both nodes. Pithos+ will save all
77 a96ec00f Constantinos Venetsanopoulos
data inside this directory. By 'all data', we mean files, images, and pithos
78 a96ec00f Constantinos Venetsanopoulos
specific mapping data. If you plan to upload more than one basic image, this
79 a96ec00f Constantinos Venetsanopoulos
directory should have at least 50GB of free space. During this guide, we will
80 a96ec00f Constantinos Venetsanopoulos
assume that node1 acts as an NFS server and serves the directory ``/srv/pithos``
81 ba173277 Ilias Tsitsimpis
to node2 (be sure to set no_root_squash flag). Node2 has this directory
82 ba173277 Ilias Tsitsimpis
mounted under ``/srv/pithos``, too.
83 a96ec00f Constantinos Venetsanopoulos
84 a96ec00f Constantinos Venetsanopoulos
Before starting the synnefo installation, you will need basic third party
85 a96ec00f Constantinos Venetsanopoulos
software to be installed and configured on the physical nodes. We will describe
86 a96ec00f Constantinos Venetsanopoulos
each node's general prerequisites separately. Any additional configuration,
87 a96ec00f Constantinos Venetsanopoulos
specific to a synnefo service for each node, will be described at the service's
88 a96ec00f Constantinos Venetsanopoulos
section.
89 a96ec00f Constantinos Venetsanopoulos
90 7df5a742 Christos Stavrakakis
Finally, it is required for Cyclades and Ganeti nodes to have synchronized
91 7df5a742 Christos Stavrakakis
system clocks (e.g. by running ntpd).
92 7df5a742 Christos Stavrakakis
93 a96ec00f Constantinos Venetsanopoulos
Node1
94 a96ec00f Constantinos Venetsanopoulos
-----
95 a96ec00f Constantinos Venetsanopoulos
96 a96ec00f Constantinos Venetsanopoulos
General Synnefo dependencies
97 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98 a96ec00f Constantinos Venetsanopoulos
99 a96ec00f Constantinos Venetsanopoulos
 * apache (http server)
100 a96ec00f Constantinos Venetsanopoulos
 * gunicorn (WSGI http server)
101 a96ec00f Constantinos Venetsanopoulos
 * postgresql (database)
102 a96ec00f Constantinos Venetsanopoulos
 * rabbitmq (message queue)
103 7df5a742 Christos Stavrakakis
 * ntp (NTP daemon)
104 a96ec00f Constantinos Venetsanopoulos
105 7df5a742 Christos Stavrakakis
You can install apache2, progresql and ntp by running:
106 a96ec00f Constantinos Venetsanopoulos
107 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
108 a96ec00f Constantinos Venetsanopoulos
109 7df5a742 Christos Stavrakakis
   # apt-get install apache2 postgresql ntp
110 a96ec00f Constantinos Venetsanopoulos
111 73ff1d54 Constantinos Venetsanopoulos
Make sure to install gunicorn >= v0.12.2. You can do this by installing from
112 73ff1d54 Constantinos Venetsanopoulos
the official debian backports:
113 73ff1d54 Constantinos Venetsanopoulos
114 73ff1d54 Constantinos Venetsanopoulos
.. code-block:: console
115 73ff1d54 Constantinos Venetsanopoulos
116 73ff1d54 Constantinos Venetsanopoulos
   # apt-get -t squeeze-backports install gunicorn
117 73ff1d54 Constantinos Venetsanopoulos
118 73ff1d54 Constantinos Venetsanopoulos
On node1, we will create our databases, so you will also need the
119 73ff1d54 Constantinos Venetsanopoulos
python-psycopg2 package:
120 a96ec00f Constantinos Venetsanopoulos
121 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
122 a96ec00f Constantinos Venetsanopoulos
123 a96ec00f Constantinos Venetsanopoulos
   # apt-get install python-psycopg2
124 a96ec00f Constantinos Venetsanopoulos
125 29291ebc Christos Stavrakakis
To install RabbitMQ>=2.8.4, use the RabbitMQ APT repository by adding the
126 29291ebc Christos Stavrakakis
following line to ``/etc/apt/sources.list``:
127 29291ebc Christos Stavrakakis
128 29291ebc Christos Stavrakakis
.. code-block:: console
129 29291ebc Christos Stavrakakis
130 29291ebc Christos Stavrakakis
  deb http://www.rabbitmq.com/debian testing main
131 29291ebc Christos Stavrakakis
132 29291ebc Christos Stavrakakis
Add RabbitMQ public key, to trusted key list:
133 29291ebc Christos Stavrakakis
134 29291ebc Christos Stavrakakis
.. code-block:: console
135 29291ebc Christos Stavrakakis
136 29291ebc Christos Stavrakakis
  # wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
137 29291ebc Christos Stavrakakis
  # apt-key add rabbitmq-signing-key-public.asc
138 29291ebc Christos Stavrakakis
139 29291ebc Christos Stavrakakis
Finally, to install the package run:
140 29291ebc Christos Stavrakakis
141 29291ebc Christos Stavrakakis
.. code-block:: console
142 29291ebc Christos Stavrakakis
143 29291ebc Christos Stavrakakis
  # apt-get update
144 29291ebc Christos Stavrakakis
  # apt-get install rabbitmq-server
145 29291ebc Christos Stavrakakis
146 a96ec00f Constantinos Venetsanopoulos
Database setup
147 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~
148 a96ec00f Constantinos Venetsanopoulos
149 a96ec00f Constantinos Venetsanopoulos
On node1, we create a database called ``snf_apps``, that will host all django
150 a96ec00f Constantinos Venetsanopoulos
apps related tables. We also create the user ``synnefo`` and grant him all
151 a96ec00f Constantinos Venetsanopoulos
privileges on the database. We do this by running:
152 a96ec00f Constantinos Venetsanopoulos
153 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
154 a96ec00f Constantinos Venetsanopoulos
155 a96ec00f Constantinos Venetsanopoulos
   root@node1:~ # su - postgres
156 a96ec00f Constantinos Venetsanopoulos
   postgres@node1:~ $ psql
157 a96ec00f Constantinos Venetsanopoulos
   postgres=# CREATE DATABASE snf_apps WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
158 a96ec00f Constantinos Venetsanopoulos
   postgres=# CREATE USER synnefo WITH PASSWORD 'example_passw0rd';
159 a96ec00f Constantinos Venetsanopoulos
   postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_apps TO synnefo;
160 a96ec00f Constantinos Venetsanopoulos
161 a96ec00f Constantinos Venetsanopoulos
We also create the database ``snf_pithos`` needed by the pithos+ backend and
162 a96ec00f Constantinos Venetsanopoulos
grant the ``synnefo`` user all privileges on the database. This database could
163 a96ec00f Constantinos Venetsanopoulos
be created on node2 instead, but we do it on node1 for simplicity. We will
164 a96ec00f Constantinos Venetsanopoulos
create all needed databases on node1 and then node2 will connect to them.
165 a96ec00f Constantinos Venetsanopoulos
166 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
167 a96ec00f Constantinos Venetsanopoulos
168 73ff1d54 Constantinos Venetsanopoulos
   postgres=# CREATE DATABASE snf_pithos WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
169 a96ec00f Constantinos Venetsanopoulos
   postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_pithos TO synnefo;
170 a96ec00f Constantinos Venetsanopoulos
171 a96ec00f Constantinos Venetsanopoulos
Configure the database to listen to all network interfaces. You can do this by
172 a96ec00f Constantinos Venetsanopoulos
editting the file ``/etc/postgresql/8.4/main/postgresql.conf`` and change
173 a96ec00f Constantinos Venetsanopoulos
``listen_addresses`` to ``'*'`` :
174 a96ec00f Constantinos Venetsanopoulos
175 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
176 a96ec00f Constantinos Venetsanopoulos
177 a96ec00f Constantinos Venetsanopoulos
   listen_addresses = '*'
178 a96ec00f Constantinos Venetsanopoulos
179 a96ec00f Constantinos Venetsanopoulos
Furthermore, edit ``/etc/postgresql/8.4/main/pg_hba.conf`` to allow node1 and
180 a96ec00f Constantinos Venetsanopoulos
node2 to connect to the database. Add the following lines under ``#IPv4 local
181 a96ec00f Constantinos Venetsanopoulos
connections:`` :
182 a96ec00f Constantinos Venetsanopoulos
183 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
184 a96ec00f Constantinos Venetsanopoulos
185 a96ec00f Constantinos Venetsanopoulos
   host		all	all	4.3.2.1/32	md5
186 a96ec00f Constantinos Venetsanopoulos
   host		all	all	4.3.2.2/32	md5
187 a96ec00f Constantinos Venetsanopoulos
188 a96ec00f Constantinos Venetsanopoulos
Make sure to substitute "4.3.2.1" and "4.3.2.2" with node1's and node2's
189 a96ec00f Constantinos Venetsanopoulos
actual IPs. Now, restart the server to apply the changes:
190 a96ec00f Constantinos Venetsanopoulos
191 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
192 a96ec00f Constantinos Venetsanopoulos
193 a96ec00f Constantinos Venetsanopoulos
   # /etc/init.d/postgresql restart
194 a96ec00f Constantinos Venetsanopoulos
195 a96ec00f Constantinos Venetsanopoulos
Gunicorn setup
196 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~
197 a96ec00f Constantinos Venetsanopoulos
198 a96ec00f Constantinos Venetsanopoulos
Create the file ``synnefo`` under ``/etc/gunicorn.d/`` containing the following:
199 a96ec00f Constantinos Venetsanopoulos
200 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
201 a96ec00f Constantinos Venetsanopoulos
202 a96ec00f Constantinos Venetsanopoulos
   CONFIG = {
203 a96ec00f Constantinos Venetsanopoulos
    'mode': 'django',
204 a96ec00f Constantinos Venetsanopoulos
    'environment': {
205 a96ec00f Constantinos Venetsanopoulos
      'DJANGO_SETTINGS_MODULE': 'synnefo.settings',
206 a96ec00f Constantinos Venetsanopoulos
    },
207 a96ec00f Constantinos Venetsanopoulos
    'working_dir': '/etc/synnefo',
208 a96ec00f Constantinos Venetsanopoulos
    'user': 'www-data',
209 a96ec00f Constantinos Venetsanopoulos
    'group': 'www-data',
210 a96ec00f Constantinos Venetsanopoulos
    'args': (
211 a96ec00f Constantinos Venetsanopoulos
      '--bind=127.0.0.1:8080',
212 3c833282 Christos Stavrakakis
      '--workers=8',
213 a96ec00f Constantinos Venetsanopoulos
      '--log-level=debug',
214 a96ec00f Constantinos Venetsanopoulos
    ),
215 a96ec00f Constantinos Venetsanopoulos
   }
216 a96ec00f Constantinos Venetsanopoulos
217 0c40b4ac Kostas Papadimitriou
.. warning:: Do NOT start the server yet, because it won't find the
218 1ec63d8a Kostas Papadimitriou
    ``synnefo.settings`` module. We will start the server after successful
219 4de94e15 Kostas Papadimitriou
    installation of astakos. If the server is running::
220 a96ec00f Constantinos Venetsanopoulos
221 4de94e15 Kostas Papadimitriou
       # /etc/init.d/gunicorn stop
222 a96ec00f Constantinos Venetsanopoulos
223 a96ec00f Constantinos Venetsanopoulos
Apache2 setup
224 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~
225 a96ec00f Constantinos Venetsanopoulos
226 a96ec00f Constantinos Venetsanopoulos
Create the file ``synnefo`` under ``/etc/apache2/sites-available/`` containing
227 a96ec00f Constantinos Venetsanopoulos
the following:
228 a96ec00f Constantinos Venetsanopoulos
229 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
230 a96ec00f Constantinos Venetsanopoulos
231 a96ec00f Constantinos Venetsanopoulos
   <VirtualHost *:80>
232 a96ec00f Constantinos Venetsanopoulos
     ServerName node1.example.com
233 a96ec00f Constantinos Venetsanopoulos
234 a96ec00f Constantinos Venetsanopoulos
     RewriteEngine On
235 d2a9f85f Sofia Papagiannaki
     RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
236 d2a9f85f Sofia Papagiannaki
     RewriteRule ^(.*)$ - [F,L]
237 a96ec00f Constantinos Venetsanopoulos
     RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
238 a96ec00f Constantinos Venetsanopoulos
   </VirtualHost>
239 a96ec00f Constantinos Venetsanopoulos
240 a96ec00f Constantinos Venetsanopoulos
Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``
241 a96ec00f Constantinos Venetsanopoulos
containing the following:
242 a96ec00f Constantinos Venetsanopoulos
243 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
244 a96ec00f Constantinos Venetsanopoulos
245 a96ec00f Constantinos Venetsanopoulos
   <IfModule mod_ssl.c>
246 a96ec00f Constantinos Venetsanopoulos
   <VirtualHost _default_:443>
247 a96ec00f Constantinos Venetsanopoulos
     ServerName node1.example.com
248 a96ec00f Constantinos Venetsanopoulos
249 a96ec00f Constantinos Venetsanopoulos
     Alias /static "/usr/share/synnefo/static"
250 a96ec00f Constantinos Venetsanopoulos
251 a96ec00f Constantinos Venetsanopoulos
   #  SetEnv no-gzip
252 a96ec00f Constantinos Venetsanopoulos
   #  SetEnv dont-vary
253 a96ec00f Constantinos Venetsanopoulos
254 04427415 Constantinos Venetsanopoulos
     AllowEncodedSlashes On
255 04427415 Constantinos Venetsanopoulos
256 a96ec00f Constantinos Venetsanopoulos
     RequestHeader set X-Forwarded-Protocol "https"
257 a96ec00f Constantinos Venetsanopoulos
258 a96ec00f Constantinos Venetsanopoulos
     <Proxy * >
259 a96ec00f Constantinos Venetsanopoulos
       Order allow,deny
260 a96ec00f Constantinos Venetsanopoulos
       Allow from all
261 a96ec00f Constantinos Venetsanopoulos
     </Proxy>
262 a96ec00f Constantinos Venetsanopoulos
263 a96ec00f Constantinos Venetsanopoulos
     SetEnv                proxy-sendchunked
264 a96ec00f Constantinos Venetsanopoulos
     SSLProxyEngine        off
265 a96ec00f Constantinos Venetsanopoulos
     ProxyErrorOverride    off
266 a96ec00f Constantinos Venetsanopoulos
267 a96ec00f Constantinos Venetsanopoulos
     ProxyPass        /static !
268 a96ec00f Constantinos Venetsanopoulos
     ProxyPass        / http://localhost:8080/ retry=0
269 a96ec00f Constantinos Venetsanopoulos
     ProxyPassReverse / http://localhost:8080/
270 a96ec00f Constantinos Venetsanopoulos
271 a96ec00f Constantinos Venetsanopoulos
     RewriteEngine On
272 d2a9f85f Sofia Papagiannaki
     RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
273 d2a9f85f Sofia Papagiannaki
     RewriteRule ^(.*)$ - [F,L]
274 a96ec00f Constantinos Venetsanopoulos
275 a96ec00f Constantinos Venetsanopoulos
     SSLEngine on
276 a96ec00f Constantinos Venetsanopoulos
     SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
277 a96ec00f Constantinos Venetsanopoulos
     SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
278 a96ec00f Constantinos Venetsanopoulos
   </VirtualHost>
279 a96ec00f Constantinos Venetsanopoulos
   </IfModule>
280 a96ec00f Constantinos Venetsanopoulos
281 a96ec00f Constantinos Venetsanopoulos
Now enable sites and modules by running:
282 a96ec00f Constantinos Venetsanopoulos
283 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
284 a96ec00f Constantinos Venetsanopoulos
285 a96ec00f Constantinos Venetsanopoulos
   # a2enmod ssl
286 a96ec00f Constantinos Venetsanopoulos
   # a2enmod rewrite
287 a96ec00f Constantinos Venetsanopoulos
   # a2dissite default
288 a96ec00f Constantinos Venetsanopoulos
   # a2ensite synnefo
289 a96ec00f Constantinos Venetsanopoulos
   # a2ensite synnefo-ssl
290 a96ec00f Constantinos Venetsanopoulos
   # a2enmod headers
291 a96ec00f Constantinos Venetsanopoulos
   # a2enmod proxy_http
292 a96ec00f Constantinos Venetsanopoulos
293 4de94e15 Kostas Papadimitriou
.. warning:: Do NOT start/restart the server yet. If the server is running::
294 a96ec00f Constantinos Venetsanopoulos
295 4de94e15 Kostas Papadimitriou
       # /etc/init.d/apache2 stop
296 a96ec00f Constantinos Venetsanopoulos
297 2c85833e Constantinos Venetsanopoulos
.. _rabbitmq-setup:
298 2c85833e Constantinos Venetsanopoulos
299 bdfd94c9 Constantinos Venetsanopoulos
Message Queue setup
300 bdfd94c9 Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~
301 bdfd94c9 Constantinos Venetsanopoulos
302 bdfd94c9 Constantinos Venetsanopoulos
The message queue will run on node1, so we need to create the appropriate
303 bdfd94c9 Constantinos Venetsanopoulos
rabbitmq user. The user is named ``synnefo`` and gets full privileges on all
304 bdfd94c9 Constantinos Venetsanopoulos
exchanges:
305 bdfd94c9 Constantinos Venetsanopoulos
306 bdfd94c9 Constantinos Venetsanopoulos
.. code-block:: console
307 bdfd94c9 Constantinos Venetsanopoulos
308 69aa7f21 Ilias Tsitsimpis
   # rabbitmqctl add_user synnefo "example_rabbitmq_passw0rd"
309 bdfd94c9 Constantinos Venetsanopoulos
   # rabbitmqctl set_permissions synnefo ".*" ".*" ".*"
310 bdfd94c9 Constantinos Venetsanopoulos
311 bdfd94c9 Constantinos Venetsanopoulos
We do not need to initialize the exchanges. This will be done automatically,
312 bdfd94c9 Constantinos Venetsanopoulos
during the Cyclades setup.
313 bdfd94c9 Constantinos Venetsanopoulos
314 a96ec00f Constantinos Venetsanopoulos
Pithos+ data directory setup
315 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
316 a96ec00f Constantinos Venetsanopoulos
317 a96ec00f Constantinos Venetsanopoulos
As mentioned in the General Prerequisites section, there is a directory called
318 a96ec00f Constantinos Venetsanopoulos
``/srv/pithos`` visible by both nodes. We create and setup the ``data``
319 a96ec00f Constantinos Venetsanopoulos
directory inside it:
320 a96ec00f Constantinos Venetsanopoulos
321 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
322 a96ec00f Constantinos Venetsanopoulos
323 a96ec00f Constantinos Venetsanopoulos
   # cd /srv/pithos
324 a96ec00f Constantinos Venetsanopoulos
   # mkdir data
325 a96ec00f Constantinos Venetsanopoulos
   # chown www-data:www-data data
326 a96ec00f Constantinos Venetsanopoulos
   # chmod g+ws data
327 a96ec00f Constantinos Venetsanopoulos
328 a96ec00f Constantinos Venetsanopoulos
You are now ready with all general prerequisites concerning node1. Let's go to
329 a96ec00f Constantinos Venetsanopoulos
node2.
330 a96ec00f Constantinos Venetsanopoulos
331 a96ec00f Constantinos Venetsanopoulos
Node2
332 a96ec00f Constantinos Venetsanopoulos
-----
333 a96ec00f Constantinos Venetsanopoulos
334 a96ec00f Constantinos Venetsanopoulos
General Synnefo dependencies
335 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
336 a96ec00f Constantinos Venetsanopoulos
337 a96ec00f Constantinos Venetsanopoulos
 * apache (http server)
338 a96ec00f Constantinos Venetsanopoulos
 * gunicorn (WSGI http server)
339 a96ec00f Constantinos Venetsanopoulos
 * postgresql (database)
340 7df5a742 Christos Stavrakakis
 * ntp (NTP daemon)
341 a96ec00f Constantinos Venetsanopoulos
342 a96ec00f Constantinos Venetsanopoulos
You can install the above by running:
343 a96ec00f Constantinos Venetsanopoulos
344 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
345 a96ec00f Constantinos Venetsanopoulos
346 7df5a742 Christos Stavrakakis
   # apt-get install apache2 postgresql ntp
347 73ff1d54 Constantinos Venetsanopoulos
348 73ff1d54 Constantinos Venetsanopoulos
Make sure to install gunicorn >= v0.12.2. You can do this by installing from
349 73ff1d54 Constantinos Venetsanopoulos
the official debian backports:
350 73ff1d54 Constantinos Venetsanopoulos
351 73ff1d54 Constantinos Venetsanopoulos
.. code-block:: console
352 73ff1d54 Constantinos Venetsanopoulos
353 73ff1d54 Constantinos Venetsanopoulos
   # apt-get -t squeeze-backports install gunicorn
354 a96ec00f Constantinos Venetsanopoulos
355 73ff1d54 Constantinos Venetsanopoulos
Node2 will connect to the databases on node1, so you will also need the
356 73ff1d54 Constantinos Venetsanopoulos
python-psycopg2 package:
357 a96ec00f Constantinos Venetsanopoulos
358 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
359 a96ec00f Constantinos Venetsanopoulos
360 a96ec00f Constantinos Venetsanopoulos
   # apt-get install python-psycopg2
361 a96ec00f Constantinos Venetsanopoulos
362 a96ec00f Constantinos Venetsanopoulos
Database setup
363 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~
364 a96ec00f Constantinos Venetsanopoulos
365 a96ec00f Constantinos Venetsanopoulos
All databases have been created and setup on node1, so we do not need to take
366 a96ec00f Constantinos Venetsanopoulos
any action here. From node2, we will just connect to them. When you get familiar
367 a96ec00f Constantinos Venetsanopoulos
with the software you may choose to run different databases on different nodes,
368 a96ec00f Constantinos Venetsanopoulos
for performance/scalability/redundancy reasons, but those kind of setups are out
369 a96ec00f Constantinos Venetsanopoulos
of the purpose of this guide.
370 a96ec00f Constantinos Venetsanopoulos
371 a96ec00f Constantinos Venetsanopoulos
Gunicorn setup
372 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~
373 a96ec00f Constantinos Venetsanopoulos
374 a96ec00f Constantinos Venetsanopoulos
Create the file ``synnefo`` under ``/etc/gunicorn.d/`` containing the following
375 a96ec00f Constantinos Venetsanopoulos
(same contents as in node1; you can just copy/paste the file):
376 a96ec00f Constantinos Venetsanopoulos
377 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
378 a96ec00f Constantinos Venetsanopoulos
379 a96ec00f Constantinos Venetsanopoulos
   CONFIG = {
380 a96ec00f Constantinos Venetsanopoulos
    'mode': 'django',
381 a96ec00f Constantinos Venetsanopoulos
    'environment': {
382 a96ec00f Constantinos Venetsanopoulos
      'DJANGO_SETTINGS_MODULE': 'synnefo.settings',
383 a96ec00f Constantinos Venetsanopoulos
    },
384 a96ec00f Constantinos Venetsanopoulos
    'working_dir': '/etc/synnefo',
385 a96ec00f Constantinos Venetsanopoulos
    'user': 'www-data',
386 a96ec00f Constantinos Venetsanopoulos
    'group': 'www-data',
387 a96ec00f Constantinos Venetsanopoulos
    'args': (
388 a96ec00f Constantinos Venetsanopoulos
      '--bind=127.0.0.1:8080',
389 a96ec00f Constantinos Venetsanopoulos
      '--workers=4',
390 a96ec00f Constantinos Venetsanopoulos
      '--log-level=debug',
391 699c8773 Sofia Papagiannaki
      '--timeout=43200'
392 a96ec00f Constantinos Venetsanopoulos
    ),
393 a96ec00f Constantinos Venetsanopoulos
   }
394 a96ec00f Constantinos Venetsanopoulos
395 0c40b4ac Kostas Papadimitriou
.. warning:: Do NOT start the server yet, because it won't find the
396 1ec63d8a Kostas Papadimitriou
    ``synnefo.settings`` module. We will start the server after successful
397 4de94e15 Kostas Papadimitriou
    installation of astakos. If the server is running::
398 a96ec00f Constantinos Venetsanopoulos
399 4de94e15 Kostas Papadimitriou
       # /etc/init.d/gunicorn stop
400 bdb83fd6 cven
401 a96ec00f Constantinos Venetsanopoulos
Apache2 setup
402 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~
403 5b6feb88 Vangelis Koukis
404 a96ec00f Constantinos Venetsanopoulos
Create the file ``synnefo`` under ``/etc/apache2/sites-available/`` containing
405 a96ec00f Constantinos Venetsanopoulos
the following:
406 5b6feb88 Vangelis Koukis
407 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
408 a96ec00f Constantinos Venetsanopoulos
409 a96ec00f Constantinos Venetsanopoulos
   <VirtualHost *:80>
410 a96ec00f Constantinos Venetsanopoulos
     ServerName node2.example.com
411 a96ec00f Constantinos Venetsanopoulos
412 a96ec00f Constantinos Venetsanopoulos
     RewriteEngine On
413 d2a9f85f Sofia Papagiannaki
     RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
414 d2a9f85f Sofia Papagiannaki
     RewriteRule ^(.*)$ - [F,L]
415 a96ec00f Constantinos Venetsanopoulos
     RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
416 a96ec00f Constantinos Venetsanopoulos
   </VirtualHost>
417 a96ec00f Constantinos Venetsanopoulos
418 a96ec00f Constantinos Venetsanopoulos
Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``
419 a96ec00f Constantinos Venetsanopoulos
containing the following:
420 a96ec00f Constantinos Venetsanopoulos
421 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
422 a96ec00f Constantinos Venetsanopoulos
423 a96ec00f Constantinos Venetsanopoulos
   <IfModule mod_ssl.c>
424 a96ec00f Constantinos Venetsanopoulos
   <VirtualHost _default_:443>
425 a96ec00f Constantinos Venetsanopoulos
     ServerName node2.example.com
426 a96ec00f Constantinos Venetsanopoulos
427 a96ec00f Constantinos Venetsanopoulos
     Alias /static "/usr/share/synnefo/static"
428 a96ec00f Constantinos Venetsanopoulos
429 a96ec00f Constantinos Venetsanopoulos
     SetEnv no-gzip
430 a96ec00f Constantinos Venetsanopoulos
     SetEnv dont-vary
431 04427415 Constantinos Venetsanopoulos
     AllowEncodedSlashes On
432 a96ec00f Constantinos Venetsanopoulos
433 a96ec00f Constantinos Venetsanopoulos
     RequestHeader set X-Forwarded-Protocol "https"
434 a96ec00f Constantinos Venetsanopoulos
435 a96ec00f Constantinos Venetsanopoulos
     <Proxy * >
436 a96ec00f Constantinos Venetsanopoulos
       Order allow,deny
437 a96ec00f Constantinos Venetsanopoulos
       Allow from all
438 a96ec00f Constantinos Venetsanopoulos
     </Proxy>
439 a96ec00f Constantinos Venetsanopoulos
440 a96ec00f Constantinos Venetsanopoulos
     SetEnv                proxy-sendchunked
441 a96ec00f Constantinos Venetsanopoulos
     SSLProxyEngine        off
442 a96ec00f Constantinos Venetsanopoulos
     ProxyErrorOverride    off
443 a96ec00f Constantinos Venetsanopoulos
444 a96ec00f Constantinos Venetsanopoulos
     ProxyPass        /static !
445 a96ec00f Constantinos Venetsanopoulos
     ProxyPass        / http://localhost:8080/ retry=0
446 a96ec00f Constantinos Venetsanopoulos
     ProxyPassReverse / http://localhost:8080/
447 a96ec00f Constantinos Venetsanopoulos
448 a96ec00f Constantinos Venetsanopoulos
     SSLEngine on
449 a96ec00f Constantinos Venetsanopoulos
     SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
450 a96ec00f Constantinos Venetsanopoulos
     SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
451 a96ec00f Constantinos Venetsanopoulos
   </VirtualHost>
452 a96ec00f Constantinos Venetsanopoulos
   </IfModule>
453 a96ec00f Constantinos Venetsanopoulos
454 a96ec00f Constantinos Venetsanopoulos
As in node1, enable sites and modules by running:
455 a96ec00f Constantinos Venetsanopoulos
456 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
457 a96ec00f Constantinos Venetsanopoulos
458 a96ec00f Constantinos Venetsanopoulos
   # a2enmod ssl
459 a96ec00f Constantinos Venetsanopoulos
   # a2enmod rewrite
460 a96ec00f Constantinos Venetsanopoulos
   # a2dissite default
461 a96ec00f Constantinos Venetsanopoulos
   # a2ensite synnefo
462 a96ec00f Constantinos Venetsanopoulos
   # a2ensite synnefo-ssl
463 a96ec00f Constantinos Venetsanopoulos
   # a2enmod headers
464 a96ec00f Constantinos Venetsanopoulos
   # a2enmod proxy_http
465 a96ec00f Constantinos Venetsanopoulos
466 4de94e15 Kostas Papadimitriou
.. warning:: Do NOT start/restart the server yet. If the server is running::
467 a96ec00f Constantinos Venetsanopoulos
468 4de94e15 Kostas Papadimitriou
       # /etc/init.d/apache2 stop
469 a96ec00f Constantinos Venetsanopoulos
470 a96ec00f Constantinos Venetsanopoulos
We are now ready with all general prerequisites for node2. Now that we have
471 a96ec00f Constantinos Venetsanopoulos
finished with all general prerequisites for both nodes, we can start installing
472 a96ec00f Constantinos Venetsanopoulos
the services. First, let's install Astakos on node1.
473 a96ec00f Constantinos Venetsanopoulos
474 a96ec00f Constantinos Venetsanopoulos
475 a96ec00f Constantinos Venetsanopoulos
Installation of Astakos on node1
476 a96ec00f Constantinos Venetsanopoulos
================================
477 a96ec00f Constantinos Venetsanopoulos
478 a96ec00f Constantinos Venetsanopoulos
To install astakos, grab the package from our repository (make sure  you made
479 a96ec00f Constantinos Venetsanopoulos
the additions needed in your ``/etc/apt/sources.list`` file, as described
480 a96ec00f Constantinos Venetsanopoulos
previously), by running:
481 a96ec00f Constantinos Venetsanopoulos
482 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
483 a96ec00f Constantinos Venetsanopoulos
484 a96ec00f Constantinos Venetsanopoulos
   # apt-get install snf-astakos-app
485 a96ec00f Constantinos Venetsanopoulos
486 a96ec00f Constantinos Venetsanopoulos
After successful installation of snf-astakos-app, make sure that also
487 a96ec00f Constantinos Venetsanopoulos
snf-webproject has been installed (marked as "Recommended" package). By default
488 a96ec00f Constantinos Venetsanopoulos
Debian installs "Recommended" packages, but if you have changed your
489 a96ec00f Constantinos Venetsanopoulos
configuration and the package didn't install automatically, you should
490 a96ec00f Constantinos Venetsanopoulos
explicitly install it manually running:
491 a96ec00f Constantinos Venetsanopoulos
492 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
493 a96ec00f Constantinos Venetsanopoulos
494 a96ec00f Constantinos Venetsanopoulos
   # apt-get install snf-webproject
495 a96ec00f Constantinos Venetsanopoulos
496 a96ec00f Constantinos Venetsanopoulos
The reason snf-webproject is "Recommended" and not a hard dependency, is to give
497 a96ec00f Constantinos Venetsanopoulos
the experienced administrator the ability to install synnefo in a custom made
498 a96ec00f Constantinos Venetsanopoulos
django project. This corner case concerns only very advanced users that know
499 a96ec00f Constantinos Venetsanopoulos
what they are doing and want to experiment with synnefo.
500 a96ec00f Constantinos Venetsanopoulos
501 04c1254b Constantinos Venetsanopoulos
502 04c1254b Constantinos Venetsanopoulos
.. _conf-astakos:
503 04c1254b Constantinos Venetsanopoulos
504 a96ec00f Constantinos Venetsanopoulos
Configuration of Astakos
505 a96ec00f Constantinos Venetsanopoulos
========================
506 a96ec00f Constantinos Venetsanopoulos
507 a96ec00f Constantinos Venetsanopoulos
Conf Files
508 a96ec00f Constantinos Venetsanopoulos
----------
509 a96ec00f Constantinos Venetsanopoulos
510 a96ec00f Constantinos Venetsanopoulos
After astakos is successfully installed, you will find the directory
511 a96ec00f Constantinos Venetsanopoulos
``/etc/synnefo`` and some configuration files inside it. The files contain
512 a96ec00f Constantinos Venetsanopoulos
commented configuration options, which are the default options. While installing
513 a96ec00f Constantinos Venetsanopoulos
new snf-* components, new configuration files will appear inside the directory.
514 a96ec00f Constantinos Venetsanopoulos
In this guide (and for all services), we will edit only the minimum necessary
515 a96ec00f Constantinos Venetsanopoulos
configuration options, to reflect our setup. Everything else will remain as is.
516 a96ec00f Constantinos Venetsanopoulos
517 a96ec00f Constantinos Venetsanopoulos
After getting familiar with synnefo, you will be able to customize the software
518 a96ec00f Constantinos Venetsanopoulos
as you wish and fits your needs. Many options are available, to empower the
519 a96ec00f Constantinos Venetsanopoulos
administrator with extensively customizable setups.
520 a96ec00f Constantinos Venetsanopoulos
521 a96ec00f Constantinos Venetsanopoulos
For the snf-webproject component (installed as an astakos dependency), we
522 a96ec00f Constantinos Venetsanopoulos
need the following:
523 a96ec00f Constantinos Venetsanopoulos
524 a96ec00f Constantinos Venetsanopoulos
Edit ``/etc/synnefo/10-snf-webproject-database.conf``. You will need to
525 a96ec00f Constantinos Venetsanopoulos
uncomment and edit the ``DATABASES`` block to reflect our database:
526 a96ec00f Constantinos Venetsanopoulos
527 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
528 a96ec00f Constantinos Venetsanopoulos
529 a96ec00f Constantinos Venetsanopoulos
   DATABASES = {
530 a96ec00f Constantinos Venetsanopoulos
    'default': {
531 a96ec00f Constantinos Venetsanopoulos
        # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
532 a96ec00f Constantinos Venetsanopoulos
        'ENGINE': 'postgresql_psycopg2',
533 a96ec00f Constantinos Venetsanopoulos
         # ATTENTION: This *must* be the absolute path if using sqlite3.
534 a96ec00f Constantinos Venetsanopoulos
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
535 a96ec00f Constantinos Venetsanopoulos
        'NAME': 'snf_apps',
536 a96ec00f Constantinos Venetsanopoulos
        'USER': 'synnefo',                      # Not used with sqlite3.
537 4404e02e Ilias Tsitsimpis
        'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
538 a96ec00f Constantinos Venetsanopoulos
        # Set to empty string for localhost. Not used with sqlite3.
539 a96ec00f Constantinos Venetsanopoulos
        'HOST': '4.3.2.1',
540 a96ec00f Constantinos Venetsanopoulos
        # Set to empty string for default. Not used with sqlite3.
541 a96ec00f Constantinos Venetsanopoulos
        'PORT': '5432',
542 a96ec00f Constantinos Venetsanopoulos
    }
543 a96ec00f Constantinos Venetsanopoulos
   }
544 a96ec00f Constantinos Venetsanopoulos
545 a96ec00f Constantinos Venetsanopoulos
Edit ``/etc/synnefo/10-snf-webproject-deploy.conf``. Uncomment and edit
546 a96ec00f Constantinos Venetsanopoulos
``SECRET_KEY``. This is a django specific setting which is used to provide a
547 a96ec00f Constantinos Venetsanopoulos
seed in secret-key hashing algorithms. Set this to a random string of your
548 a96ec00f Constantinos Venetsanopoulos
choise and keep it private:
549 a96ec00f Constantinos Venetsanopoulos
550 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
551 a96ec00f Constantinos Venetsanopoulos
552 a96ec00f Constantinos Venetsanopoulos
   SECRET_KEY = 'sy6)mw6a7x%n)-example_secret_key#zzk4jo6f2=uqu!1o%)'
553 a96ec00f Constantinos Venetsanopoulos
554 a96ec00f Constantinos Venetsanopoulos
For astakos specific configuration, edit the following options in
555 73ff1d54 Constantinos Venetsanopoulos
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :
556 a96ec00f Constantinos Venetsanopoulos
557 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
558 a96ec00f Constantinos Venetsanopoulos
559 4404e02e Ilias Tsitsimpis
   ASTAKOS_DEFAULT_ADMIN_EMAIL = None
560 4404e02e Ilias Tsitsimpis
561 a96ec00f Constantinos Venetsanopoulos
   ASTAKOS_COOKIE_DOMAIN = '.example.com'
562 a96ec00f Constantinos Venetsanopoulos
563 a96ec00f Constantinos Venetsanopoulos
   ASTAKOS_BASEURL = 'https://node1.example.com'
564 a96ec00f Constantinos Venetsanopoulos
565 4e3e3d24 Constantinos Venetsanopoulos
The ``ASTAKOS_COOKIE_DOMAIN`` should be the base url of our domain (for all
566 4e3e3d24 Constantinos Venetsanopoulos
services). ``ASTAKOS_BASEURL`` is the astakos home page.
567 5b6feb88 Vangelis Koukis
568 4404e02e Ilias Tsitsimpis
``ASTAKOS_DEFAULT_ADMIN_EMAIL`` refers to the administrator's email.
569 4404e02e Ilias Tsitsimpis
Every time a new account is created a notification is sent to this email.
570 4404e02e Ilias Tsitsimpis
For this we need access to a running mail server, so we have disabled
571 4404e02e Ilias Tsitsimpis
it for now by setting its value to None. For more informations on this,
572 4404e02e Ilias Tsitsimpis
read the relative :ref:`section <mail-server>`.
573 4404e02e Ilias Tsitsimpis
574 4e3e3d24 Constantinos Venetsanopoulos
.. note:: For the purpose of this guide, we don't enable recaptcha authentication.
575 4e3e3d24 Constantinos Venetsanopoulos
    If you would like to enable it, you have to edit the following options:
576 319b615d Ilias Tsitsimpis
577 319b615d Ilias Tsitsimpis
    .. code-block:: console
578 319b615d Ilias Tsitsimpis
579 319b615d Ilias Tsitsimpis
        ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
580 319b615d Ilias Tsitsimpis
        ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
581 319b615d Ilias Tsitsimpis
        ASTAKOS_RECAPTCHA_USE_SSL = True
582 319b615d Ilias Tsitsimpis
        ASTAKOS_RECAPTCHA_ENABLED = True
583 319b615d Ilias Tsitsimpis
584 319b615d Ilias Tsitsimpis
    For the ``ASTAKOS_RECAPTCHA_PUBLIC_KEY`` and ``ASTAKOS_RECAPTCHA_PRIVATE_KEY``
585 319b615d Ilias Tsitsimpis
    go to https://www.google.com/recaptcha/admin/create and create your own pair.
586 5b6feb88 Vangelis Koukis
587 610ff8cf Constantinos Venetsanopoulos
Then edit ``/etc/synnefo/20-snf-astakos-app-cloudbar.conf`` :
588 610ff8cf Constantinos Venetsanopoulos
589 610ff8cf Constantinos Venetsanopoulos
.. code-block:: console
590 610ff8cf Constantinos Venetsanopoulos
591 610ff8cf Constantinos Venetsanopoulos
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
592 610ff8cf Constantinos Venetsanopoulos
593 610ff8cf Constantinos Venetsanopoulos
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/im/get_services'
594 610ff8cf Constantinos Venetsanopoulos
595 610ff8cf Constantinos Venetsanopoulos
   CLOUDBAR_MENU_URL = 'https://node1.example.com/im/get_menu'
596 610ff8cf Constantinos Venetsanopoulos
597 610ff8cf Constantinos Venetsanopoulos
Those settings have to do with the black cloudbar endpoints and will be described
598 610ff8cf Constantinos Venetsanopoulos
in more detail later on in this guide. For now, just edit the domain to point at
599 610ff8cf Constantinos Venetsanopoulos
node1 which is where we have installed Astakos.
600 610ff8cf Constantinos Venetsanopoulos
601 d189d11c Constantinos Venetsanopoulos
If you are an advanced user and want to use the Shibboleth Authentication method,
602 d189d11c Constantinos Venetsanopoulos
read the relative :ref:`section <shibboleth-auth>`.
603 d2a9f85f Sofia Papagiannaki
604 5a2d4e43 Christos Stavrakakis
.. note:: Because Cyclades and Astakos are running on the same machine
605 5a2d4e43 Christos Stavrakakis
    in our example, we have to deactivate the CSRF verification. We can do so
606 5a2d4e43 Christos Stavrakakis
    by adding to
607 5a2d4e43 Christos Stavrakakis
    ``/etc/synnefo/99-local.conf``:
608 5a2d4e43 Christos Stavrakakis
609 5a2d4e43 Christos Stavrakakis
    .. code-block:: console
610 5a2d4e43 Christos Stavrakakis
611 5a2d4e43 Christos Stavrakakis
        MIDDLEWARE_CLASSES.remove('django.middleware.csrf.CsrfViewMiddleware')
612 5a2d4e43 Christos Stavrakakis
        TEMPLATE_CONTEXT_PROCESSORS.remove('django.core.context_processors.csrf')
613 5a2d4e43 Christos Stavrakakis
614 4e3e3d24 Constantinos Venetsanopoulos
Enable Pooling
615 4e3e3d24 Constantinos Venetsanopoulos
--------------
616 4e3e3d24 Constantinos Venetsanopoulos
617 4e3e3d24 Constantinos Venetsanopoulos
This section can be bypassed, but we strongly recommend you apply the following,
618 4e3e3d24 Constantinos Venetsanopoulos
since they result in a significant performance boost.
619 4e3e3d24 Constantinos Venetsanopoulos
620 4e3e3d24 Constantinos Venetsanopoulos
Synnefo includes a pooling DBAPI driver for PostgreSQL, as a thin wrapper
621 4e3e3d24 Constantinos Venetsanopoulos
around Psycopg2. This allows independent Django requests to reuse pooled DB
622 4e3e3d24 Constantinos Venetsanopoulos
connections, with significant performance gains.
623 4e3e3d24 Constantinos Venetsanopoulos
624 4e3e3d24 Constantinos Venetsanopoulos
To use, first monkey-patch psycopg2. For Django, run this before the
625 4e3e3d24 Constantinos Venetsanopoulos
``DATABASES`` setting in ``/etc/synnefo/10-snf-webproject-database.conf``:
626 4e3e3d24 Constantinos Venetsanopoulos
627 4e3e3d24 Constantinos Venetsanopoulos
.. code-block:: console
628 4e3e3d24 Constantinos Venetsanopoulos
629 4e3e3d24 Constantinos Venetsanopoulos
   from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2
630 4e3e3d24 Constantinos Venetsanopoulos
   monkey_patch_psycopg2()
631 4e3e3d24 Constantinos Venetsanopoulos
632 0167faa4 Constantinos Venetsanopoulos
If running with greenlets, we should modify psycopg2 behavior, so it works
633 0167faa4 Constantinos Venetsanopoulos
properly in a greenlet context:
634 4e3e3d24 Constantinos Venetsanopoulos
635 4e3e3d24 Constantinos Venetsanopoulos
.. code-block:: console
636 4e3e3d24 Constantinos Venetsanopoulos
637 4e3e3d24 Constantinos Venetsanopoulos
   from synnefo.lib.db.psyco_gevent import make_psycopg_green
638 4e3e3d24 Constantinos Venetsanopoulos
   make_psycopg_green()
639 4e3e3d24 Constantinos Venetsanopoulos
640 4e3e3d24 Constantinos Venetsanopoulos
Use the Psycopg2 driver as usual. For Django, this means using
641 4e3e3d24 Constantinos Venetsanopoulos
``django.db.backends.postgresql_psycopg2`` without any modifications. To enable
642 4e3e3d24 Constantinos Venetsanopoulos
connection pooling, pass a nonzero ``synnefo_poolsize`` option to the DBAPI
643 4e3e3d24 Constantinos Venetsanopoulos
driver, through ``DATABASES.OPTIONS`` in django.
644 4e3e3d24 Constantinos Venetsanopoulos
645 4e3e3d24 Constantinos Venetsanopoulos
All the above will result in an ``/etc/synnefo/10-snf-webproject-database.conf``
646 4e3e3d24 Constantinos Venetsanopoulos
file that looks like this:
647 4e3e3d24 Constantinos Venetsanopoulos
648 4e3e3d24 Constantinos Venetsanopoulos
.. code-block:: console
649 4e3e3d24 Constantinos Venetsanopoulos
650 4e3e3d24 Constantinos Venetsanopoulos
   # Monkey-patch psycopg2
651 4e3e3d24 Constantinos Venetsanopoulos
   from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2
652 4e3e3d24 Constantinos Venetsanopoulos
   monkey_patch_psycopg2()
653 4e3e3d24 Constantinos Venetsanopoulos
654 4e3e3d24 Constantinos Venetsanopoulos
   # If running with greenlets
655 4e3e3d24 Constantinos Venetsanopoulos
   from synnefo.lib.db.psyco_gevent import make_psycopg_green
656 4e3e3d24 Constantinos Venetsanopoulos
   make_psycopg_green()
657 4e3e3d24 Constantinos Venetsanopoulos
658 4e3e3d24 Constantinos Venetsanopoulos
   DATABASES = {
659 4e3e3d24 Constantinos Venetsanopoulos
    'default': {
660 4e3e3d24 Constantinos Venetsanopoulos
        # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
661 4e3e3d24 Constantinos Venetsanopoulos
        'ENGINE': 'postgresql_psycopg2',
662 4e3e3d24 Constantinos Venetsanopoulos
        'OPTIONS': {'synnefo_poolsize': 8},
663 4e3e3d24 Constantinos Venetsanopoulos
664 4e3e3d24 Constantinos Venetsanopoulos
         # ATTENTION: This *must* be the absolute path if using sqlite3.
665 4e3e3d24 Constantinos Venetsanopoulos
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
666 4e3e3d24 Constantinos Venetsanopoulos
        'NAME': 'snf_apps',
667 4e3e3d24 Constantinos Venetsanopoulos
        'USER': 'synnefo',                      # Not used with sqlite3.
668 4e3e3d24 Constantinos Venetsanopoulos
        'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
669 4e3e3d24 Constantinos Venetsanopoulos
        # Set to empty string for localhost. Not used with sqlite3.
670 4e3e3d24 Constantinos Venetsanopoulos
        'HOST': '4.3.2.1',
671 4e3e3d24 Constantinos Venetsanopoulos
        # Set to empty string for default. Not used with sqlite3.
672 4e3e3d24 Constantinos Venetsanopoulos
        'PORT': '5432',
673 4e3e3d24 Constantinos Venetsanopoulos
    }
674 4e3e3d24 Constantinos Venetsanopoulos
   }
675 5a2d4e43 Christos Stavrakakis
676 a96ec00f Constantinos Venetsanopoulos
Database Initialization
677 a96ec00f Constantinos Venetsanopoulos
-----------------------
678 a96ec00f Constantinos Venetsanopoulos
679 de20a465 Constantinos Venetsanopoulos
After configuration is done, we initialize the database by running:
680 5b6feb88 Vangelis Koukis
681 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
682 a96ec00f Constantinos Venetsanopoulos
683 a96ec00f Constantinos Venetsanopoulos
   # snf-manage syncdb
684 a96ec00f Constantinos Venetsanopoulos
685 a96ec00f Constantinos Venetsanopoulos
At this example we don't need to create a django superuser, so we select
686 a96ec00f Constantinos Venetsanopoulos
``[no]`` to the question. After a successful sync, we run the migration needed
687 a96ec00f Constantinos Venetsanopoulos
for astakos:
688 5b6feb88 Vangelis Koukis
689 5b6feb88 Vangelis Koukis
.. code-block:: console
690 5b6feb88 Vangelis Koukis
691 a96ec00f Constantinos Venetsanopoulos
   # snf-manage migrate im
692 a96ec00f Constantinos Venetsanopoulos
693 de20a465 Constantinos Venetsanopoulos
Then, we load the pre-defined user groups
694 d2a9f85f Sofia Papagiannaki
695 d2a9f85f Sofia Papagiannaki
.. code-block:: console
696 d2a9f85f Sofia Papagiannaki
697 d2a9f85f Sofia Papagiannaki
   # snf-manage loaddata groups
698 d2a9f85f Sofia Papagiannaki
699 de20a465 Constantinos Venetsanopoulos
.. _services-reg:
700 de20a465 Constantinos Venetsanopoulos
701 de20a465 Constantinos Venetsanopoulos
Services Registration
702 de20a465 Constantinos Venetsanopoulos
---------------------
703 de20a465 Constantinos Venetsanopoulos
704 de20a465 Constantinos Venetsanopoulos
When the database is ready, we configure the elements of the Astakos cloudbar,
705 de20a465 Constantinos Venetsanopoulos
to point to our future services:
706 de20a465 Constantinos Venetsanopoulos
707 de20a465 Constantinos Venetsanopoulos
.. code-block:: console
708 de20a465 Constantinos Venetsanopoulos
709 62b10e3f Constantinos Venetsanopoulos
   # snf-manage service-add "~okeanos home" https://node1.example.com/im/ home-icon.png
710 62b10e3f Constantinos Venetsanopoulos
   # snf-manage service-add "cyclades" https://node1.example.com/ui/
711 62b10e3f Constantinos Venetsanopoulos
   # snf-manage service-add "pithos+" https://node2.example.com/ui/
712 de20a465 Constantinos Venetsanopoulos
713 de20a465 Constantinos Venetsanopoulos
Servers Initialization
714 de20a465 Constantinos Venetsanopoulos
----------------------
715 de20a465 Constantinos Venetsanopoulos
716 de20a465 Constantinos Venetsanopoulos
Finally, we initialize the servers on node1:
717 de20a465 Constantinos Venetsanopoulos
718 de20a465 Constantinos Venetsanopoulos
.. code-block:: console
719 de20a465 Constantinos Venetsanopoulos
720 de20a465 Constantinos Venetsanopoulos
   root@node1:~ # /etc/init.d/gunicorn restart
721 de20a465 Constantinos Venetsanopoulos
   root@node1:~ # /etc/init.d/apache2 restart
722 de20a465 Constantinos Venetsanopoulos
723 de20a465 Constantinos Venetsanopoulos
We have now finished the Astakos setup. Let's test it now.
724 a96ec00f Constantinos Venetsanopoulos
725 a96ec00f Constantinos Venetsanopoulos
726 a96ec00f Constantinos Venetsanopoulos
Testing of Astakos
727 a96ec00f Constantinos Venetsanopoulos
==================
728 a96ec00f Constantinos Venetsanopoulos
729 a96ec00f Constantinos Venetsanopoulos
Open your favorite browser and go to:
730 a96ec00f Constantinos Venetsanopoulos
731 a96ec00f Constantinos Venetsanopoulos
``http://node1.example.com/im``
732 a96ec00f Constantinos Venetsanopoulos
733 a96ec00f Constantinos Venetsanopoulos
If this redirects you to ``https://node1.example.com/im`` and you can see
734 a96ec00f Constantinos Venetsanopoulos
the "welcome" door of Astakos, then you have successfully setup Astakos.
735 a96ec00f Constantinos Venetsanopoulos
736 a96ec00f Constantinos Venetsanopoulos
Let's create our first user. At the homepage click the "CREATE ACCOUNT" button
737 a96ec00f Constantinos Venetsanopoulos
and fill all your data at the sign up form. Then click "SUBMIT". You should now
738 a96ec00f Constantinos Venetsanopoulos
see a green box on the top, which informs you that you made a successful request
739 caa6c07d Constantinos Venetsanopoulos
and the request has been sent to the administrators. So far so good, let's assume
740 caa6c07d Constantinos Venetsanopoulos
that you created the user with username ``user@example.com``.
741 a96ec00f Constantinos Venetsanopoulos
742 a96ec00f Constantinos Venetsanopoulos
Now we need to activate that user. Return to a command prompt at node1 and run:
743 a96ec00f Constantinos Venetsanopoulos
744 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
745 a96ec00f Constantinos Venetsanopoulos
746 62b10e3f Constantinos Venetsanopoulos
   root@node1:~ # snf-manage user-list
747 a96ec00f Constantinos Venetsanopoulos
748 a96ec00f Constantinos Venetsanopoulos
This command should show you a list with only one user; the one we just created.
749 a96ec00f Constantinos Venetsanopoulos
This user should have an id with a value of ``1``. It should also have an
750 a96ec00f Constantinos Venetsanopoulos
"active" status with the value of ``0`` (inactive). Now run:
751 a96ec00f Constantinos Venetsanopoulos
752 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
753 a96ec00f Constantinos Venetsanopoulos
754 18e300b2 Christos Stavrakakis
   root@node1:~ # snf-manage user-update --set-active 1
755 a96ec00f Constantinos Venetsanopoulos
756 a96ec00f Constantinos Venetsanopoulos
This modifies the active value to ``1``, and actually activates the user.
757 a96ec00f Constantinos Venetsanopoulos
When running in production, the activation is done automatically with different
758 a96ec00f Constantinos Venetsanopoulos
types of moderation, that Astakos supports. You can see the moderation methods
759 a96ec00f Constantinos Venetsanopoulos
(by invitation, whitelists, matching regexp, etc.) at the Astakos specific
760 f846d8df Constantinos Venetsanopoulos
documentation. In production, you can also manually activate a user, by sending
761 f846d8df Constantinos Venetsanopoulos
him/her an activation email. See how to do this at the :ref:`User
762 f846d8df Constantinos Venetsanopoulos
activation <user_activation>` section.
763 a96ec00f Constantinos Venetsanopoulos
764 a96ec00f Constantinos Venetsanopoulos
Now let's go back to the homepage. Open ``http://node1.example.com/im`` with
765 a96ec00f Constantinos Venetsanopoulos
your browser again. Try to sign in using your new credentials. If the astakos
766 a96ec00f Constantinos Venetsanopoulos
menu appears and you can see your profile, then you have successfully setup
767 a96ec00f Constantinos Venetsanopoulos
Astakos.
768 a96ec00f Constantinos Venetsanopoulos
769 a96ec00f Constantinos Venetsanopoulos
Let's continue to install Pithos+ now.
770 a96ec00f Constantinos Venetsanopoulos
771 a96ec00f Constantinos Venetsanopoulos
772 a96ec00f Constantinos Venetsanopoulos
Installation of Pithos+ on node2
773 a96ec00f Constantinos Venetsanopoulos
================================
774 a96ec00f Constantinos Venetsanopoulos
775 138253bc Constantinos Venetsanopoulos
To install pithos+, grab the packages from our repository (make sure  you made
776 a96ec00f Constantinos Venetsanopoulos
the additions needed in your ``/etc/apt/sources.list`` file, as described
777 a96ec00f Constantinos Venetsanopoulos
previously), by running:
778 a96ec00f Constantinos Venetsanopoulos
779 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
780 a96ec00f Constantinos Venetsanopoulos
781 a96ec00f Constantinos Venetsanopoulos
   # apt-get install snf-pithos-app
782 a96ec00f Constantinos Venetsanopoulos
783 a96ec00f Constantinos Venetsanopoulos
After successful installation of snf-pithos-app, make sure that also
784 a96ec00f Constantinos Venetsanopoulos
snf-webproject has been installed (marked as "Recommended" package). Refer to
785 a96ec00f Constantinos Venetsanopoulos
the "Installation of Astakos on node1" section, if you don't remember why this
786 138253bc Constantinos Venetsanopoulos
should happen. Now, install the pithos web interface:
787 a96ec00f Constantinos Venetsanopoulos
788 138253bc Constantinos Venetsanopoulos
.. code-block:: console
789 138253bc Constantinos Venetsanopoulos
790 138253bc Constantinos Venetsanopoulos
   # apt-get install snf-pithos-webclient
791 138253bc Constantinos Venetsanopoulos
792 138253bc Constantinos Venetsanopoulos
This package provides the standalone pithos web client. The web client is the
793 138253bc Constantinos Venetsanopoulos
web UI for pithos+ and will be accessible by clicking "pithos+" on the Astakos
794 138253bc Constantinos Venetsanopoulos
interface's cloudbar, at the top of the Astakos homepage.
795 a96ec00f Constantinos Venetsanopoulos
796 caa6c07d Constantinos Venetsanopoulos
797 04c1254b Constantinos Venetsanopoulos
.. _conf-pithos:
798 04c1254b Constantinos Venetsanopoulos
799 a96ec00f Constantinos Venetsanopoulos
Configuration of Pithos+
800 a96ec00f Constantinos Venetsanopoulos
========================
801 a96ec00f Constantinos Venetsanopoulos
802 a96ec00f Constantinos Venetsanopoulos
Conf Files
803 a96ec00f Constantinos Venetsanopoulos
----------
804 a96ec00f Constantinos Venetsanopoulos
805 a96ec00f Constantinos Venetsanopoulos
After pithos+ is successfully installed, you will find the directory
806 a96ec00f Constantinos Venetsanopoulos
``/etc/synnefo`` and some configuration files inside it, as you did in node1
807 a96ec00f Constantinos Venetsanopoulos
after installation of astakos. Here, you will not have to change anything that
808 a96ec00f Constantinos Venetsanopoulos
has to do with snf-common or snf-webproject. Everything is set at node1. You
809 138253bc Constantinos Venetsanopoulos
only need to change settings that have to do with pithos+. Specifically:
810 a96ec00f Constantinos Venetsanopoulos
811 a96ec00f Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-pithos-app-settings.conf``. There you need to set
812 11c16930 Ilias Tsitsimpis
this options:
813 a96ec00f Constantinos Venetsanopoulos
814 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
815 a96ec00f Constantinos Venetsanopoulos
816 a96ec00f Constantinos Venetsanopoulos
   PITHOS_BACKEND_DB_CONNECTION = 'postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos'
817 a96ec00f Constantinos Venetsanopoulos
818 a96ec00f Constantinos Venetsanopoulos
   PITHOS_BACKEND_BLOCK_PATH = '/srv/pithos/data'
819 bdfd94c9 Constantinos Venetsanopoulos
820 01bd066a Sofia Papagiannaki
   PITHOS_AUTHENTICATION_URL = 'https://node1.example.com/im/authenticate'
821 01bd066a Sofia Papagiannaki
   PITHOS_AUTHENTICATION_USERS = None
822 a96ec00f Constantinos Venetsanopoulos
823 3e6d0710 Constantinos Venetsanopoulos
   PITHOS_SERVICE_TOKEN = 'pithos_service_token22w=='
824 02d94254 Sofia Papagiannaki
   PITHOS_USER_CATALOG_URL = 'http://node1.example.com/user_catalogs'
825 02d94254 Sofia Papagiannaki
   PITHOS_USER_FEEDBACK_URL = 'http://node1.example.com/feedback'
826 02d94254 Sofia Papagiannaki
   PITHOS_USER_LOGIN_URL = 'http://node1.example.com/login'
827 02d94254 Sofia Papagiannaki
828 3e6d0710 Constantinos Venetsanopoulos
829 bdfd94c9 Constantinos Venetsanopoulos
The ``PITHOS_BACKEND_DB_CONNECTION`` option tells to the pithos+ app where to
830 bdfd94c9 Constantinos Venetsanopoulos
find the pithos+ backend database. Above we tell pithos+ that its database is
831 bdfd94c9 Constantinos Venetsanopoulos
``snf_pithos`` at node1 and to connect as user ``synnefo`` with password
832 bdfd94c9 Constantinos Venetsanopoulos
``example_passw0rd``.  All those settings where setup during node1's "Database
833 bdfd94c9 Constantinos Venetsanopoulos
setup" section.
834 a96ec00f Constantinos Venetsanopoulos
835 bdfd94c9 Constantinos Venetsanopoulos
The ``PITHOS_BACKEND_BLOCK_PATH`` option tells to the pithos+ app where to find
836 bdfd94c9 Constantinos Venetsanopoulos
the pithos+ backend data. Above we tell pithos+ to store its data under
837 a96ec00f Constantinos Venetsanopoulos
``/srv/pithos/data``, which is visible by both nodes. We have already setup this
838 a96ec00f Constantinos Venetsanopoulos
directory at node1's "Pithos+ data directory setup" section.
839 a96ec00f Constantinos Venetsanopoulos
840 8f85321e Sofia Papagiannaki
The ``PITHOS_AUTHENTICATION_URL`` option tells to the pithos+ app in which URI
841 bdfd94c9 Constantinos Venetsanopoulos
is available the astakos authentication api. If not set, pithos+ tries to
842 8f85321e Sofia Papagiannaki
authenticate using the ``PITHOS_AUTHENTICATION_USERS`` user pool.
843 8f85321e Sofia Papagiannaki
844 3e6d0710 Constantinos Venetsanopoulos
The ``PITHOS_SERVICE_TOKEN`` should be the Pithos+ token returned by running on
845 3e6d0710 Constantinos Venetsanopoulos
the Astakos node (node1 in our case):
846 3e6d0710 Constantinos Venetsanopoulos
847 3e6d0710 Constantinos Venetsanopoulos
.. code-block:: console
848 3e6d0710 Constantinos Venetsanopoulos
849 62b10e3f Constantinos Venetsanopoulos
   # snf-manage service-list
850 3e6d0710 Constantinos Venetsanopoulos
851 3e6d0710 Constantinos Venetsanopoulos
The token has been generated automatically during the :ref:`Pithos+ service
852 3e6d0710 Constantinos Venetsanopoulos
registration <services-reg>`.
853 3e6d0710 Constantinos Venetsanopoulos
854 138253bc Constantinos Venetsanopoulos
Then we need to setup the web UI and connect it to astakos. To do so, edit
855 138253bc Constantinos Venetsanopoulos
``/etc/synnefo/20-snf-pithos-webclient-settings.conf``:
856 138253bc Constantinos Venetsanopoulos
857 138253bc Constantinos Venetsanopoulos
.. code-block:: console
858 138253bc Constantinos Venetsanopoulos
859 138253bc Constantinos Venetsanopoulos
   PITHOS_UI_LOGIN_URL = "https://node1.example.com/im/login?next="
860 02d94254 Sofia Papagiannaki
   PITHOS_UI_FEEDBACK_URL = "https://node2.example.com/feedback"
861 138253bc Constantinos Venetsanopoulos
862 138253bc Constantinos Venetsanopoulos
The ``PITHOS_UI_LOGIN_URL`` option tells the client where to redirect you, if
863 138253bc Constantinos Venetsanopoulos
you are not logged in. The ``PITHOS_UI_FEEDBACK_URL`` option points at the
864 138253bc Constantinos Venetsanopoulos
pithos+ feedback form. Astakos already provides a generic feedback form for all
865 138253bc Constantinos Venetsanopoulos
services, so we use this one.
866 138253bc Constantinos Venetsanopoulos
867 138253bc Constantinos Venetsanopoulos
Then edit ``/etc/synnefo/20-snf-pithos-webclient-cloudbar.conf``, to connect the
868 138253bc Constantinos Venetsanopoulos
pithos+ web UI with the astakos web UI (through the top cloudbar):
869 138253bc Constantinos Venetsanopoulos
870 138253bc Constantinos Venetsanopoulos
.. code-block:: console
871 138253bc Constantinos Venetsanopoulos
872 138253bc Constantinos Venetsanopoulos
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
873 de20a465 Constantinos Venetsanopoulos
   PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE = '3'
874 138253bc Constantinos Venetsanopoulos
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/im/get_services'
875 138253bc Constantinos Venetsanopoulos
   CLOUDBAR_MENU_URL = 'https://node1.example.com/im/get_menu'
876 138253bc Constantinos Venetsanopoulos
877 138253bc Constantinos Venetsanopoulos
The ``CLOUDBAR_LOCATION`` tells the client where to find the astakos common
878 138253bc Constantinos Venetsanopoulos
cloudbar.
879 138253bc Constantinos Venetsanopoulos
880 de20a465 Constantinos Venetsanopoulos
The ``PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE`` points to an already registered
881 de20a465 Constantinos Venetsanopoulos
Astakos service. You can see all :ref:`registered services <services-reg>` by
882 de20a465 Constantinos Venetsanopoulos
running on the Astakos node (node1):
883 de20a465 Constantinos Venetsanopoulos
884 de20a465 Constantinos Venetsanopoulos
.. code-block:: console
885 de20a465 Constantinos Venetsanopoulos
886 62b10e3f Constantinos Venetsanopoulos
   # snf-manage service-list
887 de20a465 Constantinos Venetsanopoulos
888 de20a465 Constantinos Venetsanopoulos
The value of ``PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE`` should be the pithos service's
889 de20a465 Constantinos Venetsanopoulos
``id`` as shown by the above command, in our case ``3``.
890 138253bc Constantinos Venetsanopoulos
891 138253bc Constantinos Venetsanopoulos
The ``CLOUDBAR_SERVICES_URL`` and ``CLOUDBAR_MENU_URL`` options are used by the
892 138253bc Constantinos Venetsanopoulos
pithos+ web client to get from astakos all the information needed to fill its
893 bdfd94c9 Constantinos Venetsanopoulos
own cloudbar. So we put our astakos deployment urls there.
894 138253bc Constantinos Venetsanopoulos
895 4e3e3d24 Constantinos Venetsanopoulos
Pooling and Greenlets
896 4e3e3d24 Constantinos Venetsanopoulos
---------------------
897 4e3e3d24 Constantinos Venetsanopoulos
898 4e3e3d24 Constantinos Venetsanopoulos
Pithos is pooling-ready without the need of further configuration, because it
899 4e3e3d24 Constantinos Venetsanopoulos
doesn't use a Django DB. It pools HTTP connections to Astakos and pithos
900 4e3e3d24 Constantinos Venetsanopoulos
backend objects for access to the Pithos DB.
901 4e3e3d24 Constantinos Venetsanopoulos
902 4e3e3d24 Constantinos Venetsanopoulos
However, as in Astakos, if running with Greenlets, it is also recommended to
903 4e3e3d24 Constantinos Venetsanopoulos
modify psycopg2 behavior so it works properly in a greenlet context. This means
904 4e3e3d24 Constantinos Venetsanopoulos
adding the following lines at the top of your
905 4e3e3d24 Constantinos Venetsanopoulos
``/etc/synnefo/10-snf-webproject-database.conf`` file:
906 4e3e3d24 Constantinos Venetsanopoulos
907 4e3e3d24 Constantinos Venetsanopoulos
.. code-block:: console
908 4e3e3d24 Constantinos Venetsanopoulos
909 4e3e3d24 Constantinos Venetsanopoulos
   from synnefo.lib.db.psyco_gevent import make_psycopg_green
910 4e3e3d24 Constantinos Venetsanopoulos
   make_psycopg_green()
911 4e3e3d24 Constantinos Venetsanopoulos
912 4e3e3d24 Constantinos Venetsanopoulos
Furthermore, add the ``--worker-class=gevent`` argument on your
913 4e3e3d24 Constantinos Venetsanopoulos
``/etc/gunicorn.d/synnefo`` configuration file. The file should look something like
914 4e3e3d24 Constantinos Venetsanopoulos
this:
915 4e3e3d24 Constantinos Venetsanopoulos
916 4e3e3d24 Constantinos Venetsanopoulos
.. code-block:: console
917 4e3e3d24 Constantinos Venetsanopoulos
918 4e3e3d24 Constantinos Venetsanopoulos
   CONFIG = {
919 4e3e3d24 Constantinos Venetsanopoulos
    'mode': 'django',
920 4e3e3d24 Constantinos Venetsanopoulos
    'environment': {
921 4e3e3d24 Constantinos Venetsanopoulos
      'DJANGO_SETTINGS_MODULE': 'synnefo.settings',
922 4e3e3d24 Constantinos Venetsanopoulos
    },
923 4e3e3d24 Constantinos Venetsanopoulos
    'working_dir': '/etc/synnefo',
924 4e3e3d24 Constantinos Venetsanopoulos
    'user': 'www-data',
925 4e3e3d24 Constantinos Venetsanopoulos
    'group': 'www-data',
926 4e3e3d24 Constantinos Venetsanopoulos
    'args': (
927 4e3e3d24 Constantinos Venetsanopoulos
      '--bind=127.0.0.1:8080',
928 4e3e3d24 Constantinos Venetsanopoulos
      '--workers=4',
929 4e3e3d24 Constantinos Venetsanopoulos
      '--worker-class=gevent',
930 4e3e3d24 Constantinos Venetsanopoulos
      '--log-level=debug',
931 4e3e3d24 Constantinos Venetsanopoulos
      '--timeout=43200'
932 4e3e3d24 Constantinos Venetsanopoulos
    ),
933 4e3e3d24 Constantinos Venetsanopoulos
   }
934 4e3e3d24 Constantinos Venetsanopoulos
935 a96ec00f Constantinos Venetsanopoulos
Servers Initialization
936 a96ec00f Constantinos Venetsanopoulos
----------------------
937 a96ec00f Constantinos Venetsanopoulos
938 a96ec00f Constantinos Venetsanopoulos
After configuration is done, we initialize the servers on node2:
939 a96ec00f Constantinos Venetsanopoulos
940 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
941 a96ec00f Constantinos Venetsanopoulos
942 a96ec00f Constantinos Venetsanopoulos
   root@node2:~ # /etc/init.d/gunicorn restart
943 a96ec00f Constantinos Venetsanopoulos
   root@node2:~ # /etc/init.d/apache2 restart
944 a96ec00f Constantinos Venetsanopoulos
945 a96ec00f Constantinos Venetsanopoulos
You have now finished the Pithos+ setup. Let's test it now.
946 a96ec00f Constantinos Venetsanopoulos
947 a96ec00f Constantinos Venetsanopoulos
948 a96ec00f Constantinos Venetsanopoulos
Testing of Pithos+
949 a96ec00f Constantinos Venetsanopoulos
==================
950 a96ec00f Constantinos Venetsanopoulos
951 bdfd94c9 Constantinos Venetsanopoulos
Open your browser and go to the Astakos homepage:
952 bdfd94c9 Constantinos Venetsanopoulos
953 bdfd94c9 Constantinos Venetsanopoulos
``http://node1.example.com/im``
954 bdfd94c9 Constantinos Venetsanopoulos
955 bdfd94c9 Constantinos Venetsanopoulos
Login, and you will see your profile page. Now, click the "pithos+" link on the
956 bdfd94c9 Constantinos Venetsanopoulos
top black cloudbar. If everything was setup correctly, this will redirect you
957 bdfd94c9 Constantinos Venetsanopoulos
to:
958 bdfd94c9 Constantinos Venetsanopoulos
959 bdfd94c9 Constantinos Venetsanopoulos
``https://node2.example.com/ui``
960 bdfd94c9 Constantinos Venetsanopoulos
961 bdfd94c9 Constantinos Venetsanopoulos
and you will see the blue interface of the Pithos+ application.  Click the
962 bdfd94c9 Constantinos Venetsanopoulos
orange "Upload" button and upload your first file. If the file gets uploaded
963 bdfd94c9 Constantinos Venetsanopoulos
successfully, then this is your first sign of a successful Pithos+ installation.
964 bdfd94c9 Constantinos Venetsanopoulos
Go ahead and experiment with the interface to make sure everything works
965 bdfd94c9 Constantinos Venetsanopoulos
correctly.
966 bdfd94c9 Constantinos Venetsanopoulos
967 bdfd94c9 Constantinos Venetsanopoulos
You can also use the Pithos+ clients to sync data from your Windows PC or MAC.
968 bdfd94c9 Constantinos Venetsanopoulos
969 bdfd94c9 Constantinos Venetsanopoulos
If you don't stumble on any problems, then you have successfully installed
970 bdfd94c9 Constantinos Venetsanopoulos
Pithos+, which you can use as a standalone File Storage Service.
971 bdfd94c9 Constantinos Venetsanopoulos
972 bdfd94c9 Constantinos Venetsanopoulos
If you would like to do more, such as:
973 bdfd94c9 Constantinos Venetsanopoulos
974 bdfd94c9 Constantinos Venetsanopoulos
 * Spawning VMs
975 bdfd94c9 Constantinos Venetsanopoulos
 * Spawning VMs from Images stored on Pithos+
976 bdfd94c9 Constantinos Venetsanopoulos
 * Uploading your custom Images to Pithos+
977 bdfd94c9 Constantinos Venetsanopoulos
 * Spawning VMs from those custom Images
978 bdfd94c9 Constantinos Venetsanopoulos
 * Registering existing Pithos+ files as Images
979 7a8df455 Constantinos Venetsanopoulos
 * Connect VMs to the Internet
980 7a8df455 Constantinos Venetsanopoulos
 * Create Private Networks
981 7a8df455 Constantinos Venetsanopoulos
 * Add VMs to Private Networks
982 bdfd94c9 Constantinos Venetsanopoulos
983 bdfd94c9 Constantinos Venetsanopoulos
please continue with the rest of the guide.
984 a96ec00f Constantinos Venetsanopoulos
985 caa6c07d Constantinos Venetsanopoulos
986 7a8df455 Constantinos Venetsanopoulos
Cyclades (and Plankton) Prerequisites
987 7a8df455 Constantinos Venetsanopoulos
=====================================
988 bc055d09 Constantinos Venetsanopoulos
989 caa6c07d Constantinos Venetsanopoulos
Before proceeding with the Cyclades (and Plankton) installation, make sure you
990 caa6c07d Constantinos Venetsanopoulos
have successfully set up Astakos and Pithos+ first, because Cyclades depends
991 caa6c07d Constantinos Venetsanopoulos
on them. If you don't have a working Astakos and Pithos+ installation yet,
992 caa6c07d Constantinos Venetsanopoulos
please return to the :ref:`top <quick-install-admin-guide>` of this guide.
993 bc055d09 Constantinos Venetsanopoulos
994 caa6c07d Constantinos Venetsanopoulos
Besides Astakos and Pithos+, you will also need a number of additional working
995 caa6c07d Constantinos Venetsanopoulos
prerequisites, before you start the Cyclades installation.
996 bc055d09 Constantinos Venetsanopoulos
997 caa6c07d Constantinos Venetsanopoulos
Ganeti
998 7a8df455 Constantinos Venetsanopoulos
------
999 bc055d09 Constantinos Venetsanopoulos
1000 caa6c07d Constantinos Venetsanopoulos
`Ganeti <http://code.google.com/p/ganeti/>`_ handles the low level VM management
1001 caa6c07d Constantinos Venetsanopoulos
for Cyclades, so Cyclades requires a working Ganeti installation at the backend.
1002 caa6c07d Constantinos Venetsanopoulos
Please refer to the
1003 caa6c07d Constantinos Venetsanopoulos
`ganeti documentation <http://docs.ganeti.org/ganeti/2.5/html>`_ for all the
1004 bc055d09 Constantinos Venetsanopoulos
gory details. A successful Ganeti installation concludes with a working
1005 caa6c07d Constantinos Venetsanopoulos
:ref:`GANETI-MASTER <GANETI_NODES>` and a number of :ref:`GANETI-NODEs
1006 caa6c07d Constantinos Venetsanopoulos
<GANETI_NODES>`.
1007 caa6c07d Constantinos Venetsanopoulos
1008 caa6c07d Constantinos Venetsanopoulos
The above Ganeti cluster can run on different physical machines than node1 and
1009 caa6c07d Constantinos Venetsanopoulos
node2 and can scale independently, according to your needs.
1010 caa6c07d Constantinos Venetsanopoulos
1011 caa6c07d Constantinos Venetsanopoulos
For the purpose of this guide, we will assume that the :ref:`GANETI-MASTER
1012 caa6c07d Constantinos Venetsanopoulos
<GANETI_NODES>` runs on node1 and is VM-capable. Also, node2 is a
1013 caa6c07d Constantinos Venetsanopoulos
:ref:`GANETI-NODE <GANETI_NODES>` and is Master-capable and VM-capable too.
1014 caa6c07d Constantinos Venetsanopoulos
1015 caa6c07d Constantinos Venetsanopoulos
We highly recommend that you read the official Ganeti documentation, if you are
1016 caa6c07d Constantinos Venetsanopoulos
not familiar with Ganeti. If you are extremely impatient, you can result with
1017 cc16407a Dimitris Aragiorgis
the above assumed setup by running on both nodes:
1018 caa6c07d Constantinos Venetsanopoulos
1019 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1020 caa6c07d Constantinos Venetsanopoulos
1021 f1f5235e Dimitris Aragiorgis
   # apt-get install -t squeeze-backports ganeti2 ganeti-htools
1022 cc16407a Dimitris Aragiorgis
   # modprobe drbd minor_count=255 usermode_helper=/bin/true
1023 cc16407a Dimitris Aragiorgis
1024 cc16407a Dimitris Aragiorgis
Unfortunatelly, stock Ganeti doesn't support IP pool management yet (we are
1025 cc16407a Dimitris Aragiorgis
working hard to merge it upstream for Ganeti 2.7). Synnefo depends on the IP
1026 cc16407a Dimitris Aragiorgis
pool functionality of Ganeti, so you have to use GRNET's patches for now. To
1027 f1f5235e Dimitris Aragiorgis
do so you have to build your own package from source. Please clone our local
1028 f1f5235e Dimitris Aragiorgis
repo:
1029 cc16407a Dimitris Aragiorgis
1030 cc16407a Dimitris Aragiorgis
.. code-block:: console
1031 cc16407a Dimitris Aragiorgis
1032 cc16407a Dimitris Aragiorgis
   # git clone https://code.grnet.gr/git/ganeti-local
1033 cc16407a Dimitris Aragiorgis
   # cd ganeti-local
1034 f1f5235e Dimitris Aragiorgis
   # git checkout stable-2.6-ippool-hotplug-esi
1035 f1f5235e Dimitris Aragiorgis
   # git checkout debian-2.6
1036 f1f5235e Dimitris Aragiorgis
1037 f1f5235e Dimitris Aragiorgis
Then please check if you can complile ganeti:
1038 f1f5235e Dimitris Aragiorgis
1039 f1f5235e Dimitris Aragiorgis
.. code-block:: console
1040 f1f5235e Dimitris Aragiorgis
1041 f1f5235e Dimitris Aragiorgis
   # cd ganeti-local
1042 f1f5235e Dimitris Aragiorgis
   # ./automake.sh
1043 f1f5235e Dimitris Aragiorgis
   # ./configure
1044 f1f5235e Dimitris Aragiorgis
   # make
1045 f1f5235e Dimitris Aragiorgis
1046 f1f5235e Dimitris Aragiorgis
To do so you must have a correct build environment. Please refer to INSTALL
1047 f1f5235e Dimitris Aragiorgis
file in the source tree. Most of the packages needed are refered here:
1048 f1f5235e Dimitris Aragiorgis
1049 f1f5235e Dimitris Aragiorgis
.. code-block:: console
1050 f1f5235e Dimitris Aragiorgis
1051 f1f5235e Dimitris Aragiorgis
   #  apt-get install graphviz automake lvm2 ssh bridge-utils iproute iputils-arping \
1052 f1f5235e Dimitris Aragiorgis
                      ndisc6 python python-pyopenssl openssl \
1053 f1f5235e Dimitris Aragiorgis
                      python-pyparsing python-simplejson \
1054 f1f5235e Dimitris Aragiorgis
                      python-pyinotify python-pycurl socat \
1055 f1f5235e Dimitris Aragiorgis
                      python-elementtree kvm qemu-kvm \
1056 f1f5235e Dimitris Aragiorgis
                      ghc6 libghc6-json-dev libghc6-network-dev \
1057 f1f5235e Dimitris Aragiorgis
                      libghc6-parallel-dev libghc6-curl-dev \
1058 f1f5235e Dimitris Aragiorgis
                      libghc-quickcheck2-dev hscolour hlint
1059 f1f5235e Dimitris Aragiorgis
                      python-support python-paramiko \
1060 92e4d1c6 Dimitris Aragiorgis
                      python-fdsend python-ipaddr python-bitarray libjs-jquery fping pandoc
1061 f1f5235e Dimitris Aragiorgis
1062 92e4d1c6 Dimitris Aragiorgis
Now lets try to build the package:
1063 f1f5235e Dimitris Aragiorgis
1064 f1f5235e Dimitris Aragiorgis
.. code-block:: console
1065 f1f5235e Dimitris Aragiorgis
1066 f1f5235e Dimitris Aragiorgis
   # apt-get install git-buildpackage
1067 f1f5235e Dimitris Aragiorgis
   # mkdir ../build-area
1068 f1f5235e Dimitris Aragiorgis
   # git-buildpackage --git-upstream-branch=stable-2.6-ippool-hotplug-esi \
1069 f1f5235e Dimitris Aragiorgis
                   --git-debian-branch=debian-2.6 \
1070 cc16407a Dimitris Aragiorgis
                   --git-export=INDEX \
1071 cc16407a Dimitris Aragiorgis
                   --git-ignore-new
1072 cc16407a Dimitris Aragiorgis
1073 92e4d1c6 Dimitris Aragiorgis
To be able to sign the packages a key must be found in the system to comply to the
1074 92e4d1c6 Dimitris Aragiorgis
name and email of the last debian/changelog entry in debian branch. Please note
1075 92e4d1c6 Dimitris Aragiorgis
that signing is optional.
1076 92e4d1c6 Dimitris Aragiorgis
1077 cc16407a Dimitris Aragiorgis
This will create two deb packages in build-area. You should then run in both
1078 cc16407a Dimitris Aragiorgis
nodes:
1079 cc16407a Dimitris Aragiorgis
1080 cc16407a Dimitris Aragiorgis
.. code-block:: console
1081 cc16407a Dimitris Aragiorgis
1082 f1f5235e Dimitris Aragiorgis
   # dpkg -i ../build-area/ganeti-htools.*deb
1083 92e4d1c6 Dimitris Aragiorgis
   # dpkg -i ../build-area/snf-ganeti.*deb
1084 cc16407a Dimitris Aragiorgis
   # apt-get install -f
1085 caa6c07d Constantinos Venetsanopoulos
1086 caa6c07d Constantinos Venetsanopoulos
We assume that Ganeti will use the KVM hypervisor. After installing Ganeti on
1087 92e4d1c6 Dimitris Aragiorgis
both nodes, choose a domain name that resolves to a valid floating IP (let's
1088 92e4d1c6 Dimitris Aragiorgis
say it's ``ganeti.node1.example.com``). Make sure node1 and node2 have same
1089 92e4d1c6 Dimitris Aragiorgis
dsa/rsa keys and authorised_keys for password-less root ssh between each other.
1090 92e4d1c6 Dimitris Aragiorgis
If not then skip passing --no-ssh-init but be aware that it will replace
1091 92e4d1c6 Dimitris Aragiorgis
/root/.ssh/* related files and you might lose access to master node. Also,
1092 92e4d1c6 Dimitris Aragiorgis
make sure there is an lvm volume group named ``ganeti`` that will host your
1093 92e4d1c6 Dimitris Aragiorgis
VMs' disks. Finally, setup a bridge interface on the host machines (e.g: br0).
1094 92e4d1c6 Dimitris Aragiorgis
Then run on node1:
1095 caa6c07d Constantinos Venetsanopoulos
1096 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1097 caa6c07d Constantinos Venetsanopoulos
1098 11c16930 Ilias Tsitsimpis
   root@node1:~ # gnt-cluster init --enabled-hypervisors=kvm --no-ssh-init \
1099 11c16930 Ilias Tsitsimpis
                                   --no-etc-hosts --vg-name=ganeti \
1100 11c16930 Ilias Tsitsimpis
                                   --nic-parameters link=br0 --master-netdev eth0 \
1101 caa6c07d Constantinos Venetsanopoulos
                                   ganeti.node1.example.com
1102 caa6c07d Constantinos Venetsanopoulos
   root@node1:~ # gnt-cluster modify --default-iallocator hail
1103 caa6c07d Constantinos Venetsanopoulos
   root@node1:~ # gnt-cluster modify --hypervisor-parameters kvm:kernel_path=
1104 caa6c07d Constantinos Venetsanopoulos
   root@node1:~ # gnt-cluster modify --hypervisor-parameters kvm:vnc_bind_address=0.0.0.0
1105 caa6c07d Constantinos Venetsanopoulos
1106 92e4d1c6 Dimitris Aragiorgis
   root@node1:~ # gnt-node add --no-ssh-key-check --master-capable=yes \
1107 caa6c07d Constantinos Venetsanopoulos
                               --vm-capable=yes node2.example.com
1108 cc16407a Dimitris Aragiorgis
   root@node1:~ # gnt-cluster modify --disk-parameters=drbd:metavg=ganeti
1109 cc16407a Dimitris Aragiorgis
   root@node1:~ # gnt-group modify --disk-parameters=drbd:metavg=ganeti default
1110 caa6c07d Constantinos Venetsanopoulos
1111 caa6c07d Constantinos Venetsanopoulos
For any problems you may stumble upon installing Ganeti, please refer to the
1112 caa6c07d Constantinos Venetsanopoulos
`official documentation <http://docs.ganeti.org/ganeti/2.5/html>`_. Installation
1113 caa6c07d Constantinos Venetsanopoulos
of Ganeti is out of the scope of this guide.
1114 caa6c07d Constantinos Venetsanopoulos
1115 caa6c07d Constantinos Venetsanopoulos
.. _cyclades-install-snfimage:
1116 caa6c07d Constantinos Venetsanopoulos
1117 caa6c07d Constantinos Venetsanopoulos
snf-image
1118 7a8df455 Constantinos Venetsanopoulos
---------
1119 caa6c07d Constantinos Venetsanopoulos
1120 caa6c07d Constantinos Venetsanopoulos
Installation
1121 7a8df455 Constantinos Venetsanopoulos
~~~~~~~~~~~~
1122 caa6c07d Constantinos Venetsanopoulos
For :ref:`Cyclades <cyclades>` to be able to launch VMs from specified Images,
1123 caa6c07d Constantinos Venetsanopoulos
you need the :ref:`snf-image <snf-image>` OS Definition installed on *all*
1124 caa6c07d Constantinos Venetsanopoulos
VM-capable Ganeti nodes. This means we need :ref:`snf-image <snf-image>` on
1125 caa6c07d Constantinos Venetsanopoulos
node1 and node2. You can do this by running on *both* nodes:
1126 caa6c07d Constantinos Venetsanopoulos
1127 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1128 caa6c07d Constantinos Venetsanopoulos
1129 053d0dfc Constantinos Venetsanopoulos
   # apt-get install snf-image-host snf-pithos-backend python-psycopg2
1130 1a37da56 Constantinos Venetsanopoulos
1131 1a37da56 Constantinos Venetsanopoulos
snf-image also needs the `snf-pithos-backend <snf-pithos-backend>`, to be able to
1132 053d0dfc Constantinos Venetsanopoulos
handle image files stored on Pithos+. It also needs `python-psycopg2` to be able
1133 053d0dfc Constantinos Venetsanopoulos
to access the Pithos+ database. This is why, we also install them on *all*
1134 1a37da56 Constantinos Venetsanopoulos
VM-capable Ganeti nodes.
1135 caa6c07d Constantinos Venetsanopoulos
1136 cd837dad Constantinos Venetsanopoulos
After `snf-image-host` has been installed successfully, create the helper VM by
1137 caa6c07d Constantinos Venetsanopoulos
running on *both* nodes:
1138 caa6c07d Constantinos Venetsanopoulos
1139 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1140 caa6c07d Constantinos Venetsanopoulos
1141 caa6c07d Constantinos Venetsanopoulos
   # snf-image-update-helper
1142 caa6c07d Constantinos Venetsanopoulos
1143 caa6c07d Constantinos Venetsanopoulos
This will create all the needed files under ``/var/lib/snf-image/helper/`` for
1144 cd837dad Constantinos Venetsanopoulos
snf-image-host to run successfully, and it may take a few minutes depending on
1145 cd837dad Constantinos Venetsanopoulos
your Internet connection.
1146 caa6c07d Constantinos Venetsanopoulos
1147 caa6c07d Constantinos Venetsanopoulos
Configuration
1148 7a8df455 Constantinos Venetsanopoulos
~~~~~~~~~~~~~
1149 caa6c07d Constantinos Venetsanopoulos
snf-image supports native access to Images stored on Pithos+. This means that
1150 caa6c07d Constantinos Venetsanopoulos
snf-image can talk directly to the Pithos+ backend, without the need of providing
1151 caa6c07d Constantinos Venetsanopoulos
a public URL. More details, are described in the next section. For now, the only
1152 caa6c07d Constantinos Venetsanopoulos
thing we need to do, is configure snf-image to access our Pithos+ backend.
1153 caa6c07d Constantinos Venetsanopoulos
1154 caa6c07d Constantinos Venetsanopoulos
To do this, we need to set the corresponding variables in
1155 caa6c07d Constantinos Venetsanopoulos
``/etc/default/snf-image``, to reflect our Pithos+ setup:
1156 caa6c07d Constantinos Venetsanopoulos
1157 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1158 caa6c07d Constantinos Venetsanopoulos
1159 caa6c07d Constantinos Venetsanopoulos
   PITHOS_DB="postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos"
1160 caa6c07d Constantinos Venetsanopoulos
1161 caa6c07d Constantinos Venetsanopoulos
   PITHOS_DATA="/srv/pithos/data"
1162 caa6c07d Constantinos Venetsanopoulos
1163 caa6c07d Constantinos Venetsanopoulos
If you have installed your Ganeti cluster on different nodes than node1 and node2 make
1164 caa6c07d Constantinos Venetsanopoulos
sure that ``/srv/pithos/data`` is visible by all of them.
1165 caa6c07d Constantinos Venetsanopoulos
1166 caa6c07d Constantinos Venetsanopoulos
If you would like to use Images that are also/only stored locally, you need to
1167 caa6c07d Constantinos Venetsanopoulos
save them under ``IMAGE_DIR``, however this guide targets Images stored only on
1168 caa6c07d Constantinos Venetsanopoulos
Pithos+.
1169 caa6c07d Constantinos Venetsanopoulos
1170 caa6c07d Constantinos Venetsanopoulos
Testing
1171 7a8df455 Constantinos Venetsanopoulos
~~~~~~~
1172 caa6c07d Constantinos Venetsanopoulos
You can test that snf-image is successfully installed by running on the
1173 caa6c07d Constantinos Venetsanopoulos
:ref:`GANETI-MASTER <GANETI_NODES>` (in our case node1):
1174 caa6c07d Constantinos Venetsanopoulos
1175 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1176 caa6c07d Constantinos Venetsanopoulos
1177 caa6c07d Constantinos Venetsanopoulos
   # gnt-os diagnose
1178 caa6c07d Constantinos Venetsanopoulos
1179 caa6c07d Constantinos Venetsanopoulos
This should return ``valid`` for snf-image.
1180 caa6c07d Constantinos Venetsanopoulos
1181 caa6c07d Constantinos Venetsanopoulos
If you are interested to learn more about snf-image's internals (and even use
1182 caa6c07d Constantinos Venetsanopoulos
it alongside Ganeti without Synnefo), please see
1183 caa6c07d Constantinos Venetsanopoulos
`here <https://code.grnet.gr/projects/snf-image/wiki>`_ for information concerning
1184 caa6c07d Constantinos Venetsanopoulos
installation instructions, documentation on the design and implementation, and
1185 caa6c07d Constantinos Venetsanopoulos
supported Image formats.
1186 caa6c07d Constantinos Venetsanopoulos
1187 8a4cd31b Constantinos Venetsanopoulos
.. _snf-image-images:
1188 8a4cd31b Constantinos Venetsanopoulos
1189 cd837dad Constantinos Venetsanopoulos
Actual Images for snf-image
1190 cd837dad Constantinos Venetsanopoulos
---------------------------
1191 caa6c07d Constantinos Venetsanopoulos
1192 caa6c07d Constantinos Venetsanopoulos
Now that snf-image is installed successfully we need to provide it with some
1193 caa6c07d Constantinos Venetsanopoulos
Images. :ref:`snf-image <snf-image>` supports Images stored in ``extdump``,
1194 caa6c07d Constantinos Venetsanopoulos
``ntfsdump`` or ``diskdump`` format. We recommend the use of the ``diskdump``
1195 cd837dad Constantinos Venetsanopoulos
format. For more information about snf-image Image formats see `here
1196 caa6c07d Constantinos Venetsanopoulos
<https://code.grnet.gr/projects/snf-image/wiki/Image_Format>`_.
1197 caa6c07d Constantinos Venetsanopoulos
1198 caa6c07d Constantinos Venetsanopoulos
:ref:`snf-image <snf-image>` also supports three (3) different locations for the
1199 caa6c07d Constantinos Venetsanopoulos
above Images to be stored:
1200 caa6c07d Constantinos Venetsanopoulos
1201 caa6c07d Constantinos Venetsanopoulos
 * Under a local folder (usually an NFS mount, configurable as ``IMAGE_DIR`` in
1202 caa6c07d Constantinos Venetsanopoulos
   :file:`/etc/default/snf-image`)
1203 caa6c07d Constantinos Venetsanopoulos
 * On a remote host (accessible via a public URL e.g: http://... or ftp://...)
1204 caa6c07d Constantinos Venetsanopoulos
 * On Pithos+ (accessible natively, not only by its public URL)
1205 caa6c07d Constantinos Venetsanopoulos
1206 caa6c07d Constantinos Venetsanopoulos
For the purpose of this guide, we will use the `Debian Squeeze Base Image
1207 caa6c07d Constantinos Venetsanopoulos
<https://pithos.okeanos.grnet.gr/public/9epgb>`_ found on the official
1208 caa6c07d Constantinos Venetsanopoulos
`snf-image page
1209 caa6c07d Constantinos Venetsanopoulos
<https://code.grnet.gr/projects/snf-image/wiki#Sample-Images>`_. The image is
1210 caa6c07d Constantinos Venetsanopoulos
of type ``diskdump``. We will store it in our new Pithos+ installation.
1211 caa6c07d Constantinos Venetsanopoulos
1212 caa6c07d Constantinos Venetsanopoulos
To do so, do the following:
1213 caa6c07d Constantinos Venetsanopoulos
1214 caa6c07d Constantinos Venetsanopoulos
a) Download the Image from the official snf-image page (`image link
1215 caa6c07d Constantinos Venetsanopoulos
   <https://pithos.okeanos.grnet.gr/public/9epgb>`_).
1216 caa6c07d Constantinos Venetsanopoulos
1217 caa6c07d Constantinos Venetsanopoulos
b) Upload the Image to your Pithos+ installation, either using the Pithos+ Web UI
1218 caa6c07d Constantinos Venetsanopoulos
   or the command line client `kamaki
1219 caa6c07d Constantinos Venetsanopoulos
   <http://docs.dev.grnet.gr/kamaki/latest/index.html>`_.
1220 caa6c07d Constantinos Venetsanopoulos
1221 caa6c07d Constantinos Venetsanopoulos
Once the Image is uploaded successfully, download the Image's metadata file
1222 caa6c07d Constantinos Venetsanopoulos
from the official snf-image page (`image_metadata link
1223 caa6c07d Constantinos Venetsanopoulos
<https://pithos.okeanos.grnet.gr/public/gwqcv>`_). You will need it, for
1224 caa6c07d Constantinos Venetsanopoulos
spawning a VM from Ganeti, in the next section.
1225 caa6c07d Constantinos Venetsanopoulos
1226 caa6c07d Constantinos Venetsanopoulos
Of course, you can repeat the procedure to upload more Images, available from the
1227 caa6c07d Constantinos Venetsanopoulos
`official snf-image page
1228 caa6c07d Constantinos Venetsanopoulos
<https://code.grnet.gr/projects/snf-image/wiki#Sample-Images>`_.
1229 caa6c07d Constantinos Venetsanopoulos
1230 8a4cd31b Constantinos Venetsanopoulos
.. _ganeti-with-pithos-images:
1231 8a4cd31b Constantinos Venetsanopoulos
1232 caa6c07d Constantinos Venetsanopoulos
Spawning a VM from a Pithos+ Image, using Ganeti
1233 7a8df455 Constantinos Venetsanopoulos
------------------------------------------------
1234 caa6c07d Constantinos Venetsanopoulos
1235 caa6c07d Constantinos Venetsanopoulos
Now, it is time to test our installation so far. So, we have Astakos and
1236 caa6c07d Constantinos Venetsanopoulos
Pithos+ installed, we have a working Ganeti installation, the snf-image
1237 caa6c07d Constantinos Venetsanopoulos
definition installed on all VM-capable nodes and a Debian Squeeze Image on
1238 caa6c07d Constantinos Venetsanopoulos
Pithos+. Make sure you also have the `metadata file
1239 caa6c07d Constantinos Venetsanopoulos
<https://pithos.okeanos.grnet.gr/public/gwqcv>`_ for this image.
1240 caa6c07d Constantinos Venetsanopoulos
1241 caa6c07d Constantinos Venetsanopoulos
Run on the :ref:`GANETI-MASTER's <GANETI_NODES>` (node1) command line:
1242 caa6c07d Constantinos Venetsanopoulos
1243 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1244 caa6c07d Constantinos Venetsanopoulos
1245 cc16407a Dimitris Aragiorgis
   # gnt-instance add -o snf-image+default --os-parameters \
1246 cc16407a Dimitris Aragiorgis
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1247 cc16407a Dimitris Aragiorgis
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1248 caa6c07d Constantinos Venetsanopoulos
                      testvm1
1249 caa6c07d Constantinos Venetsanopoulos
1250 caa6c07d Constantinos Venetsanopoulos
In the above command:
1251 caa6c07d Constantinos Venetsanopoulos
1252 caa6c07d Constantinos Venetsanopoulos
 * ``img_passwd``: the arbitrary root password of your new instance
1253 caa6c07d Constantinos Venetsanopoulos
 * ``img_format``: set to ``diskdump`` to reflect the type of the uploaded Image
1254 caa6c07d Constantinos Venetsanopoulos
 * ``img_id``: If you want to deploy an Image stored on Pithos+ (our case), this
1255 301294a9 Constantinos Venetsanopoulos
               should have the format ``pithos://<username>/<container>/<filename>``:
1256 301294a9 Constantinos Venetsanopoulos
               * ``username``: ``user@example.com`` (defined during Astakos sign up)
1257 301294a9 Constantinos Venetsanopoulos
               * ``container``: ``pithos`` (default, if the Web UI was used)
1258 301294a9 Constantinos Venetsanopoulos
               * ``filename``: the name of file (visible also from the Web UI)
1259 caa6c07d Constantinos Venetsanopoulos
 * ``img_properties``: taken from the metadata file. Used only the two mandatory
1260 caa6c07d Constantinos Venetsanopoulos
                       properties ``OSFAMILY`` and ``ROOT_PARTITION``. `Learn more
1261 caa6c07d Constantinos Venetsanopoulos
                       <https://code.grnet.gr/projects/snf-image/wiki/Image_Format#Image-Properties>`_
1262 caa6c07d Constantinos Venetsanopoulos
1263 caa6c07d Constantinos Venetsanopoulos
If the ``gnt-instance add`` command returns successfully, then run:
1264 caa6c07d Constantinos Venetsanopoulos
1265 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1266 caa6c07d Constantinos Venetsanopoulos
1267 caa6c07d Constantinos Venetsanopoulos
   # gnt-instance info testvm1 | grep "console connection"
1268 caa6c07d Constantinos Venetsanopoulos
1269 caa6c07d Constantinos Venetsanopoulos
to find out where to connect using VNC. If you can connect successfully and can
1270 caa6c07d Constantinos Venetsanopoulos
login to your new instance using the root password ``my_vm_example_passw0rd``,
1271 caa6c07d Constantinos Venetsanopoulos
then everything works as expected and you have your new Debian Base VM up and
1272 caa6c07d Constantinos Venetsanopoulos
running.
1273 caa6c07d Constantinos Venetsanopoulos
1274 caa6c07d Constantinos Venetsanopoulos
If ``gnt-instance add`` fails, make sure that snf-image is correctly configured
1275 caa6c07d Constantinos Venetsanopoulos
to access the Pithos+ database and the Pithos+ backend data. Also, make sure
1276 caa6c07d Constantinos Venetsanopoulos
you gave the correct ``img_id`` and ``img_properties``. If ``gnt-instance add``
1277 caa6c07d Constantinos Venetsanopoulos
succeeds but you cannot connect, again find out what went wrong. Do *NOT*
1278 caa6c07d Constantinos Venetsanopoulos
proceed to the next steps unless you are sure everything works till this point.
1279 caa6c07d Constantinos Venetsanopoulos
1280 2f6143c9 Constantinos Venetsanopoulos
If everything works, you have successfully connected Ganeti with Pithos+. Let's
1281 2f6143c9 Constantinos Venetsanopoulos
move on to networking now.
1282 caa6c07d Constantinos Venetsanopoulos
1283 2f6143c9 Constantinos Venetsanopoulos
.. warning::
1284 e4404297 Christos Stavrakakis
1285 7a8df455 Constantinos Venetsanopoulos
    You can bypass the networking sections and go straight to
1286 2c85833e Constantinos Venetsanopoulos
    :ref:`Cyclades Ganeti tools <cyclades-gtools>`, if you do not want to setup
1287 2c85833e Constantinos Venetsanopoulos
    the Cyclades Network Service, but only the Cyclades Compute Service
1288 2c85833e Constantinos Venetsanopoulos
    (recommended for now).
1289 2f6143c9 Constantinos Venetsanopoulos
1290 cc16407a Dimitris Aragiorgis
Networking Setup Overview
1291 cc16407a Dimitris Aragiorgis
-------------------------
1292 2f6143c9 Constantinos Venetsanopoulos
1293 2f6143c9 Constantinos Venetsanopoulos
This part is deployment-specific and must be customized based on the specific
1294 2f6143c9 Constantinos Venetsanopoulos
needs of the system administrator. However, to do so, the administrator needs
1295 2f6143c9 Constantinos Venetsanopoulos
to understand how each level handles Virtual Networks, to be able to setup the
1296 b11446c1 Constantinos Venetsanopoulos
backend appropriately, before installing Cyclades. To do so, please read the
1297 b11446c1 Constantinos Venetsanopoulos
:ref:`Network <networks>` section before proceeding.
1298 2f6143c9 Constantinos Venetsanopoulos
1299 cc16407a Dimitris Aragiorgis
Since synnefo 0.11 all network actions are managed with the snf-manage
1300 cc16407a Dimitris Aragiorgis
network-* commands. This needs the underlying setup (Ganeti, nfdhcpd,
1301 cc16407a Dimitris Aragiorgis
snf-network, bridges, vlans) to be already configured correctly. The only
1302 cc16407a Dimitris Aragiorgis
actions needed in this point are:
1303 cc16407a Dimitris Aragiorgis
1304 cc16407a Dimitris Aragiorgis
a) Have Ganeti with IP pool management support installed.
1305 2f6143c9 Constantinos Venetsanopoulos
1306 cc16407a Dimitris Aragiorgis
b) Install :ref:`snf-network <snf-network>`, which provides a synnefo specific kvm-ifup script, etc.
1307 2f6143c9 Constantinos Venetsanopoulos
1308 cc16407a Dimitris Aragiorgis
c) Install :ref:`nfdhcpd <nfdhcpd>`, which serves DHCP requests of the VMs.
1309 2f6143c9 Constantinos Venetsanopoulos
1310 cc16407a Dimitris Aragiorgis
In order to test that everything is setup correctly before installing Cyclades,
1311 cc16407a Dimitris Aragiorgis
we will make some testing actions in this section, and the actual setup will be
1312 cc16407a Dimitris Aragiorgis
done afterwards with snf-manage commands.
1313 2f6143c9 Constantinos Venetsanopoulos
1314 cc16407a Dimitris Aragiorgis
.. _snf-network:
1315 2f6143c9 Constantinos Venetsanopoulos
1316 cc16407a Dimitris Aragiorgis
snf-network
1317 cc16407a Dimitris Aragiorgis
~~~~~~~~~~~
1318 2f6143c9 Constantinos Venetsanopoulos
1319 cc16407a Dimitris Aragiorgis
snf-network includes `kvm-vif-bridge` script that is invoked every time
1320 cc16407a Dimitris Aragiorgis
a tap (a VM's NIC) is created. Based on environment variables passed by
1321 cc16407a Dimitris Aragiorgis
Ganeti it issues various commands depending on the network type the NIC is
1322 cc16407a Dimitris Aragiorgis
connected to and sets up a corresponding dhcp lease.
1323 2f6143c9 Constantinos Venetsanopoulos
1324 cc16407a Dimitris Aragiorgis
Install snf-network on all Ganeti nodes:
1325 2f6143c9 Constantinos Venetsanopoulos
1326 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1327 2f6143c9 Constantinos Venetsanopoulos
1328 cc16407a Dimitris Aragiorgis
   # apt-get install snf-network
1329 2f6143c9 Constantinos Venetsanopoulos
1330 cc16407a Dimitris Aragiorgis
Then, in :file:`/etc/default/snf-network` set:
1331 2f6143c9 Constantinos Venetsanopoulos
1332 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1333 2f6143c9 Constantinos Venetsanopoulos
1334 cc16407a Dimitris Aragiorgis
   MAC_MASK=ff:ff:f0:00:00:00
1335 2f6143c9 Constantinos Venetsanopoulos
1336 cc16407a Dimitris Aragiorgis
.. _nfdhcpd:
1337 2f6143c9 Constantinos Venetsanopoulos
1338 cc16407a Dimitris Aragiorgis
nfdhcpd
1339 7a8df455 Constantinos Venetsanopoulos
~~~~~~~
1340 2f6143c9 Constantinos Venetsanopoulos
1341 cc16407a Dimitris Aragiorgis
Each NIC's IP is chosen by Ganeti (with IP pool management support).
1342 cc16407a Dimitris Aragiorgis
`kvm-vif-bridge` script sets up dhcp leases and when the VM boots and
1343 cc16407a Dimitris Aragiorgis
makes a dhcp request, iptables will mangle the packet and `nfdhcpd` will
1344 cc16407a Dimitris Aragiorgis
create a dhcp response.
1345 2f6143c9 Constantinos Venetsanopoulos
1346 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1347 2f6143c9 Constantinos Venetsanopoulos
1348 cc16407a Dimitris Aragiorgis
   # apt-get install nfqueue-bindings-python=0.3+physindev-1
1349 2f6143c9 Constantinos Venetsanopoulos
   # apt-get install nfdhcpd
1350 2f6143c9 Constantinos Venetsanopoulos
1351 2f6143c9 Constantinos Venetsanopoulos
Edit ``/etc/nfdhcpd/nfdhcpd.conf`` to reflect your network configuration. At
1352 2f6143c9 Constantinos Venetsanopoulos
least, set the ``dhcp_queue`` variable to ``42`` and the ``nameservers``
1353 2f6143c9 Constantinos Venetsanopoulos
variable to your DNS IP/s. Those IPs will be passed as the DNS IP/s of your new
1354 2f6143c9 Constantinos Venetsanopoulos
VMs. Once you are finished, restart the server on all nodes:
1355 2f6143c9 Constantinos Venetsanopoulos
1356 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1357 2f6143c9 Constantinos Venetsanopoulos
1358 2f6143c9 Constantinos Venetsanopoulos
   # /etc/init.d/nfdhcpd restart
1359 2f6143c9 Constantinos Venetsanopoulos
1360 2f6143c9 Constantinos Venetsanopoulos
If you are using ``ferm``, then you need to run the following:
1361 2f6143c9 Constantinos Venetsanopoulos
1362 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1363 2f6143c9 Constantinos Venetsanopoulos
1364 2f6143c9 Constantinos Venetsanopoulos
   # echo "@include 'nfdhcpd.ferm';" >> /etc/ferm/ferm.conf
1365 2f6143c9 Constantinos Venetsanopoulos
   # /etc/init.d/ferm restart
1366 2f6143c9 Constantinos Venetsanopoulos
1367 cc16407a Dimitris Aragiorgis
or make sure to run after boot:
1368 cc16407a Dimitris Aragiorgis
1369 cc16407a Dimitris Aragiorgis
.. code-block:: console
1370 cc16407a Dimitris Aragiorgis
1371 cc16407a Dimitris Aragiorgis
   # iptables -t mangle -A PREROUTING -p udp -m udp --dport 67 -j NFQUEUE --queue-num 42
1372 cc16407a Dimitris Aragiorgis
1373 cc16407a Dimitris Aragiorgis
and if you have IPv6 enabled:
1374 cc16407a Dimitris Aragiorgis
1375 cc16407a Dimitris Aragiorgis
.. code-block:: console
1376 cc16407a Dimitris Aragiorgis
1377 cc16407a Dimitris Aragiorgis
   # ip6tables -t mangle -A PREROUTING -p ipv6-icmp -m icmp6 --icmpv6-type 133 -j NFQUEUE --queue-num 43
1378 cc16407a Dimitris Aragiorgis
   # ip6tables -t mangle -A PREROUTING -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j NFQUEUE --queue-num 44
1379 cc16407a Dimitris Aragiorgis
1380 cc16407a Dimitris Aragiorgis
You can check which clients are currently served by nfdhcpd by running:
1381 cc16407a Dimitris Aragiorgis
1382 cc16407a Dimitris Aragiorgis
.. code-block:: console
1383 cc16407a Dimitris Aragiorgis
1384 cc16407a Dimitris Aragiorgis
   # kill -SIGUSR1 `cat /var/run/nfdhcpd/nfdhcpd.pid`
1385 cc16407a Dimitris Aragiorgis
1386 cc16407a Dimitris Aragiorgis
When you run the above, then check ``/var/log/nfdhcpd/nfdhcpd.log``.
1387 cc16407a Dimitris Aragiorgis
1388 cc16407a Dimitris Aragiorgis
Public Network Setup
1389 cc16407a Dimitris Aragiorgis
--------------------
1390 cc16407a Dimitris Aragiorgis
1391 cc16407a Dimitris Aragiorgis
To achieve basic networking the simplest way is to have a common bridge (e.g.
1392 cc16407a Dimitris Aragiorgis
``br0``, on the same collision domain with the router) where all VMs will connect
1393 cc16407a Dimitris Aragiorgis
to. Packets will be "forwarded" to the router and then to the Internet. If
1394 cc16407a Dimitris Aragiorgis
you want a more advanced setup (ip-less routing and proxy-arp plese refer to
1395 cc16407a Dimitris Aragiorgis
:ref:`Network <networks>` section).
1396 cc16407a Dimitris Aragiorgis
1397 cc16407a Dimitris Aragiorgis
Physical Host Setup
1398 cc16407a Dimitris Aragiorgis
~~~~~~~~~~~~~~~~~~~
1399 cc16407a Dimitris Aragiorgis
1400 cc16407a Dimitris Aragiorgis
Assuming ``eth0`` on both hosts is the public interface (directly connected
1401 cc16407a Dimitris Aragiorgis
to the router), run on every node:
1402 2f6143c9 Constantinos Venetsanopoulos
1403 cc16407a Dimitris Aragiorgis
.. code-block:: console
1404 cc16407a Dimitris Aragiorgis
1405 cc16407a Dimitris Aragiorgis
   # brctl addbr br0
1406 cc16407a Dimitris Aragiorgis
   # ip link set br0 up
1407 cc16407a Dimitris Aragiorgis
   # vconfig add eth0 100
1408 cc16407a Dimitris Aragiorgis
   # ip link set eth0.100 up
1409 cc16407a Dimitris Aragiorgis
   # brctl addif br0 eth0.100
1410 cc16407a Dimitris Aragiorgis
1411 cc16407a Dimitris Aragiorgis
1412 cc16407a Dimitris Aragiorgis
Testing a Public Network
1413 cc16407a Dimitris Aragiorgis
~~~~~~~~~~~~~~~~~~~~~~~~
1414 cc16407a Dimitris Aragiorgis
1415 cc16407a Dimitris Aragiorgis
Let's assume, that you want to assign IPs from the ``5.6.7.0/27`` range to you
1416 cc16407a Dimitris Aragiorgis
new VMs, with ``5.6.7.1`` as the router's gateway. In Ganeti you can add the
1417 cc16407a Dimitris Aragiorgis
network by running:
1418 cc16407a Dimitris Aragiorgis
1419 cc16407a Dimitris Aragiorgis
.. code-block:: console
1420 cc16407a Dimitris Aragiorgis
1421 cc16407a Dimitris Aragiorgis
   # gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network-type=public --tags=nfdhcpd test-net-public
1422 2f6143c9 Constantinos Venetsanopoulos
1423 cc16407a Dimitris Aragiorgis
Then, connect the network to all your nodegroups. We assume that we only have
1424 cc16407a Dimitris Aragiorgis
one nodegroup (``default``) in our Ganeti cluster:
1425 cc16407a Dimitris Aragiorgis
1426 cc16407a Dimitris Aragiorgis
.. code-block:: console
1427 cc16407a Dimitris Aragiorgis
1428 cc16407a Dimitris Aragiorgis
   # gnt-network connect test-net-public default bridged br0
1429 2f6143c9 Constantinos Venetsanopoulos
1430 2f6143c9 Constantinos Venetsanopoulos
Now, it is time to test that the backend infrastracture is correctly setup for
1431 cc16407a Dimitris Aragiorgis
the Public Network. We will add a new VM, the same way we did it on the
1432 cc16407a Dimitris Aragiorgis
previous testing section. However, now will also add one NIC, configured to be
1433 cc16407a Dimitris Aragiorgis
managed from our previously defined network. Run on the GANETI-MASTER (node1):
1434 2f6143c9 Constantinos Venetsanopoulos
1435 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1436 2f6143c9 Constantinos Venetsanopoulos
1437 cc16407a Dimitris Aragiorgis
   # gnt-instance add -o snf-image+default --os-parameters \
1438 cc16407a Dimitris Aragiorgis
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1439 cc16407a Dimitris Aragiorgis
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1440 cc16407a Dimitris Aragiorgis
                      --net 0:ip=pool,network=test-net-public \
1441 2f6143c9 Constantinos Venetsanopoulos
                      testvm2
1442 2f6143c9 Constantinos Venetsanopoulos
1443 2f6143c9 Constantinos Venetsanopoulos
If the above returns successfully, connect to the new VM and run:
1444 2f6143c9 Constantinos Venetsanopoulos
1445 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1446 2f6143c9 Constantinos Venetsanopoulos
1447 cc16407a Dimitris Aragiorgis
   root@testvm2:~ # ip addr
1448 cc16407a Dimitris Aragiorgis
   root@testvm2:~ # ip route
1449 cc16407a Dimitris Aragiorgis
   root@testvm2:~ # cat /etc/resolv.conf
1450 2f6143c9 Constantinos Venetsanopoulos
1451 cc16407a Dimitris Aragiorgis
to check IP address (5.6.7.2), IP routes (default via 5.6.7.1) and DNS config
1452 cc16407a Dimitris Aragiorgis
(nameserver option in nfdhcpd.conf). This shows correct configuration of
1453 cc16407a Dimitris Aragiorgis
ganeti, snf-network and nfdhcpd.
1454 2f6143c9 Constantinos Venetsanopoulos
1455 2f6143c9 Constantinos Venetsanopoulos
Now ping the outside world. If this works too, then you have also configured
1456 cc16407a Dimitris Aragiorgis
correctly your physical host and router.
1457 547c78f6 Constantinos Venetsanopoulos
1458 547c78f6 Constantinos Venetsanopoulos
Make sure everything works as expected, before proceeding with the Private
1459 2f6143c9 Constantinos Venetsanopoulos
Networks setup.
1460 2f6143c9 Constantinos Venetsanopoulos
1461 04c1254b Constantinos Venetsanopoulos
.. _private-networks-setup:
1462 04c1254b Constantinos Venetsanopoulos
1463 cc16407a Dimitris Aragiorgis
Private Networks Setup
1464 7a8df455 Constantinos Venetsanopoulos
----------------------
1465 2f6143c9 Constantinos Venetsanopoulos
1466 cc16407a Dimitris Aragiorgis
Synnefo supports two types of private networks:
1467 cc16407a Dimitris Aragiorgis
1468 cc16407a Dimitris Aragiorgis
 - based on MAC filtering
1469 cc16407a Dimitris Aragiorgis
 - based on physical VLANs
1470 cc16407a Dimitris Aragiorgis
1471 cc16407a Dimitris Aragiorgis
Both types provide Layer 2 isolation to the end-user.
1472 2f6143c9 Constantinos Venetsanopoulos
1473 cc16407a Dimitris Aragiorgis
For the first type a common bridge (e.g. ``prv0``) is needed while for the second a
1474 cc16407a Dimitris Aragiorgis
range of bridges (e.g. ``prv1..prv100``) each bridged on a different physical
1475 cc16407a Dimitris Aragiorgis
VLAN. To this end to assure isolation among end-users' private networks each
1476 cc16407a Dimitris Aragiorgis
has to have different MAC prefix (for the filtering to take place) or to be
1477 cc16407a Dimitris Aragiorgis
"connected" to a different bridge (VLAN actually).
1478 547c78f6 Constantinos Venetsanopoulos
1479 cc16407a Dimitris Aragiorgis
Physical Host Setup
1480 cc16407a Dimitris Aragiorgis
~~~~~~~~~~~~~~~~~~~
1481 cc16407a Dimitris Aragiorgis
1482 cc16407a Dimitris Aragiorgis
In order to create the necessary VLAN/bridges, one for MAC filtered private
1483 cc16407a Dimitris Aragiorgis
networks and various (e.g. 20) for private networks based on physical VLANs,
1484 cc16407a Dimitris Aragiorgis
run on every node:
1485 547c78f6 Constantinos Venetsanopoulos
1486 cc16407a Dimitris Aragiorgis
Assuming ``eth0`` of both hosts are somehow (via cable/switch with VLANs
1487 cc16407a Dimitris Aragiorgis
configured correctly) connected together, run on every node:
1488 547c78f6 Constantinos Venetsanopoulos
1489 547c78f6 Constantinos Venetsanopoulos
.. code-block:: console
1490 547c78f6 Constantinos Venetsanopoulos
1491 cc16407a Dimitris Aragiorgis
   # apt-get install vlan
1492 cc16407a Dimitris Aragiorgis
   # modprobe 8021q
1493 547c78f6 Constantinos Venetsanopoulos
   # $iface=eth0
1494 cc16407a Dimitris Aragiorgis
   # for prv in $(seq 0 20); do
1495 547c78f6 Constantinos Venetsanopoulos
	vlan=$prv
1496 547c78f6 Constantinos Venetsanopoulos
	bridge=prv$prv
1497 547c78f6 Constantinos Venetsanopoulos
	vconfig add $iface $vlan
1498 547c78f6 Constantinos Venetsanopoulos
	ifconfig $iface.$vlan up
1499 547c78f6 Constantinos Venetsanopoulos
	brctl addbr $bridge
1500 547c78f6 Constantinos Venetsanopoulos
	brctl setfd $bridge 0
1501 547c78f6 Constantinos Venetsanopoulos
	brctl addif $bridge $iface.$vlan
1502 547c78f6 Constantinos Venetsanopoulos
	ifconfig $bridge up
1503 547c78f6 Constantinos Venetsanopoulos
      done
1504 547c78f6 Constantinos Venetsanopoulos
1505 cc16407a Dimitris Aragiorgis
The above will do the following :
1506 547c78f6 Constantinos Venetsanopoulos
1507 cc16407a Dimitris Aragiorgis
 * provision 21 new bridges: ``prv0`` - ``prv20``
1508 cc16407a Dimitris Aragiorgis
 * provision 21 new vlans: ``eth0.0`` - ``eth0.20``
1509 cc16407a Dimitris Aragiorgis
 * add the corresponding vlan to the equivalent bridge
1510 547c78f6 Constantinos Venetsanopoulos
1511 547c78f6 Constantinos Venetsanopoulos
You can run ``brctl show`` on both nodes to see if everything was setup
1512 547c78f6 Constantinos Venetsanopoulos
correctly.
1513 547c78f6 Constantinos Venetsanopoulos
1514 2f6143c9 Constantinos Venetsanopoulos
Testing the Private Networks
1515 7a8df455 Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1516 caa6c07d Constantinos Venetsanopoulos
1517 547c78f6 Constantinos Venetsanopoulos
To test the Private Networks, we will create two instances and put them in the
1518 cc16407a Dimitris Aragiorgis
same Private Networks (one MAC Filtered and one Physical VLAN). This means
1519 cc16407a Dimitris Aragiorgis
that the instances will have a second NIC connected to the ``prv0``
1520 cc16407a Dimitris Aragiorgis
pre-provisioned bridge and a third to ``prv1``.
1521 547c78f6 Constantinos Venetsanopoulos
1522 547c78f6 Constantinos Venetsanopoulos
We run the same command as in the Public Network testing section, but with one
1523 547c78f6 Constantinos Venetsanopoulos
more argument for the second NIC:
1524 547c78f6 Constantinos Venetsanopoulos
1525 547c78f6 Constantinos Venetsanopoulos
.. code-block:: console
1526 547c78f6 Constantinos Venetsanopoulos
1527 cc16407a Dimitris Aragiorgis
   # gnt-network add --network=192.168.1.0/24 --mac-prefix=aa:00:55 --network-type=private --tags=nfdhcpd,private-filtered test-net-prv-mac
1528 cc16407a Dimitris Aragiorgis
   # gnt-network connect test-net-prv-mac default bridged prv0
1529 cc16407a Dimitris Aragiorgis
1530 cc16407a Dimitris Aragiorgis
   # gnt-network add --network=10.0.0.0/24 --tags=nfdhcpd --network-type=private test-net-prv-vlan
1531 cc16407a Dimitris Aragiorgis
   # gnt-network connect test-net-prv-vlan default bridged prv1
1532 cc16407a Dimitris Aragiorgis
1533 cc16407a Dimitris Aragiorgis
   # gnt-instance add -o snf-image+default --os-parameters \
1534 cc16407a Dimitris Aragiorgis
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1535 cc16407a Dimitris Aragiorgis
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1536 cc16407a Dimitris Aragiorgis
                      --net 0:ip=pool,network=test-net-public \
1537 cc16407a Dimitris Aragiorgis
                      --net 1:ip=pool,network=test-net-prv-mac \
1538 cc16407a Dimitris Aragiorgis
                      --net 2:ip=none,network=test-net-prv-vlan \
1539 547c78f6 Constantinos Venetsanopoulos
                      testvm3
1540 547c78f6 Constantinos Venetsanopoulos
1541 cc16407a Dimitris Aragiorgis
   # gnt-instance add -o snf-image+default --os-parameters \
1542 cc16407a Dimitris Aragiorgis
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1543 cc16407a Dimitris Aragiorgis
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1544 cc16407a Dimitris Aragiorgis
                      --net 0:ip=pool,network=test-net-public \
1545 cc16407a Dimitris Aragiorgis
                      --net 1:ip=pool,network=test-net-prv-mac \
1546 cc16407a Dimitris Aragiorgis
                      --net 2:ip=none,network=test-net-prv-vlan \
1547 547c78f6 Constantinos Venetsanopoulos
                      testvm4
1548 547c78f6 Constantinos Venetsanopoulos
1549 cc16407a Dimitris Aragiorgis
Above, we create two instances with first NIC connected to the internet, their
1550 cc16407a Dimitris Aragiorgis
second NIC connected to a MAC filtered private Network and their third NIC
1551 cc16407a Dimitris Aragiorgis
connected to the first Physical VLAN Private Network. Now, connect to the
1552 cc16407a Dimitris Aragiorgis
instances using VNC and make sure everything works as expected:
1553 547c78f6 Constantinos Venetsanopoulos
1554 cc16407a Dimitris Aragiorgis
 a) The instances have access to the public internet through their first eth
1555 cc16407a Dimitris Aragiorgis
    interface (``eth0``), which has been automatically assigned a public IP.
1556 547c78f6 Constantinos Venetsanopoulos
1557 cc16407a Dimitris Aragiorgis
 b) ``eth1`` will have mac prefix ``aa:00:55``, while ``eth2`` default one (``aa:00:00``)
1558 547c78f6 Constantinos Venetsanopoulos
1559 cc16407a Dimitris Aragiorgis
 c) ip link set ``eth1``/``eth2`` up
1560 547c78f6 Constantinos Venetsanopoulos
1561 cc16407a Dimitris Aragiorgis
 d) dhclient ``eth1``/``eth2``
1562 547c78f6 Constantinos Venetsanopoulos
1563 cc16407a Dimitris Aragiorgis
 e) On testvm3  ping 192.168.1.2/10.0.0.2
1564 547c78f6 Constantinos Venetsanopoulos
1565 547c78f6 Constantinos Venetsanopoulos
If everything works as expected, then you have finished the Network Setup at the
1566 547c78f6 Constantinos Venetsanopoulos
backend for both types of Networks (Public & Private).
1567 547c78f6 Constantinos Venetsanopoulos
1568 2c85833e Constantinos Venetsanopoulos
.. _cyclades-gtools:
1569 2c85833e Constantinos Venetsanopoulos
1570 2c85833e Constantinos Venetsanopoulos
Cyclades Ganeti tools
1571 2c85833e Constantinos Venetsanopoulos
---------------------
1572 2c85833e Constantinos Venetsanopoulos
1573 2c85833e Constantinos Venetsanopoulos
In order for Ganeti to be connected with Cyclades later on, we need the
1574 2c85833e Constantinos Venetsanopoulos
`Cyclades Ganeti tools` available on all Ganeti nodes (node1 & node2 in our
1575 2c85833e Constantinos Venetsanopoulos
case). You can install them by running in both nodes:
1576 2c85833e Constantinos Venetsanopoulos
1577 2c85833e Constantinos Venetsanopoulos
.. code-block:: console
1578 2c85833e Constantinos Venetsanopoulos
1579 2c85833e Constantinos Venetsanopoulos
   # apt-get install snf-cyclades-gtools
1580 2c85833e Constantinos Venetsanopoulos
1581 2c85833e Constantinos Venetsanopoulos
This will install the following:
1582 2c85833e Constantinos Venetsanopoulos
1583 2c85833e Constantinos Venetsanopoulos
 * ``snf-ganeti-eventd`` (daemon to publish Ganeti related messages on RabbitMQ)
1584 2c85833e Constantinos Venetsanopoulos
 * ``snf-ganeti-hook`` (all necessary hooks under ``/etc/ganeti/hooks``)
1585 2c85833e Constantinos Venetsanopoulos
 * ``snf-progress-monitor`` (used by ``snf-image`` to publish progress messages)
1586 2c85833e Constantinos Venetsanopoulos
1587 2c85833e Constantinos Venetsanopoulos
Configure ``snf-cyclades-gtools``
1588 2c85833e Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1589 2c85833e Constantinos Venetsanopoulos
1590 04c1254b Constantinos Venetsanopoulos
The package will install the ``/etc/synnefo/10-snf-cyclades-gtools-backend.conf``
1591 2c85833e Constantinos Venetsanopoulos
configuration file. At least we need to set the RabbitMQ endpoint for all tools
1592 2c85833e Constantinos Venetsanopoulos
that need it:
1593 2c85833e Constantinos Venetsanopoulos
1594 2c85833e Constantinos Venetsanopoulos
.. code-block:: console
1595 2c85833e Constantinos Venetsanopoulos
1596 ad003186 Constantinos Venetsanopoulos
   AMQP_HOSTS=["amqp://synnefo:example_rabbitmq_passw0rd@node1.example.com:5672"]
1597 2c85833e Constantinos Venetsanopoulos
1598 2c85833e Constantinos Venetsanopoulos
The above variables should reflect your :ref:`Message Queue setup
1599 2c85833e Constantinos Venetsanopoulos
<rabbitmq-setup>`. This file should be editted in all Ganeti nodes.
1600 2c85833e Constantinos Venetsanopoulos
1601 2c85833e Constantinos Venetsanopoulos
Connect ``snf-image`` with ``snf-progress-monitor``
1602 2c85833e Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1603 2c85833e Constantinos Venetsanopoulos
1604 2c85833e Constantinos Venetsanopoulos
Finally, we need to configure ``snf-image`` to publish progress messages during
1605 2c85833e Constantinos Venetsanopoulos
the deployment of each Image. To do this, we edit ``/etc/default/snf-image`` and
1606 2c85833e Constantinos Venetsanopoulos
set the corresponding variable to ``snf-progress-monitor``:
1607 2c85833e Constantinos Venetsanopoulos
1608 2c85833e Constantinos Venetsanopoulos
.. code-block:: console
1609 2c85833e Constantinos Venetsanopoulos
1610 2c85833e Constantinos Venetsanopoulos
   PROGRESS_MONITOR="snf-progress-monitor"
1611 2c85833e Constantinos Venetsanopoulos
1612 2c85833e Constantinos Venetsanopoulos
This file should be editted in all Ganeti nodes.
1613 2c85833e Constantinos Venetsanopoulos
1614 7a8df455 Constantinos Venetsanopoulos
.. _rapi-user:
1615 7a8df455 Constantinos Venetsanopoulos
1616 caa6c07d Constantinos Venetsanopoulos
Synnefo RAPI user
1617 7a8df455 Constantinos Venetsanopoulos
-----------------
1618 caa6c07d Constantinos Venetsanopoulos
1619 7a8df455 Constantinos Venetsanopoulos
As a last step before installing Cyclades, create a new RAPI user that will
1620 caa6c07d Constantinos Venetsanopoulos
have ``write`` access. Cyclades will use this user to issue commands to Ganeti,
1621 2c85833e Constantinos Venetsanopoulos
so we will call the user ``cyclades`` with password ``example_rapi_passw0rd``.
1622 2c85833e Constantinos Venetsanopoulos
You can do this, by first running:
1623 2c85833e Constantinos Venetsanopoulos
1624 2c85833e Constantinos Venetsanopoulos
.. code-block:: console
1625 2c85833e Constantinos Venetsanopoulos
1626 2c85833e Constantinos Venetsanopoulos
   # echo -n 'cyclades:Ganeti Remote API:example_rapi_passw0rd' | openssl md5
1627 2c85833e Constantinos Venetsanopoulos
1628 2c85833e Constantinos Venetsanopoulos
and then putting the output in ``/var/lib/ganeti/rapi/users`` as follows:
1629 caa6c07d Constantinos Venetsanopoulos
1630 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1631 caa6c07d Constantinos Venetsanopoulos
1632 2c85833e Constantinos Venetsanopoulos
   cyclades {HA1}55aec7050aa4e4b111ca43cb505a61a0 write
1633 caa6c07d Constantinos Venetsanopoulos
1634 caa6c07d Constantinos Venetsanopoulos
More about Ganeti's RAPI users `here.
1635 caa6c07d Constantinos Venetsanopoulos
<http://docs.ganeti.org/ganeti/2.5/html/rapi.html#introduction>`_
1636 bc055d09 Constantinos Venetsanopoulos
1637 7a8df455 Constantinos Venetsanopoulos
You have now finished with all needed Prerequisites for Cyclades (and
1638 7a8df455 Constantinos Venetsanopoulos
Plankton). Let's move on to the actual Cyclades installation.
1639 7a8df455 Constantinos Venetsanopoulos
1640 7a8df455 Constantinos Venetsanopoulos
1641 7a8df455 Constantinos Venetsanopoulos
Installation of Cyclades (and Plankton) on node1
1642 7a8df455 Constantinos Venetsanopoulos
================================================
1643 7a8df455 Constantinos Venetsanopoulos
1644 7a8df455 Constantinos Venetsanopoulos
This section describes the installation of Cyclades. Cyclades is Synnefo's
1645 7a8df455 Constantinos Venetsanopoulos
Compute service. Plankton (the Image Registry service) will get installed
1646 7a8df455 Constantinos Venetsanopoulos
automatically along with Cyclades, because it is contained in the same Synnefo
1647 7a8df455 Constantinos Venetsanopoulos
component right now.
1648 7a8df455 Constantinos Venetsanopoulos
1649 04c1254b Constantinos Venetsanopoulos
We will install Cyclades (and Plankton) on node1. To do so, we install the
1650 04c1254b Constantinos Venetsanopoulos
corresponding package by running on node1:
1651 04c1254b Constantinos Venetsanopoulos
1652 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1653 04c1254b Constantinos Venetsanopoulos
1654 04c1254b Constantinos Venetsanopoulos
   # apt-get install snf-cyclades-app
1655 04c1254b Constantinos Venetsanopoulos
1656 7d07d924 Constantinos Venetsanopoulos
.. warning:: Make sure you have installed ``python-gevent`` version >= 0.13.6.
1657 7d07d924 Constantinos Venetsanopoulos
    This version is available at squeeze-backports and can be installed by
1658 7d07d924 Constantinos Venetsanopoulos
    running: ``apt-get install -t squeeze-backports python-gevent``
1659 7d07d924 Constantinos Venetsanopoulos
1660 7d07d924 Constantinos Venetsanopoulos
If all packages install successfully, then Cyclades and Plankton are installed
1661 04c1254b Constantinos Venetsanopoulos
and we proceed with their configuration.
1662 04c1254b Constantinos Venetsanopoulos
1663 bc055d09 Constantinos Venetsanopoulos
1664 a96ec00f Constantinos Venetsanopoulos
Configuration of Cyclades (and Plankton)
1665 a96ec00f Constantinos Venetsanopoulos
========================================
1666 5b6feb88 Vangelis Koukis
1667 04c1254b Constantinos Venetsanopoulos
Conf files
1668 04c1254b Constantinos Venetsanopoulos
----------
1669 bc055d09 Constantinos Venetsanopoulos
1670 04c1254b Constantinos Venetsanopoulos
After installing Cyclades, a number of new configuration files will appear under
1671 04c1254b Constantinos Venetsanopoulos
``/etc/synnefo/`` prefixed with ``20-snf-cyclades-app-``. We will descibe here
1672 04c1254b Constantinos Venetsanopoulos
only the minimal needed changes to result with a working system. In general, sane
1673 04c1254b Constantinos Venetsanopoulos
defaults have been chosen for the most of the options, to cover most of the
1674 04c1254b Constantinos Venetsanopoulos
common scenarios. However, if you want to tweak Cyclades feel free to do so,
1675 04c1254b Constantinos Venetsanopoulos
once you get familiar with the different options.
1676 bc055d09 Constantinos Venetsanopoulos
1677 04c1254b Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-cyclades-app-api.conf``:
1678 bc055d09 Constantinos Venetsanopoulos
1679 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1680 04c1254b Constantinos Venetsanopoulos
1681 bc3a7b5a Constantinos Venetsanopoulos
   ASTAKOS_URL = 'https://node1.example.com/im/authenticate'
1682 04c1254b Constantinos Venetsanopoulos
1683 04c1254b Constantinos Venetsanopoulos
The ``ASTAKOS_URL`` denotes the authentication endpoint for Cyclades and is set
1684 04c1254b Constantinos Venetsanopoulos
to point to Astakos (this should have the same value with Pithos+'s
1685 04c1254b Constantinos Venetsanopoulos
``PITHOS_AUTHENTICATION_URL``, setup :ref:`previously <conf-pithos>`).
1686 04c1254b Constantinos Venetsanopoulos
1687 053d0dfc Constantinos Venetsanopoulos
TODO: Document the Network Options here
1688 04c1254b Constantinos Venetsanopoulos
1689 04c1254b Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-cyclades-app-cloudbar.conf``:
1690 04c1254b Constantinos Venetsanopoulos
1691 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1692 04c1254b Constantinos Venetsanopoulos
1693 bc3a7b5a Constantinos Venetsanopoulos
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
1694 de20a465 Constantinos Venetsanopoulos
   CLOUDBAR_ACTIVE_SERVICE = '2'
1695 bc3a7b5a Constantinos Venetsanopoulos
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/im/get_services'
1696 04c1254b Constantinos Venetsanopoulos
   CLOUDBAR_MENU_URL = 'https://account.node1.example.com/im/get_menu'
1697 04c1254b Constantinos Venetsanopoulos
1698 04c1254b Constantinos Venetsanopoulos
``CLOUDBAR_LOCATION`` tells the client where to find the Astakos common
1699 04c1254b Constantinos Venetsanopoulos
cloudbar. The ``CLOUDBAR_SERVICES_URL`` and ``CLOUDBAR_MENU_URL`` options are
1700 04c1254b Constantinos Venetsanopoulos
used by the Cyclades Web UI to get from Astakos all the information needed to
1701 04c1254b Constantinos Venetsanopoulos
fill its own cloudbar. So, we put our Astakos deployment urls there. All the
1702 04c1254b Constantinos Venetsanopoulos
above should have the same values we put in the corresponding variables in
1703 04c1254b Constantinos Venetsanopoulos
``/etc/synnefo/20-snf-pithos-webclient-cloudbar.conf`` on the previous
1704 04c1254b Constantinos Venetsanopoulos
:ref:`Pithos configuration <conf-pithos>` section.
1705 04c1254b Constantinos Venetsanopoulos
1706 de20a465 Constantinos Venetsanopoulos
The ``CLOUDBAR_ACTIVE_SERVICE`` points to an already registered Astakos
1707 de20a465 Constantinos Venetsanopoulos
service. You can see all :ref:`registered services <services-reg>` by running
1708 de20a465 Constantinos Venetsanopoulos
on the Astakos node (node1):
1709 de20a465 Constantinos Venetsanopoulos
1710 de20a465 Constantinos Venetsanopoulos
.. code-block:: console
1711 de20a465 Constantinos Venetsanopoulos
1712 62b10e3f Constantinos Venetsanopoulos
   # snf-manage service-list
1713 de20a465 Constantinos Venetsanopoulos
1714 de20a465 Constantinos Venetsanopoulos
The value of ``CLOUDBAR_ACTIVE_SERVICE`` should be the cyclades service's
1715 de20a465 Constantinos Venetsanopoulos
``id`` as shown by the above command, in our case ``2``.
1716 04c1254b Constantinos Venetsanopoulos
1717 04c1254b Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-cyclades-app-plankton.conf``:
1718 04c1254b Constantinos Venetsanopoulos
1719 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1720 04c1254b Constantinos Venetsanopoulos
1721 04c1254b Constantinos Venetsanopoulos
   BACKEND_DB_CONNECTION = 'postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos'
1722 04c1254b Constantinos Venetsanopoulos
   BACKEND_BLOCK_PATH = '/srv/pithos/data/'
1723 04c1254b Constantinos Venetsanopoulos
1724 04c1254b Constantinos Venetsanopoulos
In this file we configure the Plankton Service. ``BACKEND_DB_CONNECTION``
1725 04c1254b Constantinos Venetsanopoulos
denotes the Pithos+ database (where the Image files are stored). So we set that
1726 04c1254b Constantinos Venetsanopoulos
to point to our Pithos+ database. ``BACKEND_BLOCK_PATH`` denotes the actual
1727 04c1254b Constantinos Venetsanopoulos
Pithos+ data location.
1728 04c1254b Constantinos Venetsanopoulos
1729 04c1254b Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-cyclades-app-queues.conf``:
1730 04c1254b Constantinos Venetsanopoulos
1731 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1732 04c1254b Constantinos Venetsanopoulos
1733 ad003186 Constantinos Venetsanopoulos
   AMQP_HOSTS=["amqp://synnefo:example_rabbitmq_passw0rd@node1.example.com:5672"]
1734 04c1254b Constantinos Venetsanopoulos
1735 04c1254b Constantinos Venetsanopoulos
The above settings denote the Message Queue. Those settings should have the same
1736 04c1254b Constantinos Venetsanopoulos
values as in ``/etc/synnefo/10-snf-cyclades-gtools-backend.conf`` file, and
1737 04c1254b Constantinos Venetsanopoulos
reflect our :ref:`Message Queue setup <rabbitmq-setup>`.
1738 04c1254b Constantinos Venetsanopoulos
1739 04c1254b Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-cyclades-app-ui.conf``:
1740 04c1254b Constantinos Venetsanopoulos
1741 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1742 04c1254b Constantinos Venetsanopoulos
1743 bc3a7b5a Constantinos Venetsanopoulos
   UI_LOGIN_URL = "https://node1.example.com/im/login"
1744 bc3a7b5a Constantinos Venetsanopoulos
   UI_LOGOUT_URL = "https://node1.example.com/im/logout"
1745 04c1254b Constantinos Venetsanopoulos
1746 04c1254b Constantinos Venetsanopoulos
The ``UI_LOGIN_URL`` option tells the Cyclades Web UI where to redirect users,
1747 04c1254b Constantinos Venetsanopoulos
if they are not logged in. We point that to Astakos.
1748 04c1254b Constantinos Venetsanopoulos
1749 04c1254b Constantinos Venetsanopoulos
The ``UI_LOGOUT_URL`` option tells the Cyclades Web UI where to redirect the
1750 04c1254b Constantinos Venetsanopoulos
user when he/she logs out. We point that to Astakos, too.
1751 04c1254b Constantinos Venetsanopoulos
1752 053d0dfc Constantinos Venetsanopoulos
Edit ``/etc/default/vncauthproxy``:
1753 053d0dfc Constantinos Venetsanopoulos
1754 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
1755 053d0dfc Constantinos Venetsanopoulos
1756 053d0dfc Constantinos Venetsanopoulos
   CHUID="www-data:nogroup"
1757 053d0dfc Constantinos Venetsanopoulos
1758 04c1254b Constantinos Venetsanopoulos
We have now finished with the basic Cyclades and Plankton configuration.
1759 04c1254b Constantinos Venetsanopoulos
1760 04c1254b Constantinos Venetsanopoulos
Database Initialization
1761 04c1254b Constantinos Venetsanopoulos
-----------------------
1762 04c1254b Constantinos Venetsanopoulos
1763 04c1254b Constantinos Venetsanopoulos
Once Cyclades is configured, we sync the database:
1764 bc055d09 Constantinos Venetsanopoulos
1765 bc055d09 Constantinos Venetsanopoulos
.. code-block:: console
1766 bc055d09 Constantinos Venetsanopoulos
1767 bc055d09 Constantinos Venetsanopoulos
   $ snf-manage syncdb
1768 bc055d09 Constantinos Venetsanopoulos
   $ snf-manage migrate
1769 bc055d09 Constantinos Venetsanopoulos
1770 04c1254b Constantinos Venetsanopoulos
and load the initial server flavors:
1771 bc055d09 Constantinos Venetsanopoulos
1772 bc055d09 Constantinos Venetsanopoulos
.. code-block:: console
1773 bc055d09 Constantinos Venetsanopoulos
1774 bc055d09 Constantinos Venetsanopoulos
   $ snf-manage loaddata flavors
1775 bc055d09 Constantinos Venetsanopoulos
1776 04c1254b Constantinos Venetsanopoulos
If everything returns successfully, our database is ready.
1777 04c1254b Constantinos Venetsanopoulos
1778 053d0dfc Constantinos Venetsanopoulos
Add the Ganeti backend
1779 053d0dfc Constantinos Venetsanopoulos
----------------------
1780 053d0dfc Constantinos Venetsanopoulos
1781 7a3439cf Constantinos Venetsanopoulos
In our installation we assume that we only have one Ganeti cluster, the one we
1782 b2764de1 Dimitris Aragiorgis
setup earlier.  At this point you have to add this backend (Ganeti cluster) to
1783 b2764de1 Dimitris Aragiorgis
cyclades assuming that you have setup the :ref:`Rapi User <rapi-user>`
1784 b2764de1 Dimitris Aragiorgis
correctly.
1785 b2764de1 Dimitris Aragiorgis
1786 b2764de1 Dimitris Aragiorgis
.. code-block:: console
1787 b2764de1 Dimitris Aragiorgis
1788 69aa7f21 Ilias Tsitsimpis
   $ snf-manage backend-add --clustername=ganeti.node1.example.com --user=cyclades --pass=example_rapi_passw0rd
1789 b2764de1 Dimitris Aragiorgis
1790 b2764de1 Dimitris Aragiorgis
You can see everything has been setup correctly by running:
1791 053d0dfc Constantinos Venetsanopoulos
1792 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
1793 053d0dfc Constantinos Venetsanopoulos
1794 053d0dfc Constantinos Venetsanopoulos
   $ snf-manage backend-list
1795 053d0dfc Constantinos Venetsanopoulos
1796 7a3439cf Constantinos Venetsanopoulos
If something is not set correctly, you can modify the backend with the
1797 7a3439cf Constantinos Venetsanopoulos
``snf-manage backend-modify`` command. If something has gone wrong, you could
1798 7a3439cf Constantinos Venetsanopoulos
modify the backend to reflect the Ganeti installation by running:
1799 053d0dfc Constantinos Venetsanopoulos
1800 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
1801 053d0dfc Constantinos Venetsanopoulos
1802 053d0dfc Constantinos Venetsanopoulos
   $ snf-manage backend-modify --clustername "ganeti.node1.example.com"
1803 7a3439cf Constantinos Venetsanopoulos
                               --user=cyclades
1804 7a3439cf Constantinos Venetsanopoulos
                               --pass=example_rapi_passw0rd
1805 053d0dfc Constantinos Venetsanopoulos
                               1
1806 053d0dfc Constantinos Venetsanopoulos
1807 053d0dfc Constantinos Venetsanopoulos
``clustername`` denotes the Ganeti-cluster's name. We provide the corresponding
1808 053d0dfc Constantinos Venetsanopoulos
domain that resolves to the master IP, than the IP itself, to ensure Cyclades
1809 053d0dfc Constantinos Venetsanopoulos
can talk to Ganeti even after a Ganeti master-failover.
1810 053d0dfc Constantinos Venetsanopoulos
1811 7a3439cf Constantinos Venetsanopoulos
``user`` and ``pass`` denote the RAPI user's username and the RAPI user's
1812 b2764de1 Dimitris Aragiorgis
password.  Once we setup the first backend to point at our Ganeti cluster, we
1813 b2764de1 Dimitris Aragiorgis
update the Cyclades backends status by running:
1814 053d0dfc Constantinos Venetsanopoulos
1815 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
1816 053d0dfc Constantinos Venetsanopoulos
1817 053d0dfc Constantinos Venetsanopoulos
   $ snf-manage backend-update-status
1818 053d0dfc Constantinos Venetsanopoulos
1819 7a3439cf Constantinos Venetsanopoulos
Cyclades can manage multiple Ganeti backends, but for the purpose of this
1820 7a3439cf Constantinos Venetsanopoulos
guide,we won't get into more detail regarding mulitple backends. If you want to
1821 7a3439cf Constantinos Venetsanopoulos
learn more please see /*TODO*/.
1822 7a3439cf Constantinos Venetsanopoulos
1823 7a3439cf Constantinos Venetsanopoulos
Add a Public Network
1824 053d0dfc Constantinos Venetsanopoulos
----------------------
1825 053d0dfc Constantinos Venetsanopoulos
1826 7a3439cf Constantinos Venetsanopoulos
Cyclades supports different Public Networks on different Ganeti backends.
1827 cc16407a Dimitris Aragiorgis
After connecting Cyclades with our Ganeti cluster, we need to setup a Public
1828 cc16407a Dimitris Aragiorgis
Network for this Ganeti backend (`id = 1`). The basic setup is to bridge every
1829 cc16407a Dimitris Aragiorgis
created NIC on a bridge. After having a bridge (e.g. br0) created in every
1830 cc16407a Dimitris Aragiorgis
backend node edit Synnefo setting CUSTOM_BRIDGED_BRIDGE to 'br0':
1831 053d0dfc Constantinos Venetsanopoulos
1832 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
1833 053d0dfc Constantinos Venetsanopoulos
1834 69aa7f21 Ilias Tsitsimpis
   $ snf-manage network-create --subnet=5.6.7.0/27 \
1835 69aa7f21 Ilias Tsitsimpis
                               --gateway=5.6.7.1 \
1836 69aa7f21 Ilias Tsitsimpis
                               --subnet6=2001:648:2FFC:1322::/64 \
1837 69aa7f21 Ilias Tsitsimpis
                               --gateway6=2001:648:2FFC:1322::1 \
1838 18e300b2 Christos Stavrakakis
                               --public --dhcp --flavor=CUSTOM \
1839 92e4d1c6 Dimitris Aragiorgis
                               --link=br0 --mode=bridged \
1840 69aa7f21 Ilias Tsitsimpis
                               --name=public_network \
1841 7a3439cf Constantinos Venetsanopoulos
                               --backend-id=1
1842 053d0dfc Constantinos Venetsanopoulos
1843 053d0dfc Constantinos Venetsanopoulos
This will create the Public Network on both Cyclades and the Ganeti backend. To
1844 053d0dfc Constantinos Venetsanopoulos
make sure everything was setup correctly, also run:
1845 053d0dfc Constantinos Venetsanopoulos
1846 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
1847 053d0dfc Constantinos Venetsanopoulos
1848 053d0dfc Constantinos Venetsanopoulos
   $ snf-manage reconcile-networks
1849 053d0dfc Constantinos Venetsanopoulos
1850 053d0dfc Constantinos Venetsanopoulos
You can see all available networks by running:
1851 053d0dfc Constantinos Venetsanopoulos
1852 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
1853 053d0dfc Constantinos Venetsanopoulos
1854 7a3439cf Constantinos Venetsanopoulos
   $ snf-manage network-list
1855 053d0dfc Constantinos Venetsanopoulos
1856 053d0dfc Constantinos Venetsanopoulos
and inspect each network's state by running:
1857 053d0dfc Constantinos Venetsanopoulos
1858 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
1859 053d0dfc Constantinos Venetsanopoulos
1860 053d0dfc Constantinos Venetsanopoulos
   $ snf-manage network-inspect <net_id>
1861 053d0dfc Constantinos Venetsanopoulos
1862 053d0dfc Constantinos Venetsanopoulos
Finally, you can see the networks from the Ganeti perspective by running on the
1863 053d0dfc Constantinos Venetsanopoulos
Ganeti MASTER:
1864 053d0dfc Constantinos Venetsanopoulos
1865 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
1866 053d0dfc Constantinos Venetsanopoulos
1867 053d0dfc Constantinos Venetsanopoulos
   $ gnt-network list
1868 053d0dfc Constantinos Venetsanopoulos
   $ gnt-network info <network_name>
1869 053d0dfc Constantinos Venetsanopoulos
1870 19425707 Christos Stavrakakis
1871 19425707 Christos Stavrakakis
Create pools for Private Networks
1872 19425707 Christos Stavrakakis
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1873 19425707 Christos Stavrakakis
1874 19425707 Christos Stavrakakis
To prevent duplicate assignment of resources to different private networks,
1875 19425707 Christos Stavrakakis
Cyclades supports two types of pools:
1876 19425707 Christos Stavrakakis
1877 19425707 Christos Stavrakakis
 - MAC prefix Pool
1878 19425707 Christos Stavrakakis
 - Bridge Pool
1879 19425707 Christos Stavrakakis
1880 19425707 Christos Stavrakakis
As long as those resourses have been provisioned, admin has to define two
1881 19425707 Christos Stavrakakis
these pools in Synnefo:
1882 19425707 Christos Stavrakakis
1883 19425707 Christos Stavrakakis
1884 19425707 Christos Stavrakakis
.. code-block:: console
1885 19425707 Christos Stavrakakis
1886 19425707 Christos Stavrakakis
   root@testvm1:~ # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
1887 19425707 Christos Stavrakakis
1888 19425707 Christos Stavrakakis
   root@testvm1:~ # snf-manage pool-create --type=bridge --base=prv --size=20
1889 19425707 Christos Stavrakakis
1890 19425707 Christos Stavrakakis
Also, change the Synnefo setting in :file:`20-snf-cyclades-app-api.conf`:
1891 19425707 Christos Stavrakakis
1892 19425707 Christos Stavrakakis
.. code-block:: console
1893 19425707 Christos Stavrakakis
1894 18e300b2 Christos Stavrakakis
   DEFAULT_MAC_FILTERED_BRIDGE = 'prv0'
1895 19425707 Christos Stavrakakis
1896 04c1254b Constantinos Venetsanopoulos
Servers restart
1897 04c1254b Constantinos Venetsanopoulos
---------------
1898 04c1254b Constantinos Venetsanopoulos
1899 053d0dfc Constantinos Venetsanopoulos
Restart gunicorn on node1:
1900 04c1254b Constantinos Venetsanopoulos
1901 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1902 bc055d09 Constantinos Venetsanopoulos
1903 04c1254b Constantinos Venetsanopoulos
   # /etc/init.d/gunicorn restart
1904 bc055d09 Constantinos Venetsanopoulos
1905 04c1254b Constantinos Venetsanopoulos
Now let's do the final connections of Cyclades with Ganeti.
1906 bc055d09 Constantinos Venetsanopoulos
1907 04c1254b Constantinos Venetsanopoulos
``snf-dispatcher`` initialization
1908 04c1254b Constantinos Venetsanopoulos
---------------------------------
1909 04c1254b Constantinos Venetsanopoulos
1910 04c1254b Constantinos Venetsanopoulos
``snf-dispatcher`` dispatches all messages published to the Message Queue and
1911 04c1254b Constantinos Venetsanopoulos
manages the Cyclades database accordingly. It also initializes all exchanges. By
1912 04c1254b Constantinos Venetsanopoulos
default it is not enabled during installation of Cyclades, so let's enable it in
1913 04c1254b Constantinos Venetsanopoulos
its configuration file ``/etc/default/snf-dispatcher``:
1914 04c1254b Constantinos Venetsanopoulos
1915 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1916 04c1254b Constantinos Venetsanopoulos
1917 04c1254b Constantinos Venetsanopoulos
   SNF_DSPTCH_ENABLE=true
1918 04c1254b Constantinos Venetsanopoulos
1919 04c1254b Constantinos Venetsanopoulos
and start the daemon:
1920 04c1254b Constantinos Venetsanopoulos
1921 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1922 04c1254b Constantinos Venetsanopoulos
1923 04c1254b Constantinos Venetsanopoulos
   # /etc/init.d/snf-dispatcher start
1924 04c1254b Constantinos Venetsanopoulos
1925 04c1254b Constantinos Venetsanopoulos
You can see that everything works correctly by tailing its log file
1926 04c1254b Constantinos Venetsanopoulos
``/var/log/synnefo/dispatcher.log``.
1927 04c1254b Constantinos Venetsanopoulos
1928 04c1254b Constantinos Venetsanopoulos
``snf-ganeti-eventd`` on GANETI MASTER
1929 04c1254b Constantinos Venetsanopoulos
--------------------------------------
1930 04c1254b Constantinos Venetsanopoulos
1931 04c1254b Constantinos Venetsanopoulos
The last step of the Cyclades setup is enabling the ``snf-ganeti-eventd``
1932 04c1254b Constantinos Venetsanopoulos
daemon (part of the :ref:`Cyclades Ganeti tools <cyclades-gtools>` package).
1933 04c1254b Constantinos Venetsanopoulos
The daemon is already installed on the GANETI MASTER (node1 in our case).
1934 04c1254b Constantinos Venetsanopoulos
``snf-ganeti-eventd`` is disabled by default during the ``snf-cyclades-gtools``
1935 04c1254b Constantinos Venetsanopoulos
installation, so we enable it in its configuration file
1936 04c1254b Constantinos Venetsanopoulos
``/etc/default/snf-ganeti-eventd``:
1937 04c1254b Constantinos Venetsanopoulos
1938 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1939 04c1254b Constantinos Venetsanopoulos
1940 04c1254b Constantinos Venetsanopoulos
   SNF_EVENTD_ENABLE=true
1941 04c1254b Constantinos Venetsanopoulos
1942 04c1254b Constantinos Venetsanopoulos
and start the daemon:
1943 04c1254b Constantinos Venetsanopoulos
1944 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1945 bc055d09 Constantinos Venetsanopoulos
1946 04c1254b Constantinos Venetsanopoulos
   # /etc/init.d/snf-ganeti-eventd start
1947 bc055d09 Constantinos Venetsanopoulos
1948 04c1254b Constantinos Venetsanopoulos
.. warning:: Make sure you start ``snf-ganeti-eventd`` *ONLY* on GANETI MASTER
1949 bc055d09 Constantinos Venetsanopoulos
1950 04c1254b Constantinos Venetsanopoulos
If all the above return successfully, then you have finished with the Cyclades
1951 04c1254b Constantinos Venetsanopoulos
and Plankton installation and setup. Let's test our installation now.
1952 bc055d09 Constantinos Venetsanopoulos
1953 5b6feb88 Vangelis Koukis
1954 a96ec00f Constantinos Venetsanopoulos
Testing of Cyclades (and Plankton)
1955 a96ec00f Constantinos Venetsanopoulos
==================================
1956 5b6feb88 Vangelis Koukis
1957 8a4cd31b Constantinos Venetsanopoulos
Cyclades Web UI
1958 8a4cd31b Constantinos Venetsanopoulos
---------------
1959 8a4cd31b Constantinos Venetsanopoulos
1960 8a4cd31b Constantinos Venetsanopoulos
First of all we need to test that our Cyclades Web UI works correctly. Open your
1961 8a4cd31b Constantinos Venetsanopoulos
browser and go to the Astakos home page. Login and then click 'cyclades' on the
1962 8a4cd31b Constantinos Venetsanopoulos
top cloud bar. This should redirect you to:
1963 8a4cd31b Constantinos Venetsanopoulos
1964 8a4cd31b Constantinos Venetsanopoulos
 `http://node1.example.com/ui/`
1965 8a4cd31b Constantinos Venetsanopoulos
1966 8a4cd31b Constantinos Venetsanopoulos
and the Cyclades home page should appear. If not, please go back and find what
1967 8a4cd31b Constantinos Venetsanopoulos
went wrong. Do not proceed if you don't see the Cyclades home page.
1968 8a4cd31b Constantinos Venetsanopoulos
1969 8a4cd31b Constantinos Venetsanopoulos
If the Cyclades home page appears, click on the orange button 'New machine'. The
1970 8a4cd31b Constantinos Venetsanopoulos
first step of the 'New machine wizard' will appear. This step shows all the
1971 8a4cd31b Constantinos Venetsanopoulos
available Images from which you can spawn new VMs. The list should be currently
1972 8a4cd31b Constantinos Venetsanopoulos
empty, as we haven't registered any Images yet. Close the wizard and browse the
1973 8a4cd31b Constantinos Venetsanopoulos
interface (not many things to see yet). If everything seems to work, let's
1974 8a4cd31b Constantinos Venetsanopoulos
register our first Image file.
1975 8a4cd31b Constantinos Venetsanopoulos
1976 8a4cd31b Constantinos Venetsanopoulos
Cyclades Images
1977 8a4cd31b Constantinos Venetsanopoulos
---------------
1978 8a4cd31b Constantinos Venetsanopoulos
1979 8a4cd31b Constantinos Venetsanopoulos
To test our Cyclades (and Plankton) installation, we will use an Image stored on
1980 8a4cd31b Constantinos Venetsanopoulos
Pithos+ to spawn a new VM from the Cyclades interface. We will describe all
1981 8a4cd31b Constantinos Venetsanopoulos
steps, even though you may already have uploaded an Image on Pithos+ from a
1982 8a4cd31b Constantinos Venetsanopoulos
:ref:`previous <snf-image-images>` section:
1983 8a4cd31b Constantinos Venetsanopoulos
1984 8a4cd31b Constantinos Venetsanopoulos
 * Upload an Image file to Pithos+
1985 8a4cd31b Constantinos Venetsanopoulos
 * Register that Image file to Plankton
1986 8a4cd31b Constantinos Venetsanopoulos
 * Spawn a new VM from that Image from the Cyclades Web UI
1987 8a4cd31b Constantinos Venetsanopoulos
1988 8a4cd31b Constantinos Venetsanopoulos
We will use the `kamaki <http://docs.dev.grnet.gr/kamaki/latest/index.html>`_
1989 8a4cd31b Constantinos Venetsanopoulos
command line client to do the uploading and registering of the Image.
1990 8a4cd31b Constantinos Venetsanopoulos
1991 8a4cd31b Constantinos Venetsanopoulos
Installation of `kamaki`
1992 8a4cd31b Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~
1993 8a4cd31b Constantinos Venetsanopoulos
1994 8a4cd31b Constantinos Venetsanopoulos
You can install `kamaki` anywhere you like, since it is a standalone client of
1995 8a4cd31b Constantinos Venetsanopoulos
the APIs and talks to the installation over `http`. For the purpose of this
1996 8a4cd31b Constantinos Venetsanopoulos
guide we will assume that we have downloaded the `Debian Squeeze Base Image
1997 8a4cd31b Constantinos Venetsanopoulos
<https://pithos.okeanos.grnet.gr/public/9epgb>`_ and stored it under node1's
1998 8a4cd31b Constantinos Venetsanopoulos
``/srv/images`` directory. For that reason we will install `kamaki` on node1,
1999 8a4cd31b Constantinos Venetsanopoulos
too. We do this by running:
2000 8a4cd31b Constantinos Venetsanopoulos
2001 8a4cd31b Constantinos Venetsanopoulos
.. code-block:: console
2002 8a4cd31b Constantinos Venetsanopoulos
2003 8a4cd31b Constantinos Venetsanopoulos
   # apt-get install kamaki
2004 8a4cd31b Constantinos Venetsanopoulos
2005 8a4cd31b Constantinos Venetsanopoulos
Configuration of kamaki
2006 8a4cd31b Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~
2007 8a4cd31b Constantinos Venetsanopoulos
2008 8a4cd31b Constantinos Venetsanopoulos
Now we need to setup kamaki, by adding the appropriate URLs and tokens of our
2009 8a4cd31b Constantinos Venetsanopoulos
installation. We do this by running:
2010 8a4cd31b Constantinos Venetsanopoulos
2011 8a4cd31b Constantinos Venetsanopoulos
.. code-block:: console
2012 8a4cd31b Constantinos Venetsanopoulos
2013 8a4cd31b Constantinos Venetsanopoulos
   $ kamaki config set astakos.url "https://node1.example.com"
2014 69aa7f21 Ilias Tsitsimpis
   $ kamaki config set compute.url "https://node1.example.com/api/v1.1"
2015 69aa7f21 Ilias Tsitsimpis
   $ kamaki config set image.url "https://node1.example.com/plankton"
2016 69aa7f21 Ilias Tsitsimpis
   $ kamaki config set store.url "https://node2.example.com/v1"
2017 69aa7f21 Ilias Tsitsimpis
   $ kamaki config set global.account "user@example.com"
2018 92e4d1c6 Dimitris Aragiorgis
   $ kamaki config set store.enable on
2019 92e4d1c6 Dimitris Aragiorgis
   $ kamaki config set store.pithos_extensions on
2020 92e4d1c6 Dimitris Aragiorgis
   $ kamaki config set store.url "https://node2.example.com/v1"
2021 92e4d1c6 Dimitris Aragiorgis
   $ kamaki config set store.account USER_UUID
2022 92e4d1c6 Dimitris Aragiorgis
   $ kamaki config set global.token USER_TOKEN
2023 8a4cd31b Constantinos Venetsanopoulos
2024 92e4d1c6 Dimitris Aragiorgis
The USER_TOKEN and USER_UUID appear on the user's (``user@example.com``) `Profile` web
2025 92e4d1c6 Dimitris Aragiorgis
page on the Astakos Web UI.
2026 8a4cd31b Constantinos Venetsanopoulos
2027 8a4cd31b Constantinos Venetsanopoulos
You can see that the new configuration options have been applied correctly, by
2028 8a4cd31b Constantinos Venetsanopoulos
running:
2029 8a4cd31b Constantinos Venetsanopoulos
2030 8a4cd31b Constantinos Venetsanopoulos
.. code-block:: console
2031 8a4cd31b Constantinos Venetsanopoulos
2032 8a4cd31b Constantinos Venetsanopoulos
   $ kamaki config list
2033 8a4cd31b Constantinos Venetsanopoulos
2034 8a4cd31b Constantinos Venetsanopoulos
Upload an Image file to Pithos+
2035 8a4cd31b Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2036 8a4cd31b Constantinos Venetsanopoulos
2037 8a4cd31b Constantinos Venetsanopoulos
Now, that we have set up `kamaki` we will upload the Image that we have
2038 8a4cd31b Constantinos Venetsanopoulos
downloaded and stored under ``/srv/images/``. Although we can upload the Image
2039 8a4cd31b Constantinos Venetsanopoulos
under the root ``Pithos`` container (as you may have done when uploading the
2040 8a4cd31b Constantinos Venetsanopoulos
Image from the Pithos+ Web UI), we will create a new container called ``images``
2041 8a4cd31b Constantinos Venetsanopoulos
and store the Image under that container. We do this for two reasons:
2042 8a4cd31b Constantinos Venetsanopoulos
2043 8a4cd31b Constantinos Venetsanopoulos
a) To demonstrate how to create containers other than the default ``Pithos``.
2044 8a4cd31b Constantinos Venetsanopoulos
   This can be done only with the `kamaki` client and not through the Web UI.
2045 8a4cd31b Constantinos Venetsanopoulos
2046 8a4cd31b Constantinos Venetsanopoulos
b) As a best organization practise, so that you won't have your Image files
2047 8a4cd31b Constantinos Venetsanopoulos
   tangled along with all your other Pithos+ files and directory structures.
2048 8a4cd31b Constantinos Venetsanopoulos
2049 8a4cd31b Constantinos Venetsanopoulos
We create the new ``images`` container by running:
2050 8a4cd31b Constantinos Venetsanopoulos
2051 8a4cd31b Constantinos Venetsanopoulos
.. code-block:: console
2052 8a4cd31b Constantinos Venetsanopoulos
2053 8a4cd31b Constantinos Venetsanopoulos
   $ kamaki store create images
2054 8a4cd31b Constantinos Venetsanopoulos
2055 8a4cd31b Constantinos Venetsanopoulos
Then, we upload the Image file to that container:
2056 8a4cd31b Constantinos Venetsanopoulos
2057 8a4cd31b Constantinos Venetsanopoulos
.. code-block:: console
2058 8a4cd31b Constantinos Venetsanopoulos
2059 8a4cd31b Constantinos Venetsanopoulos
   $ kamaki store upload --container images \
2060 8a4cd31b Constantinos Venetsanopoulos
                         /srv/images/debian_base-6.0-7-x86_64.diskdump \
2061 8a4cd31b Constantinos Venetsanopoulos
                         debian_base-6.0-7-x86_64.diskdump
2062 8a4cd31b Constantinos Venetsanopoulos
2063 8a4cd31b Constantinos Venetsanopoulos
The first is the local path and the second is the remote path on Pithos+. If
2064 8a4cd31b Constantinos Venetsanopoulos
the new container and the file appears on the Pithos+ Web UI, then you have
2065 8a4cd31b Constantinos Venetsanopoulos
successfully created the container and uploaded the Image file.
2066 8a4cd31b Constantinos Venetsanopoulos
2067 8a4cd31b Constantinos Venetsanopoulos
Register an existing Image file to Plankton
2068 8a4cd31b Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2069 8a4cd31b Constantinos Venetsanopoulos
2070 8a4cd31b Constantinos Venetsanopoulos
Once the Image file has been successfully uploaded on Pithos+, then we register
2071 8a4cd31b Constantinos Venetsanopoulos
it to Plankton (so that it becomes visible to Cyclades), by running:
2072 8a4cd31b Constantinos Venetsanopoulos
2073 8a4cd31b Constantinos Venetsanopoulos
.. code-block:: console
2074 8a4cd31b Constantinos Venetsanopoulos
2075 69aa7f21 Ilias Tsitsimpis
   $ kamaki image register "Debian Base" \
2076 92e4d1c6 Dimitris Aragiorgis
                           pithos://USER_UUID/images/debian_base-6.0-7-x86_64.diskdump \
2077 69aa7f21 Ilias Tsitsimpis
                           --public \
2078 69aa7f21 Ilias Tsitsimpis
                           --disk-format=diskdump \
2079 69aa7f21 Ilias Tsitsimpis
                           --property OSFAMILY=linux --property ROOT_PARTITION=1 \
2080 69aa7f21 Ilias Tsitsimpis
                           --property description="Debian Squeeze Base System" \
2081 69aa7f21 Ilias Tsitsimpis
                           --property size=451 --property kernel=2.6.32 --property GUI="No GUI" \
2082 8a4cd31b Constantinos Venetsanopoulos
                           --property sortorder=1 --property USERS=root --property OS=debian
2083 8a4cd31b Constantinos Venetsanopoulos
2084 8a4cd31b Constantinos Venetsanopoulos
This command registers the Pithos+ file
2085 69aa7f21 Ilias Tsitsimpis
``pithos://user@example.com/images/debian_base-6.0-7-x86_64.diskdump`` as an
2086 8a4cd31b Constantinos Venetsanopoulos
Image in Plankton. This Image will be public (``--public``), so all users will
2087 8a4cd31b Constantinos Venetsanopoulos
be able to spawn VMs from it and is of type ``diskdump``. The first two
2088 8a4cd31b Constantinos Venetsanopoulos
properties (``OSFAMILY`` and ``ROOT_PARTITION``) are mandatory. All the rest
2089 8a4cd31b Constantinos Venetsanopoulos
properties are optional, but recommended, so that the Images appear nicely on
2090 8a4cd31b Constantinos Venetsanopoulos
the Cyclades Web UI. ``Debian Base`` will appear as the name of this Image. The
2091 8a4cd31b Constantinos Venetsanopoulos
``OS`` property's valid values may be found in the ``IMAGE_ICONS`` variable
2092 8a4cd31b Constantinos Venetsanopoulos
inside the ``20-snf-cyclades-app-ui.conf`` configuration file.
2093 8a4cd31b Constantinos Venetsanopoulos
2094 8a4cd31b Constantinos Venetsanopoulos
``OSFAMILY`` and ``ROOT_PARTITION`` are mandatory because they will be passed
2095 8a4cd31b Constantinos Venetsanopoulos
from Plankton to Cyclades and then to Ganeti and `snf-image` (also see
2096 8a4cd31b Constantinos Venetsanopoulos
:ref:`previous section <ganeti-with-pithos-images>`). All other properties are
2097 8a4cd31b Constantinos Venetsanopoulos
used to show information on the Cyclades UI.
2098 8a4cd31b Constantinos Venetsanopoulos
2099 8a4cd31b Constantinos Venetsanopoulos
Spawn a VM from the Cyclades Web UI
2100 8a4cd31b Constantinos Venetsanopoulos
-----------------------------------
2101 8a4cd31b Constantinos Venetsanopoulos
2102 8a4cd31b Constantinos Venetsanopoulos
If the registration completes successfully, then go to the Cyclades Web UI from
2103 8a4cd31b Constantinos Venetsanopoulos
your browser at:
2104 8a4cd31b Constantinos Venetsanopoulos
2105 8a4cd31b Constantinos Venetsanopoulos
 `https://node1.example.com/ui/`
2106 8a4cd31b Constantinos Venetsanopoulos
2107 8a4cd31b Constantinos Venetsanopoulos
Click on the 'New Machine' button and the first step of the wizard will appear.
2108 8a4cd31b Constantinos Venetsanopoulos
Click on 'My Images' (right after 'System' Images) on the left pane of the
2109 8a4cd31b Constantinos Venetsanopoulos
wizard. Your previously registered Image "Debian Base" should appear under
2110 8a4cd31b Constantinos Venetsanopoulos
'Available Images'. If not, something has gone wrong with the registration. Make
2111 8a4cd31b Constantinos Venetsanopoulos
sure you can see your Image file on the Pithos+ Web UI and ``kamaki image
2112 8a4cd31b Constantinos Venetsanopoulos
register`` returns successfully with all options and properties as shown above.
2113 8a4cd31b Constantinos Venetsanopoulos
2114 8a4cd31b Constantinos Venetsanopoulos
If the Image appears on the list, select it and complete the wizard by selecting
2115 8a4cd31b Constantinos Venetsanopoulos
a flavor and a name for your VM. Then finish by clicking 'Create'. Make sure you
2116 8a4cd31b Constantinos Venetsanopoulos
write down your password, because you *WON'T* be able to retrieve it later.
2117 8a4cd31b Constantinos Venetsanopoulos
2118 8a4cd31b Constantinos Venetsanopoulos
If everything was setup correctly, after a few minutes your new machine will go
2119 8a4cd31b Constantinos Venetsanopoulos
to state 'Running' and you will be able to use it. Click 'Console' to connect
2120 8a4cd31b Constantinos Venetsanopoulos
through VNC out of band, or click on the machine's icon to connect directly via
2121 8a4cd31b Constantinos Venetsanopoulos
SSH or RDP (for windows machines).
2122 8a4cd31b Constantinos Venetsanopoulos
2123 8a4cd31b Constantinos Venetsanopoulos
Congratulations. You have successfully installed the whole Synnefo stack and
2124 8a4cd31b Constantinos Venetsanopoulos
connected all components. Go ahead in the next section to test the Network
2125 8a4cd31b Constantinos Venetsanopoulos
functionality from inside Cyclades and discover even more features.
2126 8a4cd31b Constantinos Venetsanopoulos
2127 5b6feb88 Vangelis Koukis
2128 a96ec00f Constantinos Venetsanopoulos
General Testing
2129 a96ec00f Constantinos Venetsanopoulos
===============
2130 5b6feb88 Vangelis Koukis
2131 5b6feb88 Vangelis Koukis
2132 a96ec00f Constantinos Venetsanopoulos
Notes
2133 a96ec00f Constantinos Venetsanopoulos
=====