Statistics
| Branch: | Tag: | Revision:

root / docs / quick-install-admin-guide.rst @ a1d0bacb

History | View | Annotate | Download (82.8 kB)

1 bdb83fd6 cven
.. _quick-install-admin-guide:
2 bdb83fd6 cven
3 454dca28 Constantinos Venetsanopoulos
Administrator's Installation Guide
4 454dca28 Constantinos Venetsanopoulos
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5 a96ec00f Constantinos Venetsanopoulos
6 454dca28 Constantinos Venetsanopoulos
This is the Administrator's installation guide.
7 a96ec00f Constantinos Venetsanopoulos
8 f8cdf6ec Dionysis Grigoropoulos
It describes how to install the whole Synnefo stack on two (2) physical nodes,
9 a96ec00f Constantinos Venetsanopoulos
with minimum configuration. It installs synnefo from Debian packages, and
10 f8cdf6ec Dionysis Grigoropoulos
assumes the nodes run Debian Wheezy. After successful installation, you will
11 a96ec00f Constantinos Venetsanopoulos
have the following services running:
12 a96ec00f Constantinos Venetsanopoulos
13 d587329c Konstantinos Tompoulidis
    * Identity Management (Astakos)
14 e5d8df8c Constantinos Venetsanopoulos
    * Object Storage Service (Pithos)
15 d587329c Konstantinos Tompoulidis
    * Compute Service (Cyclades)
16 73ebcd68 Constantinos Venetsanopoulos
    * Image Service (part of Cyclades)
17 73ebcd68 Constantinos Venetsanopoulos
    * Network Service (part of Cyclades)
18 a96ec00f Constantinos Venetsanopoulos
19 a96ec00f Constantinos Venetsanopoulos
and a single unified Web UI to manage them all.
20 a96ec00f Constantinos Venetsanopoulos
21 e5d8df8c Constantinos Venetsanopoulos
If you just want to install the Object Storage Service (Pithos), follow the
22 e5d8df8c Constantinos Venetsanopoulos
guide and just stop after the "Testing of Pithos" section.
23 a96ec00f Constantinos Venetsanopoulos
24 a96ec00f Constantinos Venetsanopoulos
25 a96ec00f Constantinos Venetsanopoulos
Installation of Synnefo / Introduction
26 a96ec00f Constantinos Venetsanopoulos
======================================
27 a96ec00f Constantinos Venetsanopoulos
28 73ebcd68 Constantinos Venetsanopoulos
We will install the services with the above list's order. The last three
29 73ebcd68 Constantinos Venetsanopoulos
services will be installed in a single step (at the end), because at the moment
30 73ebcd68 Constantinos Venetsanopoulos
they are contained in the same software component (Cyclades). Furthermore, we
31 e5d8df8c Constantinos Venetsanopoulos
will install all services in the first physical node, except Pithos which will
32 73ebcd68 Constantinos Venetsanopoulos
be installed in the second, due to a conflict between the snf-pithos-app and
33 73ebcd68 Constantinos Venetsanopoulos
snf-cyclades-app component (scheduled to be fixed in the next version).
34 a96ec00f Constantinos Venetsanopoulos
35 a96ec00f Constantinos Venetsanopoulos
For the rest of the documentation we will refer to the first physical node as
36 a96ec00f Constantinos Venetsanopoulos
"node1" and the second as "node2". We will also assume that their domain names
37 f8cdf6ec Dionysis Grigoropoulos
are "node1.example.com" and "node2.example.com" and their public IPs are "203.0.113.1" and
38 f8cdf6ec Dionysis Grigoropoulos
"203.0.113.2" respectively. It is important that the two machines are under the same domain name.
39 0c068fc6 marioskogias
In case you choose to follow a private installation you will need to
40 dd6062f2 Christos Stavrakakis
set up a private dns server, using dnsmasq for example. See node1 below for
41 f8cdf6ec Dionysis Grigoropoulos
more information on how to do so.
42 a96ec00f Constantinos Venetsanopoulos
43 a96ec00f Constantinos Venetsanopoulos
General Prerequisites
44 a96ec00f Constantinos Venetsanopoulos
=====================
45 a96ec00f Constantinos Venetsanopoulos
46 a96ec00f Constantinos Venetsanopoulos
These are the general synnefo prerequisites, that you need on node1 and node2
47 e5d8df8c Constantinos Venetsanopoulos
and are related to all the services (Astakos, Pithos, Cyclades).
48 a96ec00f Constantinos Venetsanopoulos
49 a96ec00f Constantinos Venetsanopoulos
To be able to download all synnefo components you need to add the following
50 a96ec00f Constantinos Venetsanopoulos
lines in your ``/etc/apt/sources.list`` file:
51 a96ec00f Constantinos Venetsanopoulos
52 f8cdf6ec Dionysis Grigoropoulos
| ``deb http://apt.dev.grnet.gr wheezy/``
53 f8cdf6ec Dionysis Grigoropoulos
| ``deb-src http://apt.dev.grnet.gr wheezy/``
54 a96ec00f Constantinos Venetsanopoulos
55 ec9862dd Stratos Psomadakis
and import the repo's GPG key:
56 ec9862dd Stratos Psomadakis
57 ec9862dd Stratos Psomadakis
| ``curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -``
58 ec9862dd Stratos Psomadakis
59 f8cdf6ec Dionysis Grigoropoulos
Update your list of packages and continue with the installation:
60 f8cdf6ec Dionysis Grigoropoulos
61 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
62 169f7d38 Vangelis Koukis
63 f8cdf6ec Dionysis Grigoropoulos
   # apt-get update
64 169f7d38 Vangelis Koukis
65 e5d8df8c Constantinos Venetsanopoulos
You also need a shared directory visible by both nodes. Pithos will save all
66 f8cdf6ec Dionysis Grigoropoulos
data inside this directory. By 'all data', we mean files, images, and Pithos
67 a96ec00f Constantinos Venetsanopoulos
specific mapping data. If you plan to upload more than one basic image, this
68 a96ec00f Constantinos Venetsanopoulos
directory should have at least 50GB of free space. During this guide, we will
69 a96ec00f Constantinos Venetsanopoulos
assume that node1 acts as an NFS server and serves the directory ``/srv/pithos``
70 ba173277 Ilias Tsitsimpis
to node2 (be sure to set no_root_squash flag). Node2 has this directory
71 ba173277 Ilias Tsitsimpis
mounted under ``/srv/pithos``, too.
72 a96ec00f Constantinos Venetsanopoulos
73 a96ec00f Constantinos Venetsanopoulos
Before starting the synnefo installation, you will need basic third party
74 a96ec00f Constantinos Venetsanopoulos
software to be installed and configured on the physical nodes. We will describe
75 a96ec00f Constantinos Venetsanopoulos
each node's general prerequisites separately. Any additional configuration,
76 a96ec00f Constantinos Venetsanopoulos
specific to a synnefo service for each node, will be described at the service's
77 a96ec00f Constantinos Venetsanopoulos
section.
78 a96ec00f Constantinos Venetsanopoulos
79 7df5a742 Christos Stavrakakis
Finally, it is required for Cyclades and Ganeti nodes to have synchronized
80 7df5a742 Christos Stavrakakis
system clocks (e.g. by running ntpd).
81 7df5a742 Christos Stavrakakis
82 a96ec00f Constantinos Venetsanopoulos
Node1
83 a96ec00f Constantinos Venetsanopoulos
-----
84 a96ec00f Constantinos Venetsanopoulos
85 0c068fc6 marioskogias
86 a96ec00f Constantinos Venetsanopoulos
General Synnefo dependencies
87 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
88 a96ec00f Constantinos Venetsanopoulos
89 0c068fc6 marioskogias
		* apache (http server)
90 0c068fc6 marioskogias
		* public certificate
91 0c068fc6 marioskogias
		* gunicorn (WSGI http server)
92 0c068fc6 marioskogias
		* postgresql (database)
93 0c068fc6 marioskogias
		* rabbitmq (message queue)
94 0c068fc6 marioskogias
		* ntp (NTP daemon)
95 0c068fc6 marioskogias
		* gevent
96 f8cdf6ec Dionysis Grigoropoulos
		* dnsmasq (DNS server)
97 a96ec00f Constantinos Venetsanopoulos
98 d75bf4c1 Christos Stavrakakis
You can install apache2, postgresql, ntp and rabbitmq by running:
99 a96ec00f Constantinos Venetsanopoulos
100 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
101 a96ec00f Constantinos Venetsanopoulos
102 d75bf4c1 Christos Stavrakakis
   # apt-get install apache2 postgresql ntp rabbitmq-server
103 a96ec00f Constantinos Venetsanopoulos
104 f8cdf6ec Dionysis Grigoropoulos
To install gunicorn and gevent, run:
105 73ff1d54 Constantinos Venetsanopoulos
106 73ff1d54 Constantinos Venetsanopoulos
.. code-block:: console
107 73ff1d54 Constantinos Venetsanopoulos
108 f8cdf6ec Dionysis Grigoropoulos
   # apt-get install gunicorn python-gevent
109 576e059b Constantinos Venetsanopoulos
110 73ff1d54 Constantinos Venetsanopoulos
On node1, we will create our databases, so you will also need the
111 73ff1d54 Constantinos Venetsanopoulos
python-psycopg2 package:
112 a96ec00f Constantinos Venetsanopoulos
113 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
114 a96ec00f Constantinos Venetsanopoulos
115 a96ec00f Constantinos Venetsanopoulos
   # apt-get install python-psycopg2
116 a96ec00f Constantinos Venetsanopoulos
117 a96ec00f Constantinos Venetsanopoulos
Database setup
118 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~
119 a96ec00f Constantinos Venetsanopoulos
120 a96ec00f Constantinos Venetsanopoulos
On node1, we create a database called ``snf_apps``, that will host all django
121 a96ec00f Constantinos Venetsanopoulos
apps related tables. We also create the user ``synnefo`` and grant him all
122 a96ec00f Constantinos Venetsanopoulos
privileges on the database. We do this by running:
123 a96ec00f Constantinos Venetsanopoulos
124 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
125 a96ec00f Constantinos Venetsanopoulos
126 d587329c Konstantinos Tompoulidis
    root@node1:~ # su - postgres
127 d587329c Konstantinos Tompoulidis
    postgres@node1:~ $ psql
128 d587329c Konstantinos Tompoulidis
    postgres=# CREATE DATABASE snf_apps WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
129 d587329c Konstantinos Tompoulidis
    postgres=# CREATE USER synnefo WITH PASSWORD 'example_passw0rd';
130 d587329c Konstantinos Tompoulidis
    postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_apps TO synnefo;
131 a96ec00f Constantinos Venetsanopoulos
132 e5d8df8c Constantinos Venetsanopoulos
We also create the database ``snf_pithos`` needed by the Pithos backend and
133 a96ec00f Constantinos Venetsanopoulos
grant the ``synnefo`` user all privileges on the database. This database could
134 a96ec00f Constantinos Venetsanopoulos
be created on node2 instead, but we do it on node1 for simplicity. We will
135 a96ec00f Constantinos Venetsanopoulos
create all needed databases on node1 and then node2 will connect to them.
136 a96ec00f Constantinos Venetsanopoulos
137 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
138 a96ec00f Constantinos Venetsanopoulos
139 d587329c Konstantinos Tompoulidis
    postgres=# CREATE DATABASE snf_pithos WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
140 d587329c Konstantinos Tompoulidis
    postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_pithos TO synnefo;
141 a96ec00f Constantinos Venetsanopoulos
142 a96ec00f Constantinos Venetsanopoulos
Configure the database to listen to all network interfaces. You can do this by
143 36fea6f9 Dionysis Grigoropoulos
editing the file ``/etc/postgresql/9.1/main/postgresql.conf`` and change
144 a96ec00f Constantinos Venetsanopoulos
``listen_addresses`` to ``'*'`` :
145 a96ec00f Constantinos Venetsanopoulos
146 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
147 a96ec00f Constantinos Venetsanopoulos
148 d587329c Konstantinos Tompoulidis
    listen_addresses = '*'
149 a96ec00f Constantinos Venetsanopoulos
150 f8cdf6ec Dionysis Grigoropoulos
Furthermore, edit ``/etc/postgresql/9.1/main/pg_hba.conf`` to allow node1 and
151 a96ec00f Constantinos Venetsanopoulos
node2 to connect to the database. Add the following lines under ``#IPv4 local
152 a96ec00f Constantinos Venetsanopoulos
connections:`` :
153 a96ec00f Constantinos Venetsanopoulos
154 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
155 a96ec00f Constantinos Venetsanopoulos
156 f8cdf6ec Dionysis Grigoropoulos
    host		all	all	203.0.113.1/32	md5
157 f8cdf6ec Dionysis Grigoropoulos
    host		all	all	203.0.113.2/32	md5
158 a96ec00f Constantinos Venetsanopoulos
159 f8cdf6ec Dionysis Grigoropoulos
Make sure to substitute "203.0.113.1" and "203.0.113.2" with node1's and node2's
160 a96ec00f Constantinos Venetsanopoulos
actual IPs. Now, restart the server to apply the changes:
161 a96ec00f Constantinos Venetsanopoulos
162 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
163 a96ec00f Constantinos Venetsanopoulos
164 a96ec00f Constantinos Venetsanopoulos
   # /etc/init.d/postgresql restart
165 a96ec00f Constantinos Venetsanopoulos
166 a96ec00f Constantinos Venetsanopoulos
167 0c068fc6 marioskogias
Certificate Creation
168 0c068fc6 marioskogias
~~~~~~~~~~~~~~~~~~~~~
169 0c068fc6 marioskogias
170 dd6062f2 Christos Stavrakakis
Node1 will host Cyclades. Cyclades should communicate with the other Synnefo
171 dd6062f2 Christos Stavrakakis
Services and users over a secure channel. In order for the connection to be
172 f8cdf6ec Dionysis Grigoropoulos
trusted, the keys provided to Apache below should be signed with a certificate.
173 0c068fc6 marioskogias
This certificate should be added to all nodes. In case you don't have signed keys you can create a self-signed certificate
174 f8cdf6ec Dionysis Grigoropoulos
and sign your keys with this. To do so on node1 run:
175 0c068fc6 marioskogias
176 0c068fc6 marioskogias
.. code-block:: console
177 0c068fc6 marioskogias
178 f8cdf6ec Dionysis Grigoropoulos
		# apt-get install openvpn
179 0c068fc6 marioskogias
		# mkdir /etc/openvpn/easy-rsa
180 0c068fc6 marioskogias
		# cp -ai /usr/share/doc/openvpn/examples/easy-rsa/2.0/ /etc/openvpn/easy-rsa
181 0c068fc6 marioskogias
		# cd /etc/openvpn/easy-rsa/2.0
182 0c068fc6 marioskogias
		# vim vars
183 0c068fc6 marioskogias
184 0c068fc6 marioskogias
In vars you can set your own parameters such as KEY_COUNTRY
185 0c068fc6 marioskogias
186 0c068fc6 marioskogias
.. code-block:: console
187 0c068fc6 marioskogias
188 0c068fc6 marioskogias
	# . ./vars
189 0c068fc6 marioskogias
	# ./clean-all
190 0c068fc6 marioskogias
191 0c068fc6 marioskogias
Now you can create the certificate
192 0c068fc6 marioskogias
193 dd45ee44 Dionysis Grigoropoulos
.. code-block:: console
194 dd45ee44 Dionysis Grigoropoulos
195 0c068fc6 marioskogias
		# ./build-ca
196 0c068fc6 marioskogias
197 f8cdf6ec Dionysis Grigoropoulos
The previous will create a ``ca.crt`` file in the directory ``/etc/openvpn/easy-rsa/2.0/keys``.
198 f8cdf6ec Dionysis Grigoropoulos
Copy this file under ``/usr/local/share/ca-certificates/`` directory and run :
199 0c068fc6 marioskogias
200 0c068fc6 marioskogias
.. code-block:: console
201 0c068fc6 marioskogias
202 0c068fc6 marioskogias
		# update-ca-certificates
203 0c068fc6 marioskogias
204 0c068fc6 marioskogias
to update the records. You will have to do the following on node2 as well.
205 0c068fc6 marioskogias
206 0c068fc6 marioskogias
Now you can create the keys and sign them with the certificate
207 0c068fc6 marioskogias
208 0c068fc6 marioskogias
.. code-block:: console
209 dd45ee44 Dionysis Grigoropoulos
210 0c068fc6 marioskogias
		# ./build-key-server node1.example.com
211 0c068fc6 marioskogias
212 dd6062f2 Christos Stavrakakis
This will create a ``01.pem`` and a ``node1.example.com.key`` files in the
213 f8cdf6ec Dionysis Grigoropoulos
``/etc/openvpn/easy-rsa/2.0/keys`` directory. Copy these in ``/etc/ssl/certs/``
214 cbb596d4 Dionysis Grigoropoulos
and ``/etc/ssl/private/`` respectively and use them in the apache2
215 f8cdf6ec Dionysis Grigoropoulos
configuration file below instead of the defaults.
216 0c068fc6 marioskogias
217 a96ec00f Constantinos Venetsanopoulos
Apache2 setup
218 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~
219 a96ec00f Constantinos Venetsanopoulos
220 d587329c Konstantinos Tompoulidis
Create the file ``/etc/apache2/sites-available/synnefo`` containing the
221 d587329c Konstantinos Tompoulidis
following:
222 a96ec00f Constantinos Venetsanopoulos
223 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
224 a96ec00f Constantinos Venetsanopoulos
225 d587329c Konstantinos Tompoulidis
    <VirtualHost *:80>
226 d587329c Konstantinos Tompoulidis
        ServerName node1.example.com
227 a96ec00f Constantinos Venetsanopoulos
228 d587329c Konstantinos Tompoulidis
        RewriteEngine On
229 d587329c Konstantinos Tompoulidis
        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
230 d587329c Konstantinos Tompoulidis
        RewriteRule ^(.*)$ - [F,L]
231 d587329c Konstantinos Tompoulidis
        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
232 d587329c Konstantinos Tompoulidis
    </VirtualHost>
233 a96ec00f Constantinos Venetsanopoulos
234 0c068fc6 marioskogias
235 d587329c Konstantinos Tompoulidis
Create the file ``/etc/apache2/sites-available/synnefo-ssl`` containing the
236 d587329c Konstantinos Tompoulidis
following:
237 a96ec00f Constantinos Venetsanopoulos
238 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
239 a96ec00f Constantinos Venetsanopoulos
240 d587329c Konstantinos Tompoulidis
    <IfModule mod_ssl.c>
241 d587329c Konstantinos Tompoulidis
    <VirtualHost _default_:443>
242 d587329c Konstantinos Tompoulidis
        ServerName node1.example.com
243 a96ec00f Constantinos Venetsanopoulos
244 d587329c Konstantinos Tompoulidis
        Alias /static "/usr/share/synnefo/static"
245 a96ec00f Constantinos Venetsanopoulos
246 d587329c Konstantinos Tompoulidis
        #  SetEnv no-gzip
247 d587329c Konstantinos Tompoulidis
        #  SetEnv dont-vary
248 a96ec00f Constantinos Venetsanopoulos
249 d587329c Konstantinos Tompoulidis
       AllowEncodedSlashes On
250 04427415 Constantinos Venetsanopoulos
251 d587329c Konstantinos Tompoulidis
       RequestHeader set X-Forwarded-Protocol "https"
252 a96ec00f Constantinos Venetsanopoulos
253 d587329c Konstantinos Tompoulidis
    <Proxy * >
254 d587329c Konstantinos Tompoulidis
        Order allow,deny
255 d587329c Konstantinos Tompoulidis
        Allow from all
256 d587329c Konstantinos Tompoulidis
    </Proxy>
257 a96ec00f Constantinos Venetsanopoulos
258 d587329c Konstantinos Tompoulidis
        SetEnv                proxy-sendchunked
259 d587329c Konstantinos Tompoulidis
        SSLProxyEngine        off
260 d587329c Konstantinos Tompoulidis
        ProxyErrorOverride    off
261 a96ec00f Constantinos Venetsanopoulos
262 d587329c Konstantinos Tompoulidis
        ProxyPass        /static !
263 d587329c Konstantinos Tompoulidis
        ProxyPass        / http://localhost:8080/ retry=0
264 d587329c Konstantinos Tompoulidis
        ProxyPassReverse / http://localhost:8080/
265 a96ec00f Constantinos Venetsanopoulos
266 d587329c Konstantinos Tompoulidis
        RewriteEngine On
267 d587329c Konstantinos Tompoulidis
        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
268 d587329c Konstantinos Tompoulidis
        RewriteRule ^(.*)$ - [F,L]
269 a96ec00f Constantinos Venetsanopoulos
270 d587329c Konstantinos Tompoulidis
        SSLEngine on
271 d587329c Konstantinos Tompoulidis
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
272 d587329c Konstantinos Tompoulidis
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
273 d587329c Konstantinos Tompoulidis
    </VirtualHost>
274 d587329c Konstantinos Tompoulidis
    </IfModule>
275 a96ec00f Constantinos Venetsanopoulos
276 a96ec00f Constantinos Venetsanopoulos
Now enable sites and modules by running:
277 a96ec00f Constantinos Venetsanopoulos
278 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
279 a96ec00f Constantinos Venetsanopoulos
280 a96ec00f Constantinos Venetsanopoulos
   # a2enmod ssl
281 a96ec00f Constantinos Venetsanopoulos
   # a2enmod rewrite
282 a96ec00f Constantinos Venetsanopoulos
   # a2dissite default
283 a96ec00f Constantinos Venetsanopoulos
   # a2ensite synnefo
284 a96ec00f Constantinos Venetsanopoulos
   # a2ensite synnefo-ssl
285 a96ec00f Constantinos Venetsanopoulos
   # a2enmod headers
286 a96ec00f Constantinos Venetsanopoulos
   # a2enmod proxy_http
287 a96ec00f Constantinos Venetsanopoulos
288 dd45ee44 Dionysis Grigoropoulos
.. note:: This isn't really needed, but it's a good security practice to disable
289 dd45ee44 Dionysis Grigoropoulos
    directory listing in apache::
290 dd45ee44 Dionysis Grigoropoulos
291 dd45ee44 Dionysis Grigoropoulos
        # a2dismod autoindex
292 dd45ee44 Dionysis Grigoropoulos
293 dd45ee44 Dionysis Grigoropoulos
294 4de94e15 Kostas Papadimitriou
.. warning:: Do NOT start/restart the server yet. If the server is running::
295 a96ec00f Constantinos Venetsanopoulos
296 4de94e15 Kostas Papadimitriou
       # /etc/init.d/apache2 stop
297 a96ec00f Constantinos Venetsanopoulos
298 0c068fc6 marioskogias
299 2c85833e Constantinos Venetsanopoulos
.. _rabbitmq-setup:
300 2c85833e Constantinos Venetsanopoulos
301 bdfd94c9 Constantinos Venetsanopoulos
Message Queue setup
302 bdfd94c9 Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~
303 bdfd94c9 Constantinos Venetsanopoulos
304 bdfd94c9 Constantinos Venetsanopoulos
The message queue will run on node1, so we need to create the appropriate
305 bdfd94c9 Constantinos Venetsanopoulos
rabbitmq user. The user is named ``synnefo`` and gets full privileges on all
306 bdfd94c9 Constantinos Venetsanopoulos
exchanges:
307 bdfd94c9 Constantinos Venetsanopoulos
308 bdfd94c9 Constantinos Venetsanopoulos
.. code-block:: console
309 bdfd94c9 Constantinos Venetsanopoulos
310 69aa7f21 Ilias Tsitsimpis
   # rabbitmqctl add_user synnefo "example_rabbitmq_passw0rd"
311 bdfd94c9 Constantinos Venetsanopoulos
   # rabbitmqctl set_permissions synnefo ".*" ".*" ".*"
312 bdfd94c9 Constantinos Venetsanopoulos
313 bdfd94c9 Constantinos Venetsanopoulos
We do not need to initialize the exchanges. This will be done automatically,
314 bdfd94c9 Constantinos Venetsanopoulos
during the Cyclades setup.
315 bdfd94c9 Constantinos Venetsanopoulos
316 e5d8df8c Constantinos Venetsanopoulos
Pithos data directory setup
317 e5d8df8c Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~
318 a96ec00f Constantinos Venetsanopoulos
319 dd6062f2 Christos Stavrakakis
As mentioned in the General Prerequisites section, there should be a directory
320 f8cdf6ec Dionysis Grigoropoulos
called ``/srv/pithos`` visible by both nodes. We create and setup the ``data``
321 a96ec00f Constantinos Venetsanopoulos
directory inside it:
322 a96ec00f Constantinos Venetsanopoulos
323 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
324 a96ec00f Constantinos Venetsanopoulos
325 f8cdf6ec Dionysis Grigoropoulos
   # mkdir /srv/pithos
326 a96ec00f Constantinos Venetsanopoulos
   # cd /srv/pithos
327 a96ec00f Constantinos Venetsanopoulos
   # mkdir data
328 a96ec00f Constantinos Venetsanopoulos
   # chown www-data:www-data data
329 a96ec00f Constantinos Venetsanopoulos
   # chmod g+ws data
330 a96ec00f Constantinos Venetsanopoulos
331 f8cdf6ec Dionysis Grigoropoulos
This directory must be shared via `NFS <https://en.wikipedia.org/wiki/Network_File_System>`_.
332 f8cdf6ec Dionysis Grigoropoulos
In order to do this, run:
333 f8cdf6ec Dionysis Grigoropoulos
334 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
335 f8cdf6ec Dionysis Grigoropoulos
336 f8cdf6ec Dionysis Grigoropoulos
   # apt-get install rpcbind nfs-kernel-server
337 f8cdf6ec Dionysis Grigoropoulos
338 f8cdf6ec Dionysis Grigoropoulos
Now edit ``/etc/exports`` and add the following line:
339 f8cdf6ec Dionysis Grigoropoulos
340 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
341 dd6062f2 Christos Stavrakakis
342 f8cdf6ec Dionysis Grigoropoulos
   /srv/pithos/ 203.0.113.2(rw,no_root_squash,sync,subtree_check)
343 f8cdf6ec Dionysis Grigoropoulos
344 f8cdf6ec Dionysis Grigoropoulos
Once done, run:
345 f8cdf6ec Dionysis Grigoropoulos
346 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
347 f8cdf6ec Dionysis Grigoropoulos
348 f8cdf6ec Dionysis Grigoropoulos
   # /etc/init.d/nfs-kernel-server restart
349 f8cdf6ec Dionysis Grigoropoulos
350 f8cdf6ec Dionysis Grigoropoulos
351 0c068fc6 marioskogias
DNS server setup
352 0c068fc6 marioskogias
~~~~~~~~~~~~~~~~
353 0c068fc6 marioskogias
354 f8cdf6ec Dionysis Grigoropoulos
If your machines are not under the same domain name you have to set up a dns server.
355 f8cdf6ec Dionysis Grigoropoulos
In order to set up a dns server using dnsmasq do the following:
356 0c068fc6 marioskogias
357 0c068fc6 marioskogias
.. code-block:: console
358 dd45ee44 Dionysis Grigoropoulos
359 f8cdf6ec Dionysis Grigoropoulos
   # apt-get install dnsmasq
360 0c068fc6 marioskogias
361 f8cdf6ec Dionysis Grigoropoulos
Then edit your ``/etc/hosts/`` file as follows:
362 0c068fc6 marioskogias
363 0c068fc6 marioskogias
.. code-block:: console
364 0c068fc6 marioskogias
365 f8cdf6ec Dionysis Grigoropoulos
		203.0.113.1     node1.example.com
366 f8cdf6ec Dionysis Grigoropoulos
		203.0.113.2     node2.example.com
367 0c068fc6 marioskogias
368 f8cdf6ec Dionysis Grigoropoulos
dnsmasq will serve any IPs/domains found in ``/etc/resolv.conf``.
369 0c068fc6 marioskogias
370 f8cdf6ec Dionysis Grigoropoulos
There is a `"bug" in libevent 2.0.5 <http://sourceforge.net/p/levent/bugs/193/>`_
371 f8cdf6ec Dionysis Grigoropoulos
, where if you have multiple nameservers in your ``/etc/resolv.conf``, libevent
372 f8cdf6ec Dionysis Grigoropoulos
will round-robin against them. To avoid this, you must use a single nameserver
373 dd6062f2 Christos Stavrakakis
for all your needs. Edit your ``/etc/resolv.conf`` to include your dns server:
374 0c068fc6 marioskogias
375 0c068fc6 marioskogias
.. code-block:: console
376 0c068fc6 marioskogias
377 f8cdf6ec Dionysis Grigoropoulos
   nameserver 203.0.113.1
378 f8cdf6ec Dionysis Grigoropoulos
379 f8cdf6ec Dionysis Grigoropoulos
Because of the aforementioned bug, you can't specify more than one DNS servers
380 dd6062f2 Christos Stavrakakis
in your ``/etc/resolv.conf``. In order for dnsmasq to serve domains not in
381 dd6062f2 Christos Stavrakakis
``/etc/hosts``, edit ``/etc/dnsmasq.conf`` and change the line starting with
382 f8cdf6ec Dionysis Grigoropoulos
``#resolv-file=`` to:
383 f8cdf6ec Dionysis Grigoropoulos
384 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
385 f8cdf6ec Dionysis Grigoropoulos
386 f8cdf6ec Dionysis Grigoropoulos
   resolv-file=/etc/external-dns
387 f8cdf6ec Dionysis Grigoropoulos
388 f8cdf6ec Dionysis Grigoropoulos
Now create the file ``/etc/external-dns`` and specify any extra DNS servers you
389 f8cdf6ec Dionysis Grigoropoulos
want dnsmasq to query for domains, e.g., 8.8.8.8:
390 f8cdf6ec Dionysis Grigoropoulos
391 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
392 f8cdf6ec Dionysis Grigoropoulos
393 f8cdf6ec Dionysis Grigoropoulos
   nameserver 8.8.8.8
394 f8cdf6ec Dionysis Grigoropoulos
395 dd6062f2 Christos Stavrakakis
In the ``/etc/dnsmasq.conf`` file, you can also specify the ``listen-address``
396 f8cdf6ec Dionysis Grigoropoulos
and the ``interface`` you would like dnsmasq to listen to.
397 f8cdf6ec Dionysis Grigoropoulos
398 f8cdf6ec Dionysis Grigoropoulos
Finally, restart dnsmasq:
399 f8cdf6ec Dionysis Grigoropoulos
400 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
401 f8cdf6ec Dionysis Grigoropoulos
402 f8cdf6ec Dionysis Grigoropoulos
   # /etc/init.d/dnsmasq restart
403 0c068fc6 marioskogias
404 a96ec00f Constantinos Venetsanopoulos
You are now ready with all general prerequisites concerning node1. Let's go to
405 a96ec00f Constantinos Venetsanopoulos
node2.
406 a96ec00f Constantinos Venetsanopoulos
407 a96ec00f Constantinos Venetsanopoulos
Node2
408 a96ec00f Constantinos Venetsanopoulos
-----
409 a96ec00f Constantinos Venetsanopoulos
410 a96ec00f Constantinos Venetsanopoulos
General Synnefo dependencies
411 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
412 a96ec00f Constantinos Venetsanopoulos
413 d587329c Konstantinos Tompoulidis
    * apache (http server)
414 d587329c Konstantinos Tompoulidis
    * gunicorn (WSGI http server)
415 d587329c Konstantinos Tompoulidis
    * postgresql (database)
416 d587329c Konstantinos Tompoulidis
    * ntp (NTP daemon)
417 d587329c Konstantinos Tompoulidis
    * gevent
418 0c068fc6 marioskogias
    * certificates
419 f8cdf6ec Dionysis Grigoropoulos
    * dnsmasq (DNS server)
420 a96ec00f Constantinos Venetsanopoulos
421 a96ec00f Constantinos Venetsanopoulos
You can install the above by running:
422 a96ec00f Constantinos Venetsanopoulos
423 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
424 a96ec00f Constantinos Venetsanopoulos
425 7df5a742 Christos Stavrakakis
   # apt-get install apache2 postgresql ntp
426 73ff1d54 Constantinos Venetsanopoulos
427 f8cdf6ec Dionysis Grigoropoulos
To install gunicorn and gevent, run:
428 73ff1d54 Constantinos Venetsanopoulos
429 73ff1d54 Constantinos Venetsanopoulos
.. code-block:: console
430 73ff1d54 Constantinos Venetsanopoulos
431 f8cdf6ec Dionysis Grigoropoulos
   # apt-get install gunicorn python-gevent
432 576e059b Constantinos Venetsanopoulos
433 73ff1d54 Constantinos Venetsanopoulos
Node2 will connect to the databases on node1, so you will also need the
434 73ff1d54 Constantinos Venetsanopoulos
python-psycopg2 package:
435 a96ec00f Constantinos Venetsanopoulos
436 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
437 a96ec00f Constantinos Venetsanopoulos
438 a96ec00f Constantinos Venetsanopoulos
   # apt-get install python-psycopg2
439 a96ec00f Constantinos Venetsanopoulos
440 a96ec00f Constantinos Venetsanopoulos
Database setup
441 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~~
442 a96ec00f Constantinos Venetsanopoulos
443 a96ec00f Constantinos Venetsanopoulos
All databases have been created and setup on node1, so we do not need to take
444 a96ec00f Constantinos Venetsanopoulos
any action here. From node2, we will just connect to them. When you get familiar
445 a96ec00f Constantinos Venetsanopoulos
with the software you may choose to run different databases on different nodes,
446 a96ec00f Constantinos Venetsanopoulos
for performance/scalability/redundancy reasons, but those kind of setups are out
447 a96ec00f Constantinos Venetsanopoulos
of the purpose of this guide.
448 a96ec00f Constantinos Venetsanopoulos
449 a96ec00f Constantinos Venetsanopoulos
Apache2 setup
450 a96ec00f Constantinos Venetsanopoulos
~~~~~~~~~~~~~
451 5b6feb88 Vangelis Koukis
452 d587329c Konstantinos Tompoulidis
Create the file ``/etc/apache2/sites-available/synnefo`` containing the
453 d587329c Konstantinos Tompoulidis
following:
454 5b6feb88 Vangelis Koukis
455 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
456 a96ec00f Constantinos Venetsanopoulos
457 d587329c Konstantinos Tompoulidis
    <VirtualHost *:80>
458 d587329c Konstantinos Tompoulidis
        ServerName node2.example.com
459 a96ec00f Constantinos Venetsanopoulos
460 d587329c Konstantinos Tompoulidis
        RewriteEngine On
461 d587329c Konstantinos Tompoulidis
        RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
462 d587329c Konstantinos Tompoulidis
        RewriteRule ^(.*)$ - [F,L]
463 d587329c Konstantinos Tompoulidis
        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
464 d587329c Konstantinos Tompoulidis
    </VirtualHost>
465 a96ec00f Constantinos Venetsanopoulos
466 a96ec00f Constantinos Venetsanopoulos
Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``
467 a96ec00f Constantinos Venetsanopoulos
containing the following:
468 a96ec00f Constantinos Venetsanopoulos
469 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
470 a96ec00f Constantinos Venetsanopoulos
471 d587329c Konstantinos Tompoulidis
    <IfModule mod_ssl.c>
472 d587329c Konstantinos Tompoulidis
    <VirtualHost _default_:443>
473 d587329c Konstantinos Tompoulidis
        ServerName node2.example.com
474 a96ec00f Constantinos Venetsanopoulos
475 d587329c Konstantinos Tompoulidis
        Alias /static "/usr/share/synnefo/static"
476 a96ec00f Constantinos Venetsanopoulos
477 d587329c Konstantinos Tompoulidis
        SetEnv no-gzip
478 d587329c Konstantinos Tompoulidis
        SetEnv dont-vary
479 d587329c Konstantinos Tompoulidis
        AllowEncodedSlashes On
480 a96ec00f Constantinos Venetsanopoulos
481 d587329c Konstantinos Tompoulidis
        RequestHeader set X-Forwarded-Protocol "https"
482 a96ec00f Constantinos Venetsanopoulos
483 d587329c Konstantinos Tompoulidis
        <Proxy * >
484 d587329c Konstantinos Tompoulidis
            Order allow,deny
485 d587329c Konstantinos Tompoulidis
            Allow from all
486 d587329c Konstantinos Tompoulidis
        </Proxy>
487 a96ec00f Constantinos Venetsanopoulos
488 d587329c Konstantinos Tompoulidis
        SetEnv                proxy-sendchunked
489 d587329c Konstantinos Tompoulidis
        SSLProxyEngine        off
490 d587329c Konstantinos Tompoulidis
        ProxyErrorOverride    off
491 a96ec00f Constantinos Venetsanopoulos
492 d587329c Konstantinos Tompoulidis
        ProxyPass        /static !
493 d587329c Konstantinos Tompoulidis
        ProxyPass        / http://localhost:8080/ retry=0
494 d587329c Konstantinos Tompoulidis
        ProxyPassReverse / http://localhost:8080/
495 a96ec00f Constantinos Venetsanopoulos
496 d587329c Konstantinos Tompoulidis
        SSLEngine on
497 d587329c Konstantinos Tompoulidis
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
498 d587329c Konstantinos Tompoulidis
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
499 d587329c Konstantinos Tompoulidis
    </VirtualHost>
500 d587329c Konstantinos Tompoulidis
    </IfModule>
501 a96ec00f Constantinos Venetsanopoulos
502 a96ec00f Constantinos Venetsanopoulos
As in node1, enable sites and modules by running:
503 a96ec00f Constantinos Venetsanopoulos
504 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
505 a96ec00f Constantinos Venetsanopoulos
506 a96ec00f Constantinos Venetsanopoulos
   # a2enmod ssl
507 a96ec00f Constantinos Venetsanopoulos
   # a2enmod rewrite
508 a96ec00f Constantinos Venetsanopoulos
   # a2dissite default
509 a96ec00f Constantinos Venetsanopoulos
   # a2ensite synnefo
510 a96ec00f Constantinos Venetsanopoulos
   # a2ensite synnefo-ssl
511 a96ec00f Constantinos Venetsanopoulos
   # a2enmod headers
512 a96ec00f Constantinos Venetsanopoulos
   # a2enmod proxy_http
513 a96ec00f Constantinos Venetsanopoulos
514 dd45ee44 Dionysis Grigoropoulos
.. note:: This isn't really needed, but it's a good security practice to disable
515 dd45ee44 Dionysis Grigoropoulos
    directory listing in apache::
516 dd45ee44 Dionysis Grigoropoulos
517 dd45ee44 Dionysis Grigoropoulos
        # a2dismod autoindex
518 dd45ee44 Dionysis Grigoropoulos
519 4de94e15 Kostas Papadimitriou
.. warning:: Do NOT start/restart the server yet. If the server is running::
520 a96ec00f Constantinos Venetsanopoulos
521 4de94e15 Kostas Papadimitriou
       # /etc/init.d/apache2 stop
522 a96ec00f Constantinos Venetsanopoulos
523 0c068fc6 marioskogias
524 0c068fc6 marioskogias
Acquire certificate
525 0c068fc6 marioskogias
~~~~~~~~~~~~~~~~~~~
526 0c068fc6 marioskogias
527 0c068fc6 marioskogias
Copy the certificate you created before on node1 (`ca.crt`) under the directory
528 f8cdf6ec Dionysis Grigoropoulos
``/usr/local/share/ca-certificate`` and run:
529 0c068fc6 marioskogias
530 0c068fc6 marioskogias
.. code-block:: console
531 0c068fc6 marioskogias
532 f8cdf6ec Dionysis Grigoropoulos
   # update-ca-certificates
533 0c068fc6 marioskogias
534 0c068fc6 marioskogias
to update the records.
535 0c068fc6 marioskogias
536 0c068fc6 marioskogias
537 0c068fc6 marioskogias
DNS Setup
538 0c068fc6 marioskogias
~~~~~~~~~
539 0c068fc6 marioskogias
540 0c068fc6 marioskogias
Add the following line in ``/etc/resolv.conf`` file
541 0c068fc6 marioskogias
542 0c068fc6 marioskogias
.. code-block:: console
543 dd45ee44 Dionysis Grigoropoulos
544 f8cdf6ec Dionysis Grigoropoulos
   nameserver 203.0.113.1
545 0c068fc6 marioskogias
546 f8cdf6ec Dionysis Grigoropoulos
to inform the node about the new DNS server.
547 f8cdf6ec Dionysis Grigoropoulos
548 dd6062f2 Christos Stavrakakis
As mentioned before, this should be the only ``nameserver`` entry in
549 f8cdf6ec Dionysis Grigoropoulos
``/etc/resolv.conf``.
550 0c068fc6 marioskogias
551 a96ec00f Constantinos Venetsanopoulos
We are now ready with all general prerequisites for node2. Now that we have
552 a96ec00f Constantinos Venetsanopoulos
finished with all general prerequisites for both nodes, we can start installing
553 a96ec00f Constantinos Venetsanopoulos
the services. First, let's install Astakos on node1.
554 a96ec00f Constantinos Venetsanopoulos
555 a96ec00f Constantinos Venetsanopoulos
Installation of Astakos on node1
556 a96ec00f Constantinos Venetsanopoulos
================================
557 a96ec00f Constantinos Venetsanopoulos
558 f8cdf6ec Dionysis Grigoropoulos
To install Astakos, grab the package from our repository (make sure  you made
559 dd6062f2 Christos Stavrakakis
the additions needed in your ``/etc/apt/sources.list`` file and updated, as
560 f8cdf6ec Dionysis Grigoropoulos
described previously), by running:
561 a96ec00f Constantinos Venetsanopoulos
562 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
563 a96ec00f Constantinos Venetsanopoulos
564 b446c082 Giorgos Korfiatis
   # apt-get install snf-astakos-app snf-pithos-backend
565 a96ec00f Constantinos Venetsanopoulos
566 04c1254b Constantinos Venetsanopoulos
.. _conf-astakos:
567 04c1254b Constantinos Venetsanopoulos
568 a96ec00f Constantinos Venetsanopoulos
Configuration of Astakos
569 a96ec00f Constantinos Venetsanopoulos
========================
570 a96ec00f Constantinos Venetsanopoulos
571 f8cdf6ec Dionysis Grigoropoulos
Gunicorn setup
572 f8cdf6ec Dionysis Grigoropoulos
--------------
573 f8cdf6ec Dionysis Grigoropoulos
574 f8cdf6ec Dionysis Grigoropoulos
Copy the file ``/etc/gunicorn.d/synnefo.example`` to
575 f8cdf6ec Dionysis Grigoropoulos
``/etc/gunicorn.d/synnefo``, to make it a valid gunicorn configuration file:
576 f8cdf6ec Dionysis Grigoropoulos
577 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
578 f8cdf6ec Dionysis Grigoropoulos
579 f8cdf6ec Dionysis Grigoropoulos
    # mv /etc/gunicorn.d/synnefo.example /etc/gunicorn.d/synnefo
580 f8cdf6ec Dionysis Grigoropoulos
581 f8cdf6ec Dionysis Grigoropoulos
582 f8cdf6ec Dionysis Grigoropoulos
.. warning:: Do NOT start the server yet, because it won't find the
583 f8cdf6ec Dionysis Grigoropoulos
    ``synnefo.settings`` module. Also, in case you are using ``/etc/hosts``
584 f8cdf6ec Dionysis Grigoropoulos
    instead of a DNS to get the hostnames, change ``--worker-class=gevent`` to
585 f8cdf6ec Dionysis Grigoropoulos
    ``--worker-class=sync``. We will start the server after successful
586 f8cdf6ec Dionysis Grigoropoulos
    installation of Astakos. If the server is running::
587 f8cdf6ec Dionysis Grigoropoulos
588 f8cdf6ec Dionysis Grigoropoulos
       # /etc/init.d/gunicorn stop
589 f8cdf6ec Dionysis Grigoropoulos
590 a96ec00f Constantinos Venetsanopoulos
Conf Files
591 a96ec00f Constantinos Venetsanopoulos
----------
592 a96ec00f Constantinos Venetsanopoulos
593 f8cdf6ec Dionysis Grigoropoulos
After Astakos is successfully installed, you will find the directory
594 a96ec00f Constantinos Venetsanopoulos
``/etc/synnefo`` and some configuration files inside it. The files contain
595 a96ec00f Constantinos Venetsanopoulos
commented configuration options, which are the default options. While installing
596 a96ec00f Constantinos Venetsanopoulos
new snf-* components, new configuration files will appear inside the directory.
597 a96ec00f Constantinos Venetsanopoulos
In this guide (and for all services), we will edit only the minimum necessary
598 a96ec00f Constantinos Venetsanopoulos
configuration options, to reflect our setup. Everything else will remain as is.
599 a96ec00f Constantinos Venetsanopoulos
600 f8cdf6ec Dionysis Grigoropoulos
After getting familiar with Synnefo, you will be able to customize the software
601 a96ec00f Constantinos Venetsanopoulos
as you wish and fits your needs. Many options are available, to empower the
602 a96ec00f Constantinos Venetsanopoulos
administrator with extensively customizable setups.
603 a96ec00f Constantinos Venetsanopoulos
604 f8cdf6ec Dionysis Grigoropoulos
For the snf-webproject component (installed as an Astakos dependency), we
605 a96ec00f Constantinos Venetsanopoulos
need the following:
606 a96ec00f Constantinos Venetsanopoulos
607 a96ec00f Constantinos Venetsanopoulos
Edit ``/etc/synnefo/10-snf-webproject-database.conf``. You will need to
608 a96ec00f Constantinos Venetsanopoulos
uncomment and edit the ``DATABASES`` block to reflect our database:
609 a96ec00f Constantinos Venetsanopoulos
610 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
611 a96ec00f Constantinos Venetsanopoulos
612 d587329c Konstantinos Tompoulidis
    DATABASES = {
613 d587329c Konstantinos Tompoulidis
     'default': {
614 d587329c Konstantinos Tompoulidis
         # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
615 49026a89 Ilias Tsitsimpis
         'ENGINE': 'django.db.backends.postgresql_psycopg2',
616 a96ec00f Constantinos Venetsanopoulos
         # ATTENTION: This *must* be the absolute path if using sqlite3.
617 a96ec00f Constantinos Venetsanopoulos
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
618 d587329c Konstantinos Tompoulidis
         'NAME': 'snf_apps',
619 d587329c Konstantinos Tompoulidis
         'USER': 'synnefo',                      # Not used with sqlite3.
620 d587329c Konstantinos Tompoulidis
         'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
621 d587329c Konstantinos Tompoulidis
         # Set to empty string for localhost. Not used with sqlite3.
622 f8cdf6ec Dionysis Grigoropoulos
         'HOST': '203.0.113.1',
623 d587329c Konstantinos Tompoulidis
         # Set to empty string for default. Not used with sqlite3.
624 d587329c Konstantinos Tompoulidis
         'PORT': '5432',
625 d587329c Konstantinos Tompoulidis
     }
626 a96ec00f Constantinos Venetsanopoulos
    }
627 a96ec00f Constantinos Venetsanopoulos
628 a96ec00f Constantinos Venetsanopoulos
Edit ``/etc/synnefo/10-snf-webproject-deploy.conf``. Uncomment and edit
629 d587329c Konstantinos Tompoulidis
``SECRET_KEY``. This is a Django specific setting which is used to provide a
630 a96ec00f Constantinos Venetsanopoulos
seed in secret-key hashing algorithms. Set this to a random string of your
631 a14f152f Giorgos Korfiatis
choice and keep it private:
632 a96ec00f Constantinos Venetsanopoulos
633 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
634 a96ec00f Constantinos Venetsanopoulos
635 d587329c Konstantinos Tompoulidis
    SECRET_KEY = 'sy6)mw6a7x%n)-example_secret_key#zzk4jo6f2=uqu!1o%)'
636 a96ec00f Constantinos Venetsanopoulos
637 f8cdf6ec Dionysis Grigoropoulos
For Astakos specific configuration, edit the following options in
638 73ff1d54 Constantinos Venetsanopoulos
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :
639 a96ec00f Constantinos Venetsanopoulos
640 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
641 a96ec00f Constantinos Venetsanopoulos
642 d587329c Konstantinos Tompoulidis
    ASTAKOS_COOKIE_DOMAIN = '.example.com'
643 a96ec00f Constantinos Venetsanopoulos
644 a14f152f Giorgos Korfiatis
    ASTAKOS_BASE_URL = 'https://node1.example.com/astakos'
645 a96ec00f Constantinos Venetsanopoulos
646 4e3e3d24 Constantinos Venetsanopoulos
The ``ASTAKOS_COOKIE_DOMAIN`` should be the base url of our domain (for all
647 f8cdf6ec Dionysis Grigoropoulos
services). ``ASTAKOS_BASE_URL`` is the Astakos top-level URL. Appending an
648 0d87ef78 Giorgos Korfiatis
extra path (``/astakos`` here) is recommended in order to distinguish
649 0d87ef78 Giorgos Korfiatis
components, if more than one are installed on the same machine.
650 5b6feb88 Vangelis Koukis
651 4e3e3d24 Constantinos Venetsanopoulos
.. note:: For the purpose of this guide, we don't enable recaptcha authentication.
652 4e3e3d24 Constantinos Venetsanopoulos
    If you would like to enable it, you have to edit the following options:
653 319b615d Ilias Tsitsimpis
654 319b615d Ilias Tsitsimpis
    .. code-block:: console
655 319b615d Ilias Tsitsimpis
656 319b615d Ilias Tsitsimpis
        ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
657 319b615d Ilias Tsitsimpis
        ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
658 319b615d Ilias Tsitsimpis
        ASTAKOS_RECAPTCHA_USE_SSL = True
659 319b615d Ilias Tsitsimpis
        ASTAKOS_RECAPTCHA_ENABLED = True
660 319b615d Ilias Tsitsimpis
661 319b615d Ilias Tsitsimpis
    For the ``ASTAKOS_RECAPTCHA_PUBLIC_KEY`` and ``ASTAKOS_RECAPTCHA_PRIVATE_KEY``
662 319b615d Ilias Tsitsimpis
    go to https://www.google.com/recaptcha/admin/create and create your own pair.
663 5b6feb88 Vangelis Koukis
664 610ff8cf Constantinos Venetsanopoulos
Then edit ``/etc/synnefo/20-snf-astakos-app-cloudbar.conf`` :
665 610ff8cf Constantinos Venetsanopoulos
666 610ff8cf Constantinos Venetsanopoulos
.. code-block:: console
667 610ff8cf Constantinos Venetsanopoulos
668 d587329c Konstantinos Tompoulidis
    CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
669 610ff8cf Constantinos Venetsanopoulos
670 a14f152f Giorgos Korfiatis
    CLOUDBAR_SERVICES_URL = 'https://node1.example.com/astakos/ui/get_services'
671 610ff8cf Constantinos Venetsanopoulos
672 a14f152f Giorgos Korfiatis
    CLOUDBAR_MENU_URL = 'https://node1.example.com/astakos/ui/get_menu'
673 610ff8cf Constantinos Venetsanopoulos
674 d587329c Konstantinos Tompoulidis
Those settings have to do with the black cloudbar endpoints and will be
675 d587329c Konstantinos Tompoulidis
described in more detail later on in this guide. For now, just edit the domain
676 d587329c Konstantinos Tompoulidis
to point at node1 which is where we have installed Astakos.
677 610ff8cf Constantinos Venetsanopoulos
678 d587329c Konstantinos Tompoulidis
If you are an advanced user and want to use the Shibboleth Authentication
679 d587329c Konstantinos Tompoulidis
method, read the relative :ref:`section <shibboleth-auth>`.
680 d2a9f85f Sofia Papagiannaki
681 a68c2667 Kostas Papadimitriou
.. _email-configuration:
682 657f8ad7 Kostas Papadimitriou
683 657f8ad7 Kostas Papadimitriou
Email delivery configuration
684 657f8ad7 Kostas Papadimitriou
----------------------------
685 657f8ad7 Kostas Papadimitriou
686 dd6062f2 Christos Stavrakakis
Many of the ``Astakos`` operations require the server to notify service users
687 dd6062f2 Christos Stavrakakis
and administrators via email. e.g. right after the signup process, the service
688 dd6062f2 Christos Stavrakakis
sents an email to the registered email address containing an verification url.
689 dd6062f2 Christos Stavrakakis
After the user verifies the email address, Astakos once again needs to
690 f8cdf6ec Dionysis Grigoropoulos
notify administrators with a notice that a new account has just been verified.
691 657f8ad7 Kostas Papadimitriou
692 f8cdf6ec Dionysis Grigoropoulos
More specifically Astakos sends emails in the following cases
693 657f8ad7 Kostas Papadimitriou
694 657f8ad7 Kostas Papadimitriou
- An email containing a verification link after each signup process.
695 dd45ee44 Dionysis Grigoropoulos
- An email to the people listed in ``ADMINS`` setting after each email
696 dd45ee44 Dionysis Grigoropoulos
  verification if ``ASTAKOS_MODERATION`` setting is ``True``. The email
697 dd45ee44 Dionysis Grigoropoulos
  notifies administrators that an additional action is required in order to
698 657f8ad7 Kostas Papadimitriou
  activate the user.
699 dd45ee44 Dionysis Grigoropoulos
- A welcome email to the user email and an admin notification to ``ADMINS``
700 657f8ad7 Kostas Papadimitriou
  right after each account activation.
701 f8cdf6ec Dionysis Grigoropoulos
- Feedback messages submited from Astakos contact view and Astakos feedback
702 657f8ad7 Kostas Papadimitriou
  API endpoint are sent to contacts listed in ``HELPDESK`` setting.
703 dd45ee44 Dionysis Grigoropoulos
- Project application request notifications to people included in ``HELPDESK``
704 657f8ad7 Kostas Papadimitriou
  and ``MANAGERS`` settings.
705 dd45ee44 Dionysis Grigoropoulos
- Notifications after each project members action (join request, membership
706 657f8ad7 Kostas Papadimitriou
  accepted/declinde etc.) to project members or project owners.
707 657f8ad7 Kostas Papadimitriou
708 dd45ee44 Dionysis Grigoropoulos
Astakos uses the Django internal email delivering mechanism to send email
709 dd45ee44 Dionysis Grigoropoulos
notifications. A simple configuration, using an external smtp server to
710 dd45ee44 Dionysis Grigoropoulos
deliver messages, is shown below. Alter the following example to meet your
711 0c068fc6 marioskogias
smtp server characteristics. Notice that the smtp server is needed for a proper
712 f8cdf6ec Dionysis Grigoropoulos
installation.
713 f8cdf6ec Dionysis Grigoropoulos
714 f8cdf6ec Dionysis Grigoropoulos
Edit ``/etc/synnefo/00-snf-common-admins.conf``:
715 657f8ad7 Kostas Papadimitriou
716 657f8ad7 Kostas Papadimitriou
.. code-block:: python
717 dd45ee44 Dionysis Grigoropoulos
718 f8cdf6ec Dionysis Grigoropoulos
    EMAIL_HOST = "mysmtp.server.example.com"
719 657f8ad7 Kostas Papadimitriou
    EMAIL_HOST_USER = "<smtpuser>"
720 657f8ad7 Kostas Papadimitriou
    EMAIL_HOST_PASSWORD = "<smtppassword>"
721 657f8ad7 Kostas Papadimitriou
722 657f8ad7 Kostas Papadimitriou
    # this gets appended in all email subjects
723 f8cdf6ec Dionysis Grigoropoulos
    EMAIL_SUBJECT_PREFIX = "[example.com] "
724 dd45ee44 Dionysis Grigoropoulos
725 657f8ad7 Kostas Papadimitriou
    # Address to use for outgoing emails
726 f8cdf6ec Dionysis Grigoropoulos
    DEFAULT_FROM_EMAIL = "server@example.com"
727 657f8ad7 Kostas Papadimitriou
728 dd45ee44 Dionysis Grigoropoulos
    # Email where users can contact for support. This is used in html/email
729 657f8ad7 Kostas Papadimitriou
    # templates.
730 f8cdf6ec Dionysis Grigoropoulos
    CONTACT_EMAIL = "server@example.com"
731 657f8ad7 Kostas Papadimitriou
732 657f8ad7 Kostas Papadimitriou
    # The email address that error messages come from
733 f8cdf6ec Dionysis Grigoropoulos
    SERVER_EMAIL = "server-errors@example.com"
734 657f8ad7 Kostas Papadimitriou
735 657f8ad7 Kostas Papadimitriou
Notice that since email settings might be required by applications other than
736 f8cdf6ec Dionysis Grigoropoulos
Astakos, they are defined in a different configuration file than the one
737 f8cdf6ec Dionysis Grigoropoulos
previously used to set Astakos specific settings.
738 657f8ad7 Kostas Papadimitriou
739 dd45ee44 Dionysis Grigoropoulos
Refer to
740 6b256427 Christos Stavrakakis
`Django documentation <https://docs.djangoproject.com/en/1.4/topics/email/>`_
741 657f8ad7 Kostas Papadimitriou
for additional information on available email settings.
742 657f8ad7 Kostas Papadimitriou
743 dd45ee44 Dionysis Grigoropoulos
As refered in the previous section, based on the operation that triggers
744 f8cdf6ec Dionysis Grigoropoulos
an email notification, the recipients list differs. Specifically, for
745 dd45ee44 Dionysis Grigoropoulos
emails whose recipients include contacts from your service team
746 dd45ee44 Dionysis Grigoropoulos
(administrators, managers, helpdesk etc) synnefo provides the following
747 52188a27 Kostis Fardelas
settings located in ``00-snf-common-admins.conf``:
748 657f8ad7 Kostas Papadimitriou
749 657f8ad7 Kostas Papadimitriou
.. code-block:: python
750 657f8ad7 Kostas Papadimitriou
751 f8cdf6ec Dionysis Grigoropoulos
    ADMINS = (('Admin name', 'admin@example.com'),
752 f8cdf6ec Dionysis Grigoropoulos
              ('Admin2 name', 'admin2@example.com))
753 f8cdf6ec Dionysis Grigoropoulos
    MANAGERS = (('Manager name', 'manager@example.com'),)
754 f8cdf6ec Dionysis Grigoropoulos
    HELPDESK = (('Helpdesk user name', 'helpdesk@example.com'),)
755 657f8ad7 Kostas Papadimitriou
756 6dd3e7c2 Mpampis Stylianopoulos
Alternatively, it may be convenient to send e-mails to a file, instead of an actual smtp server, using the file backend. Do so by creating a configuration file ``/etc/synnefo/99-local.conf`` including the folowing:
757 6dd3e7c2 Mpampis Stylianopoulos
758 6dd3e7c2 Mpampis Stylianopoulos
.. code-block:: python
759 6dd3e7c2 Mpampis Stylianopoulos
760 6dd3e7c2 Mpampis Stylianopoulos
    EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend'
761 cbb596d4 Dionysis Grigoropoulos
    EMAIL_FILE_PATH = '/tmp/app-messages'
762 657f8ad7 Kostas Papadimitriou
763 657f8ad7 Kostas Papadimitriou
764 4e3e3d24 Constantinos Venetsanopoulos
Enable Pooling
765 4e3e3d24 Constantinos Venetsanopoulos
--------------
766 4e3e3d24 Constantinos Venetsanopoulos
767 4e3e3d24 Constantinos Venetsanopoulos
This section can be bypassed, but we strongly recommend you apply the following,
768 4e3e3d24 Constantinos Venetsanopoulos
since they result in a significant performance boost.
769 4e3e3d24 Constantinos Venetsanopoulos
770 4e3e3d24 Constantinos Venetsanopoulos
Synnefo includes a pooling DBAPI driver for PostgreSQL, as a thin wrapper
771 4e3e3d24 Constantinos Venetsanopoulos
around Psycopg2. This allows independent Django requests to reuse pooled DB
772 4e3e3d24 Constantinos Venetsanopoulos
connections, with significant performance gains.
773 4e3e3d24 Constantinos Venetsanopoulos
774 4e3e3d24 Constantinos Venetsanopoulos
To use, first monkey-patch psycopg2. For Django, run this before the
775 4e3e3d24 Constantinos Venetsanopoulos
``DATABASES`` setting in ``/etc/synnefo/10-snf-webproject-database.conf``:
776 4e3e3d24 Constantinos Venetsanopoulos
777 4e3e3d24 Constantinos Venetsanopoulos
.. code-block:: console
778 4e3e3d24 Constantinos Venetsanopoulos
779 d587329c Konstantinos Tompoulidis
    from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2
780 d587329c Konstantinos Tompoulidis
    monkey_patch_psycopg2()
781 4e3e3d24 Constantinos Venetsanopoulos
782 576e059b Constantinos Venetsanopoulos
Since we are running with greenlets, we should modify psycopg2 behavior, so it
783 576e059b Constantinos Venetsanopoulos
works properly in a greenlet context:
784 4e3e3d24 Constantinos Venetsanopoulos
785 4e3e3d24 Constantinos Venetsanopoulos
.. code-block:: console
786 4e3e3d24 Constantinos Venetsanopoulos
787 d587329c Konstantinos Tompoulidis
    from synnefo.lib.db.psyco_gevent import make_psycopg_green
788 d587329c Konstantinos Tompoulidis
    make_psycopg_green()
789 4e3e3d24 Constantinos Venetsanopoulos
790 4e3e3d24 Constantinos Venetsanopoulos
Use the Psycopg2 driver as usual. For Django, this means using
791 4e3e3d24 Constantinos Venetsanopoulos
``django.db.backends.postgresql_psycopg2`` without any modifications. To enable
792 4e3e3d24 Constantinos Venetsanopoulos
connection pooling, pass a nonzero ``synnefo_poolsize`` option to the DBAPI
793 d587329c Konstantinos Tompoulidis
driver, through ``DATABASES.OPTIONS`` in Django.
794 4e3e3d24 Constantinos Venetsanopoulos
795 4e3e3d24 Constantinos Venetsanopoulos
All the above will result in an ``/etc/synnefo/10-snf-webproject-database.conf``
796 4e3e3d24 Constantinos Venetsanopoulos
file that looks like this:
797 4e3e3d24 Constantinos Venetsanopoulos
798 4e3e3d24 Constantinos Venetsanopoulos
.. code-block:: console
799 4e3e3d24 Constantinos Venetsanopoulos
800 d587329c Konstantinos Tompoulidis
    # Monkey-patch psycopg2
801 d587329c Konstantinos Tompoulidis
    from synnefo.lib.db.pooled_psycopg2 import monkey_patch_psycopg2
802 d587329c Konstantinos Tompoulidis
    monkey_patch_psycopg2()
803 4e3e3d24 Constantinos Venetsanopoulos
804 d587329c Konstantinos Tompoulidis
    # If running with greenlets
805 d587329c Konstantinos Tompoulidis
    from synnefo.lib.db.psyco_gevent import make_psycopg_green
806 d587329c Konstantinos Tompoulidis
    make_psycopg_green()
807 4e3e3d24 Constantinos Venetsanopoulos
808 d587329c Konstantinos Tompoulidis
    DATABASES = {
809 d587329c Konstantinos Tompoulidis
     'default': {
810 d587329c Konstantinos Tompoulidis
         # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
811 49026a89 Ilias Tsitsimpis
         'ENGINE': 'django.db.backends.postgresql_psycopg2',
812 d587329c Konstantinos Tompoulidis
         'OPTIONS': {'synnefo_poolsize': 8},
813 4e3e3d24 Constantinos Venetsanopoulos
814 4e3e3d24 Constantinos Venetsanopoulos
         # ATTENTION: This *must* be the absolute path if using sqlite3.
815 4e3e3d24 Constantinos Venetsanopoulos
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
816 d587329c Konstantinos Tompoulidis
         'NAME': 'snf_apps',
817 d587329c Konstantinos Tompoulidis
         'USER': 'synnefo',                      # Not used with sqlite3.
818 d587329c Konstantinos Tompoulidis
         'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
819 d587329c Konstantinos Tompoulidis
         # Set to empty string for localhost. Not used with sqlite3.
820 f8cdf6ec Dionysis Grigoropoulos
         'HOST': '203.0.113.1',
821 d587329c Konstantinos Tompoulidis
         # Set to empty string for default. Not used with sqlite3.
822 d587329c Konstantinos Tompoulidis
         'PORT': '5432',
823 d587329c Konstantinos Tompoulidis
     }
824 4e3e3d24 Constantinos Venetsanopoulos
    }
825 5a2d4e43 Christos Stavrakakis
826 a96ec00f Constantinos Venetsanopoulos
Database Initialization
827 a96ec00f Constantinos Venetsanopoulos
-----------------------
828 a96ec00f Constantinos Venetsanopoulos
829 de20a465 Constantinos Venetsanopoulos
After configuration is done, we initialize the database by running:
830 5b6feb88 Vangelis Koukis
831 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
832 a96ec00f Constantinos Venetsanopoulos
833 d587329c Konstantinos Tompoulidis
    # snf-manage syncdb
834 a96ec00f Constantinos Venetsanopoulos
835 a96ec00f Constantinos Venetsanopoulos
At this example we don't need to create a django superuser, so we select
836 a96ec00f Constantinos Venetsanopoulos
``[no]`` to the question. After a successful sync, we run the migration needed
837 f8cdf6ec Dionysis Grigoropoulos
for Astakos:
838 5b6feb88 Vangelis Koukis
839 5b6feb88 Vangelis Koukis
.. code-block:: console
840 5b6feb88 Vangelis Koukis
841 d587329c Konstantinos Tompoulidis
    # snf-manage migrate im
842 b446c082 Giorgos Korfiatis
    # snf-manage migrate quotaholder_app
843 c126bd02 Giorgos Korfiatis
    # snf-manage migrate oa2
844 a96ec00f Constantinos Venetsanopoulos
845 de20a465 Constantinos Venetsanopoulos
Then, we load the pre-defined user groups
846 d2a9f85f Sofia Papagiannaki
847 d2a9f85f Sofia Papagiannaki
.. code-block:: console
848 d2a9f85f Sofia Papagiannaki
849 d587329c Konstantinos Tompoulidis
    # snf-manage loaddata groups
850 d2a9f85f Sofia Papagiannaki
851 de20a465 Constantinos Venetsanopoulos
.. _services-reg:
852 de20a465 Constantinos Venetsanopoulos
853 de20a465 Constantinos Venetsanopoulos
Services Registration
854 de20a465 Constantinos Venetsanopoulos
---------------------
855 de20a465 Constantinos Venetsanopoulos
856 b446c082 Giorgos Korfiatis
When the database is ready, we need to register the services. The following
857 f8cdf6ec Dionysis Grigoropoulos
command will ask you to register the standard Synnefo components (Astakos,
858 f8cdf6ec Dionysis Grigoropoulos
Cyclades and Pithos) along with the services they provide. Note that you
859 f8cdf6ec Dionysis Grigoropoulos
have to register at least Astakos in order to have a usable authentication
860 0d87ef78 Giorgos Korfiatis
system. For each component, you will be asked to provide two URLs: its base
861 0d87ef78 Giorgos Korfiatis
URL and its UI URL.
862 0d87ef78 Giorgos Korfiatis
863 0d87ef78 Giorgos Korfiatis
The former is the location where the component resides; it should equal
864 0d87ef78 Giorgos Korfiatis
the ``<component_name>_BASE_URL`` as specified in the respective component
865 f8cdf6ec Dionysis Grigoropoulos
settings. For example, the base URL for Astakos would be
866 0d87ef78 Giorgos Korfiatis
``https://node1.example.com/astakos``.
867 0d87ef78 Giorgos Korfiatis
868 0d87ef78 Giorgos Korfiatis
The latter is the URL that appears in the Cloudbar and leads to the
869 0d87ef78 Giorgos Korfiatis
component UI. If you want to follow the default setup, set
870 0d87ef78 Giorgos Korfiatis
the UI URL to ``<base_url>/ui/`` where ``base_url`` the component's base
871 0d87ef78 Giorgos Korfiatis
URL as explained before. (You can later change the UI URL with
872 c126bd02 Giorgos Korfiatis
``snf-manage component-modify <component_name> --ui-url new_ui_url``.)
873 0d87ef78 Giorgos Korfiatis
874 0d87ef78 Giorgos Korfiatis
The command will also register automatically the resource definitions
875 3f31a848 Giorgos Korfiatis
offered by the services.
876 de20a465 Constantinos Venetsanopoulos
877 de20a465 Constantinos Venetsanopoulos
.. code-block:: console
878 de20a465 Constantinos Venetsanopoulos
879 b4109758 Giorgos Korfiatis
    # snf-component-register
880 3f31a848 Giorgos Korfiatis
881 3f31a848 Giorgos Korfiatis
.. note::
882 3f31a848 Giorgos Korfiatis
883 3f31a848 Giorgos Korfiatis
   This command is equivalent to running the following series of commands;
884 f8cdf6ec Dionysis Grigoropoulos
   it registers the three components in Astakos and then in each host it
885 3f31a848 Giorgos Korfiatis
   exports the respective service definitions, copies the exported json file
886 f8cdf6ec Dionysis Grigoropoulos
   to the Astakos host, where it finally imports it:
887 3f31a848 Giorgos Korfiatis
888 3f31a848 Giorgos Korfiatis
    .. code-block:: console
889 3f31a848 Giorgos Korfiatis
890 eb765213 Giorgos Korfiatis
       astakos-host$ snf-manage component-add astakos --base-url astakos_base_url --ui-url astakos_ui_url
891 eb765213 Giorgos Korfiatis
       astakos-host$ snf-manage component-add cyclades --base-url cyclades_base_url --ui-url cyclades_ui_url
892 eb765213 Giorgos Korfiatis
       astakos-host$ snf-manage component-add pithos --base-url pithos_base_url --ui-url pithos_ui_url
893 3f31a848 Giorgos Korfiatis
       astakos-host$ snf-manage service-export-astakos > astakos.json
894 3f31a848 Giorgos Korfiatis
       astakos-host$ snf-manage service-import --json astakos.json
895 3f31a848 Giorgos Korfiatis
       cyclades-host$ snf-manage service-export-cyclades > cyclades.json
896 3f31a848 Giorgos Korfiatis
       # copy the file to astakos-host
897 3f31a848 Giorgos Korfiatis
       astakos-host$ snf-manage service-import --json cyclades.json
898 3f31a848 Giorgos Korfiatis
       pithos-host$ snf-manage service-export-pithos > pithos.json
899 3f31a848 Giorgos Korfiatis
       # copy the file to astakos-host
900 3f31a848 Giorgos Korfiatis
       astakos-host$ snf-manage service-import --json pithos.json
901 b446c082 Giorgos Korfiatis
902 f8cdf6ec Dionysis Grigoropoulos
Notice that in this installation astakos and cyclades are in node1 and pithos is in node2.
903 0c068fc6 marioskogias
904 b446c082 Giorgos Korfiatis
Setting Default Base Quota for Resources
905 b446c082 Giorgos Korfiatis
----------------------------------------
906 b446c082 Giorgos Korfiatis
907 b446c082 Giorgos Korfiatis
We now have to specify the limit on resources that each user can employ
908 52188a27 Kostis Fardelas
(exempting resources offered by projects). When specifying storage or
909 c126bd02 Giorgos Korfiatis
memory size limits you can append a unit to the value, i.e. 10240 MB,
910 c126bd02 Giorgos Korfiatis
10 GB etc. Use the special value ``inf``, if you don't want to restrict a
911 c126bd02 Giorgos Korfiatis
resource.
912 b446c082 Giorgos Korfiatis
913 b446c082 Giorgos Korfiatis
.. code-block:: console
914 b446c082 Giorgos Korfiatis
915 d61d6007 Giorgos Korfiatis
    # snf-manage resource-modify --default-quota-interactive
916 b446c082 Giorgos Korfiatis
917 c126bd02 Giorgos Korfiatis
Setting Resource Visibility
918 c126bd02 Giorgos Korfiatis
---------------------------
919 c126bd02 Giorgos Korfiatis
920 c126bd02 Giorgos Korfiatis
It is possible to control whether a resource is visible to the users via the
921 c126bd02 Giorgos Korfiatis
API or the Web UI. The default value for these options is denoted inside the
922 c126bd02 Giorgos Korfiatis
default resource definitions. Note that the system always checks and
923 c126bd02 Giorgos Korfiatis
enforces resource quota, regardless of their visibility. You can inspect the
924 c126bd02 Giorgos Korfiatis
current status with::
925 c126bd02 Giorgos Korfiatis
926 c126bd02 Giorgos Korfiatis
   # snf-manage resource-list
927 c126bd02 Giorgos Korfiatis
928 c126bd02 Giorgos Korfiatis
You can change a resource's visibility with::
929 c126bd02 Giorgos Korfiatis
930 c126bd02 Giorgos Korfiatis
   # snf-manage resource-modify <resource> --api-visible=True (or --ui-visible=True)
931 c126bd02 Giorgos Korfiatis
932 5547485e Sofia Papagiannaki
.. _pithos_view_registration:
933 5547485e Sofia Papagiannaki
934 5547485e Sofia Papagiannaki
Register pithos view as an OAuth 2.0 client
935 5547485e Sofia Papagiannaki
-------------------------------------------
936 5547485e Sofia Papagiannaki
937 5547485e Sofia Papagiannaki
Starting from synnefo version 0.15, the pithos view, in order to get access to
938 c126bd02 Giorgos Korfiatis
the data of a protected pithos resource, has to be granted authorization for
939 c126bd02 Giorgos Korfiatis
the specific resource by astakos.
940 5547485e Sofia Papagiannaki
941 5547485e Sofia Papagiannaki
During the authorization grant procedure, it has to authenticate itself with
942 c126bd02 Giorgos Korfiatis
astakos since the latter has to prevent serving requests by
943 c126bd02 Giorgos Korfiatis
unknown/unauthorized clients.
944 5547485e Sofia Papagiannaki
945 8a41dc10 Sofia Papagiannaki
Each oauth 2.0 client is identified by a client identifier (client_id).
946 8a41dc10 Sofia Papagiannaki
Moreover, the confidential clients are authenticated via a password
947 8a41dc10 Sofia Papagiannaki
(client_secret).
948 8a41dc10 Sofia Papagiannaki
Then, each client has to declare at least a redirect URI so that astakos will
949 8a41dc10 Sofia Papagiannaki
be able to validate the redirect URI provided during the authorization code
950 8a41dc10 Sofia Papagiannaki
request.
951 c126bd02 Giorgos Korfiatis
If a client is trusted (like a pithos view), astakos grants access on behalf
952 8a41dc10 Sofia Papagiannaki
of the resource owner, otherwise the resource owner has to be asked.
953 8a41dc10 Sofia Papagiannaki
954 5547485e Sofia Papagiannaki
To register the pithos view as an OAuth 2.0 client in astakos, we have to run
955 5547485e Sofia Papagiannaki
the following command::
956 5547485e Sofia Papagiannaki
957 5547485e Sofia Papagiannaki
    snf-manage oauth2-client-add pithos-view --secret=<secret> --is-trusted --url https://node2.example.com/pithos/ui/view
958 de20a465 Constantinos Venetsanopoulos
959 de20a465 Constantinos Venetsanopoulos
Servers Initialization
960 de20a465 Constantinos Venetsanopoulos
----------------------
961 de20a465 Constantinos Venetsanopoulos
962 de20a465 Constantinos Venetsanopoulos
Finally, we initialize the servers on node1:
963 de20a465 Constantinos Venetsanopoulos
964 de20a465 Constantinos Venetsanopoulos
.. code-block:: console
965 de20a465 Constantinos Venetsanopoulos
966 d587329c Konstantinos Tompoulidis
    root@node1:~ # /etc/init.d/gunicorn restart
967 d587329c Konstantinos Tompoulidis
    root@node1:~ # /etc/init.d/apache2 restart
968 de20a465 Constantinos Venetsanopoulos
969 de20a465 Constantinos Venetsanopoulos
We have now finished the Astakos setup. Let's test it now.
970 a96ec00f Constantinos Venetsanopoulos
971 a96ec00f Constantinos Venetsanopoulos
972 a96ec00f Constantinos Venetsanopoulos
Testing of Astakos
973 a96ec00f Constantinos Venetsanopoulos
==================
974 a96ec00f Constantinos Venetsanopoulos
975 a96ec00f Constantinos Venetsanopoulos
Open your favorite browser and go to:
976 a96ec00f Constantinos Venetsanopoulos
977 a14f152f Giorgos Korfiatis
``http://node1.example.com/astakos``
978 a96ec00f Constantinos Venetsanopoulos
979 a14f152f Giorgos Korfiatis
If this redirects you to ``https://node1.example.com/astakos/ui/`` and you can see
980 a96ec00f Constantinos Venetsanopoulos
the "welcome" door of Astakos, then you have successfully setup Astakos.
981 a96ec00f Constantinos Venetsanopoulos
982 a96ec00f Constantinos Venetsanopoulos
Let's create our first user. At the homepage click the "CREATE ACCOUNT" button
983 a96ec00f Constantinos Venetsanopoulos
and fill all your data at the sign up form. Then click "SUBMIT". You should now
984 a96ec00f Constantinos Venetsanopoulos
see a green box on the top, which informs you that you made a successful request
985 d587329c Konstantinos Tompoulidis
and the request has been sent to the administrators. So far so good, let's
986 d587329c Konstantinos Tompoulidis
assume that you created the user with username ``user@example.com``.
987 a96ec00f Constantinos Venetsanopoulos
988 a96ec00f Constantinos Venetsanopoulos
Now we need to activate that user. Return to a command prompt at node1 and run:
989 a96ec00f Constantinos Venetsanopoulos
990 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
991 a96ec00f Constantinos Venetsanopoulos
992 d587329c Konstantinos Tompoulidis
    root@node1:~ # snf-manage user-list
993 a96ec00f Constantinos Venetsanopoulos
994 a96ec00f Constantinos Venetsanopoulos
This command should show you a list with only one user; the one we just created.
995 a14f152f Giorgos Korfiatis
This user should have an id with a value of ``1`` and flag "active" and
996 a14f152f Giorgos Korfiatis
"verified" set to False. Now run:
997 a96ec00f Constantinos Venetsanopoulos
998 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
999 a96ec00f Constantinos Venetsanopoulos
1000 a14f152f Giorgos Korfiatis
    root@node1:~ # snf-manage user-modify 1 --verify --accept
1001 a96ec00f Constantinos Venetsanopoulos
1002 a14f152f Giorgos Korfiatis
This verifies the user email and activates the user.
1003 a96ec00f Constantinos Venetsanopoulos
When running in production, the activation is done automatically with different
1004 a96ec00f Constantinos Venetsanopoulos
types of moderation, that Astakos supports. You can see the moderation methods
1005 a96ec00f Constantinos Venetsanopoulos
(by invitation, whitelists, matching regexp, etc.) at the Astakos specific
1006 f846d8df Constantinos Venetsanopoulos
documentation. In production, you can also manually activate a user, by sending
1007 f846d8df Constantinos Venetsanopoulos
him/her an activation email. See how to do this at the :ref:`User
1008 f846d8df Constantinos Venetsanopoulos
activation <user_activation>` section.
1009 a96ec00f Constantinos Venetsanopoulos
1010 c126bd02 Giorgos Korfiatis
Now let's go back to the homepage. Open ``http://node1.example.com/astakos/ui/`` with
1011 f8cdf6ec Dionysis Grigoropoulos
your browser again. Try to sign in using your new credentials. If the Astakos
1012 a96ec00f Constantinos Venetsanopoulos
menu appears and you can see your profile, then you have successfully setup
1013 a96ec00f Constantinos Venetsanopoulos
Astakos.
1014 a96ec00f Constantinos Venetsanopoulos
1015 e5d8df8c Constantinos Venetsanopoulos
Let's continue to install Pithos now.
1016 a96ec00f Constantinos Venetsanopoulos
1017 a96ec00f Constantinos Venetsanopoulos
1018 e5d8df8c Constantinos Venetsanopoulos
Installation of Pithos on node2
1019 e5d8df8c Constantinos Venetsanopoulos
===============================
1020 a96ec00f Constantinos Venetsanopoulos
1021 e5d8df8c Constantinos Venetsanopoulos
To install Pithos, grab the packages from our repository (make sure  you made
1022 a96ec00f Constantinos Venetsanopoulos
the additions needed in your ``/etc/apt/sources.list`` file, as described
1023 a96ec00f Constantinos Venetsanopoulos
previously), by running:
1024 a96ec00f Constantinos Venetsanopoulos
1025 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
1026 a96ec00f Constantinos Venetsanopoulos
1027 d587329c Konstantinos Tompoulidis
   # apt-get install snf-pithos-app snf-pithos-backend
1028 a96ec00f Constantinos Venetsanopoulos
1029 fb5187f9 Giorgos Korfiatis
Now, install the pithos web interface:
1030 a96ec00f Constantinos Venetsanopoulos
1031 138253bc Constantinos Venetsanopoulos
.. code-block:: console
1032 138253bc Constantinos Venetsanopoulos
1033 138253bc Constantinos Venetsanopoulos
   # apt-get install snf-pithos-webclient
1034 138253bc Constantinos Venetsanopoulos
1035 f8cdf6ec Dionysis Grigoropoulos
This package provides the standalone Pithos web client. The web client is the
1036 f8cdf6ec Dionysis Grigoropoulos
web UI for Pithos and will be accessible by clicking "Pithos" on the Astakos
1037 138253bc Constantinos Venetsanopoulos
interface's cloudbar, at the top of the Astakos homepage.
1038 a96ec00f Constantinos Venetsanopoulos
1039 caa6c07d Constantinos Venetsanopoulos
1040 04c1254b Constantinos Venetsanopoulos
.. _conf-pithos:
1041 04c1254b Constantinos Venetsanopoulos
1042 e5d8df8c Constantinos Venetsanopoulos
Configuration of Pithos
1043 e5d8df8c Constantinos Venetsanopoulos
=======================
1044 a96ec00f Constantinos Venetsanopoulos
1045 f8cdf6ec Dionysis Grigoropoulos
Gunicorn setup
1046 f8cdf6ec Dionysis Grigoropoulos
--------------
1047 f8cdf6ec Dionysis Grigoropoulos
1048 f8cdf6ec Dionysis Grigoropoulos
Copy the file ``/etc/gunicorn.d/synnefo.example`` to
1049 f8cdf6ec Dionysis Grigoropoulos
``/etc/gunicorn.d/synnefo``, to make it a valid gunicorn configuration file
1050 f8cdf6ec Dionysis Grigoropoulos
(as happened for node1):
1051 f8cdf6ec Dionysis Grigoropoulos
1052 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1053 f8cdf6ec Dionysis Grigoropoulos
1054 f8cdf6ec Dionysis Grigoropoulos
    # cp /etc/gunicorn.d/synnefo.example /etc/gunicorn.d/synnefo
1055 f8cdf6ec Dionysis Grigoropoulos
1056 f8cdf6ec Dionysis Grigoropoulos
1057 f8cdf6ec Dionysis Grigoropoulos
.. warning:: Do NOT start the server yet, because it won't find the
1058 f8cdf6ec Dionysis Grigoropoulos
    ``synnefo.settings`` module. Also, in case you are using ``/etc/hosts``
1059 f8cdf6ec Dionysis Grigoropoulos
    instead of a DNS to get the hostnames, change ``--worker-class=gevent`` to
1060 f8cdf6ec Dionysis Grigoropoulos
    ``--worker-class=sync``. We will start the server after successful
1061 f8cdf6ec Dionysis Grigoropoulos
    installation of Astakos. If the server is running::
1062 f8cdf6ec Dionysis Grigoropoulos
1063 f8cdf6ec Dionysis Grigoropoulos
       # /etc/init.d/gunicorn stop
1064 f8cdf6ec Dionysis Grigoropoulos
1065 a96ec00f Constantinos Venetsanopoulos
Conf Files
1066 a96ec00f Constantinos Venetsanopoulos
----------
1067 a96ec00f Constantinos Venetsanopoulos
1068 e5d8df8c Constantinos Venetsanopoulos
After Pithos is successfully installed, you will find the directory
1069 a96ec00f Constantinos Venetsanopoulos
``/etc/synnefo`` and some configuration files inside it, as you did in node1
1070 f8cdf6ec Dionysis Grigoropoulos
after installation of Astakos. Here, you will not have to change anything that
1071 a96ec00f Constantinos Venetsanopoulos
has to do with snf-common or snf-webproject. Everything is set at node1. You
1072 e5d8df8c Constantinos Venetsanopoulos
only need to change settings that have to do with Pithos. Specifically:
1073 a96ec00f Constantinos Venetsanopoulos
1074 a96ec00f Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-pithos-app-settings.conf``. There you need to set
1075 11c16930 Ilias Tsitsimpis
this options:
1076 a96ec00f Constantinos Venetsanopoulos
1077 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
1078 a96ec00f Constantinos Venetsanopoulos
1079 cceaebef Ilias Tsitsimpis
   ASTAKOS_AUTH_URL = 'https://node1.example.com/astakos/identity/v2.0'
1080 a96ec00f Constantinos Venetsanopoulos
1081 a14f152f Giorgos Korfiatis
   PITHOS_BASE_URL = 'https://node2.example.com/pithos'
1082 4ab620b6 Christos Stavrakakis
   PITHOS_BACKEND_DB_CONNECTION = 'postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos'
1083 a96ec00f Constantinos Venetsanopoulos
   PITHOS_BACKEND_BLOCK_PATH = '/srv/pithos/data'
1084 bdfd94c9 Constantinos Venetsanopoulos
1085 26498848 Giorgos Korfiatis
   PITHOS_SERVICE_TOKEN = 'pithos_service_token22w'
1086 d587329c Konstantinos Tompoulidis
1087 3e6d0710 Constantinos Venetsanopoulos
1088 e5d8df8c Constantinos Venetsanopoulos
The ``PITHOS_BACKEND_DB_CONNECTION`` option tells to the Pithos app where to
1089 e5d8df8c Constantinos Venetsanopoulos
find the Pithos backend database. Above we tell Pithos that its database is
1090 bdfd94c9 Constantinos Venetsanopoulos
``snf_pithos`` at node1 and to connect as user ``synnefo`` with password
1091 bdfd94c9 Constantinos Venetsanopoulos
``example_passw0rd``.  All those settings where setup during node1's "Database
1092 bdfd94c9 Constantinos Venetsanopoulos
setup" section.
1093 a96ec00f Constantinos Venetsanopoulos
1094 e5d8df8c Constantinos Venetsanopoulos
The ``PITHOS_BACKEND_BLOCK_PATH`` option tells to the Pithos app where to find
1095 e5d8df8c Constantinos Venetsanopoulos
the Pithos backend data. Above we tell Pithos to store its data under
1096 a96ec00f Constantinos Venetsanopoulos
``/srv/pithos/data``, which is visible by both nodes. We have already setup this
1097 e5d8df8c Constantinos Venetsanopoulos
directory at node1's "Pithos data directory setup" section.
1098 a96ec00f Constantinos Venetsanopoulos
1099 cceaebef Ilias Tsitsimpis
The ``ASTAKOS_AUTH_URL`` option informs the Pithos app where Astakos is.
1100 e3ff6830 Georgios D. Tsoukalas
The Astakos service is used for user management (authentication, quotas, etc.)
1101 8f85321e Sofia Papagiannaki
1102 a14f152f Giorgos Korfiatis
The ``PITHOS_BASE_URL`` setting must point to the top-level Pithos URL.
1103 a14f152f Giorgos Korfiatis
1104 f8cdf6ec Dionysis Grigoropoulos
The ``PITHOS_SERVICE_TOKEN`` is the token used for authentication with Astakos.
1105 a14f152f Giorgos Korfiatis
It can be retrieved by running on the Astakos node (node1 in our case):
1106 3e6d0710 Constantinos Venetsanopoulos
1107 3e6d0710 Constantinos Venetsanopoulos
.. code-block:: console
1108 3e6d0710 Constantinos Venetsanopoulos
1109 a14f152f Giorgos Korfiatis
   # snf-manage component-list
1110 3e6d0710 Constantinos Venetsanopoulos
1111 e5d8df8c Constantinos Venetsanopoulos
The token has been generated automatically during the :ref:`Pithos service
1112 3e6d0710 Constantinos Venetsanopoulos
registration <services-reg>`.
1113 3e6d0710 Constantinos Venetsanopoulos
1114 591e1df0 Sofia Papagiannaki
The ``PITHOS_UPDATE_MD5`` option by default disables the computation of the
1115 591e1df0 Sofia Papagiannaki
object checksums. This results to improved performance during object uploading.
1116 591e1df0 Sofia Papagiannaki
However, if compatibility with the OpenStack Object Storage API is important
1117 591e1df0 Sofia Papagiannaki
then it should be changed to ``True``.
1118 591e1df0 Sofia Papagiannaki
1119 138253bc Constantinos Venetsanopoulos
Then edit ``/etc/synnefo/20-snf-pithos-webclient-cloudbar.conf``, to connect the
1120 f8cdf6ec Dionysis Grigoropoulos
Pithos web UI with the Astakos web UI (through the top cloudbar):
1121 138253bc Constantinos Venetsanopoulos
1122 138253bc Constantinos Venetsanopoulos
.. code-block:: console
1123 138253bc Constantinos Venetsanopoulos
1124 d587329c Konstantinos Tompoulidis
    CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
1125 a14f152f Giorgos Korfiatis
    CLOUDBAR_SERVICES_URL = 'https://node1.example.com/astakos/ui/get_services'
1126 a14f152f Giorgos Korfiatis
    CLOUDBAR_MENU_URL = 'https://node1.example.com/astakos/ui/get_menu'
1127 138253bc Constantinos Venetsanopoulos
1128 f8cdf6ec Dionysis Grigoropoulos
The ``CLOUDBAR_LOCATION`` tells the client where to find the Astakos common
1129 138253bc Constantinos Venetsanopoulos
cloudbar.
1130 138253bc Constantinos Venetsanopoulos
1131 138253bc Constantinos Venetsanopoulos
The ``CLOUDBAR_SERVICES_URL`` and ``CLOUDBAR_MENU_URL`` options are used by the
1132 f8cdf6ec Dionysis Grigoropoulos
Pithos web client to get from Astakos all the information needed to fill its
1133 f8cdf6ec Dionysis Grigoropoulos
own cloudbar. So we put our Astakos deployment urls there.
1134 138253bc Constantinos Venetsanopoulos
1135 5547485e Sofia Papagiannaki
The ``PITHOS_OAUTH2_CLIENT_CREDENTIALS`` setting is used by the pithos view
1136 5547485e Sofia Papagiannaki
in order to authenticate itself with astakos during the authorization grant
1137 5547485e Sofia Papagiannaki
procedure and it should container the credentials issued for the pithos view
1138 5547485e Sofia Papagiannaki
in `the pithos view registration step`__.
1139 5547485e Sofia Papagiannaki
1140 5547485e Sofia Papagiannaki
__ pithos_view_registration_
1141 5547485e Sofia Papagiannaki
1142 4e3e3d24 Constantinos Venetsanopoulos
Pooling and Greenlets
1143 4e3e3d24 Constantinos Venetsanopoulos
---------------------
1144 4e3e3d24 Constantinos Venetsanopoulos
1145 4e3e3d24 Constantinos Venetsanopoulos
Pithos is pooling-ready without the need of further configuration, because it
1146 f8cdf6ec Dionysis Grigoropoulos
doesn't use a Django DB. It pools HTTP connections to Astakos and Pithos
1147 4e3e3d24 Constantinos Venetsanopoulos
backend objects for access to the Pithos DB.
1148 4e3e3d24 Constantinos Venetsanopoulos
1149 576e059b Constantinos Venetsanopoulos
However, as in Astakos, since we are running with Greenlets, it is also
1150 576e059b Constantinos Venetsanopoulos
recommended to modify psycopg2 behavior so it works properly in a greenlet
1151 576e059b Constantinos Venetsanopoulos
context. This means adding the following lines at the top of your
1152 4e3e3d24 Constantinos Venetsanopoulos
``/etc/synnefo/10-snf-webproject-database.conf`` file:
1153 4e3e3d24 Constantinos Venetsanopoulos
1154 4e3e3d24 Constantinos Venetsanopoulos
.. code-block:: console
1155 4e3e3d24 Constantinos Venetsanopoulos
1156 d587329c Konstantinos Tompoulidis
    from synnefo.lib.db.psyco_gevent import make_psycopg_green
1157 d587329c Konstantinos Tompoulidis
    make_psycopg_green()
1158 d587329c Konstantinos Tompoulidis
1159 d587329c Konstantinos Tompoulidis
Furthermore, add the ``--worker-class=gevent`` (or ``--worker-class=sync`` as
1160 d587329c Konstantinos Tompoulidis
mentioned above, depending on your setup) argument on your
1161 d587329c Konstantinos Tompoulidis
``/etc/gunicorn.d/synnefo`` configuration file. The file should look something
1162 d587329c Konstantinos Tompoulidis
like this:
1163 d587329c Konstantinos Tompoulidis
1164 d587329c Konstantinos Tompoulidis
.. code-block:: console
1165 d587329c Konstantinos Tompoulidis
1166 d587329c Konstantinos Tompoulidis
    CONFIG = {
1167 d587329c Konstantinos Tompoulidis
     'mode': 'django',
1168 d587329c Konstantinos Tompoulidis
     'environment': {
1169 d587329c Konstantinos Tompoulidis
       'DJANGO_SETTINGS_MODULE': 'synnefo.settings',
1170 d587329c Konstantinos Tompoulidis
     },
1171 d587329c Konstantinos Tompoulidis
     'working_dir': '/etc/synnefo',
1172 d587329c Konstantinos Tompoulidis
     'user': 'www-data',
1173 d587329c Konstantinos Tompoulidis
     'group': 'www-data',
1174 d587329c Konstantinos Tompoulidis
     'args': (
1175 d587329c Konstantinos Tompoulidis
       '--bind=127.0.0.1:8080',
1176 d587329c Konstantinos Tompoulidis
       '--workers=4',
1177 d587329c Konstantinos Tompoulidis
       '--worker-class=gevent',
1178 d587329c Konstantinos Tompoulidis
       '--log-level=debug',
1179 d587329c Konstantinos Tompoulidis
       '--timeout=43200'
1180 d587329c Konstantinos Tompoulidis
     ),
1181 d587329c Konstantinos Tompoulidis
    }
1182 4e3e3d24 Constantinos Venetsanopoulos
1183 c17bd3a7 Sofia Papagiannaki
Stamp Database Revision
1184 c17bd3a7 Sofia Papagiannaki
-----------------------
1185 c17bd3a7 Sofia Papagiannaki
1186 c17bd3a7 Sofia Papagiannaki
Pithos uses the alembic_ database migrations tool.
1187 c17bd3a7 Sofia Papagiannaki
1188 c17bd3a7 Sofia Papagiannaki
.. _alembic: http://alembic.readthedocs.org
1189 c17bd3a7 Sofia Papagiannaki
1190 61c39121 Giorgos Korfiatis
After a successful installation, we should stamp it at the most recent
1191 a33ee5d2 Sofia Papagiannaki
revision, so that future migrations know where to start upgrading in
1192 a33ee5d2 Sofia Papagiannaki
the migration history.
1193 c17bd3a7 Sofia Papagiannaki
1194 c17bd3a7 Sofia Papagiannaki
.. code-block:: console
1195 c17bd3a7 Sofia Papagiannaki
1196 61c39121 Giorgos Korfiatis
    root@node2:~ # pithos-migrate stamp head
1197 c17bd3a7 Sofia Papagiannaki
1198 f8cdf6ec Dionysis Grigoropoulos
Mount the NFS directory
1199 f8cdf6ec Dionysis Grigoropoulos
-----------------------
1200 f8cdf6ec Dionysis Grigoropoulos
1201 f8cdf6ec Dionysis Grigoropoulos
First install the package nfs-common by running:
1202 f8cdf6ec Dionysis Grigoropoulos
1203 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1204 f8cdf6ec Dionysis Grigoropoulos
1205 f8cdf6ec Dionysis Grigoropoulos
   root@node2:~ # apt-get install nfs-common
1206 f8cdf6ec Dionysis Grigoropoulos
1207 f8cdf6ec Dionysis Grigoropoulos
now create the directory /srv/pithos/ and mount the remote directory to it:
1208 f8cdf6ec Dionysis Grigoropoulos
1209 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1210 f8cdf6ec Dionysis Grigoropoulos
1211 f8cdf6ec Dionysis Grigoropoulos
   root@node2:~ # mkdir /srv/pithos/
1212 f8cdf6ec Dionysis Grigoropoulos
   root@node2:~ # mount -t nfs 203.0.113.1:/srv/pithos/ /srv/pithos/
1213 f8cdf6ec Dionysis Grigoropoulos
1214 a96ec00f Constantinos Venetsanopoulos
Servers Initialization
1215 a96ec00f Constantinos Venetsanopoulos
----------------------
1216 a96ec00f Constantinos Venetsanopoulos
1217 a96ec00f Constantinos Venetsanopoulos
After configuration is done, we initialize the servers on node2:
1218 a96ec00f Constantinos Venetsanopoulos
1219 a96ec00f Constantinos Venetsanopoulos
.. code-block:: console
1220 a96ec00f Constantinos Venetsanopoulos
1221 d587329c Konstantinos Tompoulidis
    root@node2:~ # /etc/init.d/gunicorn restart
1222 d587329c Konstantinos Tompoulidis
    root@node2:~ # /etc/init.d/apache2 restart
1223 a96ec00f Constantinos Venetsanopoulos
1224 e5d8df8c Constantinos Venetsanopoulos
You have now finished the Pithos setup. Let's test it now.
1225 a96ec00f Constantinos Venetsanopoulos
1226 e5d8df8c Constantinos Venetsanopoulos
Testing of Pithos
1227 e5d8df8c Constantinos Venetsanopoulos
=================
1228 a96ec00f Constantinos Venetsanopoulos
1229 bdfd94c9 Constantinos Venetsanopoulos
Open your browser and go to the Astakos homepage:
1230 bdfd94c9 Constantinos Venetsanopoulos
1231 a14f152f Giorgos Korfiatis
``http://node1.example.com/astakos``
1232 bdfd94c9 Constantinos Venetsanopoulos
1233 f8cdf6ec Dionysis Grigoropoulos
Login, and you will see your profile page. Now, click the "Pithos" link on the
1234 bdfd94c9 Constantinos Venetsanopoulos
top black cloudbar. If everything was setup correctly, this will redirect you
1235 bdfd94c9 Constantinos Venetsanopoulos
to:
1236 bdfd94c9 Constantinos Venetsanopoulos
1237 f8cdf6ec Dionysis Grigoropoulos
``https://node2.example.com/ui``
1238 bdfd94c9 Constantinos Venetsanopoulos
1239 e5d8df8c Constantinos Venetsanopoulos
and you will see the blue interface of the Pithos application.  Click the
1240 bdfd94c9 Constantinos Venetsanopoulos
orange "Upload" button and upload your first file. If the file gets uploaded
1241 e5d8df8c Constantinos Venetsanopoulos
successfully, then this is your first sign of a successful Pithos installation.
1242 bdfd94c9 Constantinos Venetsanopoulos
Go ahead and experiment with the interface to make sure everything works
1243 bdfd94c9 Constantinos Venetsanopoulos
correctly.
1244 bdfd94c9 Constantinos Venetsanopoulos
1245 e5d8df8c Constantinos Venetsanopoulos
You can also use the Pithos clients to sync data from your Windows PC or MAC.
1246 bdfd94c9 Constantinos Venetsanopoulos
1247 bdfd94c9 Constantinos Venetsanopoulos
If you don't stumble on any problems, then you have successfully installed
1248 e5d8df8c Constantinos Venetsanopoulos
Pithos, which you can use as a standalone File Storage Service.
1249 bdfd94c9 Constantinos Venetsanopoulos
1250 bdfd94c9 Constantinos Venetsanopoulos
If you would like to do more, such as:
1251 bdfd94c9 Constantinos Venetsanopoulos
1252 d587329c Konstantinos Tompoulidis
    * Spawning VMs
1253 e5d8df8c Constantinos Venetsanopoulos
    * Spawning VMs from Images stored on Pithos
1254 e5d8df8c Constantinos Venetsanopoulos
    * Uploading your custom Images to Pithos
1255 d587329c Konstantinos Tompoulidis
    * Spawning VMs from those custom Images
1256 e5d8df8c Constantinos Venetsanopoulos
    * Registering existing Pithos files as Images
1257 d587329c Konstantinos Tompoulidis
    * Connect VMs to the Internet
1258 d587329c Konstantinos Tompoulidis
    * Create Private Networks
1259 d587329c Konstantinos Tompoulidis
    * Add VMs to Private Networks
1260 bdfd94c9 Constantinos Venetsanopoulos
1261 bdfd94c9 Constantinos Venetsanopoulos
please continue with the rest of the guide.
1262 a96ec00f Constantinos Venetsanopoulos
1263 caa6c07d Constantinos Venetsanopoulos
1264 f8cdf6ec Dionysis Grigoropoulos
Kamaki
1265 f8cdf6ec Dionysis Grigoropoulos
======
1266 f8cdf6ec Dionysis Grigoropoulos
1267 dd6062f2 Christos Stavrakakis
`Kamaki <http://www.synnefo.org/docs/kamaki/latest/index.html>`_ is an
1268 dd6062f2 Christos Stavrakakis
Openstack API client library and command line interface with custom extentions
1269 f8cdf6ec Dionysis Grigoropoulos
specific to Synnefo.
1270 f8cdf6ec Dionysis Grigoropoulos
1271 f8cdf6ec Dionysis Grigoropoulos
Kamaki Installation and Configuration
1272 f8cdf6ec Dionysis Grigoropoulos
-------------------------------------
1273 f8cdf6ec Dionysis Grigoropoulos
1274 f8cdf6ec Dionysis Grigoropoulos
To install kamaki run:
1275 f8cdf6ec Dionysis Grigoropoulos
1276 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1277 f8cdf6ec Dionysis Grigoropoulos
1278 f8cdf6ec Dionysis Grigoropoulos
   # apt-get install kamaki
1279 f8cdf6ec Dionysis Grigoropoulos
1280 dd6062f2 Christos Stavrakakis
Now, visit
1281 f8cdf6ec Dionysis Grigoropoulos
1282 f8cdf6ec Dionysis Grigoropoulos
 `https://node1.example.com/astakos/ui/`
1283 f8cdf6ec Dionysis Grigoropoulos
1284 dd6062f2 Christos Stavrakakis
log in and click on ``API access``. Scroll all the way to the bottom of the
1285 f8cdf6ec Dionysis Grigoropoulos
page, click on the orange ``Download your .kamakirc`` button and save the file
1286 f8cdf6ec Dionysis Grigoropoulos
as ``.kamakirc`` in your home directory.
1287 f8cdf6ec Dionysis Grigoropoulos
1288 f8cdf6ec Dionysis Grigoropoulos
That's all, kamaki is now configured and you can start using it. For a list of
1289 f8cdf6ec Dionysis Grigoropoulos
commands, see the `official documentantion <http://www.synnefo.org/docs/kamaki/latest/commands.html>`_.
1290 f8cdf6ec Dionysis Grigoropoulos
1291 73ebcd68 Constantinos Venetsanopoulos
Cyclades Prerequisites
1292 73ebcd68 Constantinos Venetsanopoulos
======================
1293 bc055d09 Constantinos Venetsanopoulos
1294 73ebcd68 Constantinos Venetsanopoulos
Before proceeding with the Cyclades installation, make sure you have
1295 e5d8df8c Constantinos Venetsanopoulos
successfully set up Astakos and Pithos first, because Cyclades depends on
1296 e5d8df8c Constantinos Venetsanopoulos
them. If you don't have a working Astakos and Pithos installation yet, please
1297 73ebcd68 Constantinos Venetsanopoulos
return to the :ref:`top <quick-install-admin-guide>` of this guide.
1298 bc055d09 Constantinos Venetsanopoulos
1299 e5d8df8c Constantinos Venetsanopoulos
Besides Astakos and Pithos, you will also need a number of additional working
1300 caa6c07d Constantinos Venetsanopoulos
prerequisites, before you start the Cyclades installation.
1301 bc055d09 Constantinos Venetsanopoulos
1302 caa6c07d Constantinos Venetsanopoulos
Ganeti
1303 7a8df455 Constantinos Venetsanopoulos
------
1304 bc055d09 Constantinos Venetsanopoulos
1305 caa6c07d Constantinos Venetsanopoulos
`Ganeti <http://code.google.com/p/ganeti/>`_ handles the low level VM management
1306 caa6c07d Constantinos Venetsanopoulos
for Cyclades, so Cyclades requires a working Ganeti installation at the backend.
1307 dd6062f2 Christos Stavrakakis
Please refer to the `ganeti documentation <http://docs.ganeti.org/ganeti/2.8/html>`_ for all
1308 f8cdf6ec Dionysis Grigoropoulos
the gory details. A successful Ganeti installation concludes with a working
1309 caa6c07d Constantinos Venetsanopoulos
:ref:`GANETI-MASTER <GANETI_NODES>` and a number of :ref:`GANETI-NODEs
1310 caa6c07d Constantinos Venetsanopoulos
<GANETI_NODES>`.
1311 caa6c07d Constantinos Venetsanopoulos
1312 caa6c07d Constantinos Venetsanopoulos
The above Ganeti cluster can run on different physical machines than node1 and
1313 caa6c07d Constantinos Venetsanopoulos
node2 and can scale independently, according to your needs.
1314 caa6c07d Constantinos Venetsanopoulos
1315 caa6c07d Constantinos Venetsanopoulos
For the purpose of this guide, we will assume that the :ref:`GANETI-MASTER
1316 caa6c07d Constantinos Venetsanopoulos
<GANETI_NODES>` runs on node1 and is VM-capable. Also, node2 is a
1317 caa6c07d Constantinos Venetsanopoulos
:ref:`GANETI-NODE <GANETI_NODES>` and is Master-capable and VM-capable too.
1318 caa6c07d Constantinos Venetsanopoulos
1319 caa6c07d Constantinos Venetsanopoulos
We highly recommend that you read the official Ganeti documentation, if you are
1320 576e059b Constantinos Venetsanopoulos
not familiar with Ganeti.
1321 f1f5235e Dimitris Aragiorgis
1322 f8cdf6ec Dionysis Grigoropoulos
Ganeti Prerequisites
1323 f8cdf6ec Dionysis Grigoropoulos
--------------------
1324 f8cdf6ec Dionysis Grigoropoulos
You're gonna need the ``lvm2`` and ``vlan`` packages, so run:
1325 f1f5235e Dimitris Aragiorgis
1326 f1f5235e Dimitris Aragiorgis
.. code-block:: console
1327 f1f5235e Dimitris Aragiorgis
1328 f8cdf6ec Dionysis Grigoropoulos
   # apt-get install lvm2 vlan
1329 f8cdf6ec Dionysis Grigoropoulos
1330 f8cdf6ec Dionysis Grigoropoulos
Ganeti requires FQDN. To properly configure your nodes please
1331 f8cdf6ec Dionysis Grigoropoulos
see `this <http://docs.ganeti.org/ganeti/2.6/html/install.html#hostname-issues>`_.
1332 f8cdf6ec Dionysis Grigoropoulos
1333 dd6062f2 Christos Stavrakakis
Ganeti requires an extra available IP and its FQDN e.g., ``203.0.113.100`` and
1334 dd6062f2 Christos Stavrakakis
``ganeti.node1.example.com``. Add this IP to your DNS server configuration, as
1335 f8cdf6ec Dionysis Grigoropoulos
explained above.
1336 f8cdf6ec Dionysis Grigoropoulos
1337 dd6062f2 Christos Stavrakakis
Also, Ganeti will need a volume group with the same name e.g., ``ganeti``
1338 dd6062f2 Christos Stavrakakis
across all nodes, of at least 20GiB. To create the volume group,
1339 f8cdf6ec Dionysis Grigoropoulos
see `this <http://www.tldp.org/HOWTO/LVM-HOWTO/createvgs.html>`_.
1340 f8cdf6ec Dionysis Grigoropoulos
1341 f8cdf6ec Dionysis Grigoropoulos
Moreover, node1 and node2 must have the same dsa, rsa keys and authorised_keys
1342 dd6062f2 Christos Stavrakakis
under ``/root/.ssh/`` for password-less root ssh between each other. To
1343 f8cdf6ec Dionysis Grigoropoulos
generate said keys, see `this <https://wiki.debian.org/SSH#Using_shared_keys>`_.
1344 f8cdf6ec Dionysis Grigoropoulos
1345 f8cdf6ec Dionysis Grigoropoulos
In the following sections, we assume that the public interface of all nodes is
1346 dd6062f2 Christos Stavrakakis
``eth0`` and there are two extra interfaces ``eth1`` and ``eth2``, which can
1347 dd6062f2 Christos Stavrakakis
also be vlans on your primary interface e.g., ``eth0.1`` and ``eth0.2``  in
1348 dd6062f2 Christos Stavrakakis
case you don't have multiple physical interfaces. For information on how to
1349 f8cdf6ec Dionysis Grigoropoulos
create vlans, please see
1350 f8cdf6ec Dionysis Grigoropoulos
`this <https://wiki.debian.org/NetworkConfiguration#Howto_use_vlan_.28dot1q.2C_802.1q.2C_trunk.29_.28Etch.2C_Lenny.29>`_.
1351 cc16407a Dimitris Aragiorgis
1352 dd6062f2 Christos Stavrakakis
Finally, setup two bridges on the host machines (e.g: br1/br2 on eth1/eth2
1353 f8cdf6ec Dionysis Grigoropoulos
respectively), as described `here <https://wiki.debian.org/BridgeNetworkConnections>`_.
1354 f8cdf6ec Dionysis Grigoropoulos
1355 f8cdf6ec Dionysis Grigoropoulos
Ganeti Installation and Initialization
1356 f8cdf6ec Dionysis Grigoropoulos
--------------------------------------
1357 f8cdf6ec Dionysis Grigoropoulos
1358 dd6062f2 Christos Stavrakakis
We assume that Ganeti will use the KVM hypervisor. To install KVM, run on all
1359 f8cdf6ec Dionysis Grigoropoulos
Ganeti nodes:
1360 0c068fc6 marioskogias
1361 0c068fc6 marioskogias
.. code-block:: console
1362 0c068fc6 marioskogias
1363 f8cdf6ec Dionysis Grigoropoulos
   # apt-get install qemu-kvm
1364 f8cdf6ec Dionysis Grigoropoulos
1365 dd6062f2 Christos Stavrakakis
It's time to install Ganeti. To be able to use hotplug (which will be part of
1366 f8cdf6ec Dionysis Grigoropoulos
the official Ganeti 2.10), we recommend using our Ganeti package version:
1367 f8cdf6ec Dionysis Grigoropoulos
1368 9e5a9a9a Dimitris Aragiorgis
``2.8.3+snap1+b64v1+kvm1+ext1+lockfix1+ipfix1+backports1-1~wheezy``
1369 f8cdf6ec Dionysis Grigoropoulos
1370 9e5a9a9a Dimitris Aragiorgis
Let's briefly explain each patch set:
1371 f8cdf6ec Dionysis Grigoropoulos
1372 9e5a9a9a Dimitris Aragiorgis
    * snap adds snapshot support for ext disk template
1373 9e5a9a9a Dimitris Aragiorgis
    * b64 saves networks' bitarrays in a more compact representation
1374 9e5a9a9a Dimitris Aragiorgis
    * kvm exports disk geometry to kvm command and adds migration capabilities
1375 9e5a9a9a Dimitris Aragiorgis
    * ext
1376 cbb596d4 Dionysis Grigoropoulos
1377 9e5a9a9a Dimitris Aragiorgis
      * exports logical id in hooks
1378 9e5a9a9a Dimitris Aragiorgis
      * allows cache, heads, cyls arbitrary params to reach kvm command
1379 cbb596d4 Dionysis Grigoropoulos
1380 9e5a9a9a Dimitris Aragiorgis
    * lockfix is a workaround for Issue #621
1381 9e5a9a9a Dimitris Aragiorgis
    * ipfix does not require IP if mode is routed (needed for IPv6 only NICs)
1382 9e5a9a9a Dimitris Aragiorgis
    * backports is a set of patches backported from stable-2.10
1383 f8cdf6ec Dionysis Grigoropoulos
1384 9e5a9a9a Dimitris Aragiorgis
      * Hotplug support
1385 9e5a9a9a Dimitris Aragiorgis
      * Better networking support (NIC configuration scripts)
1386 9e5a9a9a Dimitris Aragiorgis
      * Change IP pool to support NAT instances
1387 9e5a9a9a Dimitris Aragiorgis
      * Change RAPI to accept depends body argument and shutdown_timeout
1388 caa6c07d Constantinos Venetsanopoulos
1389 f8cdf6ec Dionysis Grigoropoulos
To install Ganeti run:
1390 f8cdf6ec Dionysis Grigoropoulos
1391 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1392 f8cdf6ec Dionysis Grigoropoulos
1393 9e5a9a9a Dimitris Aragiorgis
   # apt-get install snf-ganeti ganeti-htools ganeti-haskell ganeti2
1394 f8cdf6ec Dionysis Grigoropoulos
1395 dd6062f2 Christos Stavrakakis
Ganeti will make use of drbd. To enable this and make the configuration
1396 f8cdf6ec Dionysis Grigoropoulos
permanent you have to do the following :
1397 f8cdf6ec Dionysis Grigoropoulos
1398 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1399 f8cdf6ec Dionysis Grigoropoulos
1400 f8cdf6ec Dionysis Grigoropoulos
   # modprobe drbd minor_count=255 usermode_helper=/bin/true
1401 f8cdf6ec Dionysis Grigoropoulos
   # echo 'drbd minor_count=255 usermode_helper=/bin/true' >> /etc/modules
1402 0c068fc6 marioskogias
1403 92e4d1c6 Dimitris Aragiorgis
Then run on node1:
1404 caa6c07d Constantinos Venetsanopoulos
1405 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1406 caa6c07d Constantinos Venetsanopoulos
1407 d587329c Konstantinos Tompoulidis
    root@node1:~ # gnt-cluster init --enabled-hypervisors=kvm --no-ssh-init \
1408 f8cdf6ec Dionysis Grigoropoulos
                    --no-etc-hosts --vg-name=ganeti --nic-parameters link=br1 \
1409 f8cdf6ec Dionysis Grigoropoulos
                    --default-iallocator hail \
1410 f8cdf6ec Dionysis Grigoropoulos
                    --hypervisor-parameters kvm:kernel_path=,vnc_bind_address=0.0.0.0 \
1411 7b1a14eb Christos Stavrakakis
                    --specs-nic-count min=0,max=16 \
1412 d587329c Konstantinos Tompoulidis
                    --master-netdev eth0 ganeti.node1.example.com
1413 dd6062f2 Christos Stavrakakis
1414 d587329c Konstantinos Tompoulidis
    root@node1:~ # gnt-node add --no-ssh-key-check --master-capable=yes \
1415 d587329c Konstantinos Tompoulidis
                    --vm-capable=yes node2.example.com
1416 d587329c Konstantinos Tompoulidis
    root@node1:~ # gnt-cluster modify --disk-parameters=drbd:metavg=ganeti
1417 d587329c Konstantinos Tompoulidis
    root@node1:~ # gnt-group modify --disk-parameters=drbd:metavg=ganeti default
1418 caa6c07d Constantinos Venetsanopoulos
1419 f8cdf6ec Dionysis Grigoropoulos
``br1`` will be the default interface for any newly created VMs.
1420 f8cdf6ec Dionysis Grigoropoulos
1421 cbb596d4 Dionysis Grigoropoulos
You can verify that the ganeti cluster is successfully setup, by running on the
1422 52188a27 Kostis Fardelas
:ref:`GANETI-MASTER <GANETI_NODES>` (in our case node1):
1423 52188a27 Kostis Fardelas
1424 52188a27 Kostis Fardelas
.. code-block:: console
1425 52188a27 Kostis Fardelas
1426 52188a27 Kostis Fardelas
   # gnt-cluster verify
1427 52188a27 Kostis Fardelas
1428 caa6c07d Constantinos Venetsanopoulos
.. _cyclades-install-snfimage:
1429 caa6c07d Constantinos Venetsanopoulos
1430 caa6c07d Constantinos Venetsanopoulos
snf-image
1431 7a8df455 Constantinos Venetsanopoulos
---------
1432 caa6c07d Constantinos Venetsanopoulos
1433 caa6c07d Constantinos Venetsanopoulos
Installation
1434 7a8df455 Constantinos Venetsanopoulos
~~~~~~~~~~~~
1435 caa6c07d Constantinos Venetsanopoulos
For :ref:`Cyclades <cyclades>` to be able to launch VMs from specified Images,
1436 f8cdf6ec Dionysis Grigoropoulos
you need the `snf-image <http://www.synnefo.org/docs/snf-image/latest/index.html>`_ OS
1437 59ab0466 Nikos Skalkotos
Definition installed on *all* VM-capable Ganeti nodes. This means we need
1438 59ab0466 Nikos Skalkotos
:ref:`snf-image <http://www.synnefo.org/docs/snf-image/latest/index.html>` on
1439 caa6c07d Constantinos Venetsanopoulos
node1 and node2. You can do this by running on *both* nodes:
1440 caa6c07d Constantinos Venetsanopoulos
1441 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1442 caa6c07d Constantinos Venetsanopoulos
1443 d587329c Konstantinos Tompoulidis
   # apt-get install snf-image snf-pithos-backend python-psycopg2
1444 1a37da56 Constantinos Venetsanopoulos
1445 d587329c Konstantinos Tompoulidis
snf-image also needs the `snf-pithos-backend <snf-pithos-backend>`, to be able
1446 e5d8df8c Constantinos Venetsanopoulos
to handle image files stored on Pithos. It also needs `python-psycopg2` to be
1447 e5d8df8c Constantinos Venetsanopoulos
able to access the Pithos database. This is why, we also install them on *all*
1448 1a37da56 Constantinos Venetsanopoulos
VM-capable Ganeti nodes.
1449 caa6c07d Constantinos Venetsanopoulos
1450 dd45ee44 Dionysis Grigoropoulos
.. warning::
1451 0c068fc6 marioskogias
		snf-image uses ``curl`` for handling URLs. This means that it will
1452 0c068fc6 marioskogias
		not  work out of the box if you try to use URLs served by servers which do
1453 0c068fc6 marioskogias
		not have a valid certificate. In case you haven't followed the guide's
1454 f8cdf6ec Dionysis Grigoropoulos
		directions about the certificates, in order to circumvent this you should edit the file
1455 0c068fc6 marioskogias
		``/etc/default/snf-image``. Change ``#CURL="curl"`` to ``CURL="curl -k"`` on every node.
1456 d587329c Konstantinos Tompoulidis
1457 caa6c07d Constantinos Venetsanopoulos
Configuration
1458 7a8df455 Constantinos Venetsanopoulos
~~~~~~~~~~~~~
1459 e5d8df8c Constantinos Venetsanopoulos
snf-image supports native access to Images stored on Pithos. This means that
1460 e5d8df8c Constantinos Venetsanopoulos
it can talk directly to the Pithos backend, without the need of providing a
1461 d587329c Konstantinos Tompoulidis
public URL. More details, are described in the next section. For now, the only
1462 e5d8df8c Constantinos Venetsanopoulos
thing we need to do, is configure snf-image to access our Pithos backend.
1463 caa6c07d Constantinos Venetsanopoulos
1464 f8cdf6ec Dionysis Grigoropoulos
To do this, we need to set the corresponding variable in
1465 e5d8df8c Constantinos Venetsanopoulos
``/etc/default/snf-image``, to reflect our Pithos setup:
1466 caa6c07d Constantinos Venetsanopoulos
1467 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1468 caa6c07d Constantinos Venetsanopoulos
1469 d587329c Konstantinos Tompoulidis
    PITHOS_DATA="/srv/pithos/data"
1470 caa6c07d Constantinos Venetsanopoulos
1471 d587329c Konstantinos Tompoulidis
If you have installed your Ganeti cluster on different nodes than node1 and
1472 d587329c Konstantinos Tompoulidis
node2 make sure that ``/srv/pithos/data`` is visible by all of them.
1473 caa6c07d Constantinos Venetsanopoulos
1474 caa6c07d Constantinos Venetsanopoulos
If you would like to use Images that are also/only stored locally, you need to
1475 caa6c07d Constantinos Venetsanopoulos
save them under ``IMAGE_DIR``, however this guide targets Images stored only on
1476 e5d8df8c Constantinos Venetsanopoulos
Pithos.
1477 caa6c07d Constantinos Venetsanopoulos
1478 caa6c07d Constantinos Venetsanopoulos
Testing
1479 7a8df455 Constantinos Venetsanopoulos
~~~~~~~
1480 caa6c07d Constantinos Venetsanopoulos
You can test that snf-image is successfully installed by running on the
1481 caa6c07d Constantinos Venetsanopoulos
:ref:`GANETI-MASTER <GANETI_NODES>` (in our case node1):
1482 caa6c07d Constantinos Venetsanopoulos
1483 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1484 caa6c07d Constantinos Venetsanopoulos
1485 caa6c07d Constantinos Venetsanopoulos
   # gnt-os diagnose
1486 caa6c07d Constantinos Venetsanopoulos
1487 caa6c07d Constantinos Venetsanopoulos
This should return ``valid`` for snf-image.
1488 caa6c07d Constantinos Venetsanopoulos
1489 caa6c07d Constantinos Venetsanopoulos
If you are interested to learn more about snf-image's internals (and even use
1490 caa6c07d Constantinos Venetsanopoulos
it alongside Ganeti without Synnefo), please see
1491 5beef175 Nikos Skalkotos
`here <http://www.synnefo.org/docs/snf-image/latest/index.html>`_ for information
1492 d587329c Konstantinos Tompoulidis
concerning installation instructions, documentation on the design and
1493 d587329c Konstantinos Tompoulidis
implementation, and supported Image formats.
1494 caa6c07d Constantinos Venetsanopoulos
1495 8a4cd31b Constantinos Venetsanopoulos
.. _snf-image-images:
1496 8a4cd31b Constantinos Venetsanopoulos
1497 cd837dad Constantinos Venetsanopoulos
Actual Images for snf-image
1498 cd837dad Constantinos Venetsanopoulos
---------------------------
1499 caa6c07d Constantinos Venetsanopoulos
1500 caa6c07d Constantinos Venetsanopoulos
Now that snf-image is installed successfully we need to provide it with some
1501 59ab0466 Nikos Skalkotos
Images.
1502 59ab0466 Nikos Skalkotos
:ref:`snf-image <http://www.synnefo.org/docs/snf-image/latest/index.html>`
1503 59ab0466 Nikos Skalkotos
supports Images stored in ``extdump``, ``ntfsdump`` or ``diskdump`` format. We
1504 59ab0466 Nikos Skalkotos
recommend the use of the ``diskdump`` format. For more information about
1505 59ab0466 Nikos Skalkotos
snf-image Image formats see `here
1506 5beef175 Nikos Skalkotos
<http://www.synnefo.org/docs/snf-image/latest/usage.html#image-format>`_.
1507 caa6c07d Constantinos Venetsanopoulos
1508 59ab0466 Nikos Skalkotos
:ref:`snf-image <http://www.synnefo.org/docs/snf-image/latest/index.html>`
1509 59ab0466 Nikos Skalkotos
also supports three (3) different locations for the above Images to be stored:
1510 caa6c07d Constantinos Venetsanopoulos
1511 d587329c Konstantinos Tompoulidis
    * Under a local folder (usually an NFS mount, configurable as ``IMAGE_DIR``
1512 d587329c Konstantinos Tompoulidis
      in :file:`/etc/default/snf-image`)
1513 d587329c Konstantinos Tompoulidis
    * On a remote host (accessible via public URL e.g: http://... or ftp://...)
1514 e5d8df8c Constantinos Venetsanopoulos
    * On Pithos (accessible natively, not only by its public URL)
1515 caa6c07d Constantinos Venetsanopoulos
1516 d587329c Konstantinos Tompoulidis
For the purpose of this guide, we will use the Debian Squeeze Base Image found
1517 d587329c Konstantinos Tompoulidis
on the official `snf-image page
1518 5beef175 Nikos Skalkotos
<http://www.synnefo.org/docs/snf-image/latest/usage.html#sample-images>`_. The
1519 5beef175 Nikos Skalkotos
image is of type ``diskdump``. We will store it in our new Pithos installation.
1520 caa6c07d Constantinos Venetsanopoulos
1521 caa6c07d Constantinos Venetsanopoulos
To do so, do the following:
1522 caa6c07d Constantinos Venetsanopoulos
1523 d587329c Konstantinos Tompoulidis
a) Download the Image from the official snf-image page.
1524 caa6c07d Constantinos Venetsanopoulos
1525 e5d8df8c Constantinos Venetsanopoulos
b) Upload the Image to your Pithos installation, either using the Pithos Web
1526 d587329c Konstantinos Tompoulidis
   UI or the command line client `kamaki
1527 34e79416 Constantinos Venetsanopoulos
   <http://www.synnefo.org/docs/kamaki/latest/index.html>`_.
1528 caa6c07d Constantinos Venetsanopoulos
1529 f8cdf6ec Dionysis Grigoropoulos
To upload the file using kamaki, run:
1530 f8cdf6ec Dionysis Grigoropoulos
1531 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1532 dd6062f2 Christos Stavrakakis
1533 f8cdf6ec Dionysis Grigoropoulos
   # kamaki file upload debian_base-6.0-x86_64.diskdump pithos
1534 f8cdf6ec Dionysis Grigoropoulos
1535 caa6c07d Constantinos Venetsanopoulos
Once the Image is uploaded successfully, download the Image's metadata file
1536 d587329c Konstantinos Tompoulidis
from the official snf-image page. You will need it, for spawning a VM from
1537 d587329c Konstantinos Tompoulidis
Ganeti, in the next section.
1538 caa6c07d Constantinos Venetsanopoulos
1539 d587329c Konstantinos Tompoulidis
Of course, you can repeat the procedure to upload more Images, available from
1540 d587329c Konstantinos Tompoulidis
the `official snf-image page
1541 5beef175 Nikos Skalkotos
<http://www.synnefo.org/docs/snf-image/latest/usage.html#sample-images>`_.
1542 caa6c07d Constantinos Venetsanopoulos
1543 8a4cd31b Constantinos Venetsanopoulos
.. _ganeti-with-pithos-images:
1544 8a4cd31b Constantinos Venetsanopoulos
1545 e5d8df8c Constantinos Venetsanopoulos
Spawning a VM from a Pithos Image, using Ganeti
1546 e5d8df8c Constantinos Venetsanopoulos
-----------------------------------------------
1547 caa6c07d Constantinos Venetsanopoulos
1548 caa6c07d Constantinos Venetsanopoulos
Now, it is time to test our installation so far. So, we have Astakos and
1549 e5d8df8c Constantinos Venetsanopoulos
Pithos installed, we have a working Ganeti installation, the snf-image
1550 f8cdf6ec Dionysis Grigoropoulos
definition installed on all VM-capable nodes, a Debian Squeeze Image on
1551 dd6062f2 Christos Stavrakakis
Pithos and kamaki installed and configured. Make sure you also have the
1552 dd6062f2 Christos Stavrakakis
`metadata file <http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump.meta>`_
1553 f8cdf6ec Dionysis Grigoropoulos
for this image.
1554 f8cdf6ec Dionysis Grigoropoulos
1555 f8cdf6ec Dionysis Grigoropoulos
To spawn a VM from a Pithos file, we need to know:
1556 f8cdf6ec Dionysis Grigoropoulos
1557 f8cdf6ec Dionysis Grigoropoulos
    1) The hashmap of the file
1558 f8cdf6ec Dionysis Grigoropoulos
    2) The size of the file
1559 f8cdf6ec Dionysis Grigoropoulos
1560 f8cdf6ec Dionysis Grigoropoulos
If you uploaded the file with kamaki as described above, run:
1561 dd6062f2 Christos Stavrakakis
1562 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1563 f8cdf6ec Dionysis Grigoropoulos
1564 dd6062f2 Christos Stavrakakis
   # kamaki file info pithos:debian_base-6.0-x86_64.diskdump
1565 f8cdf6ec Dionysis Grigoropoulos
1566 dd6062f2 Christos Stavrakakis
else, replace ``pithos`` and ``debian_base-6.0-x86_64.diskdump`` with the
1567 f8cdf6ec Dionysis Grigoropoulos
container and filename you used, when uploading the file.
1568 f8cdf6ec Dionysis Grigoropoulos
1569 f8cdf6ec Dionysis Grigoropoulos
The hashmap is the field ``x-object-hash``, while the size of the file is the
1570 f8cdf6ec Dionysis Grigoropoulos
``content-length`` field, that ``kamaki file info`` command returns.
1571 caa6c07d Constantinos Venetsanopoulos
1572 caa6c07d Constantinos Venetsanopoulos
Run on the :ref:`GANETI-MASTER's <GANETI_NODES>` (node1) command line:
1573 caa6c07d Constantinos Venetsanopoulos
1574 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1575 caa6c07d Constantinos Venetsanopoulos
1576 cc16407a Dimitris Aragiorgis
   # gnt-instance add -o snf-image+default --os-parameters \
1577 f8cdf6ec Dionysis Grigoropoulos
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithosmap://<HashMap>/<Size>",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1578 cc16407a Dimitris Aragiorgis
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1579 caa6c07d Constantinos Venetsanopoulos
                      testvm1
1580 caa6c07d Constantinos Venetsanopoulos
1581 caa6c07d Constantinos Venetsanopoulos
In the above command:
1582 caa6c07d Constantinos Venetsanopoulos
1583 caa6c07d Constantinos Venetsanopoulos
 * ``img_passwd``: the arbitrary root password of your new instance
1584 caa6c07d Constantinos Venetsanopoulos
 * ``img_format``: set to ``diskdump`` to reflect the type of the uploaded Image
1585 e5d8df8c Constantinos Venetsanopoulos
 * ``img_id``: If you want to deploy an Image stored on Pithos (our case), this
1586 cbb596d4 Dionysis Grigoropoulos
   should have the format ``pithosmap://<HashMap>/<size>``:
1587 cbb596d4 Dionysis Grigoropoulos
1588 f8cdf6ec Dionysis Grigoropoulos
               * ``HashMap``: the map of the file
1589 dd6062f2 Christos Stavrakakis
               * ``size``: the size of the file, same size as reported in
1590 cbb596d4 Dionysis Grigoropoulos
                 ``ls -l filename``
1591 cbb596d4 Dionysis Grigoropoulos
1592 caa6c07d Constantinos Venetsanopoulos
 * ``img_properties``: taken from the metadata file. Used only the two mandatory
1593 caa6c07d Constantinos Venetsanopoulos
                       properties ``OSFAMILY`` and ``ROOT_PARTITION``. `Learn more
1594 5beef175 Nikos Skalkotos
                       <http://www.synnefo.org/docs/snf-image/latest/usage.html#image-properties>`_
1595 caa6c07d Constantinos Venetsanopoulos
1596 caa6c07d Constantinos Venetsanopoulos
If the ``gnt-instance add`` command returns successfully, then run:
1597 caa6c07d Constantinos Venetsanopoulos
1598 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1599 caa6c07d Constantinos Venetsanopoulos
1600 caa6c07d Constantinos Venetsanopoulos
   # gnt-instance info testvm1 | grep "console connection"
1601 caa6c07d Constantinos Venetsanopoulos
1602 caa6c07d Constantinos Venetsanopoulos
to find out where to connect using VNC. If you can connect successfully and can
1603 caa6c07d Constantinos Venetsanopoulos
login to your new instance using the root password ``my_vm_example_passw0rd``,
1604 caa6c07d Constantinos Venetsanopoulos
then everything works as expected and you have your new Debian Base VM up and
1605 caa6c07d Constantinos Venetsanopoulos
running.
1606 caa6c07d Constantinos Venetsanopoulos
1607 caa6c07d Constantinos Venetsanopoulos
If ``gnt-instance add`` fails, make sure that snf-image is correctly configured
1608 e5d8df8c Constantinos Venetsanopoulos
to access the Pithos database and the Pithos backend data (newer versions
1609 d587329c Konstantinos Tompoulidis
require UUID instead of a username). Another issue you may encounter is that in
1610 d587329c Konstantinos Tompoulidis
relatively slow setups, you may need to raise the default HELPER_*_TIMEOUTS in
1611 d587329c Konstantinos Tompoulidis
/etc/default/snf-image. Also, make sure you gave the correct ``img_id`` and
1612 d587329c Konstantinos Tompoulidis
``img_properties``. If ``gnt-instance add`` succeeds but you cannot connect,
1613 d587329c Konstantinos Tompoulidis
again find out what went wrong. Do *NOT* proceed to the next steps unless you
1614 d587329c Konstantinos Tompoulidis
are sure everything works till this point.
1615 caa6c07d Constantinos Venetsanopoulos
1616 e5d8df8c Constantinos Venetsanopoulos
If everything works, you have successfully connected Ganeti with Pithos. Let's
1617 2f6143c9 Constantinos Venetsanopoulos
move on to networking now.
1618 caa6c07d Constantinos Venetsanopoulos
1619 2f6143c9 Constantinos Venetsanopoulos
.. warning::
1620 e4404297 Christos Stavrakakis
1621 7a8df455 Constantinos Venetsanopoulos
    You can bypass the networking sections and go straight to
1622 2c85833e Constantinos Venetsanopoulos
    :ref:`Cyclades Ganeti tools <cyclades-gtools>`, if you do not want to setup
1623 2c85833e Constantinos Venetsanopoulos
    the Cyclades Network Service, but only the Cyclades Compute Service
1624 2c85833e Constantinos Venetsanopoulos
    (recommended for now).
1625 2f6143c9 Constantinos Venetsanopoulos
1626 cc16407a Dimitris Aragiorgis
Networking Setup Overview
1627 cc16407a Dimitris Aragiorgis
-------------------------
1628 2f6143c9 Constantinos Venetsanopoulos
1629 2f6143c9 Constantinos Venetsanopoulos
This part is deployment-specific and must be customized based on the specific
1630 8314e2fc Constantinos Venetsanopoulos
needs of the system administrator. Synnefo supports a lot of different
1631 8314e2fc Constantinos Venetsanopoulos
networking configurations in the backend (spanning from very simple to more
1632 8314e2fc Constantinos Venetsanopoulos
advanced), which are not in the scope of this guide.
1633 2f6143c9 Constantinos Venetsanopoulos
1634 8314e2fc Constantinos Venetsanopoulos
In this section, we'll describe the simplest scenario, which will enable the
1635 8314e2fc Constantinos Venetsanopoulos
VMs to have access to the public Internet and also access to arbitrary private
1636 8314e2fc Constantinos Venetsanopoulos
networks.
1637 8314e2fc Constantinos Venetsanopoulos
1638 8314e2fc Constantinos Venetsanopoulos
At the end of this section the networking setup on the two nodes will look like
1639 8314e2fc Constantinos Venetsanopoulos
this:
1640 8314e2fc Constantinos Venetsanopoulos
1641 8314e2fc Constantinos Venetsanopoulos
.. image:: images/install-guide-networks.png
1642 8314e2fc Constantinos Venetsanopoulos
   :width: 70%
1643 8314e2fc Constantinos Venetsanopoulos
   :target: _images/install-guide-networks.png
1644 2f6143c9 Constantinos Venetsanopoulos
1645 cc16407a Dimitris Aragiorgis
.. _snf-network:
1646 2f6143c9 Constantinos Venetsanopoulos
1647 cc16407a Dimitris Aragiorgis
snf-network
1648 cc16407a Dimitris Aragiorgis
~~~~~~~~~~~
1649 2f6143c9 Constantinos Venetsanopoulos
1650 f8cdf6ec Dionysis Grigoropoulos
snf-network is a set of custom scripts, that perform all the necessary actions,
1651 f8cdf6ec Dionysis Grigoropoulos
so that VMs have a working networking configuration.
1652 2f6143c9 Constantinos Venetsanopoulos
1653 cc16407a Dimitris Aragiorgis
Install snf-network on all Ganeti nodes:
1654 2f6143c9 Constantinos Venetsanopoulos
1655 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1656 2f6143c9 Constantinos Venetsanopoulos
1657 cc16407a Dimitris Aragiorgis
   # apt-get install snf-network
1658 2f6143c9 Constantinos Venetsanopoulos
1659 cc16407a Dimitris Aragiorgis
Then, in :file:`/etc/default/snf-network` set:
1660 2f6143c9 Constantinos Venetsanopoulos
1661 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1662 2f6143c9 Constantinos Venetsanopoulos
1663 cc16407a Dimitris Aragiorgis
   MAC_MASK=ff:ff:f0:00:00:00
1664 2f6143c9 Constantinos Venetsanopoulos
1665 cc16407a Dimitris Aragiorgis
.. _nfdhcpd:
1666 2f6143c9 Constantinos Venetsanopoulos
1667 cc16407a Dimitris Aragiorgis
nfdhcpd
1668 7a8df455 Constantinos Venetsanopoulos
~~~~~~~
1669 2f6143c9 Constantinos Venetsanopoulos
1670 dd6062f2 Christos Stavrakakis
nfdhcpd is an NFQUEUE based daemon, answering DHCP requests and running locally
1671 dd6062f2 Christos Stavrakakis
on every Ganeti node. Its leases file, gets automatically updated by
1672 f8cdf6ec Dionysis Grigoropoulos
snf-network and information provided by Ganeti.
1673 2f6143c9 Constantinos Venetsanopoulos
1674 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1675 2f6143c9 Constantinos Venetsanopoulos
1676 f8cdf6ec Dionysis Grigoropoulos
   # apt-get install python-nfqueue=0.4+physindev-1~wheezy
1677 2f6143c9 Constantinos Venetsanopoulos
   # apt-get install nfdhcpd
1678 2f6143c9 Constantinos Venetsanopoulos
1679 2f6143c9 Constantinos Venetsanopoulos
Edit ``/etc/nfdhcpd/nfdhcpd.conf`` to reflect your network configuration. At
1680 2f6143c9 Constantinos Venetsanopoulos
least, set the ``dhcp_queue`` variable to ``42`` and the ``nameservers``
1681 f8cdf6ec Dionysis Grigoropoulos
variable to your DNS IP/s (the one running dnsmasq for instance or you can use
1682 f8cdf6ec Dionysis Grigoropoulos
Google's DNS server ``8.8.8.8``). Restart the server on all nodes:
1683 2f6143c9 Constantinos Venetsanopoulos
1684 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1685 2f6143c9 Constantinos Venetsanopoulos
1686 2f6143c9 Constantinos Venetsanopoulos
   # /etc/init.d/nfdhcpd restart
1687 2f6143c9 Constantinos Venetsanopoulos
1688 dd6062f2 Christos Stavrakakis
In order for nfdhcpd to receive the VMs requests, we have to mangle all DHCP
1689 f8cdf6ec Dionysis Grigoropoulos
traffic coming from the corresponding interfaces. To accomplish that run:
1690 cc16407a Dimitris Aragiorgis
1691 cc16407a Dimitris Aragiorgis
.. code-block:: console
1692 cc16407a Dimitris Aragiorgis
1693 cc16407a Dimitris Aragiorgis
   # iptables -t mangle -A PREROUTING -p udp -m udp --dport 67 -j NFQUEUE --queue-num 42
1694 cc16407a Dimitris Aragiorgis
1695 f8cdf6ec Dionysis Grigoropoulos
and append it to your ``/etc/rc.local``.
1696 cc16407a Dimitris Aragiorgis
1697 cc16407a Dimitris Aragiorgis
You can check which clients are currently served by nfdhcpd by running:
1698 cc16407a Dimitris Aragiorgis
1699 cc16407a Dimitris Aragiorgis
.. code-block:: console
1700 cc16407a Dimitris Aragiorgis
1701 cc16407a Dimitris Aragiorgis
   # kill -SIGUSR1 `cat /var/run/nfdhcpd/nfdhcpd.pid`
1702 cc16407a Dimitris Aragiorgis
1703 cc16407a Dimitris Aragiorgis
When you run the above, then check ``/var/log/nfdhcpd/nfdhcpd.log``.
1704 cc16407a Dimitris Aragiorgis
1705 cc16407a Dimitris Aragiorgis
Public Network Setup
1706 cc16407a Dimitris Aragiorgis
--------------------
1707 cc16407a Dimitris Aragiorgis
1708 f8cdf6ec Dionysis Grigoropoulos
In the following section, we'll guide you through a very basic network setup.
1709 f8cdf6ec Dionysis Grigoropoulos
This assumes the following:
1710 dd6062f2 Christos Stavrakakis
1711 f8cdf6ec Dionysis Grigoropoulos
    * Node1 has access to the public network via eth0.
1712 f8cdf6ec Dionysis Grigoropoulos
    * Node1 will become a NAT server for the VMs.
1713 f8cdf6ec Dionysis Grigoropoulos
    * All nodes have ``br1/br2`` dedicated for the VMs' public/private traffic.
1714 f8cdf6ec Dionysis Grigoropoulos
    * VMs' public network is ``10.0.0.0/24`` with gateway ``10.0.0.1``.
1715 cc16407a Dimitris Aragiorgis
1716 f8cdf6ec Dionysis Grigoropoulos
Setting up the NAT server on node1
1717 f8cdf6ec Dionysis Grigoropoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1718 cc16407a Dimitris Aragiorgis
1719 f8cdf6ec Dionysis Grigoropoulos
To setup the NAT server on node1, run:
1720 2f6143c9 Constantinos Venetsanopoulos
1721 cc16407a Dimitris Aragiorgis
.. code-block:: console
1722 dd6062f2 Christos Stavrakakis
1723 f8cdf6ec Dionysis Grigoropoulos
   # ip addr add 10.0.0.1/24 dev br1
1724 f8cdf6ec Dionysis Grigoropoulos
   # iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
1725 f8cdf6ec Dionysis Grigoropoulos
   # echo 1 > /proc/sys/net/ipv4/ip_forward
1726 bc7e4f5f Stratos Psomadakis
1727 f8cdf6ec Dionysis Grigoropoulos
and append it to your ``/etc/rc.local``.
1728 dd6062f2 Christos Stavrakakis
1729 cc16407a Dimitris Aragiorgis
1730 f8cdf6ec Dionysis Grigoropoulos
Testing the Public Networks
1731 f8cdf6ec Dionysis Grigoropoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~
1732 cc16407a Dimitris Aragiorgis
1733 f8cdf6ec Dionysis Grigoropoulos
First add the network in Ganati:
1734 cc16407a Dimitris Aragiorgis
1735 cc16407a Dimitris Aragiorgis
.. code-block:: console
1736 cc16407a Dimitris Aragiorgis
1737 f8cdf6ec Dionysis Grigoropoulos
   # gnt-network add --network=10.0.0.0/24 --gateway=10.0.0.1 --tags=nfdhcpd test-net-public
1738 2f6143c9 Constantinos Venetsanopoulos
1739 f8cdf6ec Dionysis Grigoropoulos
Then, provide connectivity mode and link to the network:
1740 cc16407a Dimitris Aragiorgis
1741 cc16407a Dimitris Aragiorgis
.. code-block:: console
1742 cc16407a Dimitris Aragiorgis
1743 f8cdf6ec Dionysis Grigoropoulos
   # gnt-network connect test-net-public bridged br1
1744 2f6143c9 Constantinos Venetsanopoulos
1745 2f6143c9 Constantinos Venetsanopoulos
Now, it is time to test that the backend infrastracture is correctly setup for
1746 f8cdf6ec Dionysis Grigoropoulos
the Public Network. We will add a new VM, almost the same way we did it on the
1747 f8cdf6ec Dionysis Grigoropoulos
previous testing section. However, now we'll also add one NIC, configured to be
1748 f8cdf6ec Dionysis Grigoropoulos
managed from our previously defined network.
1749 f8cdf6ec Dionysis Grigoropoulos
1750 f8cdf6ec Dionysis Grigoropoulos
Fetch the Debian Old Base image locally (in all nodes), by running:
1751 f8cdf6ec Dionysis Grigoropoulos
1752 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1753 f8cdf6ec Dionysis Grigoropoulos
1754 f8cdf6ec Dionysis Grigoropoulos
   # wget http://cdn.synnefo.org/debian_base-6.0-x86_64.diskdump -O /var/lib/snf-image/debian_base-6.0-x86_64.diskdump
1755 f8cdf6ec Dionysis Grigoropoulos
1756 f8cdf6ec Dionysis Grigoropoulos
Also in all nodes, bring all ``br*`` interfaces up:
1757 2f6143c9 Constantinos Venetsanopoulos
1758 2f6143c9 Constantinos Venetsanopoulos
.. code-block:: console
1759 2f6143c9 Constantinos Venetsanopoulos
1760 f8cdf6ec Dionysis Grigoropoulos
   # ifconfig br1 up
1761 f8cdf6ec Dionysis Grigoropoulos
   # ifconfig br2 up
1762 f8cdf6ec Dionysis Grigoropoulos
1763 f8cdf6ec Dionysis Grigoropoulos
Finally, run on the GANETI-MASTER (node1):
1764 f8cdf6ec Dionysis Grigoropoulos
1765 f8cdf6ec Dionysis Grigoropoulos
.. code-block:: console
1766 dd6062f2 Christos Stavrakakis
1767 cc16407a Dimitris Aragiorgis
   # gnt-instance add -o snf-image+default --os-parameters \
1768 f8cdf6ec Dionysis Grigoropoulos
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id=debian_base-6.0-x86_64,img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1769 cc16407a Dimitris Aragiorgis
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1770 cc16407a Dimitris Aragiorgis
                      --net 0:ip=pool,network=test-net-public \
1771 2f6143c9 Constantinos Venetsanopoulos
                      testvm2
1772 2f6143c9 Constantinos Venetsanopoulos
1773 f8cdf6ec Dionysis Grigoropoulos
The following things should happen:
1774 2f6143c9 Constantinos Venetsanopoulos
1775 f8cdf6ec Dionysis Grigoropoulos
    * Ganeti creates a tap interface.
1776 f8cdf6ec Dionysis Grigoropoulos
    * snf-network bridges the tap interface to ``br1`` and updates nfdhcpd state.
1777 f8cdf6ec Dionysis Grigoropoulos
    * nfdhcpd serves 10.0.0.2 IP to the interface of ``testvm2``.
1778 2f6143c9 Constantinos Venetsanopoulos
1779 dd6062f2 Christos Stavrakakis
Now try to ping the outside world e.g., ``www.synnefo.org`` from inside the VM
1780 f8cdf6ec Dionysis Grigoropoulos
(connect to the VM using VNC as before).
1781 547c78f6 Constantinos Venetsanopoulos
1782 547c78f6 Constantinos Venetsanopoulos
Make sure everything works as expected, before proceeding with the Private
1783 2f6143c9 Constantinos Venetsanopoulos
Networks setup.
1784 2f6143c9 Constantinos Venetsanopoulos
1785 04c1254b Constantinos Venetsanopoulos
.. _private-networks-setup:
1786 04c1254b Constantinos Venetsanopoulos
1787 cc16407a Dimitris Aragiorgis
Private Networks Setup
1788 7a8df455 Constantinos Venetsanopoulos
----------------------
1789 2f6143c9 Constantinos Venetsanopoulos
1790 dd6062f2 Christos Stavrakakis
In this section, we'll describe a basic network configuration, that will provide
1791 dd6062f2 Christos Stavrakakis
isolated private networks to the end-users. All private network traffic, will
1792 dd6062f2 Christos Stavrakakis
pass through ``br1`` and isolation will be guaranteed with a specific set of
1793 f8cdf6ec Dionysis Grigoropoulos
``ebtables`` rules.
1794 547c78f6 Constantinos Venetsanopoulos
1795 2f6143c9 Constantinos Venetsanopoulos
Testing the Private Networks
1796 7a8df455 Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1797 caa6c07d Constantinos Venetsanopoulos
1798 dd6062f2 Christos Stavrakakis
We'll create two instances and connect them to the same Private Network. This
1799 f8cdf6ec Dionysis Grigoropoulos
means that the instances will have a second NIC connected to the ``br1``.
1800 547c78f6 Constantinos Venetsanopoulos
1801 547c78f6 Constantinos Venetsanopoulos
.. code-block:: console
1802 547c78f6 Constantinos Venetsanopoulos
1803 f8cdf6ec Dionysis Grigoropoulos
   # gnt-network add --network=192.168.1.0/24 --mac-prefix=aa:00:55 --tags=nfdhcpd,private-filtered test-net-prv-mac
1804 f8cdf6ec Dionysis Grigoropoulos
   # gnt-network connect test-net-prv-mac bridged br1
1805 cc16407a Dimitris Aragiorgis
1806 cc16407a Dimitris Aragiorgis
   # gnt-instance add -o snf-image+default --os-parameters \
1807 f8cdf6ec Dionysis Grigoropoulos
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id=debian_base-6.0-x86_64,img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1808 cc16407a Dimitris Aragiorgis
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1809 cc16407a Dimitris Aragiorgis
                      --net 0:ip=pool,network=test-net-public \
1810 cc16407a Dimitris Aragiorgis
                      --net 1:ip=pool,network=test-net-prv-mac \
1811 547c78f6 Constantinos Venetsanopoulos
                      testvm3
1812 547c78f6 Constantinos Venetsanopoulos
1813 cc16407a Dimitris Aragiorgis
   # gnt-instance add -o snf-image+default --os-parameters \
1814 f8cdf6ec Dionysis Grigoropoulos
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id=debian_base-6.0-x86_64,img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1815 cc16407a Dimitris Aragiorgis
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1816 cc16407a Dimitris Aragiorgis
                      --net 0:ip=pool,network=test-net-public \
1817 f8cdf6ec Dionysis Grigoropoulos
                      --net 1:ip=pool,network=test-net-prv-mac -n node2 \
1818 547c78f6 Constantinos Venetsanopoulos
                      testvm4
1819 547c78f6 Constantinos Venetsanopoulos
1820 f8cdf6ec Dionysis Grigoropoulos
Above, we create two instances with the first NIC connected to the internet and
1821 f8cdf6ec Dionysis Grigoropoulos
their second NIC connected to a MAC filtered private Network. Now, connect to the
1822 cc16407a Dimitris Aragiorgis
instances using VNC and make sure everything works as expected:
1823 547c78f6 Constantinos Venetsanopoulos
1824 cc16407a Dimitris Aragiorgis
 a) The instances have access to the public internet through their first eth
1825 f8cdf6ec Dionysis Grigoropoulos
    interface (``eth0``), which has been automatically assigned a "public" IP.
1826 547c78f6 Constantinos Venetsanopoulos
1827 f8cdf6ec Dionysis Grigoropoulos
 b) ``eth1`` will have mac prefix ``aa:00:55``
1828 547c78f6 Constantinos Venetsanopoulos
1829 f8cdf6ec Dionysis Grigoropoulos
 c) On testvm3  ping 192.168.1.2
1830 547c78f6 Constantinos Venetsanopoulos
1831 547c78f6 Constantinos Venetsanopoulos
If everything works as expected, then you have finished the Network Setup at the
1832 547c78f6 Constantinos Venetsanopoulos
backend for both types of Networks (Public & Private).
1833 547c78f6 Constantinos Venetsanopoulos
1834 2c85833e Constantinos Venetsanopoulos
.. _cyclades-gtools:
1835 2c85833e Constantinos Venetsanopoulos
1836 2c85833e Constantinos Venetsanopoulos
Cyclades Ganeti tools
1837 2c85833e Constantinos Venetsanopoulos
---------------------
1838 2c85833e Constantinos Venetsanopoulos
1839 2c85833e Constantinos Venetsanopoulos
In order for Ganeti to be connected with Cyclades later on, we need the
1840 2c85833e Constantinos Venetsanopoulos
`Cyclades Ganeti tools` available on all Ganeti nodes (node1 & node2 in our
1841 2c85833e Constantinos Venetsanopoulos
case). You can install them by running in both nodes:
1842 2c85833e Constantinos Venetsanopoulos
1843 2c85833e Constantinos Venetsanopoulos
.. code-block:: console
1844 2c85833e Constantinos Venetsanopoulos
1845 2c85833e Constantinos Venetsanopoulos
   # apt-get install snf-cyclades-gtools
1846 2c85833e Constantinos Venetsanopoulos
1847 2c85833e Constantinos Venetsanopoulos
This will install the following:
1848 2c85833e Constantinos Venetsanopoulos
1849 2c85833e Constantinos Venetsanopoulos
 * ``snf-ganeti-eventd`` (daemon to publish Ganeti related messages on RabbitMQ)
1850 2c85833e Constantinos Venetsanopoulos
 * ``snf-progress-monitor`` (used by ``snf-image`` to publish progress messages)
1851 2c85833e Constantinos Venetsanopoulos
1852 2c85833e Constantinos Venetsanopoulos
Configure ``snf-cyclades-gtools``
1853 2c85833e Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1854 2c85833e Constantinos Venetsanopoulos
1855 d587329c Konstantinos Tompoulidis
The package will install the ``/etc/synnefo/20-snf-cyclades-gtools-backend.conf``
1856 2c85833e Constantinos Venetsanopoulos
configuration file. At least we need to set the RabbitMQ endpoint for all tools
1857 2c85833e Constantinos Venetsanopoulos
that need it:
1858 2c85833e Constantinos Venetsanopoulos
1859 2c85833e Constantinos Venetsanopoulos
.. code-block:: console
1860 2c85833e Constantinos Venetsanopoulos
1861 d587329c Konstantinos Tompoulidis
  AMQP_HOSTS=["amqp://synnefo:example_rabbitmq_passw0rd@node1.example.com:5672"]
1862 2c85833e Constantinos Venetsanopoulos
1863 2c85833e Constantinos Venetsanopoulos
The above variables should reflect your :ref:`Message Queue setup
1864 2c85833e Constantinos Venetsanopoulos
<rabbitmq-setup>`. This file should be editted in all Ganeti nodes.
1865 2c85833e Constantinos Venetsanopoulos
1866 2c85833e Constantinos Venetsanopoulos
Connect ``snf-image`` with ``snf-progress-monitor``
1867 2c85833e Constantinos Venetsanopoulos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1868 2c85833e Constantinos Venetsanopoulos
1869 2c85833e Constantinos Venetsanopoulos
Finally, we need to configure ``snf-image`` to publish progress messages during
1870 2c85833e Constantinos Venetsanopoulos
the deployment of each Image. To do this, we edit ``/etc/default/snf-image`` and
1871 2c85833e Constantinos Venetsanopoulos
set the corresponding variable to ``snf-progress-monitor``:
1872 2c85833e Constantinos Venetsanopoulos
1873 2c85833e Constantinos Venetsanopoulos
.. code-block:: console
1874 2c85833e Constantinos Venetsanopoulos
1875 2c85833e Constantinos Venetsanopoulos
   PROGRESS_MONITOR="snf-progress-monitor"
1876 2c85833e Constantinos Venetsanopoulos
1877 2c85833e Constantinos Venetsanopoulos
This file should be editted in all Ganeti nodes.
1878 2c85833e Constantinos Venetsanopoulos
1879 7a8df455 Constantinos Venetsanopoulos
.. _rapi-user:
1880 7a8df455 Constantinos Venetsanopoulos
1881 caa6c07d Constantinos Venetsanopoulos
Synnefo RAPI user
1882 7a8df455 Constantinos Venetsanopoulos
-----------------
1883 caa6c07d Constantinos Venetsanopoulos
1884 7a8df455 Constantinos Venetsanopoulos
As a last step before installing Cyclades, create a new RAPI user that will
1885 caa6c07d Constantinos Venetsanopoulos
have ``write`` access. Cyclades will use this user to issue commands to Ganeti,
1886 2c85833e Constantinos Venetsanopoulos
so we will call the user ``cyclades`` with password ``example_rapi_passw0rd``.
1887 2c85833e Constantinos Venetsanopoulos
You can do this, by first running:
1888 2c85833e Constantinos Venetsanopoulos
1889 2c85833e Constantinos Venetsanopoulos
.. code-block:: console
1890 2c85833e Constantinos Venetsanopoulos
1891 2c85833e Constantinos Venetsanopoulos
   # echo -n 'cyclades:Ganeti Remote API:example_rapi_passw0rd' | openssl md5
1892 2c85833e Constantinos Venetsanopoulos
1893 2c85833e Constantinos Venetsanopoulos
and then putting the output in ``/var/lib/ganeti/rapi/users`` as follows:
1894 caa6c07d Constantinos Venetsanopoulos
1895 caa6c07d Constantinos Venetsanopoulos
.. code-block:: console
1896 caa6c07d Constantinos Venetsanopoulos
1897 2c85833e Constantinos Venetsanopoulos
   cyclades {HA1}55aec7050aa4e4b111ca43cb505a61a0 write
1898 caa6c07d Constantinos Venetsanopoulos
1899 caa6c07d Constantinos Venetsanopoulos
More about Ganeti's RAPI users `here.
1900 e332c1fd Stratos Psomadakis
<http://docs.ganeti.org/ganeti/2.6/html/rapi.html#introduction>`_
1901 bc055d09 Constantinos Venetsanopoulos
1902 73ebcd68 Constantinos Venetsanopoulos
You have now finished with all needed Prerequisites for Cyclades. Let's move on
1903 73ebcd68 Constantinos Venetsanopoulos
to the actual Cyclades installation.
1904 7a8df455 Constantinos Venetsanopoulos
1905 7a8df455 Constantinos Venetsanopoulos
1906 73ebcd68 Constantinos Venetsanopoulos
Installation of Cyclades on node1
1907 73ebcd68 Constantinos Venetsanopoulos
=================================
1908 7a8df455 Constantinos Venetsanopoulos
1909 7a8df455 Constantinos Venetsanopoulos
This section describes the installation of Cyclades. Cyclades is Synnefo's
1910 73ebcd68 Constantinos Venetsanopoulos
Compute service. The Image Service will get installed automatically along with
1911 73ebcd68 Constantinos Venetsanopoulos
Cyclades, because it is contained in the same Synnefo component.
1912 7a8df455 Constantinos Venetsanopoulos
1913 73ebcd68 Constantinos Venetsanopoulos
We will install Cyclades on node1. To do so, we install the corresponding
1914 73ebcd68 Constantinos Venetsanopoulos
package by running on node1:
1915 04c1254b Constantinos Venetsanopoulos
1916 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1917 04c1254b Constantinos Venetsanopoulos
1918 d587329c Konstantinos Tompoulidis
   # apt-get install snf-cyclades-app memcached python-memcache
1919 04c1254b Constantinos Venetsanopoulos
1920 73ebcd68 Constantinos Venetsanopoulos
If all packages install successfully, then Cyclades are installed and we
1921 73ebcd68 Constantinos Venetsanopoulos
proceed with their configuration.
1922 04c1254b Constantinos Venetsanopoulos
1923 d587329c Konstantinos Tompoulidis
Since version 0.13, Synnefo uses the VMAPI in order to prevent sensitive data
1924 d587329c Konstantinos Tompoulidis
needed by 'snf-image' to be stored in Ganeti configuration (e.g. VM password).
1925 d587329c Konstantinos Tompoulidis
This is achieved by storing all sensitive information to a CACHE backend and
1926 d587329c Konstantinos Tompoulidis
exporting it via VMAPI. The cache entries are invalidated after the first
1927 d587329c Konstantinos Tompoulidis
request. Synnefo uses `memcached <http://memcached.org/>`_ as a
1928 d587329c Konstantinos Tompoulidis
`Django <https://www.djangoproject.com/>`_ cache backend.
1929 bc055d09 Constantinos Venetsanopoulos
1930 73ebcd68 Constantinos Venetsanopoulos
Configuration of Cyclades
1931 73ebcd68 Constantinos Venetsanopoulos
=========================
1932 5b6feb88 Vangelis Koukis
1933 04c1254b Constantinos Venetsanopoulos
Conf files
1934 04c1254b Constantinos Venetsanopoulos
----------
1935 bc055d09 Constantinos Venetsanopoulos
1936 04c1254b Constantinos Venetsanopoulos
After installing Cyclades, a number of new configuration files will appear under
1937 d587329c Konstantinos Tompoulidis
``/etc/synnefo/`` prefixed with ``20-snf-cyclades-app-``. We will describe here
1938 d587329c Konstantinos Tompoulidis
only the minimal needed changes to result with a working system. In general,
1939 d587329c Konstantinos Tompoulidis
sane defaults have been chosen for the most of the options, to cover most of the
1940 04c1254b Constantinos Venetsanopoulos
common scenarios. However, if you want to tweak Cyclades feel free to do so,
1941 04c1254b Constantinos Venetsanopoulos
once you get familiar with the different options.
1942 bc055d09 Constantinos Venetsanopoulos
1943 04c1254b Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-cyclades-app-api.conf``:
1944 bc055d09 Constantinos Venetsanopoulos
1945 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1946 04c1254b Constantinos Venetsanopoulos
1947 e3ff6830 Georgios D. Tsoukalas
   CYCLADES_BASE_URL = 'https://node1.example.com/cyclades'
1948 cceaebef Ilias Tsitsimpis
   ASTAKOS_AUTH_URL = 'https://node1.example.com/astakos/identity/v2.0'
1949 04c1254b Constantinos Venetsanopoulos
1950 26498848 Giorgos Korfiatis
   CYCLADES_SERVICE_TOKEN = 'cyclades_service_token22w'
1951 a14f152f Giorgos Korfiatis
1952 cceaebef Ilias Tsitsimpis
The ``ASTAKOS_AUTH_URL`` denotes the Astakos endpoint for Cyclades,
1953 e3ff6830 Georgios D. Tsoukalas
which is used for all user management, including authentication.
1954 e3ff6830 Georgios D. Tsoukalas
Since our Astakos, Cyclades, and Pithos installations belong together,
1955 cceaebef Ilias Tsitsimpis
they should all have identical ``ASTAKOS_AUTH_URL`` setting
1956 e3ff6830 Georgios D. Tsoukalas
(see also, :ref:`previously <conf-pithos>`).
1957 d587329c Konstantinos Tompoulidis
1958 0d87ef78 Giorgos Korfiatis
The ``CYCLADES_BASE_URL`` setting must point to the top-level Cyclades URL.
1959 0d87ef78 Giorgos Korfiatis
Appending an extra path (``/cyclades`` here) is recommended in order to
1960 0d87ef78 Giorgos Korfiatis
distinguish components, if more than one are installed on the same machine.
1961 0d87ef78 Giorgos Korfiatis
1962 f8cdf6ec Dionysis Grigoropoulos
The ``CYCLADES_SERVICE_TOKEN`` is the token used for authentication with Astakos.
1963 a14f152f Giorgos Korfiatis
It can be retrieved by running on the Astakos node (node1 in our case):
1964 a14f152f Giorgos Korfiatis
1965 a14f152f Giorgos Korfiatis
.. code-block:: console
1966 a14f152f Giorgos Korfiatis
1967 a14f152f Giorgos Korfiatis
   # snf-manage component-list
1968 a14f152f Giorgos Korfiatis
1969 a14f152f Giorgos Korfiatis
The token has been generated automatically during the :ref:`Cyclades service
1970 a14f152f Giorgos Korfiatis
registration <services-reg>`.
1971 a14f152f Giorgos Korfiatis
1972 04c1254b Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-cyclades-app-cloudbar.conf``:
1973 04c1254b Constantinos Venetsanopoulos
1974 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1975 04c1254b Constantinos Venetsanopoulos
1976 bc3a7b5a Constantinos Venetsanopoulos
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
1977 a14f152f Giorgos Korfiatis
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/astakos/ui/get_services'
1978 0c068fc6 marioskogias
   CLOUDBAR_MENU_URL = 'https://node1.example.com/astakos/ui/get_menu'
1979 04c1254b Constantinos Venetsanopoulos
1980 04c1254b Constantinos Venetsanopoulos
``CLOUDBAR_LOCATION`` tells the client where to find the Astakos common
1981 04c1254b Constantinos Venetsanopoulos
cloudbar. The ``CLOUDBAR_SERVICES_URL`` and ``CLOUDBAR_MENU_URL`` options are
1982 04c1254b Constantinos Venetsanopoulos
used by the Cyclades Web UI to get from Astakos all the information needed to
1983 04c1254b Constantinos Venetsanopoulos
fill its own cloudbar. So, we put our Astakos deployment urls there. All the
1984 04c1254b Constantinos Venetsanopoulos
above should have the same values we put in the corresponding variables in
1985 04c1254b Constantinos Venetsanopoulos
``/etc/synnefo/20-snf-pithos-webclient-cloudbar.conf`` on the previous
1986 04c1254b Constantinos Venetsanopoulos
:ref:`Pithos configuration <conf-pithos>` section.
1987 04c1254b Constantinos Venetsanopoulos
1988 04c1254b Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-cyclades-app-plankton.conf``:
1989 04c1254b Constantinos Venetsanopoulos
1990 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
1991 04c1254b Constantinos Venetsanopoulos
1992 04c1254b Constantinos Venetsanopoulos
   BACKEND_DB_CONNECTION = 'postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos'
1993 04c1254b Constantinos Venetsanopoulos
   BACKEND_BLOCK_PATH = '/srv/pithos/data/'
1994 04c1254b Constantinos Venetsanopoulos
1995 73ebcd68 Constantinos Venetsanopoulos
In this file we configure the Image Service. ``BACKEND_DB_CONNECTION``
1996 e5d8df8c Constantinos Venetsanopoulos
denotes the Pithos database (where the Image files are stored). So we set that
1997 e5d8df8c Constantinos Venetsanopoulos
to point to our Pithos database. ``BACKEND_BLOCK_PATH`` denotes the actual
1998 e5d8df8c Constantinos Venetsanopoulos
Pithos data location.
1999 04c1254b Constantinos Venetsanopoulos
2000 04c1254b Constantinos Venetsanopoulos
Edit ``/etc/synnefo/20-snf-cyclades-app-queues.conf``:
2001 04c1254b Constantinos Venetsanopoulos
2002 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
2003 04c1254b Constantinos Venetsanopoulos
2004 ad003186 Constantinos Venetsanopoulos
   AMQP_HOSTS=["amqp://synnefo:example_rabbitmq_passw0rd@node1.example.com:5672"]
2005 04c1254b Constantinos Venetsanopoulos
2006 04c1254b Constantinos Venetsanopoulos
The above settings denote the Message Queue. Those settings should have the same
2007 52188a27 Kostis Fardelas
values as in ``/etc/synnefo/20-snf-cyclades-gtools-backend.conf`` file, and
2008 04c1254b Constantinos Venetsanopoulos
reflect our :ref:`Message Queue setup <rabbitmq-setup>`.
2009 04c1254b Constantinos Venetsanopoulos
2010 d587329c Konstantinos Tompoulidis
Edit ``/etc/synnefo/20-snf-cyclades-app-vmapi.conf``:
2011 d587329c Konstantinos Tompoulidis
2012 d587329c Konstantinos Tompoulidis
.. code-block:: console
2013 d587329c Konstantinos Tompoulidis
2014 d587329c Konstantinos Tompoulidis
   VMAPI_CACHE_BACKEND = "memcached://127.0.0.1:11211/?timeout=3600"
2015 d587329c Konstantinos Tompoulidis
2016 b5a93045 Stratos Psomadakis
Add a vncauthproxy user:
2017 053d0dfc Constantinos Venetsanopoulos
2018 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
2019 053d0dfc Constantinos Venetsanopoulos
2020 b5a93045 Stratos Psomadakis
    # vncauthproxy-passwd /var/lib/vncauthproxy/users synnefo
2021 b5a93045 Stratos Psomadakis
    # /etc/init.d/vncauthproxy restart
2022 b5a93045 Stratos Psomadakis
2023 b5a93045 Stratos Psomadakis
Configure the vncauthproxy settings in
2024 0fcb1b3e Ilias Tsitsimpis
``/etc/synnefo/20-snf-cyclades-app-api.conf``:
2025 b5a93045 Stratos Psomadakis
2026 b5a93045 Stratos Psomadakis
.. code-block:: console
2027 b5a93045 Stratos Psomadakis
2028 b5a93045 Stratos Psomadakis
    CYCLADES_VNCAUTHPROXY_OPTS = {
2029 b5a93045 Stratos Psomadakis
        'auth_user': 'synnefo',
2030 b5a93045 Stratos Psomadakis
        'auth_password': 'secret_password',
2031 b5a93045 Stratos Psomadakis
        'server_address': '127.0.0.1',
2032 b5a93045 Stratos Psomadakis
        'server_port': 24999,
2033 b5a93045 Stratos Psomadakis
        'enable_ssl': False,
2034 b5a93045 Stratos Psomadakis
        'ca_cert': None,
2035 b5a93045 Stratos Psomadakis
        'strict': False,
2036 b5a93045 Stratos Psomadakis
    }
2037 b5a93045 Stratos Psomadakis
2038 b5a93045 Stratos Psomadakis
Depending on your snf-vncauthproxy setup, you might want to tweak the above
2039 b5a93045 Stratos Psomadakis
settings. Check the `documentation
2040 b5a93045 Stratos Psomadakis
<http://www.synnefo.org/docs/snf-vncauthproxy/latest/index.html>`_ of
2041 b5a93045 Stratos Psomadakis
snf-vncauthproxy for more information.
2042 053d0dfc Constantinos Venetsanopoulos
2043 73ebcd68 Constantinos Venetsanopoulos
We have now finished with the basic Cyclades configuration.
2044 04c1254b Constantinos Venetsanopoulos
2045 04c1254b Constantinos Venetsanopoulos
Database Initialization
2046 04c1254b Constantinos Venetsanopoulos
-----------------------
2047 04c1254b Constantinos Venetsanopoulos
2048 04c1254b Constantinos Venetsanopoulos
Once Cyclades is configured, we sync the database:
2049 bc055d09 Constantinos Venetsanopoulos
2050 bc055d09 Constantinos Venetsanopoulos
.. code-block:: console
2051 bc055d09 Constantinos Venetsanopoulos
2052 bc055d09 Constantinos Venetsanopoulos
   $ snf-manage syncdb
2053 bc055d09 Constantinos Venetsanopoulos
   $ snf-manage migrate
2054 bc055d09 Constantinos Venetsanopoulos
2055 04c1254b Constantinos Venetsanopoulos
and load the initial server flavors:
2056 bc055d09 Constantinos Venetsanopoulos
2057 bc055d09 Constantinos Venetsanopoulos
.. code-block:: console
2058 bc055d09 Constantinos Venetsanopoulos
2059 bc055d09 Constantinos Venetsanopoulos
   $ snf-manage loaddata flavors
2060 bc055d09 Constantinos Venetsanopoulos
2061 04c1254b Constantinos Venetsanopoulos
If everything returns successfully, our database is ready.
2062 04c1254b Constantinos Venetsanopoulos
2063 053d0dfc Constantinos Venetsanopoulos
Add the Ganeti backend
2064 053d0dfc Constantinos Venetsanopoulos
----------------------
2065 053d0dfc Constantinos Venetsanopoulos
2066 7a3439cf Constantinos Venetsanopoulos
In our installation we assume that we only have one Ganeti cluster, the one we
2067 b2764de1 Dimitris Aragiorgis
setup earlier.  At this point you have to add this backend (Ganeti cluster) to
2068 f8cdf6ec Dionysis Grigoropoulos
Cyclades assuming that you have setup the :ref:`Rapi User <rapi-user>`
2069 b2764de1 Dimitris Aragiorgis
correctly.
2070 b2764de1 Dimitris Aragiorgis
2071 b2764de1 Dimitris Aragiorgis
.. code-block:: console
2072 b2764de1 Dimitris Aragiorgis
2073 69aa7f21 Ilias Tsitsimpis
   $ snf-manage backend-add --clustername=ganeti.node1.example.com --user=cyclades --pass=example_rapi_passw0rd
2074 b2764de1 Dimitris Aragiorgis
2075 b2764de1 Dimitris Aragiorgis
You can see everything has been setup correctly by running:
2076 053d0dfc Constantinos Venetsanopoulos
2077 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
2078 053d0dfc Constantinos Venetsanopoulos
2079 053d0dfc Constantinos Venetsanopoulos
   $ snf-manage backend-list
2080 053d0dfc Constantinos Venetsanopoulos
2081 d587329c Konstantinos Tompoulidis
Enable the new backend by running:
2082 d587329c Konstantinos Tompoulidis
2083 d587329c Konstantinos Tompoulidis
.. code-block::
2084 d587329c Konstantinos Tompoulidis
2085 d587329c Konstantinos Tompoulidis
   $ snf-manage backend-modify --drained False 1
2086 d587329c Konstantinos Tompoulidis
2087 d587329c Konstantinos Tompoulidis
.. warning:: Since version 0.13, the backend is set to "drained" by default.
2088 d587329c Konstantinos Tompoulidis
    This means that you cannot add VMs to it. The reason for this is that the
2089 d587329c Konstantinos Tompoulidis
    nodes should be unavailable to Synnefo until the Administrator explicitly
2090 d587329c Konstantinos Tompoulidis
    releases them. To change this setting, use ``snf-manage backend-modify
2091 d587329c Konstantinos Tompoulidis
    --drained False <backend-id>``.
2092 d587329c Konstantinos Tompoulidis
2093 7a3439cf Constantinos Venetsanopoulos
If something is not set correctly, you can modify the backend with the
2094 7a3439cf Constantinos Venetsanopoulos
``snf-manage backend-modify`` command. If something has gone wrong, you could
2095 7a3439cf Constantinos Venetsanopoulos
modify the backend to reflect the Ganeti installation by running:
2096 053d0dfc Constantinos Venetsanopoulos
2097 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
2098 053d0dfc Constantinos Venetsanopoulos
2099 053d0dfc Constantinos Venetsanopoulos
   $ snf-manage backend-modify --clustername "ganeti.node1.example.com"
2100 7a3439cf Constantinos Venetsanopoulos
                               --user=cyclades
2101 7a3439cf Constantinos Venetsanopoulos
                               --pass=example_rapi_passw0rd
2102 053d0dfc Constantinos Venetsanopoulos
                               1
2103 053d0dfc Constantinos Venetsanopoulos
2104 053d0dfc Constantinos Venetsanopoulos
``clustername`` denotes the Ganeti-cluster's name. We provide the corresponding
2105 053d0dfc Constantinos Venetsanopoulos
domain that resolves to the master IP, than the IP itself, to ensure Cyclades
2106 053d0dfc Constantinos Venetsanopoulos
can talk to Ganeti even after a Ganeti master-failover.
2107 053d0dfc Constantinos Venetsanopoulos
2108 7a3439cf Constantinos Venetsanopoulos
``user`` and ``pass`` denote the RAPI user's username and the RAPI user's
2109 b2764de1 Dimitris Aragiorgis
password.  Once we setup the first backend to point at our Ganeti cluster, we
2110 b2764de1 Dimitris Aragiorgis
update the Cyclades backends status by running:
2111 053d0dfc Constantinos Venetsanopoulos
2112 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
2113 053d0dfc Constantinos Venetsanopoulos
2114 053d0dfc Constantinos Venetsanopoulos
   $ snf-manage backend-update-status
2115 053d0dfc Constantinos Venetsanopoulos
2116 7a3439cf Constantinos Venetsanopoulos
Cyclades can manage multiple Ganeti backends, but for the purpose of this
2117 7a3439cf Constantinos Venetsanopoulos
guide,we won't get into more detail regarding mulitple backends. If you want to
2118 7a3439cf Constantinos Venetsanopoulos
learn more please see /*TODO*/.
2119 7a3439cf Constantinos Venetsanopoulos
2120 7a3439cf Constantinos Venetsanopoulos
Add a Public Network
2121 053d0dfc Constantinos Venetsanopoulos
----------------------
2122 053d0dfc Constantinos Venetsanopoulos
2123 7a3439cf Constantinos Venetsanopoulos
Cyclades supports different Public Networks on different Ganeti backends.
2124 cc16407a Dimitris Aragiorgis
After connecting Cyclades with our Ganeti cluster, we need to setup a Public
2125 cc16407a Dimitris Aragiorgis
Network for this Ganeti backend (`id = 1`). The basic setup is to bridge every
2126 dd6062f2 Christos Stavrakakis
created NIC on a bridge.
2127 053d0dfc Constantinos Venetsanopoulos
2128 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
2129 053d0dfc Constantinos Venetsanopoulos
2130 f8cdf6ec Dionysis Grigoropoulos
   $ snf-manage network-create --subnet=10.0.0.0/24 \
2131 f8cdf6ec Dionysis Grigoropoulos
                               --gateway=10.0.0.1 \
2132 850586cb Christos Stavrakakis
                               --public --dhcp=True --flavor=CUSTOM \
2133 f8cdf6ec Dionysis Grigoropoulos
                               --link=br1 --mode=bridged \
2134 69aa7f21 Ilias Tsitsimpis
                               --name=public_network \
2135 7a3439cf Constantinos Venetsanopoulos
                               --backend-id=1
2136 053d0dfc Constantinos Venetsanopoulos
2137 053d0dfc Constantinos Venetsanopoulos
This will create the Public Network on both Cyclades and the Ganeti backend. To
2138 053d0dfc Constantinos Venetsanopoulos
make sure everything was setup correctly, also run:
2139 053d0dfc Constantinos Venetsanopoulos
2140 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
2141 053d0dfc Constantinos Venetsanopoulos
2142 f8cdf6ec Dionysis Grigoropoulos
   # snf-manage reconcile-networks
2143 f8cdf6ec Dionysis Grigoropoulos
2144 f8cdf6ec Dionysis Grigoropoulos
You can use ``snf-manage reconcile-networks --fix-all`` to fix any
2145 f8cdf6ec Dionysis Grigoropoulos
inconsistencies that may have arisen.
2146 053d0dfc Constantinos Venetsanopoulos
2147 053d0dfc Constantinos Venetsanopoulos
You can see all available networks by running:
2148 053d0dfc Constantinos Venetsanopoulos
2149 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
2150 053d0dfc Constantinos Venetsanopoulos
2151 f8cdf6ec Dionysis Grigoropoulos
   # snf-manage network-list
2152 053d0dfc Constantinos Venetsanopoulos
2153 053d0dfc Constantinos Venetsanopoulos
and inspect each network's state by running:
2154 053d0dfc Constantinos Venetsanopoulos
2155 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
2156 053d0dfc Constantinos Venetsanopoulos
2157 f8cdf6ec Dionysis Grigoropoulos
   # snf-manage network-inspect <net_id>
2158 053d0dfc Constantinos Venetsanopoulos
2159 053d0dfc Constantinos Venetsanopoulos
Finally, you can see the networks from the Ganeti perspective by running on the
2160 053d0dfc Constantinos Venetsanopoulos
Ganeti MASTER:
2161 053d0dfc Constantinos Venetsanopoulos
2162 053d0dfc Constantinos Venetsanopoulos
.. code-block:: console
2163 053d0dfc Constantinos Venetsanopoulos
2164 f8cdf6ec Dionysis Grigoropoulos
   # gnt-network list
2165 f8cdf6ec Dionysis Grigoropoulos
   # gnt-network info <network_name>
2166 053d0dfc Constantinos Venetsanopoulos
2167 19425707 Christos Stavrakakis
Create pools for Private Networks
2168 d587329c Konstantinos Tompoulidis
---------------------------------
2169 19425707 Christos Stavrakakis
2170 19425707 Christos Stavrakakis
To prevent duplicate assignment of resources to different private networks,
2171 19425707 Christos Stavrakakis
Cyclades supports two types of pools:
2172 19425707 Christos Stavrakakis
2173 19425707 Christos Stavrakakis
 - MAC prefix Pool
2174 19425707 Christos Stavrakakis
 - Bridge Pool
2175 19425707 Christos Stavrakakis
2176 19425707 Christos Stavrakakis
As long as those resourses have been provisioned, admin has to define two
2177 19425707 Christos Stavrakakis
these pools in Synnefo:
2178 19425707 Christos Stavrakakis
2179 19425707 Christos Stavrakakis
2180 19425707 Christos Stavrakakis
.. code-block:: console
2181 19425707 Christos Stavrakakis
2182 f8cdf6ec Dionysis Grigoropoulos
   # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
2183 19425707 Christos Stavrakakis
2184 f8cdf6ec Dionysis Grigoropoulos
Also, change the Synnefo setting in :file:`/etc/synnefo/20-snf-cyclades-app-api.conf`:
2185 19425707 Christos Stavrakakis
2186 19425707 Christos Stavrakakis
.. code-block:: console
2187 19425707 Christos Stavrakakis
2188 f8cdf6ec Dionysis Grigoropoulos
   DEFAULT_MAC_FILTERED_BRIDGE = 'br2'
2189 19425707 Christos Stavrakakis
2190 04c1254b Constantinos Venetsanopoulos
Servers restart
2191 04c1254b Constantinos Venetsanopoulos
---------------
2192 04c1254b Constantinos Venetsanopoulos
2193 053d0dfc Constantinos Venetsanopoulos
Restart gunicorn on node1:
2194 04c1254b Constantinos Venetsanopoulos
2195 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
2196 bc055d09 Constantinos Venetsanopoulos
2197 04c1254b Constantinos Venetsanopoulos
   # /etc/init.d/gunicorn restart
2198 bc055d09 Constantinos Venetsanopoulos
2199 04c1254b Constantinos Venetsanopoulos
Now let's do the final connections of Cyclades with Ganeti.
2200 bc055d09 Constantinos Venetsanopoulos
2201 04c1254b Constantinos Venetsanopoulos
``snf-dispatcher`` initialization
2202 04c1254b Constantinos Venetsanopoulos
---------------------------------
2203 04c1254b Constantinos Venetsanopoulos
2204 04c1254b Constantinos Venetsanopoulos
``snf-dispatcher`` dispatches all messages published to the Message Queue and
2205 04c1254b Constantinos Venetsanopoulos
manages the Cyclades database accordingly. It also initializes all exchanges. By
2206 04c1254b Constantinos Venetsanopoulos
default it is not enabled during installation of Cyclades, so let's enable it in
2207 04c1254b Constantinos Venetsanopoulos
its configuration file ``/etc/default/snf-dispatcher``:
2208 04c1254b Constantinos Venetsanopoulos
2209 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
2210 04c1254b Constantinos Venetsanopoulos
2211 04c1254b Constantinos Venetsanopoulos
   SNF_DSPTCH_ENABLE=true
2212 04c1254b Constantinos Venetsanopoulos
2213 04c1254b Constantinos Venetsanopoulos
and start the daemon:
2214 04c1254b Constantinos Venetsanopoulos
2215 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
2216 04c1254b Constantinos Venetsanopoulos
2217 04c1254b Constantinos Venetsanopoulos
   # /etc/init.d/snf-dispatcher start
2218 04c1254b Constantinos Venetsanopoulos
2219 04c1254b Constantinos Venetsanopoulos
You can see that everything works correctly by tailing its log file
2220 04c1254b Constantinos Venetsanopoulos
``/var/log/synnefo/dispatcher.log``.
2221 04c1254b Constantinos Venetsanopoulos
2222 04c1254b Constantinos Venetsanopoulos
``snf-ganeti-eventd`` on GANETI MASTER
2223 04c1254b Constantinos Venetsanopoulos
--------------------------------------
2224 04c1254b Constantinos Venetsanopoulos
2225 04c1254b Constantinos Venetsanopoulos
The last step of the Cyclades setup is enabling the ``snf-ganeti-eventd``
2226 04c1254b Constantinos Venetsanopoulos
daemon (part of the :ref:`Cyclades Ganeti tools <cyclades-gtools>` package).
2227 04c1254b Constantinos Venetsanopoulos
The daemon is already installed on the GANETI MASTER (node1 in our case).
2228 04c1254b Constantinos Venetsanopoulos
``snf-ganeti-eventd`` is disabled by default during the ``snf-cyclades-gtools``
2229 04c1254b Constantinos Venetsanopoulos
installation, so we enable it in its configuration file
2230 04c1254b Constantinos Venetsanopoulos
``/etc/default/snf-ganeti-eventd``:
2231 04c1254b Constantinos Venetsanopoulos
2232 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
2233 04c1254b Constantinos Venetsanopoulos
2234 04c1254b Constantinos Venetsanopoulos
   SNF_EVENTD_ENABLE=true
2235 04c1254b Constantinos Venetsanopoulos
2236 04c1254b Constantinos Venetsanopoulos
and start the daemon:
2237 04c1254b Constantinos Venetsanopoulos
2238 04c1254b Constantinos Venetsanopoulos
.. code-block:: console
2239 bc055d09 Constantinos Venetsanopoulos
2240 04c1254b Constantinos Venetsanopoulos
   # /etc/init.d/snf-ganeti-eventd start
2241 bc055d09 Constantinos Venetsanopoulos
2242 04c1254b Constantinos Venetsanopoulos
.. warning:: Make sure you start ``snf-ganeti-eventd`` *ONLY* on GANETI MASTER
2243 bc055d09 Constantinos Venetsanopoulos
2244 b446c082 Giorgos Korfiatis
Apply Quota
2245 b446c082 Giorgos Korfiatis
-----------
2246 b446c082 Giorgos Korfiatis
2247 b446c082 Giorgos Korfiatis
The following commands will check and fix the integrity of user quota.
2248 b446c082 Giorgos Korfiatis
In a freshly installed system, these commands have no effect and can be
2249 b446c082 Giorgos Korfiatis
skipped.
2250 d587329c Konstantinos Tompoulidis
2251 d587329c Konstantinos Tompoulidis
.. code-block:: console
2252 d587329c Konstantinos Tompoulidis
2253 075b91de Giorgos Korfiatis
   node1 # snf-manage quota --sync
2254 b446c082 Giorgos Korfiatis
   node1 # snf-manage reconcile-resources-astakos --fix
2255 b446c082 Giorgos Korfiatis
   node2 # snf-manage reconcile-resources-pithos --fix
2256 62c86226 Christos Stavrakakis
   node1 # snf-manage reconcile-resources-cyclades --fix
2257 d587329c Konstantinos Tompoulidis
2258 62c0a9e1 Stratos Psomadakis
VM stats configuration
2259 62c0a9e1 Stratos Psomadakis
----------------------
2260 62c0a9e1 Stratos Psomadakis
2261 62c0a9e1 Stratos Psomadakis
Please refer to the documentation in the :ref:`admin guide <admin-guide-stats>`
2262 62c0a9e1 Stratos Psomadakis
for deploying and configuring snf-stats-app and collectd.
2263 62c0a9e1 Stratos Psomadakis
2264 0c068fc6 marioskogias
2265 04c1254b Constantinos Venetsanopoulos
If all the above return successfully, then you have finished with the Cyclades
2266 73ebcd68 Constantinos Venetsanopoulos
installation and setup.
2267 d587329c Konstantinos Tompoulidis
2268 d587329c Konstantinos Tompoulidis
Let's test our installation now.
2269 bc055d09 Constantinos Venetsanopoulos
2270 5b6feb88 Vangelis Koukis
2271 73ebcd68 Constantinos Venetsanopoulos
Testing of Cyclades
2272 73ebcd68 Constantinos Venetsanopoulos
===================
2273 5b6feb88 Vangelis Koukis
2274 8a4cd31b Constantinos Venetsanopoulos
Cyclades Web UI
2275 8a4cd31b Constantinos Venetsanopoulos
---------------
2276 8a4cd31b Constantinos Venetsanopoulos
2277 8a4cd31b Constantinos Venetsanopoulos
First of all we need to test that our Cyclades Web UI works correctly. Open your
2278 f8cdf6ec Dionysis Grigoropoulos
browser and go to the Astakos home page. Login and then click 'Cyclades' on the
2279 8a4cd31b Constantinos Venetsanopoulos
top cloud bar. This should redirect you to:
2280 8a4cd31b Constantinos Venetsanopoulos
2281 a14f152f Giorgos Korfiatis
 `http://node1.example.com/cyclades/ui/`
2282 8a4cd31b Constantinos Venetsanopoulos
2283 8a4cd31b Constantinos Venetsanopoulos
and the Cyclades home page should appear. If not, please go back and find what
2284 8a4cd31b Constantinos Venetsanopoulos
went wrong. Do not proceed if you don't see the Cyclades home page.
2285 8a4cd31b Constantinos Venetsanopoulos
2286 8a4cd31b Constantinos Venetsanopoulos
If the Cyclades home page appears, click on the orange button 'New machine'. The
2287 8a4cd31b Constantinos Venetsanopoulos
first step of the 'New machine wizard' will appear. This step shows all the
2288 8a4cd31b Constantinos Venetsanopoulos
available Images from which you can spawn new VMs. The list should be currently
2289 8a4cd31b Constantinos Venetsanopoulos
empty, as we haven't registered any Images yet. Close the wizard and browse the
2290 8a4cd31b Constantinos Venetsanopoulos
interface (not many things to see yet). If everything seems to work, let's
2291 8a4cd31b Constantinos Venetsanopoulos
register our first Image file.
2292 8a4cd31b Constantinos Venetsanopoulos
2293 8a4cd31b Constantinos Venetsanopoulos
Cyclades Images
2294 8a4cd31b Constantinos Venetsanopoulos
---------------
2295 8a4cd31b Constantinos Venetsanopoulos
2296 e5d8df8c Constantinos Venetsanopoulos
To test our Cyclades installation, we will use an Image stored on Pithos to
2297 73ebcd68 Constantinos Venetsanopoulos
spawn a new VM from the Cyclades interface. We will describe all steps, even
2298 e5d8df8c Constantinos Venetsanopoulos
though you may already have uploaded an Image on Pithos from a :ref:`previous
2299 73ebcd68 Constantinos Venetsanopoulos
<snf-image-images>` section:
2300 8a4cd31b Constantinos Venetsanopoulos
2301 e5d8df8c Constantinos Venetsanopoulos
 * Upload an Image file to Pithos
2302 73ebcd68 Constantinos Venetsanopoulos
 * Register that Image file to Cyclades
2303 8a4cd31b Constantinos Venetsanopoulos
 * Spawn a new VM from that Image from the Cyclades Web UI
2304 8a4cd31b Constantinos Venetsanopoulos
2305 34e79416 Constantinos Venetsanopoulos
We will use the `kamaki <http://www.synnefo.org/docs/kamaki/latest/index.html>`_
2306 8a4cd31b Constantinos Venetsanopoulos
command line client to do the uploading and registering of the Image.
2307 8a4cd31b Constantinos Venetsanopoulos
2308 d3cf0b89 Stavros Sachtouris
Installation of `kamaki`
2309 d3cf0b89 Stavros Sachtouris
~~~~~~~~~~~~~~~~~~~~~~~~
2310 d3cf0b89 Stavros Sachtouris
2311 d3cf0b89 Stavros Sachtouris
You can install `kamaki` anywhere you like, since it is a standalone client of
2312 d3cf0b89 Stavros Sachtouris
the APIs and talks to the installation over `http`. For the purpose of this
2313 d3cf0b89 Stavros Sachtouris
guide we will assume that we have downloaded the `Debian Squeeze Base Image
2314 d3cf0b89 Stavros Sachtouris
<https://pithos.okeanos.grnet.gr/public/9epgb>`_ and stored it under node1's
2315 d3cf0b89 Stavros Sachtouris
``/srv/images`` directory. For that reason we will install `kamaki` on node1,
2316 d3cf0b89 Stavros Sachtouris
too. We do this by running:
2317 d3cf0b89 Stavros Sachtouris
2318 d3cf0b89 Stavros Sachtouris
.. code-block:: console
2319 d3cf0b89 Stavros Sachtouris
2320 d3cf0b89 Stavros Sachtouris
   # apt-get install kamaki
2321 d3cf0b89 Stavros Sachtouris
2322 d3cf0b89 Stavros Sachtouris
Configuration of kamaki
2323 d3cf0b89 Stavros Sachtouris
~~~~~~~~~~~~~~~~~~~~~~~
2324 d3cf0b89 Stavros Sachtouris
2325 d3cf0b89 Stavros Sachtouris
Now we need to setup kamaki, by adding the appropriate URLs and tokens of our
2326 d3cf0b89 Stavros Sachtouris
installation. We do this by running:
2327 d3cf0b89 Stavros Sachtouris
2328 d3cf0b89 Stavros Sachtouris
.. code-block:: console
2329 d3cf0b89 Stavros Sachtouris
2330 d3cf0b89 Stavros Sachtouris
   $ kamaki config set cloud.default.url \
2331 d3cf0b89 Stavros Sachtouris
       "https://node1.example.com/astakos/identity/v2.0"
2332 d3cf0b89 Stavros Sachtouris
   $ kamaki config set cloud.default.token USER_TOKEN
2333 d3cf0b89 Stavros Sachtouris
2334 d3cf0b89 Stavros Sachtouris
Both the Authentication URL and the USER_TOKEN appear on the user's
2335 d3cf0b89 Stavros Sachtouris
`API access` web page on the Astakos Web UI.
2336 d3cf0b89 Stavros Sachtouris
2337 d3cf0b89 Stavros Sachtouris
You can see that the new configuration options have been applied correctly,
2338 d3cf0b89 Stavros Sachtouris
either by checking the editable file ``~/.kamakirc`` or by running:
2339 d3cf0b89 Stavros Sachtouris
2340 d3cf0b89 Stavros Sachtouris
.. code-block:: console
2341 d3cf0b89 Stavros Sachtouris
2342 d3cf0b89 Stavros Sachtouris
   $ kamaki config list
2343 d3cf0b89 Stavros Sachtouris
2344 d3cf0b89 Stavros Sachtouris
A quick test to check that kamaki is configured correctly, is to try to
2345 d3cf0b89 Stavros Sachtouris
authenticate a user based on his/her token (in this case the user is you):