Statistics
| Branch: | Tag: | Revision:

root / docs / quick-install-admin-guide.rst @ 11c16930

History | View | Annotate | Download (68 kB)

1
.. _quick-install-admin-guide:
2

    
3
Administrator's Quick Installation Guide
4
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5

    
6
This is the Administrator's quick installation guide.
7

    
8
It describes how to install the whole synnefo stack on two (2) physical nodes,
9
with minimum configuration. It installs synnefo from Debian packages, and
10
assumes the nodes run Debian Squeeze. After successful installation, you will
11
have the following services running:
12

    
13
 * Identity Management (Astakos)
14
 * Object Storage Service (Pithos+)
15
 * Compute Service (Cyclades)
16
 * Image Registry Service (Plankton)
17

    
18
and a single unified Web UI to manage them all.
19

    
20
The Volume Storage Service (Archipelago) and the Billing Service (Aquarium) are
21
not released yet.
22

    
23
If you just want to install the Object Storage Service (Pithos+), follow the guide
24
and just stop after the "Testing of Pithos+" section.
25

    
26

    
27
Installation of Synnefo / Introduction
28
======================================
29

    
30
We will install the services with the above list's order. Cyclades and Plankton
31
will be installed in a single step (at the end), because at the moment they are
32
contained in the same software component. Furthermore, we will install all
33
services in the first physical node, except Pithos+ which will be installed in
34
the second, due to a conflict between the snf-pithos-app and snf-cyclades-app
35
component (scheduled to be fixed in the next version).
36

    
37
For the rest of the documentation we will refer to the first physical node as
38
"node1" and the second as "node2". We will also assume that their domain names
39
are "node1.example.com" and "node2.example.com" and their IPs are "4.3.2.1" and
40
"4.3.2.2" respectively.
41

    
42
.. note:: It is import that the two machines are under the same domain name.
43
    If they are not, you can do this by editting the file ``/etc/hosts``
44
    on both machines, and add the following lines:
45

    
46
    .. code-block:: console
47

    
48
        4.3.2.1     node1.example.com
49
        4.3.2.2     node2.example.com
50

    
51

    
52
General Prerequisites
53
=====================
54

    
55
These are the general synnefo prerequisites, that you need on node1 and node2
56
and are related to all the services (Astakos, Pithos+, Cyclades, Plankton).
57

    
58
To be able to download all synnefo components you need to add the following
59
lines in your ``/etc/apt/sources.list`` file:
60

    
61
| ``deb http://apt.dev.grnet.gr squeeze main``
62
| ``deb-src http://apt.dev.grnet.gr squeeze main``
63
| ``deb http://apt.dev.grnet.gr squeeze-backports main``
64

    
65
and import the repo's GPG key:
66

    
67
| ``curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -``
68

    
69
Also add the following line to enable the ``squeeze-backports`` repository,
70
which may provide more recent versions of certain packages. The repository
71
is deactivated by default and must be specified expicitly in ``apt-get``
72
operations:
73

    
74
| ``deb http://backports.debian.org/debian-backports squeeze-backports main``
75

    
76
You also need a shared directory visible by both nodes. Pithos+ will save all
77
data inside this directory. By 'all data', we mean files, images, and pithos
78
specific mapping data. If you plan to upload more than one basic image, this
79
directory should have at least 50GB of free space. During this guide, we will
80
assume that node1 acts as an NFS server and serves the directory ``/srv/pithos``
81
to node2. Node2 has this directory mounted under ``/srv/pithos``, too.
82

    
83
Before starting the synnefo installation, you will need basic third party
84
software to be installed and configured on the physical nodes. We will describe
85
each node's general prerequisites separately. Any additional configuration,
86
specific to a synnefo service for each node, will be described at the service's
87
section.
88

    
89
Node1
90
-----
91

    
92
General Synnefo dependencies
93
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
94

    
95
 * apache (http server)
96
 * gunicorn (WSGI http server)
97
 * postgresql (database)
98
 * rabbitmq (message queue)
99

    
100
You can install the above by running:
101

    
102
.. code-block:: console
103

    
104
   # apt-get install apache2 postgresql rabbitmq-server
105

    
106
Make sure to install gunicorn >= v0.12.2. You can do this by installing from
107
the official debian backports:
108

    
109
.. code-block:: console
110

    
111
   # apt-get -t squeeze-backports install gunicorn
112

    
113
On node1, we will create our databases, so you will also need the
114
python-psycopg2 package:
115

    
116
.. code-block:: console
117

    
118
   # apt-get install python-psycopg2
119

    
120
Database setup
121
~~~~~~~~~~~~~~
122

    
123
On node1, we create a database called ``snf_apps``, that will host all django
124
apps related tables. We also create the user ``synnefo`` and grant him all
125
privileges on the database. We do this by running:
126

    
127
.. code-block:: console
128

    
129
   root@node1:~ # su - postgres
130
   postgres@node1:~ $ psql
131
   postgres=# CREATE DATABASE snf_apps WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
132
   postgres=# CREATE USER synnefo WITH PASSWORD 'example_passw0rd';
133
   postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_apps TO synnefo;
134

    
135
We also create the database ``snf_pithos`` needed by the pithos+ backend and
136
grant the ``synnefo`` user all privileges on the database. This database could
137
be created on node2 instead, but we do it on node1 for simplicity. We will
138
create all needed databases on node1 and then node2 will connect to them.
139

    
140
.. code-block:: console
141

    
142
   postgres=# CREATE DATABASE snf_pithos WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
143
   postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_pithos TO synnefo;
144

    
145
Configure the database to listen to all network interfaces. You can do this by
146
editting the file ``/etc/postgresql/8.4/main/postgresql.conf`` and change
147
``listen_addresses`` to ``'*'`` :
148

    
149
.. code-block:: console
150

    
151
   listen_addresses = '*'
152

    
153
Furthermore, edit ``/etc/postgresql/8.4/main/pg_hba.conf`` to allow node1 and
154
node2 to connect to the database. Add the following lines under ``#IPv4 local
155
connections:`` :
156

    
157
.. code-block:: console
158

    
159
   host		all	all	4.3.2.1/32	md5
160
   host		all	all	4.3.2.2/32	md5
161

    
162
Make sure to substitute "4.3.2.1" and "4.3.2.2" with node1's and node2's
163
actual IPs. Now, restart the server to apply the changes:
164

    
165
.. code-block:: console
166

    
167
   # /etc/init.d/postgresql restart
168

    
169
Gunicorn setup
170
~~~~~~~~~~~~~~
171

    
172
Create the file ``synnefo`` under ``/etc/gunicorn.d/`` containing the following:
173

    
174
.. code-block:: console
175

    
176
   CONFIG = {
177
    'mode': 'django',
178
    'environment': {
179
      'DJANGO_SETTINGS_MODULE': 'synnefo.settings',
180
    },
181
    'working_dir': '/etc/synnefo',
182
    'user': 'www-data',
183
    'group': 'www-data',
184
    'args': (
185
      '--bind=127.0.0.1:8080',
186
      '--workers=4',
187
      '--log-level=debug',
188
    ),
189
   }
190

    
191
.. warning:: Do NOT start the server yet, because it won't find the
192
    ``synnefo.settings`` module. We will start the server after successful
193
    installation of astakos. If the server is running::
194

    
195
       # /etc/init.d/gunicorn stop
196

    
197
Apache2 setup
198
~~~~~~~~~~~~~
199

    
200
Create the file ``synnefo`` under ``/etc/apache2/sites-available/`` containing
201
the following:
202

    
203
.. code-block:: console
204

    
205
   <VirtualHost *:80>
206
     ServerName node1.example.com
207

    
208
     RewriteEngine On
209
     RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
210
     RewriteRule ^(.*)$ - [F,L]
211
     RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
212
   </VirtualHost>
213

    
214
Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``
215
containing the following:
216

    
217
.. code-block:: console
218

    
219
   <IfModule mod_ssl.c>
220
   <VirtualHost _default_:443>
221
     ServerName node1.example.com
222

    
223
     Alias /static "/usr/share/synnefo/static"
224

    
225
   #  SetEnv no-gzip
226
   #  SetEnv dont-vary
227

    
228
     AllowEncodedSlashes On
229

    
230
     RequestHeader set X-Forwarded-Protocol "https"
231

    
232
     <Proxy * >
233
       Order allow,deny
234
       Allow from all
235
     </Proxy>
236

    
237
     SetEnv                proxy-sendchunked
238
     SSLProxyEngine        off
239
     ProxyErrorOverride    off
240

    
241
     ProxyPass        /static !
242
     ProxyPass        / http://localhost:8080/ retry=0
243
     ProxyPassReverse / http://localhost:8080/
244

    
245
     RewriteEngine On
246
     RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
247
     RewriteRule ^(.*)$ - [F,L]
248
     RewriteRule ^/login(.*) /im/login/redirect$1 [PT,NE]
249

    
250
     SSLEngine on
251
     SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
252
     SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
253
   </VirtualHost>
254
   </IfModule>
255

    
256
Now enable sites and modules by running:
257

    
258
.. code-block:: console
259

    
260
   # a2enmod ssl
261
   # a2enmod rewrite
262
   # a2dissite default
263
   # a2ensite synnefo
264
   # a2ensite synnefo-ssl
265
   # a2enmod headers
266
   # a2enmod proxy_http
267

    
268
.. warning:: Do NOT start/restart the server yet. If the server is running::
269

    
270
       # /etc/init.d/apache2 stop
271

    
272
.. _rabbitmq-setup:
273

    
274
Message Queue setup
275
~~~~~~~~~~~~~~~~~~~
276

    
277
The message queue will run on node1, so we need to create the appropriate
278
rabbitmq user. The user is named ``synnefo`` and gets full privileges on all
279
exchanges:
280

    
281
.. code-block:: console
282

    
283
   # rabbitmqctl add_user synnefo "examle_rabbitmq_passw0rd"
284
   # rabbitmqctl set_permissions synnefo ".*" ".*" ".*"
285

    
286
We do not need to initialize the exchanges. This will be done automatically,
287
during the Cyclades setup.
288

    
289
Pithos+ data directory setup
290
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
291

    
292
As mentioned in the General Prerequisites section, there is a directory called
293
``/srv/pithos`` visible by both nodes. We create and setup the ``data``
294
directory inside it:
295

    
296
.. code-block:: console
297

    
298
   # cd /srv/pithos
299
   # mkdir data
300
   # chown www-data:www-data data
301
   # chmod g+ws data
302

    
303
You are now ready with all general prerequisites concerning node1. Let's go to
304
node2.
305

    
306
Node2
307
-----
308

    
309
General Synnefo dependencies
310
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
311

    
312
 * apache (http server)
313
 * gunicorn (WSGI http server)
314
 * postgresql (database)
315

    
316
You can install the above by running:
317

    
318
.. code-block:: console
319

    
320
   # apt-get install apache2 postgresql
321

    
322
Make sure to install gunicorn >= v0.12.2. You can do this by installing from
323
the official debian backports:
324

    
325
.. code-block:: console
326

    
327
   # apt-get -t squeeze-backports install gunicorn
328

    
329
Node2 will connect to the databases on node1, so you will also need the
330
python-psycopg2 package:
331

    
332
.. code-block:: console
333

    
334
   # apt-get install python-psycopg2
335

    
336
Database setup
337
~~~~~~~~~~~~~~
338

    
339
All databases have been created and setup on node1, so we do not need to take
340
any action here. From node2, we will just connect to them. When you get familiar
341
with the software you may choose to run different databases on different nodes,
342
for performance/scalability/redundancy reasons, but those kind of setups are out
343
of the purpose of this guide.
344

    
345
Gunicorn setup
346
~~~~~~~~~~~~~~
347

    
348
Create the file ``synnefo`` under ``/etc/gunicorn.d/`` containing the following
349
(same contents as in node1; you can just copy/paste the file):
350

    
351
.. code-block:: console
352

    
353
   CONFIG = {
354
    'mode': 'django',
355
    'environment': {
356
      'DJANGO_SETTINGS_MODULE': 'synnefo.settings',
357
    },
358
    'working_dir': '/etc/synnefo',
359
    'user': 'www-data',
360
    'group': 'www-data',
361
    'args': (
362
      '--bind=127.0.0.1:8080',
363
      '--workers=4',
364
      '--log-level=debug',
365
      '--timeout=43200'
366
    ),
367
   }
368

    
369
.. warning:: Do NOT start the server yet, because it won't find the
370
    ``synnefo.settings`` module. We will start the server after successful
371
    installation of astakos. If the server is running::
372

    
373
       # /etc/init.d/gunicorn stop
374

    
375
Apache2 setup
376
~~~~~~~~~~~~~
377

    
378
Create the file ``synnefo`` under ``/etc/apache2/sites-available/`` containing
379
the following:
380

    
381
.. code-block:: console
382

    
383
   <VirtualHost *:80>
384
     ServerName node2.example.com
385

    
386
     RewriteEngine On
387
     RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
388
     RewriteRule ^(.*)$ - [F,L]
389
     RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
390
   </VirtualHost>
391

    
392
Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``
393
containing the following:
394

    
395
.. code-block:: console
396

    
397
   <IfModule mod_ssl.c>
398
   <VirtualHost _default_:443>
399
     ServerName node2.example.com
400

    
401
     Alias /static "/usr/share/synnefo/static"
402

    
403
     SetEnv no-gzip
404
     SetEnv dont-vary
405
     AllowEncodedSlashes On
406

    
407
     RequestHeader set X-Forwarded-Protocol "https"
408

    
409
     <Proxy * >
410
       Order allow,deny
411
       Allow from all
412
     </Proxy>
413

    
414
     SetEnv                proxy-sendchunked
415
     SSLProxyEngine        off
416
     ProxyErrorOverride    off
417

    
418
     ProxyPass        /static !
419
     ProxyPass        / http://localhost:8080/ retry=0
420
     ProxyPassReverse / http://localhost:8080/
421

    
422
     SSLEngine on
423
     SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
424
     SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
425
   </VirtualHost>
426
   </IfModule>
427

    
428
As in node1, enable sites and modules by running:
429

    
430
.. code-block:: console
431

    
432
   # a2enmod ssl
433
   # a2enmod rewrite
434
   # a2dissite default
435
   # a2ensite synnefo
436
   # a2ensite synnefo-ssl
437
   # a2enmod headers
438
   # a2enmod proxy_http
439

    
440
.. warning:: Do NOT start/restart the server yet. If the server is running::
441

    
442
       # /etc/init.d/apache2 stop
443

    
444
We are now ready with all general prerequisites for node2. Now that we have
445
finished with all general prerequisites for both nodes, we can start installing
446
the services. First, let's install Astakos on node1.
447

    
448

    
449
Installation of Astakos on node1
450
================================
451

    
452
To install astakos, grab the package from our repository (make sure  you made
453
the additions needed in your ``/etc/apt/sources.list`` file, as described
454
previously), by running:
455

    
456
.. code-block:: console
457

    
458
   # apt-get install snf-astakos-app
459

    
460
After successful installation of snf-astakos-app, make sure that also
461
snf-webproject has been installed (marked as "Recommended" package). By default
462
Debian installs "Recommended" packages, but if you have changed your
463
configuration and the package didn't install automatically, you should
464
explicitly install it manually running:
465

    
466
.. code-block:: console
467

    
468
   # apt-get install snf-webproject
469

    
470
The reason snf-webproject is "Recommended" and not a hard dependency, is to give
471
the experienced administrator the ability to install synnefo in a custom made
472
django project. This corner case concerns only very advanced users that know
473
what they are doing and want to experiment with synnefo.
474

    
475

    
476
.. _conf-astakos:
477

    
478
Configuration of Astakos
479
========================
480

    
481
Conf Files
482
----------
483

    
484
After astakos is successfully installed, you will find the directory
485
``/etc/synnefo`` and some configuration files inside it. The files contain
486
commented configuration options, which are the default options. While installing
487
new snf-* components, new configuration files will appear inside the directory.
488
In this guide (and for all services), we will edit only the minimum necessary
489
configuration options, to reflect our setup. Everything else will remain as is.
490

    
491
After getting familiar with synnefo, you will be able to customize the software
492
as you wish and fits your needs. Many options are available, to empower the
493
administrator with extensively customizable setups.
494

    
495
For the snf-webproject component (installed as an astakos dependency), we
496
need the following:
497

    
498
Edit ``/etc/synnefo/10-snf-webproject-database.conf``. You will need to
499
uncomment and edit the ``DATABASES`` block to reflect our database:
500

    
501
.. code-block:: console
502

    
503
   DATABASES = {
504
    'default': {
505
        # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
506
        'ENGINE': 'postgresql_psycopg2',
507
         # ATTENTION: This *must* be the absolute path if using sqlite3.
508
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
509
        'NAME': 'snf_apps',
510
        'USER': 'synnefo',                      # Not used with sqlite3.
511
        'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
512
        # Set to empty string for localhost. Not used with sqlite3.
513
        'HOST': '4.3.2.1',
514
        # Set to empty string for default. Not used with sqlite3.
515
        'PORT': '5432',
516
    }
517
   }
518

    
519
Edit ``/etc/synnefo/10-snf-webproject-deploy.conf``. Uncomment and edit
520
``SECRET_KEY``. This is a django specific setting which is used to provide a
521
seed in secret-key hashing algorithms. Set this to a random string of your
522
choise and keep it private:
523

    
524
.. code-block:: console
525

    
526
   SECRET_KEY = 'sy6)mw6a7x%n)-example_secret_key#zzk4jo6f2=uqu!1o%)'
527

    
528
For astakos specific configuration, edit the following options in
529
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :
530

    
531
.. code-block:: console
532

    
533
   ASTAKOS_DEFAULT_ADMIN_EMAIL = None
534

    
535
   ASTAKOS_IM_MODULES = ['local']
536

    
537
   ASTAKOS_COOKIE_DOMAIN = '.example.com'
538

    
539
   ASTAKOS_BASEURL = 'https://node1.example.com'
540

    
541
   ASTAKOS_SITENAME = '~okeanos demo example'
542

    
543
   ASTAKOS_RECAPTCHA_ENABLED = False
544

    
545
``ASTAKOS_IM_MODULES`` refers to the astakos login methods. For now only local
546
is supported. The ``ASTAKOS_COOKIE_DOMAIN`` should be the base url of our
547
domain (for all services). ``ASTAKOS_BASEURL`` is the astakos home page.
548

    
549
``ASTAKOS_DEFAULT_ADMIN_EMAIL`` refers to the administrator's email.
550
Every time a new account is created a notification is sent to this email.
551
For this we need access to a running mail server, so we have disabled
552
it for now by setting its value to None. For more informations on this,
553
read the relative :ref:`section <mail-server>`.
554

    
555
.. note:: For the purpose of this guide, we have disabled recaptcha authentication.
556
    If you would like to enable it you have to edit the following options:
557

    
558
    .. code-block:: console
559

    
560
        ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
561
        ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
562
        ASTAKOS_RECAPTCHA_USE_SSL = True
563
        ASTAKOS_RECAPTCHA_ENABLED = True
564

    
565
    For the ``ASTAKOS_RECAPTCHA_PUBLIC_KEY`` and ``ASTAKOS_RECAPTCHA_PRIVATE_KEY``
566
    go to https://www.google.com/recaptcha/admin/create and create your own pair.
567

    
568
Then edit ``/etc/synnefo/20-snf-astakos-app-cloudbar.conf`` :
569

    
570
.. code-block:: console
571

    
572
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
573

    
574
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/im/get_services'
575

    
576
   CLOUDBAR_MENU_URL = 'https://node1.example.com/im/get_menu'
577

    
578
Those settings have to do with the black cloudbar endpoints and will be described
579
in more detail later on in this guide. For now, just edit the domain to point at
580
node1 which is where we have installed Astakos.
581

    
582
If you are an advanced user and want to use the Shibboleth Authentication method,
583
read the relative :ref:`section <shibboleth-auth>`.
584

    
585
Database Initialization
586
-----------------------
587

    
588
After configuration is done, we initialize the database by running:
589

    
590
.. code-block:: console
591

    
592
   # snf-manage syncdb
593

    
594
At this example we don't need to create a django superuser, so we select
595
``[no]`` to the question. After a successful sync, we run the migration needed
596
for astakos:
597

    
598
.. code-block:: console
599

    
600
   # snf-manage migrate im
601

    
602
Then, we load the pre-defined user groups
603

    
604
.. code-block:: console
605

    
606
   # snf-manage loaddata groups
607

    
608
.. _services-reg:
609

    
610
Services Registration
611
---------------------
612

    
613
When the database is ready, we configure the elements of the Astakos cloudbar,
614
to point to our future services:
615

    
616
.. code-block:: console
617

    
618
   # snf-manage service-add "~okeanos home" https://node1.example.com/im/ home-icon.png
619
   # snf-manage service-add "cyclades" https://node1.example.com/ui/
620
   # snf-manage service-add "pithos+" https://node2.example.com/ui/
621

    
622
Servers Initialization
623
----------------------
624

    
625
Finally, we initialize the servers on node1:
626

    
627
.. code-block:: console
628

    
629
   root@node1:~ # /etc/init.d/gunicorn restart
630
   root@node1:~ # /etc/init.d/apache2 restart
631

    
632
We have now finished the Astakos setup. Let's test it now.
633

    
634

    
635
Testing of Astakos
636
==================
637

    
638
Open your favorite browser and go to:
639

    
640
``http://node1.example.com/im``
641

    
642
If this redirects you to ``https://node1.example.com/im`` and you can see
643
the "welcome" door of Astakos, then you have successfully setup Astakos.
644

    
645
Let's create our first user. At the homepage click the "CREATE ACCOUNT" button
646
and fill all your data at the sign up form. Then click "SUBMIT". You should now
647
see a green box on the top, which informs you that you made a successful request
648
and the request has been sent to the administrators. So far so good, let's assume
649
that you created the user with username ``user@example.com``.
650

    
651
Now we need to activate that user. Return to a command prompt at node1 and run:
652

    
653
.. code-block:: console
654

    
655
   root@node1:~ # snf-manage user-list
656

    
657
This command should show you a list with only one user; the one we just created.
658
This user should have an id with a value of ``1``. It should also have an
659
"active" status with the value of ``0`` (inactive). Now run:
660

    
661
.. code-block:: console
662

    
663
   root@node1:~ # snf-manage user-modify --set-active 1
664

    
665
This modifies the active value to ``1``, and actually activates the user.
666
When running in production, the activation is done automatically with different
667
types of moderation, that Astakos supports. You can see the moderation methods
668
(by invitation, whitelists, matching regexp, etc.) at the Astakos specific
669
documentation. In production, you can also manually activate a user, by sending
670
him/her an activation email. See how to do this at the :ref:`User
671
activation <user_activation>` section.
672

    
673
Now let's go back to the homepage. Open ``http://node1.example.com/im`` with
674
your browser again. Try to sign in using your new credentials. If the astakos
675
menu appears and you can see your profile, then you have successfully setup
676
Astakos.
677

    
678
Let's continue to install Pithos+ now.
679

    
680

    
681
Installation of Pithos+ on node2
682
================================
683

    
684
To install pithos+, grab the packages from our repository (make sure  you made
685
the additions needed in your ``/etc/apt/sources.list`` file, as described
686
previously), by running:
687

    
688
.. code-block:: console
689

    
690
   # apt-get install snf-pithos-app
691

    
692
After successful installation of snf-pithos-app, make sure that also
693
snf-webproject has been installed (marked as "Recommended" package). Refer to
694
the "Installation of Astakos on node1" section, if you don't remember why this
695
should happen. Now, install the pithos web interface:
696

    
697
.. code-block:: console
698

    
699
   # apt-get install snf-pithos-webclient
700

    
701
This package provides the standalone pithos web client. The web client is the
702
web UI for pithos+ and will be accessible by clicking "pithos+" on the Astakos
703
interface's cloudbar, at the top of the Astakos homepage.
704

    
705

    
706
.. _conf-pithos:
707

    
708
Configuration of Pithos+
709
========================
710

    
711
Conf Files
712
----------
713

    
714
After pithos+ is successfully installed, you will find the directory
715
``/etc/synnefo`` and some configuration files inside it, as you did in node1
716
after installation of astakos. Here, you will not have to change anything that
717
has to do with snf-common or snf-webproject. Everything is set at node1. You
718
only need to change settings that have to do with pithos+. Specifically:
719

    
720
Edit ``/etc/synnefo/20-snf-pithos-app-settings.conf``. There you need to set
721
this options:
722

    
723
.. code-block:: console
724

    
725
   PITHOS_BACKEND_DB_CONNECTION = 'postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos'
726

    
727
   PITHOS_BACKEND_BLOCK_PATH = '/srv/pithos/data'
728

    
729
   PITHOS_AUTHENTICATION_URL = 'https://node1.example.com/im/authenticate'
730
   PITHOS_AUTHENTICATION_USERS = None
731

    
732
   PITHOS_SERVICE_TOKEN = 'pithos_service_token22w=='
733

    
734
The ``PITHOS_BACKEND_DB_CONNECTION`` option tells to the pithos+ app where to
735
find the pithos+ backend database. Above we tell pithos+ that its database is
736
``snf_pithos`` at node1 and to connect as user ``synnefo`` with password
737
``example_passw0rd``.  All those settings where setup during node1's "Database
738
setup" section.
739

    
740
The ``PITHOS_BACKEND_BLOCK_PATH`` option tells to the pithos+ app where to find
741
the pithos+ backend data. Above we tell pithos+ to store its data under
742
``/srv/pithos/data``, which is visible by both nodes. We have already setup this
743
directory at node1's "Pithos+ data directory setup" section.
744

    
745
The ``PITHOS_AUTHENTICATION_URL`` option tells to the pithos+ app in which URI
746
is available the astakos authentication api. If not set, pithos+ tries to
747
authenticate using the ``PITHOS_AUTHENTICATION_USERS`` user pool.
748

    
749
The ``PITHOS_SERVICE_TOKEN`` should be the Pithos+ token returned by running on
750
the Astakos node (node1 in our case):
751

    
752
.. code-block:: console
753

    
754
   # snf-manage service-list
755

    
756
The token has been generated automatically during the :ref:`Pithos+ service
757
registration <services-reg>`.
758

    
759
Then we need to setup the web UI and connect it to astakos. To do so, edit
760
``/etc/synnefo/20-snf-pithos-webclient-settings.conf``:
761

    
762
.. code-block:: console
763

    
764
   PITHOS_UI_LOGIN_URL = "https://node1.example.com/im/login?next="
765
   PITHOS_UI_FEEDBACK_URL = "https://node1.example.com/im/feedback"
766

    
767
The ``PITHOS_UI_LOGIN_URL`` option tells the client where to redirect you, if
768
you are not logged in. The ``PITHOS_UI_FEEDBACK_URL`` option points at the
769
pithos+ feedback form. Astakos already provides a generic feedback form for all
770
services, so we use this one.
771

    
772
Then edit ``/etc/synnefo/20-snf-pithos-webclient-cloudbar.conf``, to connect the
773
pithos+ web UI with the astakos web UI (through the top cloudbar):
774

    
775
.. code-block:: console
776

    
777
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
778
   PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE = '3'
779
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/im/get_services'
780
   CLOUDBAR_MENU_URL = 'https://node1.example.com/im/get_menu'
781

    
782
The ``CLOUDBAR_LOCATION`` tells the client where to find the astakos common
783
cloudbar.
784

    
785
The ``PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE`` points to an already registered
786
Astakos service. You can see all :ref:`registered services <services-reg>` by
787
running on the Astakos node (node1):
788

    
789
.. code-block:: console
790

    
791
   # snf-manage service-list
792

    
793
The value of ``PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE`` should be the pithos service's
794
``id`` as shown by the above command, in our case ``3``.
795

    
796
The ``CLOUDBAR_SERVICES_URL`` and ``CLOUDBAR_MENU_URL`` options are used by the
797
pithos+ web client to get from astakos all the information needed to fill its
798
own cloudbar. So we put our astakos deployment urls there.
799

    
800
Servers Initialization
801
----------------------
802

    
803
After configuration is done, we initialize the servers on node2:
804

    
805
.. code-block:: console
806

    
807
   root@node2:~ # /etc/init.d/gunicorn restart
808
   root@node2:~ # /etc/init.d/apache2 restart
809

    
810
You have now finished the Pithos+ setup. Let's test it now.
811

    
812

    
813
Testing of Pithos+
814
==================
815

    
816
Open your browser and go to the Astakos homepage:
817

    
818
``http://node1.example.com/im``
819

    
820
Login, and you will see your profile page. Now, click the "pithos+" link on the
821
top black cloudbar. If everything was setup correctly, this will redirect you
822
to:
823

    
824
``https://node2.example.com/ui``
825

    
826
and you will see the blue interface of the Pithos+ application.  Click the
827
orange "Upload" button and upload your first file. If the file gets uploaded
828
successfully, then this is your first sign of a successful Pithos+ installation.
829
Go ahead and experiment with the interface to make sure everything works
830
correctly.
831

    
832
You can also use the Pithos+ clients to sync data from your Windows PC or MAC.
833

    
834
If you don't stumble on any problems, then you have successfully installed
835
Pithos+, which you can use as a standalone File Storage Service.
836

    
837
If you would like to do more, such as:
838

    
839
 * Spawning VMs
840
 * Spawning VMs from Images stored on Pithos+
841
 * Uploading your custom Images to Pithos+
842
 * Spawning VMs from those custom Images
843
 * Registering existing Pithos+ files as Images
844
 * Connect VMs to the Internet
845
 * Create Private Networks
846
 * Add VMs to Private Networks
847

    
848
please continue with the rest of the guide.
849

    
850

    
851
Cyclades (and Plankton) Prerequisites
852
=====================================
853

    
854
Before proceeding with the Cyclades (and Plankton) installation, make sure you
855
have successfully set up Astakos and Pithos+ first, because Cyclades depends
856
on them. If you don't have a working Astakos and Pithos+ installation yet,
857
please return to the :ref:`top <quick-install-admin-guide>` of this guide.
858

    
859
Besides Astakos and Pithos+, you will also need a number of additional working
860
prerequisites, before you start the Cyclades installation.
861

    
862
Ganeti
863
------
864

    
865
`Ganeti <http://code.google.com/p/ganeti/>`_ handles the low level VM management
866
for Cyclades, so Cyclades requires a working Ganeti installation at the backend.
867
Please refer to the
868
`ganeti documentation <http://docs.ganeti.org/ganeti/2.5/html>`_ for all the
869
gory details. A successful Ganeti installation concludes with a working
870
:ref:`GANETI-MASTER <GANETI_NODES>` and a number of :ref:`GANETI-NODEs
871
<GANETI_NODES>`.
872

    
873
The above Ganeti cluster can run on different physical machines than node1 and
874
node2 and can scale independently, according to your needs.
875

    
876
For the purpose of this guide, we will assume that the :ref:`GANETI-MASTER
877
<GANETI_NODES>` runs on node1 and is VM-capable. Also, node2 is a
878
:ref:`GANETI-NODE <GANETI_NODES>` and is Master-capable and VM-capable too.
879

    
880
We highly recommend that you read the official Ganeti documentation, if you are
881
not familiar with Ganeti. If you are extremely impatient, you can result with
882
the above assumed setup by running on both nodes:
883

    
884
.. code-block:: console
885

    
886
   # apt-get install ganeti2
887
   # apt-get install ganeti-htools
888
   # modprobe drbd minor_count=255 usermode_helper=/bin/true
889

    
890
Unfortunatelly, stock Ganeti doesn't support IP pool management yet (we are
891
working hard to merge it upstream for Ganeti 2.7). Synnefo depends on the IP
892
pool functionality of Ganeti, so you have to use GRNET's patches for now. To
893
do so you have to build your own package from source:
894

    
895
.. code-block:: console
896

    
897
   # apt-get install -t squeeze-backports python-bitarray
898
   # apt-get install git-buildpackage
899
   # git clone https://code.grnet.gr/git/ganeti-local
900
   # mkdir build-area
901
   # cd ganeti-local
902
   # git checkout stable-2.6-grnet
903
   # git checkout debian-2.6-grnet
904
   # git-buildpackage --git-upstream-branch=stable-2.6-grnet \
905
                   --git-debian-branch=debian-2.6-grnet \
906
                   --git-export=INDEX \
907
                   --git-ignore-new
908

    
909
This will create two deb packages in build-area. You should then run in both
910
nodes:
911

    
912
.. code-block:: console
913

    
914
   # dpkg -i build-area/\*deb
915
   # apt-get install -f
916

    
917
We assume that Ganeti will use the KVM hypervisor. After installing Ganeti on
918
both nodes, choose a domain name that resolves to a valid floating IP (let's say
919
it's ``ganeti.node1.example.com``). Make sure node1 and node2 have root access
920
between each other using ssh keys and not passwords. Also, make sure there is an
921
lvm volume group named ``ganeti`` that will host your VMs' disks. Finally, setup
922
a bridge interface on the host machines (e.g: br0). Then run on node1:
923

    
924
.. code-block:: console
925

    
926
   root@node1:~ # gnt-cluster init --enabled-hypervisors=kvm --no-ssh-init \
927
                                   --no-etc-hosts --vg-name=ganeti \
928
                                   --nic-parameters link=br0 --master-netdev eth0 \
929
                                   ganeti.node1.example.com
930
   root@node1:~ # gnt-cluster modify --default-iallocator hail
931
   root@node1:~ # gnt-cluster modify --hypervisor-parameters kvm:kernel_path=
932
   root@node1:~ # gnt-cluster modify --hypervisor-parameters kvm:vnc_bind_address=0.0.0.0
933

    
934
   root@node1:~ # gnt-node add --no-node-setup --master-capable=yes \
935
                               --vm-capable=yes node2.example.com
936
   root@node1:~ # gnt-cluster modify --disk-parameters=drbd:metavg=ganeti
937
   root@node1:~ # gnt-group modify --disk-parameters=drbd:metavg=ganeti default
938

    
939
For any problems you may stumble upon installing Ganeti, please refer to the
940
`official documentation <http://docs.ganeti.org/ganeti/2.5/html>`_. Installation
941
of Ganeti is out of the scope of this guide.
942

    
943
.. _cyclades-install-snfimage:
944

    
945
snf-image
946
---------
947

    
948
Installation
949
~~~~~~~~~~~~
950
For :ref:`Cyclades <cyclades>` to be able to launch VMs from specified Images,
951
you need the :ref:`snf-image <snf-image>` OS Definition installed on *all*
952
VM-capable Ganeti nodes. This means we need :ref:`snf-image <snf-image>` on
953
node1 and node2. You can do this by running on *both* nodes:
954

    
955
.. code-block:: console
956

    
957
   # apt-get install snf-image-host snf-pithos-backend python-psycopg2
958

    
959
snf-image also needs the `snf-pithos-backend <snf-pithos-backend>`, to be able to
960
handle image files stored on Pithos+. It also needs `python-psycopg2` to be able
961
to access the Pithos+ database. This is why, we also install them on *all*
962
VM-capable Ganeti nodes.
963

    
964
Now, you need to download and save the corresponding helper package. Please see
965
`here <https://code.grnet.gr/projects/snf-image/files>`_ for the latest package. Let's
966
assume that you installed snf-image-host version 0.4.4-1. Then, you need
967
snf-image-helper v0.4.4-1 on *both* nodes:
968

    
969
.. code-block:: console
970

    
971
   # cd /var/lib/snf-image/helper/
972
   # wget https://code.grnet.gr/attachments/download/1058/snf-image-helper_0.4.4-1_all.deb
973

    
974
.. warning:: Be careful: Do NOT install the snf-image-helper debian package.
975
             Just put it under /var/lib/snf-image/helper/
976

    
977
Once, you have downloaded the snf-image-helper package, create the helper VM by
978
running on *both* nodes:
979

    
980
.. code-block:: console
981

    
982
   # ln -s snf-image-helper_0.4.4-1_all.deb snf-image-helper.deb
983
   # snf-image-update-helper
984

    
985
This will create all the needed files under ``/var/lib/snf-image/helper/`` for
986
snf-image-host to run successfully.
987

    
988
Configuration
989
~~~~~~~~~~~~~
990
snf-image supports native access to Images stored on Pithos+. This means that
991
snf-image can talk directly to the Pithos+ backend, without the need of providing
992
a public URL. More details, are described in the next section. For now, the only
993
thing we need to do, is configure snf-image to access our Pithos+ backend.
994

    
995
To do this, we need to set the corresponding variables in
996
``/etc/default/snf-image``, to reflect our Pithos+ setup:
997

    
998
.. code-block:: console
999

    
1000
   PITHOS_DB="postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos"
1001

    
1002
   PITHOS_DATA="/srv/pithos/data"
1003

    
1004
If you have installed your Ganeti cluster on different nodes than node1 and node2 make
1005
sure that ``/srv/pithos/data`` is visible by all of them.
1006

    
1007
If you would like to use Images that are also/only stored locally, you need to
1008
save them under ``IMAGE_DIR``, however this guide targets Images stored only on
1009
Pithos+.
1010

    
1011
Testing
1012
~~~~~~~
1013
You can test that snf-image is successfully installed by running on the
1014
:ref:`GANETI-MASTER <GANETI_NODES>` (in our case node1):
1015

    
1016
.. code-block:: console
1017

    
1018
   # gnt-os diagnose
1019

    
1020
This should return ``valid`` for snf-image.
1021

    
1022
If you are interested to learn more about snf-image's internals (and even use
1023
it alongside Ganeti without Synnefo), please see
1024
`here <https://code.grnet.gr/projects/snf-image/wiki>`_ for information concerning
1025
installation instructions, documentation on the design and implementation, and
1026
supported Image formats.
1027

    
1028
.. _snf-image-images:
1029

    
1030
snf-image's actual Images
1031
-------------------------
1032

    
1033
Now that snf-image is installed successfully we need to provide it with some
1034
Images. :ref:`snf-image <snf-image>` supports Images stored in ``extdump``,
1035
``ntfsdump`` or ``diskdump`` format. We recommend the use of the ``diskdump``
1036
format. For more information about snf-image's Image formats see `here
1037
<https://code.grnet.gr/projects/snf-image/wiki/Image_Format>`_.
1038

    
1039
:ref:`snf-image <snf-image>` also supports three (3) different locations for the
1040
above Images to be stored:
1041

    
1042
 * Under a local folder (usually an NFS mount, configurable as ``IMAGE_DIR`` in
1043
   :file:`/etc/default/snf-image`)
1044
 * On a remote host (accessible via a public URL e.g: http://... or ftp://...)
1045
 * On Pithos+ (accessible natively, not only by its public URL)
1046

    
1047
For the purpose of this guide, we will use the `Debian Squeeze Base Image
1048
<https://pithos.okeanos.grnet.gr/public/9epgb>`_ found on the official
1049
`snf-image page
1050
<https://code.grnet.gr/projects/snf-image/wiki#Sample-Images>`_. The image is
1051
of type ``diskdump``. We will store it in our new Pithos+ installation.
1052

    
1053
To do so, do the following:
1054

    
1055
a) Download the Image from the official snf-image page (`image link
1056
   <https://pithos.okeanos.grnet.gr/public/9epgb>`_).
1057

    
1058
b) Upload the Image to your Pithos+ installation, either using the Pithos+ Web UI
1059
   or the command line client `kamaki
1060
   <http://docs.dev.grnet.gr/kamaki/latest/index.html>`_.
1061

    
1062
Once the Image is uploaded successfully, download the Image's metadata file
1063
from the official snf-image page (`image_metadata link
1064
<https://pithos.okeanos.grnet.gr/public/gwqcv>`_). You will need it, for
1065
spawning a VM from Ganeti, in the next section.
1066

    
1067
Of course, you can repeat the procedure to upload more Images, available from the
1068
`official snf-image page
1069
<https://code.grnet.gr/projects/snf-image/wiki#Sample-Images>`_.
1070

    
1071
.. _ganeti-with-pithos-images:
1072

    
1073
Spawning a VM from a Pithos+ Image, using Ganeti
1074
------------------------------------------------
1075

    
1076
Now, it is time to test our installation so far. So, we have Astakos and
1077
Pithos+ installed, we have a working Ganeti installation, the snf-image
1078
definition installed on all VM-capable nodes and a Debian Squeeze Image on
1079
Pithos+. Make sure you also have the `metadata file
1080
<https://pithos.okeanos.grnet.gr/public/gwqcv>`_ for this image.
1081

    
1082
Run on the :ref:`GANETI-MASTER's <GANETI_NODES>` (node1) command line:
1083

    
1084
.. code-block:: console
1085

    
1086
   # gnt-instance add -o snf-image+default --os-parameters \
1087
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1088
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1089
                      testvm1
1090

    
1091
In the above command:
1092

    
1093
 * ``img_passwd``: the arbitrary root password of your new instance
1094
 * ``img_format``: set to ``diskdump`` to reflect the type of the uploaded Image
1095
 * ``img_id``: If you want to deploy an Image stored on Pithos+ (our case), this
1096
               should have the format
1097
               ``pithos://<username>/<container>/<filename>``:
1098
                * ``username``: ``user@example.com`` (defined during Astakos sign up)
1099
                * ``container``: ``pithos`` (default, if the Web UI was used)
1100
                * ``filename``: the name of file (visible also from the Web UI)
1101
 * ``img_properties``: taken from the metadata file. Used only the two mandatory
1102
                       properties ``OSFAMILY`` and ``ROOT_PARTITION``. `Learn more
1103
                       <https://code.grnet.gr/projects/snf-image/wiki/Image_Format#Image-Properties>`_
1104

    
1105
If the ``gnt-instance add`` command returns successfully, then run:
1106

    
1107
.. code-block:: console
1108

    
1109
   # gnt-instance info testvm1 | grep "console connection"
1110

    
1111
to find out where to connect using VNC. If you can connect successfully and can
1112
login to your new instance using the root password ``my_vm_example_passw0rd``,
1113
then everything works as expected and you have your new Debian Base VM up and
1114
running.
1115

    
1116
If ``gnt-instance add`` fails, make sure that snf-image is correctly configured
1117
to access the Pithos+ database and the Pithos+ backend data. Also, make sure
1118
you gave the correct ``img_id`` and ``img_properties``. If ``gnt-instance add``
1119
succeeds but you cannot connect, again find out what went wrong. Do *NOT*
1120
proceed to the next steps unless you are sure everything works till this point.
1121

    
1122
If everything works, you have successfully connected Ganeti with Pithos+. Let's
1123
move on to networking now.
1124

    
1125
.. warning::
1126
    You can bypass the networking sections and go straight to
1127
    :ref:`Cyclades Ganeti tools <cyclades-gtools>`, if you do not want to setup
1128
    the Cyclades Network Service, but only the Cyclades Compute Service
1129
    (recommended for now).
1130

    
1131
Networking Setup Overview
1132
-------------------------
1133

    
1134
This part is deployment-specific and must be customized based on the specific
1135
needs of the system administrator. However, to do so, the administrator needs
1136
to understand how each level handles Virtual Networks, to be able to setup the
1137
backend appropriately, before installing Cyclades. To do so, please read the
1138
:ref:`Network <networks>` section before proceeding.
1139

    
1140
Since synnefo 0.11 all network actions are managed with the snf-manage
1141
network-* commands. This needs the underlying setup (Ganeti, nfdhcpd,
1142
snf-network, bridges, vlans) to be already configured correctly. The only
1143
actions needed in this point are:
1144

    
1145
a) Have Ganeti with IP pool management support installed.
1146

    
1147
b) Install :ref:`snf-network <snf-network>`, which provides a synnefo specific kvm-ifup script, etc.
1148

    
1149
c) Install :ref:`nfdhcpd <nfdhcpd>`, which serves DHCP requests of the VMs.
1150

    
1151
In order to test that everything is setup correctly before installing Cyclades,
1152
we will make some testing actions in this section, and the actual setup will be
1153
done afterwards with snf-manage commands.
1154

    
1155
.. _snf-network:
1156

    
1157
snf-network
1158
~~~~~~~~~~~
1159

    
1160
snf-network includes `kvm-vif-bridge` script that is invoked every time
1161
a tap (a VM's NIC) is created. Based on environment variables passed by
1162
Ganeti it issues various commands depending on the network type the NIC is
1163
connected to and sets up a corresponding dhcp lease.
1164

    
1165
Install snf-network on all Ganeti nodes:
1166

    
1167
.. code-block:: console
1168

    
1169
   # apt-get install snf-network
1170

    
1171
Then, in :file:`/etc/default/snf-network` set:
1172

    
1173
.. code-block:: console
1174

    
1175
   MAC_MASK=ff:ff:f0:00:00:00
1176

    
1177
.. _nfdhcpd:
1178

    
1179
nfdhcpd
1180
~~~~~~~
1181

    
1182
Each NIC's IP is chosen by Ganeti (with IP pool management support).
1183
`kvm-vif-bridge` script sets up dhcp leases and when the VM boots and
1184
makes a dhcp request, iptables will mangle the packet and `nfdhcpd` will
1185
create a dhcp response.
1186

    
1187
.. code-block:: console
1188

    
1189
   # apt-get install nfqueue-bindings-python=0.3+physindev-1
1190
   # apt-get install nfdhcpd
1191

    
1192
Edit ``/etc/nfdhcpd/nfdhcpd.conf`` to reflect your network configuration. At
1193
least, set the ``dhcp_queue`` variable to ``42`` and the ``nameservers``
1194
variable to your DNS IP/s. Those IPs will be passed as the DNS IP/s of your new
1195
VMs. Once you are finished, restart the server on all nodes:
1196

    
1197
.. code-block:: console
1198

    
1199
   # /etc/init.d/nfdhcpd restart
1200

    
1201
If you are using ``ferm``, then you need to run the following:
1202

    
1203
.. code-block:: console
1204

    
1205
   # echo "@include 'nfdhcpd.ferm';" >> /etc/ferm/ferm.conf
1206
   # /etc/init.d/ferm restart
1207

    
1208
or make sure to run after boot:
1209

    
1210
.. code-block:: console
1211

    
1212
   # iptables -t mangle -A PREROUTING -p udp -m udp --dport 67 -j NFQUEUE --queue-num 42
1213

    
1214
and if you have IPv6 enabled:
1215

    
1216
.. code-block:: console
1217

    
1218
   # ip6tables -t mangle -A PREROUTING -p ipv6-icmp -m icmp6 --icmpv6-type 133 -j NFQUEUE --queue-num 43
1219
   # ip6tables -t mangle -A PREROUTING -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j NFQUEUE --queue-num 44
1220

    
1221
You can check which clients are currently served by nfdhcpd by running:
1222

    
1223
.. code-block:: console
1224

    
1225
   # kill -SIGUSR1 `cat /var/run/nfdhcpd/nfdhcpd.pid`
1226

    
1227
When you run the above, then check ``/var/log/nfdhcpd/nfdhcpd.log``.
1228

    
1229
Public Network Setup
1230
--------------------
1231

    
1232
To achieve basic networking the simplest way is to have a common bridge (e.g.
1233
``br0``, on the same collision domain with the router) where all VMs will connect
1234
to. Packets will be "forwarded" to the router and then to the Internet. If
1235
you want a more advanced setup (ip-less routing and proxy-arp plese refer to
1236
:ref:`Network <networks>` section).
1237

    
1238
Physical Host Setup
1239
~~~~~~~~~~~~~~~~~~~
1240

    
1241
Assuming ``eth0`` on both hosts is the public interface (directly connected
1242
to the router), run on every node:
1243

    
1244
.. code-block:: console
1245

    
1246
   # brctl addbr br0
1247
   # ip link set br0 up
1248
   # vconfig add eth0 100
1249
   # ip link set eth0.100 up
1250
   # brctl addif br0 eth0.100
1251

    
1252

    
1253
Testing a Public Network
1254
~~~~~~~~~~~~~~~~~~~~~~~~
1255

    
1256
Let's assume, that you want to assign IPs from the ``5.6.7.0/27`` range to you
1257
new VMs, with ``5.6.7.1`` as the router's gateway. In Ganeti you can add the
1258
network by running:
1259

    
1260
.. code-block:: console
1261

    
1262
   # gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network-type=public --tags=nfdhcpd test-net-public
1263

    
1264
Then, connect the network to all your nodegroups. We assume that we only have
1265
one nodegroup (``default``) in our Ganeti cluster:
1266

    
1267
.. code-block:: console
1268

    
1269
   # gnt-network connect test-net-public default bridged br0
1270

    
1271
Now, it is time to test that the backend infrastracture is correctly setup for
1272
the Public Network. We will add a new VM, the same way we did it on the
1273
previous testing section. However, now will also add one NIC, configured to be
1274
managed from our previously defined network. Run on the GANETI-MASTER (node1):
1275

    
1276
.. code-block:: console
1277

    
1278
   # gnt-instance add -o snf-image+default --os-parameters \
1279
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1280
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1281
                      --net 0:ip=pool,network=test-net-public \
1282
                      testvm2
1283

    
1284
If the above returns successfully, connect to the new VM and run:
1285

    
1286
.. code-block:: console
1287

    
1288
   root@testvm2:~ # ip addr
1289
   root@testvm2:~ # ip route
1290
   root@testvm2:~ # cat /etc/resolv.conf
1291

    
1292
to check IP address (5.6.7.2), IP routes (default via 5.6.7.1) and DNS config
1293
(nameserver option in nfdhcpd.conf). This shows correct configuration of
1294
ganeti, snf-network and nfdhcpd.
1295

    
1296
Now ping the outside world. If this works too, then you have also configured
1297
correctly your physical host and router.
1298

    
1299
Make sure everything works as expected, before proceeding with the Private
1300
Networks setup.
1301

    
1302
.. _private-networks-setup:
1303

    
1304
Private Networks Setup
1305
----------------------
1306

    
1307
Synnefo supports two types of private networks:
1308

    
1309
 - based on MAC filtering
1310
 - based on physical VLANs
1311

    
1312
Both types provide Layer 2 isolation to the end-user.
1313

    
1314
For the first type a common bridge (e.g. ``prv0``) is needed while for the second a
1315
range of bridges (e.g. ``prv1..prv100``) each bridged on a different physical
1316
VLAN. To this end to assure isolation among end-users' private networks each
1317
has to have different MAC prefix (for the filtering to take place) or to be
1318
"connected" to a different bridge (VLAN actually).
1319

    
1320
Physical Host Setup
1321
~~~~~~~~~~~~~~~~~~~
1322

    
1323
In order to create the necessary VLAN/bridges, one for MAC filtered private
1324
networks and various (e.g. 20) for private networks based on physical VLANs,
1325
run on every node:
1326

    
1327
Assuming ``eth0`` of both hosts are somehow (via cable/switch with VLANs
1328
configured correctly) connected together, run on every node:
1329

    
1330
.. code-block:: console
1331

    
1332
   # apt-get install vlan
1333
   # modprobe 8021q
1334
   # $iface=eth0
1335
   # for prv in $(seq 0 20); do
1336
	vlan=$prv
1337
	bridge=prv$prv
1338
	vconfig add $iface $vlan
1339
	ifconfig $iface.$vlan up
1340
	brctl addbr $bridge
1341
	brctl setfd $bridge 0
1342
	brctl addif $bridge $iface.$vlan
1343
	ifconfig $bridge up
1344
      done
1345

    
1346
The above will do the following :
1347

    
1348
 * provision 21 new bridges: ``prv0`` - ``prv20``
1349
 * provision 21 new vlans: ``eth0.0`` - ``eth0.20``
1350
 * add the corresponding vlan to the equivalent bridge
1351

    
1352
You can run ``brctl show`` on both nodes to see if everything was setup
1353
correctly.
1354

    
1355
Synnefo Setup
1356
~~~~~~~~~~~~~
1357

    
1358
As long as those resourses have been provisioned, admin has to define two
1359
different pools in Synnefo:
1360

    
1361
 - MAC prefix Pool
1362
 - Bridge Pool
1363

    
1364
.. code-block:: console
1365

    
1366
   root@testvm1:~ # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
1367

    
1368
   root@testvm1:~ # snf-manage pool-create --type=bridge --base=prv --size=20
1369

    
1370
Change the Synnefo setting in :file:`20-snf-cyclades-app-api.conf`:
1371

    
1372
.. code-block:: console
1373

    
1374
   PRIVATE_MAC_FILTERED_BRIDGE = 'prv0'
1375

    
1376
Testing the Private Networks
1377
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1378

    
1379
To test the Private Networks, we will create two instances and put them in the
1380
same Private Networks (one MAC Filtered and one Physical VLAN). This means
1381
that the instances will have a second NIC connected to the ``prv0``
1382
pre-provisioned bridge and a third to ``prv1``.
1383

    
1384
We run the same command as in the Public Network testing section, but with one
1385
more argument for the second NIC:
1386

    
1387
.. code-block:: console
1388

    
1389
   # gnt-network add --network=192.168.1.0/24 --mac-prefix=aa:00:55 --network-type=private --tags=nfdhcpd,private-filtered test-net-prv-mac
1390
   # gnt-network connect test-net-prv-mac default bridged prv0
1391

    
1392
   # gnt-network add --network=10.0.0.0/24 --tags=nfdhcpd --network-type=private test-net-prv-vlan
1393
   # gnt-network connect test-net-prv-vlan default bridged prv1
1394

    
1395
   # gnt-instance add -o snf-image+default --os-parameters \
1396
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1397
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1398
                      --net 0:ip=pool,network=test-net-public \
1399
                      --net 1:ip=pool,network=test-net-prv-mac \
1400
                      --net 2:ip=none,network=test-net-prv-vlan \
1401
                      testvm3
1402

    
1403
   # gnt-instance add -o snf-image+default --os-parameters \
1404
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1405
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1406
                      --net 0:ip=pool,network=test-net-public \
1407
                      --net 1:ip=pool,network=test-net-prv-mac \
1408
                      --net 2:ip=none,network=test-net-prv-vlan \
1409
                      testvm4
1410

    
1411
Above, we create two instances with first NIC connected to the internet, their
1412
second NIC connected to a MAC filtered private Network and their third NIC
1413
connected to the first Physical VLAN Private Network. Now, connect to the
1414
instances using VNC and make sure everything works as expected:
1415

    
1416
 a) The instances have access to the public internet through their first eth
1417
    interface (``eth0``), which has been automatically assigned a public IP.
1418

    
1419
 b) ``eth1`` will have mac prefix ``aa:00:55``, while ``eth2`` default one (``aa:00:00``)
1420

    
1421
 c) ip link set ``eth1``/``eth2`` up
1422

    
1423
 d) dhclient ``eth1``/``eth2``
1424

    
1425
 e) On testvm3  ping 192.168.1.2/10.0.0.2
1426

    
1427
If everything works as expected, then you have finished the Network Setup at the
1428
backend for both types of Networks (Public & Private).
1429

    
1430
.. _cyclades-gtools:
1431

    
1432
Cyclades Ganeti tools
1433
---------------------
1434

    
1435
In order for Ganeti to be connected with Cyclades later on, we need the
1436
`Cyclades Ganeti tools` available on all Ganeti nodes (node1 & node2 in our
1437
case). You can install them by running in both nodes:
1438

    
1439
.. code-block:: console
1440

    
1441
   # apt-get install snf-cyclades-gtools
1442

    
1443
This will install the following:
1444

    
1445
 * ``snf-ganeti-eventd`` (daemon to publish Ganeti related messages on RabbitMQ)
1446
 * ``snf-ganeti-hook`` (all necessary hooks under ``/etc/ganeti/hooks``)
1447
 * ``snf-progress-monitor`` (used by ``snf-image`` to publish progress messages)
1448

    
1449
Configure ``snf-cyclades-gtools``
1450
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1451

    
1452
The package will install the ``/etc/synnefo/10-snf-cyclades-gtools-backend.conf``
1453
configuration file. At least we need to set the RabbitMQ endpoint for all tools
1454
that need it:
1455

    
1456
.. code-block:: console
1457

    
1458
   AMQP_HOSTS=["amqp://synnefo:example_rabbitmq_passw0rd@node1.example.com:5672"]
1459

    
1460
The above variables should reflect your :ref:`Message Queue setup
1461
<rabbitmq-setup>`. This file should be editted in all Ganeti nodes.
1462

    
1463
Connect ``snf-image`` with ``snf-progress-monitor``
1464
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1465

    
1466
Finally, we need to configure ``snf-image`` to publish progress messages during
1467
the deployment of each Image. To do this, we edit ``/etc/default/snf-image`` and
1468
set the corresponding variable to ``snf-progress-monitor``:
1469

    
1470
.. code-block:: console
1471

    
1472
   PROGRESS_MONITOR="snf-progress-monitor"
1473

    
1474
This file should be editted in all Ganeti nodes.
1475

    
1476
.. _rapi-user:
1477

    
1478
Synnefo RAPI user
1479
-----------------
1480

    
1481
As a last step before installing Cyclades, create a new RAPI user that will
1482
have ``write`` access. Cyclades will use this user to issue commands to Ganeti,
1483
so we will call the user ``cyclades`` with password ``example_rapi_passw0rd``.
1484
You can do this, by first running:
1485

    
1486
.. code-block:: console
1487

    
1488
   # echo -n 'cyclades:Ganeti Remote API:example_rapi_passw0rd' | openssl md5
1489

    
1490
and then putting the output in ``/var/lib/ganeti/rapi/users`` as follows:
1491

    
1492
.. code-block:: console
1493

    
1494
   cyclades {HA1}55aec7050aa4e4b111ca43cb505a61a0 write
1495

    
1496
More about Ganeti's RAPI users `here.
1497
<http://docs.ganeti.org/ganeti/2.5/html/rapi.html#introduction>`_
1498

    
1499
You have now finished with all needed Prerequisites for Cyclades (and
1500
Plankton). Let's move on to the actual Cyclades installation.
1501

    
1502

    
1503
Installation of Cyclades (and Plankton) on node1
1504
================================================
1505

    
1506
This section describes the installation of Cyclades. Cyclades is Synnefo's
1507
Compute service. Plankton (the Image Registry service) will get installed
1508
automatically along with Cyclades, because it is contained in the same Synnefo
1509
component right now.
1510

    
1511
We will install Cyclades (and Plankton) on node1. To do so, we install the
1512
corresponding package by running on node1:
1513

    
1514
.. code-block:: console
1515

    
1516
   # apt-get install snf-cyclades-app
1517

    
1518
.. warning:: Make sure you have installed ``python-gevent`` version >= 0.13.6.
1519
    This version is available at squeeze-backports and can be installed by
1520
    running: ``apt-get install -t squeeze-backports python-gevent``
1521

    
1522
If all packages install successfully, then Cyclades and Plankton are installed
1523
and we proceed with their configuration.
1524

    
1525

    
1526
Configuration of Cyclades (and Plankton)
1527
========================================
1528

    
1529
Conf files
1530
----------
1531

    
1532
After installing Cyclades, a number of new configuration files will appear under
1533
``/etc/synnefo/`` prefixed with ``20-snf-cyclades-app-``. We will descibe here
1534
only the minimal needed changes to result with a working system. In general, sane
1535
defaults have been chosen for the most of the options, to cover most of the
1536
common scenarios. However, if you want to tweak Cyclades feel free to do so,
1537
once you get familiar with the different options.
1538

    
1539
Edit ``/etc/synnefo/20-snf-cyclades-app-api.conf``:
1540

    
1541
.. code-block:: console
1542

    
1543
   ASTAKOS_URL = 'https://node1.example.com/im/authenticate'
1544

    
1545
The ``ASTAKOS_URL`` denotes the authentication endpoint for Cyclades and is set
1546
to point to Astakos (this should have the same value with Pithos+'s
1547
``PITHOS_AUTHENTICATION_URL``, setup :ref:`previously <conf-pithos>`).
1548

    
1549
TODO: Document the Network Options here
1550

    
1551
Edit ``/etc/synnefo/20-snf-cyclades-app-cloudbar.conf``:
1552

    
1553
.. code-block:: console
1554

    
1555
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
1556
   CLOUDBAR_ACTIVE_SERVICE = '2'
1557
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/im/get_services'
1558
   CLOUDBAR_MENU_URL = 'https://account.node1.example.com/im/get_menu'
1559

    
1560
``CLOUDBAR_LOCATION`` tells the client where to find the Astakos common
1561
cloudbar. The ``CLOUDBAR_SERVICES_URL`` and ``CLOUDBAR_MENU_URL`` options are
1562
used by the Cyclades Web UI to get from Astakos all the information needed to
1563
fill its own cloudbar. So, we put our Astakos deployment urls there. All the
1564
above should have the same values we put in the corresponding variables in
1565
``/etc/synnefo/20-snf-pithos-webclient-cloudbar.conf`` on the previous
1566
:ref:`Pithos configuration <conf-pithos>` section.
1567

    
1568
The ``CLOUDBAR_ACTIVE_SERVICE`` points to an already registered Astakos
1569
service. You can see all :ref:`registered services <services-reg>` by running
1570
on the Astakos node (node1):
1571

    
1572
.. code-block:: console
1573

    
1574
   # snf-manage service-list
1575

    
1576
The value of ``CLOUDBAR_ACTIVE_SERVICE`` should be the cyclades service's
1577
``id`` as shown by the above command, in our case ``2``.
1578

    
1579
Edit ``/etc/synnefo/20-snf-cyclades-app-plankton.conf``:
1580

    
1581
.. code-block:: console
1582

    
1583
   BACKEND_DB_CONNECTION = 'postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos'
1584
   BACKEND_BLOCK_PATH = '/srv/pithos/data/'
1585

    
1586
In this file we configure the Plankton Service. ``BACKEND_DB_CONNECTION``
1587
denotes the Pithos+ database (where the Image files are stored). So we set that
1588
to point to our Pithos+ database. ``BACKEND_BLOCK_PATH`` denotes the actual
1589
Pithos+ data location.
1590

    
1591
Edit ``/etc/synnefo/20-snf-cyclades-app-queues.conf``:
1592

    
1593
.. code-block:: console
1594

    
1595
   AMQP_HOSTS=["amqp://synnefo:example_rabbitmq_passw0rd@node1.example.com:5672"]
1596

    
1597
The above settings denote the Message Queue. Those settings should have the same
1598
values as in ``/etc/synnefo/10-snf-cyclades-gtools-backend.conf`` file, and
1599
reflect our :ref:`Message Queue setup <rabbitmq-setup>`.
1600

    
1601
Edit ``/etc/synnefo/20-snf-cyclades-app-ui.conf``:
1602

    
1603
.. code-block:: console
1604

    
1605
   UI_LOGIN_URL = "https://node1.example.com/im/login"
1606
   UI_LOGOUT_URL = "https://node1.example.com/im/logout"
1607

    
1608
The ``UI_LOGIN_URL`` option tells the Cyclades Web UI where to redirect users,
1609
if they are not logged in. We point that to Astakos.
1610

    
1611
The ``UI_LOGOUT_URL`` option tells the Cyclades Web UI where to redirect the
1612
user when he/she logs out. We point that to Astakos, too.
1613

    
1614
Edit ``/etc/default/vncauthproxy``:
1615

    
1616
.. code-block:: console
1617

    
1618
   CHUID="www-data:nogroup"
1619

    
1620
We have now finished with the basic Cyclades and Plankton configuration.
1621

    
1622
Database Initialization
1623
-----------------------
1624

    
1625
Once Cyclades is configured, we sync the database:
1626

    
1627
.. code-block:: console
1628

    
1629
   $ snf-manage syncdb
1630
   $ snf-manage migrate
1631

    
1632
and load the initial server flavors:
1633

    
1634
.. code-block:: console
1635

    
1636
   $ snf-manage loaddata flavors
1637

    
1638
If everything returns successfully, our database is ready.
1639

    
1640
Add the Ganeti backend
1641
----------------------
1642

    
1643
In our installation we assume that we only have one Ganeti cluster, the one we
1644
setup earlier.  At this point you have to add this backend (Ganeti cluster) to
1645
cyclades assuming that you have setup the :ref:`Rapi User <rapi-user>`
1646
correctly.
1647

    
1648
.. code-block:: console
1649

    
1650
   $ snf-manage backend-add --clustername=ganeti.example.com --user=cyclades --pass=example_rapi_passw0rd
1651

    
1652
You can see everything has been setup correctly by running:
1653

    
1654
.. code-block:: console
1655

    
1656
   $ snf-manage backend-list
1657

    
1658
If something is not set correctly, you can modify the backend with the
1659
``snf-manage backend-modify`` command. If something has gone wrong, you could
1660
modify the backend to reflect the Ganeti installation by running:
1661

    
1662
.. code-block:: console
1663

    
1664
   $ snf-manage backend-modify --clustername "ganeti.node1.example.com"
1665
                               --user=cyclades
1666
                               --pass=example_rapi_passw0rd
1667
                               1
1668

    
1669
``clustername`` denotes the Ganeti-cluster's name. We provide the corresponding
1670
domain that resolves to the master IP, than the IP itself, to ensure Cyclades
1671
can talk to Ganeti even after a Ganeti master-failover.
1672

    
1673
``user`` and ``pass`` denote the RAPI user's username and the RAPI user's
1674
password.  Once we setup the first backend to point at our Ganeti cluster, we
1675
update the Cyclades backends status by running:
1676

    
1677
.. code-block:: console
1678

    
1679
   $ snf-manage backend-update-status
1680

    
1681
Cyclades can manage multiple Ganeti backends, but for the purpose of this
1682
guide,we won't get into more detail regarding mulitple backends. If you want to
1683
learn more please see /*TODO*/.
1684

    
1685
Add a Public Network
1686
----------------------
1687

    
1688
Cyclades supports different Public Networks on different Ganeti backends.
1689
After connecting Cyclades with our Ganeti cluster, we need to setup a Public
1690
Network for this Ganeti backend (`id = 1`). The basic setup is to bridge every
1691
created NIC on a bridge. After having a bridge (e.g. br0) created in every
1692
backend node edit Synnefo setting CUSTOM_BRIDGED_BRIDGE to 'br0':
1693

    
1694
.. code-block:: console
1695

    
1696
   $ snf-manage network-create --subnet=5.6.7.0/27
1697
                               --gateway=5.6.7.1
1698
                               --subnet6=2001:648:2FFC:1322::/64
1699
                               --gateway6=2001:648:2FFC:1322::1
1700
                               --public --dhcp --type=CUSTOM_BRIDGED
1701
                               --name=public_network
1702
                               --backend-id=1
1703

    
1704
This will create the Public Network on both Cyclades and the Ganeti backend. To
1705
make sure everything was setup correctly, also run:
1706

    
1707
.. code-block:: console
1708

    
1709
   $ snf-manage reconcile-networks
1710

    
1711
You can see all available networks by running:
1712

    
1713
.. code-block:: console
1714

    
1715
   $ snf-manage network-list
1716

    
1717
and inspect each network's state by running:
1718

    
1719
.. code-block:: console
1720

    
1721
   $ snf-manage network-inspect <net_id>
1722

    
1723
Finally, you can see the networks from the Ganeti perspective by running on the
1724
Ganeti MASTER:
1725

    
1726
.. code-block:: console
1727

    
1728
   $ gnt-network list
1729
   $ gnt-network info <network_name>
1730

    
1731
Servers restart
1732
---------------
1733

    
1734
Restart gunicorn on node1:
1735

    
1736
.. code-block:: console
1737

    
1738
   # /etc/init.d/gunicorn restart
1739

    
1740
Now let's do the final connections of Cyclades with Ganeti.
1741

    
1742
``snf-dispatcher`` initialization
1743
---------------------------------
1744

    
1745
``snf-dispatcher`` dispatches all messages published to the Message Queue and
1746
manages the Cyclades database accordingly. It also initializes all exchanges. By
1747
default it is not enabled during installation of Cyclades, so let's enable it in
1748
its configuration file ``/etc/default/snf-dispatcher``:
1749

    
1750
.. code-block:: console
1751

    
1752
   SNF_DSPTCH_ENABLE=true
1753

    
1754
and start the daemon:
1755

    
1756
.. code-block:: console
1757

    
1758
   # /etc/init.d/snf-dispatcher start
1759

    
1760
You can see that everything works correctly by tailing its log file
1761
``/var/log/synnefo/dispatcher.log``.
1762

    
1763
``snf-ganeti-eventd`` on GANETI MASTER
1764
--------------------------------------
1765

    
1766
The last step of the Cyclades setup is enabling the ``snf-ganeti-eventd``
1767
daemon (part of the :ref:`Cyclades Ganeti tools <cyclades-gtools>` package).
1768
The daemon is already installed on the GANETI MASTER (node1 in our case).
1769
``snf-ganeti-eventd`` is disabled by default during the ``snf-cyclades-gtools``
1770
installation, so we enable it in its configuration file
1771
``/etc/default/snf-ganeti-eventd``:
1772

    
1773
.. code-block:: console
1774

    
1775
   SNF_EVENTD_ENABLE=true
1776

    
1777
and start the daemon:
1778

    
1779
.. code-block:: console
1780

    
1781
   # /etc/init.d/snf-ganeti-eventd start
1782

    
1783
.. warning:: Make sure you start ``snf-ganeti-eventd`` *ONLY* on GANETI MASTER
1784

    
1785
If all the above return successfully, then you have finished with the Cyclades
1786
and Plankton installation and setup. Let's test our installation now.
1787

    
1788

    
1789
Testing of Cyclades (and Plankton)
1790
==================================
1791

    
1792
Cyclades Web UI
1793
---------------
1794

    
1795
First of all we need to test that our Cyclades Web UI works correctly. Open your
1796
browser and go to the Astakos home page. Login and then click 'cyclades' on the
1797
top cloud bar. This should redirect you to:
1798

    
1799
 `http://node1.example.com/ui/`
1800

    
1801
and the Cyclades home page should appear. If not, please go back and find what
1802
went wrong. Do not proceed if you don't see the Cyclades home page.
1803

    
1804
If the Cyclades home page appears, click on the orange button 'New machine'. The
1805
first step of the 'New machine wizard' will appear. This step shows all the
1806
available Images from which you can spawn new VMs. The list should be currently
1807
empty, as we haven't registered any Images yet. Close the wizard and browse the
1808
interface (not many things to see yet). If everything seems to work, let's
1809
register our first Image file.
1810

    
1811
Cyclades Images
1812
---------------
1813

    
1814
To test our Cyclades (and Plankton) installation, we will use an Image stored on
1815
Pithos+ to spawn a new VM from the Cyclades interface. We will describe all
1816
steps, even though you may already have uploaded an Image on Pithos+ from a
1817
:ref:`previous <snf-image-images>` section:
1818

    
1819
 * Upload an Image file to Pithos+
1820
 * Register that Image file to Plankton
1821
 * Spawn a new VM from that Image from the Cyclades Web UI
1822

    
1823
We will use the `kamaki <http://docs.dev.grnet.gr/kamaki/latest/index.html>`_
1824
command line client to do the uploading and registering of the Image.
1825

    
1826
Installation of `kamaki`
1827
~~~~~~~~~~~~~~~~~~~~~~~~
1828

    
1829
You can install `kamaki` anywhere you like, since it is a standalone client of
1830
the APIs and talks to the installation over `http`. For the purpose of this
1831
guide we will assume that we have downloaded the `Debian Squeeze Base Image
1832
<https://pithos.okeanos.grnet.gr/public/9epgb>`_ and stored it under node1's
1833
``/srv/images`` directory. For that reason we will install `kamaki` on node1,
1834
too. We do this by running:
1835

    
1836
.. code-block:: console
1837

    
1838
   # apt-get install kamaki
1839

    
1840
Configuration of kamaki
1841
~~~~~~~~~~~~~~~~~~~~~~~
1842

    
1843
Now we need to setup kamaki, by adding the appropriate URLs and tokens of our
1844
installation. We do this by running:
1845

    
1846
.. code-block:: console
1847

    
1848
   $ kamaki config set astakos.url "https://node1.example.com"
1849
   $ kamaki config set compute.url="https://node1.example.com/api/v1.1"
1850
   $ kamaki config set image.url "https://node1.examle.com/plankton"
1851
   $ kamaki config set storage.url "https://node2.example.com/v1"
1852
   $ kamaki config set storage.account "user@example.com"
1853
   $ kamaki config set global.token "bdY_example_user_tokenYUff=="
1854

    
1855
The token at the last kamaki command is our user's (``user@example.com``) token,
1856
as it appears on the user's `Profile` web page on the Astakos Web UI.
1857

    
1858
You can see that the new configuration options have been applied correctly, by
1859
running:
1860

    
1861
.. code-block:: console
1862

    
1863
   $ kamaki config list
1864

    
1865
Upload an Image file to Pithos+
1866
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1867

    
1868
Now, that we have set up `kamaki` we will upload the Image that we have
1869
downloaded and stored under ``/srv/images/``. Although we can upload the Image
1870
under the root ``Pithos`` container (as you may have done when uploading the
1871
Image from the Pithos+ Web UI), we will create a new container called ``images``
1872
and store the Image under that container. We do this for two reasons:
1873

    
1874
a) To demonstrate how to create containers other than the default ``Pithos``.
1875
   This can be done only with the `kamaki` client and not through the Web UI.
1876

    
1877
b) As a best organization practise, so that you won't have your Image files
1878
   tangled along with all your other Pithos+ files and directory structures.
1879

    
1880
We create the new ``images`` container by running:
1881

    
1882
.. code-block:: console
1883

    
1884
   $ kamaki store create images
1885

    
1886
Then, we upload the Image file to that container:
1887

    
1888
.. code-block:: console
1889

    
1890
   $ kamaki store upload --container images \
1891
                         /srv/images/debian_base-6.0-7-x86_64.diskdump \
1892
                         debian_base-6.0-7-x86_64.diskdump
1893

    
1894
The first is the local path and the second is the remote path on Pithos+. If
1895
the new container and the file appears on the Pithos+ Web UI, then you have
1896
successfully created the container and uploaded the Image file.
1897

    
1898
Register an existing Image file to Plankton
1899
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1900

    
1901
Once the Image file has been successfully uploaded on Pithos+, then we register
1902
it to Plankton (so that it becomes visible to Cyclades), by running:
1903

    
1904
.. code-block:: console
1905

    
1906
   $ kamaki image register "Debian Base"
1907
                           pithos://user@examle.com/images/debian_base-6.0-7-x86_64.diskdump
1908
                           --public
1909
                           --disk-format=diskdump
1910
                           --property OSFAMILY=linux --property ROOT_PARTITION=1
1911
                           --property description="Debian Squeeze Base System"
1912
                           --property size=451 --property kernel=2.6.32 --property GUI="No GUI"
1913
                           --property sortorder=1 --property USERS=root --property OS=debian
1914

    
1915
This command registers the Pithos+ file
1916
``pithos://user@examle.com/images/debian_base-6.0-7-x86_64.diskdump`` as an
1917
Image in Plankton. This Image will be public (``--public``), so all users will
1918
be able to spawn VMs from it and is of type ``diskdump``. The first two
1919
properties (``OSFAMILY`` and ``ROOT_PARTITION``) are mandatory. All the rest
1920
properties are optional, but recommended, so that the Images appear nicely on
1921
the Cyclades Web UI. ``Debian Base`` will appear as the name of this Image. The
1922
``OS`` property's valid values may be found in the ``IMAGE_ICONS`` variable
1923
inside the ``20-snf-cyclades-app-ui.conf`` configuration file.
1924

    
1925
``OSFAMILY`` and ``ROOT_PARTITION`` are mandatory because they will be passed
1926
from Plankton to Cyclades and then to Ganeti and `snf-image` (also see
1927
:ref:`previous section <ganeti-with-pithos-images>`). All other properties are
1928
used to show information on the Cyclades UI.
1929

    
1930
Spawn a VM from the Cyclades Web UI
1931
-----------------------------------
1932

    
1933
If the registration completes successfully, then go to the Cyclades Web UI from
1934
your browser at:
1935

    
1936
 `https://node1.example.com/ui/`
1937

    
1938
Click on the 'New Machine' button and the first step of the wizard will appear.
1939
Click on 'My Images' (right after 'System' Images) on the left pane of the
1940
wizard. Your previously registered Image "Debian Base" should appear under
1941
'Available Images'. If not, something has gone wrong with the registration. Make
1942
sure you can see your Image file on the Pithos+ Web UI and ``kamaki image
1943
register`` returns successfully with all options and properties as shown above.
1944

    
1945
If the Image appears on the list, select it and complete the wizard by selecting
1946
a flavor and a name for your VM. Then finish by clicking 'Create'. Make sure you
1947
write down your password, because you *WON'T* be able to retrieve it later.
1948

    
1949
If everything was setup correctly, after a few minutes your new machine will go
1950
to state 'Running' and you will be able to use it. Click 'Console' to connect
1951
through VNC out of band, or click on the machine's icon to connect directly via
1952
SSH or RDP (for windows machines).
1953

    
1954
Congratulations. You have successfully installed the whole Synnefo stack and
1955
connected all components. Go ahead in the next section to test the Network
1956
functionality from inside Cyclades and discover even more features.
1957

    
1958

    
1959
General Testing
1960
===============
1961

    
1962

    
1963
Notes
1964
=====