Statistics
| Branch: | Tag: | Revision:

root / docs / quick-install-admin-guide.rst @ 7df5a742

History | View | Annotate | Download (70.3 kB)

1
.. _quick-install-admin-guide:
2

    
3
Administrator's Quick Installation Guide
4
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5

    
6
This is the Administrator's quick installation guide.
7

    
8
It describes how to install the whole synnefo stack on two (2) physical nodes,
9
with minimum configuration. It installs synnefo from Debian packages, and
10
assumes the nodes run Debian Squeeze. After successful installation, you will
11
have the following services running:
12

    
13
 * Identity Management (Astakos)
14
 * Object Storage Service (Pithos+)
15
 * Compute Service (Cyclades)
16
 * Image Registry Service (Plankton)
17

    
18
and a single unified Web UI to manage them all.
19

    
20
The Volume Storage Service (Archipelago) and the Billing Service (Aquarium) are
21
not released yet.
22

    
23
If you just want to install the Object Storage Service (Pithos+), follow the guide
24
and just stop after the "Testing of Pithos+" section.
25

    
26

    
27
Installation of Synnefo / Introduction
28
======================================
29

    
30
We will install the services with the above list's order. Cyclades and Plankton
31
will be installed in a single step (at the end), because at the moment they are
32
contained in the same software component. Furthermore, we will install all
33
services in the first physical node, except Pithos+ which will be installed in
34
the second, due to a conflict between the snf-pithos-app and snf-cyclades-app
35
component (scheduled to be fixed in the next version).
36

    
37
For the rest of the documentation we will refer to the first physical node as
38
"node1" and the second as "node2". We will also assume that their domain names
39
are "node1.example.com" and "node2.example.com" and their IPs are "4.3.2.1" and
40
"4.3.2.2" respectively.
41

    
42
.. note:: It is import that the two machines are under the same domain name.
43
    If they are not, you can do this by editting the file ``/etc/hosts``
44
    on both machines, and add the following lines:
45

    
46
    .. code-block:: console
47

    
48
        4.3.2.1     node1.example.com
49
        4.3.2.2     node2.example.com
50

    
51

    
52
General Prerequisites
53
=====================
54

    
55
These are the general synnefo prerequisites, that you need on node1 and node2
56
and are related to all the services (Astakos, Pithos+, Cyclades, Plankton).
57

    
58
To be able to download all synnefo components you need to add the following
59
lines in your ``/etc/apt/sources.list`` file:
60

    
61
| ``deb http://apt.dev.grnet.gr squeeze main``
62
| ``deb-src http://apt.dev.grnet.gr squeeze main``
63
| ``deb http://apt.dev.grnet.gr squeeze-backports main``
64

    
65
and import the repo's GPG key:
66

    
67
| ``curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -``
68

    
69
Also add the following line to enable the ``squeeze-backports`` repository,
70
which may provide more recent versions of certain packages. The repository
71
is deactivated by default and must be specified expicitly in ``apt-get``
72
operations:
73

    
74
| ``deb http://backports.debian.org/debian-backports squeeze-backports main``
75

    
76
You also need a shared directory visible by both nodes. Pithos+ will save all
77
data inside this directory. By 'all data', we mean files, images, and pithos
78
specific mapping data. If you plan to upload more than one basic image, this
79
directory should have at least 50GB of free space. During this guide, we will
80
assume that node1 acts as an NFS server and serves the directory ``/srv/pithos``
81
to node2 (be sure to set no_root_squash flag). Node2 has this directory
82
mounted under ``/srv/pithos``, too.
83

    
84
Before starting the synnefo installation, you will need basic third party
85
software to be installed and configured on the physical nodes. We will describe
86
each node's general prerequisites separately. Any additional configuration,
87
specific to a synnefo service for each node, will be described at the service's
88
section.
89

    
90
Finally, it is required for Cyclades and Ganeti nodes to have synchronized
91
system clocks (e.g. by running ntpd).
92

    
93
Node1
94
-----
95

    
96
General Synnefo dependencies
97
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98

    
99
 * apache (http server)
100
 * gunicorn (WSGI http server)
101
 * postgresql (database)
102
 * rabbitmq (message queue)
103
 * ntp (NTP daemon)
104

    
105
You can install apache2, progresql and ntp by running:
106

    
107
.. code-block:: console
108

    
109
   # apt-get install apache2 postgresql ntp
110

    
111
Make sure to install gunicorn >= v0.12.2. You can do this by installing from
112
the official debian backports:
113

    
114
.. code-block:: console
115

    
116
   # apt-get -t squeeze-backports install gunicorn
117

    
118
On node1, we will create our databases, so you will also need the
119
python-psycopg2 package:
120

    
121
.. code-block:: console
122

    
123
   # apt-get install python-psycopg2
124

    
125
To install RabbitMQ>=2.8.4, use the RabbitMQ APT repository by adding the
126
following line to ``/etc/apt/sources.list``:
127

    
128
.. code-block:: console
129

    
130
  deb http://www.rabbitmq.com/debian testing main
131

    
132
Add RabbitMQ public key, to trusted key list:
133

    
134
.. code-block:: console
135

    
136
  # wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
137
  # apt-key add rabbitmq-signing-key-public.asc
138

    
139
Finally, to install the package run:
140

    
141
.. code-block:: console
142

    
143
  # apt-get update
144
  # apt-get install rabbitmq-server
145

    
146
Database setup
147
~~~~~~~~~~~~~~
148

    
149
On node1, we create a database called ``snf_apps``, that will host all django
150
apps related tables. We also create the user ``synnefo`` and grant him all
151
privileges on the database. We do this by running:
152

    
153
.. code-block:: console
154

    
155
   root@node1:~ # su - postgres
156
   postgres@node1:~ $ psql
157
   postgres=# CREATE DATABASE snf_apps WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
158
   postgres=# CREATE USER synnefo WITH PASSWORD 'example_passw0rd';
159
   postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_apps TO synnefo;
160

    
161
We also create the database ``snf_pithos`` needed by the pithos+ backend and
162
grant the ``synnefo`` user all privileges on the database. This database could
163
be created on node2 instead, but we do it on node1 for simplicity. We will
164
create all needed databases on node1 and then node2 will connect to them.
165

    
166
.. code-block:: console
167

    
168
   postgres=# CREATE DATABASE snf_pithos WITH ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C' TEMPLATE=template0;
169
   postgres=# GRANT ALL PRIVILEGES ON DATABASE snf_pithos TO synnefo;
170

    
171
Configure the database to listen to all network interfaces. You can do this by
172
editting the file ``/etc/postgresql/8.4/main/postgresql.conf`` and change
173
``listen_addresses`` to ``'*'`` :
174

    
175
.. code-block:: console
176

    
177
   listen_addresses = '*'
178

    
179
Furthermore, edit ``/etc/postgresql/8.4/main/pg_hba.conf`` to allow node1 and
180
node2 to connect to the database. Add the following lines under ``#IPv4 local
181
connections:`` :
182

    
183
.. code-block:: console
184

    
185
   host		all	all	4.3.2.1/32	md5
186
   host		all	all	4.3.2.2/32	md5
187

    
188
Make sure to substitute "4.3.2.1" and "4.3.2.2" with node1's and node2's
189
actual IPs. Now, restart the server to apply the changes:
190

    
191
.. code-block:: console
192

    
193
   # /etc/init.d/postgresql restart
194

    
195
Gunicorn setup
196
~~~~~~~~~~~~~~
197

    
198
Create the file ``synnefo`` under ``/etc/gunicorn.d/`` containing the following:
199

    
200
.. code-block:: console
201

    
202
   CONFIG = {
203
    'mode': 'django',
204
    'environment': {
205
      'DJANGO_SETTINGS_MODULE': 'synnefo.settings',
206
    },
207
    'working_dir': '/etc/synnefo',
208
    'user': 'www-data',
209
    'group': 'www-data',
210
    'args': (
211
      '--bind=127.0.0.1:8080',
212
      '--workers=4',
213
      '--log-level=debug',
214
    ),
215
   }
216

    
217
.. warning:: Do NOT start the server yet, because it won't find the
218
    ``synnefo.settings`` module. We will start the server after successful
219
    installation of astakos. If the server is running::
220

    
221
       # /etc/init.d/gunicorn stop
222

    
223
Apache2 setup
224
~~~~~~~~~~~~~
225

    
226
Create the file ``synnefo`` under ``/etc/apache2/sites-available/`` containing
227
the following:
228

    
229
.. code-block:: console
230

    
231
   <VirtualHost *:80>
232
     ServerName node1.example.com
233

    
234
     RewriteEngine On
235
     RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
236
     RewriteRule ^(.*)$ - [F,L]
237
     RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
238
   </VirtualHost>
239

    
240
Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``
241
containing the following:
242

    
243
.. code-block:: console
244

    
245
   <IfModule mod_ssl.c>
246
   <VirtualHost _default_:443>
247
     ServerName node1.example.com
248

    
249
     Alias /static "/usr/share/synnefo/static"
250

    
251
   #  SetEnv no-gzip
252
   #  SetEnv dont-vary
253

    
254
     AllowEncodedSlashes On
255

    
256
     RequestHeader set X-Forwarded-Protocol "https"
257

    
258
     <Proxy * >
259
       Order allow,deny
260
       Allow from all
261
     </Proxy>
262

    
263
     SetEnv                proxy-sendchunked
264
     SSLProxyEngine        off
265
     ProxyErrorOverride    off
266

    
267
     ProxyPass        /static !
268
     ProxyPass        / http://localhost:8080/ retry=0
269
     ProxyPassReverse / http://localhost:8080/
270

    
271
     RewriteEngine On
272
     RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
273
     RewriteRule ^(.*)$ - [F,L]
274
     RewriteRule ^/login(.*) /im/login/redirect$1 [PT,NE]
275

    
276
     SSLEngine on
277
     SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
278
     SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
279
   </VirtualHost>
280
   </IfModule>
281

    
282
Now enable sites and modules by running:
283

    
284
.. code-block:: console
285

    
286
   # a2enmod ssl
287
   # a2enmod rewrite
288
   # a2dissite default
289
   # a2ensite synnefo
290
   # a2ensite synnefo-ssl
291
   # a2enmod headers
292
   # a2enmod proxy_http
293

    
294
.. warning:: Do NOT start/restart the server yet. If the server is running::
295

    
296
       # /etc/init.d/apache2 stop
297

    
298
.. _rabbitmq-setup:
299

    
300
Message Queue setup
301
~~~~~~~~~~~~~~~~~~~
302

    
303
The message queue will run on node1, so we need to create the appropriate
304
rabbitmq user. The user is named ``synnefo`` and gets full privileges on all
305
exchanges:
306

    
307
.. code-block:: console
308

    
309
   # rabbitmqctl add_user synnefo "example_rabbitmq_passw0rd"
310
   # rabbitmqctl set_permissions synnefo ".*" ".*" ".*"
311

    
312
We do not need to initialize the exchanges. This will be done automatically,
313
during the Cyclades setup.
314

    
315
Pithos+ data directory setup
316
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
317

    
318
As mentioned in the General Prerequisites section, there is a directory called
319
``/srv/pithos`` visible by both nodes. We create and setup the ``data``
320
directory inside it:
321

    
322
.. code-block:: console
323

    
324
   # cd /srv/pithos
325
   # mkdir data
326
   # chown www-data:www-data data
327
   # chmod g+ws data
328

    
329
You are now ready with all general prerequisites concerning node1. Let's go to
330
node2.
331

    
332
Node2
333
-----
334

    
335
General Synnefo dependencies
336
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
337

    
338
 * apache (http server)
339
 * gunicorn (WSGI http server)
340
 * postgresql (database)
341
 * ntp (NTP daemon)
342

    
343
You can install the above by running:
344

    
345
.. code-block:: console
346

    
347
   # apt-get install apache2 postgresql ntp
348

    
349
Make sure to install gunicorn >= v0.12.2. You can do this by installing from
350
the official debian backports:
351

    
352
.. code-block:: console
353

    
354
   # apt-get -t squeeze-backports install gunicorn
355

    
356
Node2 will connect to the databases on node1, so you will also need the
357
python-psycopg2 package:
358

    
359
.. code-block:: console
360

    
361
   # apt-get install python-psycopg2
362

    
363
Database setup
364
~~~~~~~~~~~~~~
365

    
366
All databases have been created and setup on node1, so we do not need to take
367
any action here. From node2, we will just connect to them. When you get familiar
368
with the software you may choose to run different databases on different nodes,
369
for performance/scalability/redundancy reasons, but those kind of setups are out
370
of the purpose of this guide.
371

    
372
Gunicorn setup
373
~~~~~~~~~~~~~~
374

    
375
Create the file ``synnefo`` under ``/etc/gunicorn.d/`` containing the following
376
(same contents as in node1; you can just copy/paste the file):
377

    
378
.. code-block:: console
379

    
380
   CONFIG = {
381
    'mode': 'django',
382
    'environment': {
383
      'DJANGO_SETTINGS_MODULE': 'synnefo.settings',
384
    },
385
    'working_dir': '/etc/synnefo',
386
    'user': 'www-data',
387
    'group': 'www-data',
388
    'args': (
389
      '--bind=127.0.0.1:8080',
390
      '--workers=4',
391
      '--log-level=debug',
392
      '--timeout=43200'
393
    ),
394
   }
395

    
396
.. warning:: Do NOT start the server yet, because it won't find the
397
    ``synnefo.settings`` module. We will start the server after successful
398
    installation of astakos. If the server is running::
399

    
400
       # /etc/init.d/gunicorn stop
401

    
402
Apache2 setup
403
~~~~~~~~~~~~~
404

    
405
Create the file ``synnefo`` under ``/etc/apache2/sites-available/`` containing
406
the following:
407

    
408
.. code-block:: console
409

    
410
   <VirtualHost *:80>
411
     ServerName node2.example.com
412

    
413
     RewriteEngine On
414
     RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
415
     RewriteRule ^(.*)$ - [F,L]
416
     RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
417
   </VirtualHost>
418

    
419
Create the file ``synnefo-ssl`` under ``/etc/apache2/sites-available/``
420
containing the following:
421

    
422
.. code-block:: console
423

    
424
   <IfModule mod_ssl.c>
425
   <VirtualHost _default_:443>
426
     ServerName node2.example.com
427

    
428
     Alias /static "/usr/share/synnefo/static"
429

    
430
     SetEnv no-gzip
431
     SetEnv dont-vary
432
     AllowEncodedSlashes On
433

    
434
     RequestHeader set X-Forwarded-Protocol "https"
435

    
436
     <Proxy * >
437
       Order allow,deny
438
       Allow from all
439
     </Proxy>
440

    
441
     SetEnv                proxy-sendchunked
442
     SSLProxyEngine        off
443
     ProxyErrorOverride    off
444

    
445
     ProxyPass        /static !
446
     ProxyPass        / http://localhost:8080/ retry=0
447
     ProxyPassReverse / http://localhost:8080/
448

    
449
     SSLEngine on
450
     SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
451
     SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
452
   </VirtualHost>
453
   </IfModule>
454

    
455
As in node1, enable sites and modules by running:
456

    
457
.. code-block:: console
458

    
459
   # a2enmod ssl
460
   # a2enmod rewrite
461
   # a2dissite default
462
   # a2ensite synnefo
463
   # a2ensite synnefo-ssl
464
   # a2enmod headers
465
   # a2enmod proxy_http
466

    
467
.. warning:: Do NOT start/restart the server yet. If the server is running::
468

    
469
       # /etc/init.d/apache2 stop
470

    
471
We are now ready with all general prerequisites for node2. Now that we have
472
finished with all general prerequisites for both nodes, we can start installing
473
the services. First, let's install Astakos on node1.
474

    
475

    
476
Installation of Astakos on node1
477
================================
478

    
479
To install astakos, grab the package from our repository (make sure  you made
480
the additions needed in your ``/etc/apt/sources.list`` file, as described
481
previously), by running:
482

    
483
.. code-block:: console
484

    
485
   # apt-get install snf-astakos-app
486

    
487
After successful installation of snf-astakos-app, make sure that also
488
snf-webproject has been installed (marked as "Recommended" package). By default
489
Debian installs "Recommended" packages, but if you have changed your
490
configuration and the package didn't install automatically, you should
491
explicitly install it manually running:
492

    
493
.. code-block:: console
494

    
495
   # apt-get install snf-webproject
496

    
497
The reason snf-webproject is "Recommended" and not a hard dependency, is to give
498
the experienced administrator the ability to install synnefo in a custom made
499
django project. This corner case concerns only very advanced users that know
500
what they are doing and want to experiment with synnefo.
501

    
502

    
503
.. _conf-astakos:
504

    
505
Configuration of Astakos
506
========================
507

    
508
Conf Files
509
----------
510

    
511
After astakos is successfully installed, you will find the directory
512
``/etc/synnefo`` and some configuration files inside it. The files contain
513
commented configuration options, which are the default options. While installing
514
new snf-* components, new configuration files will appear inside the directory.
515
In this guide (and for all services), we will edit only the minimum necessary
516
configuration options, to reflect our setup. Everything else will remain as is.
517

    
518
After getting familiar with synnefo, you will be able to customize the software
519
as you wish and fits your needs. Many options are available, to empower the
520
administrator with extensively customizable setups.
521

    
522
For the snf-webproject component (installed as an astakos dependency), we
523
need the following:
524

    
525
Edit ``/etc/synnefo/10-snf-webproject-database.conf``. You will need to
526
uncomment and edit the ``DATABASES`` block to reflect our database:
527

    
528
.. code-block:: console
529

    
530
   DATABASES = {
531
    'default': {
532
        # 'postgresql_psycopg2', 'postgresql','mysql', 'sqlite3' or 'oracle'
533
        'ENGINE': 'postgresql_psycopg2',
534
         # ATTENTION: This *must* be the absolute path if using sqlite3.
535
         # See: http://docs.djangoproject.com/en/dev/ref/settings/#name
536
        'NAME': 'snf_apps',
537
        'USER': 'synnefo',                      # Not used with sqlite3.
538
        'PASSWORD': 'example_passw0rd',         # Not used with sqlite3.
539
        # Set to empty string for localhost. Not used with sqlite3.
540
        'HOST': '4.3.2.1',
541
        # Set to empty string for default. Not used with sqlite3.
542
        'PORT': '5432',
543
    }
544
   }
545

    
546
Edit ``/etc/synnefo/10-snf-webproject-deploy.conf``. Uncomment and edit
547
``SECRET_KEY``. This is a django specific setting which is used to provide a
548
seed in secret-key hashing algorithms. Set this to a random string of your
549
choise and keep it private:
550

    
551
.. code-block:: console
552

    
553
   SECRET_KEY = 'sy6)mw6a7x%n)-example_secret_key#zzk4jo6f2=uqu!1o%)'
554

    
555
For astakos specific configuration, edit the following options in
556
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :
557

    
558
.. code-block:: console
559

    
560
   ASTAKOS_DEFAULT_ADMIN_EMAIL = None
561

    
562
   ASTAKOS_IM_MODULES = ['local']
563

    
564
   ASTAKOS_COOKIE_DOMAIN = '.example.com'
565

    
566
   ASTAKOS_BASEURL = 'https://node1.example.com'
567

    
568
   ASTAKOS_SITENAME = '~okeanos demo example'
569

    
570
   ASTAKOS_RECAPTCHA_ENABLED = False
571

    
572
``ASTAKOS_IM_MODULES`` refers to the astakos login methods. For now only local
573
is supported. The ``ASTAKOS_COOKIE_DOMAIN`` should be the base url of our
574
domain (for all services). ``ASTAKOS_BASEURL`` is the astakos home page.
575

    
576
``ASTAKOS_DEFAULT_ADMIN_EMAIL`` refers to the administrator's email.
577
Every time a new account is created a notification is sent to this email.
578
For this we need access to a running mail server, so we have disabled
579
it for now by setting its value to None. For more informations on this,
580
read the relative :ref:`section <mail-server>`.
581

    
582
.. note:: For the purpose of this guide, we have disabled recaptcha authentication.
583
    If you would like to enable it you have to edit the following options:
584

    
585
    .. code-block:: console
586

    
587
        ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
588
        ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
589
        ASTAKOS_RECAPTCHA_USE_SSL = True
590
        ASTAKOS_RECAPTCHA_ENABLED = True
591

    
592
    For the ``ASTAKOS_RECAPTCHA_PUBLIC_KEY`` and ``ASTAKOS_RECAPTCHA_PRIVATE_KEY``
593
    go to https://www.google.com/recaptcha/admin/create and create your own pair.
594

    
595
Then edit ``/etc/synnefo/20-snf-astakos-app-cloudbar.conf`` :
596

    
597
.. code-block:: console
598

    
599
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
600

    
601
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/im/get_services'
602

    
603
   CLOUDBAR_MENU_URL = 'https://node1.example.com/im/get_menu'
604

    
605
Those settings have to do with the black cloudbar endpoints and will be described
606
in more detail later on in this guide. For now, just edit the domain to point at
607
node1 which is where we have installed Astakos.
608

    
609
If you are an advanced user and want to use the Shibboleth Authentication method,
610
read the relative :ref:`section <shibboleth-auth>`.
611

    
612
.. note:: Because Cyclades and Astakos are running on the same machine
613
    in our example, we have to deactivate the CSRF verification. We can do so
614
    by adding to
615
    ``/etc/synnefo/99-local.conf``:
616

    
617
    .. code-block:: console
618

    
619
        MIDDLEWARE_CLASSES.remove('django.middleware.csrf.CsrfViewMiddleware')
620
        TEMPLATE_CONTEXT_PROCESSORS.remove('django.core.context_processors.csrf')
621

    
622

    
623
Database Initialization
624
-----------------------
625

    
626
After configuration is done, we initialize the database by running:
627

    
628
.. code-block:: console
629

    
630
   # snf-manage syncdb
631

    
632
At this example we don't need to create a django superuser, so we select
633
``[no]`` to the question. After a successful sync, we run the migration needed
634
for astakos:
635

    
636
.. code-block:: console
637

    
638
   # snf-manage migrate im
639

    
640
Then, we load the pre-defined user groups
641

    
642
.. code-block:: console
643

    
644
   # snf-manage loaddata groups
645

    
646
.. _services-reg:
647

    
648
Services Registration
649
---------------------
650

    
651
When the database is ready, we configure the elements of the Astakos cloudbar,
652
to point to our future services:
653

    
654
.. code-block:: console
655

    
656
   # snf-manage service-add "~okeanos home" https://node1.example.com/im/ home-icon.png
657
   # snf-manage service-add "cyclades" https://node1.example.com/ui/
658
   # snf-manage service-add "pithos+" https://node2.example.com/ui/
659

    
660
Servers Initialization
661
----------------------
662

    
663
Finally, we initialize the servers on node1:
664

    
665
.. code-block:: console
666

    
667
   root@node1:~ # /etc/init.d/gunicorn restart
668
   root@node1:~ # /etc/init.d/apache2 restart
669

    
670
We have now finished the Astakos setup. Let's test it now.
671

    
672

    
673
Testing of Astakos
674
==================
675

    
676
Open your favorite browser and go to:
677

    
678
``http://node1.example.com/im``
679

    
680
If this redirects you to ``https://node1.example.com/im`` and you can see
681
the "welcome" door of Astakos, then you have successfully setup Astakos.
682

    
683
Let's create our first user. At the homepage click the "CREATE ACCOUNT" button
684
and fill all your data at the sign up form. Then click "SUBMIT". You should now
685
see a green box on the top, which informs you that you made a successful request
686
and the request has been sent to the administrators. So far so good, let's assume
687
that you created the user with username ``user@example.com``.
688

    
689
Now we need to activate that user. Return to a command prompt at node1 and run:
690

    
691
.. code-block:: console
692

    
693
   root@node1:~ # snf-manage user-list
694

    
695
This command should show you a list with only one user; the one we just created.
696
This user should have an id with a value of ``1``. It should also have an
697
"active" status with the value of ``0`` (inactive). Now run:
698

    
699
.. code-block:: console
700

    
701
   root@node1:~ # snf-manage user-modify --set-active 1
702

    
703
This modifies the active value to ``1``, and actually activates the user.
704
When running in production, the activation is done automatically with different
705
types of moderation, that Astakos supports. You can see the moderation methods
706
(by invitation, whitelists, matching regexp, etc.) at the Astakos specific
707
documentation. In production, you can also manually activate a user, by sending
708
him/her an activation email. See how to do this at the :ref:`User
709
activation <user_activation>` section.
710

    
711
Now let's go back to the homepage. Open ``http://node1.example.com/im`` with
712
your browser again. Try to sign in using your new credentials. If the astakos
713
menu appears and you can see your profile, then you have successfully setup
714
Astakos.
715

    
716
Let's continue to install Pithos+ now.
717

    
718

    
719
Installation of Pithos+ on node2
720
================================
721

    
722
To install pithos+, grab the packages from our repository (make sure  you made
723
the additions needed in your ``/etc/apt/sources.list`` file, as described
724
previously), by running:
725

    
726
.. code-block:: console
727

    
728
   # apt-get install snf-pithos-app
729

    
730
After successful installation of snf-pithos-app, make sure that also
731
snf-webproject has been installed (marked as "Recommended" package). Refer to
732
the "Installation of Astakos on node1" section, if you don't remember why this
733
should happen. Now, install the pithos web interface:
734

    
735
.. code-block:: console
736

    
737
   # apt-get install snf-pithos-webclient
738

    
739
This package provides the standalone pithos web client. The web client is the
740
web UI for pithos+ and will be accessible by clicking "pithos+" on the Astakos
741
interface's cloudbar, at the top of the Astakos homepage.
742

    
743

    
744
.. _conf-pithos:
745

    
746
Configuration of Pithos+
747
========================
748

    
749
Conf Files
750
----------
751

    
752
After pithos+ is successfully installed, you will find the directory
753
``/etc/synnefo`` and some configuration files inside it, as you did in node1
754
after installation of astakos. Here, you will not have to change anything that
755
has to do with snf-common or snf-webproject. Everything is set at node1. You
756
only need to change settings that have to do with pithos+. Specifically:
757

    
758
Edit ``/etc/synnefo/20-snf-pithos-app-settings.conf``. There you need to set
759
this options:
760

    
761
.. code-block:: console
762

    
763
   PITHOS_BACKEND_DB_CONNECTION = 'postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos'
764

    
765
   PITHOS_BACKEND_BLOCK_PATH = '/srv/pithos/data'
766

    
767
   PITHOS_AUTHENTICATION_URL = 'https://node1.example.com/im/authenticate'
768
   PITHOS_AUTHENTICATION_USERS = None
769

    
770
   PITHOS_SERVICE_TOKEN = 'pithos_service_token22w=='
771

    
772
The ``PITHOS_BACKEND_DB_CONNECTION`` option tells to the pithos+ app where to
773
find the pithos+ backend database. Above we tell pithos+ that its database is
774
``snf_pithos`` at node1 and to connect as user ``synnefo`` with password
775
``example_passw0rd``.  All those settings where setup during node1's "Database
776
setup" section.
777

    
778
The ``PITHOS_BACKEND_BLOCK_PATH`` option tells to the pithos+ app where to find
779
the pithos+ backend data. Above we tell pithos+ to store its data under
780
``/srv/pithos/data``, which is visible by both nodes. We have already setup this
781
directory at node1's "Pithos+ data directory setup" section.
782

    
783
The ``PITHOS_AUTHENTICATION_URL`` option tells to the pithos+ app in which URI
784
is available the astakos authentication api. If not set, pithos+ tries to
785
authenticate using the ``PITHOS_AUTHENTICATION_USERS`` user pool.
786

    
787
The ``PITHOS_SERVICE_TOKEN`` should be the Pithos+ token returned by running on
788
the Astakos node (node1 in our case):
789

    
790
.. code-block:: console
791

    
792
   # snf-manage service-list
793

    
794
The token has been generated automatically during the :ref:`Pithos+ service
795
registration <services-reg>`.
796

    
797
Then we need to setup the web UI and connect it to astakos. To do so, edit
798
``/etc/synnefo/20-snf-pithos-webclient-settings.conf``:
799

    
800
.. code-block:: console
801

    
802
   PITHOS_UI_LOGIN_URL = "https://node1.example.com/im/login?next="
803
   PITHOS_UI_FEEDBACK_URL = "https://node1.example.com/im/feedback"
804

    
805
The ``PITHOS_UI_LOGIN_URL`` option tells the client where to redirect you, if
806
you are not logged in. The ``PITHOS_UI_FEEDBACK_URL`` option points at the
807
pithos+ feedback form. Astakos already provides a generic feedback form for all
808
services, so we use this one.
809

    
810
Then edit ``/etc/synnefo/20-snf-pithos-webclient-cloudbar.conf``, to connect the
811
pithos+ web UI with the astakos web UI (through the top cloudbar):
812

    
813
.. code-block:: console
814

    
815
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
816
   PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE = '3'
817
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/im/get_services'
818
   CLOUDBAR_MENU_URL = 'https://node1.example.com/im/get_menu'
819

    
820
The ``CLOUDBAR_LOCATION`` tells the client where to find the astakos common
821
cloudbar.
822

    
823
The ``PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE`` points to an already registered
824
Astakos service. You can see all :ref:`registered services <services-reg>` by
825
running on the Astakos node (node1):
826

    
827
.. code-block:: console
828

    
829
   # snf-manage service-list
830

    
831
The value of ``PITHOS_UI_CLOUDBAR_ACTIVE_SERVICE`` should be the pithos service's
832
``id`` as shown by the above command, in our case ``3``.
833

    
834
The ``CLOUDBAR_SERVICES_URL`` and ``CLOUDBAR_MENU_URL`` options are used by the
835
pithos+ web client to get from astakos all the information needed to fill its
836
own cloudbar. So we put our astakos deployment urls there.
837

    
838
Servers Initialization
839
----------------------
840

    
841
After configuration is done, we initialize the servers on node2:
842

    
843
.. code-block:: console
844

    
845
   root@node2:~ # /etc/init.d/gunicorn restart
846
   root@node2:~ # /etc/init.d/apache2 restart
847

    
848
You have now finished the Pithos+ setup. Let's test it now.
849

    
850

    
851
Testing of Pithos+
852
==================
853

    
854
Open your browser and go to the Astakos homepage:
855

    
856
``http://node1.example.com/im``
857

    
858
Login, and you will see your profile page. Now, click the "pithos+" link on the
859
top black cloudbar. If everything was setup correctly, this will redirect you
860
to:
861

    
862
``https://node2.example.com/ui``
863

    
864
and you will see the blue interface of the Pithos+ application.  Click the
865
orange "Upload" button and upload your first file. If the file gets uploaded
866
successfully, then this is your first sign of a successful Pithos+ installation.
867
Go ahead and experiment with the interface to make sure everything works
868
correctly.
869

    
870
You can also use the Pithos+ clients to sync data from your Windows PC or MAC.
871

    
872
If you don't stumble on any problems, then you have successfully installed
873
Pithos+, which you can use as a standalone File Storage Service.
874

    
875
If you would like to do more, such as:
876

    
877
 * Spawning VMs
878
 * Spawning VMs from Images stored on Pithos+
879
 * Uploading your custom Images to Pithos+
880
 * Spawning VMs from those custom Images
881
 * Registering existing Pithos+ files as Images
882
 * Connect VMs to the Internet
883
 * Create Private Networks
884
 * Add VMs to Private Networks
885

    
886
please continue with the rest of the guide.
887

    
888

    
889
Cyclades (and Plankton) Prerequisites
890
=====================================
891

    
892
Before proceeding with the Cyclades (and Plankton) installation, make sure you
893
have successfully set up Astakos and Pithos+ first, because Cyclades depends
894
on them. If you don't have a working Astakos and Pithos+ installation yet,
895
please return to the :ref:`top <quick-install-admin-guide>` of this guide.
896

    
897
Besides Astakos and Pithos+, you will also need a number of additional working
898
prerequisites, before you start the Cyclades installation.
899

    
900
Ganeti
901
------
902

    
903
`Ganeti <http://code.google.com/p/ganeti/>`_ handles the low level VM management
904
for Cyclades, so Cyclades requires a working Ganeti installation at the backend.
905
Please refer to the
906
`ganeti documentation <http://docs.ganeti.org/ganeti/2.5/html>`_ for all the
907
gory details. A successful Ganeti installation concludes with a working
908
:ref:`GANETI-MASTER <GANETI_NODES>` and a number of :ref:`GANETI-NODEs
909
<GANETI_NODES>`.
910

    
911
The above Ganeti cluster can run on different physical machines than node1 and
912
node2 and can scale independently, according to your needs.
913

    
914
For the purpose of this guide, we will assume that the :ref:`GANETI-MASTER
915
<GANETI_NODES>` runs on node1 and is VM-capable. Also, node2 is a
916
:ref:`GANETI-NODE <GANETI_NODES>` and is Master-capable and VM-capable too.
917

    
918
We highly recommend that you read the official Ganeti documentation, if you are
919
not familiar with Ganeti. If you are extremely impatient, you can result with
920
the above assumed setup by running on both nodes:
921

    
922
.. code-block:: console
923

    
924
   # apt-get install -t squeeze-backports ganeti2 ganeti-htools
925
   # modprobe drbd minor_count=255 usermode_helper=/bin/true
926

    
927
Unfortunatelly, stock Ganeti doesn't support IP pool management yet (we are
928
working hard to merge it upstream for Ganeti 2.7). Synnefo depends on the IP
929
pool functionality of Ganeti, so you have to use GRNET's patches for now. To
930
do so you have to build your own package from source. Please clone our local
931
repo:
932

    
933
.. code-block:: console
934

    
935
   # git clone https://code.grnet.gr/git/ganeti-local
936
   # cd ganeti-local
937
   # git checkout stable-2.6-ippool-hotplug-esi
938
   # git checkout debian-2.6
939

    
940
Then please check if you can complile ganeti:
941

    
942
.. code-block:: console
943

    
944
   # cd ganeti-local
945
   # ./automake.sh
946
   # ./configure
947
   # make
948

    
949
To do so you must have a correct build environment. Please refer to INSTALL
950
file in the source tree. Most of the packages needed are refered here:
951

    
952
.. code-block:: console
953

    
954
   #  apt-get install graphviz automake lvm2 ssh bridge-utils iproute iputils-arping \
955
                      ndisc6 python python-pyopenssl openssl \
956
                      python-pyparsing python-simplejson \
957
                      python-pyinotify python-pycurl socat \
958
                      python-elementtree kvm qemu-kvm \
959
                      ghc6 libghc6-json-dev libghc6-network-dev \
960
                      libghc6-parallel-dev libghc6-curl-dev \
961
                      libghc-quickcheck2-dev hscolour hlint
962
                      python-support python-paramiko \
963
                      python-fdsend python-ipaddr python-bitarray libjs-jquery fping
964

    
965
Now let try to build the package:
966

    
967
.. code-block:: console
968

    
969
   # apt-get install git-buildpackage
970
   # mkdir ../build-area
971
   # git-buildpackage --git-upstream-branch=stable-2.6-ippool-hotplug-esi \
972
                   --git-debian-branch=debian-2.6 \
973
                   --git-export=INDEX \
974
                   --git-ignore-new
975

    
976
This will create two deb packages in build-area. You should then run in both
977
nodes:
978

    
979
.. code-block:: console
980

    
981
   # dpkg -i ../build-area/snf-ganeti.*deb
982
   # dpkg -i ../build-area/ganeti-htools.*deb
983
   # apt-get install -f
984

    
985
We assume that Ganeti will use the KVM hypervisor. After installing Ganeti on
986
both nodes, choose a domain name that resolves to a valid floating IP (let's say
987
it's ``ganeti.node1.example.com``). Make sure node1 and node2 have root access
988
between each other using ssh keys and not passwords. Also, make sure there is an
989
lvm volume group named ``ganeti`` that will host your VMs' disks. Finally, setup
990
a bridge interface on the host machines (e.g: br0). Then run on node1:
991

    
992
.. code-block:: console
993

    
994
   root@node1:~ # gnt-cluster init --enabled-hypervisors=kvm --no-ssh-init \
995
                                   --no-etc-hosts --vg-name=ganeti \
996
                                   --nic-parameters link=br0 --master-netdev eth0 \
997
                                   ganeti.node1.example.com
998
   root@node1:~ # gnt-cluster modify --default-iallocator hail
999
   root@node1:~ # gnt-cluster modify --hypervisor-parameters kvm:kernel_path=
1000
   root@node1:~ # gnt-cluster modify --hypervisor-parameters kvm:vnc_bind_address=0.0.0.0
1001

    
1002
   root@node1:~ # gnt-node add --no-node-setup --master-capable=yes \
1003
                               --vm-capable=yes node2.example.com
1004
   root@node1:~ # gnt-cluster modify --disk-parameters=drbd:metavg=ganeti
1005
   root@node1:~ # gnt-group modify --disk-parameters=drbd:metavg=ganeti default
1006

    
1007
For any problems you may stumble upon installing Ganeti, please refer to the
1008
`official documentation <http://docs.ganeti.org/ganeti/2.5/html>`_. Installation
1009
of Ganeti is out of the scope of this guide.
1010

    
1011
.. _cyclades-install-snfimage:
1012

    
1013
snf-image
1014
---------
1015

    
1016
Installation
1017
~~~~~~~~~~~~
1018
For :ref:`Cyclades <cyclades>` to be able to launch VMs from specified Images,
1019
you need the :ref:`snf-image <snf-image>` OS Definition installed on *all*
1020
VM-capable Ganeti nodes. This means we need :ref:`snf-image <snf-image>` on
1021
node1 and node2. You can do this by running on *both* nodes:
1022

    
1023
.. code-block:: console
1024

    
1025
   # apt-get install snf-image-host snf-pithos-backend python-psycopg2
1026

    
1027
snf-image also needs the `snf-pithos-backend <snf-pithos-backend>`, to be able to
1028
handle image files stored on Pithos+. It also needs `python-psycopg2` to be able
1029
to access the Pithos+ database. This is why, we also install them on *all*
1030
VM-capable Ganeti nodes.
1031

    
1032
Now, you need to download and save the corresponding helper package. Please see
1033
`here <https://code.grnet.gr/projects/snf-image/files>`_ for the latest package. Let's
1034
assume that you installed snf-image-host version 0.4.4-1. Then, you need
1035
snf-image-helper v0.4.4-1 on *both* nodes:
1036

    
1037
.. code-block:: console
1038

    
1039
   # cd /var/lib/snf-image/helper/
1040
   # wget https://code.grnet.gr/attachments/download/1058/snf-image-helper_0.4.4-1_all.deb
1041

    
1042
.. warning:: Be careful: Do NOT install the snf-image-helper debian package.
1043
             Just put it under /var/lib/snf-image/helper/
1044

    
1045
Once, you have downloaded the snf-image-helper package, create the helper VM by
1046
running on *both* nodes:
1047

    
1048
.. code-block:: console
1049

    
1050
   # ln -s snf-image-helper_0.4.4-1_all.deb snf-image-helper.deb
1051
   # snf-image-update-helper
1052

    
1053
This will create all the needed files under ``/var/lib/snf-image/helper/`` for
1054
snf-image-host to run successfully.
1055

    
1056
Configuration
1057
~~~~~~~~~~~~~
1058
snf-image supports native access to Images stored on Pithos+. This means that
1059
snf-image can talk directly to the Pithos+ backend, without the need of providing
1060
a public URL. More details, are described in the next section. For now, the only
1061
thing we need to do, is configure snf-image to access our Pithos+ backend.
1062

    
1063
To do this, we need to set the corresponding variables in
1064
``/etc/default/snf-image``, to reflect our Pithos+ setup:
1065

    
1066
.. code-block:: console
1067

    
1068
   PITHOS_DB="postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos"
1069

    
1070
   PITHOS_DATA="/srv/pithos/data"
1071

    
1072
If you have installed your Ganeti cluster on different nodes than node1 and node2 make
1073
sure that ``/srv/pithos/data`` is visible by all of them.
1074

    
1075
If you would like to use Images that are also/only stored locally, you need to
1076
save them under ``IMAGE_DIR``, however this guide targets Images stored only on
1077
Pithos+.
1078

    
1079
Testing
1080
~~~~~~~
1081
You can test that snf-image is successfully installed by running on the
1082
:ref:`GANETI-MASTER <GANETI_NODES>` (in our case node1):
1083

    
1084
.. code-block:: console
1085

    
1086
   # gnt-os diagnose
1087

    
1088
This should return ``valid`` for snf-image.
1089

    
1090
If you are interested to learn more about snf-image's internals (and even use
1091
it alongside Ganeti without Synnefo), please see
1092
`here <https://code.grnet.gr/projects/snf-image/wiki>`_ for information concerning
1093
installation instructions, documentation on the design and implementation, and
1094
supported Image formats.
1095

    
1096
.. _snf-image-images:
1097

    
1098
snf-image's actual Images
1099
-------------------------
1100

    
1101
Now that snf-image is installed successfully we need to provide it with some
1102
Images. :ref:`snf-image <snf-image>` supports Images stored in ``extdump``,
1103
``ntfsdump`` or ``diskdump`` format. We recommend the use of the ``diskdump``
1104
format. For more information about snf-image's Image formats see `here
1105
<https://code.grnet.gr/projects/snf-image/wiki/Image_Format>`_.
1106

    
1107
:ref:`snf-image <snf-image>` also supports three (3) different locations for the
1108
above Images to be stored:
1109

    
1110
 * Under a local folder (usually an NFS mount, configurable as ``IMAGE_DIR`` in
1111
   :file:`/etc/default/snf-image`)
1112
 * On a remote host (accessible via a public URL e.g: http://... or ftp://...)
1113
 * On Pithos+ (accessible natively, not only by its public URL)
1114

    
1115
For the purpose of this guide, we will use the `Debian Squeeze Base Image
1116
<https://pithos.okeanos.grnet.gr/public/9epgb>`_ found on the official
1117
`snf-image page
1118
<https://code.grnet.gr/projects/snf-image/wiki#Sample-Images>`_. The image is
1119
of type ``diskdump``. We will store it in our new Pithos+ installation.
1120

    
1121
To do so, do the following:
1122

    
1123
a) Download the Image from the official snf-image page (`image link
1124
   <https://pithos.okeanos.grnet.gr/public/9epgb>`_).
1125

    
1126
b) Upload the Image to your Pithos+ installation, either using the Pithos+ Web UI
1127
   or the command line client `kamaki
1128
   <http://docs.dev.grnet.gr/kamaki/latest/index.html>`_.
1129

    
1130
Once the Image is uploaded successfully, download the Image's metadata file
1131
from the official snf-image page (`image_metadata link
1132
<https://pithos.okeanos.grnet.gr/public/gwqcv>`_). You will need it, for
1133
spawning a VM from Ganeti, in the next section.
1134

    
1135
Of course, you can repeat the procedure to upload more Images, available from the
1136
`official snf-image page
1137
<https://code.grnet.gr/projects/snf-image/wiki#Sample-Images>`_.
1138

    
1139
.. _ganeti-with-pithos-images:
1140

    
1141
Spawning a VM from a Pithos+ Image, using Ganeti
1142
------------------------------------------------
1143

    
1144
Now, it is time to test our installation so far. So, we have Astakos and
1145
Pithos+ installed, we have a working Ganeti installation, the snf-image
1146
definition installed on all VM-capable nodes and a Debian Squeeze Image on
1147
Pithos+. Make sure you also have the `metadata file
1148
<https://pithos.okeanos.grnet.gr/public/gwqcv>`_ for this image.
1149

    
1150
Run on the :ref:`GANETI-MASTER's <GANETI_NODES>` (node1) command line:
1151

    
1152
.. code-block:: console
1153

    
1154
   # gnt-instance add -o snf-image+default --os-parameters \
1155
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1156
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1157
                      testvm1
1158

    
1159
In the above command:
1160

    
1161
 * ``img_passwd``: the arbitrary root password of your new instance
1162
 * ``img_format``: set to ``diskdump`` to reflect the type of the uploaded Image
1163
 * ``img_id``: If you want to deploy an Image stored on Pithos+ (our case), this
1164
               should have the format
1165
               ``pithos://<username>/<container>/<filename>``:
1166
                * ``username``: ``user@example.com`` (defined during Astakos sign up)
1167
                * ``container``: ``pithos`` (default, if the Web UI was used)
1168
                * ``filename``: the name of file (visible also from the Web UI)
1169
 * ``img_properties``: taken from the metadata file. Used only the two mandatory
1170
                       properties ``OSFAMILY`` and ``ROOT_PARTITION``. `Learn more
1171
                       <https://code.grnet.gr/projects/snf-image/wiki/Image_Format#Image-Properties>`_
1172

    
1173
If the ``gnt-instance add`` command returns successfully, then run:
1174

    
1175
.. code-block:: console
1176

    
1177
   # gnt-instance info testvm1 | grep "console connection"
1178

    
1179
to find out where to connect using VNC. If you can connect successfully and can
1180
login to your new instance using the root password ``my_vm_example_passw0rd``,
1181
then everything works as expected and you have your new Debian Base VM up and
1182
running.
1183

    
1184
If ``gnt-instance add`` fails, make sure that snf-image is correctly configured
1185
to access the Pithos+ database and the Pithos+ backend data. Also, make sure
1186
you gave the correct ``img_id`` and ``img_properties``. If ``gnt-instance add``
1187
succeeds but you cannot connect, again find out what went wrong. Do *NOT*
1188
proceed to the next steps unless you are sure everything works till this point.
1189

    
1190
If everything works, you have successfully connected Ganeti with Pithos+. Let's
1191
move on to networking now.
1192

    
1193
.. warning::
1194
    You can bypass the networking sections and go straight to
1195
    :ref:`Cyclades Ganeti tools <cyclades-gtools>`, if you do not want to setup
1196
    the Cyclades Network Service, but only the Cyclades Compute Service
1197
    (recommended for now).
1198

    
1199
Networking Setup Overview
1200
-------------------------
1201

    
1202
This part is deployment-specific and must be customized based on the specific
1203
needs of the system administrator. However, to do so, the administrator needs
1204
to understand how each level handles Virtual Networks, to be able to setup the
1205
backend appropriately, before installing Cyclades. To do so, please read the
1206
:ref:`Network <networks>` section before proceeding.
1207

    
1208
Since synnefo 0.11 all network actions are managed with the snf-manage
1209
network-* commands. This needs the underlying setup (Ganeti, nfdhcpd,
1210
snf-network, bridges, vlans) to be already configured correctly. The only
1211
actions needed in this point are:
1212

    
1213
a) Have Ganeti with IP pool management support installed.
1214

    
1215
b) Install :ref:`snf-network <snf-network>`, which provides a synnefo specific kvm-ifup script, etc.
1216

    
1217
c) Install :ref:`nfdhcpd <nfdhcpd>`, which serves DHCP requests of the VMs.
1218

    
1219
In order to test that everything is setup correctly before installing Cyclades,
1220
we will make some testing actions in this section, and the actual setup will be
1221
done afterwards with snf-manage commands.
1222

    
1223
.. _snf-network:
1224

    
1225
snf-network
1226
~~~~~~~~~~~
1227

    
1228
snf-network includes `kvm-vif-bridge` script that is invoked every time
1229
a tap (a VM's NIC) is created. Based on environment variables passed by
1230
Ganeti it issues various commands depending on the network type the NIC is
1231
connected to and sets up a corresponding dhcp lease.
1232

    
1233
Install snf-network on all Ganeti nodes:
1234

    
1235
.. code-block:: console
1236

    
1237
   # apt-get install snf-network
1238

    
1239
Then, in :file:`/etc/default/snf-network` set:
1240

    
1241
.. code-block:: console
1242

    
1243
   MAC_MASK=ff:ff:f0:00:00:00
1244

    
1245
.. _nfdhcpd:
1246

    
1247
nfdhcpd
1248
~~~~~~~
1249

    
1250
Each NIC's IP is chosen by Ganeti (with IP pool management support).
1251
`kvm-vif-bridge` script sets up dhcp leases and when the VM boots and
1252
makes a dhcp request, iptables will mangle the packet and `nfdhcpd` will
1253
create a dhcp response.
1254

    
1255
.. code-block:: console
1256

    
1257
   # apt-get install nfqueue-bindings-python=0.3+physindev-1
1258
   # apt-get install nfdhcpd
1259

    
1260
Edit ``/etc/nfdhcpd/nfdhcpd.conf`` to reflect your network configuration. At
1261
least, set the ``dhcp_queue`` variable to ``42`` and the ``nameservers``
1262
variable to your DNS IP/s. Those IPs will be passed as the DNS IP/s of your new
1263
VMs. Once you are finished, restart the server on all nodes:
1264

    
1265
.. code-block:: console
1266

    
1267
   # /etc/init.d/nfdhcpd restart
1268

    
1269
If you are using ``ferm``, then you need to run the following:
1270

    
1271
.. code-block:: console
1272

    
1273
   # echo "@include 'nfdhcpd.ferm';" >> /etc/ferm/ferm.conf
1274
   # /etc/init.d/ferm restart
1275

    
1276
or make sure to run after boot:
1277

    
1278
.. code-block:: console
1279

    
1280
   # iptables -t mangle -A PREROUTING -p udp -m udp --dport 67 -j NFQUEUE --queue-num 42
1281

    
1282
and if you have IPv6 enabled:
1283

    
1284
.. code-block:: console
1285

    
1286
   # ip6tables -t mangle -A PREROUTING -p ipv6-icmp -m icmp6 --icmpv6-type 133 -j NFQUEUE --queue-num 43
1287
   # ip6tables -t mangle -A PREROUTING -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j NFQUEUE --queue-num 44
1288

    
1289
You can check which clients are currently served by nfdhcpd by running:
1290

    
1291
.. code-block:: console
1292

    
1293
   # kill -SIGUSR1 `cat /var/run/nfdhcpd/nfdhcpd.pid`
1294

    
1295
When you run the above, then check ``/var/log/nfdhcpd/nfdhcpd.log``.
1296

    
1297
Public Network Setup
1298
--------------------
1299

    
1300
To achieve basic networking the simplest way is to have a common bridge (e.g.
1301
``br0``, on the same collision domain with the router) where all VMs will connect
1302
to. Packets will be "forwarded" to the router and then to the Internet. If
1303
you want a more advanced setup (ip-less routing and proxy-arp plese refer to
1304
:ref:`Network <networks>` section).
1305

    
1306
Physical Host Setup
1307
~~~~~~~~~~~~~~~~~~~
1308

    
1309
Assuming ``eth0`` on both hosts is the public interface (directly connected
1310
to the router), run on every node:
1311

    
1312
.. code-block:: console
1313

    
1314
   # brctl addbr br0
1315
   # ip link set br0 up
1316
   # vconfig add eth0 100
1317
   # ip link set eth0.100 up
1318
   # brctl addif br0 eth0.100
1319

    
1320

    
1321
Testing a Public Network
1322
~~~~~~~~~~~~~~~~~~~~~~~~
1323

    
1324
Let's assume, that you want to assign IPs from the ``5.6.7.0/27`` range to you
1325
new VMs, with ``5.6.7.1`` as the router's gateway. In Ganeti you can add the
1326
network by running:
1327

    
1328
.. code-block:: console
1329

    
1330
   # gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network-type=public --tags=nfdhcpd test-net-public
1331

    
1332
Then, connect the network to all your nodegroups. We assume that we only have
1333
one nodegroup (``default``) in our Ganeti cluster:
1334

    
1335
.. code-block:: console
1336

    
1337
   # gnt-network connect test-net-public default bridged br0
1338

    
1339
Now, it is time to test that the backend infrastracture is correctly setup for
1340
the Public Network. We will add a new VM, the same way we did it on the
1341
previous testing section. However, now will also add one NIC, configured to be
1342
managed from our previously defined network. Run on the GANETI-MASTER (node1):
1343

    
1344
.. code-block:: console
1345

    
1346
   # gnt-instance add -o snf-image+default --os-parameters \
1347
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1348
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1349
                      --net 0:ip=pool,network=test-net-public \
1350
                      testvm2
1351

    
1352
If the above returns successfully, connect to the new VM and run:
1353

    
1354
.. code-block:: console
1355

    
1356
   root@testvm2:~ # ip addr
1357
   root@testvm2:~ # ip route
1358
   root@testvm2:~ # cat /etc/resolv.conf
1359

    
1360
to check IP address (5.6.7.2), IP routes (default via 5.6.7.1) and DNS config
1361
(nameserver option in nfdhcpd.conf). This shows correct configuration of
1362
ganeti, snf-network and nfdhcpd.
1363

    
1364
Now ping the outside world. If this works too, then you have also configured
1365
correctly your physical host and router.
1366

    
1367
Make sure everything works as expected, before proceeding with the Private
1368
Networks setup.
1369

    
1370
.. _private-networks-setup:
1371

    
1372
Private Networks Setup
1373
----------------------
1374

    
1375
Synnefo supports two types of private networks:
1376

    
1377
 - based on MAC filtering
1378
 - based on physical VLANs
1379

    
1380
Both types provide Layer 2 isolation to the end-user.
1381

    
1382
For the first type a common bridge (e.g. ``prv0``) is needed while for the second a
1383
range of bridges (e.g. ``prv1..prv100``) each bridged on a different physical
1384
VLAN. To this end to assure isolation among end-users' private networks each
1385
has to have different MAC prefix (for the filtering to take place) or to be
1386
"connected" to a different bridge (VLAN actually).
1387

    
1388
Physical Host Setup
1389
~~~~~~~~~~~~~~~~~~~
1390

    
1391
In order to create the necessary VLAN/bridges, one for MAC filtered private
1392
networks and various (e.g. 20) for private networks based on physical VLANs,
1393
run on every node:
1394

    
1395
Assuming ``eth0`` of both hosts are somehow (via cable/switch with VLANs
1396
configured correctly) connected together, run on every node:
1397

    
1398
.. code-block:: console
1399

    
1400
   # apt-get install vlan
1401
   # modprobe 8021q
1402
   # $iface=eth0
1403
   # for prv in $(seq 0 20); do
1404
	vlan=$prv
1405
	bridge=prv$prv
1406
	vconfig add $iface $vlan
1407
	ifconfig $iface.$vlan up
1408
	brctl addbr $bridge
1409
	brctl setfd $bridge 0
1410
	brctl addif $bridge $iface.$vlan
1411
	ifconfig $bridge up
1412
      done
1413

    
1414
The above will do the following :
1415

    
1416
 * provision 21 new bridges: ``prv0`` - ``prv20``
1417
 * provision 21 new vlans: ``eth0.0`` - ``eth0.20``
1418
 * add the corresponding vlan to the equivalent bridge
1419

    
1420
You can run ``brctl show`` on both nodes to see if everything was setup
1421
correctly.
1422

    
1423
Testing the Private Networks
1424
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1425

    
1426
To test the Private Networks, we will create two instances and put them in the
1427
same Private Networks (one MAC Filtered and one Physical VLAN). This means
1428
that the instances will have a second NIC connected to the ``prv0``
1429
pre-provisioned bridge and a third to ``prv1``.
1430

    
1431
We run the same command as in the Public Network testing section, but with one
1432
more argument for the second NIC:
1433

    
1434
.. code-block:: console
1435

    
1436
   # gnt-network add --network=192.168.1.0/24 --mac-prefix=aa:00:55 --network-type=private --tags=nfdhcpd,private-filtered test-net-prv-mac
1437
   # gnt-network connect test-net-prv-mac default bridged prv0
1438

    
1439
   # gnt-network add --network=10.0.0.0/24 --tags=nfdhcpd --network-type=private test-net-prv-vlan
1440
   # gnt-network connect test-net-prv-vlan default bridged prv1
1441

    
1442
   # gnt-instance add -o snf-image+default --os-parameters \
1443
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1444
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1445
                      --net 0:ip=pool,network=test-net-public \
1446
                      --net 1:ip=pool,network=test-net-prv-mac \
1447
                      --net 2:ip=none,network=test-net-prv-vlan \
1448
                      testvm3
1449

    
1450
   # gnt-instance add -o snf-image+default --os-parameters \
1451
                      img_passwd=my_vm_example_passw0rd,img_format=diskdump,img_id="pithos://user@example.com/pithos/debian_base-6.0-7-x86_64.diskdump",img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' \
1452
                      -t plain --disk 0:size=2G --no-name-check --no-ip-check \
1453
                      --net 0:ip=pool,network=test-net-public \
1454
                      --net 1:ip=pool,network=test-net-prv-mac \
1455
                      --net 2:ip=none,network=test-net-prv-vlan \
1456
                      testvm4
1457

    
1458
Above, we create two instances with first NIC connected to the internet, their
1459
second NIC connected to a MAC filtered private Network and their third NIC
1460
connected to the first Physical VLAN Private Network. Now, connect to the
1461
instances using VNC and make sure everything works as expected:
1462

    
1463
 a) The instances have access to the public internet through their first eth
1464
    interface (``eth0``), which has been automatically assigned a public IP.
1465

    
1466
 b) ``eth1`` will have mac prefix ``aa:00:55``, while ``eth2`` default one (``aa:00:00``)
1467

    
1468
 c) ip link set ``eth1``/``eth2`` up
1469

    
1470
 d) dhclient ``eth1``/``eth2``
1471

    
1472
 e) On testvm3  ping 192.168.1.2/10.0.0.2
1473

    
1474
If everything works as expected, then you have finished the Network Setup at the
1475
backend for both types of Networks (Public & Private).
1476

    
1477
.. _cyclades-gtools:
1478

    
1479
Cyclades Ganeti tools
1480
---------------------
1481

    
1482
In order for Ganeti to be connected with Cyclades later on, we need the
1483
`Cyclades Ganeti tools` available on all Ganeti nodes (node1 & node2 in our
1484
case). You can install them by running in both nodes:
1485

    
1486
.. code-block:: console
1487

    
1488
   # apt-get install snf-cyclades-gtools
1489

    
1490
This will install the following:
1491

    
1492
 * ``snf-ganeti-eventd`` (daemon to publish Ganeti related messages on RabbitMQ)
1493
 * ``snf-ganeti-hook`` (all necessary hooks under ``/etc/ganeti/hooks``)
1494
 * ``snf-progress-monitor`` (used by ``snf-image`` to publish progress messages)
1495

    
1496
Configure ``snf-cyclades-gtools``
1497
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1498

    
1499
The package will install the ``/etc/synnefo/10-snf-cyclades-gtools-backend.conf``
1500
configuration file. At least we need to set the RabbitMQ endpoint for all tools
1501
that need it:
1502

    
1503
.. code-block:: console
1504

    
1505
   AMQP_HOSTS=["amqp://synnefo:example_rabbitmq_passw0rd@node1.example.com:5672"]
1506

    
1507
The above variables should reflect your :ref:`Message Queue setup
1508
<rabbitmq-setup>`. This file should be editted in all Ganeti nodes.
1509

    
1510
Connect ``snf-image`` with ``snf-progress-monitor``
1511
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1512

    
1513
Finally, we need to configure ``snf-image`` to publish progress messages during
1514
the deployment of each Image. To do this, we edit ``/etc/default/snf-image`` and
1515
set the corresponding variable to ``snf-progress-monitor``:
1516

    
1517
.. code-block:: console
1518

    
1519
   PROGRESS_MONITOR="snf-progress-monitor"
1520

    
1521
This file should be editted in all Ganeti nodes.
1522

    
1523
.. _rapi-user:
1524

    
1525
Synnefo RAPI user
1526
-----------------
1527

    
1528
As a last step before installing Cyclades, create a new RAPI user that will
1529
have ``write`` access. Cyclades will use this user to issue commands to Ganeti,
1530
so we will call the user ``cyclades`` with password ``example_rapi_passw0rd``.
1531
You can do this, by first running:
1532

    
1533
.. code-block:: console
1534

    
1535
   # echo -n 'cyclades:Ganeti Remote API:example_rapi_passw0rd' | openssl md5
1536

    
1537
and then putting the output in ``/var/lib/ganeti/rapi/users`` as follows:
1538

    
1539
.. code-block:: console
1540

    
1541
   cyclades {HA1}55aec7050aa4e4b111ca43cb505a61a0 write
1542

    
1543
More about Ganeti's RAPI users `here.
1544
<http://docs.ganeti.org/ganeti/2.5/html/rapi.html#introduction>`_
1545

    
1546
You have now finished with all needed Prerequisites for Cyclades (and
1547
Plankton). Let's move on to the actual Cyclades installation.
1548

    
1549

    
1550
Installation of Cyclades (and Plankton) on node1
1551
================================================
1552

    
1553
This section describes the installation of Cyclades. Cyclades is Synnefo's
1554
Compute service. Plankton (the Image Registry service) will get installed
1555
automatically along with Cyclades, because it is contained in the same Synnefo
1556
component right now.
1557

    
1558
We will install Cyclades (and Plankton) on node1. To do so, we install the
1559
corresponding package by running on node1:
1560

    
1561
.. code-block:: console
1562

    
1563
   # apt-get install snf-cyclades-app
1564

    
1565
.. warning:: Make sure you have installed ``python-gevent`` version >= 0.13.6.
1566
    This version is available at squeeze-backports and can be installed by
1567
    running: ``apt-get install -t squeeze-backports python-gevent``
1568

    
1569
If all packages install successfully, then Cyclades and Plankton are installed
1570
and we proceed with their configuration.
1571

    
1572

    
1573
Configuration of Cyclades (and Plankton)
1574
========================================
1575

    
1576
Conf files
1577
----------
1578

    
1579
After installing Cyclades, a number of new configuration files will appear under
1580
``/etc/synnefo/`` prefixed with ``20-snf-cyclades-app-``. We will descibe here
1581
only the minimal needed changes to result with a working system. In general, sane
1582
defaults have been chosen for the most of the options, to cover most of the
1583
common scenarios. However, if you want to tweak Cyclades feel free to do so,
1584
once you get familiar with the different options.
1585

    
1586
Edit ``/etc/synnefo/20-snf-cyclades-app-api.conf``:
1587

    
1588
.. code-block:: console
1589

    
1590
   ASTAKOS_URL = 'https://node1.example.com/im/authenticate'
1591

    
1592
The ``ASTAKOS_URL`` denotes the authentication endpoint for Cyclades and is set
1593
to point to Astakos (this should have the same value with Pithos+'s
1594
``PITHOS_AUTHENTICATION_URL``, setup :ref:`previously <conf-pithos>`).
1595

    
1596
TODO: Document the Network Options here
1597

    
1598
Edit ``/etc/synnefo/20-snf-cyclades-app-cloudbar.conf``:
1599

    
1600
.. code-block:: console
1601

    
1602
   CLOUDBAR_LOCATION = 'https://node1.example.com/static/im/cloudbar/'
1603
   CLOUDBAR_ACTIVE_SERVICE = '2'
1604
   CLOUDBAR_SERVICES_URL = 'https://node1.example.com/im/get_services'
1605
   CLOUDBAR_MENU_URL = 'https://account.node1.example.com/im/get_menu'
1606

    
1607
``CLOUDBAR_LOCATION`` tells the client where to find the Astakos common
1608
cloudbar. The ``CLOUDBAR_SERVICES_URL`` and ``CLOUDBAR_MENU_URL`` options are
1609
used by the Cyclades Web UI to get from Astakos all the information needed to
1610
fill its own cloudbar. So, we put our Astakos deployment urls there. All the
1611
above should have the same values we put in the corresponding variables in
1612
``/etc/synnefo/20-snf-pithos-webclient-cloudbar.conf`` on the previous
1613
:ref:`Pithos configuration <conf-pithos>` section.
1614

    
1615
The ``CLOUDBAR_ACTIVE_SERVICE`` points to an already registered Astakos
1616
service. You can see all :ref:`registered services <services-reg>` by running
1617
on the Astakos node (node1):
1618

    
1619
.. code-block:: console
1620

    
1621
   # snf-manage service-list
1622

    
1623
The value of ``CLOUDBAR_ACTIVE_SERVICE`` should be the cyclades service's
1624
``id`` as shown by the above command, in our case ``2``.
1625

    
1626
Edit ``/etc/synnefo/20-snf-cyclades-app-plankton.conf``:
1627

    
1628
.. code-block:: console
1629

    
1630
   BACKEND_DB_CONNECTION = 'postgresql://synnefo:example_passw0rd@node1.example.com:5432/snf_pithos'
1631
   BACKEND_BLOCK_PATH = '/srv/pithos/data/'
1632

    
1633
In this file we configure the Plankton Service. ``BACKEND_DB_CONNECTION``
1634
denotes the Pithos+ database (where the Image files are stored). So we set that
1635
to point to our Pithos+ database. ``BACKEND_BLOCK_PATH`` denotes the actual
1636
Pithos+ data location.
1637

    
1638
Edit ``/etc/synnefo/20-snf-cyclades-app-queues.conf``:
1639

    
1640
.. code-block:: console
1641

    
1642
   AMQP_HOSTS=["amqp://synnefo:example_rabbitmq_passw0rd@node1.example.com:5672"]
1643

    
1644
The above settings denote the Message Queue. Those settings should have the same
1645
values as in ``/etc/synnefo/10-snf-cyclades-gtools-backend.conf`` file, and
1646
reflect our :ref:`Message Queue setup <rabbitmq-setup>`.
1647

    
1648
Edit ``/etc/synnefo/20-snf-cyclades-app-ui.conf``:
1649

    
1650
.. code-block:: console
1651

    
1652
   UI_LOGIN_URL = "https://node1.example.com/im/login"
1653
   UI_LOGOUT_URL = "https://node1.example.com/im/logout"
1654

    
1655
The ``UI_LOGIN_URL`` option tells the Cyclades Web UI where to redirect users,
1656
if they are not logged in. We point that to Astakos.
1657

    
1658
The ``UI_LOGOUT_URL`` option tells the Cyclades Web UI where to redirect the
1659
user when he/she logs out. We point that to Astakos, too.
1660

    
1661
Edit ``/etc/default/vncauthproxy``:
1662

    
1663
.. code-block:: console
1664

    
1665
   CHUID="www-data:nogroup"
1666

    
1667
We have now finished with the basic Cyclades and Plankton configuration.
1668

    
1669
Database Initialization
1670
-----------------------
1671

    
1672
Once Cyclades is configured, we sync the database:
1673

    
1674
.. code-block:: console
1675

    
1676
   $ snf-manage syncdb
1677
   $ snf-manage migrate
1678

    
1679
and load the initial server flavors:
1680

    
1681
.. code-block:: console
1682

    
1683
   $ snf-manage loaddata flavors
1684

    
1685
If everything returns successfully, our database is ready.
1686

    
1687
Add the Ganeti backend
1688
----------------------
1689

    
1690
In our installation we assume that we only have one Ganeti cluster, the one we
1691
setup earlier.  At this point you have to add this backend (Ganeti cluster) to
1692
cyclades assuming that you have setup the :ref:`Rapi User <rapi-user>`
1693
correctly.
1694

    
1695
.. code-block:: console
1696

    
1697
   $ snf-manage backend-add --clustername=ganeti.node1.example.com --user=cyclades --pass=example_rapi_passw0rd
1698

    
1699
You can see everything has been setup correctly by running:
1700

    
1701
.. code-block:: console
1702

    
1703
   $ snf-manage backend-list
1704

    
1705
If something is not set correctly, you can modify the backend with the
1706
``snf-manage backend-modify`` command. If something has gone wrong, you could
1707
modify the backend to reflect the Ganeti installation by running:
1708

    
1709
.. code-block:: console
1710

    
1711
   $ snf-manage backend-modify --clustername "ganeti.node1.example.com"
1712
                               --user=cyclades
1713
                               --pass=example_rapi_passw0rd
1714
                               1
1715

    
1716
``clustername`` denotes the Ganeti-cluster's name. We provide the corresponding
1717
domain that resolves to the master IP, than the IP itself, to ensure Cyclades
1718
can talk to Ganeti even after a Ganeti master-failover.
1719

    
1720
``user`` and ``pass`` denote the RAPI user's username and the RAPI user's
1721
password.  Once we setup the first backend to point at our Ganeti cluster, we
1722
update the Cyclades backends status by running:
1723

    
1724
.. code-block:: console
1725

    
1726
   $ snf-manage backend-update-status
1727

    
1728
Cyclades can manage multiple Ganeti backends, but for the purpose of this
1729
guide,we won't get into more detail regarding mulitple backends. If you want to
1730
learn more please see /*TODO*/.
1731

    
1732
Add a Public Network
1733
----------------------
1734

    
1735
Cyclades supports different Public Networks on different Ganeti backends.
1736
After connecting Cyclades with our Ganeti cluster, we need to setup a Public
1737
Network for this Ganeti backend (`id = 1`). The basic setup is to bridge every
1738
created NIC on a bridge. After having a bridge (e.g. br0) created in every
1739
backend node edit Synnefo setting CUSTOM_BRIDGED_BRIDGE to 'br0':
1740

    
1741
.. code-block:: console
1742

    
1743
   $ snf-manage network-create --subnet=5.6.7.0/27 \
1744
                               --gateway=5.6.7.1 \
1745
                               --subnet6=2001:648:2FFC:1322::/64 \
1746
                               --gateway6=2001:648:2FFC:1322::1 \
1747
                               --public --dhcp --type=CUSTOM_BRIDGED \
1748
                               --name=public_network \
1749
                               --backend-id=1
1750

    
1751
This will create the Public Network on both Cyclades and the Ganeti backend. To
1752
make sure everything was setup correctly, also run:
1753

    
1754
.. code-block:: console
1755

    
1756
   $ snf-manage reconcile-networks
1757

    
1758
You can see all available networks by running:
1759

    
1760
.. code-block:: console
1761

    
1762
   $ snf-manage network-list
1763

    
1764
and inspect each network's state by running:
1765

    
1766
.. code-block:: console
1767

    
1768
   $ snf-manage network-inspect <net_id>
1769

    
1770
Finally, you can see the networks from the Ganeti perspective by running on the
1771
Ganeti MASTER:
1772

    
1773
.. code-block:: console
1774

    
1775
   $ gnt-network list
1776
   $ gnt-network info <network_name>
1777

    
1778

    
1779
Create pools for Private Networks
1780
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1781

    
1782
To prevent duplicate assignment of resources to different private networks,
1783
Cyclades supports two types of pools:
1784

    
1785
 - MAC prefix Pool
1786
 - Bridge Pool
1787

    
1788
As long as those resourses have been provisioned, admin has to define two
1789
these pools in Synnefo:
1790

    
1791

    
1792
.. code-block:: console
1793

    
1794
   root@testvm1:~ # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
1795

    
1796
   root@testvm1:~ # snf-manage pool-create --type=bridge --base=prv --size=20
1797

    
1798
Also, change the Synnefo setting in :file:`20-snf-cyclades-app-api.conf`:
1799

    
1800
.. code-block:: console
1801

    
1802
   PRIVATE_MAC_FILTERED_BRIDGE = 'prv0'
1803

    
1804
Servers restart
1805
---------------
1806

    
1807
Restart gunicorn on node1:
1808

    
1809
.. code-block:: console
1810

    
1811
   # /etc/init.d/gunicorn restart
1812

    
1813
Now let's do the final connections of Cyclades with Ganeti.
1814

    
1815
``snf-dispatcher`` initialization
1816
---------------------------------
1817

    
1818
``snf-dispatcher`` dispatches all messages published to the Message Queue and
1819
manages the Cyclades database accordingly. It also initializes all exchanges. By
1820
default it is not enabled during installation of Cyclades, so let's enable it in
1821
its configuration file ``/etc/default/snf-dispatcher``:
1822

    
1823
.. code-block:: console
1824

    
1825
   SNF_DSPTCH_ENABLE=true
1826

    
1827
and start the daemon:
1828

    
1829
.. code-block:: console
1830

    
1831
   # /etc/init.d/snf-dispatcher start
1832

    
1833
You can see that everything works correctly by tailing its log file
1834
``/var/log/synnefo/dispatcher.log``.
1835

    
1836
``snf-ganeti-eventd`` on GANETI MASTER
1837
--------------------------------------
1838

    
1839
The last step of the Cyclades setup is enabling the ``snf-ganeti-eventd``
1840
daemon (part of the :ref:`Cyclades Ganeti tools <cyclades-gtools>` package).
1841
The daemon is already installed on the GANETI MASTER (node1 in our case).
1842
``snf-ganeti-eventd`` is disabled by default during the ``snf-cyclades-gtools``
1843
installation, so we enable it in its configuration file
1844
``/etc/default/snf-ganeti-eventd``:
1845

    
1846
.. code-block:: console
1847

    
1848
   SNF_EVENTD_ENABLE=true
1849

    
1850
and start the daemon:
1851

    
1852
.. code-block:: console
1853

    
1854
   # /etc/init.d/snf-ganeti-eventd start
1855

    
1856
.. warning:: Make sure you start ``snf-ganeti-eventd`` *ONLY* on GANETI MASTER
1857

    
1858
If all the above return successfully, then you have finished with the Cyclades
1859
and Plankton installation and setup. Let's test our installation now.
1860

    
1861

    
1862
Testing of Cyclades (and Plankton)
1863
==================================
1864

    
1865
Cyclades Web UI
1866
---------------
1867

    
1868
First of all we need to test that our Cyclades Web UI works correctly. Open your
1869
browser and go to the Astakos home page. Login and then click 'cyclades' on the
1870
top cloud bar. This should redirect you to:
1871

    
1872
 `http://node1.example.com/ui/`
1873

    
1874
and the Cyclades home page should appear. If not, please go back and find what
1875
went wrong. Do not proceed if you don't see the Cyclades home page.
1876

    
1877
If the Cyclades home page appears, click on the orange button 'New machine'. The
1878
first step of the 'New machine wizard' will appear. This step shows all the
1879
available Images from which you can spawn new VMs. The list should be currently
1880
empty, as we haven't registered any Images yet. Close the wizard and browse the
1881
interface (not many things to see yet). If everything seems to work, let's
1882
register our first Image file.
1883

    
1884
Cyclades Images
1885
---------------
1886

    
1887
To test our Cyclades (and Plankton) installation, we will use an Image stored on
1888
Pithos+ to spawn a new VM from the Cyclades interface. We will describe all
1889
steps, even though you may already have uploaded an Image on Pithos+ from a
1890
:ref:`previous <snf-image-images>` section:
1891

    
1892
 * Upload an Image file to Pithos+
1893
 * Register that Image file to Plankton
1894
 * Spawn a new VM from that Image from the Cyclades Web UI
1895

    
1896
We will use the `kamaki <http://docs.dev.grnet.gr/kamaki/latest/index.html>`_
1897
command line client to do the uploading and registering of the Image.
1898

    
1899
Installation of `kamaki`
1900
~~~~~~~~~~~~~~~~~~~~~~~~
1901

    
1902
You can install `kamaki` anywhere you like, since it is a standalone client of
1903
the APIs and talks to the installation over `http`. For the purpose of this
1904
guide we will assume that we have downloaded the `Debian Squeeze Base Image
1905
<https://pithos.okeanos.grnet.gr/public/9epgb>`_ and stored it under node1's
1906
``/srv/images`` directory. For that reason we will install `kamaki` on node1,
1907
too. We do this by running:
1908

    
1909
.. code-block:: console
1910

    
1911
   # apt-get install kamaki
1912

    
1913
Configuration of kamaki
1914
~~~~~~~~~~~~~~~~~~~~~~~
1915

    
1916
Now we need to setup kamaki, by adding the appropriate URLs and tokens of our
1917
installation. We do this by running:
1918

    
1919
.. code-block:: console
1920

    
1921
   $ kamaki config set astakos.url "https://node1.example.com"
1922
   $ kamaki config set compute.url "https://node1.example.com/api/v1.1"
1923
   $ kamaki config set image.url "https://node1.example.com/plankton"
1924
   $ kamaki config set store.url "https://node2.example.com/v1"
1925
   $ kamaki config set global.account "user@example.com"
1926
   $ kamaki config set global.token "bdY_example_user_tokenYUff=="
1927

    
1928
The token at the last kamaki command is our user's (``user@example.com``) token,
1929
as it appears on the user's `Profile` web page on the Astakos Web UI.
1930

    
1931
You can see that the new configuration options have been applied correctly, by
1932
running:
1933

    
1934
.. code-block:: console
1935

    
1936
   $ kamaki config list
1937

    
1938
Upload an Image file to Pithos+
1939
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1940

    
1941
Now, that we have set up `kamaki` we will upload the Image that we have
1942
downloaded and stored under ``/srv/images/``. Although we can upload the Image
1943
under the root ``Pithos`` container (as you may have done when uploading the
1944
Image from the Pithos+ Web UI), we will create a new container called ``images``
1945
and store the Image under that container. We do this for two reasons:
1946

    
1947
a) To demonstrate how to create containers other than the default ``Pithos``.
1948
   This can be done only with the `kamaki` client and not through the Web UI.
1949

    
1950
b) As a best organization practise, so that you won't have your Image files
1951
   tangled along with all your other Pithos+ files and directory structures.
1952

    
1953
We create the new ``images`` container by running:
1954

    
1955
.. code-block:: console
1956

    
1957
   $ kamaki store create images
1958

    
1959
Then, we upload the Image file to that container:
1960

    
1961
.. code-block:: console
1962

    
1963
   $ kamaki store upload --container images \
1964
                         /srv/images/debian_base-6.0-7-x86_64.diskdump \
1965
                         debian_base-6.0-7-x86_64.diskdump
1966

    
1967
The first is the local path and the second is the remote path on Pithos+. If
1968
the new container and the file appears on the Pithos+ Web UI, then you have
1969
successfully created the container and uploaded the Image file.
1970

    
1971
Register an existing Image file to Plankton
1972
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1973

    
1974
Once the Image file has been successfully uploaded on Pithos+, then we register
1975
it to Plankton (so that it becomes visible to Cyclades), by running:
1976

    
1977
.. code-block:: console
1978

    
1979
   $ kamaki image register "Debian Base" \
1980
                           pithos://user@example.com/images/debian_base-6.0-7-x86_64.diskdump \
1981
                           --public \
1982
                           --disk-format=diskdump \
1983
                           --property OSFAMILY=linux --property ROOT_PARTITION=1 \
1984
                           --property description="Debian Squeeze Base System" \
1985
                           --property size=451 --property kernel=2.6.32 --property GUI="No GUI" \
1986
                           --property sortorder=1 --property USERS=root --property OS=debian
1987

    
1988
This command registers the Pithos+ file
1989
``pithos://user@example.com/images/debian_base-6.0-7-x86_64.diskdump`` as an
1990
Image in Plankton. This Image will be public (``--public``), so all users will
1991
be able to spawn VMs from it and is of type ``diskdump``. The first two
1992
properties (``OSFAMILY`` and ``ROOT_PARTITION``) are mandatory. All the rest
1993
properties are optional, but recommended, so that the Images appear nicely on
1994
the Cyclades Web UI. ``Debian Base`` will appear as the name of this Image. The
1995
``OS`` property's valid values may be found in the ``IMAGE_ICONS`` variable
1996
inside the ``20-snf-cyclades-app-ui.conf`` configuration file.
1997

    
1998
``OSFAMILY`` and ``ROOT_PARTITION`` are mandatory because they will be passed
1999
from Plankton to Cyclades and then to Ganeti and `snf-image` (also see
2000
:ref:`previous section <ganeti-with-pithos-images>`). All other properties are
2001
used to show information on the Cyclades UI.
2002

    
2003
Spawn a VM from the Cyclades Web UI
2004
-----------------------------------
2005

    
2006
If the registration completes successfully, then go to the Cyclades Web UI from
2007
your browser at:
2008

    
2009
 `https://node1.example.com/ui/`
2010

    
2011
Click on the 'New Machine' button and the first step of the wizard will appear.
2012
Click on 'My Images' (right after 'System' Images) on the left pane of the
2013
wizard. Your previously registered Image "Debian Base" should appear under
2014
'Available Images'. If not, something has gone wrong with the registration. Make
2015
sure you can see your Image file on the Pithos+ Web UI and ``kamaki image
2016
register`` returns successfully with all options and properties as shown above.
2017

    
2018
If the Image appears on the list, select it and complete the wizard by selecting
2019
a flavor and a name for your VM. Then finish by clicking 'Create'. Make sure you
2020
write down your password, because you *WON'T* be able to retrieve it later.
2021

    
2022
If everything was setup correctly, after a few minutes your new machine will go
2023
to state 'Running' and you will be able to use it. Click 'Console' to connect
2024
through VNC out of band, or click on the machine's icon to connect directly via
2025
SSH or RDP (for windows machines).
2026

    
2027
Congratulations. You have successfully installed the whole Synnefo stack and
2028
connected all components. Go ahead in the next section to test the Network
2029
functionality from inside Cyclades and discover even more features.
2030

    
2031

    
2032
General Testing
2033
===============
2034

    
2035

    
2036
Notes
2037
=====