Statistics
| Branch: | Tag: | Revision:

root / docs / admin-guide.rst @ f92dffed

History | View | Annotate | Download (92.3 kB)

1
.. _admin-guide:
2

    
3
Synnefo Administrator's Guide
4
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5

    
6
This is the complete Synnefo Administrator's Guide.
7

    
8

    
9
.. _syn+archip:
10

    
11
General Synnefo Architecture
12
============================
13

    
14
The following figure shows a detailed view of the whole Synnefo architecture
15
and how it interacts with multiple Ganeti clusters. We hope that after reading
16
the Administrator's Guide you will be able to understand every component and
17
all the interactions between them.
18

    
19
.. image:: images/synnefo-arch2.png
20
   :width: 100%
21
   :target: _images/synnefo-arch2.png
22

    
23
Synnefo also supports RADOS as an alternative storage backend for
24
Files/Images/VM disks. You will find the :ref:`corresponding figure
25
<syn+archip+rados>` later in this guide.
26

    
27

    
28
Identity Service (Astakos)
29
==========================
30

    
31

    
32
Authentication methods
33
----------------------
34

    
35
Astakos supports multiple authentication methods:
36

    
37
 * local username/password
38
 * LDAP / Active Directory
39
 * SAML 2.0 (Shibboleth) federated logins
40
 * Google
41
 * Twitter
42
 * LinkedIn
43

    
44
.. _shibboleth-auth:
45

    
46
Shibboleth Authentication
47
~~~~~~~~~~~~~~~~~~~~~~~~~
48

    
49
Astakos can delegate user authentication to a Shibboleth federation.
50

    
51
To setup shibboleth, install package::
52

    
53
  apt-get install libapache2-mod-shib2
54

    
55
Change appropriately the configuration files in ``/etc/shibboleth``.
56

    
57
Add in ``/etc/apache2/sites-available/synnefo-ssl``::
58

    
59
  ShibConfig /etc/shibboleth/shibboleth2.xml
60
  Alias      /shibboleth-sp /usr/share/shibboleth
61

    
62
  <Location /ui/login/shibboleth>
63
    AuthType shibboleth
64
    ShibRequireSession On
65
    ShibUseHeaders On
66
    require valid-user
67
  </Location>
68

    
69
and before the line containing::
70

    
71
  ProxyPass        / http://localhost:8080/ retry=0
72

    
73
add::
74

    
75
  ProxyPass /Shibboleth.sso !
76

    
77
Then, enable the shibboleth module::
78

    
79
  a2enmod shib2
80

    
81
After passing through the apache module, the following tokens should be
82
available at the destination::
83

    
84
  eppn # eduPersonPrincipalName
85
  Shib-InetOrgPerson-givenName
86
  Shib-Person-surname
87
  Shib-Person-commonName
88
  Shib-InetOrgPerson-displayName
89
  Shib-EP-Affiliation
90
  Shib-Session-ID
91

    
92
Astakos keeps a map of shibboleth users using the value of the ``REMOTE_USER``
93
header, passed by the ``mod_shib2`` module. This happens in order to be able to
94
identify the astakos account the shibboleth user is associated to, every time
95
the user logs in from an affiliate shibboleth IdP. 
96

    
97
The shibboleth attribute which gets mapped to the ``REMOTE_USER`` header can be
98
changed in ``/etc/shibboleth/shibboleth2.xml`` configuration file.
99

    
100
.. code-block:: xml
101

    
102
    <!-- The ApplicationDefaults element is where most of Shibboleth's SAML bits are defined. -->
103
        <ApplicationDefaults entityID="https://sp.example.org/shibboleth" 
104
         REMOTE_USER="eppn persistent-id targeted-id">
105

    
106
.. warning::
107

    
108
 Changing ``mod_shib2`` ``REMOTE_USER`` to map to different shibboleth
109
 attributes will probably invalidate any existing shibboleth enabled users in
110
 astakos database. Those users won't be able to login to their existing accounts.
111

    
112

    
113
Finally, add 'shibboleth' in ``ASTAKOS_IM_MODULES`` list. The variable resides
114
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
115

    
116
Twitter Authentication
117
~~~~~~~~~~~~~~~~~~~~~~
118

    
119
To enable twitter authentication while signed in under a Twitter account,
120
visit dev.twitter.com/apps.
121

    
122
Click Create an application.
123

    
124
Fill the necessary information and for callback URL give::
125

    
126
    https://node1.example.com/ui/login/twitter/authenticated
127

    
128
Finally, add 'twitter' in ``ASTAKOS_IM_MODULES`` list. The variable resides
129
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
130

    
131
Google Authentication
132
~~~~~~~~~~~~~~~~~~~~~
133

    
134
To enable google authentication while signed in under a Google account,
135
visit https://code.google.com/apis/console/.
136

    
137
Under API Access select Create another client ID, select Web application,
138
expand more options in Your site or hostname section and in Authorized
139
Redirect URIs add:
140

    
141

    
142
Fill the necessary information and for callback URL give::
143

    
144
    https://node1.example.com/ui/login/google/authenticated
145

    
146
Finally, add 'google' in ``ASTAKOS_IM_MODULES`` list. The variable resides
147
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
148

    
149

    
150
Working with Astakos
151
--------------------
152

    
153
User registration
154
~~~~~~~~~~~~~~~~~
155

    
156
When a new user signs up, he/she is not directly marked as active. You can see
157
his/her state by running (on the machine that runs the Astakos app):
158

    
159
.. code-block:: console
160

    
161
   $ snf-manage user-list
162

    
163
More detailed user status is provided in the `status` field of the `user-show`
164
command:
165

    
166
.. code-block:: console
167

    
168
  $ snf-manage user-show <user-id>
169

    
170
  id                  : 6
171
  uuid                : 78661411-5eed-412f-a9ea-2de24f542c2e
172
  status              : Accepted/Active (accepted policy: manual)
173
  email               : user@synnefo.org
174
  ....
175

    
176
Based on the `astakos-app` configuration, there are several ways for a user to
177
get verified and activated in order to be able to login. We discuss the user
178
verification and activation flow in the following section.
179

    
180
User activation flow
181
````````````````````
182

    
183
A user can register for an account using the astakos signup form. Once the form
184
is submited successfully a user entry is created in astakos database. That entry
185
is passed through the astakos activation backend which handles whether the user
186
should be automatically verified and activated.
187

    
188
Email verification
189
``````````````````
190

    
191
The verification process takes place in order to ensure that the user owns the
192
email provided during the signup process. By default, after each successful
193
signup astakos notifies user with an verification url via email.
194

    
195
At this stage:
196

    
197
    * subsequent registrations invalidate and delete the previous registrations
198
      of the same email address.
199

    
200
    * in case user misses the initial notification, additional emails can be
201
      send either via the url which is prompted to the user if he tries to
202
      login, or by the administrator using the ``snf-manage user-activation-send
203
      <userid>`` command.
204

    
205
    * administrator may also enforce a user to get verified using the
206
      ``snf-manage user-modify --verify <userid>`` command.
207

    
208
Account activation
209
``````````````````
210

    
211
Once the user gets verified, it is time for Astakos to decide whether or not to
212
proceed through user activation process. If ``ASTAKOS_MODERATION_ENABLED``
213
setting is set to ``False`` (default value) user gets activated automatically.
214

    
215
In case the moderation is enabled Astakos may still automatically activate the
216
user in the following cases:
217

    
218
    * User email matches any of the regular expressions defined in
219
      ``ASTAKOS_RE_USER_EMAIL_PATTERNS`` (defaults to ``[]``)
220
    * User used a signup method (e.g. ``shibboleth``) for which automatic
221
      activation is enabled (see
222
      :ref:`authentication methods policies <auth_methods_policies>`).
223

    
224
If all of the above fail to trigger automatic activation, an email is sent to
225
the persons listed in ``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings,
226
notifing that there is a new user pending for moderation and that it's up to
227
the administrator to decide if the user should be activated. The UI also shows
228
a corresponding 'pending moderation' message to the user. The administrator can
229
activate a user using the ``snf-manage user-modify`` command:
230

    
231
.. code-block:: console
232

    
233
    # command to activate a pending user
234
    $ snf-manage user-modify --accept <userid>
235

    
236
    # command to reject a pending user
237
    $ snf-manage user-modify --reject --reject-reason="spammer" <userid>
238

    
239
Once the activation process finishes, a greeting message is sent to the user
240
email address and a notification for the activation to the persons listed in
241
``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings. Once activated the user is
242
able to login and access the Synnefo services.
243

    
244
Additional authentication methods
245
`````````````````````````````````
246

    
247
Astakos supports third party logins from external identity providers. This
248
can be usefull since it allows users to use their existing credentials to
249
login to astakos service.
250

    
251
Currently astakos supports the following identity providers:
252

    
253
    * `Shibboleth <http://www.internet2.edu/shibboleth>`_ (module name
254
      ``shibboleth``)
255
    * `Google <https://developers.google.com/accounts/docs/OAuth2>`_ (module
256
      name ``google``)
257
    * `Twitter <https://dev.twitter.com/docs/auth>`_ (module name ``twitter``)
258
    * `LinkedIn <http://developer.linkedin.com/documents/authentication>`_
259
      (module name ``linkedin``)
260

    
261
To enable any of the above modules (by default only ``local`` accounts are
262
allowed), retrieve and set the required provider settings and append the
263
module name in ``ASTAKOS_IM_MODULES``.
264

    
265
.. code-block:: python
266

    
267
    # settings from https://code.google.com/apis/console/
268
    ASTAKOS_GOOGLE_CLIENT_ID = '1111111111-epi60tvimgha63qqnjo40cljkojcann3.apps.googleusercontent.com'
269
    ASTAKOS_GOOGLE_SECRET = 'tNDQqTDKlTf7_LaeUcWTWwZM'
270

    
271
    # let users signup and login using their google account
272
    ASTAKOS_IM_MODULES = ['local', 'google']
273

    
274

    
275
.. _auth_methods_policies:
276

    
277
Authentication method policies
278
``````````````````````````````
279

    
280
Astakos allows you to override the default policies for each enabled provider
281
separately by adding the approriate settings in your ``.conf`` files in the
282
following format:
283

    
284
**ASTAKOS_AUTH_PROVIDER_<module>_<policy>_POLICY**
285

    
286
Available policies are:
287

    
288
    * **CREATE** Users can signup using that provider (default: ``True``)
289
    * **REMOVE/ADD** Users can remove/add login method from their profile
290
      (default: ``True``)
291
    * **AUTOMODERATE** Automatically activate users that signup using that
292
      provider (default: ``False``)
293
    * **LOGIN** Whether or not users can use the provider to login (default:
294
      ``True``).
295

    
296
e.g. to enable automatic activation for your academic users, while keeping
297
locally signed up users under moderation you can apply the following settings.
298

    
299
.. code-block:: python
300

    
301
    ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_AUTOMODERATE_POLICY = True
302
    ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_REMOVE_POLICY = False
303

    
304
User login
305
~~~~~~~~~~
306

    
307
During the logging procedure, the user is authenticated by the respective
308
identity provider.
309

    
310
If ``ASTAKOS_RECAPTCHA_ENABLED`` is set and the user fails several times
311
(``ASTAKOS_RATELIMIT_RETRIES_ALLOWED`` setting) to provide the correct
312
credentials for a local account, he/she is then prompted to solve a captcha
313
challenge.
314

    
315
Upon success, the system renews the token (if it has expired), logins the user
316
and sets the cookie, before redirecting the user to the ``next`` parameter
317
value.
318

    
319
Setting quota limits
320
~~~~~~~~~~~~~~~~~~~~
321

    
322
Set default quota
323
`````````````````
324
To inspect current default base quota limits, run::
325

    
326
   # snf-manage resource-list
327

    
328
You can modify the default base quota limit for all future users with::
329

    
330
   # snf-manage resource-modify <resource_name> --default-quota <value>
331

    
332
Set base quota for individual users
333
```````````````````````````````````
334

    
335
For individual users that need different quota than the default
336
you can set it for each resource like this::
337

    
338
    # use this to display quota / uuid
339
    # snf-manage user-show 'uuid or email' --quota
340

    
341
    # snf-manage user-modify <user-uuid> --base-quota 'cyclades.vm' 10
342

    
343
You can set base quota for all existing users, with possible exceptions, using::
344

    
345
    # snf-manage user-modify --all --base-quota cyclades.vm 10 --exclude uuid1,uuid2
346

    
347
All quota for which values different from the default have been set,
348
can be listed with::
349

    
350
    # snf-manage quota-list --with-custom=True
351

    
352

    
353
Enable the Projects feature
354
~~~~~~~~~~~~~~~~~~~~~~~~~~~
355

    
356
If you want to enable the projects feature so that users may apply
357
on their own for resources by creating and joining projects,
358
in ``20-snf-astakos-app-settings.conf`` set::
359

    
360
    # this will make the 'projects' page visible in the dashboard
361
    ASTAKOS_PROJECTS_VISIBLE = True
362

    
363
You can change the maximum allowed number of pending project applications
364
per user with::
365

    
366
    # snf-manage resource-modify astakos.pending_app --default-quota <number>
367

    
368
You can also set a user-specific limit with::
369

    
370
    # snf-manage user-modify <user-uuid> --base-quota 'astakos.pending_app' 5
371

    
372
When users apply for projects they are not automatically granted
373
the resources. They must first be approved by the administrator.
374

    
375
To list pending project applications in astakos::
376

    
377
    # snf-manage project-list --pending
378

    
379
Note the last column, the application id. To approve it::
380

    
381
    # <app id> from the last column of project-list
382
    # snf-manage project-control --approve <app id>
383

    
384
To deny an application::
385

    
386
    # snf-manage project-control --deny <app id>
387

    
388
Users designated as *project admins* can approve, deny, or modify
389
an application through the web interface. In
390
``20-snf-astakos-app-settings.conf`` set::
391

    
392
    # UUIDs of users that can approve or deny project applications from the web.
393
    ASTAKOS_PROJECT_ADMINS = [<uuid>, ...]
394

    
395

    
396
Astakos advanced operations
397
---------------------------
398

    
399
Adding "Terms of Use"
400
~~~~~~~~~~~~~~~~~~~~~
401

    
402
Astakos supports versioned terms-of-use. First of all you need to create an
403
html file that will contain your terms. For example, create the file
404
``/usr/share/synnefo/sample-terms.html``, which contains the following:
405

    
406
.. code-block:: console
407

    
408
   <h1>My cloud service terms</h1>
409

    
410
   These are the example terms for my cloud service
411

    
412
Then, add those terms-of-use with the snf-manage command:
413

    
414
.. code-block:: console
415

    
416
   $ snf-manage term-add /usr/share/synnefo/sample-terms.html
417

    
418
Your terms have been successfully added and you will see the corresponding link
419
appearing in the Astakos web pages' footer.
420

    
421
During the account registration, if there are approval terms, the user is
422
presented with an "I agree with the Terms" checkbox that needs to get checked
423
in order to proceed.
424

    
425
In case there are new approval terms that the user has not signed yet, the
426
``signed_terms_required`` view decorator redirects to the ``approval_terms``
427
view, so the user will be presented with the new terms the next time he/she
428
logins.
429

    
430
Enabling reCAPTCHA
431
~~~~~~~~~~~~~~~~~~
432

    
433
Astakos supports the `reCAPTCHA <http://www.google.com/recaptcha>`_ feature.
434
If enabled, it protects the Astakos forms from bots. To enable the feature, go
435
to https://www.google.com/recaptcha/admin/create and create your own reCAPTCHA
436
key pair. Then edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` and set
437
the corresponding variables to reflect your newly created key pair. Finally, set
438
the ``ASTAKOS_RECAPTCHA_ENABLED`` variable to ``True``:
439

    
440
.. code-block:: console
441

    
442
   ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
443
   ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
444

    
445
   ASTAKOS_RECAPTCHA_ENABLED = True
446

    
447
Restart the service on the Astakos node(s) and you are ready:
448

    
449
.. code-block:: console
450

    
451
   # /etc/init.d/gunicorn restart
452

    
453
Checkout your new Sign up page. If you see the reCAPTCHA box, you have setup
454
everything correctly.
455

    
456

    
457
Astakos internals
458
-----------------
459

    
460
X-Auth-Token
461
~~~~~~~~~~~~
462

    
463
Alice requests a specific resource from a cloud service e.g.: Pithos. In the
464
request she supplies the `X-Auth-Token` to identify whether she is eligible to
465
perform the specific task. The service contacts Astakos through its
466
``/account/v1.0/authenticate`` api call (see :ref:`authenticate-api-label`)
467
providing the specific ``X-Auth-Token``. Astakos checkes whether the token
468
belongs to an active user and it has not expired and returns a dictionary
469
containing user related information. Finally the service uses the ``uniq``
470
field included in the dictionary as the account string to identify the user
471
accessible resources.
472

    
473
.. _authentication-label:
474

    
475
Django Auth methods and Backends
476
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
477

    
478
Astakos incorporates Django user authentication system and extends its User model.
479

    
480
Since username field of django User model has a limitation of 30 characters,
481
AstakosUser is **uniquely** identified by the ``email`` instead. Therefore,
482
``astakos.im.authentication_backends.EmailBackend`` is served to authenticate a
483
user using email if the first argument is actually an email, otherwise tries
484
the username.
485

    
486
A new AstakosUser instance is assigned with a uui as username and also with a
487
``auth_token`` used by the cloud services to authenticate the user.
488
``astakos.im.authentication_backends.TokenBackend`` is also specified in order
489
to authenticate the user using the email and the token fields.
490

    
491
Logged on users can perform a number of actions:
492

    
493
 * access and edit their profile via: ``/im/profile``.
494
 * change their password via: ``/im/password``
495
 * send feedback for grnet services via: ``/im/send_feedback``
496
 * logout (and delete cookie) via: ``/im/logout``
497

    
498
Internal Astakos requests are handled using cookie-based Django user sessions.
499

    
500
External systems should forward to the ``/login`` URI. The server,
501
depending on its configuration will redirect to the appropriate login page.
502
When done with logging in, the service's login URI should redirect to the URI
503
provided with next, adding user and token parameters, which contain the email
504
and token fields respectively.
505

    
506
The login URI accepts the following parameters:
507

    
508
======================  =========================
509
Request Parameter Name  Value
510
======================  =========================
511
next                    The URI to redirect to when the process is finished
512
renew                   Force token renewal (no value parameter)
513
force                   Force logout current user (no value parameter)
514
======================  =========================
515

    
516
External systems inside the ``ASTAKOS_COOKIE_DOMAIN`` scope can acquire the
517
user information by the cookie identified by ``ASTAKOS_COOKIE_NAME`` setting
518
(set during the login procedure).
519

    
520
Finally, backend systems having acquired a token can use the
521
:ref:`authenticate-api-label` API call from a private network or through HTTPS.
522

    
523

    
524
Compute/Network/Image Service (Cyclades)
525
========================================
526

    
527
Introduction
528
------------
529

    
530
Cyclades is the Synnefo component that implements Compute, Network and Image
531
services and exposes the associated OpenStack REST APIs. By running Cyclades
532
you can provide a cloud that can handle thousands of virtual servers and
533
networks.
534

    
535
Cyclades does not include any virtualization software and knows nothing about
536
the low-level VM management operations, e.g. handling of VM creation or
537
migrations among physical nodes. Instead, Cyclades is the component that
538
handles multiple Ganeti backends and exposes the REST APIs. The administrator
539
can expand the infrastructure dynamically either by adding more Ganeti nodes
540
or by adding new Ganeti clusters. Cyclades issue VM control commands to Ganeti
541
via Ganeti's remote API and receive asynchronous notifications from Ganeti
542
backends whenever the state of a VM changes, due to Synnefo- or
543
administrator-initiated commands.
544

    
545
Cyclades is the action orchestrator and the API layer on top of multiple Ganeti
546
clusters. By this decoupled design, Ganeti cluster are self-contained and
547
the administrator has complete control on them without Cyclades knowing about
548
it. For example a VM migration to a different physical node is transparent
549
to Cyclades.
550

    
551
Working with Cyclades
552
---------------------
553

    
554
Flavors
555
~~~~~~~
556

    
557
When creating a VM, the user must specify the `flavor` of the virtual server.
558
Flavors are the virtual hardware templates, and provide a description about
559
the number of CPUs, the amount of RAM, and the size of the disk of the VM.
560
Besides the size of the disk, Cyclades flavors describe the storage backend
561
that will be used for the virtual server.
562

    
563
Flavors are created by the administrator and the user can select one of the
564
available flavors. After VM creation, the user can resize his VM, by
565
adding/removing CPU and RAM.
566

    
567
Cyclades support different storage backends that are described by the disk
568
template of the flavor, which is mapped to Ganeti's instance `disk template`.
569
Currently the available disk templates are the following:
570

    
571
* `file`: regulars file
572
* `sharedfile`: regular files on a shared directory, e.g. NFS
573
* `plain`: logical volumes
574
* `drbd`: drbd on top of lvm volumes
575
* `rbd`: rbd volumes residing inside a RADOS cluster
576
* `ext`: disks provided by an external shared storage.
577

    
578
  - `ext_archipelago`: External shared storage provided by
579
    `Archipelago <http://www.synnefo.org/docs/archipelago/latest/index.html>`_.
580

    
581
Flavors are created by the administrator using `snf-manage flavor-create`
582
command. The command takes as argument number of CPUs, amount of RAM, the size
583
of the disks and the disk templates and create the flavors that belong to the
584
cartesian product of the specified arguments. For example, the following
585
command will create two flavors of `40G` disk size with `drbd` disk template,
586
`4G` RAM and `2` or `4` CPUs.
587

    
588
.. code-block:: console
589

    
590
  $ snf-manage flavor-create 2,4 4096 40 drbd
591

    
592
To see the available flavors, run `snf-manage flavor-list` command. Finally,
593
the administrator can delete a flavor by using `flavor-modify` command:
594

    
595
.. code-block:: console
596

    
597
  $ snf-manage flavor-modify --deleted=True <flavor_id>
598

    
599
Images
600
~~~~~~
601

    
602
When creating a VM the user must also specify the `image` of the virtual
603
server. Images are the static templates from which VM instances are
604
initiated. Cyclades uses Pithos to store system and user-provided images,
605
taking advantage of all Pithos features, like deduplication and syncing
606
protocol. An image is a file stored to Pithos with additional metadata that
607
are describing the image, e.g. the OS family or the root partition. To create
608
a new image, the administrator or the user has to upload it a file to Pithos,
609
and then register it as an Image with Cyclades. Then the user can use this
610
image to spawn new VMs from it.
611

    
612
Images can be private, public or shared between users, exactly like Pithos
613
files. Since user-provided public images can be untrusted, the administrator
614
can denote which users are trusted by adding them to the
615
``UI_SYSTEM_IMAGES_OWNERS`` setting in the
616
`/etc/synnefo/20-snf-cyclades-app-ui.conf` file. Images of those users are
617
properly displayed in the UI.
618

    
619
When creating a new VM, Cyclades pass the location of the image and it's
620
metadata to Ganeti. After Ganeti creates the instance's disk, `snf-image`
621
will copy the image to the new disk and perform the image customization
622
phase. During the phase, `snf-image` sends notifications to Cyclades about
623
the progress of the image deployment and customization. Customization includes
624
resizing the root file system, file injection (e.g. SSH keys) and setting
625
a custom hostname. For better understanding of `snf-image` read the
626
corresponding `documentation
627
<http://www.synnefo.org/docs/snf-image/latest/index.html>`_.
628

    
629
For passing sensitive data about the image to Ganeti, like the VMs password,
630
Cyclades keeps all sensitive data in memory caches (memcache) and never allows
631
them to hit the disk. The data are exposed to `snf-image` via an one-time URL
632
that is exposed from the `vmapi` application. So, instead of passing sensitive
633
data to `snf-image` via Ganeti, Cyclades pass an one-time configuration URL
634
that contains a random UUID. After `snf-image` gets the sensitive data, the
635
URL is invalidated so no one else can access them.
636

    
637
The administrator can register images, exactly like users, using a system user
638
(a user that is defined in the ``UI_SYSTEM_IMAGES_OWNERS`` setting). For
639
example, the following command will register the
640
`pithos://u53r-un1qu3-1d/images/debian_base-6.0-7-x86_64.diskdump` as an
641
image to Cyclades:
642

    
643
.. code-block:: console
644

    
645
 $ kamaki image register --name="Debian Base" \
646
        --location="pithos://u53r-un1qu3-1d/images/debian_base-6.0-7-x86_64.diskdump" \
647
        --public \
648
        --disk-format=diskdump \
649
        --property OSFAMILY=linux --property ROOT_PARTITION=1 \
650
        --property description="Debian Squeeze Base System" \
651
        --property size=451 --property kernel=2.6.32 --property GUI="No GUI" \
652
        --property sortorder=1 --property USERS=root --property OS=debian
653

    
654
Deletion of an image is done via `kamaki image unregister` command, which will
655
delete the Cyclades Images but will leave the Pithos file as is (unregister).
656

    
657
Apart from using `kamaki` to see and hangle the available images, the
658
administrator can use `snf-manage image-list` and `snf-manage image-show`
659
commands to list and inspect the available public images. Also, the `--user-id`
660
option can be used the see the images of a specific user.
661

    
662
Virtual Servers
663
~~~~~~~~~~~~~~~
664

    
665
As mentioned, Cyclades uses Ganeti for management of VMs. The administrator can
666
handle Cyclades VMs just like any other Ganeti instance, via `gnt-instance`
667
commands. All Ganeti instances that belong to Synnefo, are separated from
668
others, by a prefix in their names. This prefix is defined in
669
``BACKEND_PREFIX_ID`` setting in
670
``/etc/synnefo/20-snf-cyclades-app-backend.conf``.
671

    
672
Apart from handling Cyclades VM at the Ganeti level, the administrator can
673
also use the `snf-manage server-*` commands. These command cover the most
674
common tasks that are relative with VM handling. Below we describe come
675
of them, but for more information you can use the `--help` option of all
676
`snf-manage server-* commands`. These command cover the most
677

    
678
The `snf-manage server-create` command can be used to create a new VM for some
679
user. This command can be useful when the administrator wants to test Cyclades
680
functionality without starting the API service, e.g. after an upgrade. Also, by
681
using `--backend-id` option, the VM will be created in the specified backend,
682
bypassing automatic VM allocation.
683

    
684
.. code-block:: console
685

    
686
 $ snf-manage server-create --flavor-id=1 --image-id=fc0f6858-f962-42ce-bf9a-1345f89b3d5e \
687
    --user-id=7cf4d078-67bf-424d-8ff2-8669eb4841ea --backend-id=2 \
688
    --password='example_passw0rd' --name='test_vm'
689

    
690
The above commnd will create a new VM for user
691
`7cf4d078-67bf-424d-8ff2-8669eb4841ea` in the Ganeti backend with ID 2. By
692
default this command will issue a Ganeti job to create the VM
693
(`OP_INSTANCE_CREATE`) and return. As in other commands, the `--wait=True`
694
option can be used in order to wait for the successful completion of the job.
695

    
696
`snf-manage server-list` command can be used to list all the available servers.
697
The command supports some useful options, like listing servers of a user,
698
listing servers that exist in a Ganeti backend and listing deleted servers.
699
Also, as in most of `*-list` commands, the `--filter-by` option can be used to
700
filter the results. For example, the following command will only display the
701
started servers of a specific flavor:
702

    
703
.. code-block:: console
704

    
705
 $ snf-manage server-list --filter-by="operstate=STARTED,flavor=<flavor_id>"
706

    
707
Another very useful command is the `server-inspect` command which will display
708
all available information about the state of the server in DB and the state
709
of the server in the Ganeti backend. The output will give you an easy overview
710
about the state of the VM which can be useful for debugging.
711

    
712
Also the administrator can `suspend` a user's VM, using the `server-modify`
713
command:
714

    
715
.. code-block:: console
716

    
717
 $ snf-manage server-modify --suspended=True <server_id>
718

    
719
The user is forbidden to do any action on an administratively suspended VM,
720
which is useful for abuse cases.
721

    
722
Ganeti backends
723
~~~~~~~~~~~~~~~
724

    
725
Since v0.11, Synnefo is able to manage multiple Ganeti clusters (backends)
726
making it capable to scale linearly to tens of thousands of VMs. Backends
727
can be dynamically added or removed via `snf-manage` commands.
728

    
729
Each newly created VM is allocated to a Ganeti backend by the Cyclades backend
730
allocator. The VM is "pinned" to this backend, and can not change through its
731
lifetime. The backend allocator decides in which backend to spawn the VM based
732
on the available resources of each backend, trying to balance the load between
733
them. Also, Networks are created to all Ganeti backends, in order to ensure
734
that VMs residing on different backends can be connected to the same networks.
735

    
736
A backend can be marked as `drained` in order to be excluded from automatic
737
servers allocation and not receive new servers. Also, a backend can be marked
738
as `offline` to denote that the backend is not healthy (e.g. broken master)
739
and avoid the penalty of connection timeouts.
740

    
741
Finally, Cyclades is able to manage Ganeti backends with different enabled
742
hypervisors (`kvm`, `xen`), and different enabled disk templates.
743

    
744
Listing existing backends
745
`````````````````````````
746
To list all the Ganeti backends known to Synnefo, we run:
747

    
748
.. code-block:: console
749

    
750
   $ snf-manage backend-list
751

    
752
Adding a new Ganeti backend
753
```````````````````````````
754
Backends are dynamically added under the control of Synnefo with `snf-manage
755
backend-add` command. In this section it is assumed that a Ganeti cluster,
756
named ``cluster.example.com`` is already up and running and configured to be
757
able to host Synnefo VMs.
758

    
759
To add this Ganeti cluster, we run:
760

    
761
.. code-block:: console
762

    
763
   $ snf-manage backend-add --clustername=cluster.example.com --user="synnefo_user" --pass="synnefo_pass"
764

    
765
where ``clustername`` is the Cluster hostname of the Ganeti cluster, and
766
``user`` and ``pass`` are the credentials for the `Ganeti RAPI user
767
<http://docs.ganeti.org/ganeti/2.8/html/rapi.html#users-and-passwords>`_.  All
768
backend attributes can be also changed dynamically using the `snf-manage
769
backend-modify` command.
770

    
771
``snf-manage backend-add`` will also create all existing public networks to
772
the new backend. You can verify that the backend is added, by running
773
`snf-manage backend-list`.
774

    
775
Note that no VMs will be spawned to this backend, since by default it is in a
776
``drained`` state after addition in order to manually verify the state of the
777
backend.
778

    
779
So, after making sure everything works as expected, make the new backend active
780
by un-setting the ``drained`` flag. You can do this by running:
781

    
782
.. code-block:: console
783

    
784
   $ snf-manage backend-modify --drained=False <backend_id>
785

    
786
Allocation of VMs in Ganeti backends
787
````````````````````````````````````
788
As already mentioned, the Cyclades backend allocator is responsible for
789
allocating new VMs to backends. This allocator does not choose the exact Ganeti
790
node that will host the VM but just the Ganeti backend. The exact node is
791
chosen by the Ganeti cluster's allocator (hail).
792

    
793
The decision about which backend will host a VM is based on the available
794
resources. The allocator computes a score for each backend, that shows its load
795
factor, and the one with the minimum score is chosen. The admin can exclude
796
backends from the allocation phase by marking them as ``drained`` by running:
797

    
798
.. code-block:: console
799

    
800
   $ snf-manage backend-modify --drained=True <backend_id>
801

    
802
The backend resources are periodically updated, at a period defined by
803
the ``BACKEND_REFRESH_MIN`` setting, or by running `snf-manage
804
backend-update-status` command. It is advised to have a cron job running this
805
command at a smaller interval than ``BACKEND_REFRESH_MIN`` in order to remove
806
the load of refreshing the backends stats from the VM creation phase.
807

    
808
Finally, the admin can decide to have a user's VMs being allocated to a
809
specific backend, with the ``BACKEND_PER_USER`` setting. This is a mapping
810
between users and backends. If the user is found in ``BACKEND_PER_USER``, then
811
Synnefo allocates all his/hers VMs to the specific backend in the variable,
812
even if is marked as drained (useful for testing).
813

    
814
.. _alloc_disk_templates:
815

    
816
Allocation based on disk-templates
817
**********************************
818

    
819
Besides the available resources of each Ganeti backend, the allocator takes
820
into consideration the disk template of the instance when trying to allocate it
821
to a Ganeti backend. Specifically, the allocator checks if the flavor of the
822
instance belongs to the available disk templates of each Ganeti backend.
823

    
824
A Ganeti cluster has a list of enabled disk templates
825
(`--enabled-disk-templates`) and a list of allowed disk templates for new
826
instances (`--ipolicy-disk-templates`). See the `gnt-cluster` manpage for more
827
details about these options.
828

    
829
When Synnefo allocates an instance, it checks whether the disk template of the
830
new instance belongs both in the enabled and ipolicy disk templates. You can
831
see the list of the available disk-templates by running `snf-manage
832
backend-list`. This list should be updated automatically after changing
833
these options in Ganeti and it can also be updated by running `snf-manage
834
backend-update-status`.
835

    
836
So the administrator, can route instances on different backends based on their
837
flavor disk template, by modifying the enabled or ipolicy disk templates of
838
each backend.  Also, the administrator can route instances between different
839
nodes of the same Ganeti backend, by modifying the same options at the
840
nodegroup level (see `gnt-group` manpage for mor details).
841

    
842
Removing an existing Ganeti backend
843
```````````````````````````````````
844
In order to remove an existing backend from Synnefo, you must first make
845
sure that there are not active servers in the backend, and then run:
846

    
847
.. code-block:: console
848

    
849
   $ snf-manage backend-remove <backend_id>
850

    
851

    
852
Virtual Networks
853
~~~~~~~~~~~~~~~~
854

    
855
Cyclades also implements the Network service and exposes the Quantum Openstack
856
API. Cyclades supports full IPv4 and IPv6 connectivity to the public internet
857
for it's VMs. Also, Cyclades provides L2 and L3 virtual private networks,
858
giving the user freedom to create arbitraty network topologies of
859
interconnected VMs.
860

    
861
Public networking is desployment specific and must be customized based on the
862
specific needs of the system administrator. Private virtual networks can be
863
provided by different network technologies which are exposed as different
864
network flavors. For better understanding of networking please refer to the
865
:ref:`Network <networks>` section.
866

    
867
A Cyclades virtual network is an isolated Layer-2 broadcast domain. A network
868
can also have an associated IPv4 and IPv6 subnet representing the Layer-3
869
characteristics of the network. Each subnet represents an IP address block
870
that is used in order to assign addresses to VMs.
871

    
872
To connect a VM to a network, a port must be created, which represent a virtual
873
port on a network switch. VMs are connected to networks by attaching a virtual
874
interface to a port.
875

    
876
Cyclades also supports `floating IPs`, which are public IPv4 addresses that
877
can dynamically(hotplug-able) be added and removed to VMs. Floating IPs are
878
a quotable resource that is allocated to each user. Unlike other cloud
879
platforms, floating IPs are not implemented using 1-1 NAT to a ports private
880
IP. Instead, floating IPs are directly assigned to virtual interfaces of VMs.
881

    
882
Exactly like VMS, networks can be handled as Ganeti networks via `gnt-network`
883
commands. All Ganeti networks that belong to Synnefo are named with the prefix
884
`${BACKEND_PREFIX_ID}-net-`. Also, there are a number of `snf-manage` commands
885
for handling of `networks`, `subnets`, `ports` and `floating IPs`. Below
886
we will present a use case scenario using some of these commands. For better
887
understanding of these commands, refer to their help messages.
888

    
889
Create a virtual private network for user
890
`7cf4d078-67bf-424d-8ff2-8669eb4841ea` using the `PHYSICAL_VLAN` flavor, which
891
means that the network will be uniquely assigned a phsyical VLAN. The network
892
is assigned an IPv4 subnet, described by it's CIDR and gateway. Also,
893
the `--dhcp=True` option is used, to make `nfdhcpd` response to DHCP queries
894
from VMs.
895

    
896
.. code-block:: console
897

    
898
 $ snf-manage network-create --owner=7cf4d078-67bf-424d-8ff2-8669eb4841ea --name=prv_net-1 \
899
    --subnet=192.168.2.0/24 --gateway=192.168.2.1 --dhcp=True --flavor=PHYSICAL_VLAN
900

    
901
Inspect the state of the network in Cyclades DB and in all the Ganeti backends:
902

    
903
.. code-block:: console
904

    
905
  $ snf-manage network-inspect <network_id>
906

    
907
Inspect the state of the network's subnet, containg an overview of the
908
subnet's IPv4 address allocation pool:
909

    
910
.. code-block:: console
911

    
912
  $ snf-manage subnet-inspect <subnet_id>
913

    
914
Connect a VM to the created private network. The port will be automatically
915
be assigned an IPv4 address from one of the network's available IPs. This
916
command will result in sending an `OP_INSTANCE_MODIFY` Ganeti command and
917
attaching a NIC to the specified Ganeti instance.
918

    
919
.. code-block:: console
920

    
921
 $ snf-manage port-create --network=<network_id> --server=<server_id>
922

    
923
Inspect the state of the the port in Cyclades DB and in the Ganeti backend:
924

    
925
.. code-block:: console
926

    
927
 $ snf-manage port-inspect <port_id>
928

    
929
Disconnect the VM from the network and delete the network:
930

    
931
.. code-block:: console
932

    
933
 $ snf-manage port-remove <port_id>
934
 $ snf-manage network-remove <network_id>
935

    
936

    
937
Enabling DHCP
938
`````````````
939

    
940
When connecting a VM to a network, Cyclades will automatically assign an IPv4
941
address from the IPv4 or/and IPv6 subnets of the network. If the network has
942
no subnets, then it will not be assigned any IP address.
943

    
944
If the network has DHCP enabled, then `nfdhcpd` daemon, which must be running
945
on all Ganeti nodes, will respond to DHCP queries from VMs and assign to them
946
the IP address that was allocated by Cyclades. DCHP can be enabled/disabled
947
using the `--dhcp` option of `network-create` command.
948

    
949

    
950
Public network connectivity
951
```````````````````````````
952

    
953
Since v0.14, users are able to dynamically connect and disconnect their VMs
954
from public networks. In order to do that, they have to use a `floating IP`.
955
Floating IPs are basically public IPv4 addresses that can be dynamically
956
attached and detached from VMs. The user creates a floating IP address from a
957
network that has set the `floating_ip_pool` attribute. The floating IP is
958
accounted to the user, who can then connect his VMs to public networks by
959
creating ports that they are using this floating IP. Performing this work-flow
960
from `snf-manage` would look like this:
961

    
962
.. code-block:: console
963

    
964
 $ snf-manage network-list --filter-by="floating_ip_pool=True"
965
 id      name  user.uuid   state  public  subnet.ipv4  gateway.ipv4  drained  floating_ip_pool
966
 ---------------------------------------------------------------------------------------------
967
  1  Internet       None  ACTIVE    True  10.2.1.0/24      10.2.1.1    False              True
968

    
969
 $ snf-manage floating-ip-create --owner=7cf4d078-67bf-424d-8ff2-8669eb4841ea --network=1
970

    
971
 $ snf-manage floating-ip-list --user=7cf4d078-67bf-424d-8ff2-8669eb4841ea
972
 id   address       network                             user.uuid  server
973
 ------------------------------------------------------------------------
974
 38  10.2.1.2             1  7cf4d078-67bf-424d-8ff2-8669eb4841ea      42
975

    
976
 $ snf-manage port-create --owner=7cf4d078-67bf-424d-8ff2-8669eb4841ea --network=1 \
977
                          --ipv4-address=10.2.1.2 --floating-ip=38
978

    
979
 $ snf-manage port-list --user=7cf4d078-67bf-424d-8ff2-8669eb4841ea
980
 id                            user.uuid        mac_address  network  server_id  fixed_ips   state
981
 --------------------------------------------------------------------------------------------------
982
 163 7cf4d078-67bf-424d-8ff2-8669eb4841ea  aa:00:00:45:13:98       1         77   10.2.1.2  ACTIVE
983

    
984
 $ snf-manage port-remove 163
985
 $ snf-manage floating-ip-remove 38
986

    
987
Users do not have permission to connect and disconnect VMs from public
988
networks without using a floating IP address. However, the administrator
989
have the ability to perform this tasks, using `port-create` and `port-remove`
990
commands.
991

    
992
Network connectivity for newly created servers
993
``````````````````````````````````````````````
994

    
995
When creating a virtual server, the user can specify the networks that the
996
newly created server will be connected to. Beyond this, the administrator can
997
define a list of networks that every new server will be forced to connect to.
998
For example, you can enforce all VMs to be connected to a public network
999
containing a metadata server. The networks must be specified in the
1000
``CYCLADES_FORCED_SERVER_NETWORKS`` that exists in the
1001
``/etc/synnefo/20-snf-cyclades-app-api.conf``. For the networks in this
1002
setting, no access control or quota policy are enforced!
1003

    
1004
Finally, the administrator can define a list of networks that new servers will
1005
be connected, *if the user has not* specified networks in the request to create
1006
the server. Access control and quota policy are enforced, just as if the user
1007
had specified these networks. The list of these networks is defined in the
1008
``CYCLADES_DEFAULT_SERVER_NETWORKS`` that exists in the
1009
``/etc/synnefo/20-snf-cyclades-app-api.conf``. This setting should only
1010
be used if Cyclades are being accessed by external clients that are
1011
unaware of the `Neutron API extensions` in the `Compute API`.
1012

    
1013
Each member of the above mentioned settings can be:
1014

    
1015
* a network UUID
1016
* a tuple of network UUIDs: the server will be connected to only one of these
1017
  networks, e.g. one that has a free IPv4 address
1018
* `SNF:ANY_PUBLIC_IPV4`: the server will be connected to any network with
1019
  an IPv4 subnet defined
1020
* `SNF:ANY_PUBLIC_IPV6`: the server will be connected to any network with
1021
  only an IPv6 subnet defined.
1022
* `SNF:ANY_PUBLIC`: the server will be connected to any public network.
1023

    
1024
Public IP accounting
1025
````````````````````
1026

    
1027
There are many use cases, e.g. abuse ports, where you need to find which user
1028
or which server had a public IP address. For this reason, Cyclades keeps track
1029
usage of public IPv4/IPv6 addresses. Specifically, it keeps the date and time
1030
that each public IP address was allocated and released from a virtual server.
1031
This information can be found using `ip-list` command:
1032

    
1033
.. code-block:: console
1034

    
1035
 $ snf-manage ip-list
1036

    
1037
 Show usage of a specific address:
1038
 $ snf-manage ip-list --address=192.168.2.1
1039

    
1040
 Show public IPs of a specific server:
1041
 $ snf-manage ip-list --server=<server_id>
1042

    
1043

    
1044
Managing Network Resources
1045
``````````````````````````
1046

    
1047
Proper operation of the Cyclades Network Service depends on the unique
1048
assignment of specific resources to each type of virtual network. Specifically,
1049
these resources are:
1050

    
1051
* IP addresses. Cyclades creates a Pool of IPs for each Network, and assigns a
1052
  unique IP address to each VM, thus connecting it to this Network. You can see
1053
  the IP pool of each network by running `snf-manage subnet-inspect
1054
  <subnet_ID>`. IP pools are automatically created and managed by Cyclades,
1055
  depending on the subnet of the Network.
1056
* Bridges corresponding to physical VLANs, which are required for networks of
1057
  type `PRIVATE_PHYSICAL_VLAN`.
1058
* One Bridge corresponding to one physical VLAN which is required for networks of
1059
  type `PRIVATE_MAC_PREFIX`.
1060

    
1061
IPv4 addresses
1062
**************
1063

    
1064
An allocation pool of IPv4 addresses is automatically created for every network
1065
with an IPv4 subnet. By default, the allocation pool contains the range of IP
1066
addresses that are included in the subnet, except from the gateway and the
1067
broadcast address of the network. The range of IP addresses can be restricted
1068
using the `--allocation-pool` option of `snf-manage network-create` command.
1069
The admin can externally reserve IP addresses to exclude them from automatic
1070
allocation with the `--add-reserved-ips` option of `snf-manage network-modify`
1071
command. For example the following command will reserve two IP addresses from
1072
network with ID `42`:
1073

    
1074
.. code-block:: console
1075

    
1076
 snf-manage network-modify --add-reserved-ips=10.0.0.21,10.0.0.22 42
1077

    
1078
.. warning:: Externally reserving IP addresses is also available at the Ganeti.
1079
 However, when using Cyclades with multiple Ganeti backends, the handling of
1080
 IP pools must be performed from Cyclades!
1081

    
1082
Bridges
1083
*******
1084

    
1085
As already mentioned Cyclades use a pool of Bridges that must correspond
1086
to Physical VLAN at the Ganeti level. A bridge from the pool is assigned to
1087
each network of flavor `PHYSICAL_VLAN`. Creation of this pool is done
1088
using `snf-manage pool-create` command. For example the following command
1089
will create a pool containing the brdiges from `prv1` to `prv21`.
1090

    
1091
.. code-block:: console
1092

    
1093
   # snf-manage pool-create --type=bridge --base=prv --size=20
1094

    
1095
You can verify the creation of the pool, and check its contents by running:
1096

    
1097
.. code-block:: console
1098

    
1099
   # snf-manage pool-list
1100
   # snf-manage pool-show --type=bridge 1
1101

    
1102
Finally you can use the `pool-modify` management command in order to externally
1103
reserve the values from pool, extend or shrink the pool if possible.
1104

    
1105
MAC Prefixes
1106
************
1107

    
1108
Cyclades also use a pool of MAC prefixes to assign to networks of flavor
1109
`MAC_FILTERED`. Handling of this pool is done exactly as with pool of bridges,
1110
except that the type option must be set to mac-prefix:
1111

    
1112
.. code-block:: console
1113

    
1114
   # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
1115

    
1116
The above command will create a pool of MAC prefixes from ``aa:00:1`` to
1117
``b9:ff:f``. The MAC prefix pool is responsible for providing only unicast and
1118
locally administered MAC addresses, so many of these prefixes will be
1119
externally reserved, to exclude from allocation.
1120

    
1121
Quotas
1122
~~~~~~
1123

    
1124
Handling of quotas for Cyclades resources is powered by Astakos quota
1125
mechanism. During registration of Cyclades service to Astakos, the Cyclades
1126
resources are also imported to Astakos for accounting and presentation.
1127

    
1128
Upon a request that will result in a resource creation or removal, Cyclades
1129
will communicate with Astakos to ensure that user quotas are within limits and
1130
update the corresponding usage. If a limit is reached, the request will be
1131
denied with an `overLimit(413)` fault.
1132

    
1133
The resources that are exported by Cyclades are the following:
1134

    
1135
* `cyclades.vm`: Number of virtual machines
1136
* `cyclades.total_cpu`: Number of virtual machine processors
1137
* `cyclades.cpu`: Number of virtual machine processors of running VMs
1138
* `cyclades.total_ram`: Virtual machine memory size
1139
* `cyclades.ram`: Virtual machine memory size of running VMs
1140
* `cyclades.disk`: Virtual machine disk size
1141
* `cyclades.floating_ip`: Number of floating IP addresses
1142
* `cyclades.network.private`: Number of private virtual networks
1143

    
1144
Cyclades advanced operations
1145
----------------------------
1146

    
1147
Reconciliation mechanism
1148
~~~~~~~~~~~~~~~~~~~~~~~~
1149

    
1150
Cyclades - Ganeti reconciliation
1151
````````````````````````````````
1152

    
1153
On certain occasions, such as a Ganeti or RabbitMQ failure, the state of
1154
Cyclades database may differ from the real state of VMs and networks in the
1155
Ganeti backends. The reconciliation process is designed to synchronize the
1156
state of the Cyclades DB with Ganeti. There are two management commands for
1157
reconciling VMs and Networks that will detect stale, orphans and out-of-sync
1158
VMs and networks. To fix detected inconsistencies, use the `--fix-all`.
1159

    
1160
.. code-block:: console
1161

    
1162
  $ snf-manage reconcile-servers
1163
  $ snf-manage reconcile-servers --fix-all
1164

    
1165
  $ snf-manage reconcile-networks
1166
  $ snf-manage reconcile-networks --fix-all
1167

    
1168
Please see ``snf-manage reconcile-servers --help`` and ``snf-manage
1169
reconcile--networks --help`` for all the details.
1170

    
1171

    
1172
Cyclades - Astakos reconciliation
1173
`````````````````````````````````
1174

    
1175
As already mentioned, Cyclades communicates with Astakos for resource
1176
accounting and quota enforcement. In rare cases, e.g. unexpected
1177
failures, the two services may get unsynchronized. For this reason there
1178
are the `reconcile-commissions-cyclades` and `reconcile-resources-cyclades`
1179
command that will synchronize the state of the two services. The first
1180
command will detect any pending commissions, while the second command will
1181
detect that the usage that is reported by Astakos is correct.
1182
To fix detected inconsistencies, use the `--fix` option.
1183

    
1184
.. code-block:: console
1185

    
1186
  $ snf-manage reconcile-commissions-cyclades
1187
  $ snf-manage reconcile-commissions-cyclades --fix
1188

    
1189
  $ snf-manage reconcile-resources-cyclades
1190
  $ snf-manage reconcile-resources-cyclades --fix
1191

    
1192

    
1193
Cyclades resources reconciliation
1194
`````````````````````````````````
1195

    
1196
Reconciliation of pools will check the consistency of available pools by
1197
checking that the values from each pool are not used more than once, and also
1198
that the only reserved values in a pool are the ones used. Pool reconciliation
1199
will check pools of bridges, MAC prefixes, and IPv4 addresses for all networks.
1200
To fix detected inconsistencies, use the `--fix` option.
1201

    
1202

    
1203
.. code-block:: console
1204

    
1205
  $ snf-manage reconcile-pools
1206
  $ snf-manage reconcile-pools --fix
1207

    
1208
.. _admin-guide-stats:
1209

    
1210
VM stats collecting
1211
~~~~~~~~~~~~~~~~~~~
1212

    
1213
snf-cyclades-gtools comes with a collectd plugin to collect CPU and network
1214
stats for Ganeti VMs and an example collectd configuration. snf-stats-app is a
1215
Django (snf-webproject) app that serves the VM stats graphs by reading the VM
1216
stats (from RRD files) and serves graphs.
1217

    
1218
The snf-stats-app was originally written by `GRNET NOC <http://noc.grnet.gr>`_
1219
as a WSGI Python app and was ported to a Synnefo (snf-webproject) app.
1220

    
1221
snf-stats-app configuration
1222
```````````````````````````
1223

    
1224
The snf-stats-app node should have collectd installed. The collectd
1225
configuration should enable the network plugin, assuming the server role, and
1226
the RRD plugin / backend, to store the incoming stats. Your
1227
``/etc/collectd/collectd.conf`` should look like:
1228

    
1229
.. code-block:: console
1230

    
1231
    FQDNLookup true
1232
    LoadPlugin syslog
1233
    <Plugin syslog>
1234
        LogLevel info
1235
    </Plugin>
1236

    
1237
    LoadPlugin network
1238
    LoadPlugin rrdtool
1239
    <Plugin network>
1240
        TimeToLive 128
1241
        <Listen "okeanos.io" "25826">
1242
            SecurityLevel "Sign"
1243
            AuthFile "/etc/collectd/passwd"
1244
        </Listen>
1245

    
1246
        ReportStats false
1247
        MaxPacketSize 65535
1248
    </Plugin>
1249

    
1250

    
1251
    <Plugin rrdtool>
1252
        DataDir "/var/lib/collectd/rrd"
1253
        CacheTimeout 120
1254
        CacheFlush 900
1255
        WritesPerSecond 30
1256
        RandomTimeout 0
1257
    </Plugin>
1258

    
1259
    Include "/etc/collectd/filters.conf"
1260
    Include "/etc/collectd/thresholds.conf"
1261

    
1262

    
1263
An example collectd config file is provided in
1264
``/usr/share/doc/snf-stats-app/examples/stats-colletcd.conf``.
1265

    
1266
The recommended deployment is to run snf-stats-app using gunicorn with an
1267
Apache2 or nginx reverse proxy (using the same configuration as the other
1268
Synnefo services / apps). An example gunicorn config file is provided in
1269
``/usr/share/doc/snf-stats-app/examples/stats.gunicorn``.
1270

    
1271
Make sure to edit the settings under
1272
``/etc/synnefo/20-snf-stats-app-settings.conf`` to match your deployment.
1273
More specifically, you should change the ``STATS_BASE_URL`` setting (refer
1274
to previous documentation on the BASE_URL settings used by the other Synnefo
1275
services / apps) and the ``RRD_PREFIX`` and ``GRAPH_PREFIX`` settings.
1276

    
1277
You should also set the ``STATS_SECRET_KEY`` to a random string and make sure
1278
it's the same at the ``CYCLADES_STATS_SECRET_KEY`` on the Cyclades host (see
1279
below).
1280

    
1281
``RRD_PREFIX`` is the directory where collectd stores the RRD files. The
1282
default setting matches the default RRD directory for the collectd RRDtool
1283
plugin. In a more complex setup, the collectd daemon could run on a separate
1284
host and export the RRD directory to the snf-stats-app node via e.g. NFS.
1285

    
1286
``GRAPH_PREFIX`` is the directory where collectd stores the resulting
1287
stats graphs. You should create it manually, in case it doesn't exist.
1288

    
1289
.. code-block::
1290

    
1291
    # mkdir /var/cache/snf-stats-app/
1292
    # chown www-data:wwwdata /var/cache/snf-stats-app/
1293

    
1294
The snf-stats-app will typically run as the ``www-data`` user. In that case,
1295
make sure that the ``www-data`` user should have read access to the
1296
``RRD_PREFIX`` directory and read / write access to the ``GRAPH_PREFIX``
1297
directory.
1298

    
1299
snf-stats-app, based on the ``STATS_BASE_URL`` setting will export the
1300
following URL 'endpoints`:
1301
 * CPU stats bar: ``STATS_BASE_URL``/v1.0/cpu-bar/<encrypted VM hostname>
1302
 * Network stats bar: ``STATS_BASE_URL``/v1.0/net-bar/<encrypted VM hostname>
1303
 * CPU stats daily graph: ``STATS_BASE_URL``/v1.0/cpu-ts/<encrypted VM hostname>
1304
 * Network stats daily graph: ``STATS_BASE_URL``/v1.0/net-ts/<encrypted VM hostname>
1305
 * CPU stats weekly graph: ``STATS_BASE_URL``/v1.0/cpu-ts-w/<encrypted VM hostname>
1306
 * Network stats weekly graph: ``STATS_BASE_URL``/v1.0/net-ts-w/<encrypted VM hostname>
1307

    
1308
You can verify that these endpoints are exported by issuing:
1309

    
1310
.. code-block::
1311

    
1312
    # snf-manage show_urls
1313

    
1314
snf-cyclades-gtools configuration
1315
`````````````````````````````````
1316

    
1317
To enable VM stats collecting, you will need to:
1318
 * Install collectd on the every Ganeti (VM-capable) node.
1319
 * Enable the Ganeti stats plugin in your collectd configuration. This can be
1320
   achived by either copying the example collectd conf file that comes with
1321
   snf-cyclades-gtools
1322
   (``/usr/share/doc/snf-cyclades-gtools/examples/ganeti-stats-collectd.conf``)
1323
   or by adding the following line to your existing (or default) collectd
1324
   conf file:
1325

    
1326
       Include /etc/collectd/ganeti-stats.conf
1327

    
1328
   In the latter case, make sure to configure collectd to send the collected
1329
   stats to your collectd server (via the network plugin). For more details on
1330
   how to do this, check the collectd example config file provided by the
1331
   package and the collectd documentation.
1332

    
1333
snf-cyclades-app configuration
1334
``````````````````````````````
1335

    
1336
At this point, stats collecting should be enabled and working. You can check
1337
that everything is ok by checking the contents of ``/var/lib/collectd/rrd/``
1338
directory (it will gradually get populated with directories containing RRD
1339
files / stats for every Synnefo instances).
1340

    
1341
You should also check that gunicorn and Apache2 are configured correctly by
1342
accessing the graph URLs for a VM (whose stats have been populated in
1343
``/var/lib/collectd/rrd``).
1344

    
1345
Cyclades uses the ``CYCLADES_STATS_SECRET_KEY`` setting in
1346
``20-snf-cyclades-app`` to encrypt the instance hostname in the stats graph
1347
URL. This settings should be set to a random value and match the
1348
``STATS_SECRET_KEY`` on the Stats host.
1349

    
1350
Cyclades (snf-cyclades-app) fetches the stat graphs for VMs based on four
1351
settings in ``20-snf-cyclades-app-api.conf``. The settings are:
1352

    
1353
 * CPU_BAR_GRAPH_URL = 'https://stats.host/stats/v1.0/cpu-bar/%s'
1354
 * CPU_TIMESERIES_GRAPH_URL = 'https://stats.host/stats/v1.0/cpu-ts/%s'
1355
 * NET_BAR_GRAPH_URL = 'https://stats.host/stats/v1.0/net-bar/%s'
1356
 * NET_TIMESERIES_GRAPH_URL = 'https://stats.host/stats/v1.0/net-ts/%s'
1357

    
1358
Make sure that you change this settings to match your ``STATS_BASE_URL``
1359
(and generally the Apache2 / gunicorn deployment on your stats host).
1360

    
1361
Cyclades will pass these URLs to the Cyclades UI and the user's browser will
1362
fetch them when needed.
1363

    
1364

    
1365
Helpdesk
1366
--------
1367

    
1368
Helpdesk application provides the ability to view the virtual servers and
1369
networks of all users, along with the ability to perform some basic actions
1370
like administratively suspending a server. You can perform look-ups by
1371
user UUID or email, by server ID (vm-$id) or by an IPv4 address.
1372

    
1373
If you want to activate the helpdesk application you can set to `True` the
1374
`HELPDESK_ENABLED` setting. Access to helpdesk views (under
1375
`$BASE_URL/helpdesk`) is only to allowed to users that belong to Astakos
1376
groups defined in the `HELPDESK_PERMITTED_GROUPS` setting, which by default
1377
contains the `helpdesk` group. For example, to allow <user_id>
1378
to access helpdesk view, you should run the following command in the Astakos
1379
node:
1380

    
1381
.. code-block:: console
1382

    
1383
 snf-manage group-add helpdesk
1384
 snf-manage user-modify --add-group=helpdesk <user_id>
1385

    
1386

    
1387
Cyclades internals
1388
------------------
1389

    
1390
Asynchronous communication with Ganeti backends
1391
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1392
Synnefo uses Google Ganeti backends for VM cluster management. In order for
1393
Cyclades to be able to handle thousands of user requests, Cyclades and Ganeti
1394
communicate asynchronously. Briefly, requests are submitted to Ganeti through
1395
Ganeti's RAPI/HTTP interface, and then asynchronous notifications about the
1396
progress of Ganeti jobs are being created and pushed upwards to Cyclades. The
1397
architecture and communication with a Ganeti backend is shown in the graph
1398
below:
1399

    
1400
.. image:: images/cyclades-ganeti-communication.png
1401
   :width: 40%
1402
   :target: _images/cyclades-ganeti-communication.png
1403

    
1404
The Cyclades API server is responsible for handling user requests. Read-only
1405
requests are directly served by looking up the Cyclades DB. If the request
1406
needs an action in the Ganeti backend, Cyclades submit jobs to the Ganeti
1407
master using the `Ganeti RAPI interface
1408
<http://docs.ganeti.org/ganeti/2.8/html/rapi.html>`_.
1409

    
1410
While Ganeti executes the job, `snf-ganeti-eventd`, and `snf-progress-monitor`
1411
are monitoring the progress of the job and send corresponding messages to the
1412
RabbitMQ servers. These components are part of `snf-cyclades-gtools` and must
1413
be installed on all Ganeti nodes. Specially:
1414

    
1415
* *snf-ganeti-eventd* sends messages about operations affecting the operating
1416
  state of instances and networks. Works by monitoring the Ganeti job queue.
1417
* *snf-progress_monitor* sends messages about the progress of the Image deployment
1418
  phase which is done by the Ganeti OS Definition `snf-image`.
1419

    
1420
Finally, `snf-dispatcher` consumes messages from the RabbitMQ queues, processes
1421
these messages and properly updates the state of the Cyclades DB. Subsequent
1422
requests to the Cyclades API, will retrieve the updated state from the DB.
1423

    
1424

    
1425
List of all Synnefo components
1426
==============================
1427

    
1428
They are also available from our apt repository: ``apt.dev.grnet.gr``
1429

    
1430
 * `snf-common <http://www.synnefo.org/docs/snf-common/latest/index.html>`_
1431
 * `snf-webproject <http://www.synnefo.org/docs/snf-webproject/latest/index.html>`_
1432
 * `snf-astakos-app <http://www.synnefo.org/docs/astakos/latest/index.html>`_
1433
 * `snf-pithos-backend <http://www.synnefo.org/docs/pithos/latest/backends.html>`_
1434
 * `snf-pithos-app <http://www.synnefo.org/docs/pithos/latest/index.html>`_
1435
 * `snf-pithos-webclient <http://www.synnefo.org/docs/pithos-webclient/latest/index.html>`_
1436
 * `snf-cyclades-app <http://www.synnefo.org/docs/snf-cyclades-app/latest/index.html>`_
1437
 * `snf-cyclades-gtools <http://www.synnefo.org/docs/snf-cyclades-gtools/latest/index.html>`_
1438
 * `astakosclient <http://www.synnefo.org/docs/astakosclient/latest/index.html>`_
1439
 * `snf-vncauthproxy <https://code.grnet.gr/projects/vncauthproxy>`_
1440
 * `snf-image <http://www.synnefo.org/docs/snf-image/latest/index.html/>`_
1441
 * `snf-image-creator <http://www.synnefo.org/docs/snf-image-creator/latest/index.html>`_
1442
 * `snf-occi <http://www.synnefo.org/docs/snf-occi/latest/index.html>`_
1443
 * `snf-cloudcms <http://www.synnefo.org/docs/snf-cloudcms/latest/index.html>`_
1444
 * `nfdhcpd <https://code.grnet.gr/projects/nfdhcpd>`_
1445

    
1446

    
1447
Synnefo management commands ("snf-manage")
1448
==========================================
1449

    
1450
Each Synnefo service, Astakos, Pithos and Cyclades are controlled by the
1451
administrator using the "snf-manage" admin tool. This tool is an extension of
1452
the Django command-line management utility. It is run on the host that runs
1453
each service and provides different types of commands depending the services
1454
running on the host. If you are running more than one service on the same host
1455
"snf-manage" adds all the corresponding commands for each service dynamically,
1456
providing a unified admin environment.
1457

    
1458
To run "snf-manage" you just type:
1459

    
1460
.. code-block:: console
1461

    
1462
   # snf-manage <command> [arguments]
1463

    
1464
on the corresponding host that runs the service. For example, if you have all
1465
services running on different physical hosts you would do:
1466

    
1467
.. code-block:: console
1468

    
1469
   root@astakos-host # snf-manage <astakos-command> [argument]
1470
   root@pithos-host # snf-manage <pithos-command> [argument]
1471
   root@cyclades-host # snf-manage <cyclades-command> [argument]
1472

    
1473
If you have all services running on the same host you would do:
1474

    
1475
.. code-block:: console
1476

    
1477
   root@synnefo-host # snf-manage <{astakos,pithos,cyclades}-command> [argument]
1478

    
1479
Note that you cannot execute a service's command on a host that is not running
1480
this service. For example, the following will return an error if Astakos and
1481
Cyclades are installed on different physical hosts:
1482

    
1483
.. code-block:: console
1484

    
1485
   root@astakos-host # snf-manage <cyclades-command> [argument]
1486
   Unknown command: 'cyclades-command'
1487
   Type 'snf-manage help' for usage.
1488

    
1489
This is the complete list of "snf-manage" commands for each service.
1490

    
1491
Astakos snf-manage commands
1492
---------------------------
1493

    
1494
============================  ===========================
1495
Name                          Description
1496
============================  ===========================
1497
fix-superusers                Transform superusers created by syncdb into AstakosUser instances
1498
cleanup-full                  Cleanup sessions and session catalog
1499
commission-list               List pending commissions
1500
commission-show               Show details for a pending commission
1501
component-add                 Register a component
1502
component-list                List components
1503
component-modify              Modify component attributes
1504
component-show                Show component details
1505
project-control               Manage projects and applications
1506
project-list                  List projects
1507
project-show                  Show project details
1508
quota-list                    List user quota
1509
quota-verify                  Check the integrity of user quota
1510
reconcile-resources-astakos   Reconcile resource usage of Quotaholder with Astakos DB
1511
resource-list                 List resources
1512
resource-modify               Modify a resource's default base quota and boolean flags
1513
service-export-astakos        Export Astakos services and resources in JSON format
1514
service-import                Register services
1515
service-list                  List services
1516
service-show                  Show service details
1517
term-add                      Add approval terms
1518
user-activation-send          Send user activation
1519
user-add                      Add user
1520
authpolicy-add                Create a new authentication provider policy profile
1521
authpolicy-list               List existing authentication provider policy profiles
1522
authpolicy-remove             Remove an authentication provider policy
1523
authpolicy-set                Assign an existing authentication provider policy profile to a user or group
1524
authpolicy-show               Show authentication provider profile details
1525
group-add                     Create a group with the given name
1526
group-list                    List available groups
1527
user-list                     List users
1528
user-modify                   Modify user
1529
user-show                     Show user details
1530
oauth2-client-add             Create an oauth2 client
1531
oauth2-client-list            List oauth2 clients
1532
oauth2-client-remove          Remove an oauth2 client along with its registered redirect urls
1533
============================  ===========================
1534

    
1535
Pithos snf-manage commands
1536
--------------------------
1537

    
1538
============================  ===========================
1539
Name                          Description
1540
============================  ===========================
1541
reconcile-commissions-pithos  Display unresolved commissions and trigger their recovery
1542
service-export-pithos         Export Pithos services and resources in JSON format
1543
reconcile-resources-pithos    Detect unsynchronized usage between Astakos and Pithos DB resources and synchronize them if specified so.
1544
file-show                     Display object information
1545
============================  ===========================
1546

    
1547
Cyclades snf-manage commands
1548
----------------------------
1549

    
1550
============================== ===========================
1551
Name                           Description
1552
============================== ===========================
1553
backend-add                    Add a new Ganeti backend
1554
backend-list                   List backends
1555
backend-modify                 Modify a backend
1556
backend-update-status          Update backend statistics for instance allocation
1557
backend-remove                 Remove a Ganeti backend
1558
enforce-resources-cyclades     Check and fix quota violations for Cyclades resources
1559
server-create                  Create a new server
1560
server-show                    Show server details
1561
server-list                    List servers
1562
server-modify                  Modify a server
1563
server-import                  Import an existing Ganeti VM into synnefo
1564
server-inspect                 Inspect a server in DB and Ganeti
1565
network-create                 Create a new network
1566
network-list                   List networks
1567
network-modify                 Modify a network
1568
network-inspect                Inspect network state in DB and Ganeti
1569
network-remove                 Delete a network
1570
flavor-create                  Create a new flavor
1571
flavor-list                    List flavors
1572
flavor-modify                  Modify a flavor
1573
image-list                     List images
1574
image-show                     Show image details
1575
pool-create                    Create a bridge or mac-prefix pool
1576
pool-show                      Show pool details
1577
pool-list                      List pools
1578
pool-modify                    Modify a pool
1579
pool-remove                    Delete a pool
1580
port-create                    Create a port connecting a server to a network
1581
port-inspect                   Inspect the state of a port in DB and Ganeti
1582
port-list                      List ports
1583
port-remove                    Delete a port
1584
floating-ip-create             Create a new floating IP
1585
floating-ip-attach             Attach a floating IP to a server
1586
floating-ip-detach             Detach a floating IP from a server
1587
floating-ip-list               List floating IPs
1588
floating-ip-remove             Delete a floating IP
1589
queue-inspect                  Inspect the messages of a RabbitMQ queue
1590
queue-retry                    Resend messages from Dead Letter queues to original exchanges
1591
service-export-cyclades        Export Cyclades services and resources in JSON format
1592
subnet-create                  Create a subnet
1593
subnet-inspect                 Inspect a subnet in DB
1594
subnet-list                    List subnets
1595
subnet-modify                  Modify a subnet
1596
reconcile-servers              Reconcile servers of Synnefo DB with state of Ganeti backend
1597
reconcile-networks             Reconcile networks of Synnefo DB with state of Ganeti backend
1598
reconcile-pools                Check consistency of pool resources
1599
reconcile-commissions-cyclades Detect and resolve pending commissions to Quotaholder
1600
reconcile-resources-cyclades   Reconcile resource usage of Astakos with Cyclades DB.
1601
============================== ===========================
1602

    
1603

    
1604
Astakos helper scripts
1605
======================
1606

    
1607
Astakos includes two scripts to facilitate the installation procedure.
1608
Running:
1609

    
1610
.. code-block:: console
1611

    
1612
   snf-component-register [<component_name>]
1613

    
1614
automates the registration of the standard Synnefo components (astakos,
1615
cyclades, and pithos) in astakos database. It internally uses the script:
1616

    
1617
.. code-block:: console
1618

    
1619
   snf-service-export <component_name> <base_url>
1620

    
1621
which simulates the export of service and resource definitions of the
1622
standard Synnefo components.
1623

    
1624

    
1625
Pithos managing accounts
1626
========================
1627

    
1628
Pithos provides a utility tool for managing accounts.
1629
To run you just type:
1630

    
1631
.. code-block:: console
1632

    
1633
   # pithos-manage-accounts <command> [arguments]
1634

    
1635
This is the list of the available commands:
1636

    
1637
============================  ===========================
1638
Name                          Description
1639
============================  ===========================
1640
delete                        Remove an account from the Pithos DB
1641
export-quota                  Export account quota in a file
1642
list                          List existing/dublicate accounts
1643
merge                         Move an account contents in another account
1644
set-container-quota           Set container quota for all or a specific account
1645
============================  ===========================
1646

    
1647

    
1648
The "kamaki" API client
1649
=======================
1650

    
1651
To upload, register or modify an image you will need the **kamaki** tool.
1652
Before proceeding make sure that it is configured properly. Verify that
1653
*image.url*, *file.url*, *user.url* and *token* are set as needed:
1654

    
1655
.. code-block:: console
1656

    
1657
   $ kamaki config list
1658

    
1659
To change a setting use ``kamaki config set``:
1660

    
1661
.. code-block:: console
1662

    
1663
   $ kamaki config set image.url https://cyclades.example.com/image
1664
   $ kamaki config set file.url https://pithos.example.com/v1
1665
   $ kamaki config set user.url https://accounts.example.com
1666
   $ kamaki config set token ...
1667

    
1668
To test that everything works, try authenticating the current account with
1669
kamaki:
1670

    
1671
.. code-block:: console
1672

    
1673
  $ kamaki user authenticate
1674

    
1675
This will output user information.
1676

    
1677
Upload Image
1678
------------
1679

    
1680
By convention, images are stored in a container called ``images``. Check if the
1681
container exists, by listing all containers in your account:
1682

    
1683
.. code-block:: console
1684

    
1685
   $ kamaki file list
1686

    
1687
If the container ``images`` does not exist, create it:
1688

    
1689
.. code-block:: console
1690

    
1691
  $ kamaki file create images
1692

    
1693
You are now ready to upload an image to container ``images``. You can upload it
1694
with a Pithos client, or use kamaki directly:
1695

    
1696
.. code-block:: console
1697

    
1698
   $ kamaki file upload ubuntu.iso images
1699

    
1700
You can use any Pithos client to verify that the image was uploaded correctly,
1701
or you can list the contents of the container with kamaki:
1702

    
1703
.. code-block:: console
1704

    
1705
  $ kamaki file list images
1706

    
1707
The full Pithos URL for the previous example will be
1708
``pithos://u53r-un1qu3-1d/images/ubuntu.iso`` where ``u53r-un1qu3-1d`` is the
1709
unique user id (uuid).
1710

    
1711
Register Image
1712
--------------
1713

    
1714
To register an image you will need to use the full Pithos URL. To register as
1715
a public image the one from the previous example use:
1716

    
1717
.. code-block:: console
1718

    
1719
   $ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso --public
1720

    
1721
The ``--public`` flag is important, if missing the registered image will not
1722
be listed by ``kamaki image list``.
1723

    
1724
Use ``kamaki image register`` with no arguments to see a list of available
1725
options. A more complete example would be the following:
1726

    
1727
.. code-block:: console
1728

    
1729
   $ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso \
1730
            --public --disk-format diskdump --property kernel=3.1.2
1731

    
1732
To verify that the image was registered successfully use:
1733

    
1734
.. code-block:: console
1735

    
1736
   $ kamaki image list --name-like=ubuntu
1737

    
1738

    
1739
Miscellaneous
1740
=============
1741

    
1742
.. _branding:
1743

    
1744
Branding
1745
--------
1746

    
1747
Since Synnefo v0.14, you are able to adapt the Astakos, Pithos and Cyclades Web
1748
UI to your company’s visual identity. This is possible using the snf-branding
1749
component, which is automatically installed on the nodes running the API
1750
servers for Astakos, Pithos and Cyclades.
1751

    
1752
Configuration
1753
~~~~~~~~~~~~~
1754

    
1755
This can be done by modifing the settings provided by the snf-branding component
1756
to match your service identity. The settings for the snf-branding application
1757
can be found inside the configuration file ``/etc/synnefo/15-snf-branding.conf``
1758
on the nodes that have Astakos, Pithos and Cyclades installed.
1759

    
1760
By default, the global service name is "Synnefo" and the company name is
1761
"GRNET". These names and their respective logos and URLs are used throughout
1762
the Astakos, Pithos and Cyclades UI.
1763

    
1764
**Names and URLs:**
1765

    
1766
The first group of branding customization refers to the service's and company's
1767
information.
1768

    
1769
You can overwrite the company and the service name and URL respectively by
1770
uncommenting and setting the following:
1771

    
1772
.. code-block:: python
1773

    
1774
  # setting used in Astakos Dashboard/Projects pages
1775
  BRANDING_SERVICE_NAME = 'My cloud'
1776
  BRANDING_SERVICE_URL = 'http://www.mycloud.synnefo.org/'
1777

    
1778
  # settings used in Astakos, Pithos, Cyclades footer only if
1779
  # BRANDING_SHOW_COPYRIGHT is set to True
1780
  BRANDING_SHOW_COPYRIGHT = True
1781
  BRANDING_COMPANY_NAME = 'Company LTD'
1782
  BRANDING_COMPANY_URL = 'https://www.company-ltd.synnefo.org/'
1783

    
1784

    
1785
**Copyright and footer options:**
1786

    
1787
By default, no Copyright message is shown in the UI footer. If you want to make
1788
it visible in the footer of Astakos, Pithos and Cyclades UI, you can uncomment
1789
and set to ``True`` the ``BRANDING_SHOW_COPYRIGHT`` setting:
1790

    
1791
.. code-block:: python
1792

    
1793
  #BRANDING_SHOW_COPYRIGHT = False
1794

    
1795
Copyright message defaults to 'Copyright (c) 2011-<current_year>
1796
<BRANDING_COMPANY_NAME>.' but you can overwrite it to a completely custom one by
1797
setting the following option:
1798

    
1799
.. code-block:: python
1800

    
1801
  BRANDING_COPYRIGHT_MESSAGE = 'Copyright (c) 2011-2013 GRNET'
1802

    
1803
If you want to include a custom message in the footer, you can uncomment and
1804
set the ``BRANDING_FOOTER_EXTRA_MESSAGE`` setting. You can use html markup.
1805
Your custom message will appear  above Copyright message at the Compute
1806
templates and the Dashboard UI.
1807

    
1808
.. code-block:: python
1809

    
1810
  #BRANDING_FOOTER_EXTRA_MESSAGE = ''
1811

    
1812

    
1813
**Images:**
1814

    
1815
The Astakos, Pithos and Cyclades Web UI has some logos and images.
1816

    
1817
The branding-related images are presented in  the following table:
1818

    
1819
===============  ============================  =========
1820
Image            Name/extension  convention    Usage
1821
===============  ============================  =========
1822
Favicon          favicon.ico                   Favicon for all services
1823
Dashboard logo   dashboard_logo.png            Visible in all Astakos UI pages
1824
Compute logo     compute_logo.png              Visible in all Cyclades UI pages
1825
Console logo     console_logo.png              Visible in the Cyclades Console Window
1826
Storage logo     storage_logo.png              Visible in all Pithos UI pages
1827
===============  ============================  =========
1828

    
1829
There are two methods  available for replacing all, or individual,
1830
branding-related images:
1831

    
1832
1. Create a new directory inside ``/usr/share/synnefo/static/`` (e.g.
1833
   ``mybranding``) and place there some or all of your images.
1834

    
1835
   If you want to replace all of your images, keep the name/extension
1836
   conventions as indicated in the above table and change the
1837
   ``BRANDING_IMAGE_MEDIA_URL`` setting accordingly:
1838

    
1839
   .. code-block:: python
1840

    
1841
      # using relative path
1842
      BRANDING_IMAGE_MEDIA_URL= '/static/mybranding/images/'
1843

    
1844
      # or if you already host them in a separate domain (e.g. cdn)
1845
      BRANDING_IMAGE_MEDIA_URL= 'https://cdn.synnefo.org/branding/images/'
1846

    
1847

    
1848
   If you wish to replace individual images, **do not uncomment**
1849
   ``BRANDING_IMAGE_MEDIA_URL``, but instead provide a relative path, pointing to
1850
   the file inside your directory for each ``BRANDING_<image>_URL`` that you wish
1851
   to replace.
1852

    
1853
2. Upload some or all of your images to a server and replace each
1854
   ``BRANDING_<image>_URL`` with the absolute url of the image (i.e.
1855
   ``BRANDING_DASHBOARD_URL = 'https://www.synnefo.com/images/my_dashboard.jpg'``).
1856

    
1857
   Note that the alternative text  for each image tag inside html documents is
1858
   alt=“BRANDING_SERVICE_NAME {Dashboard, Compute. Console, Storage}” respectively.
1859

    
1860
.. note:: Retina optimized images:
1861

    
1862
   Synnefo UI is optimized for Retina displays. As far as images are concerned,
1863
   `retina.js <http://retinajs.com/>`_ is used.
1864

    
1865
   Retina.js checks each image on a page to see if there is a high-resolution
1866
   version of that image on your server. If a high-resolution variant exists,
1867
   the script will swap in that image in-place.
1868

    
1869
   The script assumes you use  `Apple's prescribed high-resolution modifier (@2x)
1870
   <http://developer.apple.com/library/ios/#documentation/2DDrawing/Conceptual/
1871
   DrawingPrintingiOS/SupportingHiResScreensInViews/SupportingHiResScreensInViews
1872
   .html#//apple_ref/doc/uid/TP40010156-CH15-SW1>`_ to denote high-resolution
1873
   image variants on your server.
1874

    
1875
   For each of the images that you wish the script to  replace, you must have a
1876
   high-resolution variant in the same folder  named correctly and it will be
1877
   detected automatically. For example if your image is in <my_directory> and is
1878
   named "my_image.jpg" the script will look in the same directory for an image
1879
   named "my_image@2x.jpg".
1880

    
1881
   In case that you don’t want to use a high-resolution image, the
1882
   normal-resolution image will be visible.
1883

    
1884
More branding
1885
~~~~~~~~~~~~~
1886

    
1887
Although, it is not 100% branding-related, further verbal customization is
1888
feasible.
1889

    
1890
**EMAILS**
1891

    
1892
The output of all email `*`.txt files will be already customized to contain your
1893
company and service names but you can further alter their content if you feel it
1894
best fits your needs as simple as creasynnefo template.
1895

    
1896
In order to overwrite one or more email-templates you need to place your
1897
modified <email-file>.txt files respecting the following structure:
1898

    
1899
  **/etc/synnefo/templates/**
1900
      **im/**
1901
          | activation_email.txt
1902
          | email.txt
1903
          | invitation.txt
1904
          | switch_accounts_email.txt
1905
          | welcome_email.txt
1906
          **projects/**
1907
              | project_approval_notification.txt
1908
              | project_denial_notification.txt
1909
              | project_membership_change_notification.txt
1910
              | project_membership_enroll_notification.txt
1911
              | project_membership_leave_request_notification.txt
1912
              | project_membership_request_notification.txt
1913
              | project_suspension_notification.txt
1914
              | project_termination_notification.txt
1915
      **registration/**
1916
          | email_change_email.txt
1917
          | password_email.txt
1918

    
1919
Feel free to omit any of the above files you do not wish to overwrite.
1920

    
1921
Below is a list of all emails sent by Synnefo to users along with a short
1922
description and a link to their content:
1923

    
1924
* ``snf-astakos-app/astakos/im/templates/im/email.txt``
1925
  Base email template. Contains a contact email and a “thank you” message.
1926
  (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/email.txt>`_)
1927
* ``snf-astakos-app/astakos/im/templates/im/activation_email.txt`` Email sent to
1928
  user that prompts  him/her to click on a link provided to activate the account.
1929
  Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/activation_email.txt>`_)
1930
* ``snf-astakos-app/astakos/im/templates/im/invitation.txt`` Email sent to an
1931
  invited user. He/she has to click on a link provided to activate the account.
1932
  Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/invitation.txt>`_)
1933
* ``snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt`` Email
1934
  sent to user upon his/her request to associate this email address with a
1935
  shibboleth account. He/she has to click on a link provided to activate the
1936
  association. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt>`_)
1937
* ``snf-astakos-app/astakos/im/templates/im/welcome_email.txt`` Email sent to
1938
  inform the user that his/ her account has been activated. Extends “email.txt”
1939
  (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/welcome_email.txt>`_)
1940
* ``snf-astakos-app/astakos/im/templates/registration/email_change_email.txt``
1941
  Email sent to user when he/she has requested new email address assignment. The
1942
  user has to click on a link provided to validate this action. Extends
1943
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/email_change_email.txt>`_)
1944
* ``snf-astakos-app/astakos/im/templates/registration/password_email.txt`` Email
1945
  sent for resetting password purpose. The user has to click on a link provided
1946
  to validate this action. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/password_email.txt>`_)
1947
* ``snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt``
1948
  Informs  the project owner that his/her project has been approved. Extends
1949
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt>`_)
1950
* ``snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt``
1951
  Informs the project owner that his/her  project application has been denied
1952
  explaining the reasons. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt>`_)
1953
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt``
1954
  An email is sent to a user containing information about his project membership
1955
  (whether he has been accepted, rejected or removed). Extends “email.txt” (`Link
1956
  <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt>`_)
1957
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt``
1958
  Informs a user that he/she  has been enrolled to a project. Extends
1959
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt>`_)
1960
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt``
1961
  An email is sent to the project owner to make him aware of a  user having
1962
  requested to leave his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt>`_)
1963
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt``
1964
  An email is sent to the project owner to make him/her aware of a user having
1965
  requested to join  his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt>`_)
1966
* ``snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt``
1967
  An email is sent to the project owner to make him/her aware of his/her project
1968
  having been suspended. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt>`_)
1969
* ``snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt``
1970
  An email is sent to the project owner to make him/her aware of his/her project
1971
  having been terminated. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt>`_)
1972

    
1973
.. warning:: Django templates language:
1974

    
1975
  If you choose to  overwrite these email templates, be mindful of the necessary
1976
  information contained in django template variables that must not be omitted,
1977
  such as the activation link for activating one’s account and many more.
1978
  These variables are contained into {{}} inside the templates.
1979

    
1980

    
1981
.. RabbitMQ
1982

    
1983
RabbitMQ Broker
1984
---------------
1985

    
1986
Queue nodes run the RabbitMQ sofware, which provides AMQP functionality. To
1987
guarantee high-availability, more than one Queue nodes should be deployed, each
1988
of them belonging to the same `RabbitMQ cluster
1989
<http://www.rabbitmq.com/clustering.html>`_. Synnefo uses the RabbitMQ
1990
active/active `High Available Queues <http://www.rabbitmq.com/ha.html>`_ which
1991
are mirrored between two nodes within a RabbitMQ cluster.
1992

    
1993
The RabbitMQ nodes that form the cluster, are declared to Synnefo through the
1994
`AMQP_HOSTS` setting. Each time a Synnefo component needs to connect to
1995
RabbitMQ, one of these nodes is chosen in a random way. The client that Synnefo
1996
uses to connect to RabbitMQ, handles connection failures transparently and
1997
tries to reconnect to a different node. As long as one of these nodes are up
1998
and running, functionality of Synnefo should not be downgraded by the RabbitMQ
1999
node failures.
2000

    
2001
All the queues that are being used are declared as durable, meaning that
2002
messages are persistently stored to RabbitMQ, until they get successfully
2003
processed by a client.
2004

    
2005
Currently, RabbitMQ is used by the following components:
2006

    
2007
* `snf-ganeti-eventd` and `snf-progress-monitor`:
2008
  These components send messages concerning the status and progress of
2009
  jobs in the Ganeti backend.
2010
* `snf-dispatcher`: This daemon, consumes the messages that are sent from
2011
  the above components, and updates the Cyclades DB accordingly.
2012

    
2013

    
2014
Installation
2015
~~~~~~~~~~~~
2016

    
2017
Please check the RabbitMQ documentation which covers extensively the
2018
`installation of RabbitMQ server <http://www.rabbitmq.com/download.html>`_ and
2019
the setup of a `RabbitMQ cluster <http://www.rabbitmq.com/clustering.html>`_.
2020
Also, check out the `web management plugin
2021
<http://www.rabbitmq.com/management.html>`_ that can be useful for managing and
2022
monitoring RabbitMQ.
2023

    
2024
For a basic installation of RabbitMQ on two nodes (node1 and node2) you can do
2025
the following:
2026

    
2027
On both nodes, install rabbitmq-server and create a Synnefo user:
2028

    
2029
.. code-block:: console
2030

    
2031
  $ apt-get install rabbitmq-server
2032
  $ rabbitmqctl add_user synnefo "example_pass"
2033
  $ rabbitmqctl set_permissions synnefo  ".*" ".*" ".*"
2034

    
2035
Also guarantee that both nodes share the same cookie, by running:
2036

    
2037
.. code-block:: console
2038

    
2039
  $ scp node1:/var/lib/rabbitmq/.erlang.cookie node2:/var/lib/rabbitmq/.erlang.cookie
2040

    
2041
and restart the nodes:
2042

    
2043
.. code-block:: console
2044

    
2045
  $ /etc/init.d/rabbitmq-server restart
2046

    
2047

    
2048
To setup the RabbitMQ cluster run:
2049

    
2050
.. code-block:: console
2051

    
2052
  root@node2: rabbitmqctl stop_app
2053
  root@node2: rabbitmqctl reset
2054
  root@node2: rabbitmqctl cluster rabbit@node1 rabbit@node2
2055
  root@node2: rabbitmqctl start_app
2056

    
2057
You can verify that the cluster is set up correctly by running:
2058

    
2059
.. code-block:: console
2060

    
2061
  root@node2: rabbitmqctl cluster_status
2062

    
2063

    
2064
Logging
2065
-------
2066

    
2067
Logging in Synnefo is using Python's logging module. The module is configured
2068
using dictionary configuration, whose format is described here:
2069

    
2070
http://docs.python.org/release/2.7.1/library/logging.html#logging-config-dictschema
2071

    
2072
The logging configuration dictionary is defined in
2073
``/etc/synnefo/10-snf-webproject-logging.conf``
2074

    
2075
The administrator can have logging control by modifying the ``LOGGING_SETUP``
2076
dictionary, and defining subloggers with different handlers and log levels.
2077

    
2078

    
2079
.. _scale-up:
2080

    
2081
Scaling up to multiple nodes
2082
============================
2083

    
2084
Here we will describe how should a large scale Synnefo deployment look like. Make
2085
sure you are familiar with Synnefo and Ganeti before proceeding with this section.
2086
This means you should at least have already set up successfully a working Synnefo
2087
deployment as described in the :ref:`Admin's Installation Guide
2088
<quick-install-admin-guide>` and also read the Administrator's Guide until this
2089
section.
2090

    
2091
Graph of a scale-out Synnefo deployment
2092
---------------------------------------
2093

    
2094
Each box in the following graph corresponds to a distinct physical node:
2095

    
2096
.. image:: images/synnefo-arch2-roles.png
2097
   :width: 100%
2098
   :target: _images/synnefo-arch2-roles.png
2099

    
2100
The above graph is actually the same with the one at the beginning of this
2101
:ref:`guide <admin-guide>`, with the only difference that here we show the
2102
Synnefo roles of each physical node. These roles are described in the
2103
following section.
2104

    
2105
.. _physical-node-roles:
2106

    
2107
Physical Node roles
2108
-------------------
2109

    
2110
As appears in the previous graph, a scale-out Synnefo deployment consists of
2111
multiple physical nodes that have the following roles:
2112

    
2113
* **WEBSERVER**: A web server running in front of gunicorn (e.g.: Apache, nginx)
2114
* **ASTAKOS**: The Astakos application (gunicorn)
2115
* **ASTAKOS_DB**: The Astakos database (postgresql)
2116
* **PITHOS**: The Pithos application (gunicorn)
2117
* **PITHOS_DB**: The Pithos database (postgresql)
2118
* **CYCLADES**: The Cyclades application (gunicorn)
2119
* **CYCLADES_DB**: The Cyclades database (postgresql)
2120
* **MQ**: The message queue (RabbitMQ)
2121
* **GANETI_MASTER**: The Ganeti master of a Ganeti cluster
2122
* **GANETI_NODE** : A VM-capable Ganeti node of a Ganeti cluster
2123

    
2124
You will probably also have:
2125

    
2126
* **CMS**: The CMS used as a frotend portal for the Synnefo services
2127
* **NS**: A nameserver serving all other Synnefo nodes and resolving Synnefo FQDNs
2128
* **CLIENT**: A machine that runs the Synnefo clients (e.g.: kamaki, Web UI),
2129
              most of the times, the end user's local machine
2130

    
2131
From this point we will also refer to the following groups of roles:
2132

    
2133
* **SYNNEFO**: [ **ASTAKOS**, **ASTAKOS_DB**, **PITHOS**, **PITHOS_DB**, **CYCLADES**, **CYCLADES_DB**, **MQ**, **CMS**]
2134
* **G_BACKEND**: [**GANETI_MASTER**, **GANETI_NODE**]
2135

    
2136
Of course, when deploying Synnefo you can combine multiple of the above roles on a
2137
single physical node, but if you are trying to scale out, the above separation
2138
gives you significant advantages.
2139

    
2140
So, in the next section we will take a look on what components you will have to
2141
install on each physical node depending on its Synnefo role. We assume the graph's
2142
architecture.
2143

    
2144
Components for each role
2145
------------------------
2146

    
2147
When deploying Synnefo in large scale, you need to install different Synnefo
2148
or/and third party components on different physical nodes according to their
2149
Synnefo role, as stated in the previous section.
2150

    
2151
Specifically:
2152

    
2153
Role **WEBSERVER**
2154
    * Synnefo components: `None`
2155
    * 3rd party components: Apache
2156
Role **ASTAKOS**
2157
    * Synnefo components: `snf-webproject`, `snf-astakos-app`
2158
    * 3rd party components: Django, Gunicorn
2159
Role **ASTAKOS_DB**
2160
    * Synnefo components: `None`
2161
    * 3rd party components: PostgreSQL
2162
Role **PITHOS**
2163
    * Synnefo components: `snf-webproject`, `snf-pithos-app`, `snf-pithos-webclient`
2164
    * 3rd party components: Django, Gunicorn
2165
Role **PITHOS_DB**
2166
    * Synnefo components: `None`
2167
    * 3rd party components: PostgreSQL
2168
Role **CYCLADES**
2169
    * Synnefo components: `snf-webproject`, `snf-cyclades-app`, `snf-vncauthproxy`
2170
    * 3rd party components: Django Gunicorn
2171
Role **CYCLADES_DB**
2172
    * Synnefo components: `None`
2173
    * 3rd party components: PostgreSQL
2174
Role **MQ**
2175
    * Synnefo components: `None`
2176
    * 3rd party components: RabbitMQ
2177
Role **GANETI_MASTER**
2178
    * Synnefo components: `snf-cyclades-gtools`
2179
    * 3rd party components: Ganeti
2180
Role **GANETI_NODE**
2181
    * Synnefo components: `snf-cyclades-gtools`, `snf-network`, `snf-image`, `nfdhcpd`
2182
    * 3rd party components: Ganeti
2183
Role **CMS**
2184
    * Synnefo components: `snf-webproject`, `snf-cloudcms`
2185
    * 3rd party components: Django, Gunicorn
2186
Role **NS**
2187
    * Synnefo components: `None`
2188
    * 3rd party components: BIND
2189
Role **CLIENT**
2190
    * Synnefo components: `kamaki`, `snf-image-creator`
2191
    * 3rd party components: `None`
2192

    
2193
Example scale out installation
2194
------------------------------
2195

    
2196
In this section we describe an example of a medium scale installation which
2197
combines multiple roles on 10 different physical nodes. We also provide a
2198
:ref:`guide <i-synnefo>` to help with such an install.
2199

    
2200
We assume that we have the following 10 physical nodes with the corresponding
2201
roles:
2202

    
2203
Node1:
2204
    **WEBSERVER**, **ASTAKOS**
2205
      Guide sections:
2206
        * :ref:`apt <i-apt>`
2207
        * :ref:`gunicorn <i-gunicorn>`
2208
        * :ref:`apache <i-apache>`
2209
        * :ref:`snf-webproject <i-webproject>`
2210
        * :ref:`snf-astakos-app <i-astakos>`
2211
Node2:
2212
    **WEBSERVER**, **PITHOS**
2213
      Guide sections:
2214
        * :ref:`apt <i-apt>`
2215
        * :ref:`gunicorn <i-gunicorn>`
2216
        * :ref:`apache <i-apache>`
2217
        * :ref:`snf-webproject <i-webproject>`
2218
        * :ref:`snf-pithos-app <i-pithos>`
2219
        * :ref:`snf-pithos-webclient <i-pithos>`
2220
Node3:
2221
    **WEBSERVER**, **CYCLADES**
2222
      Guide sections:
2223
        * :ref:`apt <i-apt>`
2224
        * :ref:`gunicorn <i-gunicorn>`
2225
        * :ref:`apache <i-apache>`
2226
        * :ref:`snf-webproject <i-webproject>`
2227
        * :ref:`snf-cyclades-app <i-cyclades>`
2228
        * :ref:`snf-vncauthproxy <i-cyclades>`
2229
Node4:
2230
    **WEBSERVER**, **CMS**
2231
      Guide sections:
2232
        * :ref:`apt <i-apt>`
2233
        * :ref:`gunicorn <i-gunicorn>`
2234
        * :ref:`apache <i-apache>`
2235
        * :ref:`snf-webproject <i-webproject>`
2236
        * :ref:`snf-cloudcms <i-cms>`
2237
Node5:
2238
    **ASTAKOS_DB**, **PITHOS_DB**, **CYCLADES_DB**
2239
      Guide sections:
2240
        * :ref:`apt <i-apt>`
2241
        * :ref:`postgresql <i-db>`
2242
Node6:
2243
    **MQ**
2244
      Guide sections:
2245
        * :ref:`apt <i-apt>`
2246
        * :ref:`rabbitmq <i-mq>`
2247
Node7:
2248
    **GANETI_MASTER**, **GANETI_NODE**
2249
      Guide sections:
2250
        * :ref:`apt <i-apt>`
2251
        * :ref:`general <i-backends>`
2252
        * :ref:`ganeti <i-ganeti>`
2253
        * :ref:`snf-cyclades-gtools <i-gtools>`
2254
        * :ref:`snf-network <i-network>`
2255
        * :ref:`snf-image <i-image>`
2256
        * :ref:`nfdhcpd <i-network>`
2257
Node8:
2258
    **GANETI_NODE**
2259
      Guide sections:
2260
        * :ref:`apt <i-apt>`
2261
        * :ref:`general <i-backends>`
2262
        * :ref:`ganeti <i-ganeti>`
2263
        * :ref:`snf-cyclades-gtools <i-gtools>`
2264
        * :ref:`snf-network <i-network>`
2265
        * :ref:`snf-image <i-image>`
2266
        * :ref:`nfdhcpd <i-network>`
2267
Node9:
2268
    **GANETI_NODE**
2269
      Guide sections:
2270
        `Same as Node8`
2271
Node10:
2272
    **GANETI_NODE**
2273
      Guide sections:
2274
        `Same as Node8`
2275

    
2276
All sections: :ref:`Scale out Guide <i-synnefo>`
2277

    
2278

    
2279
Upgrade Notes
2280
=============
2281

    
2282
.. toctree::
2283
   :maxdepth: 1
2284

    
2285
   v0.12 -> v0.13 <upgrade/upgrade-0.13>
2286
   v0.13 -> v0.14 <upgrade/upgrade-0.14>
2287
   v0.14 -> v0.14.2 <upgrade/upgrade-0.14.2>
2288
   v0.14.5 -> v0.14.6 <upgrade/upgrade-0.14.6>
2289
   v0.14.7 -> v0.14.8 <upgrade/upgrade-0.14.8>
2290
   v0.14.9 -> v0.14.10 <upgrade/upgrade-0.14.10>
2291
   v0.14 -> v0.15 <upgrade/upgrade-0.15>
2292

    
2293

    
2294
Changelog, NEWS
2295
===============
2296

    
2297

    
2298
* v0.14.10 :ref:`Changelog <Changelog-0.14.10>`, :ref:`NEWS <NEWS-0.14.10>`
2299
* v0.14.9 :ref:`Changelog <Changelog-0.14.9>`, :ref:`NEWS <NEWS-0.14.9>`
2300
* v0.14.8 :ref:`Changelog <Changelog-0.14.8>`, :ref:`NEWS <NEWS-0.14.8>`
2301
* v0.14.7 :ref:`Changelog <Changelog-0.14.7>`, :ref:`NEWS <NEWS-0.14.7>`
2302
* v0.14.6 :ref:`Changelog <Changelog-0.14.6>`, :ref:`NEWS <NEWS-0.14.6>`
2303
* v0.14.5 :ref:`Changelog <Changelog-0.14.5>`, :ref:`NEWS <NEWS-0.14.5>`
2304
* v0.14.4 :ref:`Changelog <Changelog-0.14.4>`, :ref:`NEWS <NEWS-0.14.4>`
2305
* v0.14.3 :ref:`Changelog <Changelog-0.14.3>`, :ref:`NEWS <NEWS-0.14.3>`
2306
* v0.14.2 :ref:`Changelog <Changelog-0.14.2>`, :ref:`NEWS <NEWS-0.14.2>`
2307
* v0.14 :ref:`Changelog <Changelog-0.14>`, :ref:`NEWS <NEWS-0.14>`
2308
* v0.13 :ref:`Changelog <Changelog-0.13>`, :ref:`NEWS <NEWS-0.13>`