Statistics
| Branch: | Tag: | Revision:

root / docs / admin-guide.rst @ 408cd6a5

History | View | Annotate | Download (100.3 kB)

1
.. _admin-guide:
2

    
3
Synnefo Administrator's Guide
4
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5

    
6
This is the complete Synnefo Administrator's Guide.
7

    
8

    
9
.. _syn+archip:
10

    
11
General Synnefo Architecture
12
============================
13

    
14
The following figure shows a detailed view of the whole Synnefo architecture
15
and how it interacts with multiple Ganeti clusters. We hope that after reading
16
the Administrator's Guide you will be able to understand every component and
17
all the interactions between them.
18

    
19
.. image:: images/synnefo-arch2.png
20
   :width: 100%
21
   :target: _images/synnefo-arch2.png
22

    
23
Synnefo also supports RADOS as an alternative storage backend for
24
Files/Images/VM disks. You will find the :ref:`corresponding figure
25
<syn+archip+rados>` later in this guide.
26

    
27

    
28
Identity Service (Astakos)
29
==========================
30

    
31

    
32
Authentication methods
33
----------------------
34

    
35
Astakos supports multiple authentication methods:
36

    
37
 * local username/password
38
 * LDAP / Active Directory
39
 * SAML 2.0 (Shibboleth) federated logins
40
 * Google
41
 * Twitter
42
 * LinkedIn
43

    
44
.. _shibboleth-auth:
45

    
46
Shibboleth Authentication
47
~~~~~~~~~~~~~~~~~~~~~~~~~
48

    
49
Astakos can delegate user authentication to a Shibboleth federation.
50

    
51
To setup shibboleth, install package::
52

    
53
  apt-get install libapache2-mod-shib2
54

    
55
Change appropriately the configuration files in ``/etc/shibboleth``.
56

    
57
Add in ``/etc/apache2/sites-available/synnefo-ssl``::
58

    
59
  ShibConfig /etc/shibboleth/shibboleth2.xml
60
  Alias      /shibboleth-sp /usr/share/shibboleth
61

    
62
  <Location /ui/login/shibboleth>
63
    AuthType shibboleth
64
    ShibRequireSession On
65
    ShibUseHeaders On
66
    require valid-user
67
  </Location>
68

    
69
and before the line containing::
70

    
71
  ProxyPass        / http://localhost:8080/ retry=0
72

    
73
add::
74

    
75
  ProxyPass /Shibboleth.sso !
76

    
77
Then, enable the shibboleth module::
78

    
79
  a2enmod shib2
80

    
81
After passing through the apache module, the following tokens should be
82
available at the destination::
83

    
84
  eppn # eduPersonPrincipalName
85
  Shib-InetOrgPerson-givenName
86
  Shib-Person-surname
87
  Shib-Person-commonName
88
  Shib-InetOrgPerson-displayName
89
  Shib-EP-Affiliation
90
  Shib-Session-ID
91

    
92
Astakos keeps a map of shibboleth users using the value of the ``REMOTE_USER``
93
header, passed by the ``mod_shib2`` module. This happens in order to be able to
94
identify the astakos account the shibboleth user is associated to, every time
95
the user logs in from an affiliate shibboleth IdP. 
96

    
97
The shibboleth attribute which gets mapped to the ``REMOTE_USER`` header can be
98
changed in ``/etc/shibboleth/shibboleth2.xml`` configuration file.
99

    
100
.. code-block:: xml
101

    
102
    <!-- The ApplicationDefaults element is where most of Shibboleth's SAML bits are defined. -->
103
        <ApplicationDefaults entityID="https://sp.example.org/shibboleth" 
104
         REMOTE_USER="eppn persistent-id targeted-id">
105

    
106
.. warning::
107

    
108
 Changing ``mod_shib2`` ``REMOTE_USER`` to map to different shibboleth
109
 attributes will probably invalidate any existing shibboleth enabled users in
110
 astakos database. Those users won't be able to login to their existing accounts.
111

    
112

    
113
Finally, add 'shibboleth' in ``ASTAKOS_IM_MODULES`` list. The variable resides
114
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
115

    
116
Twitter Authentication
117
~~~~~~~~~~~~~~~~~~~~~~
118

    
119
To enable twitter authentication while signed in under a Twitter account,
120
visit dev.twitter.com/apps.
121

    
122
Click Create an application.
123

    
124
Fill the necessary information and for callback URL give::
125

    
126
    https://node1.example.com/ui/login/twitter/authenticated
127

    
128
Finally, add 'twitter' in ``ASTAKOS_IM_MODULES`` list. The variable resides
129
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
130

    
131
Google Authentication
132
~~~~~~~~~~~~~~~~~~~~~
133

    
134
To enable google authentication while signed in under a Google account,
135
visit https://code.google.com/apis/console/.
136

    
137
Under API Access select Create another client ID, select Web application,
138
expand more options in Your site or hostname section and in Authorized
139
Redirect URIs add:
140

    
141

    
142
Fill the necessary information and for callback URL give::
143

    
144
    https://node1.example.com/ui/login/google/authenticated
145

    
146
Finally, add 'google' in ``ASTAKOS_IM_MODULES`` list. The variable resides
147
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
148

    
149

    
150
Working with Astakos
151
--------------------
152

    
153
User registration
154
~~~~~~~~~~~~~~~~~
155

    
156
When a new user signs up, he/she is not directly marked as active. You can see
157
his/her state by running (on the machine that runs the Astakos app):
158

    
159
.. code-block:: console
160

    
161
   $ snf-manage user-list
162

    
163
More detailed user status is provided in the `status` field of the `user-show`
164
command:
165

    
166
.. code-block:: console
167

    
168
  $ snf-manage user-show <user-id>
169

    
170
  id                  : 6
171
  uuid                : 78661411-5eed-412f-a9ea-2de24f542c2e
172
  status              : Accepted/Active (accepted policy: manual)
173
  email               : user@synnefo.org
174
  ....
175

    
176
Based on the `astakos-app` configuration, there are several ways for a user to
177
get verified and activated in order to be able to login. We discuss the user
178
verification and activation flow in the following section.
179

    
180
User activation flow
181
````````````````````
182

    
183
A user can register for an account using the astakos signup form. Once the form
184
is submited successfully a user entry is created in astakos database. That entry
185
is passed through the astakos activation backend which handles whether the user
186
should be automatically verified and activated.
187

    
188
Email verification
189
``````````````````
190

    
191
The verification process takes place in order to ensure that the user owns the
192
email provided during the signup process. By default, after each successful
193
signup astakos notifies user with an verification url via email.
194

    
195
At this stage:
196

    
197
    * subsequent registrations invalidate and delete the previous registrations
198
      of the same email address.
199

    
200
    * in case user misses the initial notification, additional emails can be
201
      send either via the url which is prompted to the user if he tries to
202
      login, or by the administrator using the ``snf-manage user-activation-send
203
      <userid>`` command.
204

    
205
    * administrator may also enforce a user to get verified using the
206
      ``snf-manage user-modify --verify <userid>`` command.
207

    
208
Account activation
209
``````````````````
210

    
211
Once the user gets verified, it is time for Astakos to decide whether or not to
212
proceed through user activation process. If ``ASTAKOS_MODERATION_ENABLED``
213
setting is set to ``False`` (default value) user gets activated automatically.
214

    
215
In case the moderation is enabled Astakos may still automatically activate the
216
user in the following cases:
217

    
218
    * User email matches any of the regular expressions defined in
219
      ``ASTAKOS_RE_USER_EMAIL_PATTERNS`` (defaults to ``[]``)
220
    * User used a signup method (e.g. ``shibboleth``) for which automatic
221
      activation is enabled (see
222
      :ref:`authentication methods policies <auth_methods_policies>`).
223

    
224
If all of the above fail to trigger automatic activation, an email is sent to
225
the persons listed in ``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings,
226
notifing that there is a new user pending for moderation and that it's up to
227
the administrator to decide if the user should be activated. The UI also shows
228
a corresponding 'pending moderation' message to the user. The administrator can
229
activate a user using the ``snf-manage user-modify`` command:
230

    
231
.. code-block:: console
232

    
233
    # command to activate a pending user
234
    $ snf-manage user-modify --accept <userid>
235

    
236
    # command to reject a pending user
237
    $ snf-manage user-modify --reject --reject-reason="spammer" <userid>
238

    
239
Once the activation process finishes, a greeting message is sent to the user
240
email address and a notification for the activation to the persons listed in
241
``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings. Once activated the user is
242
able to login and access the Synnefo services.
243

    
244
Additional authentication methods
245
`````````````````````````````````
246

    
247
Astakos supports third party logins from external identity providers. This
248
can be usefull since it allows users to use their existing credentials to
249
login to astakos service.
250

    
251
Currently astakos supports the following identity providers:
252

    
253
    * `Shibboleth <http://www.internet2.edu/shibboleth>`_ (module name
254
      ``shibboleth``)
255
    * `Google <https://developers.google.com/accounts/docs/OAuth2>`_ (module
256
      name ``google``)
257
    * `Twitter <https://dev.twitter.com/docs/auth>`_ (module name ``twitter``)
258
    * `LinkedIn <http://developer.linkedin.com/documents/authentication>`_
259
      (module name ``linkedin``)
260

    
261
To enable any of the above modules (by default only ``local`` accounts are
262
allowed), retrieve and set the required provider settings and append the
263
module name in ``ASTAKOS_IM_MODULES``.
264

    
265
.. code-block:: python
266

    
267
    # settings from https://code.google.com/apis/console/
268
    ASTAKOS_GOOGLE_CLIENT_ID = '1111111111-epi60tvimgha63qqnjo40cljkojcann3.apps.googleusercontent.com'
269
    ASTAKOS_GOOGLE_SECRET = 'tNDQqTDKlTf7_LaeUcWTWwZM'
270

    
271
    # let users signup and login using their google account
272
    ASTAKOS_IM_MODULES = ['local', 'google']
273

    
274

    
275
.. _auth_methods_policies:
276

    
277
Authentication method policies
278
``````````````````````````````
279

    
280
Astakos allows you to override the default policies for each enabled provider
281
separately by adding the approriate settings in your ``.conf`` files in the
282
following format:
283

    
284
**ASTAKOS_AUTH_PROVIDER_<module>_<policy>_POLICY**
285

    
286
Available policies are:
287

    
288
    * **CREATE** Users can signup using that provider (default: ``True``)
289
    * **REMOVE/ADD** Users can remove/add login method from their profile
290
      (default: ``True``)
291
    * **AUTOMODERATE** Automatically activate users that signup using that
292
      provider (default: ``False``)
293
    * **LOGIN** Whether or not users can use the provider to login (default:
294
      ``True``).
295

    
296
e.g. to enable automatic activation for your academic users, while keeping
297
locally signed up users under moderation you can apply the following settings.
298

    
299
.. code-block:: python
300

    
301
    ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_AUTOMODERATE_POLICY = True
302
    ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_REMOVE_POLICY = False
303

    
304
User login
305
~~~~~~~~~~
306

    
307
During the logging procedure, the user is authenticated by the respective
308
identity provider.
309

    
310
If ``ASTAKOS_RECAPTCHA_ENABLED`` is set and the user fails several times
311
(``ASTAKOS_RATELIMIT_RETRIES_ALLOWED`` setting) to provide the correct
312
credentials for a local account, he/she is then prompted to solve a captcha
313
challenge.
314

    
315
Upon success, the system renews the token (if it has expired), logins the user
316
and sets the cookie, before redirecting the user to the ``next`` parameter
317
value.
318

    
319
Setting quota limits
320
~~~~~~~~~~~~~~~~~~~~
321

    
322
Set default quota
323
`````````````````
324
To inspect current default base quota limits, run::
325

    
326
   # snf-manage resource-list
327

    
328
You can modify the default base quota limit for all future users with::
329

    
330
   # snf-manage resource-modify <resource_name> --default-quota <value>
331

    
332
Set base quota for individual users
333
```````````````````````````````````
334

    
335
For individual users that need different quota than the default
336
you can set it for each resource like this::
337

    
338
    # use this to display quota / uuid
339
    # snf-manage user-show 'uuid or email' --quota
340

    
341
    # snf-manage user-modify <user-uuid> --base-quota 'cyclades.vm' 10
342

    
343
You can set base quota for all existing users, with possible exceptions, using::
344

    
345
    # snf-manage user-modify --all --base-quota cyclades.vm 10 --exclude uuid1,uuid2
346

    
347
All quota for which values different from the default have been set,
348
can be listed with::
349

    
350
    # snf-manage quota-list --with-custom=True
351

    
352

    
353
Enable the Projects feature
354
~~~~~~~~~~~~~~~~~~~~~~~~~~~
355

    
356
If you want to enable the projects feature so that users may apply
357
on their own for resources by creating and joining projects,
358
in ``20-snf-astakos-app-settings.conf`` set::
359

    
360
    # this will make the 'projects' page visible in the dashboard
361
    ASTAKOS_PROJECTS_VISIBLE = True
362

    
363
You can change the maximum allowed number of pending project applications
364
per user with::
365

    
366
    # snf-manage resource-modify astakos.pending_app --default-quota <number>
367

    
368
You can also set a user-specific limit with::
369

    
370
    # snf-manage user-modify <user-uuid> --base-quota 'astakos.pending_app' 5
371

    
372
When users apply for projects they are not automatically granted
373
the resources. They must first be approved by the administrator.
374

    
375
To list pending project applications in astakos::
376

    
377
    # snf-manage project-list --pending
378

    
379
Note the last column, the application id. To approve it::
380

    
381
    # <app id> from the last column of project-list
382
    # snf-manage project-control --approve <app id>
383

    
384
To deny an application::
385

    
386
    # snf-manage project-control --deny <app id>
387

    
388
Users designated as *project admins* can approve, deny, or modify
389
an application through the web interface. In
390
``20-snf-astakos-app-settings.conf`` set::
391

    
392
    # UUIDs of users that can approve or deny project applications from the web.
393
    ASTAKOS_PROJECT_ADMINS = [<uuid>, ...]
394

    
395

    
396
Astakos advanced operations
397
---------------------------
398

    
399
Adding "Terms of Use"
400
~~~~~~~~~~~~~~~~~~~~~
401

    
402
Astakos supports versioned terms-of-use. First of all you need to create an
403
html file that will contain your terms. For example, create the file
404
``/usr/share/synnefo/sample-terms.html``, which contains the following:
405

    
406
.. code-block:: console
407

    
408
   <h1>My cloud service terms</h1>
409

    
410
   These are the example terms for my cloud service
411

    
412
Then, add those terms-of-use with the snf-manage command:
413

    
414
.. code-block:: console
415

    
416
   $ snf-manage term-add /usr/share/synnefo/sample-terms.html
417

    
418
Your terms have been successfully added and you will see the corresponding link
419
appearing in the Astakos web pages' footer.
420

    
421
During the account registration, if there are approval terms, the user is
422
presented with an "I agree with the Terms" checkbox that needs to get checked
423
in order to proceed.
424

    
425
In case there are new approval terms that the user has not signed yet, the
426
``signed_terms_required`` view decorator redirects to the ``approval_terms``
427
view, so the user will be presented with the new terms the next time he/she
428
logins.
429

    
430
Enabling reCAPTCHA
431
~~~~~~~~~~~~~~~~~~
432

    
433
Astakos supports the `reCAPTCHA <http://www.google.com/recaptcha>`_ feature.
434
If enabled, it protects the Astakos forms from bots. To enable the feature, go
435
to https://www.google.com/recaptcha/admin/create and create your own reCAPTCHA
436
key pair. Then edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` and set
437
the corresponding variables to reflect your newly created key pair. Finally, set
438
the ``ASTAKOS_RECAPTCHA_ENABLED`` variable to ``True``:
439

    
440
.. code-block:: console
441

    
442
   ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
443
   ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
444

    
445
   ASTAKOS_RECAPTCHA_ENABLED = True
446

    
447
Restart the service on the Astakos node(s) and you are ready:
448

    
449
.. code-block:: console
450

    
451
   # /etc/init.d/gunicorn restart
452

    
453
Checkout your new Sign up page. If you see the reCAPTCHA box, you have setup
454
everything correctly.
455

    
456

    
457
Astakos internals
458
-----------------
459

    
460
X-Auth-Token
461
~~~~~~~~~~~~
462

    
463
Alice requests a specific resource from a cloud service e.g.: Pithos. In the
464
request she supplies the `X-Auth-Token` to identify whether she is eligible to
465
perform the specific task. The service contacts Astakos through its
466
``/account/v1.0/authenticate`` api call (see :ref:`authenticate-api-label`)
467
providing the specific ``X-Auth-Token``. Astakos checkes whether the token
468
belongs to an active user and it has not expired and returns a dictionary
469
containing user related information. Finally the service uses the ``uniq``
470
field included in the dictionary as the account string to identify the user
471
accessible resources.
472

    
473
.. _authentication-label:
474

    
475
Django Auth methods and Backends
476
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
477

    
478
Astakos incorporates Django user authentication system and extends its User model.
479

    
480
Since username field of django User model has a limitation of 30 characters,
481
AstakosUser is **uniquely** identified by the ``email`` instead. Therefore,
482
``astakos.im.authentication_backends.EmailBackend`` is served to authenticate a
483
user using email if the first argument is actually an email, otherwise tries
484
the username.
485

    
486
A new AstakosUser instance is assigned with a uui as username and also with a
487
``auth_token`` used by the cloud services to authenticate the user.
488
``astakos.im.authentication_backends.TokenBackend`` is also specified in order
489
to authenticate the user using the email and the token fields.
490

    
491
Logged on users can perform a number of actions:
492

    
493
 * access and edit their profile via: ``/im/profile``.
494
 * change their password via: ``/im/password``
495
 * send feedback for grnet services via: ``/im/send_feedback``
496
 * logout (and delete cookie) via: ``/im/logout``
497

    
498
Internal Astakos requests are handled using cookie-based Django user sessions.
499

    
500
External systems should forward to the ``/login`` URI. The server,
501
depending on its configuration will redirect to the appropriate login page.
502
When done with logging in, the service's login URI should redirect to the URI
503
provided with next, adding user and token parameters, which contain the email
504
and token fields respectively.
505

    
506
The login URI accepts the following parameters:
507

    
508
======================  =========================
509
Request Parameter Name  Value
510
======================  =========================
511
next                    The URI to redirect to when the process is finished
512
renew                   Force token renewal (no value parameter)
513
force                   Force logout current user (no value parameter)
514
======================  =========================
515

    
516
External systems inside the ``ASTAKOS_COOKIE_DOMAIN`` scope can acquire the
517
user information by the cookie identified by ``ASTAKOS_COOKIE_NAME`` setting
518
(set during the login procedure).
519

    
520
Finally, backend systems having acquired a token can use the
521
:ref:`authenticate-api-label` API call from a private network or through HTTPS.
522

    
523

    
524
File/Object Storage Service (Pithos+)
525
====================================
526

    
527
Pithos+ is the Synnefo component that implements a storage service and exposes
528
the associated OpenStack REST APIs with custom extensions.
529

    
530
Pithos+ advanced operations
531
---------------------------
532

    
533
Enable separate domain for serving user content
534
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
535

    
536
Since Synnefo v0.15, there is a possibility to serve untrusted user content
537
in an isolated domain.
538

    
539
Enabling this feature consists of the following steps:
540

    
541
#. **Declare new domain in apache server**
542

    
543
   In order to enable the apache server to serve several domains it is required
544
   to setup several virtual hosts.
545
   Therefore, for adding the new domain e.g. "user-content.example.com", append
546
   the following in ``/etc/apache2/sites-available/synnefo-ssl``:
547

    
548
    .. code-block:: console
549

    
550
        <VirtualHost _default_:443>
551
            ServerName user-content.example.com
552

    
553
            Alias /static "/usr/share/synnefo/static"
554

    
555
            #  SetEnv no-gzip
556
            #  SetEnv dont-vary
557

    
558
           AllowEncodedSlashes On
559

    
560
           RequestHeader set X-Forwarded-Protocol "https"
561

    
562
        <Proxy * >
563
            Order allow,deny
564
            Allow from all
565
        </Proxy>
566

    
567
            SetEnv                proxy-sendchunked
568
            SSLProxyEngine        off
569
            ProxyErrorOverride    off
570

    
571
            ProxyPass        /static !
572
            ProxyPass        / http://localhost:8080/ retry=0
573
            ProxyPassReverse / http://localhost:8080/
574

    
575
            RewriteEngine On
576
            RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC]
577
            RewriteRule ^(.*)$ - [F,L]
578

    
579
            SSLEngine on
580
            SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
581
            SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
582
        </VirtualHost>
583

    
584
    .. note:: Consider also to purchase and install a certificate for the new
585
              domain.
586

    
587

    
588
    Finally, restart the apache server::
589

    
590
        pithos-host$ /etc/init.d/apache2 restart
591

    
592
#. **Register Pithos+ as an OAuth2 client in Astakos**
593

    
594
   Starting from synnefo version 0.15, in order to view the content of a
595
   protected resource, Pithos+ (on behalf of the user) has to be granted
596
   authorization for the specific resource by Astakos.
597

    
598
   During the authorization grant procedure, Pithos+ has to authenticate
599
   itself with Astakos since the latter has to prevent serving requests by
600
   unknown/unauthorized clients.
601

    
602
   Therefore, in the installation guide you were guided to register Pithos+
603
   as an OAuth2 client in Astakos.
604

    
605
   .. note:: You can see the registered clients by running::
606
    astakos-host$ snf-manage oauth2-client-list -o identifier,redirect_urls,is_trusted
607

    
608
   However, requests originated from the new domain will be rejected since
609
   Astakos is ignorant about the new domain.
610

    
611
   Therefore, you need to register a new client pointing to the unsafe domain.
612
   To do so, use the following command::
613

    
614
        astakos-host$ snf-manage oauth2-client-add pithos-unsafe-domain --secret=<secret> --is-trusted --url https://user-content.example.com/pithos/ui/view
615

    
616

    
617
   .. note:: You can also unregister the client pointing to the safe domain,
618
       since it will no longer be useful.
619
       To do so, run the following::
620

    
621
        astakos-host$ snf-manage oauth2-client-remove pithos-view
622

    
623
#. **Update Pithos+ configuration**
624

    
625
   Respectively, the ``PITHOS_OAUTH2_CLIENT_CREDENTIALS`` setting should be
626
   updated to contain the credentials of the client registered in the previous
627
   step.
628

    
629
   Furthermore, you need to restrict all the requests for user content
630
   to be served exclusively by the unsafe domain.
631

    
632
   To enable this, set the ``PITHOS_UNSAFE_DOMAIN`` setting to the value
633
   of the new domain e.g. "user-content.example.com"
634

    
635
   Finally, restart the gunicorn server::
636

    
637
        pithos-host$ /etc/init.d/gunicorn restart
638

    
639

    
640
Compute/Network/Image Service (Cyclades)
641
========================================
642

    
643
Introduction
644
------------
645

    
646
Cyclades is the Synnefo component that implements Compute, Network and Image
647
services and exposes the associated OpenStack REST APIs. By running Cyclades
648
you can provide a cloud that can handle thousands of virtual servers and
649
networks.
650

    
651
Cyclades does not include any virtualization software and knows nothing about
652
the low-level VM management operations, e.g. handling of VM creation or
653
migrations among physical nodes. Instead, Cyclades is the component that
654
handles multiple Ganeti backends and exposes the REST APIs. The administrator
655
can expand the infrastructure dynamically either by adding more Ganeti nodes
656
or by adding new Ganeti clusters. Cyclades issue VM control commands to Ganeti
657
via Ganeti's remote API and receive asynchronous notifications from Ganeti
658
backends whenever the state of a VM changes, due to Synnefo- or
659
administrator-initiated commands.
660

    
661
Cyclades is the action orchestrator and the API layer on top of multiple Ganeti
662
clusters. By this decoupled design, Ganeti cluster are self-contained and
663
the administrator has complete control on them without Cyclades knowing about
664
it. For example a VM migration to a different physical node is transparent
665
to Cyclades.
666

    
667
Working with Cyclades
668
---------------------
669

    
670
Flavors
671
~~~~~~~
672

    
673
When creating a VM, the user must specify the `flavor` of the virtual server.
674
Flavors are the virtual hardware templates, and provide a description about
675
the number of CPUs, the amount of RAM, and the size of the disk of the VM.
676
Besides the size of the disk, Cyclades flavors describe the storage backend
677
that will be used for the virtual server.
678

    
679
Flavors are created by the administrator and the user can select one of the
680
available flavors. After VM creation, the user can resize his VM, by
681
adding/removing CPU and RAM.
682

    
683
Cyclades support different storage backends that are described by the disk
684
template of the flavor, which is mapped to Ganeti's instance `disk template`.
685
Currently the available disk templates are the following:
686

    
687
* `file`: regulars file
688
* `sharedfile`: regular files on a shared directory, e.g. NFS
689
* `plain`: logical volumes
690
* `drbd`: drbd on top of lvm volumes
691
* `rbd`: rbd volumes residing inside a RADOS cluster
692
* `ext`: disks provided by an external shared storage.
693

    
694
  - `ext_archipelago`: External shared storage provided by
695
    `Archipelago <http://www.synnefo.org/docs/archipelago/latest/index.html>`_.
696

    
697
Flavors are created by the administrator using `snf-manage flavor-create`
698
command. The command takes as argument number of CPUs, amount of RAM, the size
699
of the disks and the disk templates and create the flavors that belong to the
700
cartesian product of the specified arguments. For example, the following
701
command will create two flavors of `40G` disk size with `drbd` disk template,
702
`4G` RAM and `2` or `4` CPUs.
703

    
704
.. code-block:: console
705

    
706
  $ snf-manage flavor-create 2,4 4096 40 drbd
707

    
708
To see the available flavors, run `snf-manage flavor-list` command. The
709
administrator can delete a flavor by using `flavor-modify` command:
710

    
711
.. code-block:: console
712

    
713
  $ snf-manage flavor-modify --deleted=True <flavor_id>
714

    
715
Finally, the administrator can set if new servers can be created from a flavor
716
or not, by setting the `allow_create` attribute:
717

    
718
.. code-block:: console
719

    
720
  $ snf-manage flavor-modify --allow-create=False <flavor_id>
721

    
722
Flavors that are marked with `allow_create=False` cannot be used by users to
723
create new servers. However, they can still be used to resize existing VMs.
724

    
725

    
726
Images
727
~~~~~~
728

    
729
When creating a VM the user must also specify the `image` of the virtual
730
server. Images are the static templates from which VM instances are
731
initiated. Cyclades uses Pithos to store system and user-provided images,
732
taking advantage of all Pithos features, like deduplication and syncing
733
protocol. An image is a file stored to Pithos with additional metadata that
734
are describing the image, e.g. the OS family or the root partition. To create
735
a new image, the administrator or the user has to upload it a file to Pithos,
736
and then register it as an Image with Cyclades. Then the user can use this
737
image to spawn new VMs from it.
738

    
739
Images can be private, public or shared between users, exactly like Pithos
740
files. Since user-provided public images can be untrusted, the administrator
741
can denote which users are trusted by adding them to the
742
``UI_SYSTEM_IMAGES_OWNERS`` setting in the
743
`/etc/synnefo/20-snf-cyclades-app-ui.conf` file. Images of those users are
744
properly displayed in the UI.
745

    
746
When creating a new VM, Cyclades pass the location of the image and it's
747
metadata to Ganeti. After Ganeti creates the instance's disk, `snf-image`
748
will copy the image to the new disk and perform the image customization
749
phase. During the phase, `snf-image` sends notifications to Cyclades about
750
the progress of the image deployment and customization. Customization includes
751
resizing the root file system, file injection (e.g. SSH keys) and setting
752
a custom hostname. For better understanding of `snf-image` read the
753
corresponding `documentation
754
<http://www.synnefo.org/docs/snf-image/latest/index.html>`_.
755

    
756
For passing sensitive data about the image to Ganeti, like the VMs password,
757
Cyclades keeps all sensitive data in memory caches (memcache) and never allows
758
them to hit the disk. The data are exposed to `snf-image` via an one-time URL
759
that is exposed from the `vmapi` application. So, instead of passing sensitive
760
data to `snf-image` via Ganeti, Cyclades pass an one-time configuration URL
761
that contains a random UUID. After `snf-image` gets the sensitive data, the
762
URL is invalidated so no one else can access them.
763

    
764
The administrator can register images, exactly like users, using a system user
765
(a user that is defined in the ``UI_SYSTEM_IMAGES_OWNERS`` setting). For
766
example, the following command will register the
767
`pithos://u53r-un1qu3-1d/images/debian_base-6.0-7-x86_64.diskdump` as an
768
image to Cyclades:
769

    
770
.. code-block:: console
771

    
772
 $ kamaki image register --name="Debian Base" \
773
        --location=pithos://u53r-un1qu3-1d/images/debian_base-6.0-7-x86_64.diskdump \
774
        --public \
775
        --disk-format=diskdump \
776
        --property OSFAMILY=linux --property ROOT_PARTITION=1 \
777
        --property description="Debian Squeeze Base System" \
778
        --property size=451 --property kernel=2.6.32 --property GUI="No GUI" \
779
        --property sortorder=1 --property USERS=root --property OS=debian
780

    
781
Deletion of an image is done via `kamaki image unregister` command, which will
782
delete the Cyclades Images but will leave the Pithos file as is (unregister).
783

    
784
Apart from using `kamaki` to see and hangle the available images, the
785
administrator can use `snf-manage image-list` and `snf-manage image-show`
786
commands to list and inspect the available public images. Also, the `--user-id`
787
option can be used the see the images of a specific user.
788

    
789
Virtual Servers
790
~~~~~~~~~~~~~~~
791

    
792
As mentioned, Cyclades uses Ganeti for management of VMs. The administrator can
793
handle Cyclades VMs just like any other Ganeti instance, via `gnt-instance`
794
commands. All Ganeti instances that belong to Synnefo, are separated from
795
others, by a prefix in their names. This prefix is defined in
796
``BACKEND_PREFIX_ID`` setting in
797
``/etc/synnefo/20-snf-cyclades-app-backend.conf``.
798

    
799
Apart from handling Cyclades VM at the Ganeti level, the administrator can
800
also use the `snf-manage server-*` commands. These command cover the most
801
common tasks that are relative with VM handling. Below we describe come
802
of them, but for more information you can use the `--help` option of all
803
`snf-manage server-* commands`. These command cover the most
804

    
805
The `snf-manage server-create` command can be used to create a new VM for some
806
user. This command can be useful when the administrator wants to test Cyclades
807
functionality without starting the API service, e.g. after an upgrade. Also, by
808
using `--backend-id` option, the VM will be created in the specified backend,
809
bypassing automatic VM allocation.
810

    
811
.. code-block:: console
812

    
813
 $ snf-manage server-create --flavor-id=1 --image-id=fc0f6858-f962-42ce-bf9a-1345f89b3d5e \
814
    --user-id=7cf4d078-67bf-424d-8ff2-8669eb4841ea --backend-id=2 \
815
    --password='example_passw0rd' --name='test_vm'
816

    
817
The above commnd will create a new VM for user
818
`7cf4d078-67bf-424d-8ff2-8669eb4841ea` in the Ganeti backend with ID 2. By
819
default this command will issue a Ganeti job to create the VM
820
(`OP_INSTANCE_CREATE`) and return. As in other commands, the `--wait=True`
821
option can be used in order to wait for the successful completion of the job.
822

    
823
`snf-manage server-list` command can be used to list all the available servers.
824
The command supports some useful options, like listing servers of a user,
825
listing servers that exist in a Ganeti backend and listing deleted servers.
826
Also, as in most of `*-list` commands, the `--filter-by` option can be used to
827
filter the results. For example, the following command will only display the
828
started servers of a specific flavor:
829

    
830
.. code-block:: console
831

    
832
 $ snf-manage server-list --filter-by="operstate=STARTED,flavor=<flavor_id>"
833

    
834
Another very useful command is the `server-inspect` command which will display
835
all available information about the state of the server in DB and the state
836
of the server in the Ganeti backend. The output will give you an easy overview
837
about the state of the VM which can be useful for debugging.
838

    
839
Also the administrator can `suspend` a user's VM, using the `server-modify`
840
command:
841

    
842
.. code-block:: console
843

    
844
 $ snf-manage server-modify --suspended=True <server_id>
845

    
846
The user is forbidden to do any action on an administratively suspended VM,
847
which is useful for abuse cases.
848

    
849
Ganeti backends
850
~~~~~~~~~~~~~~~
851

    
852
Since v0.11, Synnefo is able to manage multiple Ganeti clusters (backends)
853
making it capable to scale linearly to tens of thousands of VMs. Backends
854
can be dynamically added or removed via `snf-manage` commands.
855

    
856
Each newly created VM is allocated to a Ganeti backend by the Cyclades backend
857
allocator. The VM is "pinned" to this backend, and can not change through its
858
lifetime. The backend allocator decides in which backend to spawn the VM based
859
on the available resources of each backend, trying to balance the load between
860
them. Also, Networks are created to all Ganeti backends, in order to ensure
861
that VMs residing on different backends can be connected to the same networks.
862

    
863
A backend can be marked as `drained` in order to be excluded from automatic
864
servers allocation and not receive new servers. Also, a backend can be marked
865
as `offline` to denote that the backend is not healthy (e.g. broken master)
866
and avoid the penalty of connection timeouts.
867

    
868
Finally, Cyclades is able to manage Ganeti backends with different enabled
869
hypervisors (`kvm`, `xen`), and different enabled disk templates.
870

    
871
Listing existing backends
872
`````````````````````````
873
To list all the Ganeti backends known to Synnefo, we run:
874

    
875
.. code-block:: console
876

    
877
   $ snf-manage backend-list
878

    
879
Adding a new Ganeti backend
880
```````````````````````````
881
Backends are dynamically added under the control of Synnefo with `snf-manage
882
backend-add` command. In this section it is assumed that a Ganeti cluster,
883
named ``cluster.example.com`` is already up and running and configured to be
884
able to host Synnefo VMs.
885

    
886
To add this Ganeti cluster, we run:
887

    
888
.. code-block:: console
889

    
890
   $ snf-manage backend-add --clustername=cluster.example.com --user="synnefo_user" --pass="synnefo_pass"
891

    
892
where ``clustername`` is the Cluster hostname of the Ganeti cluster, and
893
``user`` and ``pass`` are the credentials for the `Ganeti RAPI user
894
<http://docs.ganeti.org/ganeti/2.8/html/rapi.html#users-and-passwords>`_.  All
895
backend attributes can be also changed dynamically using the `snf-manage
896
backend-modify` command.
897

    
898
``snf-manage backend-add`` will also create all existing public networks to
899
the new backend. You can verify that the backend is added, by running
900
`snf-manage backend-list`.
901

    
902
Note that no VMs will be spawned to this backend, since by default it is in a
903
``drained`` state after addition in order to manually verify the state of the
904
backend.
905

    
906
So, after making sure everything works as expected, make the new backend active
907
by un-setting the ``drained`` flag. You can do this by running:
908

    
909
.. code-block:: console
910

    
911
   $ snf-manage backend-modify --drained=False <backend_id>
912

    
913
Allocation of VMs in Ganeti backends
914
````````````````````````````````````
915
As already mentioned, the Cyclades backend allocator is responsible for
916
allocating new VMs to backends. This allocator does not choose the exact Ganeti
917
node that will host the VM but just the Ganeti backend. The exact node is
918
chosen by the Ganeti cluster's allocator (hail).
919

    
920
The decision about which backend will host a VM is based on the available
921
resources. The allocator computes a score for each backend, that shows its load
922
factor, and the one with the minimum score is chosen. The admin can exclude
923
backends from the allocation phase by marking them as ``drained`` by running:
924

    
925
.. code-block:: console
926

    
927
   $ snf-manage backend-modify --drained=True <backend_id>
928

    
929
The backend resources are periodically updated, at a period defined by
930
the ``BACKEND_REFRESH_MIN`` setting, or by running `snf-manage
931
backend-update-status` command. It is advised to have a cron job running this
932
command at a smaller interval than ``BACKEND_REFRESH_MIN`` in order to remove
933
the load of refreshing the backends stats from the VM creation phase.
934

    
935
Finally, the admin can decide to have a user's VMs being allocated to a
936
specific backend, with the ``BACKEND_PER_USER`` setting. This is a mapping
937
between users and backends. If the user is found in ``BACKEND_PER_USER``, then
938
Synnefo allocates all his/hers VMs to the specific backend in the variable,
939
even if is marked as drained (useful for testing).
940

    
941
.. _alloc_disk_templates:
942

    
943
Allocation based on disk-templates
944
**********************************
945

    
946
Besides the available resources of each Ganeti backend, the allocator takes
947
into consideration the disk template of the instance when trying to allocate it
948
to a Ganeti backend. Specifically, the allocator checks if the flavor of the
949
instance belongs to the available disk templates of each Ganeti backend.
950

    
951
A Ganeti cluster has a list of enabled disk templates
952
(`--enabled-disk-templates`) and a list of allowed disk templates for new
953
instances (`--ipolicy-disk-templates`). See the `gnt-cluster` manpage for more
954
details about these options.
955

    
956
When Synnefo allocates an instance, it checks whether the disk template of the
957
new instance belongs both in the enabled and ipolicy disk templates. You can
958
see the list of the available disk-templates by running `snf-manage
959
backend-list`. This list should be updated automatically after changing
960
these options in Ganeti and it can also be updated by running `snf-manage
961
backend-update-status`.
962

    
963
So the administrator, can route instances on different backends based on their
964
flavor disk template, by modifying the enabled or ipolicy disk templates of
965
each backend.  Also, the administrator can route instances between different
966
nodes of the same Ganeti backend, by modifying the same options at the
967
nodegroup level (see `gnt-group` manpage for mor details).
968

    
969
Removing an existing Ganeti backend
970
```````````````````````````````````
971
In order to remove an existing backend from Synnefo, you must first make
972
sure that there are not active servers in the backend, and then run:
973

    
974
.. code-block:: console
975

    
976
   $ snf-manage backend-remove <backend_id>
977

    
978

    
979
Virtual Networks
980
~~~~~~~~~~~~~~~~
981

    
982
Cyclades also implements the Network service and exposes the Quantum Openstack
983
API. Cyclades supports full IPv4 and IPv6 connectivity to the public internet
984
for it's VMs. Also, Cyclades provides L2 and L3 virtual private networks,
985
giving the user freedom to create arbitraty network topologies of
986
interconnected VMs.
987

    
988
Public networking is desployment specific and must be customized based on the
989
specific needs of the system administrator. Private virtual networks can be
990
provided by different network technologies which are exposed as different
991
network flavors. For better understanding of networking please refer to the
992
:ref:`Network <networks>` section.
993

    
994
A Cyclades virtual network is an isolated Layer-2 broadcast domain. A network
995
can also have an associated IPv4 and IPv6 subnet representing the Layer-3
996
characteristics of the network. Each subnet represents an IP address block
997
that is used in order to assign addresses to VMs.
998

    
999
To connect a VM to a network, a port must be created, which represent a virtual
1000
port on a network switch. VMs are connected to networks by attaching a virtual
1001
interface to a port.
1002

    
1003
Cyclades also supports `floating IPs`, which are public IPv4 addresses that
1004
can dynamically(hotplug-able) be added and removed to VMs. Floating IPs are
1005
a quotable resource that is allocated to each user. Unlike other cloud
1006
platforms, floating IPs are not implemented using 1-1 NAT to a ports private
1007
IP. Instead, floating IPs are directly assigned to virtual interfaces of VMs.
1008

    
1009
Exactly like VMS, networks can be handled as Ganeti networks via `gnt-network`
1010
commands. All Ganeti networks that belong to Synnefo are named with the prefix
1011
`${BACKEND_PREFIX_ID}-net-`. Also, there are a number of `snf-manage` commands
1012
for handling of `networks`, `subnets`, `ports` and `floating IPs`. Below
1013
we will present a use case scenario using some of these commands. For better
1014
understanding of these commands, refer to their help messages.
1015

    
1016
Create a virtual private network for user
1017
`7cf4d078-67bf-424d-8ff2-8669eb4841ea` using the `PHYSICAL_VLAN` flavor, which
1018
means that the network will be uniquely assigned a phsyical VLAN. The network
1019
is assigned an IPv4 subnet, described by it's CIDR and gateway. Also,
1020
the `--dhcp=True` option is used, to make `nfdhcpd` response to DHCP queries
1021
from VMs.
1022

    
1023
.. code-block:: console
1024

    
1025
 $ snf-manage network-create --owner=7cf4d078-67bf-424d-8ff2-8669eb4841ea --name=prv_net-1 \
1026
    --subnet=192.168.2.0/24 --gateway=192.168.2.1 --dhcp=True --flavor=PHYSICAL_VLAN
1027

    
1028
Inspect the state of the network in Cyclades DB and in all the Ganeti backends:
1029

    
1030
.. code-block:: console
1031

    
1032
  $ snf-manage network-inspect <network_id>
1033

    
1034
Inspect the state of the network's subnet, containg an overview of the
1035
subnet's IPv4 address allocation pool:
1036

    
1037
.. code-block:: console
1038

    
1039
  $ snf-manage subnet-inspect <subnet_id>
1040

    
1041
Connect a VM to the created private network. The port will be automatically
1042
be assigned an IPv4 address from one of the network's available IPs. This
1043
command will result in sending an `OP_INSTANCE_MODIFY` Ganeti command and
1044
attaching a NIC to the specified Ganeti instance.
1045

    
1046
.. code-block:: console
1047

    
1048
 $ snf-manage port-create --network=<network_id> --server=<server_id>
1049

    
1050
Inspect the state of the the port in Cyclades DB and in the Ganeti backend:
1051

    
1052
.. code-block:: console
1053

    
1054
 $ snf-manage port-inspect <port_id>
1055

    
1056
Disconnect the VM from the network and delete the network:
1057

    
1058
.. code-block:: console
1059

    
1060
 $ snf-manage port-remove <port_id>
1061
 $ snf-manage network-remove <network_id>
1062

    
1063

    
1064
Enabling DHCP
1065
`````````````
1066

    
1067
When connecting a VM to a network, Cyclades will automatically assign an IPv4
1068
address from the IPv4 or/and IPv6 subnets of the network. If the network has
1069
no subnets, then it will not be assigned any IP address.
1070

    
1071
If the network has DHCP enabled, then `nfdhcpd` daemon, which must be running
1072
on all Ganeti nodes, will respond to DHCP queries from VMs and assign to them
1073
the IP address that was allocated by Cyclades. DCHP can be enabled/disabled
1074
using the `--dhcp` option of `network-create` command.
1075

    
1076

    
1077
Public network connectivity
1078
```````````````````````````
1079

    
1080
Since v0.14, users are able to dynamically connect and disconnect their VMs
1081
from public networks. In order to do that, they have to use a `floating IP`.
1082
Floating IPs are basically public IPv4 addresses that can be dynamically
1083
attached and detached from VMs. The user creates a floating IP address from a
1084
network that has set the `floating_ip_pool` attribute. The floating IP is
1085
accounted to the user, who can then connect his VMs to public networks by
1086
creating ports that they are using this floating IP. Performing this work-flow
1087
from `snf-manage` would look like this:
1088

    
1089
.. code-block:: console
1090

    
1091
 $ snf-manage network-list --filter-by="floating_ip_pool=True"
1092
 id      name  user.uuid   state  public  subnet.ipv4  gateway.ipv4  drained  floating_ip_pool
1093
 ---------------------------------------------------------------------------------------------
1094
  1  Internet       None  ACTIVE    True  10.2.1.0/24      10.2.1.1    False              True
1095

    
1096
 $ snf-manage floating-ip-create --owner=7cf4d078-67bf-424d-8ff2-8669eb4841ea --network=1
1097

    
1098
 $ snf-manage floating-ip-list --user=7cf4d078-67bf-424d-8ff2-8669eb4841ea
1099
 id   address       network                             user.uuid  server
1100
 ------------------------------------------------------------------------
1101
 38  10.2.1.2             1  7cf4d078-67bf-424d-8ff2-8669eb4841ea      42
1102

    
1103
 $ snf-manage port-create --owner=7cf4d078-67bf-424d-8ff2-8669eb4841ea --network=1 \
1104
                          --ipv4-address=10.2.1.2 --floating-ip=38
1105

    
1106
 $ snf-manage port-list --user=7cf4d078-67bf-424d-8ff2-8669eb4841ea
1107
 id                            user.uuid        mac_address  network  server_id  fixed_ips   state
1108
 --------------------------------------------------------------------------------------------------
1109
 163 7cf4d078-67bf-424d-8ff2-8669eb4841ea  aa:00:00:45:13:98       1         77   10.2.1.2  ACTIVE
1110

    
1111
 $ snf-manage port-remove 163
1112
 $ snf-manage floating-ip-remove 38
1113

    
1114
Users do not have permission to connect and disconnect VMs from public
1115
networks without using a floating IP address. However, the administrator
1116
have the ability to perform this tasks, using `port-create` and `port-remove`
1117
commands.
1118

    
1119
Network connectivity for newly created servers
1120
``````````````````````````````````````````````
1121

    
1122
When creating a virtual server, the user can specify the networks that the
1123
newly created server will be connected to. Beyond this, the administrator can
1124
define a list of networks that every new server will be forced to connect to.
1125
For example, you can enforce all VMs to be connected to a public network
1126
containing a metadata server. The networks must be specified in the
1127
``CYCLADES_FORCED_SERVER_NETWORKS`` that exists in the
1128
``/etc/synnefo/20-snf-cyclades-app-api.conf``. For the networks in this
1129
setting, no access control or quota policy are enforced!
1130

    
1131
Finally, the administrator can define a list of networks that new servers will
1132
be connected, *if the user has not* specified networks in the request to create
1133
the server. Access control and quota policy are enforced, just as if the user
1134
had specified these networks. The list of these networks is defined in the
1135
``CYCLADES_DEFAULT_SERVER_NETWORKS`` that exists in the
1136
``/etc/synnefo/20-snf-cyclades-app-api.conf``. This setting should only
1137
be used if Cyclades are being accessed by external clients that are
1138
unaware of the `Neutron API extensions` in the `Compute API`.
1139

    
1140
Each member of the above mentioned settings can be:
1141

    
1142
* a network UUID
1143
* a tuple of network UUIDs: the server will be connected to only one of these
1144
  networks, e.g. one that has a free IPv4 address
1145
* `SNF:ANY_PUBLIC_IPV4`: the server will be connected to any network with
1146
  an IPv4 subnet defined
1147
* `SNF:ANY_PUBLIC_IPV6`: the server will be connected to any network with
1148
  only an IPv6 subnet defined.
1149
* `SNF:ANY_PUBLIC`: the server will be connected to any public network.
1150

    
1151
Public IP accounting
1152
````````````````````
1153

    
1154
There are many use cases, e.g. abuse ports, where you need to find which user
1155
or which server had a public IP address. For this reason, Cyclades keeps track
1156
usage of public IPv4/IPv6 addresses. Specifically, it keeps the date and time
1157
that each public IP address was allocated and released from a virtual server.
1158
This information can be found using `ip-list` command:
1159

    
1160
.. code-block:: console
1161

    
1162
 $ snf-manage ip-list
1163

    
1164
 Show usage of a specific address:
1165
 $ snf-manage ip-list --address=192.168.2.1
1166

    
1167
 Show public IPs of a specific server:
1168
 $ snf-manage ip-list --server=<server_id>
1169

    
1170

    
1171
Managing Network Resources
1172
``````````````````````````
1173

    
1174
Proper operation of the Cyclades Network Service depends on the unique
1175
assignment of specific resources to each type of virtual network. Specifically,
1176
these resources are:
1177

    
1178
* IP addresses. Cyclades creates a Pool of IPs for each Network, and assigns a
1179
  unique IP address to each VM, thus connecting it to this Network. You can see
1180
  the IP pool of each network by running `snf-manage subnet-inspect
1181
  <subnet_ID>`. IP pools are automatically created and managed by Cyclades,
1182
  depending on the subnet of the Network.
1183
* Bridges corresponding to physical VLANs, which are required for networks of
1184
  type `PRIVATE_PHYSICAL_VLAN`.
1185
* One Bridge corresponding to one physical VLAN which is required for networks of
1186
  type `PRIVATE_MAC_PREFIX`.
1187

    
1188
IPv4 addresses
1189
**************
1190

    
1191
An allocation pool of IPv4 addresses is automatically created for every network
1192
with an IPv4 subnet. By default, the allocation pool contains the range of IP
1193
addresses that are included in the subnet, except from the gateway and the
1194
broadcast address of the network. The range of IP addresses can be restricted
1195
using the `--allocation-pool` option of `snf-manage network-create` command.
1196
The admin can externally reserve IP addresses to exclude them from automatic
1197
allocation with the `--add-reserved-ips` option of `snf-manage network-modify`
1198
command. For example the following command will reserve two IP addresses from
1199
network with ID `42`:
1200

    
1201
.. code-block:: console
1202

    
1203
 snf-manage network-modify --add-reserved-ips=10.0.0.21,10.0.0.22 42
1204

    
1205
.. warning:: Externally reserving IP addresses is also available at the Ganeti.
1206
 However, when using Cyclades with multiple Ganeti backends, the handling of
1207
 IP pools must be performed from Cyclades!
1208

    
1209
Bridges
1210
*******
1211

    
1212
As already mentioned Cyclades use a pool of Bridges that must correspond
1213
to Physical VLAN at the Ganeti level. A bridge from the pool is assigned to
1214
each network of flavor `PHYSICAL_VLAN`. Creation of this pool is done
1215
using `snf-manage pool-create` command. For example the following command
1216
will create a pool containing the brdiges from `prv1` to `prv21`.
1217

    
1218
.. code-block:: console
1219

    
1220
   # snf-manage pool-create --type=bridge --base=prv --size=20
1221

    
1222
You can verify the creation of the pool, and check its contents by running:
1223

    
1224
.. code-block:: console
1225

    
1226
   # snf-manage pool-list
1227
   # snf-manage pool-show --type=bridge 1
1228

    
1229
Finally you can use the `pool-modify` management command in order to externally
1230
reserve the values from pool, extend or shrink the pool if possible.
1231

    
1232
MAC Prefixes
1233
************
1234

    
1235
Cyclades also use a pool of MAC prefixes to assign to networks of flavor
1236
`MAC_FILTERED`. Handling of this pool is done exactly as with pool of bridges,
1237
except that the type option must be set to mac-prefix:
1238

    
1239
.. code-block:: console
1240

    
1241
   # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
1242

    
1243
The above command will create a pool of MAC prefixes from ``aa:00:1`` to
1244
``b9:ff:f``. The MAC prefix pool is responsible for providing only unicast and
1245
locally administered MAC addresses, so many of these prefixes will be
1246
externally reserved, to exclude from allocation.
1247

    
1248
Quotas
1249
~~~~~~
1250

    
1251
Handling of quotas for Cyclades resources is powered by Astakos quota
1252
mechanism. During registration of Cyclades service to Astakos, the Cyclades
1253
resources are also imported to Astakos for accounting and presentation.
1254

    
1255
Upon a request that will result in a resource creation or removal, Cyclades
1256
will communicate with Astakos to ensure that user quotas are within limits and
1257
update the corresponding usage. If a limit is reached, the request will be
1258
denied with an `overLimit(413)` fault.
1259

    
1260
The resources that are exported by Cyclades are the following:
1261

    
1262
* `cyclades.vm`: Number of virtual machines
1263
* `cyclades.total_cpu`: Number of virtual machine processors
1264
* `cyclades.cpu`: Number of virtual machine processors of running VMs
1265
* `cyclades.total_ram`: Virtual machine memory size
1266
* `cyclades.ram`: Virtual machine memory size of running VMs
1267
* `cyclades.disk`: Virtual machine disk size
1268
* `cyclades.floating_ip`: Number of floating IP addresses
1269
* `cyclades.network.private`: Number of private virtual networks
1270

    
1271
Enforcing quotas
1272
~~~~~~~~~~~~~~~~
1273

    
1274
User quota can get overlimit, for example when a user is removed from a
1275
project granting Cyclades resources. However, no action is automatically
1276
taken to restrict users to their new limits. There is a special tool for
1277
quota enforcement:
1278

    
1279
.. code-block:: console
1280

    
1281
  # snf-manage enforce-resources-cyclades
1282

    
1283
This command will check and report which users are overlimit on their
1284
Cyclades quota; it will also suggest actions to be taken in order to enforce
1285
quota limits, dependent on the overlimit resource:
1286

    
1287
* `cyclades.vm`: Delete VMs
1288
* `cyclades.total_cpu`: Delete VMs
1289
* `cyclades.cpu`: Shutdown VMs
1290
* `cyclades.total_ram`: Delete VMs
1291
* `cyclades.ram`: Shutdown VMs
1292
* `cyclades.disk`: Delete VMs
1293
* `cyclades.floating_ip`: Detach and remove IPs
1294

    
1295
VMs to be deleted/shutdown are chosen first by state in the following order:
1296
ERROR, BUILD, STOPPED, STARTED or RESIZE and then by decreasing ID. When
1297
needing to remove IPs, we first choose IPs that are free, then those
1298
attached to VMs, using the same VM ordering.
1299

    
1300
By default, the command checks only the following resources: `cyclades.cpu`,
1301
`cyclades.ram`, and `cyclades.floating_ip`; that is, the less dangerous
1302
ones, those that do not result in *deleting* any VM. One can change the
1303
default behavior by specifying the desired resources with option
1304
``--resources``. It is also possible to specify users to be checked or
1305
excluded.
1306

    
1307
Actual enforcement is done with option ``--fix``. In order to control the
1308
load that quota enforcement may cause on Cyclades, one can limit the number
1309
of operations per backend. For example,
1310

    
1311
.. code-block:: console
1312

    
1313
  # snf-manage enforce-resources-cyclades --fix --max-operations 10
1314

    
1315
will apply only the first 10 listed actions per backend. One can repeat the
1316
operation, until nothing is left to be done.
1317

    
1318
To control load a timeout can also be set for shutting down VMs (using
1319
option ``--shutdown-timeout <sec>``). This may be needed to avoid
1320
expensive operations triggered by shutdown, such as Windows updates.
1321

    
1322
The command outputs the list of applied actions and reports whether each
1323
action succeeded or not. Failure is reported if for any reason cyclades
1324
failed to process the job and submit it to the backend.
1325

    
1326
Cyclades advanced operations
1327
----------------------------
1328

    
1329
Reconciliation mechanism
1330
~~~~~~~~~~~~~~~~~~~~~~~~
1331

    
1332
Cyclades - Ganeti reconciliation
1333
````````````````````````````````
1334

    
1335
On certain occasions, such as a Ganeti or RabbitMQ failure, the state of
1336
Cyclades database may differ from the real state of VMs and networks in the
1337
Ganeti backends. The reconciliation process is designed to synchronize the
1338
state of the Cyclades DB with Ganeti. There are two management commands for
1339
reconciling VMs and Networks that will detect stale, orphans and out-of-sync
1340
VMs and networks. To fix detected inconsistencies, use the `--fix-all`.
1341

    
1342
.. code-block:: console
1343

    
1344
  $ snf-manage reconcile-servers
1345
  $ snf-manage reconcile-servers --fix-all
1346

    
1347
  $ snf-manage reconcile-networks
1348
  $ snf-manage reconcile-networks --fix-all
1349

    
1350
Please see ``snf-manage reconcile-servers --help`` and ``snf-manage
1351
reconcile--networks --help`` for all the details.
1352

    
1353

    
1354
Cyclades - Astakos reconciliation
1355
`````````````````````````````````
1356

    
1357
As already mentioned, Cyclades communicates with Astakos for resource
1358
accounting and quota enforcement. In rare cases, e.g. unexpected
1359
failures, the two services may get unsynchronized. For this reason there
1360
are the `reconcile-commissions-cyclades` and `reconcile-resources-cyclades`
1361
command that will synchronize the state of the two services. The first
1362
command will detect any pending commissions, while the second command will
1363
detect that the usage that is reported by Astakos is correct.
1364
To fix detected inconsistencies, use the `--fix` option.
1365

    
1366
.. code-block:: console
1367

    
1368
  $ snf-manage reconcile-commissions-cyclades
1369
  $ snf-manage reconcile-commissions-cyclades --fix
1370

    
1371
  $ snf-manage reconcile-resources-cyclades
1372
  $ snf-manage reconcile-resources-cyclades --fix
1373

    
1374

    
1375
Cyclades resources reconciliation
1376
`````````````````````````````````
1377

    
1378
Reconciliation of pools will check the consistency of available pools by
1379
checking that the values from each pool are not used more than once, and also
1380
that the only reserved values in a pool are the ones used. Pool reconciliation
1381
will check pools of bridges, MAC prefixes, and IPv4 addresses for all networks.
1382
To fix detected inconsistencies, use the `--fix` option.
1383

    
1384

    
1385
.. code-block:: console
1386

    
1387
  $ snf-manage reconcile-pools
1388
  $ snf-manage reconcile-pools --fix
1389

    
1390
.. _admin-guide-stats:
1391

    
1392
VM stats collecting
1393
~~~~~~~~~~~~~~~~~~~
1394

    
1395
snf-cyclades-gtools comes with a collectd plugin to collect CPU and network
1396
stats for Ganeti VMs and an example collectd configuration. snf-stats-app is a
1397
Django (snf-webproject) app that serves the VM stats graphs by reading the VM
1398
stats (from RRD files) and serves graphs.
1399

    
1400
The snf-stats-app was originally written by `GRNET NOC <http://noc.grnet.gr>`_
1401
as a WSGI Python app and was ported to a Synnefo (snf-webproject) app.
1402

    
1403
snf-stats-app configuration
1404
```````````````````````````
1405

    
1406
The snf-stats-app node should have collectd installed. The collectd
1407
configuration should enable the network plugin, assuming the server role, and
1408
the RRD plugin / backend, to store the incoming stats. Your
1409
``/etc/collectd/collectd.conf`` should look like:
1410

    
1411
.. code-block:: console
1412

    
1413
    FQDNLookup true
1414
    LoadPlugin syslog
1415
    <Plugin syslog>
1416
        LogLevel info
1417
    </Plugin>
1418

    
1419
    LoadPlugin network
1420
    LoadPlugin rrdtool
1421
    <Plugin network>
1422
        TimeToLive 128
1423
        <Listen "okeanos.io" "25826">
1424
            SecurityLevel "Sign"
1425
            AuthFile "/etc/collectd/passwd"
1426
        </Listen>
1427

    
1428
        ReportStats false
1429
        MaxPacketSize 65535
1430
    </Plugin>
1431

    
1432

    
1433
    <Plugin rrdtool>
1434
        DataDir "/var/lib/collectd/rrd"
1435
        CacheTimeout 120
1436
        CacheFlush 900
1437
        WritesPerSecond 30
1438
        RandomTimeout 0
1439
    </Plugin>
1440

    
1441
    Include "/etc/collectd/filters.conf"
1442
    Include "/etc/collectd/thresholds.conf"
1443

    
1444

    
1445
An example collectd config file is provided in
1446
``/usr/share/doc/snf-stats-app/examples/stats-colletcd.conf``.
1447

    
1448
The recommended deployment is to run snf-stats-app using gunicorn with an
1449
Apache2 or nginx reverse proxy (using the same configuration as the other
1450
Synnefo services / apps). An example gunicorn config file is provided in
1451
``/usr/share/doc/snf-stats-app/examples/stats.gunicorn``.
1452

    
1453
Make sure to edit the settings under
1454
``/etc/synnefo/20-snf-stats-app-settings.conf`` to match your deployment.
1455
More specifically, you should change the ``STATS_BASE_URL`` setting (refer
1456
to previous documentation on the BASE_URL settings used by the other Synnefo
1457
services / apps) and the ``RRD_PREFIX`` and ``GRAPH_PREFIX`` settings.
1458

    
1459
You should also set the ``STATS_SECRET_KEY`` to a random string and make sure
1460
it's the same at the ``CYCLADES_STATS_SECRET_KEY`` on the Cyclades host (see
1461
below).
1462

    
1463
``RRD_PREFIX`` is the directory where collectd stores the RRD files. The
1464
default setting matches the default RRD directory for the collectd RRDtool
1465
plugin. In a more complex setup, the collectd daemon could run on a separate
1466
host and export the RRD directory to the snf-stats-app node via e.g. NFS.
1467

    
1468
``GRAPH_PREFIX`` is the directory where collectd stores the resulting
1469
stats graphs. You should create it manually, in case it doesn't exist.
1470

    
1471
.. code-block::
1472

    
1473
    # mkdir /var/cache/snf-stats-app/
1474
    # chown www-data:wwwdata /var/cache/snf-stats-app/
1475

    
1476
The snf-stats-app will typically run as the ``www-data`` user. In that case,
1477
make sure that the ``www-data`` user should have read access to the
1478
``RRD_PREFIX`` directory and read / write access to the ``GRAPH_PREFIX``
1479
directory.
1480

    
1481
snf-stats-app, based on the ``STATS_BASE_URL`` setting will export the
1482
following URL 'endpoints`:
1483
 * CPU stats bar: ``STATS_BASE_URL``/v1.0/cpu-bar/<encrypted VM hostname>
1484
 * Network stats bar: ``STATS_BASE_URL``/v1.0/net-bar/<encrypted VM hostname>
1485
 * CPU stats daily graph: ``STATS_BASE_URL``/v1.0/cpu-ts/<encrypted VM hostname>
1486
 * Network stats daily graph: ``STATS_BASE_URL``/v1.0/net-ts/<encrypted VM hostname>
1487
 * CPU stats weekly graph: ``STATS_BASE_URL``/v1.0/cpu-ts-w/<encrypted VM hostname>
1488
 * Network stats weekly graph: ``STATS_BASE_URL``/v1.0/net-ts-w/<encrypted VM hostname>
1489

    
1490
You can verify that these endpoints are exported by issuing:
1491

    
1492
.. code-block::
1493

    
1494
    # snf-manage show_urls
1495

    
1496
snf-cyclades-gtools configuration
1497
`````````````````````````````````
1498

    
1499
To enable VM stats collecting, you will need to:
1500
 * Install collectd on the every Ganeti (VM-capable) node.
1501
 * Enable the Ganeti stats plugin in your collectd configuration. This can be
1502
   achived by either copying the example collectd conf file that comes with
1503
   snf-cyclades-gtools
1504
   (``/usr/share/doc/snf-cyclades-gtools/examples/ganeti-stats-collectd.conf``)
1505
   or by adding the following line to your existing (or default) collectd
1506
   conf file:
1507

    
1508
       Include /etc/collectd/ganeti-stats.conf
1509

    
1510
   In the latter case, make sure to configure collectd to send the collected
1511
   stats to your collectd server (via the network plugin). For more details on
1512
   how to do this, check the collectd example config file provided by the
1513
   package and the collectd documentation.
1514

    
1515
snf-cyclades-app configuration
1516
``````````````````````````````
1517

    
1518
At this point, stats collecting should be enabled and working. You can check
1519
that everything is ok by checking the contents of ``/var/lib/collectd/rrd/``
1520
directory (it will gradually get populated with directories containing RRD
1521
files / stats for every Synnefo instances).
1522

    
1523
You should also check that gunicorn and Apache2 are configured correctly by
1524
accessing the graph URLs for a VM (whose stats have been populated in
1525
``/var/lib/collectd/rrd``).
1526

    
1527
Cyclades uses the ``CYCLADES_STATS_SECRET_KEY`` setting in
1528
``20-snf-cyclades-app`` to encrypt the instance hostname in the stats graph
1529
URL. This settings should be set to a random value and match the
1530
``STATS_SECRET_KEY`` on the Stats host.
1531

    
1532
Cyclades (snf-cyclades-app) fetches the stat graphs for VMs based on four
1533
settings in ``20-snf-cyclades-app-api.conf``. The settings are:
1534

    
1535
 * CPU_BAR_GRAPH_URL = 'https://stats.host/stats/v1.0/cpu-bar/%s'
1536
 * CPU_TIMESERIES_GRAPH_URL = 'https://stats.host/stats/v1.0/cpu-ts/%s'
1537
 * NET_BAR_GRAPH_URL = 'https://stats.host/stats/v1.0/net-bar/%s'
1538
 * NET_TIMESERIES_GRAPH_URL = 'https://stats.host/stats/v1.0/net-ts/%s'
1539

    
1540
Make sure that you change this settings to match your ``STATS_BASE_URL``
1541
(and generally the Apache2 / gunicorn deployment on your stats host).
1542

    
1543
Cyclades will pass these URLs to the Cyclades UI and the user's browser will
1544
fetch them when needed.
1545

    
1546

    
1547
Helpdesk
1548
--------
1549

    
1550
Helpdesk application provides the ability to view the virtual servers and
1551
networks of all users, along with the ability to perform some basic actions
1552
like administratively suspending a server. You can perform look-ups by
1553
user UUID or email, by server ID (vm-$id) or by an IPv4 address.
1554

    
1555
If you want to activate the helpdesk application you can set to `True` the
1556
`HELPDESK_ENABLED` setting. Access to helpdesk views (under
1557
`$BASE_URL/helpdesk`) is only to allowed to users that belong to Astakos
1558
groups defined in the `HELPDESK_PERMITTED_GROUPS` setting, which by default
1559
contains the `helpdesk` group. For example, to allow <user_id>
1560
to access helpdesk view, you should run the following command in the Astakos
1561
node:
1562

    
1563
.. code-block:: console
1564

    
1565
 snf-manage group-add helpdesk
1566
 snf-manage user-modify --add-group=helpdesk <user_id>
1567

    
1568

    
1569
Cyclades internals
1570
------------------
1571

    
1572
Asynchronous communication with Ganeti backends
1573
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1574
Synnefo uses Google Ganeti backends for VM cluster management. In order for
1575
Cyclades to be able to handle thousands of user requests, Cyclades and Ganeti
1576
communicate asynchronously. Briefly, requests are submitted to Ganeti through
1577
Ganeti's RAPI/HTTP interface, and then asynchronous notifications about the
1578
progress of Ganeti jobs are being created and pushed upwards to Cyclades. The
1579
architecture and communication with a Ganeti backend is shown in the graph
1580
below:
1581

    
1582
.. image:: images/cyclades-ganeti-communication.png
1583
   :width: 40%
1584
   :target: _images/cyclades-ganeti-communication.png
1585

    
1586
The Cyclades API server is responsible for handling user requests. Read-only
1587
requests are directly served by looking up the Cyclades DB. If the request
1588
needs an action in the Ganeti backend, Cyclades submit jobs to the Ganeti
1589
master using the `Ganeti RAPI interface
1590
<http://docs.ganeti.org/ganeti/2.8/html/rapi.html>`_.
1591

    
1592
While Ganeti executes the job, `snf-ganeti-eventd`, and `snf-progress-monitor`
1593
are monitoring the progress of the job and send corresponding messages to the
1594
RabbitMQ servers. These components are part of `snf-cyclades-gtools` and must
1595
be installed on all Ganeti nodes. Specially:
1596

    
1597
* *snf-ganeti-eventd* sends messages about operations affecting the operating
1598
  state of instances and networks. Works by monitoring the Ganeti job queue.
1599
* *snf-progress_monitor* sends messages about the progress of the Image deployment
1600
  phase which is done by the Ganeti OS Definition `snf-image`.
1601

    
1602
Finally, `snf-dispatcher` consumes messages from the RabbitMQ queues, processes
1603
these messages and properly updates the state of the Cyclades DB. Subsequent
1604
requests to the Cyclades API, will retrieve the updated state from the DB.
1605

    
1606

    
1607
List of all Synnefo components
1608
==============================
1609

    
1610
They are also available from our apt repository: ``apt.dev.grnet.gr``
1611

    
1612
 * `snf-common <http://www.synnefo.org/docs/snf-common/latest/index.html>`_
1613
 * `snf-webproject <http://www.synnefo.org/docs/snf-webproject/latest/index.html>`_
1614
 * `snf-astakos-app <http://www.synnefo.org/docs/astakos/latest/index.html>`_
1615
 * `snf-pithos-backend <http://www.synnefo.org/docs/pithos/latest/backends.html>`_
1616
 * `snf-pithos-app <http://www.synnefo.org/docs/pithos/latest/index.html>`_
1617
 * `snf-pithos-webclient <http://www.synnefo.org/docs/pithos-webclient/latest/index.html>`_
1618
 * `snf-cyclades-app <http://www.synnefo.org/docs/snf-cyclades-app/latest/index.html>`_
1619
 * `snf-cyclades-gtools <http://www.synnefo.org/docs/snf-cyclades-gtools/latest/index.html>`_
1620
 * `astakosclient <http://www.synnefo.org/docs/astakosclient/latest/index.html>`_
1621
 * `snf-vncauthproxy <https://code.grnet.gr/projects/vncauthproxy>`_
1622
 * `snf-image <http://www.synnefo.org/docs/snf-image/latest/index.html/>`_
1623
 * `snf-image-creator <http://www.synnefo.org/docs/snf-image-creator/latest/index.html>`_
1624
 * `snf-occi <http://www.synnefo.org/docs/snf-occi/latest/index.html>`_
1625
 * `snf-cloudcms <http://www.synnefo.org/docs/snf-cloudcms/latest/index.html>`_
1626
 * `nfdhcpd <https://code.grnet.gr/projects/nfdhcpd>`_
1627

    
1628

    
1629
Synnefo management commands ("snf-manage")
1630
==========================================
1631

    
1632
Each Synnefo service, Astakos, Pithos and Cyclades are controlled by the
1633
administrator using the "snf-manage" admin tool. This tool is an extension of
1634
the Django command-line management utility. It is run on the host that runs
1635
each service and provides different types of commands depending the services
1636
running on the host. If you are running more than one service on the same host
1637
"snf-manage" adds all the corresponding commands for each service dynamically,
1638
providing a unified admin environment.
1639

    
1640
To run "snf-manage" you just type:
1641

    
1642
.. code-block:: console
1643

    
1644
   # snf-manage <command> [arguments]
1645

    
1646
on the corresponding host that runs the service. For example, if you have all
1647
services running on different physical hosts you would do:
1648

    
1649
.. code-block:: console
1650

    
1651
   root@astakos-host # snf-manage <astakos-command> [argument]
1652
   root@pithos-host # snf-manage <pithos-command> [argument]
1653
   root@cyclades-host # snf-manage <cyclades-command> [argument]
1654

    
1655
If you have all services running on the same host you would do:
1656

    
1657
.. code-block:: console
1658

    
1659
   root@synnefo-host # snf-manage <{astakos,pithos,cyclades}-command> [argument]
1660

    
1661
Note that you cannot execute a service's command on a host that is not running
1662
this service. For example, the following will return an error if Astakos and
1663
Cyclades are installed on different physical hosts:
1664

    
1665
.. code-block:: console
1666

    
1667
   root@astakos-host # snf-manage <cyclades-command> [argument]
1668
   Unknown command: 'cyclades-command'
1669
   Type 'snf-manage help' for usage.
1670

    
1671
This is the complete list of "snf-manage" commands for each service.
1672

    
1673
Astakos snf-manage commands
1674
---------------------------
1675

    
1676
============================  ===========================
1677
Name                          Description
1678
============================  ===========================
1679
fix-superusers                Transform superusers created by syncdb into AstakosUser instances
1680
cleanup-full                  Cleanup sessions and session catalog
1681
commission-list               List pending commissions
1682
commission-show               Show details for a pending commission
1683
component-add                 Register a component
1684
component-list                List components
1685
component-modify              Modify component attributes
1686
component-show                Show component details
1687
project-control               Manage projects and applications
1688
project-list                  List projects
1689
project-show                  Show project details
1690
quota-list                    List user quota
1691
quota-verify                  Check the integrity of user quota
1692
reconcile-resources-astakos   Reconcile resource usage of Quotaholder with Astakos DB
1693
resource-list                 List resources
1694
resource-modify               Modify a resource's default base quota and boolean flags
1695
service-export-astakos        Export Astakos services and resources in JSON format
1696
service-import                Register services
1697
service-list                  List services
1698
service-show                  Show service details
1699
term-add                      Add approval terms
1700
user-activation-send          Send user activation
1701
user-add                      Add user
1702
authpolicy-add                Create a new authentication provider policy profile
1703
authpolicy-list               List existing authentication provider policy profiles
1704
authpolicy-remove             Remove an authentication provider policy
1705
authpolicy-set                Assign an existing authentication provider policy profile to a user or group
1706
authpolicy-show               Show authentication provider profile details
1707
group-add                     Create a group with the given name
1708
group-list                    List available groups
1709
user-list                     List users
1710
user-modify                   Modify user
1711
user-show                     Show user details
1712
oauth2-client-add             Create an oauth2 client
1713
oauth2-client-list            List oauth2 clients
1714
oauth2-client-remove          Remove an oauth2 client along with its registered redirect urls
1715
============================  ===========================
1716

    
1717
Pithos snf-manage commands
1718
--------------------------
1719

    
1720
============================  ===========================
1721
Name                          Description
1722
============================  ===========================
1723
reconcile-commissions-pithos  Display unresolved commissions and trigger their recovery
1724
service-export-pithos         Export Pithos services and resources in JSON format
1725
reconcile-resources-pithos    Detect unsynchronized usage between Astakos and Pithos DB resources and synchronize them if specified so.
1726
file-show                     Display object information
1727
============================  ===========================
1728

    
1729
Cyclades snf-manage commands
1730
----------------------------
1731

    
1732
============================== ===========================
1733
Name                           Description
1734
============================== ===========================
1735
backend-add                    Add a new Ganeti backend
1736
backend-list                   List backends
1737
backend-modify                 Modify a backend
1738
backend-update-status          Update backend statistics for instance allocation
1739
backend-remove                 Remove a Ganeti backend
1740
enforce-resources-cyclades     Check and fix quota violations for Cyclades resources
1741
server-create                  Create a new server
1742
server-show                    Show server details
1743
server-list                    List servers
1744
server-modify                  Modify a server
1745
server-import                  Import an existing Ganeti VM into synnefo
1746
server-inspect                 Inspect a server in DB and Ganeti
1747
network-create                 Create a new network
1748
network-list                   List networks
1749
network-modify                 Modify a network
1750
network-inspect                Inspect network state in DB and Ganeti
1751
network-remove                 Delete a network
1752
flavor-create                  Create a new flavor
1753
flavor-list                    List flavors
1754
flavor-modify                  Modify a flavor
1755
image-list                     List images
1756
image-show                     Show image details
1757
pool-create                    Create a bridge or mac-prefix pool
1758
pool-show                      Show pool details
1759
pool-list                      List pools
1760
pool-modify                    Modify a pool
1761
pool-remove                    Delete a pool
1762
port-create                    Create a port connecting a server to a network
1763
port-inspect                   Inspect the state of a port in DB and Ganeti
1764
port-list                      List ports
1765
port-remove                    Delete a port
1766
floating-ip-create             Create a new floating IP
1767
floating-ip-attach             Attach a floating IP to a server
1768
floating-ip-detach             Detach a floating IP from a server
1769
floating-ip-list               List floating IPs
1770
floating-ip-remove             Delete a floating IP
1771
queue-inspect                  Inspect the messages of a RabbitMQ queue
1772
queue-retry                    Resend messages from Dead Letter queues to original exchanges
1773
service-export-cyclades        Export Cyclades services and resources in JSON format
1774
subnet-create                  Create a subnet
1775
subnet-inspect                 Inspect a subnet in DB
1776
subnet-list                    List subnets
1777
subnet-modify                  Modify a subnet
1778
reconcile-servers              Reconcile servers of Synnefo DB with state of Ganeti backend
1779
reconcile-networks             Reconcile networks of Synnefo DB with state of Ganeti backend
1780
reconcile-pools                Check consistency of pool resources
1781
reconcile-commissions-cyclades Detect and resolve pending commissions to Quotaholder
1782
reconcile-resources-cyclades   Reconcile resource usage of Astakos with Cyclades DB.
1783
============================== ===========================
1784

    
1785

    
1786
Astakos helper scripts
1787
======================
1788

    
1789
Astakos includes two scripts to facilitate the installation procedure.
1790
Running:
1791

    
1792
.. code-block:: console
1793

    
1794
   snf-component-register [<component_name>]
1795

    
1796
automates the registration of the standard Synnefo components (astakos,
1797
cyclades, and pithos) in astakos database. It internally uses the script:
1798

    
1799
.. code-block:: console
1800

    
1801
   snf-service-export <component_name> <base_url>
1802

    
1803
which simulates the export of service and resource definitions of the
1804
standard Synnefo components.
1805

    
1806

    
1807
Pithos managing accounts
1808
========================
1809

    
1810
Pithos provides a utility tool for managing accounts.
1811
To run you just type:
1812

    
1813
.. code-block:: console
1814

    
1815
   # pithos-manage-accounts <command> [arguments]
1816

    
1817
This is the list of the available commands:
1818

    
1819
============================  ===========================
1820
Name                          Description
1821
============================  ===========================
1822
delete                        Remove an account from the Pithos DB
1823
export-quota                  Export account quota in a file
1824
list                          List existing/dublicate accounts
1825
merge                         Move an account contents in another account
1826
set-container-quota           Set container quota for all or a specific account
1827
============================  ===========================
1828

    
1829

    
1830
The "kamaki" API client
1831
=======================
1832

    
1833
To upload, register or modify an image you will need the **kamaki** tool.
1834
Before proceeding make sure that it is configured properly. Verify that
1835
*image.url*, *file.url*, *user.url* and *token* are set as needed:
1836

    
1837
.. code-block:: console
1838

    
1839
   $ kamaki config list
1840

    
1841
To change a setting use ``kamaki config set``:
1842

    
1843
.. code-block:: console
1844

    
1845
   $ kamaki config set cloud.default.url https://example.com/identity/v2.0
1846
   $ kamaki config set cloud.default.token ...
1847

    
1848
To test that everything works, try authenticating the current account with
1849
kamaki:
1850

    
1851
.. code-block:: console
1852

    
1853
  $ kamaki user authenticate
1854

    
1855
This will output user information.
1856

    
1857
Upload Image
1858
------------
1859

    
1860
By convention, images are stored in a container called ``images``. Check if the
1861
container exists, by listing all containers in your account:
1862

    
1863
.. code-block:: console
1864

    
1865
   $ kamaki file list /images
1866

    
1867
If the container ``images`` does not exist, create it:
1868

    
1869
.. code-block:: console
1870

    
1871
  $ kamaki container create images
1872

    
1873
You are now ready to upload an image to container ``images``. You can upload it
1874
with a Pithos client, or use kamaki directly:
1875

    
1876
.. code-block:: console
1877

    
1878
   $ kamaki file upload ubuntu.iso /images
1879

    
1880
You can use any Pithos client to verify that the image was uploaded correctly,
1881
or you can list the contents of the container with kamaki:
1882

    
1883
.. code-block:: console
1884

    
1885
  $ kamaki file list /images
1886

    
1887
The full Pithos URL for the previous example will be
1888
``pithos://u53r-un1qu3-1d/images/ubuntu.iso`` where ``u53r-un1qu3-1d`` is the
1889
unique user id (uuid).
1890

    
1891
Register Image
1892
--------------
1893

    
1894
To register an image you will need to use the full or the relative Pithos URL.
1895
To register as a public image the one from the previous example use:
1896

    
1897
.. code-block:: console
1898

    
1899
   $ kamaki image register --name=Ubuntu --location=/images/ubuntu.iso --public
1900

    
1901
The ``--public`` flag is important, if missing the registered image will not
1902
be listed by ``kamaki image list``.
1903

    
1904
Use ``kamaki image register`` with no arguments to see a list of available
1905
options. A more complete example would be the following:
1906

    
1907
.. code-block:: console
1908

    
1909
   $ kamaki image register --name Ubuntu --location /images/ubuntu.iso \
1910
            --public --disk-format diskdump --property kernel=3.1.2
1911

    
1912
To verify that the image was registered successfully use:
1913

    
1914
.. code-block:: console
1915

    
1916
   $ kamaki image list --name-like ubuntu
1917

    
1918

    
1919
Miscellaneous
1920
=============
1921

    
1922
.. _branding:
1923

    
1924
Branding
1925
--------
1926

    
1927
Since Synnefo v0.14, you are able to adapt the Astakos, Pithos and Cyclades Web
1928
UI to your company’s visual identity. This is possible using the snf-branding
1929
component, which is automatically installed on the nodes running the API
1930
servers for Astakos, Pithos and Cyclades.
1931

    
1932
Configuration
1933
~~~~~~~~~~~~~
1934

    
1935
This can be done by modifing the settings provided by the snf-branding component
1936
to match your service identity. The settings for the snf-branding application
1937
can be found inside the configuration file ``/etc/synnefo/15-snf-branding.conf``
1938
on the nodes that have Astakos, Pithos and Cyclades installed.
1939

    
1940
By default, the global service name is "Synnefo" and the company name is
1941
"GRNET". These names and their respective logos and URLs are used throughout
1942
the Astakos, Pithos and Cyclades UI.
1943

    
1944
**Names and URLs:**
1945

    
1946
The first group of branding customization refers to the service's and company's
1947
information.
1948

    
1949
You can overwrite the company and the service name and URL respectively by
1950
uncommenting and setting the following:
1951

    
1952
.. code-block:: python
1953

    
1954
  # setting used in Astakos Dashboard/Projects pages
1955
  BRANDING_SERVICE_NAME = 'My cloud'
1956
  BRANDING_SERVICE_URL = 'http://www.mycloud.synnefo.org/'
1957

    
1958
  # settings used in Astakos, Pithos, Cyclades footer only if
1959
  # BRANDING_SHOW_COPYRIGHT is set to True
1960
  BRANDING_SHOW_COPYRIGHT = True
1961
  BRANDING_COMPANY_NAME = 'Company LTD'
1962
  BRANDING_COMPANY_URL = 'https://www.company-ltd.synnefo.org/'
1963

    
1964

    
1965
**Copyright and footer options:**
1966

    
1967
By default, no Copyright message is shown in the UI footer. If you want to make
1968
it visible in the footer of Astakos, Pithos and Cyclades UI, you can uncomment
1969
and set to ``True`` the ``BRANDING_SHOW_COPYRIGHT`` setting:
1970

    
1971
.. code-block:: python
1972

    
1973
  #BRANDING_SHOW_COPYRIGHT = False
1974

    
1975
Copyright message defaults to 'Copyright (c) 2011-<current_year>
1976
<BRANDING_COMPANY_NAME>.' but you can overwrite it to a completely custom one by
1977
setting the following option:
1978

    
1979
.. code-block:: python
1980

    
1981
  BRANDING_COPYRIGHT_MESSAGE = 'Copyright (c) 2011-2013 GRNET'
1982

    
1983
If you want to include a custom message in the footer, you can uncomment and
1984
set the ``BRANDING_FOOTER_EXTRA_MESSAGE`` setting. You can use html markup.
1985
Your custom message will appear  above Copyright message at the Compute
1986
templates and the Dashboard UI.
1987

    
1988
.. code-block:: python
1989

    
1990
  #BRANDING_FOOTER_EXTRA_MESSAGE = ''
1991

    
1992

    
1993
**Images:**
1994

    
1995
The Astakos, Pithos and Cyclades Web UI has some logos and images.
1996

    
1997
The branding-related images are presented in  the following table:
1998

    
1999
===============  ============================  =========
2000
Image            Name/extension  convention    Usage
2001
===============  ============================  =========
2002
Favicon          favicon.ico                   Favicon for all services
2003
Dashboard logo   dashboard_logo.png            Visible in all Astakos UI pages
2004
Compute logo     compute_logo.png              Visible in all Cyclades UI pages
2005
Console logo     console_logo.png              Visible in the Cyclades Console Window
2006
Storage logo     storage_logo.png              Visible in all Pithos UI pages
2007
===============  ============================  =========
2008

    
2009
There are two methods  available for replacing all, or individual,
2010
branding-related images:
2011

    
2012
1. Create a new directory inside ``/usr/share/synnefo/static/`` (e.g.
2013
   ``mybranding``) and place there some or all of your images.
2014

    
2015
   If you want to replace all of your images, keep the name/extension
2016
   conventions as indicated in the above table and change the
2017
   ``BRANDING_IMAGE_MEDIA_URL`` setting accordingly:
2018

    
2019
   .. code-block:: python
2020

    
2021
      # using relative path
2022
      BRANDING_IMAGE_MEDIA_URL= '/static/mybranding/images/'
2023

    
2024
      # or if you already host them in a separate domain (e.g. cdn)
2025
      BRANDING_IMAGE_MEDIA_URL= 'https://cdn.synnefo.org/branding/images/'
2026

    
2027

    
2028
   If you wish to replace individual images, **do not uncomment**
2029
   ``BRANDING_IMAGE_MEDIA_URL``, but instead provide a relative path, pointing to
2030
   the file inside your directory for each ``BRANDING_<image>_URL`` that you wish
2031
   to replace.
2032

    
2033
2. Upload some or all of your images to a server and replace each
2034
   ``BRANDING_<image>_URL`` with the absolute url of the image (i.e.
2035
   ``BRANDING_DASHBOARD_URL = 'https://www.synnefo.com/images/my_dashboard.jpg'``).
2036

    
2037
   Note that the alternative text  for each image tag inside html documents is
2038
   alt=“BRANDING_SERVICE_NAME {Dashboard, Compute. Console, Storage}” respectively.
2039

    
2040
.. note:: Retina optimized images:
2041

    
2042
   Synnefo UI is optimized for Retina displays. As far as images are concerned,
2043
   `retina.js <http://retinajs.com/>`_ is used.
2044

    
2045
   Retina.js checks each image on a page to see if there is a high-resolution
2046
   version of that image on your server. If a high-resolution variant exists,
2047
   the script will swap in that image in-place.
2048

    
2049
   The script assumes you use  `Apple's prescribed high-resolution modifier (@2x)
2050
   <http://developer.apple.com/library/ios/#documentation/2DDrawing/Conceptual/
2051
   DrawingPrintingiOS/SupportingHiResScreensInViews/SupportingHiResScreensInViews
2052
   .html#//apple_ref/doc/uid/TP40010156-CH15-SW1>`_ to denote high-resolution
2053
   image variants on your server.
2054

    
2055
   For each of the images that you wish the script to  replace, you must have a
2056
   high-resolution variant in the same folder  named correctly and it will be
2057
   detected automatically. For example if your image is in <my_directory> and is
2058
   named "my_image.jpg" the script will look in the same directory for an image
2059
   named "my_image@2x.jpg".
2060

    
2061
   In case that you don’t want to use a high-resolution image, the
2062
   normal-resolution image will be visible.
2063

    
2064
More branding
2065
~~~~~~~~~~~~~
2066

    
2067
Although, it is not 100% branding-related, further verbal customization is
2068
feasible.
2069

    
2070
**EMAILS**
2071

    
2072
The output of all email `*`.txt files will be already customized to contain your
2073
company and service names but you can further alter their content if you feel it
2074
best fits your needs.
2075

    
2076
In order to overwrite one or more email-templates you need to place your
2077
modified <email-file>.txt files respecting the following structure:
2078

    
2079
  **/etc/synnefo/templates/**
2080
      **im/**
2081
          | activation_email.txt
2082
          | email.txt
2083
          | invitation.txt
2084
          | switch_accounts_email.txt
2085
          | welcome_email.txt
2086
          **projects/**
2087
              | project_approval_notification.txt
2088
              | project_denial_notification.txt
2089
              | project_membership_change_notification.txt
2090
              | project_membership_enroll_notification.txt
2091
              | project_membership_leave_request_notification.txt
2092
              | project_membership_request_notification.txt
2093
              | project_suspension_notification.txt
2094
              | project_termination_notification.txt
2095
      **registration/**
2096
          | email_change_email.txt
2097
          | password_email.txt
2098

    
2099
Feel free to omit any of the above files you do not wish to overwrite.
2100

    
2101
Below is a list of all emails sent by Synnefo to users along with a short
2102
description and a link to their content:
2103

    
2104
* ``snf-astakos-app/astakos/im/templates/im/email.txt``
2105
  Base email template. Contains a contact email and a “thank you” message.
2106
  (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/email.txt>`_)
2107
* ``snf-astakos-app/astakos/im/templates/im/activation_email.txt`` Email sent to
2108
  user that prompts  him/her to click on a link provided to activate the account.
2109
  Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/activation_email.txt>`_)
2110
* ``snf-astakos-app/astakos/im/templates/im/invitation.txt`` Email sent to an
2111
  invited user. He/she has to click on a link provided to activate the account.
2112
  Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/invitation.txt>`_)
2113
* ``snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt`` Email
2114
  sent to user upon his/her request to associate this email address with a
2115
  shibboleth account. He/she has to click on a link provided to activate the
2116
  association. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt>`_)
2117
* ``snf-astakos-app/astakos/im/templates/im/welcome_email.txt`` Email sent to
2118
  inform the user that his/ her account has been activated. Extends “email.txt”
2119
  (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/welcome_email.txt>`_)
2120
* ``snf-astakos-app/astakos/im/templates/registration/email_change_email.txt``
2121
  Email sent to user when he/she has requested new email address assignment. The
2122
  user has to click on a link provided to validate this action. Extends
2123
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/email_change_email.txt>`_)
2124
* ``snf-astakos-app/astakos/im/templates/registration/password_email.txt`` Email
2125
  sent for resetting password purpose. The user has to click on a link provided
2126
  to validate this action. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/password_email.txt>`_)
2127
* ``snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt``
2128
  Informs  the project owner that his/her project has been approved. Extends
2129
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt>`_)
2130
* ``snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt``
2131
  Informs the project owner that his/her  project application has been denied
2132
  explaining the reasons. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt>`_)
2133
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt``
2134
  An email is sent to a user containing information about his project membership
2135
  (whether he has been accepted, rejected or removed). Extends “email.txt” (`Link
2136
  <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt>`_)
2137
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt``
2138
  Informs a user that he/she  has been enrolled to a project. Extends
2139
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt>`_)
2140
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt``
2141
  An email is sent to the project owner to make him aware of a  user having
2142
  requested to leave his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt>`_)
2143
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt``
2144
  An email is sent to the project owner to make him/her aware of a user having
2145
  requested to join  his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt>`_)
2146
* ``snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt``
2147
  An email is sent to the project owner to make him/her aware of his/her project
2148
  having been suspended. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt>`_)
2149
* ``snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt``
2150
  An email is sent to the project owner to make him/her aware of his/her project
2151
  having been terminated. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt>`_)
2152

    
2153
.. warning:: Django templates language:
2154

    
2155
  If you choose to  overwrite these email templates, be mindful of the necessary
2156
  information contained in django template variables that must not be omitted,
2157
  such as the activation link for activating one’s account and many more.
2158
  These variables are contained into {{}} inside the templates.
2159

    
2160
**Astakos landing page**
2161

    
2162
Astakos generates sensible default values used to display component-
2163
specific details in several places across views (dashboard, cloudbar
2164
etc.). One of these places is Astakos landing page where Synnefo components are
2165
featured.
2166

    
2167
In case those values doesn't seem to suit your deployment, Astakos allows
2168
you to override any of them using ``ASTAKOS_COMPONENTS_META`` setting
2169
in your ``/etc/synnefo/20-snf-astakos-app-settings.conf`` configuration file.
2170

    
2171
So, for example if you want to add your own image for Astakos service and in the
2172
same time hide Cyclades service from Astakos landing page you can
2173
add the following line to your configuration file:
2174

    
2175
.. code-block:: python
2176

    
2177
  ASTAKOS_COMPONENTS_META = {
2178
    'astakos': {
2179
      'dashboard': {
2180
        'icon': '<path-to-your-icon>'
2181
      }
2182
    },
2183
    'cyclades': {
2184
      'dashboard': {
2185
        'show': False
2186
      }
2187
    }
2188
  }
2189

    
2190
A complete list of available keys is shown below:
2191

    
2192
.. code-block:: python
2193

    
2194
  '<component-name>' = {
2195
    'order': 1,
2196
    'dashboard': {
2197
      'order': 1,
2198
      'show': True,
2199
      'description': '<component-description>',
2200
      'icon': '<component-icon-path>',
2201
    },
2202
    'cloudbar': {
2203
      'show': True
2204
    }
2205
  }
2206

    
2207

    
2208
**403, 404 and 500 pages**
2209

    
2210
Feel free to add your own 403 (HTTP Forbidden), 404 (Page not found) and
2211
500 (server error) pages.
2212
To override the default Synnefo error views, you must write and include any of
2213
the files 403.html, 404.html and 500.html in your
2214
**/etc/synnefo/templates/** directory.
2215

    
2216
Their content is up to you, but you may use as guides the default error pages
2217
found in:
2218

    
2219
  **/synnefo/snf-webproject/synnefo/webproject/templates/**
2220
    | 403.html
2221
    | 404.html
2222
    | 500.html
2223

    
2224

    
2225

    
2226
.. RabbitMQ
2227

    
2228
RabbitMQ Broker
2229
---------------
2230

    
2231
Queue nodes run the RabbitMQ sofware, which provides AMQP functionality. To
2232
guarantee high-availability, more than one Queue nodes should be deployed, each
2233
of them belonging to the same `RabbitMQ cluster
2234
<http://www.rabbitmq.com/clustering.html>`_. Synnefo uses the RabbitMQ
2235
active/active `High Available Queues <http://www.rabbitmq.com/ha.html>`_ which
2236
are mirrored between two nodes within a RabbitMQ cluster.
2237

    
2238
The RabbitMQ nodes that form the cluster, are declared to Synnefo through the
2239
`AMQP_HOSTS` setting. Each time a Synnefo component needs to connect to
2240
RabbitMQ, one of these nodes is chosen in a random way. The client that Synnefo
2241
uses to connect to RabbitMQ, handles connection failures transparently and
2242
tries to reconnect to a different node. As long as one of these nodes are up
2243
and running, functionality of Synnefo should not be downgraded by the RabbitMQ
2244
node failures.
2245

    
2246
All the queues that are being used are declared as durable, meaning that
2247
messages are persistently stored to RabbitMQ, until they get successfully
2248
processed by a client.
2249

    
2250
Currently, RabbitMQ is used by the following components:
2251

    
2252
* `snf-ganeti-eventd` and `snf-progress-monitor`:
2253
  These components send messages concerning the status and progress of
2254
  jobs in the Ganeti backend.
2255
* `snf-dispatcher`: This daemon, consumes the messages that are sent from
2256
  the above components, and updates the Cyclades DB accordingly.
2257

    
2258

    
2259
Installation
2260
~~~~~~~~~~~~
2261

    
2262
Please check the RabbitMQ documentation which covers extensively the
2263
`installation of RabbitMQ server <http://www.rabbitmq.com/download.html>`_ and
2264
the setup of a `RabbitMQ cluster <http://www.rabbitmq.com/clustering.html>`_.
2265
Also, check out the `web management plugin
2266
<http://www.rabbitmq.com/management.html>`_ that can be useful for managing and
2267
monitoring RabbitMQ.
2268

    
2269
For a basic installation of RabbitMQ on two nodes (node1 and node2) you can do
2270
the following:
2271

    
2272
On both nodes, install rabbitmq-server and create a Synnefo user:
2273

    
2274
.. code-block:: console
2275

    
2276
  $ apt-get install rabbitmq-server
2277
  $ rabbitmqctl add_user synnefo "example_pass"
2278
  $ rabbitmqctl set_permissions synnefo  ".*" ".*" ".*"
2279

    
2280
Also guarantee that both nodes share the same cookie, by running:
2281

    
2282
.. code-block:: console
2283

    
2284
  $ scp node1:/var/lib/rabbitmq/.erlang.cookie node2:/var/lib/rabbitmq/.erlang.cookie
2285

    
2286
and restart the nodes:
2287

    
2288
.. code-block:: console
2289

    
2290
  $ /etc/init.d/rabbitmq-server restart
2291

    
2292

    
2293
To setup the RabbitMQ cluster run:
2294

    
2295
.. code-block:: console
2296

    
2297
  root@node2: rabbitmqctl stop_app
2298
  root@node2: rabbitmqctl reset
2299
  root@node2: rabbitmqctl cluster rabbit@node1 rabbit@node2
2300
  root@node2: rabbitmqctl start_app
2301

    
2302
You can verify that the cluster is set up correctly by running:
2303

    
2304
.. code-block:: console
2305

    
2306
  root@node2: rabbitmqctl cluster_status
2307

    
2308

    
2309
Logging
2310
-------
2311

    
2312
Logging in Synnefo is using Python's logging module. The module is configured
2313
using dictionary configuration, whose format is described here:
2314

    
2315
http://docs.python.org/release/2.7.1/library/logging.html#logging-config-dictschema
2316

    
2317
The logging configuration dictionary is defined in
2318
``/etc/synnefo/10-snf-webproject-logging.conf``
2319

    
2320
The administrator can have logging control by modifying the ``LOGGING_SETUP``
2321
dictionary, and defining subloggers with different handlers and log levels.
2322

    
2323

    
2324
.. _scale-up:
2325

    
2326
Scaling up to multiple nodes
2327
============================
2328

    
2329
Here we will describe how should a large scale Synnefo deployment look like. Make
2330
sure you are familiar with Synnefo and Ganeti before proceeding with this section.
2331
This means you should at least have already set up successfully a working Synnefo
2332
deployment as described in the :ref:`Admin's Installation Guide
2333
<quick-install-admin-guide>` and also read the Administrator's Guide until this
2334
section.
2335

    
2336
Graph of a scale-out Synnefo deployment
2337
---------------------------------------
2338

    
2339
Each box in the following graph corresponds to a distinct physical node:
2340

    
2341
.. image:: images/synnefo-arch2-roles.png
2342
   :width: 100%
2343
   :target: _images/synnefo-arch2-roles.png
2344

    
2345
The above graph is actually the same with the one at the beginning of this
2346
:ref:`guide <admin-guide>`, with the only difference that here we show the
2347
Synnefo roles of each physical node. These roles are described in the
2348
following section.
2349

    
2350
.. _physical-node-roles:
2351

    
2352
Physical Node roles
2353
-------------------
2354

    
2355
As appears in the previous graph, a scale-out Synnefo deployment consists of
2356
multiple physical nodes that have the following roles:
2357

    
2358
* **WEBSERVER**: A web server running in front of gunicorn (e.g.: Apache, nginx)
2359
* **ASTAKOS**: The Astakos application (gunicorn)
2360
* **ASTAKOS_DB**: The Astakos database (postgresql)
2361
* **PITHOS**: The Pithos application (gunicorn)
2362
* **PITHOS_DB**: The Pithos database (postgresql)
2363
* **CYCLADES**: The Cyclades application (gunicorn)
2364
* **CYCLADES_DB**: The Cyclades database (postgresql)
2365
* **MQ**: The message queue (RabbitMQ)
2366
* **GANETI_MASTER**: The Ganeti master of a Ganeti cluster
2367
* **GANETI_NODE** : A VM-capable Ganeti node of a Ganeti cluster
2368

    
2369
You will probably also have:
2370

    
2371
* **CMS**: The CMS used as a frotend portal for the Synnefo services
2372
* **NS**: A nameserver serving all other Synnefo nodes and resolving Synnefo FQDNs
2373
* **CLIENT**: A machine that runs the Synnefo clients (e.g.: kamaki, Web UI),
2374
              most of the times, the end user's local machine
2375

    
2376
From this point we will also refer to the following groups of roles:
2377

    
2378
* **SYNNEFO**: [ **ASTAKOS**, **ASTAKOS_DB**, **PITHOS**, **PITHOS_DB**, **CYCLADES**, **CYCLADES_DB**, **MQ**, **CMS**]
2379
* **G_BACKEND**: [**GANETI_MASTER**, **GANETI_NODE**]
2380

    
2381
Of course, when deploying Synnefo you can combine multiple of the above roles on a
2382
single physical node, but if you are trying to scale out, the above separation
2383
gives you significant advantages.
2384

    
2385
So, in the next section we will take a look on what components you will have to
2386
install on each physical node depending on its Synnefo role. We assume the graph's
2387
architecture.
2388

    
2389
Components for each role
2390
------------------------
2391

    
2392
When deploying Synnefo in large scale, you need to install different Synnefo
2393
or/and third party components on different physical nodes according to their
2394
Synnefo role, as stated in the previous section.
2395

    
2396
Specifically:
2397

    
2398
Role **WEBSERVER**
2399
    * Synnefo components: `None`
2400
    * 3rd party components: Apache
2401
Role **ASTAKOS**
2402
    * Synnefo components: `snf-webproject`, `snf-astakos-app`
2403
    * 3rd party components: Django, Gunicorn
2404
Role **ASTAKOS_DB**
2405
    * Synnefo components: `None`
2406
    * 3rd party components: PostgreSQL
2407
Role **PITHOS**
2408
    * Synnefo components: `snf-webproject`, `snf-pithos-app`, `snf-pithos-webclient`
2409
    * 3rd party components: Django, Gunicorn
2410
Role **PITHOS_DB**
2411
    * Synnefo components: `None`
2412
    * 3rd party components: PostgreSQL
2413
Role **CYCLADES**
2414
    * Synnefo components: `snf-webproject`, `snf-cyclades-app`, `snf-vncauthproxy`
2415
    * 3rd party components: Django Gunicorn
2416
Role **CYCLADES_DB**
2417
    * Synnefo components: `None`
2418
    * 3rd party components: PostgreSQL
2419
Role **MQ**
2420
    * Synnefo components: `None`
2421
    * 3rd party components: RabbitMQ
2422
Role **GANETI_MASTER**
2423
    * Synnefo components: `snf-cyclades-gtools`
2424
    * 3rd party components: Ganeti
2425
Role **GANETI_NODE**
2426
    * Synnefo components: `snf-cyclades-gtools`, `snf-network`, `snf-image`, `nfdhcpd`
2427
    * 3rd party components: Ganeti
2428
Role **CMS**
2429
    * Synnefo components: `snf-webproject`, `snf-cloudcms`
2430
    * 3rd party components: Django, Gunicorn
2431
Role **NS**
2432
    * Synnefo components: `None`
2433
    * 3rd party components: BIND
2434
Role **CLIENT**
2435
    * Synnefo components: `kamaki`, `snf-image-creator`
2436
    * 3rd party components: `None`
2437

    
2438
Example scale out installation
2439
------------------------------
2440

    
2441
In this section we describe an example of a medium scale installation which
2442
combines multiple roles on 10 different physical nodes. We also provide a
2443
:ref:`guide <i-synnefo>` to help with such an install.
2444

    
2445
We assume that we have the following 10 physical nodes with the corresponding
2446
roles:
2447

    
2448
Node1:
2449
    **WEBSERVER**, **ASTAKOS**
2450
      Guide sections:
2451
        * :ref:`apt <i-apt>`
2452
        * :ref:`gunicorn <i-gunicorn>`
2453
        * :ref:`apache <i-apache>`
2454
        * :ref:`snf-webproject <i-webproject>`
2455
        * :ref:`snf-astakos-app <i-astakos>`
2456
Node2:
2457
    **WEBSERVER**, **PITHOS**
2458
      Guide sections:
2459
        * :ref:`apt <i-apt>`
2460
        * :ref:`gunicorn <i-gunicorn>`
2461
        * :ref:`apache <i-apache>`
2462
        * :ref:`snf-webproject <i-webproject>`
2463
        * :ref:`snf-pithos-app <i-pithos>`
2464
        * :ref:`snf-pithos-webclient <i-pithos>`
2465
Node3:
2466
    **WEBSERVER**, **CYCLADES**
2467
      Guide sections:
2468
        * :ref:`apt <i-apt>`
2469
        * :ref:`gunicorn <i-gunicorn>`
2470
        * :ref:`apache <i-apache>`
2471
        * :ref:`snf-webproject <i-webproject>`
2472
        * :ref:`snf-cyclades-app <i-cyclades>`
2473
        * :ref:`snf-vncauthproxy <i-cyclades>`
2474
Node4:
2475
    **WEBSERVER**, **CMS**
2476
      Guide sections:
2477
        * :ref:`apt <i-apt>`
2478
        * :ref:`gunicorn <i-gunicorn>`
2479
        * :ref:`apache <i-apache>`
2480
        * :ref:`snf-webproject <i-webproject>`
2481
        * :ref:`snf-cloudcms <i-cms>`
2482
Node5:
2483
    **ASTAKOS_DB**, **PITHOS_DB**, **CYCLADES_DB**
2484
      Guide sections:
2485
        * :ref:`apt <i-apt>`
2486
        * :ref:`postgresql <i-db>`
2487
Node6:
2488
    **MQ**
2489
      Guide sections:
2490
        * :ref:`apt <i-apt>`
2491
        * :ref:`rabbitmq <i-mq>`
2492
Node7:
2493
    **GANETI_MASTER**, **GANETI_NODE**
2494
      Guide sections:
2495
        * :ref:`apt <i-apt>`
2496
        * :ref:`general <i-backends>`
2497
        * :ref:`ganeti <i-ganeti>`
2498
        * :ref:`snf-cyclades-gtools <i-gtools>`
2499
        * :ref:`snf-network <i-network>`
2500
        * :ref:`snf-image <i-image>`
2501
        * :ref:`nfdhcpd <i-network>`
2502
Node8:
2503
    **GANETI_NODE**
2504
      Guide sections:
2505
        * :ref:`apt <i-apt>`
2506
        * :ref:`general <i-backends>`
2507
        * :ref:`ganeti <i-ganeti>`
2508
        * :ref:`snf-cyclades-gtools <i-gtools>`
2509
        * :ref:`snf-network <i-network>`
2510
        * :ref:`snf-image <i-image>`
2511
        * :ref:`nfdhcpd <i-network>`
2512
Node9:
2513
    **GANETI_NODE**
2514
      Guide sections:
2515
        `Same as Node8`
2516
Node10:
2517
    **GANETI_NODE**
2518
      Guide sections:
2519
        `Same as Node8`
2520

    
2521
All sections: :ref:`Scale out Guide <i-synnefo>`
2522

    
2523

    
2524
Upgrade Notes
2525
=============
2526

    
2527
.. toctree::
2528
   :maxdepth: 1
2529

    
2530
   v0.12 -> v0.13 <upgrade/upgrade-0.13>
2531
   v0.13 -> v0.14 <upgrade/upgrade-0.14>
2532
   v0.14 -> v0.14.2 <upgrade/upgrade-0.14.2>
2533
   v0.14.5 -> v0.14.6 <upgrade/upgrade-0.14.6>
2534
   v0.14.7 -> v0.14.8 <upgrade/upgrade-0.14.8>
2535
   v0.14.9 -> v0.14.10 <upgrade/upgrade-0.14.10>
2536
   v0.14 -> v0.15 <upgrade/upgrade-0.15>
2537

    
2538

    
2539
Changelog, NEWS
2540
===============
2541

    
2542

    
2543
* v0.15rc5 :ref:`Changelog <Changelog-0.15rc5>`
2544
* v0.15rc4 :ref:`Changelog <Changelog-0.15rc4>`
2545
* v0.14.10 :ref:`Changelog <Changelog-0.14.10>`, :ref:`NEWS <NEWS-0.14.10>`
2546
* v0.14.9 :ref:`Changelog <Changelog-0.14.9>`, :ref:`NEWS <NEWS-0.14.9>`
2547
* v0.14.8 :ref:`Changelog <Changelog-0.14.8>`, :ref:`NEWS <NEWS-0.14.8>`
2548
* v0.14.7 :ref:`Changelog <Changelog-0.14.7>`, :ref:`NEWS <NEWS-0.14.7>`
2549
* v0.14.6 :ref:`Changelog <Changelog-0.14.6>`, :ref:`NEWS <NEWS-0.14.6>`
2550
* v0.14.5 :ref:`Changelog <Changelog-0.14.5>`, :ref:`NEWS <NEWS-0.14.5>`
2551
* v0.14.4 :ref:`Changelog <Changelog-0.14.4>`, :ref:`NEWS <NEWS-0.14.4>`
2552
* v0.14.3 :ref:`Changelog <Changelog-0.14.3>`, :ref:`NEWS <NEWS-0.14.3>`
2553
* v0.14.2 :ref:`Changelog <Changelog-0.14.2>`, :ref:`NEWS <NEWS-0.14.2>`
2554
* v0.14 :ref:`Changelog <Changelog-0.14>`, :ref:`NEWS <NEWS-0.14>`
2555
* v0.13 :ref:`Changelog <Changelog-0.13>`, :ref:`NEWS <NEWS-0.13>`