Statistics
| Branch: | Tag: | Revision:

root / docs / admin-guide.rst @ 6d8a47d0

History | View | Annotate | Download (78 kB)

1
.. _admin-guide:
2

    
3
Synnefo Administrator's Guide
4
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5

    
6
This is the complete Synnefo Administrator's Guide.
7

    
8

    
9
.. _syn+archip:
10

    
11
General Synnefo Architecture
12
============================
13

    
14
The following figure shows a detailed view of the whole Synnefo architecture
15
and how it interacts with multiple Ganeti clusters. We hope that after reading
16
the Administrator's Guide you will be able to understand every component and
17
all the interactions between them.
18

    
19
.. image:: images/synnefo-arch2.png
20
   :width: 100%
21
   :target: _images/synnefo-arch2.png
22

    
23
Synnefo also supports RADOS as an alternative storage backend for
24
Files/Images/VM disks. You will find the :ref:`corresponding figure
25
<syn+archip+rados>` later in this guide.
26

    
27

    
28
Identity Service (Astakos)
29
==========================
30

    
31

    
32
Authentication methods
33
----------------------
34

    
35
Astakos supports multiple authentication methods:
36

    
37
 * local username/password
38
 * LDAP / Active Directory
39
 * SAML 2.0 (Shibboleth) federated logins
40
 * Google
41
 * Twitter
42
 * LinkedIn
43

    
44
.. _shibboleth-auth:
45

    
46
Shibboleth Authentication
47
~~~~~~~~~~~~~~~~~~~~~~~~~
48

    
49
Astakos can delegate user authentication to a Shibboleth federation.
50

    
51
To setup shibboleth, install package::
52

    
53
  apt-get install libapache2-mod-shib2
54

    
55
Change appropriately the configuration files in ``/etc/shibboleth``.
56

    
57
Add in ``/etc/apache2/sites-available/synnefo-ssl``::
58

    
59
  ShibConfig /etc/shibboleth/shibboleth2.xml
60
  Alias      /shibboleth-sp /usr/share/shibboleth
61

    
62
  <Location /ui/login/shibboleth>
63
    AuthType shibboleth
64
    ShibRequireSession On
65
    ShibUseHeaders On
66
    require valid-user
67
  </Location>
68

    
69
and before the line containing::
70

    
71
  ProxyPass        / http://localhost:8080/ retry=0
72

    
73
add::
74

    
75
  ProxyPass /Shibboleth.sso !
76

    
77
Then, enable the shibboleth module::
78

    
79
  a2enmod shib2
80

    
81
After passing through the apache module, the following tokens should be
82
available at the destination::
83

    
84
  eppn # eduPersonPrincipalName
85
  Shib-InetOrgPerson-givenName
86
  Shib-Person-surname
87
  Shib-Person-commonName
88
  Shib-InetOrgPerson-displayName
89
  Shib-EP-Affiliation
90
  Shib-Session-ID
91

    
92
Finally, add 'shibboleth' in ``ASTAKOS_IM_MODULES`` list. The variable resides
93
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
94

    
95
Twitter Authentication
96
~~~~~~~~~~~~~~~~~~~~~~
97

    
98
To enable twitter authentication while signed in under a Twitter account,
99
visit dev.twitter.com/apps.
100

    
101
Click Create an application.
102

    
103
Fill the necessary information and for callback URL give::
104

    
105
    https://node1.example.com/ui/login/twitter/authenticated
106

    
107
Finally, add 'twitter' in ``ASTAKOS_IM_MODULES`` list. The variable resides
108
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
109

    
110
Google Authentication
111
~~~~~~~~~~~~~~~~~~~~~
112

    
113
To enable google authentication while signed in under a Google account,
114
visit https://code.google.com/apis/console/.
115

    
116
Under API Access select Create another client ID, select Web application,
117
expand more options in Your site or hostname section and in Authorized
118
Redirect URIs add:
119

    
120

    
121
Fill the necessary information and for callback URL give::
122

    
123
    https://node1.example.com/ui/login/google/authenticated
124

    
125
Finally, add 'google' in ``ASTAKOS_IM_MODULES`` list. The variable resides
126
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
127

    
128

    
129
Working with Astakos
130
--------------------
131

    
132
User registration
133
~~~~~~~~~~~~~~~~~
134

    
135
When a new user signs up, he/she is not directly marked as active. You can see 
136
his/her state by running (on the machine that runs the Astakos app):
137

    
138
.. code-block:: console
139

    
140
   $ snf-manage user-list
141

    
142
More detailed user status is provided in the `status` field of the `user-show` 
143
command:
144

    
145
.. code-block:: console
146

    
147
  $ snf-manage user-show <user-id>
148

    
149
  id                  : 6
150
  uuid                : 78661411-5eed-412f-a9ea-2de24f542c2e
151
  status              : Accepted/Active (accepted policy: manual)
152
  email               : user@synnefo.org
153
  ....
154

    
155
Based on the `astakos-app` configuration, there are several ways for a user to
156
get verified and activated in order to be able to login. We discuss the user
157
verification and activation flow in the following section.
158

    
159
User activation flow
160
````````````````````
161

    
162
A user can register for an account using the astakos signup form. Once the form
163
is submited successfully a user entry is created in astakos database. That entry
164
is passed through the astakos activation backend which handles whether the user
165
should be automatically verified and activated.
166

    
167
Email verification
168
``````````````````
169

    
170
The verification process takes place in order to ensure that the user owns the
171
email provided during the signup process. By default, after each successful
172
signup astakos notifies user with an verification url via email. 
173

    
174
At this stage:
175

    
176
    * subsequent registrations invalidate and delete the previous registrations 
177
      of the same email address.
178

    
179
    * in case user misses the initial notification, additional emails can be
180
      send either via the url which is prompted to the user if he tries to
181
      login, or by the administrator using the ``snf-manage user-activation-send
182
      <userid>`` command.
183

    
184
    * administrator may also enforce a user to get verified using the
185
      ``snf-manage user-modify --verify <userid>`` command.
186

    
187
Account activation
188
``````````````````
189

    
190
Once the user gets verified, it is time for Astakos to decide whether or not to
191
proceed through user activation process. If ``ASTAKOS_MODERATION_ENABLED``
192
setting is set to ``False`` (default value) user gets activated automatically. 
193

    
194
In case the moderation is enabled Astakos may still automatically activate the
195
user in the following cases:
196

    
197
    * User email matches any of the regular expressions defined in
198
      ``ASTAKOS_RE_USER_EMAIL_PATTERNS`` (defaults to ``[]``)
199
    * User used a signup method (e.g. ``shibboleth``) for which automatic
200
      activation is enabled (see 
201
      :ref:`authentication methods policies <auth_methods_policies>`).
202

    
203
If all of the above fail to trigger automatic activation, an email is sent to
204
the persons listed in ``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings,
205
notifing that there is a new user pending for moderation and that it's up to
206
the administrator to decide if the user should be activated. The UI also shows
207
a corresponding 'pending moderation' message to the user. The administrator can
208
activate a user using the ``snf-manage user-modify`` command:
209

    
210
.. code-block:: console
211

    
212
    # command to activate a pending user
213
    $ snf-manage user-modify --accept <userid>
214

    
215
    # command to reject a pending user
216
    $ snf-manage user-modify --reject --reject-reason="spammer" <userid>
217

    
218
Once the activation process finishes, a greeting message is sent to the user
219
email address and a notification for the activation to the persons listed in
220
``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings. Once activated the user is
221
able to login and access the Synnefo services.
222

    
223
Additional authentication methods
224
`````````````````````````````````
225

    
226
Astakos supports third party logins from external identity providers. This
227
can be usefull since it allows users to use their existing credentials to 
228
login to astakos service.
229

    
230
Currently astakos supports the following identity providers:
231

    
232
    * `Shibboleth <http://www.internet2.edu/shibboleth>`_ (module name
233
      ``shibboleth``)
234
    * `Google <https://developers.google.com/accounts/docs/OAuth2>`_ (module
235
      name ``google``)
236
    * `Twitter <https://dev.twitter.com/docs/auth>`_ (module name ``twitter``)
237
    * `LinkedIn <http://developer.linkedin.com/documents/authentication>`_
238
      (module name ``linkedin``)
239

    
240
To enable any of the above modules (by default only ``local`` accounts are
241
allowed), retrieve and set the required provider settings and append the 
242
module name in ``ASTAKOS_IM_MODULES``.
243

    
244
.. code-block:: python
245

    
246
    # settings from https://code.google.com/apis/console/
247
    ASTAKOS_GOOGLE_CLIENT_ID = '1111111111-epi60tvimgha63qqnjo40cljkojcann3.apps.googleusercontent.com'
248
    ASTAKOS_GOOGLE_SECRET = 'tNDQqTDKlTf7_LaeUcWTWwZM'
249
    
250
    # let users signup and login using their google account
251
    ASTAKOS_IM_MODULES = ['local', 'google']
252

    
253

    
254
.. _auth_methods_policies:
255

    
256
Authentication method policies
257
``````````````````````````````
258

    
259
Astakos allows you to override the default policies for each enabled provider 
260
separately by adding the approriate settings in your ``.conf`` files in the 
261
following format:
262

    
263
**ASTAKOS_AUTH_PROVIDER_<module>_<policy>_POLICY**
264

    
265
Available policies are:
266

    
267
    * **CREATE** Users can signup using that provider (default: ``True``) 
268
    * **REMOVE/ADD** Users can remove/add login method from their profile 
269
      (default: ``True``)
270
    * **AUTOMODERATE** Automatically activate users that signup using that
271
      provider (default: ``False``)
272
    * **LOGIN** Whether or not users can use the provider to login (default:
273
      ``True``).
274

    
275
e.g. to enable automatic activation for your academic users, while keeping 
276
locally signed up users under moderation you can apply the following settings.
277

    
278
.. code-block:: python
279

    
280
    ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_AUTOMODERATE_POLICY = True
281
    ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_REMOVE_POLICY = False
282

    
283
User login
284
~~~~~~~~~~
285

    
286
During the logging procedure, the user is authenticated by the respective
287
identity provider.
288

    
289
If ``ASTAKOS_RECAPTCHA_ENABLED`` is set and the user fails several times
290
(``ASTAKOS_RATELIMIT_RETRIES_ALLOWED`` setting) to provide the correct
291
credentials for a local account, he/she is then prompted to solve a captcha
292
challenge.
293

    
294
Upon success, the system renews the token (if it has expired), logins the user
295
and sets the cookie, before redirecting the user to the ``next`` parameter
296
value.
297

    
298
Setting quota limits
299
~~~~~~~~~~~~~~~~~~~~
300

    
301
Set default quota
302
`````````````````
303
To inspect current default base quota limits, run::
304

    
305
   # snf-manage resource-list
306

    
307
You can modify the default base quota limit for all future users with::
308

    
309
   # snf-manage resource-modify <resource_name> --default-quota <value>
310

    
311
Set base quota for individual users
312
```````````````````````````````````
313

    
314
For individual users that need different quota than the default
315
you can set it for each resource like this::
316

    
317
    # use this to display quota / uuid
318
    # snf-manage user-show 'uuid or email' --quota
319

    
320
    # snf-manage user-modify <user-uuid> --base-quota 'cyclades.vm' 10
321

    
322
You can set base quota for all existing users, with possible exceptions, using::
323

    
324
    # snf-manage user-modify --all --base-quota cyclades.vm 10 --exclude uuid1,uuid2
325

    
326
All quota for which values different from the default have been set,
327
can be listed with::
328

    
329
    # snf-manage quota-list --with-custom=True
330

    
331

    
332
Enable the Projects feature
333
~~~~~~~~~~~~~~~~~~~~~~~~~~~
334

    
335
If you want to enable the projects feature so that users may apply
336
on their own for resources by creating and joining projects,
337
in ``20-snf-astakos-app-settings.conf`` set::
338

    
339
    # this will make the 'projects' page visible in the dashboard
340
    ASTAKOS_PROJECTS_VISIBLE = True
341

    
342
You can change the maximum allowed number of pending project applications
343
per user with::
344

    
345
    # snf-manage resource-modify astakos.pending_app --default-quota <number>
346

    
347
You can also set a user-specific limit with::
348

    
349
    # snf-manage user-modify <user-uuid> --base-quota 'astakos.pending_app' 5
350

    
351
When users apply for projects they are not automatically granted
352
the resources. They must first be approved by the administrator.
353

    
354
To list pending project applications in astakos::
355

    
356
    # snf-manage project-list --pending
357

    
358
Note the last column, the application id. To approve it::
359

    
360
    # <app id> from the last column of project-list
361
    # snf-manage project-control --approve <app id>
362

    
363
To deny an application::
364

    
365
    # snf-manage project-control --deny <app id>
366

    
367
Users designated as *project admins* can approve, deny, or modify
368
an application through the web interface. In
369
``20-snf-astakos-app-settings.conf`` set::
370

    
371
    # UUIDs of users that can approve or deny project applications from the web.
372
    ASTAKOS_PROJECT_ADMINS = [<uuid>, ...]
373

    
374

    
375
Astakos advanced operations
376
---------------------------
377

    
378
Adding "Terms of Use"
379
~~~~~~~~~~~~~~~~~~~~~
380

    
381
Astakos supports versioned terms-of-use. First of all you need to create an
382
html file that will contain your terms. For example, create the file
383
``/usr/share/synnefo/sample-terms.html``, which contains the following:
384

    
385
.. code-block:: console
386

    
387
   <h1>My cloud service terms</h1>
388

    
389
   These are the example terms for my cloud service
390

    
391
Then, add those terms-of-use with the snf-manage command:
392

    
393
.. code-block:: console
394

    
395
   $ snf-manage term-add /usr/share/synnefo/sample-terms.html
396

    
397
Your terms have been successfully added and you will see the corresponding link
398
appearing in the Astakos web pages' footer.
399

    
400
During the account registration, if there are approval terms, the user is
401
presented with an "I agree with the Terms" checkbox that needs to get checked
402
in order to proceed.
403

    
404
In case there are new approval terms that the user has not signed yet, the
405
``signed_terms_required`` view decorator redirects to the ``approval_terms``
406
view, so the user will be presented with the new terms the next time he/she
407
logins.
408

    
409
Enabling reCAPTCHA
410
~~~~~~~~~~~~~~~~~~
411

    
412
Astakos supports the `reCAPTCHA <http://www.google.com/recaptcha>`_ feature.
413
If enabled, it protects the Astakos forms from bots. To enable the feature, go
414
to https://www.google.com/recaptcha/admin/create and create your own reCAPTCHA
415
key pair. Then edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` and set
416
the corresponding variables to reflect your newly created key pair. Finally, set
417
the ``ASTAKOS_RECAPTCHA_ENABLED`` variable to ``True``:
418

    
419
.. code-block:: console
420

    
421
   ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
422
   ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
423

    
424
   ASTAKOS_RECAPTCHA_ENABLED = True
425

    
426
Restart the service on the Astakos node(s) and you are ready:
427

    
428
.. code-block:: console
429

    
430
   # /etc/init.d/gunicorn restart
431

    
432
Checkout your new Sign up page. If you see the reCAPTCHA box, you have setup
433
everything correctly.
434

    
435

    
436
Astakos internals
437
-----------------
438

    
439
X-Auth-Token
440
~~~~~~~~~~~~
441

    
442
Alice requests a specific resource from a cloud service e.g.: Pithos. In the
443
request she supplies the `X-Auth-Token` to identify whether she is eligible to
444
perform the specific task. The service contacts Astakos through its
445
``/account/v1.0/authenticate`` api call (see :ref:`authenticate-api-label`)
446
providing the specific ``X-Auth-Token``. Astakos checkes whether the token
447
belongs to an active user and it has not expired and returns a dictionary
448
containing user related information. Finally the service uses the ``uniq``
449
field included in the dictionary as the account string to identify the user
450
accessible resources.
451

    
452
.. _authentication-label:
453

    
454
Django Auth methods and Backends
455
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
456

    
457
Astakos incorporates Django user authentication system and extends its User model.
458

    
459
Since username field of django User model has a limitation of 30 characters,
460
AstakosUser is **uniquely** identified by the ``email`` instead. Therefore,
461
``astakos.im.authentication_backends.EmailBackend`` is served to authenticate a
462
user using email if the first argument is actually an email, otherwise tries
463
the username.
464

    
465
A new AstakosUser instance is assigned with a uui as username and also with a
466
``auth_token`` used by the cloud services to authenticate the user.
467
``astakos.im.authentication_backends.TokenBackend`` is also specified in order
468
to authenticate the user using the email and the token fields.
469

    
470
Logged on users can perform a number of actions:
471

    
472
 * access and edit their profile via: ``/im/profile``.
473
 * change their password via: ``/im/password``
474
 * send feedback for grnet services via: ``/im/send_feedback``
475
 * logout (and delete cookie) via: ``/im/logout``
476

    
477
Internal Astakos requests are handled using cookie-based Django user sessions.
478

    
479
External systems should forward to the ``/login`` URI. The server,
480
depending on its configuration will redirect to the appropriate login page.
481
When done with logging in, the service's login URI should redirect to the URI
482
provided with next, adding user and token parameters, which contain the email
483
and token fields respectively.
484

    
485
The login URI accepts the following parameters:
486

    
487
======================  =========================
488
Request Parameter Name  Value
489
======================  =========================
490
next                    The URI to redirect to when the process is finished
491
renew                   Force token renewal (no value parameter)
492
force                   Force logout current user (no value parameter)
493
======================  =========================
494

    
495
External systems inside the ``ASTAKOS_COOKIE_DOMAIN`` scope can acquire the
496
user information by the cookie identified by ``ASTAKOS_COOKIE_NAME`` setting
497
(set during the login procedure).
498

    
499
Finally, backend systems having acquired a token can use the
500
:ref:`authenticate-api-label` API call from a private network or through HTTPS.
501

    
502

    
503
Compute/Network/Image Service (Cyclades)
504
========================================
505

    
506
Introduction
507
------------
508

    
509
Cyclades is the Synnefo component that implements Compute, Network and Image
510
services and exposes the associated OpenStack REST APIs. By running Cyclades
511
you can provide a cloud that can handle thousands of virtual servers and
512
networks.
513

    
514
Cyclades does not include any virtualization software and knows nothing about
515
the low-level VM management operations, e.g. handling of VM creation or
516
migrations among physical nodes. Instead, Cyclades is the component that
517
handles multiple Ganeti backends and exposes the REST APIs. The administrator
518
can expand the infrastructure dynamically either by adding more Ganeti nodes
519
or by adding new Ganeti clusters. Cyclades issue VM control commands to Ganeti
520
via Ganeti's remote API and receive asynchronous notifications from Ganeti
521
backends whenever the state of a VM changes, due to Synnefo- or
522
administrator-initiated commands.
523

    
524
Cyclades is the action orchestrator and the API layer on top of multiple Ganeti
525
clusters. By this decoupled design, Ganeti cluster are self-contained and
526
the administrator has complete control on them without Cyclades knowing about
527
it. For example a VM migration to a different physical node is transparent
528
to Cyclades.
529

    
530

    
531
Working with Cyclades
532
---------------------
533

    
534
Flavors
535
~~~~~~~
536

    
537
When creating a VM, the user must specify the `flavor` of the virtual server.
538
Flavors are the virtual hardware templates, and provide a description about
539
the number of CPUs, the amount of RAM, and the size of the disk of the VM.
540
Besides the size of the disk, Cyclades flavors describe the storage backend
541
that will be used for the virtual server.
542

    
543
Flavors are created by the administrator and the user can select one of the
544
available flavors. After VM creation, the user can dynamically resize his VM,
545
by dynamically (hotplug-able) adding or removing CPU and RAM.
546

    
547
Cyclades support different storage backends that are described by the disk
548
template of the flavor, which is mapped to Ganeti's instance `disk template`.
549
Currently the available disk templates are the following:
550

    
551
* `file`: regulars file
552
* `sharedfile`: regular files on a shared directory, e.g. NFS
553
* `plain`: logical volumes
554
* `drbd`: drbd on top of lvm volumes
555
* `rbd`: rbd volumes residing inside a RADOS cluster
556
* `ext`: disks provided by an external shared storage.
557

    
558
  - `ext_archipelago`: External shared storage provided by
559
    `Archipelago <http://www.synnefo.org/docs/archipelago/latest/index.html>`_.
560

    
561
Flavors are created by the administrator using `snf-manage flavor-create`
562
command. The command takes as argument number of CPUs, amount of RAM, the size
563
of the disks and the disk templates and create the flavors that belong to the
564
cartesian product of the specified arguments. For example, the following
565
command will create two flavors of `40G` disk size with `drbd` disk template,
566
`4G` RAM and `2` or `4` CPUs.
567

    
568
.. code-block:: console
569

    
570
  snf-manage flavor-create 2,4 4096 40 drbd
571

    
572
To see the available flavors, run `snf-manage flavor-list` command. Finally,
573
the administrator can delete a flavor by using `flavor-modify` command:
574

    
575
.. code-block:: console
576

    
577
  snf-manage flavor-modify --deleted=True <flavor_id>
578

    
579
Images
580
~~~~~~
581

    
582
When creating a VM the user must also specify the `image` of the virtual
583
server. Images are the static templates from which VM instances are
584
initiated. Cyclades uses Pithos to store system and user-provided images,
585
taking advantage of all Pithos features, like deduplication and syncing
586
protocol. An image is a file stored to Pithos with additional metadata that
587
are describing the image, e.g. the OS family or the root partition. To create
588
a new image, the administrator or the user has to upload it a file to Pithos,
589
and then register it as an Image with Cyclades. Then the user can use this
590
image to spawn new VMs from it.
591

    
592
Images can be private, public or shared between users, exactly like Pithos
593
files. Since user-provided public images can be untrusted, the administrator
594
can denote which users are trusted by adding them to the
595
`UI_SYSTEM_IMAGES_OWNERS` setting in the
596
`/etc/synnefo/20-snf-cyclades-app-ui.conf` file. Images of those users are
597
properly displayed in the UI.
598

    
599
When creating a new VM, Cyclades pass the location of the image and it's
600
metadata to Ganeti. After Ganeti creates the instance's disk, `snf-image`
601
will copy the image to the new disk and perform the image customization
602
phase. During the phase, `snf-image` sends notifications to Cyclades about
603
the progress of the image deployment and customization. Customization includes
604
resizing the root file system, file injection (e.g. SSH keys) and setting
605
a custom hostname. For better understanding of `snf-image` read the
606
corresponding `documentation
607
<http://www.synnefo.org/docs/snf-image/latest/index.html>`_.
608

    
609
For passing sensitive data about the image to Ganeti, like the VMs password,
610
Cyclades keeps all sensitive data in memory caches (memcache) and never allows
611
them to hit the disk. The data are exposed to `snf-image` via an one-time URL
612
that is exposed from the `vmapi` application. So, instead of passing sensitive
613
data to `snf-image` via Ganeti, Cyclades pass an one-time configuration URL
614
that contains a random UUID. After `snf-image` gets the sensitive data, the
615
URL is invalidated so no one else can access them.
616

    
617
The administrator can register images, exactly like users, using a system user
618
(a user that is defined in the `UI_SYSTEM_IMAGES_OWNERS` setting). For
619
example, the following command will register the
620
`pithos://u53r-un1qu3-1d/images/debian_base-6.0-7-x86_64.diskdump` as an
621
image to Cyclades:
622

    
623
.. code-block:: console
624

    
625
 kamaki image register "Debian Base" \
626
        pithos://u53r-un1qu3-1d/images/debian_base-6.0-7-x86_64.diskdump \
627
        --public \
628
        --disk-format=diskdump \
629
        --property OSFAMILY=linux --property ROOT_PARTITION=1 \
630
        --property description="Debian Squeeze Base System" \
631
        --property size=451 --property kernel=2.6.32 --property GUI="No GUI" \
632
        --property sortorder=1 --property USERS=root --property OS=debian
633

    
634
Deletion of an image is done via `kamaki image unregister` command, which will
635
delete the Cyclades Images but will leave the Pithos file as is (unregister).
636

    
637
Apart from using `kamaki` to see and hangle the available images, the
638
administrator can use `snf-manage image-list` and `snf-manage image-show`
639
commands to list and inspect the available public images. Also, the `--user-id`
640
option can be used the see the images of a specific user.
641

    
642
Virtual Servers
643
~~~~~~~~~~~~~~~
644

    
645
As mentioned, Cyclades uses Ganeti for management of VMs. The administrator can
646
handle Cyclades VMs just like any other Ganeti instance, via `gnt-instance`
647
commands. All Ganeti instances that belong to Synnefo, are separated from
648
others, by a prefix in their names. This prefix is defined in
649
``BACKEND_PREFIX_ID`` setting in
650
``/etc/synnefo/20-snf-cyclades-app-backend.conf``.
651

    
652
Apart from handling Cyclades VM at the Ganeti level, the administrator can
653
also use the `snf-manage server-*` commands. These command cover the most
654
common tasks that are relative with VM handling. Below we describe come
655
of them, but for more information you can use the `--help` option of all
656
`snf-manage server-* commands`. These command cover the most
657

    
658
The `snf-manage server-create` command can be used to create a new VM for some
659
user. This command can be useful when the administrator wants to test Cyclades
660
functionality without starting the API service, e.g. after an upgrade. Also, by
661
using `--backend-id` option, the VM will be created in the specified backend,
662
bypassing automatic VM allocation.
663

    
664
.. code-block:: console
665

    
666
 snf-manage server-create --flavor-id=1 --image-id=fc0f6858-f962-42ce-bf9a-1345f89b3d5e \
667
    --user-id=7cf4d078-67bf-424d-8ff2-8669eb4841ea --backend-id=2 \
668
    --password='example_passw0rd' --name='test_vm'
669

    
670
The above commnd will create a new VM for user
671
`7cf4d078-67bf-424d-8ff2-8669eb4841ea` in the Ganeti backend with ID 2. By
672
default this command will issue a Ganeti job to create the VM
673
(`OP_INSTANCE_CREATE`) and return. As in other commands, the `--wait=True`
674
option can be used in order to wait for the successful completion of the job.
675

    
676
`snf-manage server-list` command can be used to list all the available servers.
677
The command supports some useful options, like listing servers of a user,
678
listing servers that exist in a Ganeti backend and listing deleted servers.
679
Also, as in most of `*-list` commands, the `--filter-by` option can be used to
680
filter the results. For example, the following command will only display the
681
started servers of a specific flavor:
682

    
683
.. code-block:: console
684

    
685
 snf-manage server-list --filter-by="operstate=STARTED,flavor=<flavor_id>"
686

    
687
Another very useful command is the `server-inspect` command which will display
688
all available information about the state of the server in DB and the state
689
of the server in the Ganeti backend. The output will give you an easy overview
690
about the state of the VM which can be useful for debugging.
691

    
692
Also the administrator can `suspend` a user's VM, using the `server-modify`
693
command:
694

    
695
.. code-block:: console
696

    
697
 snf-manage server-modify --suspended=True <server_id>
698

    
699
The user is forbidden to do any action on an administratively suspended VM,
700
which is useful for abuse cases.
701

    
702
Ganeti backends
703
~~~~~~~~~~~~~~~
704

    
705
Since v0.11, Synnefo is able to manage multiple Ganeti clusters (backends)
706
making it capable to scale linearly to tens of thousands of VMs. Backends
707
can be dynamically added or removed via `snf-manage` commands.
708

    
709
Each newly created VM is allocated to a Ganeti backend by the Cyclades backend
710
allocator. The VM is "pinned" to this backend, and can not change through its
711
lifetime. The backend allocator decides in which backend to spawn the VM based
712
on the available resources of each backend, trying to balance the load between
713
them. Also, Networks are created to all Ganeti backends, in order to ensure
714
that VMs residing on different backends can be connected to the same networks.
715

    
716
A backend can be marked as `drained` in order to be excluded from automatic
717
servers allocation and not receive new servers. Also, a backend can be marked
718
as `offline` to denote that the backend is not healthy (e.g. broken master)
719
and avoid the penalty of connection timeouts.
720

    
721
Finally, Cyclades is able to manage Ganeti backends with different enabled
722
hypervisors (`kvm`, `xen`), and different enabled disk templates.
723

    
724
Listing existing backends
725
`````````````````````````
726
To list all the Ganeti backends known to Synnefo, we run:
727

    
728
.. code-block:: console
729

    
730
   $ snf-manage backend-list
731

    
732
Adding a new Ganeti backend
733
```````````````````````````
734
Backends are dynamically added under the control of Synnefo with `snf-manage
735
backend-add` command. In this section it is assumed that a Ganeti cluster,
736
named ``cluster.example.com`` is already up and running and configured to be
737
able to host Synnefo VMs.
738

    
739
To add this Ganeti cluster, we run:
740

    
741
.. code-block:: console
742

    
743
   $ snf-manage backend-add --clustername=cluster.example.com --user="synnefo_user" --pass="synnefo_pass"
744

    
745
where ``clustername`` is the Cluster hostname of the Ganeti cluster, and
746
``user`` and ``pass`` are the credentials for the `Ganeti RAPI user
747
<http://docs.ganeti.org/ganeti/2.2/html/rapi.html#users-and-passwords>`_.  All
748
backend attributes can be also changed dynamically using the `snf-manage
749
backend-modify` command.
750

    
751
``snf-manage backend-add`` will also create all existing public networks to
752
the new backend. You can verify that the backend is added, by running
753
`snf-manage backend-list`.
754

    
755
Note that no VMs will be spawned to this backend, since by default it is in a
756
``drained`` state after addition in order to manually verify the state of the
757
backend.
758

    
759
So, after making sure everything works as expected, make the new backend active
760
by un-setting the ``drained`` flag. You can do this by running:
761

    
762
.. code-block:: console
763

    
764
   $ snf-manage backend-modify --drained=False <backend_id>
765

    
766
Allocation of VMs in Ganeti backends
767
````````````````````````````````````
768
As already mentioned, the Cyclades backend allocator is responsible for
769
allocating new VMs to backends. This allocator does not choose the exact Ganeti
770
node that will host the VM but just the Ganeti backend. The exact node is
771
chosen by the Ganeti cluster's allocator (hail).
772

    
773
The decision about which backend will host a VM is based on the available
774
resources. The allocator computes a score for each backend, that shows its load
775
factor, and the one with the minimum score is chosen. The admin can exclude
776
backends from the allocation phase by marking them as ``drained`` by running:
777

    
778
.. code-block:: console
779

    
780
   $ snf-manage backend-modify --drained=True <backend_id>
781

    
782
The backend resources are periodically updated, at a period defined by
783
the ``BACKEND_REFRESH_MIN`` setting, or by running `snf-manage
784
backend-update-status` command. It is advised to have a cron job running this
785
command at a smaller interval than ``BACKEND_REFRESH_MIN`` in order to remove
786
the load of refreshing the backends stats from the VM creation phase.
787

    
788
Finally, the admin can decide to have a user's VMs being allocated to a
789
specific backend, with the ``BACKEND_PER_USER`` setting. This is a mapping
790
between users and backends. If the user is found in ``BACKEND_PER_USER``, then
791
Synnefo allocates all his/hers VMs to the specific backend in the variable,
792
even if is marked as drained (useful for testing).
793

    
794
Allocation based on disk-templates
795
**********************************
796

    
797
Besides the available resources of each Ganeti backend, the allocator takes
798
into consideration the disk template of the instance when trying to allocate it
799
to a Ganeti backend. Specifically, the allocator checks if the flavor of the
800
instance belongs to the available disk templates of each Ganeti backend.
801

    
802
A Ganeti cluster has a list of enabled disk templates
803
(`--enabled-disk-templates`) and a list of allowed disk templates for new
804
instances (`--ipolicy-disk-templates`). See the `gnt-cluster` manpage for more
805
details about these options.
806

    
807
When Synnefo allocates an instance, it checks whether the disk template of the
808
new instance belongs both in the enabled and ipolicy disk templates. You can
809
see the list of the available disk-templates by running `snf-manage
810
backend-list`. This list should be updated automatically after changing
811
these options in Ganeti and it can also be updated by running `snf-manage
812
backend-update-status`.
813

    
814
So the administrator, can route instances on different backends based on their
815
flavor disk template, by modifying the enabled or ipolicy disk templates of
816
each backend.  Also, the administrator can route instances between different
817
nodes of the same Ganeti backend, by modifying the same options at the
818
nodegroup level (see `gnt-group` manpage for mor details).
819

    
820
Removing an existing Ganeti backend
821
```````````````````````````````````
822
In order to remove an existing backend from Synnefo, you must first make
823
sure that there are not active servers in the backend, and then run:
824

    
825
.. code-block:: console
826

    
827
   # snf-manage backend-remove <backend_id>
828

    
829

    
830
Virtual Networks
831
~~~~~~~~~~~~~~~~
832

    
833
Cyclades also implements the Network service and exposes the Quantum Openstack
834
API. Cyclades supports full IPv4 and IPv6 connectivity to the public internet
835
for it's VMs. Also, Cyclades provides L2 and L3 virtual private networks,
836
giving the user freedom to create arbitraty network topologies of
837
interconnected VMs.
838

    
839
Public networking is desployment specific and must be customized based on the
840
specific needs of the system administrator. Private virtual networks can be
841
provided by different network technologies which are exposed as different
842
network flavors. For better understanding of networking please refer to the
843
:ref:`Network <networks>` section.
844

    
845
A Cyclades virtual network is an isolated Layer-2 broadcast domain. A network
846
can also have an associated IPv4 and IPv6 subnet representing the Layer-3
847
characteristics of the network. Each subnet represents an IP address block
848
that is used in order to assign addresses to VMs.
849

    
850
To connect a VM to a network, a port must be created, which represent a virtual
851
port on a network switch. VMs are connected to networks by attaching a virtual
852
interface to a port.
853

    
854
Cyclades also supports `floating IPs`, which are public IPv4 addresses that
855
can dynamically(hotplug-able) be added and removed to VMs. Floating IPs are
856
a quotable resource that is allocated to each user. Unlike other cloud
857
platforms, floating IPs are not implemented using 1-1 NAT to a ports private
858
IP. Instead, floating IPs are directly assigned to virtual interfaces of VMs.
859

    
860
Exactly like VMS, networks can be handled as Ganeti networks via `gnt-network`
861
commands. All Ganeti networks that belong to Synnefo are named with the prefix
862
`${BACKEND_PREFIX_ID}-net-`. Also, there are a number of `snf-manage` commands
863
for handling of `networks`, `subnets`, `ports` and `floating IPs`. Below
864
we will present a use case scenario using some of these commands. For better
865
understanding of these commands, refer to their help messages.
866

    
867
Create a virtual private network for user
868
`7cf4d078-67bf-424d-8ff2-8669eb4841ea` using the `PHYSICAL_VLAN` flavor, which
869
means that the network will be uniquely assigned a phsyical VLAN. The network
870
is assigned an IPv4 subnet, described by it's CIDR and gateway. Also,
871
the `--dhcp=True` option is used, to make `nfdhcpd` respone to DHCP queries
872
from VMs.
873

    
874
.. code-block:: console
875

    
876
 snf-manage network-create --owner=7cf4d078-67bf-424d-8ff2-8669eb4841ea --name=prv_net-1 \
877
    --subnet=192.168.2.0/24 --gateway=192.168.2.1 --dhcp=True --flavor=PHYSICAL_VLAN
878

    
879
Inspect the state of the network in Cyclades DB and in all the Ganeti backends:
880

    
881
.. code-block:: console
882

    
883
  snf-manage network-inspect <network_id>
884

    
885
Inspect the state of the network's subnet, containg an overview of the
886
subnet's IPv4 address allocation pool:
887

    
888
.. code-block:: console
889

    
890
  snf-manage subnet-inspect <subnet_id>
891

    
892
Connect a VM to the created private network. The port will be automatically
893
be assigned an IPv4 address from one of the network's available IPs. This
894
command will result in sending an `OP_INSTANCE_MODIFY` Ganeti command and
895
attaching a NIC to the specified Ganeti instance.
896

    
897
.. code-block:: console
898

    
899
 snf-manage port-create --network=<network_id> --server=<server_id>
900

    
901
Inspect the state of the the port in Cyclades DB and in the Ganeti backend:
902

    
903
.. code-block:: console
904

    
905
 snf-manage port-inspect <port_id>
906

    
907
Disconnect the VM from the network and delete the network:
908

    
909
.. code-block:: console
910

    
911
 snf-manage port-remove <port_id>
912
 snf-manage network-remove <network_id>
913

    
914
Managing Network Resources
915
``````````````````````````
916

    
917
Proper operation of the Cyclades Network Service depends on the unique
918
assignment of specific resources to each type of virtual network. Specifically,
919
these resources are:
920

    
921
* IP addresses. Cyclades creates a Pool of IPs for each Network, and assigns a
922
  unique IP address to each VM, thus connecting it to this Network. You can see
923
  the IP pool of each network by running `snf-manage subnet-inspect
924
  <subnet_ID>`. IP pools are automatically created and managed by Cyclades,
925
  depending on the subnet of the Network.
926
* Bridges corresponding to physical VLANs, which are required for networks of
927
  type `PRIVATE_PHYSICAL_VLAN`.
928
* One Bridge corresponding to one physical VLAN which is required for networks of
929
  type `PRIVATE_MAC_PREFIX`.
930

    
931
IPv4 addresses
932
**************
933

    
934
An allocation pool of IPv4 addresses is automatically created for every network
935
with an IPv4 subnet. By default, the allocation pool contains the range of IP
936
addresses that are included in the subnet, except from the gateway and the
937
broadcast address of the network. The range of IP addresses can be restricted
938
using the `--allocation-pool` option of `snf-manage network-create` command.
939
The admin can externally reserve IP addresses to exclude them from automatic
940
allocation with the `--add-reserved-ips` option of `snf-manage network-modify`
941
command. For example the following command will reserve two IP addresses from
942
network with ID `42`:
943

    
944
.. code-block:: console
945

    
946
 snf-manage network-modify --add-reserved-ips=10.0.0.21,10.0.0.22 42
947

    
948
.. warning:: Externally reserving IP addresses is also available at the Ganeti.
949
 However, when using Cyclades with multiple Ganeti backends, the handling of
950
 IP pools must be performed from Cyclades!
951

    
952
Bridges
953
*******
954

    
955
As already mentioned Cyclades use a pool of Bridges that must correspond
956
to Physical VLAN at the Ganeti level. A bridge from the pool is assigned to
957
each network of flavor `PHYSICAL_VLAN`. Creation of this pool is done
958
using `snf-manage pool-create` command. For example the following command
959
will create a pool containing the brdiges from `prv1` to `prv21`.
960

    
961
.. code-block:: console
962

    
963
   # snf-manage pool-create --type=bridge --base=prv --size=20
964

    
965
You can verify the creation of the pool, and check its contents by running:
966

    
967
.. code-block:: console
968

    
969
   # snf-manage pool-list
970
   # snf-manage pool-show --type=bridge 1
971

    
972
Finally you can use the `pool-modify` management command in order to externally
973
reserve the values from pool, extend or shrink the pool if possible.
974

    
975
MAC Prefixes
976
************
977

    
978
Cyclades also use a pool of MAC prefixes to assign to networks of flavor
979
`MAC_FILTERED`. Handling of this pool is done exactly as with pool of bridges,
980
except that the type option must be set to mac-prefix:
981

    
982
.. code-block:: console
983

    
984
   # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
985

    
986
The above command will create a pool of MAC prefixes from ``aa:00:1`` to
987
``b9:ff:f``. The MAC prefix pool is responsible for providing only unicast and
988
locally administered MAC addresses, so many of these prefixes will be
989
externally reserved, to exclude from allocation.
990

    
991
Pool reconciliation
992
*******************
993

    
994
The management command `snf-manage reconcile-pools` can be used that all the
995
above mentioned pools are consistent and that all values that come from the
996
pool are not used more than once.
997

    
998

    
999
Cyclades advanced operations
1000
----------------------------
1001

    
1002
Reconciliation mechanism
1003
~~~~~~~~~~~~~~~~~~~~~~~~
1004

    
1005
On certain occasions, such as a Ganeti or RabbitMQ failure, the state of
1006
Cyclades database may differ from the real state of VMs and networks in the
1007
Ganeti backends. The reconciliation process is designed to synchronize
1008
the state of the Cyclades DB with Ganeti. There are two management commands
1009
for reconciling VMs and Networks
1010

    
1011
Reconciling Virtual Machines
1012
````````````````````````````
1013

    
1014
Reconciliation of VMs detects the following conditions:
1015

    
1016
 * Stale DB servers without corresponding Ganeti instances
1017
 * Orphan Ganeti instances, without corresponding DB entries
1018
 * Out-of-sync state for DB entries wrt to Ganeti instances
1019

    
1020
To detect all inconsistencies you can just run:
1021

    
1022
.. code-block:: console
1023

    
1024
  $ snf-manage reconcile-servers
1025

    
1026
Adding the `--fix-all` option, will do the actual synchronization:
1027

    
1028
.. code-block:: console
1029

    
1030
  $ snf-manage reconcile-servers --fix-all
1031

    
1032
Please see ``snf-manage reconcile-servers --help`` for all the details.
1033

    
1034
Reconciling Networks
1035
````````````````````
1036

    
1037
Reconciliation of Networks detects the following conditions:
1038

    
1039
  * Stale DB networks without corresponding Ganeti networks
1040
  * Orphan Ganeti networks, without corresponding DB entries
1041
  * Private networks that are not created to all Ganeti backends
1042

    
1043
To detect all inconsistencies you can just run:
1044

    
1045
.. code-block:: console
1046

    
1047
  $ snf-manage reconcile-networks
1048

    
1049
Adding the `--fix-all` option, will do the actual synchronization:
1050

    
1051
.. code-block:: console
1052

    
1053
  $ snf-manage reconcile-networks --fix-all
1054

    
1055
Please see ``snf-manage reconcile-networks --help`` for all the details.
1056

    
1057
Reconciling Pools
1058
`````````````````
1059

    
1060
Reconciliation of pools will check the consistency of available pools by
1061
checking that the values from each pool are not used more than once, and also
1062
that the only reserved values in a pool are the ones used. Pool reconciliation
1063
will check pools of bridges, MAC prefixes, and IPv4 addresses for all networks.
1064

    
1065
.. code-block:: console
1066

    
1067
  $ snf-manage reconcile-pools
1068
  $ snf-manage reconcile-pools --fix
1069

    
1070
Helpdesk
1071
--------
1072

    
1073
Helpdesk application provides the ability to view the virtual servers and
1074
networks of all users, along with the ability to perform some basic actions
1075
like administratively suspending a server. You can perform look-ups by
1076
user UUID or email, by server ID (vm-$id) or by an IPv4 address.
1077

    
1078
If you want to activate the helpdesk application you can set to `True` the
1079
`HELPDESK_ENABLED` setting. Access to helpdesk views (under
1080
`$BASE_URL/helpdesk`) is only to allowed to users that belong to Astakos
1081
groups defined in the `HELPDESK_PERMITTED_GROUPS` setting, which by default
1082
contains the `helpdesk` group. For example, to allow <user_id>
1083
to access helpdesk view, you should run the following command in the Astakos
1084
node:
1085

    
1086
.. code-block:: console
1087

    
1088
 snf-manage group-add helpdesk
1089
 snf-manage user-modify --add-group=helpdesk <user_id>
1090

    
1091

    
1092
Cyclades internals
1093
------------------
1094

    
1095
Asynchronous communication with Ganeti backends
1096
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1097
Synnefo uses Google Ganeti backends for VM cluster management. In order for
1098
Cyclades to be able to handle thousands of user requests, Cyclades and Ganeti
1099
communicate asynchronously. Briefly, requests are submitted to Ganeti through
1100
Ganeti's RAPI/HTTP interface, and then asynchronous notifications about the
1101
progress of Ganeti jobs are being created and pushed upwards to Cyclades. The
1102
architecture and communication with a Ganeti backend is shown in the graph
1103
below:
1104

    
1105
.. image:: images/cyclades-ganeti-communication.png
1106
   :width: 50%
1107
   :target: _images/cyclades-ganeti-communication.png
1108

    
1109
The Cyclades API server is responsible for handling user requests. Read-only
1110
requests are directly served by looking up the Cyclades DB. If the request
1111
needs an action in the Ganeti backend, Cyclades submit jobs to the Ganeti
1112
master using the `Ganeti RAPI interface
1113
<http://docs.ganeti.org/ganeti/2.2/html/rapi.html>`_.
1114

    
1115
While Ganeti executes the job, `snf-ganeti-eventd`, and `snf-progress-monitor`
1116
are monitoring the progress of the job and send corresponding messages to the
1117
RabbitMQ servers. These components are part of `snf-cyclades-gtools` and must
1118
be installed on all Ganeti nodes. Specially:
1119

    
1120
* *snf-ganeti-eventd* sends messages about operations affecting the operating
1121
  state of instances and networks. Works by monitoring the Ganeti job queue.
1122
* *snf-progress_monitor* sends messages about the progress of the Image deployment
1123
  phase which is done by the Ganeti OS Definition `snf-image`.
1124

    
1125
Finally, `snf-dispatcher` consumes messages from the RabbitMQ queues, processes
1126
these messages and properly updates the state of the Cyclades DB. Subsequent
1127
requests to the Cyclades API, will retrieve the updated state from the DB.
1128

    
1129

    
1130
Synnefo management commands ("snf-manage")
1131
==========================================
1132

    
1133
Each Synnefo service, Astakos, Pithos and Cyclades are controlled by the
1134
administrator using the "snf-manage" admin tool. This tool is an extension of
1135
the Django command-line management utility. It is run on the host that runs
1136
each service and provides different types of commands depending the services
1137
running on the host. If you are running more than one service on the same host
1138
"snf-manage" adds all the corresponding commands for each service dynamically,
1139
providing a unified admin environment.
1140

    
1141
To run "snf-manage" you just type:
1142

    
1143
.. code-block:: console
1144

    
1145
   # snf-manage <command> [arguments]
1146

    
1147
on the corresponding host that runs the service. For example, if you have all
1148
services running on different physical hosts you would do:
1149

    
1150
.. code-block:: console
1151

    
1152
   root@astakos-host # snf-manage <astakos-command> [argument]
1153
   root@pithos-host # snf-manage <pithos-command> [argument]
1154
   root@cyclades-host # snf-manage <cyclades-command> [argument]
1155

    
1156
If you have all services running on the same host you would do:
1157

    
1158
.. code-block:: console
1159

    
1160
   root@synnefo-host # snf-manage <{astakos,pithos,cyclades}-command> [argument]
1161

    
1162
Note that you cannot execute a service's command on a host that is not running
1163
this service. For example, the following will return an error if Astakos and
1164
Cyclades are installed on different physical hosts:
1165

    
1166
.. code-block:: console
1167

    
1168
   root@astakos-host # snf-manage <cyclades-command> [argument]
1169
   Unknown command: 'cyclades-command'
1170
   Type 'snf-manage help' for usage.
1171

    
1172
This is the complete list of "snf-manage" commands for each service.
1173

    
1174
Astakos snf-manage commands
1175
---------------------------
1176

    
1177
============================  ===========================
1178
Name                          Description
1179
============================  ===========================
1180
fix-superusers                Transform superusers created by syncdb into AstakosUser instances
1181
cleanup-full                  Cleanup sessions and session catalog
1182
commission-list               List pending commissions
1183
commission-show               Show details for a pending commission
1184
component-add                 Register a component
1185
component-list                List components
1186
component-modify              Modify component attributes
1187
component-show                Show component details
1188
project-control               Manage projects and applications
1189
project-list                  List projects
1190
project-show                  Show project details
1191
quota-list                    List user quota
1192
quota-verify                  Check the integrity of user quota
1193
reconcile-resources-astakos   Reconcile resource usage of Quotaholder with Astakos DB
1194
resource-list                 List resources
1195
resource-modify               Modify a resource's default base quota and boolean flags
1196
service-export-astakos        Export Astakos services and resources in JSON format
1197
service-import                Register services
1198
service-list                  List services
1199
service-show                  Show service details
1200
term-add                      Add approval terms
1201
user-activation-send          Send user activation
1202
user-add                      Add user
1203
authpolicy-add                Create a new authentication provider policy profile
1204
authpolicy-list               List existing authentication provider policy profiles
1205
authpolicy-remove             Remove an authentication provider policy
1206
authpolicy-set                Assign an existing authentication provider policy profile to a user or group
1207
authpolicy-show               Show authentication provider profile details
1208
group-add                     Create a group with the given name
1209
group-list                    List available groups
1210
user-list                     List users
1211
user-modify                   Modify user
1212
user-show                     Show user details
1213
============================  ===========================
1214

    
1215
Pithos snf-manage commands
1216
--------------------------
1217

    
1218
============================  ===========================
1219
Name                          Description
1220
============================  ===========================
1221
reconcile-commissions-pithos  Display unresolved commissions and trigger their recovery
1222
service-export-pithos         Export Pithos services and resources in JSON format
1223
reconcile-resources-pithos    Detect unsynchronized usage between Astakos and Pithos DB resources and synchronize them if specified so.
1224
============================  ===========================
1225

    
1226
Cyclades snf-manage commands
1227
----------------------------
1228

    
1229
============================== ===========================
1230
Name                           Description
1231
============================== ===========================
1232
backend-add                    Add a new Ganeti backend
1233
backend-list                   List backends
1234
backend-modify                 Modify a backend
1235
backend-update-status          Update backend statistics for instance allocation
1236
backend-remove                 Remove a Ganeti backend
1237
enforce-resources-cyclades     Check and fix quota violations for Cyclades resources
1238
server-create                  Create a new server
1239
server-show                    Show server details
1240
server-list                    List servers
1241
server-modify                  Modify a server
1242
server-import                  Import an existing Ganeti VM into synnefo
1243
server-inspect                 Inspect a server in DB and Ganeti
1244
network-create                 Create a new network
1245
network-list                   List networks
1246
network-modify                 Modify a network
1247
network-inspect                Inspect network state in DB and Ganeti
1248
network-remove                 Delete a network
1249
flavor-create                  Create a new flavor
1250
flavor-list                    List flavors
1251
flavor-modify                  Modify a flavor
1252
image-list                     List images
1253
image-show                     Show image details
1254
pool-create                    Create a bridge or mac-prefix pool
1255
pool-show                      Show pool details
1256
pool-list                      List pools
1257
pool-modify                    Modify a pool
1258
pool-remove                    Delete a pool
1259
port-create                    Create a port connecting a server to a network
1260
port-inspect                   Inspect the state of a port in DB and Ganeti
1261
port-list                      List ports
1262
port-remove                    Delete a port
1263
floating-ip-create             Create a new floating IP
1264
floating-ip-attach             Attach a floating IP to a server
1265
floating-ip-detach             Detach a floating IP from a server
1266
floating-ip-list               List floating IPs
1267
floating-ip-remove             Delete a floating IP
1268
queue-inspect                  Inspect the messages of a RabbitMQ queue
1269
queue-retry                    Resend messages from Dead Letter queues to original exchanges
1270
service-export-cyclades        Export Cyclades services and resources in JSON format
1271
subnet-create                  Create a subnet
1272
subnet-inspect                 Inspect a subnet in DB
1273
subnet-list                    List subnets
1274
subnet-modify                  Modify a subnet
1275
reconcile-servers              Reconcile servers of Synnefo DB with state of Ganeti backend
1276
reconcile-networks             Reconcile networks of Synnefo DB with state of Ganeti backend
1277
reconcile-pools                Check consistency of pool resources
1278
reconcile-commissions-cyclades Detect and resolve pending commissions to Quotaholder
1279
reconcile-resources-cyclades   Reconcile resource usage of Astakos with Cyclades DB.
1280
============================== ===========================
1281

    
1282

    
1283
Astakos helper scripts
1284
======================
1285

    
1286
Astakos includes two scripts to facilitate the installation procedure.
1287
Running:
1288

    
1289
.. code-block:: console
1290

    
1291
   snf-component-register [<component_name>]
1292

    
1293
automates the registration of the standard Synnefo components (astakos,
1294
cyclades, and pithos) in astakos database. It internally uses the script:
1295

    
1296
.. code-block:: console
1297

    
1298
   snf-service-export <component_name> <base_url>
1299

    
1300
which simulates the export of service and resource definitions of the
1301
standard Synnefo components.
1302

    
1303

    
1304
Pithos managing accounts
1305
========================
1306

    
1307
Pithos provides a utility tool for managing accounts.
1308
To run you just type:
1309

    
1310
.. code-block:: console
1311

    
1312
   # pithos-manage-accounts <command> [arguments]
1313

    
1314
This is the list of the available commands:
1315

    
1316
============================  ===========================
1317
Name                          Description
1318
============================  ===========================
1319
delete                        Remove an account from the Pithos DB
1320
export-quota                  Export account quota in a file
1321
list                          List existing/dublicate accounts
1322
merge                         Move an account contents in another account
1323
set-container-quota           Set container quota for all or a specific account
1324
============================  ===========================
1325

    
1326

    
1327
The "kamaki" API client
1328
=======================
1329

    
1330
To upload, register or modify an image you will need the **kamaki** tool.
1331
Before proceeding make sure that it is configured properly. Verify that
1332
*image.url*, *file.url*, *user.url* and *token* are set as needed:
1333

    
1334
.. code-block:: console
1335

    
1336
   $ kamaki config list
1337

    
1338
To change a setting use ``kamaki config set``:
1339

    
1340
.. code-block:: console
1341

    
1342
   $ kamaki config set image.url https://cyclades.example.com/image
1343
   $ kamaki config set file.url https://pithos.example.com/v1
1344
   $ kamaki config set user.url https://accounts.example.com
1345
   $ kamaki config set token ...
1346

    
1347
To test that everything works, try authenticating the current account with
1348
kamaki:
1349

    
1350
.. code-block:: console
1351

    
1352
  $ kamaki user authenticate
1353

    
1354
This will output user information.
1355

    
1356
Upload Image
1357
------------
1358

    
1359
By convention, images are stored in a container called ``images``. Check if the
1360
container exists, by listing all containers in your account:
1361

    
1362
.. code-block:: console
1363

    
1364
   $ kamaki file list
1365

    
1366
If the container ``images`` does not exist, create it:
1367

    
1368
.. code-block:: console
1369

    
1370
  $ kamaki file create images
1371

    
1372
You are now ready to upload an image to container ``images``. You can upload it
1373
with a Pithos client, or use kamaki directly:
1374

    
1375
.. code-block:: console
1376

    
1377
   $ kamaki file upload ubuntu.iso images
1378

    
1379
You can use any Pithos client to verify that the image was uploaded correctly,
1380
or you can list the contents of the container with kamaki:
1381

    
1382
.. code-block:: console
1383

    
1384
  $ kamaki file list images
1385

    
1386
The full Pithos URL for the previous example will be
1387
``pithos://u53r-un1qu3-1d/images/ubuntu.iso`` where ``u53r-un1qu3-1d`` is the
1388
unique user id (uuid).
1389

    
1390
Register Image
1391
--------------
1392

    
1393
To register an image you will need to use the full Pithos URL. To register as
1394
a public image the one from the previous example use:
1395

    
1396
.. code-block:: console
1397

    
1398
   $ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso --public
1399

    
1400
The ``--public`` flag is important, if missing the registered image will not
1401
be listed by ``kamaki image list``.
1402

    
1403
Use ``kamaki image register`` with no arguments to see a list of available
1404
options. A more complete example would be the following:
1405

    
1406
.. code-block:: console
1407

    
1408
   $ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso \
1409
            --public --disk-format diskdump --property kernel=3.1.2
1410

    
1411
To verify that the image was registered successfully use:
1412

    
1413
.. code-block:: console
1414

    
1415
   $ kamaki image list --name-like=ubuntu
1416

    
1417

    
1418
Miscellaneous
1419
=============
1420

    
1421
.. _branding:
1422

    
1423
Branding
1424
--------
1425

    
1426
Since Synnefo v0.14, you are able to adapt the Astakos, Pithos and Cyclades Web
1427
UI to your company’s visual identity. This is possible using the snf-branding
1428
component, which is automatically installed on the nodes running the API
1429
servers for Astakos, Pithos and Cyclades. 
1430

    
1431
Configuration
1432
~~~~~~~~~~~~~
1433

    
1434
This can be done by modifing the settings provided by the snf-branding component
1435
to match your service identity. The settings for the snf-branding application
1436
can be found inside the configuration file ``/etc/synnefo/15-snf-branding.conf``
1437
on the nodes that have Astakos, Pithos and Cyclades installed.
1438

    
1439
By default, the global service name is "Synnefo" and the company name is
1440
"GRNET". These names and their respective logos and URLs are used throughout
1441
the Astakos, Pithos and Cyclades UI.
1442

    
1443
**Names and URLs:**
1444

    
1445
The first group of branding customization refers to the service's and company's
1446
information.
1447

    
1448
You can overwrite the company and the service name and URL respectively by
1449
uncommenting and setting the following:
1450

    
1451
.. code-block:: python
1452
  
1453
  # setting used in Astakos Dashboard/Projects pages
1454
  BRANDING_SERVICE_NAME = 'My cloud'
1455
  BRANDING_SERVICE_URL = 'http://www.mycloud.synnefo.org/'
1456

    
1457
  # settings used in Astakos, Pithos, Cyclades footer only if 
1458
  # BRANDING_SHOW_COPYRIGHT is set to True
1459
  BRANDING_SHOW_COPYRIGHT = True
1460
  BRANDING_COMPANY_NAME = 'Company LTD'
1461
  BRANDING_COMPANY_URL = 'https://www.company-ltd.synnefo.org/'
1462

    
1463

    
1464
**Copyright and footer options:**
1465

    
1466
By default, no Copyright message is shown in the UI footer. If you want to make
1467
it visible in the footer of Astakos, Pithos and Cyclades UI, you can uncomment
1468
and set to ``True`` the ``BRANDING_SHOW_COPYRIGHT`` setting:
1469

    
1470
.. code-block:: python
1471

    
1472
  #BRANDING_SHOW_COPYRIGHT = False
1473

    
1474
Copyright message defaults to 'Copyright (c) 2011-<current_year>
1475
<BRANDING_COMPANY_NAME>.' but you can overwrite it to a completely custom one by
1476
setting the following option:
1477

    
1478
.. code-block:: python
1479

    
1480
  BRANDING_COPYRIGHT_MESSAGE = 'Copyright (c) 2011-2013 GRNET'
1481

    
1482
If you want to include a custom message in the footer, you can uncomment and 
1483
set the ``BRANDING_FOOTER_EXTRA_MESSAGE`` setting. You can use html markup. 
1484
Your custom message will appear  above Copyright message at the Compute 
1485
templates and the Dashboard UI.
1486

    
1487
.. code-block:: python
1488

    
1489
  #BRANDING_FOOTER_EXTRA_MESSAGE = ''
1490

    
1491

    
1492
**Images:**
1493

    
1494
The Astakos, Pithos and Cyclades Web UI has some logos and images.
1495
 
1496
The branding-related images are presented in  the following table:
1497

    
1498
===============  ============================  =========
1499
Image            Name/extension  convention    Usage
1500
===============  ============================  =========
1501
Favicon          favicon.ico                   Favicon for all services
1502
Dashboard logo   dashboard_logo.png            Visible in all Astakos UI pages
1503
Compute logo     compute_logo.png              Visible in all Cyclades UI pages
1504
Console logo     console_logo.png              Visible in the Cyclades Console Window
1505
Storage logo     storage_logo.png              Visible in all Pithos UI pages
1506
===============  ============================  =========
1507

    
1508
There are two methods  available for replacing all, or individual, 
1509
branding-related images:
1510

    
1511
1. Create a new directory inside ``/usr/share/synnefo/static/`` (e.g.
1512
   ``mybranding``) and place there some or all of your images.
1513

    
1514
   If you want to replace all of your images, keep the name/extension
1515
   conventions as indicated in the above table and change the
1516
   ``BRANDING_IMAGE_MEDIA_URL`` setting accordingly:
1517

    
1518
   .. code-block:: python
1519
        
1520
      # using relative path
1521
      BRANDING_IMAGE_MEDIA_URL= '/static/mybranding/images/' 
1522

    
1523
      # or if you already host them in a separate domain (e.g. cdn)
1524
      BRANDING_IMAGE_MEDIA_URL= 'https://cdn.synnefo.org/branding/images/'
1525

    
1526

    
1527
   If you wish to replace individual images, **do not uncomment**
1528
   ``BRANDING_IMAGE_MEDIA_URL``, but instead provide a relative path, pointing to
1529
   the file inside your directory for each ``BRANDING_<image>_URL`` that you wish
1530
   to replace.
1531

    
1532
2. Upload some or all of your images to a server and replace each 
1533
   ``BRANDING_<image>_URL`` with the absolute url of the image (i.e.
1534
   ``BRANDING_DASHBOARD_URL = 'https://www.synnefo.com/images/my_dashboard.jpg'``).
1535

    
1536
   Note that the alternative text  for each image tag inside html documents is 
1537
   alt=“BRANDING_SERVICE_NAME {Dashboard, Compute. Console, Storage}” respectively.
1538

    
1539
.. note:: Retina optimized images:
1540

    
1541
   Synnefo UI is optimized for Retina displays. As far as images are concerned,  
1542
   `retina.js <http://retinajs.com/>`_ is used.
1543

    
1544
   Retina.js checks each image on a page to see if there is a high-resolution 
1545
   version of that image on your server. If a high-resolution variant exists, 
1546
   the script will swap in that image in-place.
1547

    
1548
   The script assumes you use  `Apple's prescribed high-resolution modifier (@2x)
1549
   <http://developer.apple.com/library/ios/#documentation/2DDrawing/Conceptual/
1550
   DrawingPrintingiOS/SupportingHiResScreensInViews/SupportingHiResScreensInViews
1551
   .html#//apple_ref/doc/uid/TP40010156-CH15-SW1>`_ to denote high-resolution 
1552
   image variants on your server.
1553

    
1554
   For each of the images that you wish the script to  replace, you must have a 
1555
   high-resolution variant in the same folder  named correctly and it will be 
1556
   detected automatically. For example if your image is in <my_directory> and is 
1557
   named "my_image.jpg" the script will look in the same directory for an image 
1558
   named "my_image@2x.jpg".
1559

    
1560
   In case that you don’t want to use a high-resolution image, the 
1561
   normal-resolution image will be visible.
1562

    
1563
More branding
1564
~~~~~~~~~~~~~
1565

    
1566
Although, it is not 100% branding-related, further verbal customization is
1567
feasible. 
1568

    
1569
**EMAILS**
1570

    
1571
The output of all email `*`.txt files will be already customized to contain your
1572
company and service names but you can further alter their content if you feel it
1573
best fits your needs as simple as creasynnefo template.    
1574

    
1575
In order to overwrite one or more email-templates you need to place your 
1576
modified <email-file>.txt files respecting the following structure:
1577
  
1578
  **/etc/synnefo/templates/**
1579
      **im/**
1580
          | activation_email.txt
1581
          | email.txt
1582
          | invitation.txt
1583
          | switch_accounts_email.txt
1584
          | welcome_email.txt
1585
          **projects/**
1586
              | project_approval_notification.txt
1587
              | project_denial_notification.txt    
1588
              | project_membership_change_notification.txt
1589
              | project_membership_enroll_notification.txt
1590
              | project_membership_leave_request_notification.txt
1591
              | project_membership_request_notification.txt
1592
              | project_suspension_notification.txt
1593
              | project_termination_notification.txt
1594
      **registration/**
1595
          | email_change_email.txt
1596
          | password_email.txt
1597

    
1598
Feel free to omit any of the above files you do not wish to overwrite.
1599

    
1600
Below is a list of all emails sent by Synnefo to users along with a short 
1601
description and a link to their content:
1602

    
1603
* ``snf-astakos-app/astakos/im/templates/im/email.txt``
1604
  Base email template. Contains a contact email and a “thank you” message.
1605
  (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/email.txt>`_)
1606
* ``snf-astakos-app/astakos/im/templates/im/activation_email.txt`` Email sent to
1607
  user that prompts  him/her to click on a link provided to activate the account.
1608
  Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/activation_email.txt>`_)
1609
* ``snf-astakos-app/astakos/im/templates/im/invitation.txt`` Email sent to an
1610
  invited user. He/she has to click on a link provided to activate the account.
1611
  Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/invitation.txt>`_)
1612
* ``snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt`` Email
1613
  sent to user upon his/her request to associate this email address with a
1614
  shibboleth account. He/she has to click on a link provided to activate the
1615
  association. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt>`_)
1616
* ``snf-astakos-app/astakos/im/templates/im/welcome_email.txt`` Email sent to
1617
  inform the user that his/ her account has been activated. Extends “email.txt”
1618
  (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/welcome_email.txt>`_)
1619
* ``snf-astakos-app/astakos/im/templates/registration/email_change_email.txt``
1620
  Email sent to user when he/she has requested new email address assignment. The
1621
  user has to click on a link provided to validate this action. Extends
1622
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/email_change_email.txt>`_)
1623
* ``snf-astakos-app/astakos/im/templates/registration/password_email.txt`` Email
1624
  sent for resetting password purpose. The user has to click on a link provided
1625
  to validate this action. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/password_email.txt>`_)
1626
* ``snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt``
1627
  Informs  the project owner that his/her project has been approved. Extends
1628
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt>`_)
1629
* ``snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt``
1630
  Informs the project owner that his/her  project application has been denied
1631
  explaining the reasons. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt>`_)
1632
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt``
1633
  An email is sent to a user containing information about his project membership
1634
  (whether he has been accepted, rejected or removed). Extends “email.txt” (`Link
1635
  <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt>`_)
1636
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt``
1637
  Informs a user that he/she  has been enrolled to a project. Extends
1638
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt>`_)
1639
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt``
1640
  An email is sent to the project owner to make him aware of a  user having
1641
  requested to leave his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt>`_)
1642
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt``
1643
  An email is sent to the project owner to make him/her aware of a user having
1644
  requested to join  his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt>`_)
1645
* ``snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt``
1646
  An email is sent to the project owner to make him/her aware of his/her project
1647
  having been suspended. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt>`_)
1648
* ``snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt``
1649
  An email is sent to the project owner to make him/her aware of his/her project
1650
  having been terminated. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt>`_)
1651

    
1652
.. warning:: Django templates language:
1653

    
1654
  If you choose to  overwrite these email templates, be mindful of the necessary 
1655
  information contained in django template variables that must not be omitted, 
1656
  such as the activation link for activating one’s account and many more. 
1657
  These variables are contained into {{}} inside the templates.
1658

    
1659

    
1660
.. RabbitMQ
1661

    
1662
RabbitMQ Broker
1663
---------------
1664

    
1665
Queue nodes run the RabbitMQ sofware, which provides AMQP functionality. To
1666
guarantee high-availability, more than one Queue nodes should be deployed, each
1667
of them belonging to the same `RabbitMQ cluster
1668
<http://www.rabbitmq.com/clustering.html>`_. Synnefo uses the RabbitMQ
1669
active/active `High Available Queues <http://www.rabbitmq.com/ha.html>`_ which
1670
are mirrored between two nodes within a RabbitMQ cluster.
1671

    
1672
The RabbitMQ nodes that form the cluster, are declared to Synnefo through the
1673
`AMQP_HOSTS` setting. Each time a Synnefo component needs to connect to
1674
RabbitMQ, one of these nodes is chosen in a random way. The client that Synnefo
1675
uses to connect to RabbitMQ, handles connection failures transparently and
1676
tries to reconnect to a different node. As long as one of these nodes are up
1677
and running, functionality of Synnefo should not be downgraded by the RabbitMQ
1678
node failures.
1679

    
1680
All the queues that are being used are declared as durable, meaning that
1681
messages are persistently stored to RabbitMQ, until they get successfully
1682
processed by a client.
1683

    
1684
Currently, RabbitMQ is used by the following components:
1685

    
1686
* `snf-ganeti-eventd` and `snf-progress-monitor`:
1687
  These components send messages concerning the status and progress of
1688
  jobs in the Ganeti backend.
1689
* `snf-dispatcher`: This daemon, consumes the messages that are sent from
1690
  the above components, and updates the Cyclades DB accordingly.
1691

    
1692

    
1693
Installation
1694
~~~~~~~~~~~~
1695

    
1696
Please check the RabbitMQ documentation which covers extensively the
1697
`installation of RabbitMQ server <http://www.rabbitmq.com/download.html>`_ and
1698
the setup of a `RabbitMQ cluster <http://www.rabbitmq.com/clustering.html>`_.
1699
Also, check out the `web management plugin
1700
<http://www.rabbitmq.com/management.html>`_ that can be useful for managing and
1701
monitoring RabbitMQ.
1702

    
1703
For a basic installation of RabbitMQ on two nodes (node1 and node2) you can do
1704
the following:
1705

    
1706
On both nodes, install rabbitmq-server and create a Synnefo user:
1707

    
1708
.. code-block:: console
1709

    
1710
  $ apt-get install rabbitmq-server
1711
  $ rabbitmqctl add_user synnefo "example_pass"
1712
  $ rabbitmqctl set_permissions synnefo  ".*" ".*" ".*"
1713

    
1714
Also guarantee that both nodes share the same cookie, by running:
1715

    
1716
.. code-block:: console
1717

    
1718
  $ scp node1:/var/lib/rabbitmq/.erlang.cookie node2:/var/lib/rabbitmq/.erlang.cookie
1719

    
1720
and restart the nodes:
1721

    
1722
.. code-block:: console
1723

    
1724
  $ /etc/init.d/rabbitmq-server restart
1725

    
1726

    
1727
To setup the RabbitMQ cluster run:
1728

    
1729
.. code-block:: console
1730

    
1731
  root@node2: rabbitmqctl stop_app
1732
  root@node2: rabbitmqctl reset
1733
  root@node2: rabbitmqctl cluster rabbit@node1 rabbit@node2
1734
  root@node2: rabbitmqctl start_app
1735

    
1736
You can verify that the cluster is set up correctly by running:
1737

    
1738
.. code-block:: console
1739

    
1740
  root@node2: rabbitmqctl cluster_status
1741

    
1742

    
1743
Logging
1744
-------
1745

    
1746
Logging in Synnefo is using Python's logging module. The module is configured
1747
using dictionary configuration, whose format is described here:
1748

    
1749
http://docs.python.org/release/2.7.1/library/logging.html#logging-config-dictschema
1750

    
1751
The logging configuration dictionary is defined in
1752
``/etc/synnefo/10-snf-webproject-logging.conf``
1753

    
1754
The administrator can have logging control by modifying the ``LOGGING_SETUP``
1755
dictionary, and defining subloggers with different handlers and log levels.
1756

    
1757

    
1758
.. _scale-up:
1759

    
1760
Scaling up to multiple nodes
1761
============================
1762

    
1763
Here we will describe how should a large scale Synnefo deployment look like. Make
1764
sure you are familiar with Synnefo and Ganeti before proceeding with this section.
1765
This means you should at least have already set up successfully a working Synnefo
1766
deployment as described in the :ref:`Admin's Installation Guide
1767
<quick-install-admin-guide>` and also read the Administrator's Guide until this
1768
section.
1769

    
1770
Graph of a scale-out Synnefo deployment
1771
---------------------------------------
1772

    
1773
Each box in the following graph corresponds to a distinct physical node:
1774

    
1775
.. image:: images/synnefo-arch2-roles.png
1776
   :width: 100%
1777
   :target: _images/synnefo-arch2-roles.png
1778

    
1779
The above graph is actually the same with the one at the beginning of this
1780
:ref:`guide <admin-guide>`, with the only difference that here we show the
1781
Synnefo roles of each physical node. These roles are described in the
1782
following section.
1783

    
1784
.. _physical-node-roles:
1785

    
1786
Physical Node roles
1787
-------------------
1788

    
1789
As appears in the previous graph, a scale-out Synnefo deployment consists of
1790
multiple physical nodes that have the following roles:
1791

    
1792
* **WEBSERVER**: A web server running in front of gunicorn (e.g.: Apache, nginx)
1793
* **ASTAKOS**: The Astakos application (gunicorn)
1794
* **ASTAKOS_DB**: The Astakos database (postgresql)
1795
* **PITHOS**: The Pithos application (gunicorn)
1796
* **PITHOS_DB**: The Pithos database (postgresql)
1797
* **CYCLADES**: The Cyclades application (gunicorn)
1798
* **CYCLADES_DB**: The Cyclades database (postgresql)
1799
* **MQ**: The message queue (RabbitMQ)
1800
* **GANETI_MASTER**: The Ganeti master of a Ganeti cluster
1801
* **GANETI_NODE** : A VM-capable Ganeti node of a Ganeti cluster
1802

    
1803
You will probably also have:
1804

    
1805
* **CMS**: The CMS used as a frotend portal for the Synnefo services
1806
* **NS**: A nameserver serving all other Synnefo nodes and resolving Synnefo FQDNs
1807
* **CLIENT**: A machine that runs the Synnefo clients (e.g.: kamaki, Web UI),
1808
              most of the times, the end user's local machine
1809

    
1810
From this point we will also refer to the following groups of roles:
1811

    
1812
* **SYNNEFO**: [ **ASTAKOS**, **ASTAKOS_DB**, **PITHOS**, **PITHOS_DB**, **CYCLADES**, **CYCLADES_DB**, **MQ**, **CMS**]
1813
* **G_BACKEND**: [**GANETI_MASTER**, **GANETI_NODE**]
1814

    
1815
Of course, when deploying Synnefo you can combine multiple of the above roles on a
1816
single physical node, but if you are trying to scale out, the above separation
1817
gives you significant advantages.
1818

    
1819
So, in the next section we will take a look on what components you will have to
1820
install on each physical node depending on its Synnefo role. We assume the graph's
1821
architecture.
1822

    
1823
Components for each role
1824
------------------------
1825

    
1826
When deploying Synnefo in large scale, you need to install different Synnefo
1827
or/and third party components on different physical nodes according to their
1828
Synnefo role, as stated in the previous section.
1829

    
1830
Specifically:
1831

    
1832
Role **WEBSERVER**
1833
    * Synnefo components: `None`
1834
    * 3rd party components: Apache
1835
Role **ASTAKOS**
1836
    * Synnefo components: `snf-webproject`, `snf-astakos-app`
1837
    * 3rd party components: Django, Gunicorn
1838
Role **ASTAKOS_DB**
1839
    * Synnefo components: `None`
1840
    * 3rd party components: PostgreSQL
1841
Role **PITHOS**
1842
    * Synnefo components: `snf-webproject`, `snf-pithos-app`, `snf-pithos-webclient`
1843
    * 3rd party components: Django, Gunicorn
1844
Role **PITHOS_DB**
1845
    * Synnefo components: `None`
1846
    * 3rd party components: PostgreSQL
1847
Role **CYCLADES**
1848
    * Synnefo components: `snf-webproject`, `snf-cyclades-app`, `snf-vncauthproxy`
1849
    * 3rd party components: Django Gunicorn
1850
Role **CYCLADES_DB**
1851
    * Synnefo components: `None`
1852
    * 3rd party components: PostgreSQL
1853
Role **MQ**
1854
    * Synnefo components: `None`
1855
    * 3rd party components: RabbitMQ
1856
Role **GANETI_MASTER**
1857
    * Synnefo components: `snf-cyclades-gtools`
1858
    * 3rd party components: Ganeti
1859
Role **GANETI_NODE**
1860
    * Synnefo components: `snf-cyclades-gtools`, `snf-network`, `snf-image`, `nfdhcpd`
1861
    * 3rd party components: Ganeti
1862
Role **CMS**
1863
    * Synnefo components: `snf-webproject`, `snf-cloudcms`
1864
    * 3rd party components: Django, Gunicorn
1865
Role **NS**
1866
    * Synnefo components: `None`
1867
    * 3rd party components: BIND
1868
Role **CLIENT**
1869
    * Synnefo components: `kamaki`, `snf-image-creator`
1870
    * 3rd party components: `None`
1871

    
1872
Example scale out installation
1873
------------------------------
1874

    
1875
In this section we describe an example of a medium scale installation which
1876
combines multiple roles on 10 different physical nodes. We also provide a
1877
:ref:`guide <i-synnefo>` to help with such an install.
1878

    
1879
We assume that we have the following 10 physical nodes with the corresponding
1880
roles:
1881

    
1882
Node1:
1883
    **WEBSERVER**, **ASTAKOS**
1884
      Guide sections:
1885
        * :ref:`apt <i-apt>`
1886
        * :ref:`gunicorn <i-gunicorn>`
1887
        * :ref:`apache <i-apache>`
1888
        * :ref:`snf-webproject <i-webproject>`
1889
        * :ref:`snf-astakos-app <i-astakos>`
1890
Node2:
1891
    **WEBSERVER**, **PITHOS**
1892
      Guide sections:
1893
        * :ref:`apt <i-apt>`
1894
        * :ref:`gunicorn <i-gunicorn>`
1895
        * :ref:`apache <i-apache>`
1896
        * :ref:`snf-webproject <i-webproject>`
1897
        * :ref:`snf-pithos-app <i-pithos>`
1898
        * :ref:`snf-pithos-webclient <i-pithos>`
1899
Node3:
1900
    **WEBSERVER**, **CYCLADES**
1901
      Guide sections:
1902
        * :ref:`apt <i-apt>`
1903
        * :ref:`gunicorn <i-gunicorn>`
1904
        * :ref:`apache <i-apache>`
1905
        * :ref:`snf-webproject <i-webproject>`
1906
        * :ref:`snf-cyclades-app <i-cyclades>`
1907
        * :ref:`snf-vncauthproxy <i-cyclades>`
1908
Node4:
1909
    **WEBSERVER**, **CMS**
1910
      Guide sections:
1911
        * :ref:`apt <i-apt>`
1912
        * :ref:`gunicorn <i-gunicorn>`
1913
        * :ref:`apache <i-apache>`
1914
        * :ref:`snf-webproject <i-webproject>`
1915
        * :ref:`snf-cloudcms <i-cms>`
1916
Node5:
1917
    **ASTAKOS_DB**, **PITHOS_DB**, **CYCLADES_DB**
1918
      Guide sections:
1919
        * :ref:`apt <i-apt>`
1920
        * :ref:`postgresql <i-db>`
1921
Node6:
1922
    **MQ**
1923
      Guide sections:
1924
        * :ref:`apt <i-apt>`
1925
        * :ref:`rabbitmq <i-mq>`
1926
Node7:
1927
    **GANETI_MASTER**, **GANETI_NODE**
1928
      Guide sections:
1929
        * :ref:`apt <i-apt>`
1930
        * :ref:`general <i-backends>`
1931
        * :ref:`ganeti <i-ganeti>`
1932
        * :ref:`snf-cyclades-gtools <i-gtools>`
1933
        * :ref:`snf-network <i-network>`
1934
        * :ref:`snf-image <i-image>`
1935
        * :ref:`nfdhcpd <i-network>`
1936
Node8:
1937
    **GANETI_NODE**
1938
      Guide sections:
1939
        * :ref:`apt <i-apt>`
1940
        * :ref:`general <i-backends>`
1941
        * :ref:`ganeti <i-ganeti>`
1942
        * :ref:`snf-cyclades-gtools <i-gtools>`
1943
        * :ref:`snf-network <i-network>`
1944
        * :ref:`snf-image <i-image>`
1945
        * :ref:`nfdhcpd <i-network>`
1946
Node9:
1947
    **GANETI_NODE**
1948
      Guide sections:
1949
        `Same as Node8`
1950
Node10:
1951
    **GANETI_NODE**
1952
      Guide sections:
1953
        `Same as Node8`
1954

    
1955
All sections: :ref:`Scale out Guide <i-synnefo>`
1956

    
1957

    
1958
Upgrade Notes
1959
=============
1960

    
1961
.. toctree::
1962
   :maxdepth: 1
1963

    
1964
   v0.12 -> v0.13 <upgrade/upgrade-0.13>
1965
   v0.13 -> v0.14 <upgrade/upgrade-0.14>
1966
   v0.14 -> v0.14.2 <upgrade/upgrade-0.14.2>
1967
   v0.14.5 -> v0.14.6 <upgrade/upgrade-0.14.6>
1968
   v0.14.7 -> v0.14.8 <upgrade/upgrade-0.14.8>
1969
   v0.14.9 -> v0.14.10 <upgrade/upgrade-0.14.10>
1970
   v0.14 -> v0.15 <upgrade/upgrade-0.15>
1971

    
1972

    
1973
Changelog, NEWS
1974
===============
1975

    
1976

    
1977
* v0.14.10 :ref:`Changelog <Changelog-0.14.10>`, :ref:`NEWS <NEWS-0.14.10>`
1978
* v0.14.9 :ref:`Changelog <Changelog-0.14.9>`, :ref:`NEWS <NEWS-0.14.9>`
1979
* v0.14.8 :ref:`Changelog <Changelog-0.14.8>`, :ref:`NEWS <NEWS-0.14.8>`
1980
* v0.14.7 :ref:`Changelog <Changelog-0.14.7>`, :ref:`NEWS <NEWS-0.14.7>`
1981
* v0.14.6 :ref:`Changelog <Changelog-0.14.6>`, :ref:`NEWS <NEWS-0.14.6>`
1982
* v0.14.5 :ref:`Changelog <Changelog-0.14.5>`, :ref:`NEWS <NEWS-0.14.5>`
1983
* v0.14.4 :ref:`Changelog <Changelog-0.14.4>`, :ref:`NEWS <NEWS-0.14.4>`
1984
* v0.14.3 :ref:`Changelog <Changelog-0.14.3>`, :ref:`NEWS <NEWS-0.14.3>`
1985
* v0.14.2 :ref:`Changelog <Changelog-0.14.2>`, :ref:`NEWS <NEWS-0.14.2>`
1986
* v0.14 :ref:`Changelog <Changelog-0.14>`, :ref:`NEWS <NEWS-0.14>`
1987
* v0.13 :ref:`Changelog <Changelog-0.13>`, :ref:`NEWS <NEWS-0.13>`