Statistics
| Branch: | Tag: | Revision:

root / docs / admin-guide.rst @ 033321fb

History | View | Annotate | Download (66.1 kB)

1
.. _admin-guide:
2

    
3
Synnefo Administrator's Guide
4
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5

    
6
This is the complete Synnefo Administrator's Guide.
7

    
8

    
9

    
10
General Synnefo Architecture
11
============================
12

    
13
The following graph shows the whole Synnefo architecture and how it interacts
14
with multiple Ganeti clusters. We hope that after reading the Administrator's
15
Guide you will be able to understand every component and all the interactions
16
between them. It is a good idea to first go through the Administrator's Guide
17
before proceeding.
18

    
19
.. image:: images/synnefo-arch2.png
20
   :width: 100%
21
   :target: _images/synnefo-arch2.png
22

    
23

    
24

    
25
Identity Service (Astakos)
26
==========================
27

    
28

    
29
Overview
30
--------
31

    
32
Authentication methods
33
~~~~~~~~~~~~~~~~~~~~~~
34

    
35
Local Authentication
36
````````````````````
37

    
38
LDAP Authentication
39
```````````````````
40

    
41
.. _shibboleth-auth:
42

    
43
Shibboleth Authentication
44
`````````````````````````
45

    
46
Astakos can delegate user authentication to a Shibboleth federation.
47

    
48
To setup shibboleth, install package::
49

    
50
  apt-get install libapache2-mod-shib2
51

    
52
Change appropriately the configuration files in ``/etc/shibboleth``.
53

    
54
Add in ``/etc/apache2/sites-available/synnefo-ssl``::
55

    
56
  ShibConfig /etc/shibboleth/shibboleth2.xml
57
  Alias      /shibboleth-sp /usr/share/shibboleth
58

    
59
  <Location /ui/login/shibboleth>
60
    AuthType shibboleth
61
    ShibRequireSession On
62
    ShibUseHeaders On
63
    require valid-user
64
  </Location>
65

    
66
and before the line containing::
67

    
68
  ProxyPass        / http://localhost:8080/ retry=0
69

    
70
add::
71

    
72
  ProxyPass /Shibboleth.sso !
73

    
74
Then, enable the shibboleth module::
75

    
76
  a2enmod shib2
77

    
78
After passing through the apache module, the following tokens should be
79
available at the destination::
80

    
81
  eppn # eduPersonPrincipalName
82
  Shib-InetOrgPerson-givenName
83
  Shib-Person-surname
84
  Shib-Person-commonName
85
  Shib-InetOrgPerson-displayName
86
  Shib-EP-Affiliation
87
  Shib-Session-ID
88

    
89
Finally, add 'shibboleth' in ``ASTAKOS_IM_MODULES`` list. The variable resides
90
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
91

    
92
Twitter Authentication
93
```````````````````````
94

    
95
To enable twitter authentication while signed in under a Twitter account,
96
visit dev.twitter.com/apps.
97

    
98
Click Create an application.
99

    
100
Fill the necessary information and for callback URL give::
101

    
102
    https://node1.example.com/ui/login/twitter/authenticated
103

    
104
Finally, add 'twitter' in ``ASTAKOS_IM_MODULES`` list. The variable resides
105
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
106

    
107

    
108
Google Authentication
109
`````````````````````
110

    
111
To enable google authentication while signed in under a Google account,
112
visit https://code.google.com/apis/console/.
113

    
114
Under API Access select Create another client ID, select Web application,
115
expand more options in Your site or hostname section and in Authorized
116
Redirect URIs add:
117

    
118

    
119
Fill the necessary information and for callback URL give::
120

    
121
    https://node1.example.com/ui/login/google/authenticated
122

    
123
Finally, add 'google' in ``ASTAKOS_IM_MODULES`` list. The variable resides
124
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
125

    
126
Architecture
127
------------
128

    
129
Prereqs
130
-------
131

    
132
Installation
133
------------
134

    
135
Configuration
136
-------------
137

    
138
Working with Astakos
139
--------------------
140

    
141
User registration
142
~~~~~~~~~~~~~~~~~
143

    
144
When a new user signs up, he/she is not directly marked as active. You can see 
145
his/her state by running (on the machine that runs the Astakos app):
146

    
147
.. code-block:: console
148

    
149
   $ snf-manage user-list
150

    
151
More detailed user status is provided in the `status` field of the `user-show` 
152
command:
153

    
154
.. code-block:: console
155

    
156
  $ snf-manage user-show <user-id>
157

    
158
  id                  : 6
159
  uuid                : 78661411-5eed-412f-a9ea-2de24f542c2e
160
  status              : Accepted/Active (accepted policy: manual)
161
  email               : user@synnefo.org
162
  ....
163

    
164

    
165
Based on how your configuration of `astakos-app`, there are several ways for a 
166
user to get activated and be able to login. We discuss the user activation 
167
flow in the following section.
168

    
169

    
170
User activation flow
171
````````````````````
172

    
173
A user can register for an account using the astakos signup form. Once the form
174
is submited successfully a user entry is created in astakos database. That entry
175
is passed through the astakos activation backend which handles whether the user
176
should be automatically verified and activated.
177

    
178

    
179
Email verification
180
``````````````````
181

    
182
The verification process takes place in order to ensure that the user owns the
183
email provided during the signup process. By default, after each successful
184
signup astakos notifies user with an verification url via email. 
185

    
186
At this stage:
187

    
188
    * subsequent registrations invalidate and delete the previous registrations 
189
      of the same email address.
190

    
191
    * in case user misses the initial notification, additional emails can be
192
      send either via the url which is prompted to the user if he tries to
193
      login, or by the administrator using the ``snf-manage user-activation-send
194
      <userid>`` command.
195

    
196
    * administrator may also enforce a user to get verified using the
197
      ``snf-manage user-modify --verify <userid>`` command.
198

    
199

    
200
Account activation
201
``````````````````
202

    
203
Once user gets verified it is time for astakos to decide whether or not to
204
proceed through user activation process. If ``ASTAKOS_MODERATION_ENABLED``
205
setting is set to ``False`` (default value) user gets activated automatically. 
206

    
207
In case the moderation is enabled astakos may still automatically activate the
208
user in the following cases:
209

    
210
    * User email matches any of the regular expressions defined in
211
      ``ASTAKOS_RE_USER_EMAIL_PATTERNS`` (defaults to ``[]``)
212
    * User used a signup method (e.g. ``shibboleth``) for which automatic
213
      activation is enabled (see 
214
      :ref:`authentication methods policies <auth_methods_policies>`).
215

    
216
If all of the above fail to trigger automatic activation, an email is sent 
217
to the persons listed in ``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings, 
218
notifing that there is a new user pending for moderation and that it's 
219
up to the administrator to decide if the user should be activated, using the 
220
``user-modify`` command.
221

    
222
.. code-block:: console
223

    
224
    # command to activate a pending user
225
    $ snf-manage user-modify --accept <userid>
226

    
227
    # command to reject a pending user
228
    $ snf-manage user-modify --reject --reject-reason="spammer" <userid>
229

    
230
Once activation process finish, a greeting message is sent to the user email
231
address and a notification for the activation to the persons listed in 
232
``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings. Once activated the user is 
233
able to login and access the synnefo services.
234

    
235

    
236
Additional authentication methods
237
`````````````````````````````````
238

    
239
Astakos supports third party logins from external identity providers. This
240
can be usefull since it allows users to use their existing credentials to 
241
login to astakos service.
242

    
243
Currently astakos supports the following identity providers:
244

    
245
    * `Shibboleth <http://www.internet2.edu/shibboleth>`_ (module name
246
      ``shibboleth``)
247
    * `Google <https://developers.google.com/accounts/docs/OAuth2>`_ (module
248
      name ``google``)
249
    * `Twitter <https://dev.twitter.com/docs/auth>`_ (module name ``twitter``)
250
    * `LinkedIn <http://developer.linkedin.com/documents/authentication>`_
251
      (module name ``linkedin``)
252

    
253
To enable any of the above modules (by default only ``local`` accounts are
254
allowed), retrieve and set the required provider settings and append the 
255
module name in ``ASTAKOS_IM_MODULES``.
256

    
257
.. code-block:: python
258

    
259
    # settings from https://code.google.com/apis/console/
260
    ASTAKOS_GOOGLE_CLIENT_ID = '1111111111-epi60tvimgha63qqnjo40cljkojcann3.apps.googleusercontent.com'
261
    ASTAKOS_GOOGLE_SECRET = 'tNDQqTDKlTf7_LaeUcWTWwZM'
262
    
263
    # let users signup and login using their google account
264
    ASTAKOS_IM_MODULES = ['local', 'google']
265

    
266

    
267
.. _auth_methods_policies:
268

    
269
Authentication method policies
270
``````````````````````````````
271

    
272
Astakos allows you to override the default policies for each enabled provider 
273
separately by adding the approriate settings in your ``.conf`` files in the 
274
following format:
275

    
276
**ASTAKOS_AUTH_PROVIDER_<module>_<policy>_POLICY**
277

    
278
Available policies are:
279

    
280
    * **CREATE** Users can signup using that provider (default: ``True``) 
281
    * **REMOVE/ADD** Users can remove/add login method from their profile 
282
      (default: ``True``)
283
    * **AUTOMODERATE** Automatically activate users that signup using that
284
      provider (default: ``False``)
285
    * **LOGIN** Whether or not users can use the provider to login (default:
286
      ``True``).
287

    
288
e.g. to enable automatic activation for your academic users, while keeping 
289
locally signed up users under moderation you can apply the following settings.
290

    
291
.. code-block:: python
292

    
293
    ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_AUTOMODERATE_POLICY = True
294
    ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_REMOVE_POLICY = False
295

    
296

    
297
Setting quota limits
298
~~~~~~~~~~~~~~~~~~~~
299

    
300
Set default quota
301
`````````````````
302

    
303
In 20-snf-astakos-app-settings.conf, 
304
uncomment the default setting ``ASTAKOS_SERVICES``
305
and customize the ``'uplimit'`` values.
306
These are the default base quota for all users.
307

    
308
To apply your configuration run::
309

    
310
    # snf-manage astakos-init --load-service-resources
311
    # snf-manage quota --sync
312

    
313
Set base quota for individual users
314
```````````````````````````````````
315

    
316
For individual users that need different quota than the default
317
you can set it for each resource like this::
318

    
319
    # use this to display quota / uuid
320
    # snf-manage user-show 'uuid or email' --quota
321

    
322
    # snf-manage user-modify 'user-uuid' --set-base-quota 'cyclades.vm' 10
323

    
324

    
325
Enable the Projects feature
326
~~~~~~~~~~~~~~~~~~~~~~~~~~~
327

    
328
If you want to enable the projects feature so that users may apply
329
on their own for resources by creating and joining projects,
330
in ``20-snf-astakos-app-settings.conf`` set::
331

    
332
    # this will make the 'projects' page visible in the dashboard
333
    ASTAKOS_PROJECTS_VISIBLE = True
334

    
335
You can change the maximum allowed number of pending project applications
336
per user with::
337

    
338
    # snf-manage resource-modify astakos.pending_app --limit <number>
339

    
340
You can also set a user-specific limit with::
341

    
342
    # snf-manage user-modify 'user-uuid' --set-base-quota 'astakos.pending_app' 5
343

    
344
When users apply for projects they are not automatically granted
345
the resources. They must first be approved by the administrator.
346

    
347
To list pending project applications in astakos::
348

    
349
    # snf-manage project-list --pending
350

    
351
Note the last column, the application id. To approve it::
352

    
353
    # <app id> from the last column of project-list
354
    # snf-manage project-control --approve <app id>
355

    
356
To deny an application::
357

    
358
    # snf-manage project-control --deny <app id>
359

    
360
Users designated as *project admins* can approve, deny, or modify
361
an application through the web interface. In
362
``20-snf-astakos-app-settings.conf`` set::
363

    
364
    # UUIDs of users that can approve or deny project applications from the web.
365
    ASTAKOS_PROJECT_ADMINS = [<uuid>, ...]
366

    
367

    
368
Astakos advanced operations
369
---------------------------
370

    
371
Adding "Terms of Use"
372
~~~~~~~~~~~~~~~~~~~~~
373

    
374
Astakos supports versioned terms-of-use. First of all you need to create an
375
html file that will contain your terms. For example, create the file
376
``/usr/share/synnefo/sample-terms.html``, which contains the following:
377

    
378
.. code-block:: console
379

    
380
   <h1>~okeanos terms</h1>
381

    
382
   These are the example terms for ~okeanos
383

    
384
Then, add those terms-of-use with the snf-manage command:
385

    
386
.. code-block:: console
387

    
388
   $ snf-manage term-add /usr/share/synnefo/sample-terms.html
389

    
390
Your terms have been successfully added and you will see the corresponding link
391
appearing in the Astakos web pages' footer.
392

    
393
Enabling reCAPTCHA
394
~~~~~~~~~~~~~~~~~~
395

    
396
Astakos supports the `reCAPTCHA <http://www.google.com/recaptcha>`_ feature.
397
If enabled, it protects the Astakos forms from bots. To enable the feature, go
398
to https://www.google.com/recaptcha/admin/create and create your own reCAPTCHA
399
key pair. Then edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` and set
400
the corresponding variables to reflect your newly created key pair. Finally, set
401
the ``ASTAKOS_RECAPTCHA_ENABLED`` variable to ``True``:
402

    
403
.. code-block:: console
404

    
405
   ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*('
406
   ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*('
407

    
408
   ASTAKOS_RECAPTCHA_ENABLED = True
409

    
410
Restart the service on the Astakos node(s) and you are ready:
411

    
412
.. code-block:: console
413

    
414
   # /etc/init.d/gunicorn restart
415

    
416
Checkout your new Sign up page. If you see the reCAPTCHA box, you have setup
417
everything correctly.
418

    
419

    
420

    
421
File Storage Service (Pithos)
422
=============================
423

    
424
Overview
425
--------
426

    
427
Architecture
428
------------
429

    
430
Prereqs
431
-------
432

    
433
Installation
434
------------
435

    
436
Configuration
437
-------------
438

    
439
Working with Pithos
440
-------------------
441

    
442
Pithos advanced operations
443
--------------------------
444

    
445

    
446

    
447
Compute/Network/Image Service (Cyclades)
448
========================================
449

    
450
Compute Overview
451
----------------
452

    
453
Network Overview
454
----------------
455

    
456
Image Overview
457
--------------
458

    
459
Architecture
460
------------
461

    
462
Asynchronous communication with Ganeti backends
463
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
464
Synnefo uses Google Ganeti backends for VM cluster management. In order for
465
Cyclades to be able to handle thousands of user requests, Cyclades and Ganeti
466
communicate asynchronously. Briefly, requests are submitted to Ganeti through
467
Ganeti's RAPI/HTTP interface, and then asynchronous notifications about the
468
progress of Ganeti jobs are being created and pushed upwards to Cyclades. The
469
architecture and communication with a Ganeti backend is shown in the graph
470
below:
471

    
472
.. image:: images/cyclades-ganeti-communication.png
473
   :width: 50%
474
   :target: _images/cyclades-ganeti-communication.png
475

    
476
The Cyclades API server is responsible for handling user requests. Read-only
477
requests are directly served by looking up the Cyclades DB. If the request
478
needs an action in the Ganeti backend, Cyclades submit jobs to the Ganeti
479
master using the `Ganeti RAPI interface
480
<http://docs.ganeti.org/ganeti/2.2/html/rapi.html>`_.
481

    
482
While Ganeti executes the job, `snf-ganeti-eventd`, `snf-ganeti-hook` and
483
`snf-progress-monitor` are monitoring the progress of the job and send
484
corresponding messages to the RabbitMQ servers. These components are part
485
of `snf-cyclades-gtools` and must be installed on all Ganeti nodes. Specially:
486

    
487
* *snf-ganeti-eventd* sends messages about operations affecting the operating
488
  state of instances and networks. Works by monitoring the Ganeti job queue.
489
* *snf-ganeti_hook* sends messages about the NICs of instances. It includes a
490
  number of `Ganeti hooks <http://docs.ganeti.org/ganeti/2.2/html/hooks.html>`_
491
  for customisation of operations.
492
* *snf-progress_monitor* sends messages about the progress of the Image deployment
493
  phase which is done by the Ganeti OS Definition `snf-image`.
494

    
495
Finally, `snf-dispatcher` consumes messages from the RabbitMQ queues, processes
496
these messages and properly updates the state of the Cyclades DB. Subsequent
497
requests to the Cyclades API, will retrieve the updated state from the DB.
498

    
499

    
500
Prereqs
501
-------
502

    
503
Work in progress. Please refer to :ref:`administrator's install quide <quick-install-admin-guide>`.
504

    
505
Installation
506
------------
507

    
508
Work in progress. Please refer to :ref:`administrator's install quide <quick-install-admin-guide>`.
509

    
510
Configuration
511
-------------
512

    
513
Work in progress. Please refer to :ref:`administrator's install quide <quick-install-admin-guide>`.
514

    
515
Working with Cyclades
516
---------------------
517

    
518
Managing Ganeti Backends
519
~~~~~~~~~~~~~~~~~~~~~~~~
520

    
521
Since v0.11, Synnefo is able to manage multiple Ganeti clusters (backends)
522
making it capable to scale linearly to tens of thousands of VMs. Backends
523
can be dynamically added or removed via `snf-manage` commands.
524

    
525
Each newly created VM is allocated to a Ganeti backend by the Cyclades backend
526
allocator. The VM is "pinned" to this backend, and can not change through its
527
lifetime. The backend allocator decides in which backend to spawn the VM based
528
on the available resources of each backend, trying to balance the load between
529
them.
530

    
531
Handling of Networks, as far as backends are concerned, is based on whether the
532
network is public or not. Public networks are created through the `snf-manage
533
network-create` command, and are only created on one backend. Private networks
534
are created on all backends, in order to ensure that VMs residing on different
535
backends can be connected to the same private network.
536

    
537
Listing existing backends
538
`````````````````````````
539
To list all the Ganeti backends known to Synnefo, we run:
540

    
541
.. code-block:: console
542

    
543
   $ snf-manage backend-list
544

    
545
Adding a new Ganeti backend
546
```````````````````````````
547
Backends are dynamically added under the control of Synnefo with `snf-manage
548
backend-add` command. In this section it is assumed that a Ganeti cluster,
549
named ``cluster.example.com`` is already up and running and configured to be
550
able to host Synnefo VMs.
551

    
552
To add this Ganeti cluster, we run:
553

    
554
.. code-block:: console
555

    
556
   $ snf-manage backend-add --clustername=cluster.example.com --user="synnefo_user" --pass="synnefo_pass"
557

    
558
where ``clustername`` is the Cluster hostname of the Ganeti cluster, and
559
``user`` and ``pass`` are the credentials for the `Ganeti RAPI user
560
<http://docs.ganeti.org/ganeti/2.2/html/rapi.html#users-and-passwords>`_.  All
561
backend attributes can be also changed dynamically using the `snf-manage
562
backend-modify` command.
563

    
564
``snf-manage backend-add`` will also create all existing private networks to
565
the new backend. You can verify that the backend is added, by running
566
`snf-manage backend-list`.
567

    
568
Note that no VMs will be spawned to this backend, since by default it is in a
569
``drained`` state after addition and also it has no public network assigned to
570
it.
571

    
572
So, first you need to create its public network, make sure everything works as
573
expected and finally make it active by un-setting the ``drained`` flag. You can
574
do this by running:
575

    
576
.. code-block:: console
577

    
578
   $ snf-manage backend-modify --drained=False <backend_id>
579

    
580
Removing an existing Ganeti backend
581
```````````````````````````````````
582
In order to remove an existing backend from Synnefo, we run:
583

    
584
.. code-block:: console
585

    
586
   # snf-manage backend-remove <backend_id>
587

    
588
This command will fail if there are active VMs on the backend. Also, the
589
backend is not cleaned before removal, so all the Synnefo private networks
590
will be left on the Ganeti nodes. You need to remove them manually.
591

    
592
Allocation of VMs in Ganeti backends
593
````````````````````````````````````
594
As already mentioned, the Cyclades backend allocator is responsible for
595
allocating new VMs to backends. This allocator does not choose the exact Ganeti
596
node that will host the VM but just the Ganeti backend. The exact node is
597
chosen by the Ganeti cluster's allocator (hail).
598

    
599
The decision about which backend will host a VM is based on the available
600
resources. The allocator computes a score for each backend, that shows its load
601
factor, and the one with the minimum score is chosen. The admin can exclude
602
backends from the allocation phase by marking them as ``drained`` by running:
603

    
604
.. code-block:: console
605

    
606
   $ snf-manage backend-modify --drained=True <backend_id>
607

    
608
The backend resources are periodically updated, at a period defined by
609
the ``BACKEND_REFRESH_MIN`` setting, or by running `snf-manage backend-update-status`
610
command. It is advised to have a cron job running this command at a smaller
611
interval than ``BACKEND_REFRESH_MIN`` in order to remove the load of refreshing
612
the backends stats from the VM creation phase.
613

    
614
Finally, the admin can decide to have a user's VMs being allocated to a
615
specific backend, with the ``BACKEND_PER_USER`` setting. This is a mapping
616
between users and backends. If the user is found in ``BACKEND_PER_USER``, then
617
Synnefo allocates all his/hers VMs to the specific backend in the variable,
618
even if is marked as drained (useful for testing).
619

    
620

    
621
Managing Virtual Machines
622
~~~~~~~~~~~~~~~~~~~~~~~~~
623

    
624
As mentioned, Cyclades uses Ganeti for management of VMs. The administrator can
625
handle Cyclades VMs just like any other Ganeti instance, via `gnt-instance`
626
commands. All Ganeti instances that belong to Synnefo, are separated from
627
others, by a prefix in their names. This prefix is defined in
628
``BACKEND_PREFIX_ID`` setting in
629
``/etc/synnefo/20-snf-cyclades-app-backend.conf``.
630

    
631
Apart from handling instances directly in the Ganeti level, a number of `snf-manage`
632
commands are available:
633

    
634
* ``snf-manage server-list``: List servers
635
* ``snf-manage server-show``: Show information about a server in the Cyclades DB
636
* ``snf-manage server-inspect``: Inspect the state of a server both in DB and Ganeti
637
* ``snf-manage server-modify``: Modify the state of a server in the Cycldes DB
638
* ``snf-manage server-create``: Create a new server
639
* ``snf-manage server-import``: Import an existing Ganeti instance to Cyclades
640

    
641

    
642
Managing Virtual Networks
643
~~~~~~~~~~~~~~~~~~~~~~~~~
644

    
645
Cyclades is able to create and manage Virtual Networks. Networking is
646
desployment specific and must be customized based on the specific needs of the
647
system administrator. For better understanding of networking please refer to
648
the :ref:`Network <networks>` section.
649

    
650
Exactly as Cyclades VMs can be handled like Ganeti instances, Cyclades Networks
651
can also by handled as Ganeti networks, via `gnt-network commands`. All Ganeti
652
networks that belong to Synnefo are named with the prefix
653
`${BACKEND_PREFIX_ID}-net-`.
654

    
655
There are also the following `snf-manage` commands for managing networks:
656

    
657
* ``snf-manage network-list``: List networks
658
* ``snf-manage network-show``: Show information about a network in the Cyclades DB
659
* ``snf-manage network-inspect``: Inspect the state of the network in DB and Ganeti backends
660
* ``snf-manage network-modify``: Modify the state of a network in the Cycldes DB
661
* ``snf-manage network-create``: Create a new network
662
* ``snf-manage network-remove``: Remove an existing network
663

    
664
Managing Network Resources
665
``````````````````````````
666

    
667
Proper operation of the Cyclades Network Service depends on the unique
668
assignment of specific resources to each type of virtual network. Specifically,
669
these resources are:
670

    
671
* IP addresses. Cyclades creates a Pool of IPs for each Network, and assigns a
672
  unique IP address to each VM, thus connecting it to this Network. You can see
673
  the IP pool of each network by running `snf-manage network-inspect
674
  <network_ID>`. IP pools are automatically created and managed by Cyclades,
675
  depending on the subnet of the Network.
676
* Bridges corresponding to physical VLANs, which are required for networks of
677
  type `PRIVATE_PHYSICAL_VLAN`.
678
* One Bridge corresponding to one physical VLAN which is required for networks of
679
  type `PRIVATE_MAC_PREFIX`.
680

    
681
Cyclades allocates those resources from pools that are created by the
682
administrator with the `snf-manage pool-create` management command.
683

    
684
Pool Creation
685
`````````````
686
Pools are created using the `snf-manage pool-create` command:
687

    
688
.. code-block:: console
689

    
690
   # snf-manage pool-create --type=bridge --base=prv --size=20
691

    
692
will create a pool of bridges, containing bridges prv1, prv2,..prv21.
693

    
694
You can verify the creation of the pool, and check its contents by running:
695

    
696
.. code-block:: console
697

    
698
   # snf-manage pool-list
699
   # snf-manage pool-show --type=bridge 1
700

    
701
With the same commands you can handle a pool of MAC prefixes. For example:
702

    
703
.. code-block:: console
704

    
705
   # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
706

    
707
will create a pool of MAC prefixes from ``aa:00:1`` to ``b9:ff:f``. The MAC
708
prefix pool is responsible for providing only unicast and locally administered
709
MAC addresses, so many of these prefixes will be externally reserved, to
710
exclude from allocation.
711

    
712
Cyclades advanced operations
713
----------------------------
714

    
715
Reconciliation mechanism
716
~~~~~~~~~~~~~~~~~~~~~~~~
717

    
718
On certain occasions, such as a Ganeti or RabbitMQ failure, the state of
719
Cyclades database may differ from the real state of VMs and networks in the
720
Ganeti backends. The reconciliation process is designed to synchronize
721
the state of the Cyclades DB with Ganeti. There are two management commands
722
for reconciling VMs and Networks
723

    
724
Reconciling Virtual Machines
725
````````````````````````````
726

    
727
Reconciliation of VMs detects the following conditions:
728

    
729
 * Stale DB servers without corresponding Ganeti instances
730
 * Orphan Ganeti instances, without corresponding DB entries
731
 * Out-of-sync state for DB entries wrt to Ganeti instances
732

    
733
To detect all inconsistencies you can just run:
734

    
735
.. code-block:: console
736

    
737
  $ snf-manage reconcile-servers
738

    
739
Adding the `--fix-all` option, will do the actual synchronization:
740

    
741
.. code-block:: console
742

    
743
  $ snf-manage reconcile-servers --fix-all
744

    
745
Please see ``snf-manage reconcile-servers --help`` for all the details.
746

    
747

    
748
Reconciling Networks
749
````````````````````
750

    
751
Reconciliation of Networks detects the following conditions:
752

    
753
  * Stale DB networks without corresponding Ganeti networks
754
  * Orphan Ganeti networks, without corresponding DB entries
755
  * Private networks that are not created to all Ganeti backends
756
  * Unsynchronized IP pools
757

    
758
To detect all inconsistencies you can just run:
759

    
760
.. code-block:: console
761

    
762
  $ snf-manage reconcile-networks
763

    
764
Adding the `--fix-all` option, will do the actual synchronization:
765

    
766
.. code-block:: console
767

    
768
  $ snf-manage reconcile-networks --fix-all
769

    
770
Please see ``snf-manage reconcile-networks --help`` for all the details.
771

    
772

    
773

    
774
Block Storage Service (Archipelago)
775
===================================
776

    
777
Overview
778
--------
779
Archipelago offers Copy-On-Write snapshotable volumes. Pithos images can be used
780
to provision a volume with Copy-On-Write semantics (i.e. a clone). Snapshots
781
offer a unique deduplicated image of a volume, that reflects the volume state
782
during snapshot creation and are indistinguishable from a Pithos image.
783

    
784
Archipelago is used by Cyclades and Ganeti for fast provisioning of VMs based on
785
CoW volumes. Moreover, it enables live migration of thinly-provisioned VMs with
786
no physically shared storage.
787

    
788
Archipelago Architecture
789
------------------------
790

    
791
.. image:: images/archipelago-architecture.png
792
   :width: 50%
793
   :target: _images/archipelago-architecture.png
794

    
795
.. _syn+archip+rados:
796

    
797
Overview of Synnefo + Archipelago + RADOS
798
-----------------------------------------
799

    
800
.. image:: images/synnefo-arch3.png
801
   :width: 100%
802
   :target: _images/synnefo-arch3.png
803

    
804
Prereqs
805
-------
806

    
807
The administrator must initialize the storage backend where archipelago volume
808
blocks will reside.
809

    
810
In case of a files backend, the administrator must create two directories. One
811
for the archipelago data blocks and one for the archipelago map blocks. These
812
should probably be over shared storage to enable sharing archipelago volumes
813
between multiple nodes. He or she, must also be able to supply a directory where
814
the pithos data and map blocks reside.
815

    
816
In case of a RADOS backend, the administrator must create two rados pools, one
817
for data blocks, and one for the map blocks. These pools, must be the same pools
818
used in pithos, in order to enable volume creation based on pithos images.
819

    
820
Installation
821
------------
822

    
823
Archipelago consists of
824

    
825
* ``libxseg0``: libxseg used to communicate over shared memory segments
826
* ``python-xseg``: python bindings for libxseg
827
* ``archipelago-kernel-dkms``: contains archipelago kernel modules to provide
828
  block devices to be used as vm disks
829
* ``python-archipelago``: archipelago python module. Includes archipelago and
830
  vlmc functionality.
831
* ``archipelago``: user space tools and peers for the archipelago management and
832
  volume composition
833
* ``archipelago-ganeti``: ganeti ext storage scripts, that enable ganeti to
834
  provision VMs over archipelago
835

    
836
Performing
837

    
838
.. code-block:: console
839

    
840
  $ apt-get install archipelago-ganeti 
841

    
842
should fetch all the required packages and get you up 'n going with archipelago
843

    
844
Bare in mind, that custom librados is required, which is provided in the apt
845
repo of GRNet.
846

    
847

    
848
For now, librados is a dependency of archipelago, even if you do not intend to
849
use archipelago over RADOS.
850

    
851
Configuration
852
-------------
853
Archipelago should work out of the box with a RADOS backend, but basic
854
configuration can be done in ``/etc/default/archipelago`` .
855

    
856
If you wish to change the storage backend to files, set
857

    
858
.. code-block:: console
859

    
860
   STORAGE="files"
861

    
862
and provide the appropriate settings for files storage backend in the conf file.
863

    
864
These are:
865

    
866
* ``FILED_IMAGES``: directory for archipelago data blocks.
867
* ``FILED_MAPS``: directory for archipelago map blocks.
868
* ``PITHOS``: directory of pithos data blocks.
869
* ``PITHOSMAPS``: directory of pithos map blocks.
870

    
871
The settings for RADOS storage backend are:
872

    
873
* ``RADOS_POOL_MAPS``: The pool where archipelago and pithos map blocks reside.
874
* ``RADOS_POOL_BLOCKS``: The pool where archipelago and pithos data blocks
875
  reside.
876

    
877
Examples can be found in the conf file.
878

    
879
Be aware that archipelago infrastructure doesn't provide default values for this
880
settings. If they are not set in the conf file, archipelago will not be able to
881
function.
882

    
883
Archipelago also provides ``VERBOSITY`` config options to control the output
884
generated by the userspace peers.
885

    
886
The available options are:
887

    
888
* ``VERBOSITY_BLOCKERB``
889
* ``VERBOSITY_BLOCKERM``
890
* ``VERBOSITY_MAPPER``
891
* ``VERBOSITY_VLMC``
892

    
893
and the available values are:
894

    
895
* 0 : Error only logging.
896
* 1 : Warning logging.
897
* 2 : Info logging.
898
* 3 : Debug logging. WARNING: This options produces tons of output, but the
899
  logrotate daemon should take care of it.
900

    
901
Working with Archipelago
902
------------------------
903

    
904
``archipelago`` provides basic functionality for archipelago.
905

    
906
Usage:
907

    
908
.. code-block:: console
909

    
910
  $ archipelago [-u] command
911

    
912

    
913
Currently it supports the following commands:
914

    
915
* ``start [peer]``
916
  Starts archipelago or the specified peer.
917
* ``stop [peer]``
918
  Stops archipelago or the specified peer.
919
* ``restart [peer]``
920
  Restarts archipelago or the specified peer.
921
* ``status``
922
  Show the status of archipelago.
923

    
924
Available peers: ``blockerm``, ``blockerb``, ``mapperd``, ``vlmcd``.
925

    
926

    
927
``start``, ``stop``, ``restart`` can be combined with the ``-u / --user`` option
928
to affect only the userspace peers supporting archipelago.
929

    
930

    
931

    
932
Archipelago advanced operations
933
-------------------------------
934
The ``vlmc`` tool provides a way to interact with archipelago volumes
935

    
936
* ``vlmc map <volumename>``: maps the volume to a xsegbd device.
937

    
938
* ``vlmc unmap </dev/xsegbd[1-..]>``: unmaps the specified device from the
939
  system.
940

    
941
* ``vlmc create <volumename> --snap <snapname> --size <size>``: creates a new
942
  volume named <volumename> from snapshot name <snapname> with size <size>.
943
  The ``--snap`` and ``--size`` are optional, but at least one of them is
944
  mandatory. e.g:
945

    
946
  ``vlmc create <volumename> --snap <snapname>`` creates a volume named
947
  volumename from snapshot snapname. The size of the volume is the same as
948
  the size of the snapshot.
949

    
950
  ``vlmc create <volumename> --size <size>`` creates an empty volume of size
951
  <size> named <volumename>.
952

    
953
* ``vlmc remove <volumename>``: removes the volume and all the related
954
  archipelago blocks from storage.
955

    
956
* ``vlmc list``: provides a list of archipelago volumes. Currently only works
957
  with RADOS storage backend.
958

    
959
* ``vlmc info <volumename>``: shows volume information. Currently returns only
960
  volume size.
961

    
962
* ``vlmc open <volumename>``: opens an archipelago volume. That is, taking all
963
  the necessary locks and also make the rest of the infrastructure aware of the
964
  operation.
965

    
966
  This operation succeeds if the volume is alread opened.
967

    
968
* ``vlmc close <volumename>``: closes an archipelago volume. That is, performing
969
  all the necessary functions in the insfrastrure to successfully release the
970
  volume. Also releases all the acquired locks.
971

    
972
  ``vlmc close`` should be performed after a ``vlmc open`` operation.
973

    
974
* ``vlmc lock <volumename>``: locks a volume. This step allow the administrator
975
  to lock an archipelago volume, independently from the rest of the
976
  infrastrure.
977

    
978
* ``vlmc unlock [-f] <volumename>``: unlocks a volume. This allow the
979
  administrator to unlock a volume, independently from the rest of the
980
  infrastructure.
981
  The unlock option can be performed only by the blocker that acquired the lock
982
  in the first place. To unlock a volume from another blocker, ``-f`` option
983
  must be used to break the lock.
984

    
985

    
986
Synnefo management commands ("snf-manage")
987
==========================================
988

    
989
Each Synnefo service, Astakos, Pithos and Cyclades are controlled by the
990
administrator using the "snf-manage" admin tool. This tool is an extension of
991
the Django command-line management utility. It is run on the host that runs
992
each service and provides different types of commands depending the services
993
running on the host. If you are running more than one service on the same host
994
"snf-manage" adds all the corresponding commands for each service dynamically,
995
providing a unified admin environment.
996

    
997
To run "snf-manage" you just type:
998

    
999
.. code-block:: console
1000

    
1001
   # snf-manage <command> [arguments]
1002

    
1003
on the corresponding host that runs the service. For example, if you have all
1004
services running on different physical hosts you would do:
1005

    
1006
.. code-block:: console
1007

    
1008
   root@astakos-host # snf-manage <astakos-command> [argument]
1009
   root@pithos-host # snf-manage <pithos-command> [argument]
1010
   root@cyclades-host # snf-manage <cyclades-command> [argument]
1011

    
1012
If you have all services running on the same host you would do:
1013

    
1014
.. code-block:: console
1015

    
1016
   root@synnefo-host # snf-manage <{astakos,pithos,cyclades}-command> [argument]
1017

    
1018
Note that you cannot execute a service's command on a host that is not running
1019
this service. For example, the following will return an error if Astakos and
1020
Cyclades are installed on different physical hosts:
1021

    
1022
.. code-block:: console
1023

    
1024
   root@astakos-host # snf-manage <cyclades-command> [argument]
1025
   Unknown command: 'cyclades-command'
1026
   Type 'snf-manage help' for usage.
1027

    
1028
This is the complete list of "snf-manage" commands for each service.
1029

    
1030
Astakos snf-manage commands
1031
---------------------------
1032

    
1033
============================  ===========================
1034
Name                          Description
1035
============================  ===========================
1036
fix-superusers                Transform superusers created by syncdb into AstakosUser instances
1037
cleanup-full                  Cleanup sessions and session catalog
1038
commission-list               List pending commissions
1039
commission-show               Show details for a pending commission
1040
component-add                 Register a component
1041
component-list                List components
1042
component-modify              Modify component attributes
1043
project-control               Manage projects and applications
1044
project-list                  List projects
1045
project-show                  Show project details
1046
quota                         List and check the integrity of user quota
1047
reconcile-resources-astakos   Reconcile resource usage of Quotaholder with Astakos DB
1048
resource-export-astakos       Export astakos resources in json format
1049
resource-import               Register resources
1050
resource-list                 List resources
1051
resource-modify               Modify a resource's default base quota and boolean flags
1052
service-import                Register services
1053
service-list                  List services
1054
service-show                  Show service details
1055
term-add                      Add approval terms
1056
user-activation-send          Send user activation
1057
user-add                      Add user
1058
authpolicy-add                Create a new authentication provider policy profile
1059
authpolicy-list               List existing authentication provider policy profiles
1060
authpolicy-remove             Remove an authentication provider policy
1061
authpolicy-set                Assign an existing authentication provider policy profile to a user or group
1062
authpolicy-show               Show authentication provider profile details
1063
group-add                     Create a group with the given name
1064
group-list                    List available groups
1065
user-list                     List users
1066
user-modify                   Modify user
1067
user-show                     Show user details
1068
============================  ===========================
1069

    
1070
Pithos snf-manage commands
1071
--------------------------
1072

    
1073
============================  ===========================
1074
Name                          Description
1075
============================  ===========================
1076
reconcile-commissions-pithos  Display unresolved commissions and trigger their recovery
1077
resource-export-pithos        Export pithos resources in json format
1078
reconcile-resources-pithos    Detect unsynchronized usage between Astakos and Pithos DB resources and synchronize them if specified so.
1079
============================  ===========================
1080

    
1081
Cyclades snf-manage commands
1082
----------------------------
1083

    
1084
============================== ===========================
1085
Name                           Description
1086
============================== ===========================
1087
backend-add                    Add a new Ganeti backend
1088
backend-list                   List backends
1089
backend-modify                 Modify a backend
1090
backend-update-status          Update backend statistics for instance allocation
1091
backend-remove                 Remove a Ganeti backend
1092
server-create                  Create a new server
1093
server-show                    Show server details
1094
server-list                    List servers
1095
server-modify                  Modify a server
1096
server-import                  Import an existing Ganeti VM into synnefo
1097
server-inspect                 Inspect a server in DB and Ganeti
1098
network-create                 Create a new network
1099
network-list                   List networks
1100
network-modify                 Modify a network
1101
network-inspect                Inspect network state in DB and Ganeti
1102
network-remove                 Delete a network
1103
flavor-create                  Create a new flavor
1104
flavor-list                    List flavors
1105
flavor-modify                  Modify a flavor
1106
image-list                     List images
1107
image-show                     Show image details
1108
pool-create                    Create a bridge or mac-prefix pool
1109
pool-show                      Show pool details
1110
pool-list                      List pools
1111
pool-modify                    Modify a pool
1112
pool-remove                    Delete a pool
1113
queue-inspect                  Inspect the messages of a RabbitMQ queue
1114
queue-retry                    Resend messages from Dead Letter queues to original exchanges
1115
resource-export-cyclades       Export Cyclades resources in JSON format.
1116
service-export-cyclades        Export Cyclades services in JSON format.
1117
reconcile-servers              Reconcile servers of Synnefo DB with state of Ganeti backend
1118
reconcile-networks             Reconcile networks of Synnefo DB with state of Ganeti backend
1119
reconcile-pools                Check consistency of pool resources
1120
reconcile-commissions-cyclades Detect and resolve pending commissions to Quotaholder
1121
reconcile-resources-cyclades   Reconcile resource usage of Astakos with Cyclades DB.
1122
============================== ===========================
1123

    
1124
Astakos helper scripts
1125
======================
1126

    
1127
Astakos includes two scripts to facilitate the installation procedure.
1128
Running:
1129

    
1130
.. code-block:: console
1131

    
1132
   snf-component-register [<component_name>]
1133

    
1134
automates the registration of the standard Synnefo components (astakos,
1135
cyclades, and pithos) in astakos database. It internally uses the script:
1136

    
1137
.. code-block:: console
1138

    
1139
   snf-service-export <component_name> <base_url>
1140

    
1141
which simulates the export of service and resource definitions of the
1142
standard Synnefo components.
1143

    
1144
Pithos managing accounts
1145
========================
1146

    
1147
Pithos provides a utility tool for managing accounts.
1148
To run you just type:
1149

    
1150
.. code-block:: console
1151

    
1152
   # pithos-manage-accounts <command> [arguments]
1153

    
1154
This is the list of the available commands:
1155

    
1156
============================  ===========================
1157
Name                          Description
1158
============================  ===========================
1159
delete                        Remove an account from the Pithos DB
1160
export-quota                  Export account quota in a file
1161
list                          List existing/dublicate accounts
1162
merge                         Move an account contents in another account
1163
set-container-quota           Set container quota for all or a specific account
1164
============================  ===========================
1165

    
1166

    
1167
The "kamaki" API client
1168
=======================
1169

    
1170
To upload, register or modify an image you will need the **kamaki** tool.
1171
Before proceeding make sure that it is configured properly. Verify that
1172
*image.url*, *file.url*, *user.url* and *token* are set as needed:
1173

    
1174
.. code-block:: console
1175

    
1176
   $ kamaki config list
1177

    
1178
To change a setting use ``kamaki config set``:
1179

    
1180
.. code-block:: console
1181

    
1182
   $ kamaki config set image.url https://cyclades.example.com/image
1183
   $ kamaki config set file.url https://pithos.example.com/v1
1184
   $ kamaki config set user.url https://accounts.example.com
1185
   $ kamaki config set token ...
1186

    
1187
To test that everything works, try authenticating the current account with
1188
kamaki:
1189

    
1190
.. code-block:: console
1191

    
1192
  $ kamaki user authenticate
1193

    
1194
This will output user information.
1195

    
1196
Upload Image
1197
------------
1198

    
1199
By convention, images are stored in a container called ``images``. Check if the
1200
container exists, by listing all containers in your account:
1201

    
1202
.. code-block:: console
1203

    
1204
   $ kamaki file list
1205

    
1206
If the container ``images`` does not exist, create it:
1207

    
1208
.. code-block:: console
1209

    
1210
  $ kamaki file create images
1211

    
1212
You are now ready to upload an image to container ``images``. You can upload it
1213
with a Pithos client, or use kamaki directly:
1214

    
1215
.. code-block:: console
1216

    
1217
   $ kamaki file upload ubuntu.iso images
1218

    
1219
You can use any Pithos client to verify that the image was uploaded correctly,
1220
or you can list the contents of the container with kamaki:
1221

    
1222
.. code-block:: console
1223

    
1224
  $ kamaki file list images
1225

    
1226
The full Pithos URL for the previous example will be
1227
``pithos://u53r-un1qu3-1d/images/ubuntu.iso`` where ``u53r-un1qu3-1d`` is the
1228
unique user id (uuid).
1229

    
1230
Register Image
1231
--------------
1232

    
1233
To register an image you will need to use the full Pithos URL. To register as
1234
a public image the one from the previous example use:
1235

    
1236
.. code-block:: console
1237

    
1238
   $ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso --public
1239

    
1240
The ``--public`` flag is important, if missing the registered image will not
1241
be listed by ``kamaki image list``.
1242

    
1243
Use ``kamaki image register`` with no arguments to see a list of available
1244
options. A more complete example would be the following:
1245

    
1246
.. code-block:: console
1247

    
1248
   $ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso \
1249
            --public --disk-format diskdump --property kernel=3.1.2
1250

    
1251
To verify that the image was registered successfully use:
1252

    
1253
.. code-block:: console
1254

    
1255
   $ kamaki image list --name-like=ubuntu
1256

    
1257

    
1258
Miscellaneous
1259
=============
1260

    
1261
.. _branding:
1262

    
1263
Branding
1264
--------
1265

    
1266
Since Synnefo v0.14, you are able to adapt the Astakos, Pithos and Cyclades Web
1267
UI to your company’s visual identity. This is possible using the snf-branding
1268
component, which is automatically installed on the nodes running the API
1269
servers for Astakos, Pithos and Cyclades. 
1270

    
1271
Configuration
1272
~~~~~~~~~~~~~
1273

    
1274
This can be done by modifing the settings provided by the snf-branding component
1275
to match your service identity. The settings for the snf-branding application
1276
can be found inside the configuration file ``/etc/synnefo/15-snf-branding.conf``
1277
on the nodes that have Astakos, Pithos and Cyclades installed.
1278

    
1279
By default, the global service name is "Synnefo" and the company name is
1280
"GRNET". These names and their respective logos and URLs are used throughout
1281
the Astakos, Pithos and Cyclades UI.
1282

    
1283
**Names and URLs:**
1284

    
1285
The first group of branding customization refers to the service's and company's
1286
information.
1287

    
1288
You can overwrite the company and the service name and URL respectively by
1289
uncommenting and setting the following:
1290

    
1291
.. code-block:: python
1292
  
1293
  # setting used in Astakos Dashboard/Projects pages
1294
  BRANDING_SERVICE_NAME = 'My cloud'
1295
  BRANDING_SERVICE_URL = 'http://www.mycloud.synnefo.org/'
1296

    
1297
  # settings used in Astakos, Pithos, Cyclades footer only if 
1298
  # BRANDING_SHOW_COPYRIGHT is set to True
1299
  BRANDING_SHOW_COPYRIGHT = True
1300
  BRANDING_COMPANY_NAME = 'Company LTD'
1301
  BRANDING_COMPANY_URL = 'https://www.company-ltd.synnefo.org/'
1302

    
1303

    
1304
**Copyright and footer options:**
1305

    
1306
By default, no Copyright message is shown in the UI footer. If you want to make
1307
it visible in the footer of Astakos, Pithos and Cyclades UI, you can uncomment
1308
and set to ``True`` the ``BRANDING_SHOW_COPYRIGHT`` setting:
1309

    
1310
.. code-block:: python
1311

    
1312
  #BRANDING_SHOW_COPYRIGHT = False
1313

    
1314
Copyright message defaults to 'Copyright (c) 2011-<current_year>
1315
<BRANDING_COMPANY_NAME>.' but you can overwrite it to a completely custom one by
1316
setting the following option:
1317

    
1318
.. code-block:: python
1319

    
1320
  BRANDING_COPYRIGHT_MESSAGE = 'Copyright (c) 2011-2013 GRNET'
1321

    
1322
If you want to include a custom message in the footer, you can uncomment and 
1323
set the ``BRANDING_FOOTER_EXTRA_MESSAGE`` setting. You can use html markup. 
1324
Your custom message will appear  above Copyright message at the Compute 
1325
templates and the Dashboard UI.
1326

    
1327
.. code-block:: python
1328

    
1329
  #BRANDING_FOOTER_EXTRA_MESSAGE = ''
1330

    
1331

    
1332
**Images:**
1333

    
1334
The Astakos, Pithos and Cyclades Web UI has some logos and images.
1335
 
1336
The branding-related images are presented in  the following table:
1337

    
1338
===============  ============================  =========
1339
Image            Name/extension  convention    Usage
1340
===============  ============================  =========
1341
Favicon          favicon.ico                   Favicon for all services
1342
Dashboard logo   dashboard_logo.png            Visible in all Astakos UI pages
1343
Compute logo     compute_logo.png              Visible in all Cyclades UI pages
1344
Console logo     console_logo.png              Visible in the Cyclades Console Window
1345
Storage logo     storage_logo.png              Visible in all Pithos UI pages
1346
===============  ============================  =========
1347

    
1348
There are two methods  available for replacing all, or individual, 
1349
branding-related images:
1350

    
1351
1. Create a new directory inside ``/usr/share/synnefo/static/`` (e.g.
1352
   ``mybranding``) and place there some or all of your images.
1353

    
1354
   If you want to replace all of your images, keep the name/extension
1355
   conventions as indicated in the above table and change the
1356
   ``BRANDING_IMAGE_MEDIA_URL`` setting accordingly:
1357

    
1358
   .. code-block:: python
1359
        
1360
      # using relative path
1361
      BRANDING_IMAGE_MEDIA_URL= '/static/mybranding/images/' 
1362

    
1363
      # or if you already host them in a separate domain (e.g. cdn)
1364
      BRANDING_IMAGE_MEDIA_URL= 'https://cdn.synnefo.org/branding/images/'
1365

    
1366

    
1367
   If you wish to replace individual images, **do not uncomment**
1368
   ``BRANDING_IMAGE_MEDIA_URL``, but instead provide a relative path, pointing to
1369
   the file inside your directory for each ``BRANDING_<image>_URL`` that you wish
1370
   to replace.
1371

    
1372
2. Upload some or all of your images to a server and replace each 
1373
   ``BRANDING_<image>_URL`` with the absolute url of the image (i.e.
1374
   ``BRANDING_DASHBOARD_URL = 'https://www.synnefo.com/images/my_dashboard.jpg'``).
1375

    
1376
   Note that the alternative text  for each image tag inside html documents is 
1377
   alt=“BRANDING_SERVICE_NAME {Dashboard, Compute. Console, Storage}” respectively.
1378

    
1379
.. note:: Retina optimized images:
1380

    
1381
   Synnefo UI is optimized for Retina displays. As far as images are concerned,  
1382
   `retina.js <http://retinajs.com/>`_ is used.
1383

    
1384
   Retina.js checks each image on a page to see if there is a high-resolution 
1385
   version of that image on your server. If a high-resolution variant exists, 
1386
   the script will swap in that image in-place.
1387

    
1388
   The script assumes you use  `Apple's prescribed high-resolution modifier (@2x)
1389
   <http://developer.apple.com/library/ios/#documentation/2DDrawing/Conceptual/
1390
   DrawingPrintingiOS/SupportingHiResScreensInViews/SupportingHiResScreensInViews
1391
   .html#//apple_ref/doc/uid/TP40010156-CH15-SW1>`_ to denote high-resolution 
1392
   image variants on your server.
1393

    
1394
   For each of the images that you wish the script to  replace, you must have a 
1395
   high-resolution variant in the same folder  named correctly and it will be 
1396
   detected automatically. For example if your image is in <my_directory> and is 
1397
   named "my_image.jpg" the script will look in the same directory for an image 
1398
   named "my_image@2x.jpg".
1399

    
1400
   In case that you don’t want to use a high-resolution image, the 
1401
   normal-resolution image will be visible.
1402

    
1403
More branding
1404
~~~~~~~~~~~~~
1405

    
1406
Although, it is not 100% branding-related, further verbal customization is
1407
feasible. 
1408

    
1409
**EMAILS**
1410

    
1411
The output of all email `*`.txt files will be already customized to contain your
1412
company and service names but you can further alter their content if you feel it
1413
best fits your needs as simple as creasynnefo template.    
1414

    
1415
In order to overwrite one or more email-templates you need to place your 
1416
modified <email-file>.txt files respecting the following structure:
1417
  
1418
  **/etc/synnefo/templates/**
1419
      **im/**
1420
          | activation_email.txt
1421
          | email.txt
1422
          | invitation.txt
1423
          | switch_accounts_email.txt
1424
          | welcome_email.txt
1425
          **projects/**
1426
              | project_approval_notification.txt
1427
              | project_denial_notification.txt    
1428
              | project_membership_change_notification.txt
1429
              | project_membership_enroll_notification.txt
1430
              | project_membership_leave_request_notification.txt
1431
              | project_membership_request_notification.txt
1432
              | project_suspension_notification.txt
1433
              | project_termination_notification.txt
1434
      **registration/**
1435
          | email_change_email.txt
1436
          | password_email.txt
1437

    
1438
Feel free to omit any of the above files you do not wish to overwrite.
1439

    
1440
Below is a list of all emails sent by Synnefo to users along with a short 
1441
description and a link to their content:
1442

    
1443
* ``snf-astakos-app/astakos/im/templates/im/email.txt``
1444
  Base email template. Contains a contact email and a “thank you” message.
1445
  (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/email.txt>`_)
1446
* ``snf-astakos-app/astakos/im/templates/im/activation_email.txt`` Email sent to
1447
  user that prompts  him/her to click on a link provided to activate the account.
1448
  Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/activation_email.txt>`_)
1449
* ``snf-astakos-app/astakos/im/templates/im/invitation.txt`` Email sent to an
1450
  invited user. He/she has to click on a link provided to activate the account.
1451
  Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/invitation.txt>`_)
1452
* ``snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt`` Email
1453
  sent to user upon his/her request to associate this email address with a
1454
  shibboleth account. He/she has to click on a link provided to activate the
1455
  association. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt>`_)
1456
* ``snf-astakos-app/astakos/im/templates/im/welcome_email.txt`` Email sent to
1457
  inform the user that his/ her account has been activated. Extends “email.txt”
1458
  (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/welcome_email.txt>`_)
1459
* ``snf-astakos-app/astakos/im/templates/registration/email_change_email.txt``
1460
  Email sent to user when he/she has requested new email address assignment. The
1461
  user has to click on a link provided to validate this action. Extends
1462
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/email_change_email.txt>`_)
1463
* ``snf-astakos-app/astakos/im/templates/registration/password_email.txt`` Email
1464
  sent for resetting password purpose. The user has to click on a link provided
1465
  to validate this action. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/password_email.txt>`_)
1466
* ``snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt``
1467
  Informs  the project owner that his/her project has been approved. Extends
1468
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt>`_)
1469
* ``snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt``
1470
  Informs the project owner that his/her  project application has been denied
1471
  explaining the reasons. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt>`_)
1472
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt``
1473
  An email is sent to a user containing information about his project membership
1474
  (whether he has been accepted, rejected or removed). Extends “email.txt” (`Link
1475
  <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt>`_)
1476
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt``
1477
  Informs a user that he/she  has been enrolled to a project. Extends
1478
  “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt>`_)
1479
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt``
1480
  An email is sent to the project owner to make him aware of a  user having
1481
  requested to leave his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt>`_)
1482
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt``
1483
  An email is sent to the project owner to make him/her aware of a user having
1484
  requested to join  his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt>`_)
1485
* ``snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt``
1486
  An email is sent to the project owner to make him/her aware of his/her project
1487
  having been suspended. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt>`_)
1488
* ``snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt``
1489
  An email is sent to the project owner to make him/her aware of his/her project
1490
  having been terminated. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt>`_)
1491

    
1492
.. warning:: Django templates language:
1493

    
1494
  If you choose to  overwrite these email templates, be mindful of the necessary 
1495
  information contained in django template variables that must not be omitted, 
1496
  such as the activation link for activating one’s account and many more. 
1497
  These variables are contained into {{}} inside the templates.
1498

    
1499

    
1500
.. RabbitMQ
1501

    
1502
RabbitMQ Broker
1503
---------------
1504

    
1505
Queue nodes run the RabbitMQ sofware, which provides AMQP functionality. To
1506
guarantee high-availability, more than one Queue nodes should be deployed, each
1507
of them belonging to the same `RabbitMQ cluster
1508
<http://www.rabbitmq.com/clustering.html>`_. Synnefo uses the RabbitMQ
1509
active/active `High Available Queues <http://www.rabbitmq.com/ha.html>`_ which
1510
are mirrored between two nodes within a RabbitMQ cluster.
1511

    
1512
The RabbitMQ nodes that form the cluster, are declared to Synnefo through the
1513
`AMQP_HOSTS` setting. Each time a Synnefo component needs to connect to
1514
RabbitMQ, one of these nodes is chosen in a random way. The client that Synnefo
1515
uses to connect to RabbitMQ, handles connection failures transparently and
1516
tries to reconnect to a different node. As long as one of these nodes are up
1517
and running, functionality of Synnefo should not be downgraded by the RabbitMQ
1518
node failures.
1519

    
1520
All the queues that are being used are declared as durable, meaning that
1521
messages are persistently stored to RabbitMQ, until they get successfully
1522
processed by a client.
1523

    
1524
Currently, RabbitMQ is used by the following components:
1525

    
1526
* `snf-ganeti-eventd`, `snf-ganeti-hook` and `snf-progress-monitor`:
1527
  These components send messages concerning the status and progress of
1528
  jobs in the Ganeti backend.
1529
* `snf-dispatcher`: This daemon, consumes the messages that are sent from
1530
  the above components, and updates the Cyclades DB accordingly.
1531

    
1532

    
1533
Installation
1534
~~~~~~~~~~~~
1535

    
1536
Please check the RabbitMQ documentation which covers extensively the
1537
`installation of RabbitMQ server <http://www.rabbitmq.com/download.html>`_ and
1538
the setup of a `RabbitMQ cluster <http://www.rabbitmq.com/clustering.html>`_.
1539
Also, check out the `web management plugin
1540
<http://www.rabbitmq.com/management.html>`_ that can be useful for managing and
1541
monitoring RabbitMQ.
1542

    
1543
For a basic installation of RabbitMQ on two nodes (node1 and node2) you can do
1544
the following:
1545

    
1546
On both nodes, install rabbitmq-server and create a Synnefo user:
1547

    
1548
.. code-block:: console
1549

    
1550
  $ apt-get install rabbitmq-server
1551
  $ rabbitmqctl add_user synnefo "example_pass"
1552
  $ rabbitmqctl set_permissions synnefo  ".*" ".*" ".*"
1553

    
1554
Also guarantee that both nodes share the same cookie, by running:
1555

    
1556
.. code-block:: console
1557

    
1558
  $ scp node1:/var/lib/rabbitmq/.erlang.cookie node2:/var/lib/rabbitmq/.erlang.cookie
1559

    
1560
and restart the nodes:
1561

    
1562
.. code-block:: console
1563

    
1564
  $ /etc/init.d/rabbitmq-server restart
1565

    
1566

    
1567
To setup the RabbitMQ cluster run:
1568

    
1569
.. code-block:: console
1570

    
1571
  root@node2: rabbitmqctl stop_app
1572
  root@node2: rabbitmqctl reset
1573
  root@node2: rabbitmqctl cluster rabbit@node1 rabbit@node2
1574
  root@node2: rabbitmqctl start_app
1575

    
1576
You can verify that the cluster is set up correctly by running:
1577

    
1578
.. code-block:: console
1579

    
1580
  root@node2: rabbitmqctl cluster_status
1581

    
1582

    
1583
Logging
1584
-------
1585

    
1586
Logging in Synnefo is using Python's logging module. The module is configured
1587
using dictionary configuration, whose format is described here:
1588

    
1589
http://docs.python.org/release/2.7.1/library/logging.html#logging-config-dictschema
1590

    
1591
Note that this is a feature of Python 2.7 that we have backported for use in
1592
Python 2.6.
1593

    
1594
The logging configuration dictionary is defined in
1595
``/etc/synnefo/10-snf-webproject-logging.conf``
1596

    
1597
The administrator can have finer logging control by modifying the
1598
``LOGGING_SETUP`` dictionary, and defining subloggers with different handlers
1599
and log levels.  e.g. To enable debug messages only for the API set the level
1600
of 'synnefo.api' to ``DEBUG``
1601

    
1602
By default, the Django webapp and snf-manage logs to syslog, while
1603
`snf-dispatcher` logs to `/var/log/synnefo/dispatcher.log`.
1604

    
1605

    
1606
.. _scale-up:
1607

    
1608
Scaling up to multiple nodes
1609
============================
1610

    
1611
Here we will describe how should a large scale Synnefo deployment look like. Make
1612
sure you are familiar with Synnefo and Ganeti before proceeding with this section.
1613
This means you should at least have already set up successfully a working Synnefo
1614
deployment as described in the :ref:`Admin's Installation Guide
1615
<quick-install-admin-guide>` and also read the Administrator's Guide until this
1616
section.
1617

    
1618
Graph of a scale-out Synnefo deployment
1619
---------------------------------------
1620

    
1621
Each box in the following graph corresponds to a distinct physical node:
1622

    
1623
.. image:: images/synnefo-arch2-roles.png
1624
   :width: 100%
1625
   :target: _images/synnefo-arch2-roles.png
1626

    
1627
The above graph is actually the same with the one at the beginning of this
1628
:ref:`guide <admin-guide>`, with the only difference that here we show the
1629
Synnefo roles of each physical node. These roles are described in the
1630
following section.
1631

    
1632
.. _physical-node-roles:
1633

    
1634
Physical Node roles
1635
-------------------
1636

    
1637
As appears in the previous graph, a scale-out Synnefo deployment consists of
1638
multiple physical nodes that have the following roles:
1639

    
1640
* **WEBSERVER**: A web server running in front of gunicorn (e.g.: Apache, nginx)
1641
* **ASTAKOS**: The Astakos application (gunicorn)
1642
* **ASTAKOS_DB**: The Astakos database (postgresql)
1643
* **PITHOS**: The Pithos application (gunicorn)
1644
* **PITHOS_DB**: The Pithos database (postgresql)
1645
* **CYCLADES**: The Cyclades application (gunicorn)
1646
* **CYCLADES_DB**: The Cyclades database (postgresql)
1647
* **MQ**: The message queue (RabbitMQ)
1648
* **GANETI_MASTER**: The Ganeti master of a Ganeti cluster
1649
* **GANETI_NODE** : A VM-capable Ganeti node of a Ganeti cluster
1650

    
1651
You will probably also have:
1652

    
1653
* **CMS**: The CMS used as a frotend portal for the Synnefo services
1654
* **NS**: A nameserver serving all other Synnefo nodes and resolving Synnefo FQDNs
1655
* **CLIENT**: A machine that runs the Synnefo clients (e.g.: kamaki, Web UI),
1656
              most of the times, the end user's local machine
1657

    
1658
From this point we will also refer to the following groups of roles:
1659

    
1660
* **SYNNEFO**: [ **ASTAKOS**, **ASTAKOS_DB**, **PITHOS**, **PITHOS_DB**, **CYCLADES**, **CYCLADES_DB**, **MQ**, **CMS**]
1661
* **G_BACKEND**: [**GANETI_MASTER**, **GANETI_NODE**]
1662

    
1663
Of course, when deploying Synnefo you can combine multiple of the above roles on a
1664
single physical node, but if you are trying to scale out, the above separation
1665
gives you significant advantages.
1666

    
1667
So, in the next section we will take a look on what components you will have to
1668
install on each physical node depending on its Synnefo role. We assume the graph's
1669
architecture.
1670

    
1671
Components for each role
1672
------------------------
1673

    
1674
When deploying Synnefo in large scale, you need to install different Synnefo
1675
or/and third party components on different physical nodes according to their
1676
Synnefo role, as stated in the previous section.
1677

    
1678
Specifically:
1679

    
1680
Role **WEBSERVER**
1681
    * Synnefo components: `None`
1682
    * 3rd party components: Apache
1683
Role **ASTAKOS**
1684
    * Synnefo components: `snf-webproject`, `snf-astakos-app`
1685
    * 3rd party components: Django, Gunicorn
1686
Role **ASTAKOS_DB**
1687
    * Synnefo components: `None`
1688
    * 3rd party components: PostgreSQL
1689
Role **PITHOS**
1690
    * Synnefo components: `snf-webproject`, `snf-pithos-app`, `snf-pithos-webclient`
1691
    * 3rd party components: Django, Gunicorn
1692
Role **PITHOS_DB**
1693
    * Synnefo components: `None`
1694
    * 3rd party components: PostgreSQL
1695
Role **CYCLADES**
1696
    * Synnefo components: `snf-webproject`, `snf-cyclades-app`, `snf-vncauthproxy`
1697
    * 3rd party components: Django Gunicorn
1698
Role **CYCLADES_DB**
1699
    * Synnefo components: `None`
1700
    * 3rd party components: PostgreSQL
1701
Role **MQ**
1702
    * Synnefo components: `None`
1703
    * 3rd party components: RabbitMQ
1704
Role **GANETI_MASTER**
1705
    * Synnefo components: `snf-cyclades-gtools`
1706
    * 3rd party components: Ganeti
1707
Role **GANETI_NODE**
1708
    * Synnefo components: `snf-cyclades-gtools`, `snf-network`, `snf-image`, `nfdhcpd`
1709
    * 3rd party components: Ganeti
1710
Role **CMS**
1711
    * Synnefo components: `snf-webproject`, `snf-cloudcms`
1712
    * 3rd party components: Django, Gunicorn
1713
Role **NS**
1714
    * Synnefo components: `None`
1715
    * 3rd party components: BIND
1716
Role **CLIENT**
1717
    * Synnefo components: `kamaki`, `snf-image-creator`
1718
    * 3rd party components: `None`
1719

    
1720
Example scale out installation
1721
------------------------------
1722

    
1723
In this section we describe an example of a medium scale installation which
1724
combines multiple roles on 10 different physical nodes. We also provide a
1725
:ref:`guide <i-synnefo>` to help with such an install.
1726

    
1727
We assume that we have the following 10 physical nodes with the corresponding
1728
roles:
1729

    
1730
Node1:
1731
    **WEBSERVER**, **ASTAKOS**
1732
      Guide sections:
1733
        * :ref:`apt <i-apt>`
1734
        * :ref:`gunicorn <i-gunicorn>`
1735
        * :ref:`apache <i-apache>`
1736
        * :ref:`snf-webproject <i-webproject>`
1737
        * :ref:`snf-astakos-app <i-astakos>`
1738
Node2:
1739
    **WEBSERVER**, **PITHOS**
1740
      Guide sections:
1741
        * :ref:`apt <i-apt>`
1742
        * :ref:`gunicorn <i-gunicorn>`
1743
        * :ref:`apache <i-apache>`
1744
        * :ref:`snf-webproject <i-webproject>`
1745
        * :ref:`snf-pithos-app <i-pithos>`
1746
        * :ref:`snf-pithos-webclient <i-pithos>`
1747
Node3:
1748
    **WEBSERVER**, **CYCLADES**
1749
      Guide sections:
1750
        * :ref:`apt <i-apt>`
1751
        * :ref:`gunicorn <i-gunicorn>`
1752
        * :ref:`apache <i-apache>`
1753
        * :ref:`snf-webproject <i-webproject>`
1754
        * :ref:`snf-cyclades-app <i-cyclades>`
1755
        * :ref:`snf-vncauthproxy <i-cyclades>`
1756
Node4:
1757
    **WEBSERVER**, **CMS**
1758
      Guide sections:
1759
        * :ref:`apt <i-apt>`
1760
        * :ref:`gunicorn <i-gunicorn>`
1761
        * :ref:`apache <i-apache>`
1762
        * :ref:`snf-webproject <i-webproject>`
1763
        * :ref:`snf-cloudcms <i-cms>`
1764
Node5:
1765
    **ASTAKOS_DB**, **PITHOS_DB**, **CYCLADES_DB**
1766
      Guide sections:
1767
        * :ref:`apt <i-apt>`
1768
        * :ref:`postgresql <i-db>`
1769
Node6:
1770
    **MQ**
1771
      Guide sections:
1772
        * :ref:`apt <i-apt>`
1773
        * :ref:`rabbitmq <i-mq>`
1774
Node7:
1775
    **GANETI_MASTER**, **GANETI_NODE**
1776
      Guide sections:
1777
        * :ref:`apt <i-apt>`
1778
        * :ref:`general <i-backends>`
1779
        * :ref:`ganeti <i-ganeti>`
1780
        * :ref:`snf-cyclades-gtools <i-gtools>`
1781
        * :ref:`snf-network <i-network>`
1782
        * :ref:`snf-image <i-image>`
1783
        * :ref:`nfdhcpd <i-network>`
1784
Node8:
1785
    **GANETI_NODE**
1786
      Guide sections:
1787
        * :ref:`apt <i-apt>`
1788
        * :ref:`general <i-backends>`
1789
        * :ref:`ganeti <i-ganeti>`
1790
        * :ref:`snf-cyclades-gtools <i-gtools>`
1791
        * :ref:`snf-network <i-network>`
1792
        * :ref:`snf-image <i-image>`
1793
        * :ref:`nfdhcpd <i-network>`
1794
Node9:
1795
    **GANETI_NODE**
1796
      Guide sections:
1797
        `Same as Node8`
1798
Node10:
1799
    **GANETI_NODE**
1800
      Guide sections:
1801
        `Same as Node8`
1802

    
1803
All sections: :ref:`Scale out Guide <i-synnefo>`
1804

    
1805

    
1806
Upgrade Notes
1807
=============
1808

    
1809
.. toctree::
1810
   :maxdepth: 1
1811

    
1812
   v0.12 -> v0.13 <upgrade/upgrade-0.13>
1813
   v0.13 -> v0.14 <upgrade/upgrade-0.14>
1814
   v0.14 -> v0.14.2 <upgrade/upgrade-0.14.2>
1815
   v0.14.5 -> v0.14.6 <upgrade/upgrade-0.14.6>
1816
   v0.14.7 -> v0.14.8 <upgrade/upgrade-0.14.8>
1817

    
1818

    
1819
Changelog, NEWS
1820
===============
1821

    
1822

    
1823
* v0.14.9 :ref:`Changelog <Changelog-0.14.9>`, :ref:`NEWS <NEWS-0.14.9>`
1824
* v0.14.8 :ref:`Changelog <Changelog-0.14.8>`, :ref:`NEWS <NEWS-0.14.8>`
1825
* v0.14.7 :ref:`Changelog <Changelog-0.14.7>`, :ref:`NEWS <NEWS-0.14.7>`
1826
* v0.14.6 :ref:`Changelog <Changelog-0.14.6>`, :ref:`NEWS <NEWS-0.14.6>`
1827
* v0.14.5 :ref:`Changelog <Changelog-0.14.5>`, :ref:`NEWS <NEWS-0.14.5>`
1828
* v0.14.4 :ref:`Changelog <Changelog-0.14.4>`, :ref:`NEWS <NEWS-0.14.4>`
1829
* v0.14.3 :ref:`Changelog <Changelog-0.14.3>`, :ref:`NEWS <NEWS-0.14.3>`
1830
* v0.14.2 :ref:`Changelog <Changelog-0.14.2>`, :ref:`NEWS <NEWS-0.14.2>`
1831
* v0.14 :ref:`Changelog <Changelog-0.14>`, :ref:`NEWS <NEWS-0.14>`
1832
* v0.13 :ref:`Changelog <Changelog-0.13>`, :ref:`NEWS <NEWS-0.13>`