root / docs / admin-guide.rst @ d14155e3
History | View | Annotate | Download (66.6 kB)
1 |
.. _admin-guide: |
---|---|
2 |
|
3 |
Synnefo Administrator's Guide |
4 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
5 |
|
6 |
This is the complete Synnefo Administrator's Guide. |
7 |
|
8 |
|
9 |
.. _syn+archip: |
10 |
|
11 |
General Synnefo Architecture |
12 |
============================ |
13 |
|
14 |
The following figure shows a detailed view of the whole Synnefo architecture |
15 |
and how it interacts with multiple Ganeti clusters. We hope that after reading |
16 |
the Administrator's Guide you will be able to understand every component and |
17 |
all the interactions between them. |
18 |
|
19 |
.. image:: images/synnefo-arch2.png |
20 |
:width: 100% |
21 |
:target: _images/synnefo-arch2.png |
22 |
|
23 |
Synnefo also supports RADOS as an alternative storage backend for |
24 |
Files/Images/VM disks. You will find the :ref:`corresponding figure |
25 |
<syn+archip+rados>` later in this guide. |
26 |
|
27 |
|
28 |
Identity Service (Astakos) |
29 |
========================== |
30 |
|
31 |
|
32 |
Authentication methods |
33 |
---------------------- |
34 |
|
35 |
Astakos supports multiple authentication methods: |
36 |
|
37 |
* local username/password |
38 |
* LDAP / Active Directory |
39 |
* SAML 2.0 (Shibboleth) federated logins |
40 |
|
41 |
|
42 |
|
43 |
|
44 |
.. _shibboleth-auth: |
45 |
|
46 |
Shibboleth Authentication |
47 |
~~~~~~~~~~~~~~~~~~~~~~~~~ |
48 |
|
49 |
Astakos can delegate user authentication to a Shibboleth federation. |
50 |
|
51 |
To setup shibboleth, install package:: |
52 |
|
53 |
apt-get install libapache2-mod-shib2 |
54 |
|
55 |
Change appropriately the configuration files in ``/etc/shibboleth``. |
56 |
|
57 |
Add in ``/etc/apache2/sites-available/synnefo-ssl``:: |
58 |
|
59 |
ShibConfig /etc/shibboleth/shibboleth2.xml |
60 |
Alias /shibboleth-sp /usr/share/shibboleth |
61 |
|
62 |
<Location /ui/login/shibboleth> |
63 |
AuthType shibboleth |
64 |
ShibRequireSession On |
65 |
ShibUseHeaders On |
66 |
require valid-user |
67 |
</Location> |
68 |
|
69 |
and before the line containing:: |
70 |
|
71 |
ProxyPass / http://localhost:8080/ retry=0 |
72 |
|
73 |
add:: |
74 |
|
75 |
ProxyPass /Shibboleth.sso ! |
76 |
|
77 |
Then, enable the shibboleth module:: |
78 |
|
79 |
a2enmod shib2 |
80 |
|
81 |
After passing through the apache module, the following tokens should be |
82 |
available at the destination:: |
83 |
|
84 |
eppn # eduPersonPrincipalName |
85 |
Shib-InetOrgPerson-givenName |
86 |
Shib-Person-surname |
87 |
Shib-Person-commonName |
88 |
Shib-InetOrgPerson-displayName |
89 |
Shib-EP-Affiliation |
90 |
Shib-Session-ID |
91 |
|
92 |
Finally, add 'shibboleth' in ``ASTAKOS_IM_MODULES`` list. The variable resides |
93 |
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf`` |
94 |
|
95 |
Twitter Authentication |
96 |
~~~~~~~~~~~~~~~~~~~~~~ |
97 |
|
98 |
To enable twitter authentication while signed in under a Twitter account, |
99 |
visit dev.twitter.com/apps. |
100 |
|
101 |
Click Create an application. |
102 |
|
103 |
Fill the necessary information and for callback URL give:: |
104 |
|
105 |
https://node1.example.com/ui/login/twitter/authenticated |
106 |
|
107 |
Finally, add 'twitter' in ``ASTAKOS_IM_MODULES`` list. The variable resides |
108 |
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf`` |
109 |
|
110 |
Google Authentication |
111 |
~~~~~~~~~~~~~~~~~~~~~ |
112 |
|
113 |
To enable google authentication while signed in under a Google account, |
114 |
visit https://code.google.com/apis/console/. |
115 |
|
116 |
Under API Access select Create another client ID, select Web application, |
117 |
expand more options in Your site or hostname section and in Authorized |
118 |
Redirect URIs add: |
119 |
|
120 |
|
121 |
Fill the necessary information and for callback URL give:: |
122 |
|
123 |
https://node1.example.com/ui/login/google/authenticated |
124 |
|
125 |
Finally, add 'google' in ``ASTAKOS_IM_MODULES`` list. The variable resides |
126 |
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf`` |
127 |
|
128 |
|
129 |
Working with Astakos |
130 |
-------------------- |
131 |
|
132 |
User registration |
133 |
~~~~~~~~~~~~~~~~~ |
134 |
|
135 |
When a new user signs up, he/she is not directly marked as active. You can see |
136 |
his/her state by running (on the machine that runs the Astakos app): |
137 |
|
138 |
.. code-block:: console |
139 |
|
140 |
$ snf-manage user-list |
141 |
|
142 |
More detailed user status is provided in the `status` field of the `user-show` |
143 |
command: |
144 |
|
145 |
.. code-block:: console |
146 |
|
147 |
$ snf-manage user-show <user-id> |
148 |
|
149 |
id : 6 |
150 |
uuid : 78661411-5eed-412f-a9ea-2de24f542c2e |
151 |
status : Accepted/Active (accepted policy: manual) |
152 |
email : user@synnefo.org |
153 |
.... |
154 |
|
155 |
Based on the `astakos-app` configuration, there are several ways for a user to |
156 |
get verified and activated in order to be able to login. We discuss the user |
157 |
verification and activation flow in the following section. |
158 |
|
159 |
User activation flow |
160 |
```````````````````` |
161 |
|
162 |
A user can register for an account using the astakos signup form. Once the form |
163 |
is submited successfully a user entry is created in astakos database. That entry |
164 |
is passed through the astakos activation backend which handles whether the user |
165 |
should be automatically verified and activated. |
166 |
|
167 |
Email verification |
168 |
`````````````````` |
169 |
|
170 |
The verification process takes place in order to ensure that the user owns the |
171 |
email provided during the signup process. By default, after each successful |
172 |
signup astakos notifies user with an verification url via email. |
173 |
|
174 |
At this stage: |
175 |
|
176 |
* subsequent registrations invalidate and delete the previous registrations |
177 |
of the same email address. |
178 |
|
179 |
* in case user misses the initial notification, additional emails can be |
180 |
send either via the url which is prompted to the user if he tries to |
181 |
login, or by the administrator using the ``snf-manage user-activation-send |
182 |
<userid>`` command. |
183 |
|
184 |
* administrator may also enforce a user to get verified using the |
185 |
``snf-manage user-modify --verify <userid>`` command. |
186 |
|
187 |
Account activation |
188 |
`````````````````` |
189 |
|
190 |
Once the user gets verified, it is time for Astakos to decide whether or not to |
191 |
proceed through user activation process. If ``ASTAKOS_MODERATION_ENABLED`` |
192 |
setting is set to ``False`` (default value) user gets activated automatically. |
193 |
|
194 |
In case the moderation is enabled Astakos may still automatically activate the |
195 |
user in the following cases: |
196 |
|
197 |
* User email matches any of the regular expressions defined in |
198 |
``ASTAKOS_RE_USER_EMAIL_PATTERNS`` (defaults to ``[]``) |
199 |
* User used a signup method (e.g. ``shibboleth``) for which automatic |
200 |
activation is enabled (see |
201 |
:ref:`authentication methods policies <auth_methods_policies>`). |
202 |
|
203 |
If all of the above fail to trigger automatic activation, an email is sent to |
204 |
the persons listed in ``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings, |
205 |
notifing that there is a new user pending for moderation and that it's up to |
206 |
the administrator to decide if the user should be activated. The UI also shows |
207 |
a corresponding 'pending moderation' message to the user. The administrator can |
208 |
activate a user using the ``snf-manage user-modify`` command: |
209 |
|
210 |
.. code-block:: console |
211 |
|
212 |
# command to activate a pending user |
213 |
$ snf-manage user-modify --accept <userid> |
214 |
|
215 |
# command to reject a pending user |
216 |
$ snf-manage user-modify --reject --reject-reason="spammer" <userid> |
217 |
|
218 |
Once the activation process finishes, a greeting message is sent to the user |
219 |
email address and a notification for the activation to the persons listed in |
220 |
``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings. Once activated the user is |
221 |
able to login and access the Synnefo services. |
222 |
|
223 |
Additional authentication methods |
224 |
````````````````````````````````` |
225 |
|
226 |
Astakos supports third party logins from external identity providers. This |
227 |
can be usefull since it allows users to use their existing credentials to |
228 |
login to astakos service. |
229 |
|
230 |
Currently astakos supports the following identity providers: |
231 |
|
232 |
* `Shibboleth <http://www.internet2.edu/shibboleth>`_ (module name |
233 |
``shibboleth``) |
234 |
* `Google <https://developers.google.com/accounts/docs/OAuth2>`_ (module |
235 |
name ``google``) |
236 |
* `Twitter <https://dev.twitter.com/docs/auth>`_ (module name ``twitter``) |
237 |
* `LinkedIn <http://developer.linkedin.com/documents/authentication>`_ |
238 |
(module name ``linkedin``) |
239 |
|
240 |
To enable any of the above modules (by default only ``local`` accounts are |
241 |
allowed), retrieve and set the required provider settings and append the |
242 |
module name in ``ASTAKOS_IM_MODULES``. |
243 |
|
244 |
.. code-block:: python |
245 |
|
246 |
# settings from https://code.google.com/apis/console/ |
247 |
ASTAKOS_GOOGLE_CLIENT_ID = '1111111111-epi60tvimgha63qqnjo40cljkojcann3.apps.googleusercontent.com' |
248 |
ASTAKOS_GOOGLE_SECRET = 'tNDQqTDKlTf7_LaeUcWTWwZM' |
249 |
|
250 |
# let users signup and login using their google account |
251 |
ASTAKOS_IM_MODULES = ['local', 'google'] |
252 |
|
253 |
|
254 |
.. _auth_methods_policies: |
255 |
|
256 |
Authentication method policies |
257 |
`````````````````````````````` |
258 |
|
259 |
Astakos allows you to override the default policies for each enabled provider |
260 |
separately by adding the approriate settings in your ``.conf`` files in the |
261 |
following format: |
262 |
|
263 |
**ASTAKOS_AUTH_PROVIDER_<module>_<policy>_POLICY** |
264 |
|
265 |
Available policies are: |
266 |
|
267 |
* **CREATE** Users can signup using that provider (default: ``True``) |
268 |
* **REMOVE/ADD** Users can remove/add login method from their profile |
269 |
(default: ``True``) |
270 |
* **AUTOMODERATE** Automatically activate users that signup using that |
271 |
provider (default: ``False``) |
272 |
* **LOGIN** Whether or not users can use the provider to login (default: |
273 |
``True``). |
274 |
|
275 |
e.g. to enable automatic activation for your academic users, while keeping |
276 |
locally signed up users under moderation you can apply the following settings. |
277 |
|
278 |
.. code-block:: python |
279 |
|
280 |
ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_AUTOMODERATE_POLICY = True |
281 |
ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_REMOVE_POLICY = False |
282 |
|
283 |
User login |
284 |
~~~~~~~~~~ |
285 |
|
286 |
During the logging procedure, the user is authenticated by the respective |
287 |
identity provider. |
288 |
|
289 |
If ``ASTAKOS_RECAPTCHA_ENABLED`` is set and the user fails several times |
290 |
(``ASTAKOS_RATELIMIT_RETRIES_ALLOWED`` setting) to provide the correct |
291 |
credentials for a local account, he/she is then prompted to solve a captcha |
292 |
challenge. |
293 |
|
294 |
Upon success, the system renews the token (if it has expired), logins the user |
295 |
and sets the cookie, before redirecting the user to the ``next`` parameter |
296 |
value. |
297 |
|
298 |
Setting quota limits |
299 |
~~~~~~~~~~~~~~~~~~~~ |
300 |
|
301 |
Set default quota |
302 |
````````````````` |
303 |
To inspect current default base quota limits, run:: |
304 |
|
305 |
# snf-manage resource-list |
306 |
|
307 |
You can modify the default base quota limit for all future users with:: |
308 |
|
309 |
# snf-manage resource-modify <resource_name> --default-quota <value> |
310 |
|
311 |
Set base quota for individual users |
312 |
``````````````````````````````````` |
313 |
|
314 |
For individual users that need different quota than the default |
315 |
you can set it for each resource like this:: |
316 |
|
317 |
# use this to display quota / uuid |
318 |
# snf-manage user-show 'uuid or email' --quota |
319 |
|
320 |
# snf-manage user-modify <user-uuid> --base-quota 'cyclades.vm' 10 |
321 |
|
322 |
You can set base quota for all existing users, with possible exceptions, using:: |
323 |
|
324 |
# snf-manage user-modify --all --base-quota cyclades.vm 10 --exclude uuid1,uuid2 |
325 |
|
326 |
All quota for which values different from the default have been set, |
327 |
can be listed with:: |
328 |
|
329 |
# snf-manage quota-list --with-custom=True |
330 |
|
331 |
|
332 |
Enable the Projects feature |
333 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
334 |
|
335 |
If you want to enable the projects feature so that users may apply |
336 |
on their own for resources by creating and joining projects, |
337 |
in ``20-snf-astakos-app-settings.conf`` set:: |
338 |
|
339 |
# this will make the 'projects' page visible in the dashboard |
340 |
ASTAKOS_PROJECTS_VISIBLE = True |
341 |
|
342 |
You can change the maximum allowed number of pending project applications |
343 |
per user with:: |
344 |
|
345 |
# snf-manage resource-modify astakos.pending_app --default-quota <number> |
346 |
|
347 |
You can also set a user-specific limit with:: |
348 |
|
349 |
# snf-manage user-modify <user-uuid> --base-quota 'astakos.pending_app' 5 |
350 |
|
351 |
When users apply for projects they are not automatically granted |
352 |
the resources. They must first be approved by the administrator. |
353 |
|
354 |
To list pending project applications in astakos:: |
355 |
|
356 |
# snf-manage project-list --pending |
357 |
|
358 |
Note the last column, the application id. To approve it:: |
359 |
|
360 |
# <app id> from the last column of project-list |
361 |
# snf-manage project-control --approve <app id> |
362 |
|
363 |
To deny an application:: |
364 |
|
365 |
# snf-manage project-control --deny <app id> |
366 |
|
367 |
Users designated as *project admins* can approve, deny, or modify |
368 |
an application through the web interface. In |
369 |
``20-snf-astakos-app-settings.conf`` set:: |
370 |
|
371 |
# UUIDs of users that can approve or deny project applications from the web. |
372 |
ASTAKOS_PROJECT_ADMINS = [<uuid>, ...] |
373 |
|
374 |
|
375 |
Astakos advanced operations |
376 |
--------------------------- |
377 |
|
378 |
Adding "Terms of Use" |
379 |
~~~~~~~~~~~~~~~~~~~~~ |
380 |
|
381 |
Astakos supports versioned terms-of-use. First of all you need to create an |
382 |
html file that will contain your terms. For example, create the file |
383 |
``/usr/share/synnefo/sample-terms.html``, which contains the following: |
384 |
|
385 |
.. code-block:: console |
386 |
|
387 |
<h1>My cloud service terms</h1> |
388 |
|
389 |
These are the example terms for my cloud service |
390 |
|
391 |
Then, add those terms-of-use with the snf-manage command: |
392 |
|
393 |
.. code-block:: console |
394 |
|
395 |
$ snf-manage term-add /usr/share/synnefo/sample-terms.html |
396 |
|
397 |
Your terms have been successfully added and you will see the corresponding link |
398 |
appearing in the Astakos web pages' footer. |
399 |
|
400 |
During the account registration, if there are approval terms, the user is |
401 |
presented with an "I agree with the Terms" checkbox that needs to get checked |
402 |
in order to proceed. |
403 |
|
404 |
In case there are new approval terms that the user has not signed yet, the |
405 |
``signed_terms_required`` view decorator redirects to the ``approval_terms`` |
406 |
view, so the user will be presented with the new terms the next time he/she |
407 |
logins. |
408 |
|
409 |
Enabling reCAPTCHA |
410 |
~~~~~~~~~~~~~~~~~~ |
411 |
|
412 |
Astakos supports the `reCAPTCHA <http://www.google.com/recaptcha>`_ feature. |
413 |
If enabled, it protects the Astakos forms from bots. To enable the feature, go |
414 |
to https://www.google.com/recaptcha/admin/create and create your own reCAPTCHA |
415 |
key pair. Then edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` and set |
416 |
the corresponding variables to reflect your newly created key pair. Finally, set |
417 |
the ``ASTAKOS_RECAPTCHA_ENABLED`` variable to ``True``: |
418 |
|
419 |
.. code-block:: console |
420 |
|
421 |
ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*(' |
422 |
ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*(' |
423 |
|
424 |
ASTAKOS_RECAPTCHA_ENABLED = True |
425 |
|
426 |
Restart the service on the Astakos node(s) and you are ready: |
427 |
|
428 |
.. code-block:: console |
429 |
|
430 |
# /etc/init.d/gunicorn restart |
431 |
|
432 |
Checkout your new Sign up page. If you see the reCAPTCHA box, you have setup |
433 |
everything correctly. |
434 |
|
435 |
|
436 |
Astakos internals |
437 |
----------------- |
438 |
|
439 |
X-Auth-Token |
440 |
~~~~~~~~~~~~ |
441 |
|
442 |
Alice requests a specific resource from a cloud service e.g.: Pithos. In the |
443 |
request she supplies the `X-Auth-Token` to identify whether she is eligible to |
444 |
perform the specific task. The service contacts Astakos through its |
445 |
``/account/v1.0/authenticate`` api call (see :ref:`authenticate-api-label`) |
446 |
providing the specific ``X-Auth-Token``. Astakos checkes whether the token |
447 |
belongs to an active user and it has not expired and returns a dictionary |
448 |
containing user related information. Finally the service uses the ``uniq`` |
449 |
field included in the dictionary as the account string to identify the user |
450 |
accessible resources. |
451 |
|
452 |
.. _authentication-label: |
453 |
|
454 |
Django Auth methods and Backends |
455 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
456 |
|
457 |
Astakos incorporates Django user authentication system and extends its User model. |
458 |
|
459 |
Since username field of django User model has a limitation of 30 characters, |
460 |
AstakosUser is **uniquely** identified by the ``email`` instead. Therefore, |
461 |
``astakos.im.authentication_backends.EmailBackend`` is served to authenticate a |
462 |
user using email if the first argument is actually an email, otherwise tries |
463 |
the username. |
464 |
|
465 |
A new AstakosUser instance is assigned with a uui as username and also with a |
466 |
``auth_token`` used by the cloud services to authenticate the user. |
467 |
``astakos.im.authentication_backends.TokenBackend`` is also specified in order |
468 |
to authenticate the user using the email and the token fields. |
469 |
|
470 |
Logged on users can perform a number of actions: |
471 |
|
472 |
* access and edit their profile via: ``/im/profile``. |
473 |
* change their password via: ``/im/password`` |
474 |
* send feedback for grnet services via: ``/im/send_feedback`` |
475 |
* logout (and delete cookie) via: ``/im/logout`` |
476 |
|
477 |
Internal Astakos requests are handled using cookie-based Django user sessions. |
478 |
|
479 |
External systems should forward to the ``/login`` URI. The server, |
480 |
depending on its configuration will redirect to the appropriate login page. |
481 |
When done with logging in, the service's login URI should redirect to the URI |
482 |
provided with next, adding user and token parameters, which contain the email |
483 |
and token fields respectively. |
484 |
|
485 |
The login URI accepts the following parameters: |
486 |
|
487 |
====================== ========================= |
488 |
Request Parameter Name Value |
489 |
====================== ========================= |
490 |
next The URI to redirect to when the process is finished |
491 |
renew Force token renewal (no value parameter) |
492 |
force Force logout current user (no value parameter) |
493 |
====================== ========================= |
494 |
|
495 |
External systems inside the ``ASTAKOS_COOKIE_DOMAIN`` scope can acquire the |
496 |
user information by the cookie identified by ``ASTAKOS_COOKIE_NAME`` setting |
497 |
(set during the login procedure). |
498 |
|
499 |
Finally, backend systems having acquired a token can use the |
500 |
:ref:`authenticate-api-label` API call from a private network or through HTTPS. |
501 |
|
502 |
|
503 |
Compute/Network/Image Service (Cyclades) |
504 |
======================================== |
505 |
|
506 |
Working with Cyclades |
507 |
--------------------- |
508 |
|
509 |
Managing Ganeti Backends |
510 |
~~~~~~~~~~~~~~~~~~~~~~~~ |
511 |
|
512 |
Since v0.11, Synnefo is able to manage multiple Ganeti clusters (backends) |
513 |
making it capable to scale linearly to tens of thousands of VMs. Backends |
514 |
can be dynamically added or removed via `snf-manage` commands. |
515 |
|
516 |
Each newly created VM is allocated to a Ganeti backend by the Cyclades backend |
517 |
allocator. The VM is "pinned" to this backend, and can not change through its |
518 |
lifetime. The backend allocator decides in which backend to spawn the VM based |
519 |
on the available resources of each backend, trying to balance the load between |
520 |
them. |
521 |
|
522 |
Handling of Networks, as far as backends are concerned, is based on whether the |
523 |
network is public or not. Public networks are created through the `snf-manage |
524 |
network-create` command, and are only created on one backend. Private networks |
525 |
are created on all backends, in order to ensure that VMs residing on different |
526 |
backends can be connected to the same private network. |
527 |
|
528 |
Listing existing backends |
529 |
````````````````````````` |
530 |
To list all the Ganeti backends known to Synnefo, we run: |
531 |
|
532 |
.. code-block:: console |
533 |
|
534 |
$ snf-manage backend-list |
535 |
|
536 |
Adding a new Ganeti backend |
537 |
``````````````````````````` |
538 |
Backends are dynamically added under the control of Synnefo with `snf-manage |
539 |
backend-add` command. In this section it is assumed that a Ganeti cluster, |
540 |
named ``cluster.example.com`` is already up and running and configured to be |
541 |
able to host Synnefo VMs. |
542 |
|
543 |
To add this Ganeti cluster, we run: |
544 |
|
545 |
.. code-block:: console |
546 |
|
547 |
$ snf-manage backend-add --clustername=cluster.example.com --user="synnefo_user" --pass="synnefo_pass" |
548 |
|
549 |
where ``clustername`` is the Cluster hostname of the Ganeti cluster, and |
550 |
``user`` and ``pass`` are the credentials for the `Ganeti RAPI user |
551 |
<http://docs.ganeti.org/ganeti/2.2/html/rapi.html#users-and-passwords>`_. All |
552 |
backend attributes can be also changed dynamically using the `snf-manage |
553 |
backend-modify` command. |
554 |
|
555 |
``snf-manage backend-add`` will also create all existing private networks to |
556 |
the new backend. You can verify that the backend is added, by running |
557 |
`snf-manage backend-list`. |
558 |
|
559 |
Note that no VMs will be spawned to this backend, since by default it is in a |
560 |
``drained`` state after addition and also it has no public network assigned to |
561 |
it. |
562 |
|
563 |
So, first you need to create its public network, make sure everything works as |
564 |
expected and finally make it active by un-setting the ``drained`` flag. You can |
565 |
do this by running: |
566 |
|
567 |
.. code-block:: console |
568 |
|
569 |
$ snf-manage backend-modify --drained=False <backend_id> |
570 |
|
571 |
Removing an existing Ganeti backend |
572 |
``````````````````````````````````` |
573 |
In order to remove an existing backend from Synnefo, we run: |
574 |
|
575 |
.. code-block:: console |
576 |
|
577 |
# snf-manage backend-remove <backend_id> |
578 |
|
579 |
This command will fail if there are active VMs on the backend. Also, the |
580 |
backend is not cleaned before removal, so all the Synnefo private networks |
581 |
will be left on the Ganeti nodes. You need to remove them manually. |
582 |
|
583 |
Allocation of VMs in Ganeti backends |
584 |
```````````````````````````````````` |
585 |
As already mentioned, the Cyclades backend allocator is responsible for |
586 |
allocating new VMs to backends. This allocator does not choose the exact Ganeti |
587 |
node that will host the VM but just the Ganeti backend. The exact node is |
588 |
chosen by the Ganeti cluster's allocator (hail). |
589 |
|
590 |
The decision about which backend will host a VM is based on the available |
591 |
resources. The allocator computes a score for each backend, that shows its load |
592 |
factor, and the one with the minimum score is chosen. The admin can exclude |
593 |
backends from the allocation phase by marking them as ``drained`` by running: |
594 |
|
595 |
.. code-block:: console |
596 |
|
597 |
$ snf-manage backend-modify --drained=True <backend_id> |
598 |
|
599 |
The backend resources are periodically updated, at a period defined by |
600 |
the ``BACKEND_REFRESH_MIN`` setting, or by running `snf-manage backend-update-status` |
601 |
command. It is advised to have a cron job running this command at a smaller |
602 |
interval than ``BACKEND_REFRESH_MIN`` in order to remove the load of refreshing |
603 |
the backends stats from the VM creation phase. |
604 |
|
605 |
Finally, the admin can decide to have a user's VMs being allocated to a |
606 |
specific backend, with the ``BACKEND_PER_USER`` setting. This is a mapping |
607 |
between users and backends. If the user is found in ``BACKEND_PER_USER``, then |
608 |
Synnefo allocates all his/hers VMs to the specific backend in the variable, |
609 |
even if is marked as drained (useful for testing). |
610 |
|
611 |
Allocation based on disk-templates |
612 |
********************************** |
613 |
|
614 |
Besides the available resources of each Ganeti backend, the allocator takes |
615 |
into consideration the disk template of the instance when trying to allocate it |
616 |
to a Ganeti backend. Specifically, the allocator checks if the flavor of the |
617 |
instance belongs to the available disk templates of each Ganeti backend. |
618 |
|
619 |
A Ganeti cluster has a list of enabled disk templates |
620 |
(`--enabled-disk-templates`) and a list of allowed disk templates for new |
621 |
instances (`--ipolicy-disk-templates`). See the `gnt-cluster` manpage for more |
622 |
details about these options. |
623 |
|
624 |
When Synnefo allocates an instance, it checks whether the disk template of the |
625 |
new instance belongs both in the enabled and ipolicy disk templates. You can |
626 |
see the list of the available disk-templates by running `snf-manage |
627 |
backend-list`. This list should be updated automatically after changing |
628 |
these options in Ganeti and it can also be updated by running `snf-manage |
629 |
backend-update-status`. |
630 |
|
631 |
So the administrator, can route instances on different backends based on their |
632 |
flavor disk template, by modifying the enabled or ipolicy disk templates of |
633 |
each backend. Also, the administrator can route instances between different |
634 |
nodes of the same Ganeti backend, by modifying the same options at the |
635 |
nodegroup level (see `gnt-group` manpage for mor details). |
636 |
|
637 |
|
638 |
Managing Virtual Machines |
639 |
~~~~~~~~~~~~~~~~~~~~~~~~~ |
640 |
|
641 |
As mentioned, Cyclades uses Ganeti for management of VMs. The administrator can |
642 |
handle Cyclades VMs just like any other Ganeti instance, via `gnt-instance` |
643 |
commands. All Ganeti instances that belong to Synnefo, are separated from |
644 |
others, by a prefix in their names. This prefix is defined in |
645 |
``BACKEND_PREFIX_ID`` setting in |
646 |
``/etc/synnefo/20-snf-cyclades-app-backend.conf``. |
647 |
|
648 |
Apart from handling instances directly in the Ganeti level, a number of `snf-manage` |
649 |
commands are available: |
650 |
|
651 |
* ``snf-manage server-list``: List servers |
652 |
* ``snf-manage server-show``: Show information about a server in the Cyclades DB |
653 |
* ``snf-manage server-inspect``: Inspect the state of a server both in DB and Ganeti |
654 |
* ``snf-manage server-modify``: Modify the state of a server in the Cycldes DB |
655 |
* ``snf-manage server-create``: Create a new server |
656 |
* ``snf-manage server-import``: Import an existing Ganeti instance to Cyclades |
657 |
|
658 |
|
659 |
Managing Virtual Networks |
660 |
~~~~~~~~~~~~~~~~~~~~~~~~~ |
661 |
|
662 |
Cyclades is able to create and manage Virtual Networks. Networking is |
663 |
desployment specific and must be customized based on the specific needs of the |
664 |
system administrator. For better understanding of networking please refer to |
665 |
the :ref:`Network <networks>` section. |
666 |
|
667 |
Exactly as Cyclades VMs can be handled like Ganeti instances, Cyclades Networks |
668 |
can also by handled as Ganeti networks, via `gnt-network commands`. All Ganeti |
669 |
networks that belong to Synnefo are named with the prefix |
670 |
`${BACKEND_PREFIX_ID}-net-`. |
671 |
|
672 |
There are also the following `snf-manage` commands for managing networks: |
673 |
|
674 |
* ``snf-manage network-list``: List networks |
675 |
* ``snf-manage network-show``: Show information about a network in the Cyclades DB |
676 |
* ``snf-manage network-inspect``: Inspect the state of the network in DB and Ganeti backends |
677 |
* ``snf-manage network-modify``: Modify the state of a network in the Cycldes DB |
678 |
* ``snf-manage network-create``: Create a new network |
679 |
* ``snf-manage network-remove``: Remove an existing network |
680 |
|
681 |
Managing Network Resources |
682 |
`````````````````````````` |
683 |
|
684 |
Proper operation of the Cyclades Network Service depends on the unique |
685 |
assignment of specific resources to each type of virtual network. Specifically, |
686 |
these resources are: |
687 |
|
688 |
* IP addresses. Cyclades creates a Pool of IPs for each Network, and assigns a |
689 |
unique IP address to each VM, thus connecting it to this Network. You can see |
690 |
the IP pool of each network by running `snf-manage network-inspect |
691 |
<network_ID>`. IP pools are automatically created and managed by Cyclades, |
692 |
depending on the subnet of the Network. |
693 |
* Bridges corresponding to physical VLANs, which are required for networks of |
694 |
type `PRIVATE_PHYSICAL_VLAN`. |
695 |
* One Bridge corresponding to one physical VLAN which is required for networks of |
696 |
type `PRIVATE_MAC_PREFIX`. |
697 |
|
698 |
IPv4 addresses |
699 |
************** |
700 |
|
701 |
An allocation pool of IPv4 addresses is automatically created for every network |
702 |
that has the attribute `dhcp` set to True. The allocation pool contains the |
703 |
range of IP addresses that are included in the subnet. The gateway and the |
704 |
broadcast address of the network are excluded from the allocation pool. The |
705 |
admin can externally reserve IP addresses to exclude them from automatic |
706 |
allocation with the `--add-reserved-ips` option of `snf-manage network-modify` |
707 |
command. For example the following command will reserve two IP addresses |
708 |
from network with ID `42`: |
709 |
|
710 |
.. code-block:: console |
711 |
|
712 |
snf-manage network-modify --add-reserved-ips=10.0.0.21,10.0.0.22 42 |
713 |
|
714 |
.. warning:: Externally reserving IP addresses is also available at the Ganeti. |
715 |
However, when using Cyclades with multiple Ganeti backends, the handling of |
716 |
IP pools must be performed from Cyclades! |
717 |
|
718 |
Bridges |
719 |
******* |
720 |
|
721 |
As already mentioned Cyclades use a pool of Bridges that must correspond |
722 |
to Physical VLAN at the Ganeti level. A bridge from the pool is assigned to |
723 |
each network of flavor `PHYSICAL_VLAN`. Creation of this pool is done |
724 |
using `snf-manage pool-create` command. For example the following command |
725 |
will create a pool containing the brdiges from `prv1` to `prv21`. |
726 |
|
727 |
.. code-block:: console |
728 |
|
729 |
# snf-manage pool-create --type=bridge --base=prv --size=20 |
730 |
|
731 |
You can verify the creation of the pool, and check its contents by running: |
732 |
|
733 |
.. code-block:: console |
734 |
|
735 |
# snf-manage pool-list |
736 |
# snf-manage pool-show --type=bridge 1 |
737 |
|
738 |
Finally you can use the `pool-modify` management command in order to externally |
739 |
reserve the values from pool, extend or shrink the pool if possible. |
740 |
|
741 |
MAC Prefixes |
742 |
************ |
743 |
|
744 |
Cyclades also use a pool of MAC prefixes to assign to networks of flavor |
745 |
`MAC_FILTERED`. Handling of this pool is done exactly as with pool of bridges, |
746 |
except that the type option must be set to mac-prefix: |
747 |
|
748 |
.. code-block:: console |
749 |
|
750 |
# snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536 |
751 |
|
752 |
The above command will create a pool of MAC prefixes from ``aa:00:1`` to |
753 |
``b9:ff:f``. The MAC prefix pool is responsible for providing only unicast and |
754 |
locally administered MAC addresses, so many of these prefixes will be |
755 |
externally reserved, to exclude from allocation. |
756 |
|
757 |
Pool reconciliation |
758 |
******************* |
759 |
|
760 |
The management command `snf-manage reconcile-pools` can be used that all the |
761 |
above mentioned pools are consistent and that all values that come from the |
762 |
pool are not used more than once. |
763 |
|
764 |
|
765 |
Cyclades advanced operations |
766 |
---------------------------- |
767 |
|
768 |
Reconciliation mechanism |
769 |
~~~~~~~~~~~~~~~~~~~~~~~~ |
770 |
|
771 |
On certain occasions, such as a Ganeti or RabbitMQ failure, the state of |
772 |
Cyclades database may differ from the real state of VMs and networks in the |
773 |
Ganeti backends. The reconciliation process is designed to synchronize |
774 |
the state of the Cyclades DB with Ganeti. There are two management commands |
775 |
for reconciling VMs and Networks |
776 |
|
777 |
Reconciling Virtual Machines |
778 |
```````````````````````````` |
779 |
|
780 |
Reconciliation of VMs detects the following conditions: |
781 |
|
782 |
* Stale DB servers without corresponding Ganeti instances |
783 |
* Orphan Ganeti instances, without corresponding DB entries |
784 |
* Out-of-sync state for DB entries wrt to Ganeti instances |
785 |
|
786 |
To detect all inconsistencies you can just run: |
787 |
|
788 |
.. code-block:: console |
789 |
|
790 |
$ snf-manage reconcile-servers |
791 |
|
792 |
Adding the `--fix-all` option, will do the actual synchronization: |
793 |
|
794 |
.. code-block:: console |
795 |
|
796 |
$ snf-manage reconcile-servers --fix-all |
797 |
|
798 |
Please see ``snf-manage reconcile-servers --help`` for all the details. |
799 |
|
800 |
Reconciling Networks |
801 |
```````````````````` |
802 |
|
803 |
Reconciliation of Networks detects the following conditions: |
804 |
|
805 |
* Stale DB networks without corresponding Ganeti networks |
806 |
* Orphan Ganeti networks, without corresponding DB entries |
807 |
* Private networks that are not created to all Ganeti backends |
808 |
* Unsynchronized IP pools |
809 |
|
810 |
To detect all inconsistencies you can just run: |
811 |
|
812 |
.. code-block:: console |
813 |
|
814 |
$ snf-manage reconcile-networks |
815 |
|
816 |
Adding the `--fix-all` option, will do the actual synchronization: |
817 |
|
818 |
.. code-block:: console |
819 |
|
820 |
$ snf-manage reconcile-networks --fix-all |
821 |
|
822 |
Please see ``snf-manage reconcile-networks --help`` for all the details. |
823 |
|
824 |
|
825 |
Cyclades internals |
826 |
------------------ |
827 |
|
828 |
Asynchronous communication with Ganeti backends |
829 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
830 |
Synnefo uses Google Ganeti backends for VM cluster management. In order for |
831 |
Cyclades to be able to handle thousands of user requests, Cyclades and Ganeti |
832 |
communicate asynchronously. Briefly, requests are submitted to Ganeti through |
833 |
Ganeti's RAPI/HTTP interface, and then asynchronous notifications about the |
834 |
progress of Ganeti jobs are being created and pushed upwards to Cyclades. The |
835 |
architecture and communication with a Ganeti backend is shown in the graph |
836 |
below: |
837 |
|
838 |
.. image:: images/cyclades-ganeti-communication.png |
839 |
:width: 50% |
840 |
:target: _images/cyclades-ganeti-communication.png |
841 |
|
842 |
The Cyclades API server is responsible for handling user requests. Read-only |
843 |
requests are directly served by looking up the Cyclades DB. If the request |
844 |
needs an action in the Ganeti backend, Cyclades submit jobs to the Ganeti |
845 |
master using the `Ganeti RAPI interface |
846 |
<http://docs.ganeti.org/ganeti/2.2/html/rapi.html>`_. |
847 |
|
848 |
While Ganeti executes the job, `snf-ganeti-eventd`, `snf-ganeti-hook` and |
849 |
`snf-progress-monitor` are monitoring the progress of the job and send |
850 |
corresponding messages to the RabbitMQ servers. These components are part |
851 |
of `snf-cyclades-gtools` and must be installed on all Ganeti nodes. Specially: |
852 |
|
853 |
* *snf-ganeti-eventd* sends messages about operations affecting the operating |
854 |
state of instances and networks. Works by monitoring the Ganeti job queue. |
855 |
* *snf-ganeti_hook* sends messages about the NICs of instances. It includes a |
856 |
number of `Ganeti hooks <http://docs.ganeti.org/ganeti/2.2/html/hooks.html>`_ |
857 |
for customisation of operations. |
858 |
* *snf-progress_monitor* sends messages about the progress of the Image deployment |
859 |
phase which is done by the Ganeti OS Definition `snf-image`. |
860 |
|
861 |
Finally, `snf-dispatcher` consumes messages from the RabbitMQ queues, processes |
862 |
these messages and properly updates the state of the Cyclades DB. Subsequent |
863 |
requests to the Cyclades API, will retrieve the updated state from the DB. |
864 |
|
865 |
|
866 |
Synnefo management commands ("snf-manage") |
867 |
========================================== |
868 |
|
869 |
Each Synnefo service, Astakos, Pithos and Cyclades are controlled by the |
870 |
administrator using the "snf-manage" admin tool. This tool is an extension of |
871 |
the Django command-line management utility. It is run on the host that runs |
872 |
each service and provides different types of commands depending the services |
873 |
running on the host. If you are running more than one service on the same host |
874 |
"snf-manage" adds all the corresponding commands for each service dynamically, |
875 |
providing a unified admin environment. |
876 |
|
877 |
To run "snf-manage" you just type: |
878 |
|
879 |
.. code-block:: console |
880 |
|
881 |
# snf-manage <command> [arguments] |
882 |
|
883 |
on the corresponding host that runs the service. For example, if you have all |
884 |
services running on different physical hosts you would do: |
885 |
|
886 |
.. code-block:: console |
887 |
|
888 |
root@astakos-host # snf-manage <astakos-command> [argument] |
889 |
root@pithos-host # snf-manage <pithos-command> [argument] |
890 |
root@cyclades-host # snf-manage <cyclades-command> [argument] |
891 |
|
892 |
If you have all services running on the same host you would do: |
893 |
|
894 |
.. code-block:: console |
895 |
|
896 |
root@synnefo-host # snf-manage <{astakos,pithos,cyclades}-command> [argument] |
897 |
|
898 |
Note that you cannot execute a service's command on a host that is not running |
899 |
this service. For example, the following will return an error if Astakos and |
900 |
Cyclades are installed on different physical hosts: |
901 |
|
902 |
.. code-block:: console |
903 |
|
904 |
root@astakos-host # snf-manage <cyclades-command> [argument] |
905 |
Unknown command: 'cyclades-command' |
906 |
Type 'snf-manage help' for usage. |
907 |
|
908 |
This is the complete list of "snf-manage" commands for each service. |
909 |
|
910 |
Astakos snf-manage commands |
911 |
--------------------------- |
912 |
|
913 |
============================ =========================== |
914 |
Name Description |
915 |
============================ =========================== |
916 |
fix-superusers Transform superusers created by syncdb into AstakosUser instances |
917 |
cleanup-full Cleanup sessions and session catalog |
918 |
commission-list List pending commissions |
919 |
commission-show Show details for a pending commission |
920 |
component-add Register a component |
921 |
component-list List components |
922 |
component-modify Modify component attributes |
923 |
component-show Show component details |
924 |
project-control Manage projects and applications |
925 |
project-list List projects |
926 |
project-show Show project details |
927 |
quota-list List user quota |
928 |
quota-verify Check the integrity of user quota |
929 |
reconcile-resources-astakos Reconcile resource usage of Quotaholder with Astakos DB |
930 |
resource-list List resources |
931 |
resource-modify Modify a resource's default base quota and boolean flags |
932 |
service-export-astakos Export Astakos services and resources in JSON format |
933 |
service-import Register services |
934 |
service-list List services |
935 |
service-show Show service details |
936 |
term-add Add approval terms |
937 |
user-activation-send Send user activation |
938 |
user-add Add user |
939 |
authpolicy-add Create a new authentication provider policy profile |
940 |
authpolicy-list List existing authentication provider policy profiles |
941 |
authpolicy-remove Remove an authentication provider policy |
942 |
authpolicy-set Assign an existing authentication provider policy profile to a user or group |
943 |
authpolicy-show Show authentication provider profile details |
944 |
group-add Create a group with the given name |
945 |
group-list List available groups |
946 |
user-list List users |
947 |
user-modify Modify user |
948 |
user-show Show user details |
949 |
============================ =========================== |
950 |
|
951 |
Pithos snf-manage commands |
952 |
-------------------------- |
953 |
|
954 |
============================ =========================== |
955 |
Name Description |
956 |
============================ =========================== |
957 |
reconcile-commissions-pithos Display unresolved commissions and trigger their recovery |
958 |
service-export-pithos Export Pithos services and resources in JSON format |
959 |
reconcile-resources-pithos Detect unsynchronized usage between Astakos and Pithos DB resources and synchronize them if specified so. |
960 |
============================ =========================== |
961 |
|
962 |
Cyclades snf-manage commands |
963 |
---------------------------- |
964 |
|
965 |
============================== =========================== |
966 |
Name Description |
967 |
============================== =========================== |
968 |
backend-add Add a new Ganeti backend |
969 |
backend-list List backends |
970 |
backend-modify Modify a backend |
971 |
backend-update-status Update backend statistics for instance allocation |
972 |
backend-remove Remove a Ganeti backend |
973 |
enforce-resources-cyclades Check and fix quota violations for Cyclades resources |
974 |
server-create Create a new server |
975 |
server-show Show server details |
976 |
server-list List servers |
977 |
server-modify Modify a server |
978 |
server-import Import an existing Ganeti VM into synnefo |
979 |
server-inspect Inspect a server in DB and Ganeti |
980 |
network-create Create a new network |
981 |
network-list List networks |
982 |
network-modify Modify a network |
983 |
network-inspect Inspect network state in DB and Ganeti |
984 |
network-remove Delete a network |
985 |
flavor-create Create a new flavor |
986 |
flavor-list List flavors |
987 |
flavor-modify Modify a flavor |
988 |
image-list List images |
989 |
image-show Show image details |
990 |
pool-create Create a bridge or mac-prefix pool |
991 |
pool-show Show pool details |
992 |
pool-list List pools |
993 |
pool-modify Modify a pool |
994 |
pool-remove Delete a pool |
995 |
port-create Create a port connecting a server to a network |
996 |
port-inspect Inspect the state of a port in DB and Ganeti |
997 |
port-list List ports |
998 |
port-remove Delete a port |
999 |
floating-ip-create Create a new floating IP |
1000 |
floating-ip-attach Attach a floating IP to a server |
1001 |
floating-ip-dettach Dettach a flotaing IP from a server |
1002 |
floating-ip-list List floating IPs |
1003 |
floating-ip-remove Delete a floating IP |
1004 |
queue-inspect Inspect the messages of a RabbitMQ queue |
1005 |
queue-retry Resend messages from Dead Letter queues to original exchanges |
1006 |
service-export-cyclades Export Cyclades services and resources in JSON format |
1007 |
subnet-create Create a subnet |
1008 |
subnet-inspect Inspect a subnet in DB |
1009 |
subnet-list List subnets |
1010 |
subnet-modify Modify a subnet |
1011 |
reconcile-servers Reconcile servers of Synnefo DB with state of Ganeti backend |
1012 |
reconcile-networks Reconcile networks of Synnefo DB with state of Ganeti backend |
1013 |
reconcile-pools Check consistency of pool resources |
1014 |
reconcile-commissions-cyclades Detect and resolve pending commissions to Quotaholder |
1015 |
reconcile-resources-cyclades Reconcile resource usage of Astakos with Cyclades DB. |
1016 |
============================== =========================== |
1017 |
|
1018 |
|
1019 |
Astakos helper scripts |
1020 |
====================== |
1021 |
|
1022 |
Astakos includes two scripts to facilitate the installation procedure. |
1023 |
Running: |
1024 |
|
1025 |
.. code-block:: console |
1026 |
|
1027 |
snf-component-register [<component_name>] |
1028 |
|
1029 |
automates the registration of the standard Synnefo components (astakos, |
1030 |
cyclades, and pithos) in astakos database. It internally uses the script: |
1031 |
|
1032 |
.. code-block:: console |
1033 |
|
1034 |
snf-service-export <component_name> <base_url> |
1035 |
|
1036 |
which simulates the export of service and resource definitions of the |
1037 |
standard Synnefo components. |
1038 |
|
1039 |
|
1040 |
Pithos managing accounts |
1041 |
======================== |
1042 |
|
1043 |
Pithos provides a utility tool for managing accounts. |
1044 |
To run you just type: |
1045 |
|
1046 |
.. code-block:: console |
1047 |
|
1048 |
# pithos-manage-accounts <command> [arguments] |
1049 |
|
1050 |
This is the list of the available commands: |
1051 |
|
1052 |
============================ =========================== |
1053 |
Name Description |
1054 |
============================ =========================== |
1055 |
delete Remove an account from the Pithos DB |
1056 |
export-quota Export account quota in a file |
1057 |
list List existing/dublicate accounts |
1058 |
merge Move an account contents in another account |
1059 |
set-container-quota Set container quota for all or a specific account |
1060 |
============================ =========================== |
1061 |
|
1062 |
|
1063 |
The "kamaki" API client |
1064 |
======================= |
1065 |
|
1066 |
To upload, register or modify an image you will need the **kamaki** tool. |
1067 |
Before proceeding make sure that it is configured properly. Verify that |
1068 |
*image.url*, *file.url*, *user.url* and *token* are set as needed: |
1069 |
|
1070 |
.. code-block:: console |
1071 |
|
1072 |
$ kamaki config list |
1073 |
|
1074 |
To change a setting use ``kamaki config set``: |
1075 |
|
1076 |
.. code-block:: console |
1077 |
|
1078 |
$ kamaki config set image.url https://cyclades.example.com/image |
1079 |
$ kamaki config set file.url https://pithos.example.com/v1 |
1080 |
$ kamaki config set user.url https://accounts.example.com |
1081 |
$ kamaki config set token ... |
1082 |
|
1083 |
To test that everything works, try authenticating the current account with |
1084 |
kamaki: |
1085 |
|
1086 |
.. code-block:: console |
1087 |
|
1088 |
$ kamaki user authenticate |
1089 |
|
1090 |
This will output user information. |
1091 |
|
1092 |
Upload Image |
1093 |
------------ |
1094 |
|
1095 |
By convention, images are stored in a container called ``images``. Check if the |
1096 |
container exists, by listing all containers in your account: |
1097 |
|
1098 |
.. code-block:: console |
1099 |
|
1100 |
$ kamaki file list |
1101 |
|
1102 |
If the container ``images`` does not exist, create it: |
1103 |
|
1104 |
.. code-block:: console |
1105 |
|
1106 |
$ kamaki file create images |
1107 |
|
1108 |
You are now ready to upload an image to container ``images``. You can upload it |
1109 |
with a Pithos client, or use kamaki directly: |
1110 |
|
1111 |
.. code-block:: console |
1112 |
|
1113 |
$ kamaki file upload ubuntu.iso images |
1114 |
|
1115 |
You can use any Pithos client to verify that the image was uploaded correctly, |
1116 |
or you can list the contents of the container with kamaki: |
1117 |
|
1118 |
.. code-block:: console |
1119 |
|
1120 |
$ kamaki file list images |
1121 |
|
1122 |
The full Pithos URL for the previous example will be |
1123 |
``pithos://u53r-un1qu3-1d/images/ubuntu.iso`` where ``u53r-un1qu3-1d`` is the |
1124 |
unique user id (uuid). |
1125 |
|
1126 |
Register Image |
1127 |
-------------- |
1128 |
|
1129 |
To register an image you will need to use the full Pithos URL. To register as |
1130 |
a public image the one from the previous example use: |
1131 |
|
1132 |
.. code-block:: console |
1133 |
|
1134 |
$ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso --public |
1135 |
|
1136 |
The ``--public`` flag is important, if missing the registered image will not |
1137 |
be listed by ``kamaki image list``. |
1138 |
|
1139 |
Use ``kamaki image register`` with no arguments to see a list of available |
1140 |
options. A more complete example would be the following: |
1141 |
|
1142 |
.. code-block:: console |
1143 |
|
1144 |
$ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso \ |
1145 |
--public --disk-format diskdump --property kernel=3.1.2 |
1146 |
|
1147 |
To verify that the image was registered successfully use: |
1148 |
|
1149 |
.. code-block:: console |
1150 |
|
1151 |
$ kamaki image list --name-like=ubuntu |
1152 |
|
1153 |
|
1154 |
Miscellaneous |
1155 |
============= |
1156 |
|
1157 |
.. _branding: |
1158 |
|
1159 |
Branding |
1160 |
-------- |
1161 |
|
1162 |
Since Synnefo v0.14, you are able to adapt the Astakos, Pithos and Cyclades Web |
1163 |
UI to your company’s visual identity. This is possible using the snf-branding |
1164 |
component, which is automatically installed on the nodes running the API |
1165 |
servers for Astakos, Pithos and Cyclades. |
1166 |
|
1167 |
Configuration |
1168 |
~~~~~~~~~~~~~ |
1169 |
|
1170 |
This can be done by modifing the settings provided by the snf-branding component |
1171 |
to match your service identity. The settings for the snf-branding application |
1172 |
can be found inside the configuration file ``/etc/synnefo/15-snf-branding.conf`` |
1173 |
on the nodes that have Astakos, Pithos and Cyclades installed. |
1174 |
|
1175 |
By default, the global service name is "Synnefo" and the company name is |
1176 |
"GRNET". These names and their respective logos and URLs are used throughout |
1177 |
the Astakos, Pithos and Cyclades UI. |
1178 |
|
1179 |
**Names and URLs:** |
1180 |
|
1181 |
The first group of branding customization refers to the service's and company's |
1182 |
information. |
1183 |
|
1184 |
You can overwrite the company and the service name and URL respectively by |
1185 |
uncommenting and setting the following: |
1186 |
|
1187 |
.. code-block:: python |
1188 |
|
1189 |
# setting used in Astakos Dashboard/Projects pages |
1190 |
BRANDING_SERVICE_NAME = 'My cloud' |
1191 |
BRANDING_SERVICE_URL = 'http://www.mycloud.synnefo.org/' |
1192 |
|
1193 |
# settings used in Astakos, Pithos, Cyclades footer only if |
1194 |
# BRANDING_SHOW_COPYRIGHT is set to True |
1195 |
BRANDING_SHOW_COPYRIGHT = True |
1196 |
BRANDING_COMPANY_NAME = 'Company LTD' |
1197 |
BRANDING_COMPANY_URL = 'https://www.company-ltd.synnefo.org/' |
1198 |
|
1199 |
|
1200 |
**Copyright and footer options:** |
1201 |
|
1202 |
By default, no Copyright message is shown in the UI footer. If you want to make |
1203 |
it visible in the footer of Astakos, Pithos and Cyclades UI, you can uncomment |
1204 |
and set to ``True`` the ``BRANDING_SHOW_COPYRIGHT`` setting: |
1205 |
|
1206 |
.. code-block:: python |
1207 |
|
1208 |
#BRANDING_SHOW_COPYRIGHT = False |
1209 |
|
1210 |
Copyright message defaults to 'Copyright (c) 2011-<current_year> |
1211 |
<BRANDING_COMPANY_NAME>.' but you can overwrite it to a completely custom one by |
1212 |
setting the following option: |
1213 |
|
1214 |
.. code-block:: python |
1215 |
|
1216 |
BRANDING_COPYRIGHT_MESSAGE = 'Copyright (c) 2011-2013 GRNET' |
1217 |
|
1218 |
If you want to include a custom message in the footer, you can uncomment and |
1219 |
set the ``BRANDING_FOOTER_EXTRA_MESSAGE`` setting. You can use html markup. |
1220 |
Your custom message will appear above Copyright message at the Compute |
1221 |
templates and the Dashboard UI. |
1222 |
|
1223 |
.. code-block:: python |
1224 |
|
1225 |
#BRANDING_FOOTER_EXTRA_MESSAGE = '' |
1226 |
|
1227 |
|
1228 |
**Images:** |
1229 |
|
1230 |
The Astakos, Pithos and Cyclades Web UI has some logos and images. |
1231 |
|
1232 |
The branding-related images are presented in the following table: |
1233 |
|
1234 |
=============== ============================ ========= |
1235 |
Image Name/extension convention Usage |
1236 |
=============== ============================ ========= |
1237 |
Favicon favicon.ico Favicon for all services |
1238 |
Dashboard logo dashboard_logo.png Visible in all Astakos UI pages |
1239 |
Compute logo compute_logo.png Visible in all Cyclades UI pages |
1240 |
Console logo console_logo.png Visible in the Cyclades Console Window |
1241 |
Storage logo storage_logo.png Visible in all Pithos UI pages |
1242 |
=============== ============================ ========= |
1243 |
|
1244 |
There are two methods available for replacing all, or individual, |
1245 |
branding-related images: |
1246 |
|
1247 |
1. Create a new directory inside ``/usr/share/synnefo/static/`` (e.g. |
1248 |
``mybranding``) and place there some or all of your images. |
1249 |
|
1250 |
If you want to replace all of your images, keep the name/extension |
1251 |
conventions as indicated in the above table and change the |
1252 |
``BRANDING_IMAGE_MEDIA_URL`` setting accordingly: |
1253 |
|
1254 |
.. code-block:: python |
1255 |
|
1256 |
# using relative path |
1257 |
BRANDING_IMAGE_MEDIA_URL= '/static/mybranding/images/' |
1258 |
|
1259 |
# or if you already host them in a separate domain (e.g. cdn) |
1260 |
BRANDING_IMAGE_MEDIA_URL= 'https://cdn.synnefo.org/branding/images/' |
1261 |
|
1262 |
|
1263 |
If you wish to replace individual images, **do not uncomment** |
1264 |
``BRANDING_IMAGE_MEDIA_URL``, but instead provide a relative path, pointing to |
1265 |
the file inside your directory for each ``BRANDING_<image>_URL`` that you wish |
1266 |
to replace. |
1267 |
|
1268 |
2. Upload some or all of your images to a server and replace each |
1269 |
``BRANDING_<image>_URL`` with the absolute url of the image (i.e. |
1270 |
``BRANDING_DASHBOARD_URL = 'https://www.synnefo.com/images/my_dashboard.jpg'``). |
1271 |
|
1272 |
Note that the alternative text for each image tag inside html documents is |
1273 |
alt=“BRANDING_SERVICE_NAME {Dashboard, Compute. Console, Storage}” respectively. |
1274 |
|
1275 |
.. note:: Retina optimized images: |
1276 |
|
1277 |
Synnefo UI is optimized for Retina displays. As far as images are concerned, |
1278 |
`retina.js <http://retinajs.com/>`_ is used. |
1279 |
|
1280 |
Retina.js checks each image on a page to see if there is a high-resolution |
1281 |
version of that image on your server. If a high-resolution variant exists, |
1282 |
the script will swap in that image in-place. |
1283 |
|
1284 |
The script assumes you use `Apple's prescribed high-resolution modifier (@2x) |
1285 |
<http://developer.apple.com/library/ios/#documentation/2DDrawing/Conceptual/ |
1286 |
DrawingPrintingiOS/SupportingHiResScreensInViews/SupportingHiResScreensInViews |
1287 |
.html#//apple_ref/doc/uid/TP40010156-CH15-SW1>`_ to denote high-resolution |
1288 |
image variants on your server. |
1289 |
|
1290 |
For each of the images that you wish the script to replace, you must have a |
1291 |
high-resolution variant in the same folder named correctly and it will be |
1292 |
detected automatically. For example if your image is in <my_directory> and is |
1293 |
named "my_image.jpg" the script will look in the same directory for an image |
1294 |
named "my_image@2x.jpg". |
1295 |
|
1296 |
In case that you don’t want to use a high-resolution image, the |
1297 |
normal-resolution image will be visible. |
1298 |
|
1299 |
More branding |
1300 |
~~~~~~~~~~~~~ |
1301 |
|
1302 |
Although, it is not 100% branding-related, further verbal customization is |
1303 |
feasible. |
1304 |
|
1305 |
**EMAILS** |
1306 |
|
1307 |
The output of all email `*`.txt files will be already customized to contain your |
1308 |
company and service names but you can further alter their content if you feel it |
1309 |
best fits your needs as simple as creasynnefo template. |
1310 |
|
1311 |
In order to overwrite one or more email-templates you need to place your |
1312 |
modified <email-file>.txt files respecting the following structure: |
1313 |
|
1314 |
**/etc/synnefo/templates/** |
1315 |
**im/** |
1316 |
| activation_email.txt |
1317 |
| email.txt |
1318 |
| invitation.txt |
1319 |
| switch_accounts_email.txt |
1320 |
| welcome_email.txt |
1321 |
**projects/** |
1322 |
| project_approval_notification.txt |
1323 |
| project_denial_notification.txt |
1324 |
| project_membership_change_notification.txt |
1325 |
| project_membership_enroll_notification.txt |
1326 |
| project_membership_leave_request_notification.txt |
1327 |
| project_membership_request_notification.txt |
1328 |
| project_suspension_notification.txt |
1329 |
| project_termination_notification.txt |
1330 |
**registration/** |
1331 |
| email_change_email.txt |
1332 |
| password_email.txt |
1333 |
|
1334 |
Feel free to omit any of the above files you do not wish to overwrite. |
1335 |
|
1336 |
Below is a list of all emails sent by Synnefo to users along with a short |
1337 |
description and a link to their content: |
1338 |
|
1339 |
* ``snf-astakos-app/astakos/im/templates/im/email.txt`` |
1340 |
Base email template. Contains a contact email and a “thank you” message. |
1341 |
(`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/email.txt>`_) |
1342 |
* ``snf-astakos-app/astakos/im/templates/im/activation_email.txt`` Email sent to |
1343 |
user that prompts him/her to click on a link provided to activate the account. |
1344 |
Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/activation_email.txt>`_) |
1345 |
* ``snf-astakos-app/astakos/im/templates/im/invitation.txt`` Email sent to an |
1346 |
invited user. He/she has to click on a link provided to activate the account. |
1347 |
Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/invitation.txt>`_) |
1348 |
* ``snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt`` Email |
1349 |
sent to user upon his/her request to associate this email address with a |
1350 |
shibboleth account. He/she has to click on a link provided to activate the |
1351 |
association. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt>`_) |
1352 |
* ``snf-astakos-app/astakos/im/templates/im/welcome_email.txt`` Email sent to |
1353 |
inform the user that his/ her account has been activated. Extends “email.txt” |
1354 |
(`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/welcome_email.txt>`_) |
1355 |
* ``snf-astakos-app/astakos/im/templates/registration/email_change_email.txt`` |
1356 |
Email sent to user when he/she has requested new email address assignment. The |
1357 |
user has to click on a link provided to validate this action. Extends |
1358 |
“email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/email_change_email.txt>`_) |
1359 |
* ``snf-astakos-app/astakos/im/templates/registration/password_email.txt`` Email |
1360 |
sent for resetting password purpose. The user has to click on a link provided |
1361 |
to validate this action. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/password_email.txt>`_) |
1362 |
* ``snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt`` |
1363 |
Informs the project owner that his/her project has been approved. Extends |
1364 |
“email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt>`_) |
1365 |
* ``snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt`` |
1366 |
Informs the project owner that his/her project application has been denied |
1367 |
explaining the reasons. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt>`_) |
1368 |
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt`` |
1369 |
An email is sent to a user containing information about his project membership |
1370 |
(whether he has been accepted, rejected or removed). Extends “email.txt” (`Link |
1371 |
<https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt>`_) |
1372 |
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt`` |
1373 |
Informs a user that he/she has been enrolled to a project. Extends |
1374 |
“email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt>`_) |
1375 |
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt`` |
1376 |
An email is sent to the project owner to make him aware of a user having |
1377 |
requested to leave his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt>`_) |
1378 |
* ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt`` |
1379 |
An email is sent to the project owner to make him/her aware of a user having |
1380 |
requested to join his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt>`_) |
1381 |
* ``snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt`` |
1382 |
An email is sent to the project owner to make him/her aware of his/her project |
1383 |
having been suspended. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt>`_) |
1384 |
* ``snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt`` |
1385 |
An email is sent to the project owner to make him/her aware of his/her project |
1386 |
having been terminated. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt>`_) |
1387 |
|
1388 |
.. warning:: Django templates language: |
1389 |
|
1390 |
If you choose to overwrite these email templates, be mindful of the necessary |
1391 |
information contained in django template variables that must not be omitted, |
1392 |
such as the activation link for activating one’s account and many more. |
1393 |
These variables are contained into {{}} inside the templates. |
1394 |
|
1395 |
|
1396 |
.. RabbitMQ |
1397 |
|
1398 |
RabbitMQ Broker |
1399 |
--------------- |
1400 |
|
1401 |
Queue nodes run the RabbitMQ sofware, which provides AMQP functionality. To |
1402 |
guarantee high-availability, more than one Queue nodes should be deployed, each |
1403 |
of them belonging to the same `RabbitMQ cluster |
1404 |
<http://www.rabbitmq.com/clustering.html>`_. Synnefo uses the RabbitMQ |
1405 |
active/active `High Available Queues <http://www.rabbitmq.com/ha.html>`_ which |
1406 |
are mirrored between two nodes within a RabbitMQ cluster. |
1407 |
|
1408 |
The RabbitMQ nodes that form the cluster, are declared to Synnefo through the |
1409 |
`AMQP_HOSTS` setting. Each time a Synnefo component needs to connect to |
1410 |
RabbitMQ, one of these nodes is chosen in a random way. The client that Synnefo |
1411 |
uses to connect to RabbitMQ, handles connection failures transparently and |
1412 |
tries to reconnect to a different node. As long as one of these nodes are up |
1413 |
and running, functionality of Synnefo should not be downgraded by the RabbitMQ |
1414 |
node failures. |
1415 |
|
1416 |
All the queues that are being used are declared as durable, meaning that |
1417 |
messages are persistently stored to RabbitMQ, until they get successfully |
1418 |
processed by a client. |
1419 |
|
1420 |
Currently, RabbitMQ is used by the following components: |
1421 |
|
1422 |
* `snf-ganeti-eventd`, `snf-ganeti-hook` and `snf-progress-monitor`: |
1423 |
These components send messages concerning the status and progress of |
1424 |
jobs in the Ganeti backend. |
1425 |
* `snf-dispatcher`: This daemon, consumes the messages that are sent from |
1426 |
the above components, and updates the Cyclades DB accordingly. |
1427 |
|
1428 |
|
1429 |
Installation |
1430 |
~~~~~~~~~~~~ |
1431 |
|
1432 |
Please check the RabbitMQ documentation which covers extensively the |
1433 |
`installation of RabbitMQ server <http://www.rabbitmq.com/download.html>`_ and |
1434 |
the setup of a `RabbitMQ cluster <http://www.rabbitmq.com/clustering.html>`_. |
1435 |
Also, check out the `web management plugin |
1436 |
<http://www.rabbitmq.com/management.html>`_ that can be useful for managing and |
1437 |
monitoring RabbitMQ. |
1438 |
|
1439 |
For a basic installation of RabbitMQ on two nodes (node1 and node2) you can do |
1440 |
the following: |
1441 |
|
1442 |
On both nodes, install rabbitmq-server and create a Synnefo user: |
1443 |
|
1444 |
.. code-block:: console |
1445 |
|
1446 |
$ apt-get install rabbitmq-server |
1447 |
$ rabbitmqctl add_user synnefo "example_pass" |
1448 |
$ rabbitmqctl set_permissions synnefo ".*" ".*" ".*" |
1449 |
|
1450 |
Also guarantee that both nodes share the same cookie, by running: |
1451 |
|
1452 |
.. code-block:: console |
1453 |
|
1454 |
$ scp node1:/var/lib/rabbitmq/.erlang.cookie node2:/var/lib/rabbitmq/.erlang.cookie |
1455 |
|
1456 |
and restart the nodes: |
1457 |
|
1458 |
.. code-block:: console |
1459 |
|
1460 |
$ /etc/init.d/rabbitmq-server restart |
1461 |
|
1462 |
|
1463 |
To setup the RabbitMQ cluster run: |
1464 |
|
1465 |
.. code-block:: console |
1466 |
|
1467 |
root@node2: rabbitmqctl stop_app |
1468 |
root@node2: rabbitmqctl reset |
1469 |
root@node2: rabbitmqctl cluster rabbit@node1 rabbit@node2 |
1470 |
root@node2: rabbitmqctl start_app |
1471 |
|
1472 |
You can verify that the cluster is set up correctly by running: |
1473 |
|
1474 |
.. code-block:: console |
1475 |
|
1476 |
root@node2: rabbitmqctl cluster_status |
1477 |
|
1478 |
|
1479 |
Logging |
1480 |
------- |
1481 |
|
1482 |
Logging in Synnefo is using Python's logging module. The module is configured |
1483 |
using dictionary configuration, whose format is described here: |
1484 |
|
1485 |
http://docs.python.org/release/2.7.1/library/logging.html#logging-config-dictschema |
1486 |
|
1487 |
Note that this is a feature of Python 2.7 that we have backported for use in |
1488 |
Python 2.6. |
1489 |
|
1490 |
The logging configuration dictionary is defined in |
1491 |
``/etc/synnefo/10-snf-webproject-logging.conf`` |
1492 |
|
1493 |
The administrator can have finer logging control by modifying the |
1494 |
``LOGGING_SETUP`` dictionary, and defining subloggers with different handlers |
1495 |
and log levels. e.g. To enable debug messages only for the API set the level |
1496 |
of 'synnefo.api' to ``DEBUG`` |
1497 |
|
1498 |
By default, the Django webapp and snf-manage logs to syslog, while |
1499 |
`snf-dispatcher` logs to `/var/log/synnefo/dispatcher.log`. |
1500 |
|
1501 |
|
1502 |
.. _scale-up: |
1503 |
|
1504 |
Scaling up to multiple nodes |
1505 |
============================ |
1506 |
|
1507 |
Here we will describe how should a large scale Synnefo deployment look like. Make |
1508 |
sure you are familiar with Synnefo and Ganeti before proceeding with this section. |
1509 |
This means you should at least have already set up successfully a working Synnefo |
1510 |
deployment as described in the :ref:`Admin's Installation Guide |
1511 |
<quick-install-admin-guide>` and also read the Administrator's Guide until this |
1512 |
section. |
1513 |
|
1514 |
Graph of a scale-out Synnefo deployment |
1515 |
--------------------------------------- |
1516 |
|
1517 |
Each box in the following graph corresponds to a distinct physical node: |
1518 |
|
1519 |
.. image:: images/synnefo-arch2-roles.png |
1520 |
:width: 100% |
1521 |
:target: _images/synnefo-arch2-roles.png |
1522 |
|
1523 |
The above graph is actually the same with the one at the beginning of this |
1524 |
:ref:`guide <admin-guide>`, with the only difference that here we show the |
1525 |
Synnefo roles of each physical node. These roles are described in the |
1526 |
following section. |
1527 |
|
1528 |
.. _physical-node-roles: |
1529 |
|
1530 |
Physical Node roles |
1531 |
------------------- |
1532 |
|
1533 |
As appears in the previous graph, a scale-out Synnefo deployment consists of |
1534 |
multiple physical nodes that have the following roles: |
1535 |
|
1536 |
* **WEBSERVER**: A web server running in front of gunicorn (e.g.: Apache, nginx) |
1537 |
* **ASTAKOS**: The Astakos application (gunicorn) |
1538 |
* **ASTAKOS_DB**: The Astakos database (postgresql) |
1539 |
* **PITHOS**: The Pithos application (gunicorn) |
1540 |
* **PITHOS_DB**: The Pithos database (postgresql) |
1541 |
* **CYCLADES**: The Cyclades application (gunicorn) |
1542 |
* **CYCLADES_DB**: The Cyclades database (postgresql) |
1543 |
* **MQ**: The message queue (RabbitMQ) |
1544 |
* **GANETI_MASTER**: The Ganeti master of a Ganeti cluster |
1545 |
* **GANETI_NODE** : A VM-capable Ganeti node of a Ganeti cluster |
1546 |
|
1547 |
You will probably also have: |
1548 |
|
1549 |
* **CMS**: The CMS used as a frotend portal for the Synnefo services |
1550 |
* **NS**: A nameserver serving all other Synnefo nodes and resolving Synnefo FQDNs |
1551 |
* **CLIENT**: A machine that runs the Synnefo clients (e.g.: kamaki, Web UI), |
1552 |
most of the times, the end user's local machine |
1553 |
|
1554 |
From this point we will also refer to the following groups of roles: |
1555 |
|
1556 |
* **SYNNEFO**: [ **ASTAKOS**, **ASTAKOS_DB**, **PITHOS**, **PITHOS_DB**, **CYCLADES**, **CYCLADES_DB**, **MQ**, **CMS**] |
1557 |
* **G_BACKEND**: [**GANETI_MASTER**, **GANETI_NODE**] |
1558 |
|
1559 |
Of course, when deploying Synnefo you can combine multiple of the above roles on a |
1560 |
single physical node, but if you are trying to scale out, the above separation |
1561 |
gives you significant advantages. |
1562 |
|
1563 |
So, in the next section we will take a look on what components you will have to |
1564 |
install on each physical node depending on its Synnefo role. We assume the graph's |
1565 |
architecture. |
1566 |
|
1567 |
Components for each role |
1568 |
------------------------ |
1569 |
|
1570 |
When deploying Synnefo in large scale, you need to install different Synnefo |
1571 |
or/and third party components on different physical nodes according to their |
1572 |
Synnefo role, as stated in the previous section. |
1573 |
|
1574 |
Specifically: |
1575 |
|
1576 |
Role **WEBSERVER** |
1577 |
* Synnefo components: `None` |
1578 |
* 3rd party components: Apache |
1579 |
Role **ASTAKOS** |
1580 |
* Synnefo components: `snf-webproject`, `snf-astakos-app` |
1581 |
* 3rd party components: Django, Gunicorn |
1582 |
Role **ASTAKOS_DB** |
1583 |
* Synnefo components: `None` |
1584 |
* 3rd party components: PostgreSQL |
1585 |
Role **PITHOS** |
1586 |
* Synnefo components: `snf-webproject`, `snf-pithos-app`, `snf-pithos-webclient` |
1587 |
* 3rd party components: Django, Gunicorn |
1588 |
Role **PITHOS_DB** |
1589 |
* Synnefo components: `None` |
1590 |
* 3rd party components: PostgreSQL |
1591 |
Role **CYCLADES** |
1592 |
* Synnefo components: `snf-webproject`, `snf-cyclades-app`, `snf-vncauthproxy` |
1593 |
* 3rd party components: Django Gunicorn |
1594 |
Role **CYCLADES_DB** |
1595 |
* Synnefo components: `None` |
1596 |
* 3rd party components: PostgreSQL |
1597 |
Role **MQ** |
1598 |
* Synnefo components: `None` |
1599 |
* 3rd party components: RabbitMQ |
1600 |
Role **GANETI_MASTER** |
1601 |
* Synnefo components: `snf-cyclades-gtools` |
1602 |
* 3rd party components: Ganeti |
1603 |
Role **GANETI_NODE** |
1604 |
* Synnefo components: `snf-cyclades-gtools`, `snf-network`, `snf-image`, `nfdhcpd` |
1605 |
* 3rd party components: Ganeti |
1606 |
Role **CMS** |
1607 |
* Synnefo components: `snf-webproject`, `snf-cloudcms` |
1608 |
* 3rd party components: Django, Gunicorn |
1609 |
Role **NS** |
1610 |
* Synnefo components: `None` |
1611 |
* 3rd party components: BIND |
1612 |
Role **CLIENT** |
1613 |
* Synnefo components: `kamaki`, `snf-image-creator` |
1614 |
* 3rd party components: `None` |
1615 |
|
1616 |
Example scale out installation |
1617 |
------------------------------ |
1618 |
|
1619 |
In this section we describe an example of a medium scale installation which |
1620 |
combines multiple roles on 10 different physical nodes. We also provide a |
1621 |
:ref:`guide <i-synnefo>` to help with such an install. |
1622 |
|
1623 |
We assume that we have the following 10 physical nodes with the corresponding |
1624 |
roles: |
1625 |
|
1626 |
Node1: |
1627 |
**WEBSERVER**, **ASTAKOS** |
1628 |
Guide sections: |
1629 |
* :ref:`apt <i-apt>` |
1630 |
* :ref:`gunicorn <i-gunicorn>` |
1631 |
* :ref:`apache <i-apache>` |
1632 |
* :ref:`snf-webproject <i-webproject>` |
1633 |
* :ref:`snf-astakos-app <i-astakos>` |
1634 |
Node2: |
1635 |
**WEBSERVER**, **PITHOS** |
1636 |
Guide sections: |
1637 |
* :ref:`apt <i-apt>` |
1638 |
* :ref:`gunicorn <i-gunicorn>` |
1639 |
* :ref:`apache <i-apache>` |
1640 |
* :ref:`snf-webproject <i-webproject>` |
1641 |
* :ref:`snf-pithos-app <i-pithos>` |
1642 |
* :ref:`snf-pithos-webclient <i-pithos>` |
1643 |
Node3: |
1644 |
**WEBSERVER**, **CYCLADES** |
1645 |
Guide sections: |
1646 |
* :ref:`apt <i-apt>` |
1647 |
* :ref:`gunicorn <i-gunicorn>` |
1648 |
* :ref:`apache <i-apache>` |
1649 |
* :ref:`snf-webproject <i-webproject>` |
1650 |
* :ref:`snf-cyclades-app <i-cyclades>` |
1651 |
* :ref:`snf-vncauthproxy <i-cyclades>` |
1652 |
Node4: |
1653 |
**WEBSERVER**, **CMS** |
1654 |
Guide sections: |
1655 |
* :ref:`apt <i-apt>` |
1656 |
* :ref:`gunicorn <i-gunicorn>` |
1657 |
* :ref:`apache <i-apache>` |
1658 |
* :ref:`snf-webproject <i-webproject>` |
1659 |
* :ref:`snf-cloudcms <i-cms>` |
1660 |
Node5: |
1661 |
**ASTAKOS_DB**, **PITHOS_DB**, **CYCLADES_DB** |
1662 |
Guide sections: |
1663 |
* :ref:`apt <i-apt>` |
1664 |
* :ref:`postgresql <i-db>` |
1665 |
Node6: |
1666 |
**MQ** |
1667 |
Guide sections: |
1668 |
* :ref:`apt <i-apt>` |
1669 |
* :ref:`rabbitmq <i-mq>` |
1670 |
Node7: |
1671 |
**GANETI_MASTER**, **GANETI_NODE** |
1672 |
Guide sections: |
1673 |
* :ref:`apt <i-apt>` |
1674 |
* :ref:`general <i-backends>` |
1675 |
* :ref:`ganeti <i-ganeti>` |
1676 |
* :ref:`snf-cyclades-gtools <i-gtools>` |
1677 |
* :ref:`snf-network <i-network>` |
1678 |
* :ref:`snf-image <i-image>` |
1679 |
* :ref:`nfdhcpd <i-network>` |
1680 |
Node8: |
1681 |
**GANETI_NODE** |
1682 |
Guide sections: |
1683 |
* :ref:`apt <i-apt>` |
1684 |
* :ref:`general <i-backends>` |
1685 |
* :ref:`ganeti <i-ganeti>` |
1686 |
* :ref:`snf-cyclades-gtools <i-gtools>` |
1687 |
* :ref:`snf-network <i-network>` |
1688 |
* :ref:`snf-image <i-image>` |
1689 |
* :ref:`nfdhcpd <i-network>` |
1690 |
Node9: |
1691 |
**GANETI_NODE** |
1692 |
Guide sections: |
1693 |
`Same as Node8` |
1694 |
Node10: |
1695 |
**GANETI_NODE** |
1696 |
Guide sections: |
1697 |
`Same as Node8` |
1698 |
|
1699 |
All sections: :ref:`Scale out Guide <i-synnefo>` |
1700 |
|
1701 |
|
1702 |
Upgrade Notes |
1703 |
============= |
1704 |
|
1705 |
.. toctree:: |
1706 |
:maxdepth: 1 |
1707 |
|
1708 |
v0.12 -> v0.13 <upgrade/upgrade-0.13> |
1709 |
v0.13 -> v0.14 <upgrade/upgrade-0.14> |
1710 |
v0.14 -> v0.14.2 <upgrade/upgrade-0.14.2> |
1711 |
v0.14.5 -> v0.14.6 <upgrade/upgrade-0.14.6> |
1712 |
v0.14.7 -> v0.14.8 <upgrade/upgrade-0.14.8> |
1713 |
v0.14 -> v0.15 <upgrade/upgrade-0.15> |
1714 |
|
1715 |
|
1716 |
Changelog, NEWS |
1717 |
=============== |
1718 |
|
1719 |
|
1720 |
* v0.14.9 :ref:`Changelog <Changelog-0.14.9>`, :ref:`NEWS <NEWS-0.14.9>` |
1721 |
* v0.14.8 :ref:`Changelog <Changelog-0.14.8>`, :ref:`NEWS <NEWS-0.14.8>` |
1722 |
* v0.14.7 :ref:`Changelog <Changelog-0.14.7>`, :ref:`NEWS <NEWS-0.14.7>` |
1723 |
* v0.14.6 :ref:`Changelog <Changelog-0.14.6>`, :ref:`NEWS <NEWS-0.14.6>` |
1724 |
* v0.14.5 :ref:`Changelog <Changelog-0.14.5>`, :ref:`NEWS <NEWS-0.14.5>` |
1725 |
* v0.14.4 :ref:`Changelog <Changelog-0.14.4>`, :ref:`NEWS <NEWS-0.14.4>` |
1726 |
* v0.14.3 :ref:`Changelog <Changelog-0.14.3>`, :ref:`NEWS <NEWS-0.14.3>` |
1727 |
* v0.14.2 :ref:`Changelog <Changelog-0.14.2>`, :ref:`NEWS <NEWS-0.14.2>` |
1728 |
* v0.14 :ref:`Changelog <Changelog-0.14>`, :ref:`NEWS <NEWS-0.14>` |
1729 |
* v0.13 :ref:`Changelog <Changelog-0.13>`, :ref:`NEWS <NEWS-0.13>` |