root / docs / admin-guide.rst @ 30fe9049
History | View | Annotate | Download (65.4 kB)
1 | bc055d09 | Constantinos Venetsanopoulos | .. _admin-guide: |
---|---|---|---|
2 | bc055d09 | Constantinos Venetsanopoulos | |
3 | bc055d09 | Constantinos Venetsanopoulos | Synnefo Administrator's Guide |
4 | bc055d09 | Constantinos Venetsanopoulos | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
5 | bc055d09 | Constantinos Venetsanopoulos | |
6 | bc055d09 | Constantinos Venetsanopoulos | This is the complete Synnefo Administrator's Guide. |
7 | bc055d09 | Constantinos Venetsanopoulos | |
8 | bc055d09 | Constantinos Venetsanopoulos | |
9 | bc055d09 | Constantinos Venetsanopoulos | |
10 | eab73849 | Constantinos Venetsanopoulos | General Synnefo Architecture |
11 | eab73849 | Constantinos Venetsanopoulos | ============================ |
12 | eab73849 | Constantinos Venetsanopoulos | |
13 | eab73849 | Constantinos Venetsanopoulos | The following graph shows the whole Synnefo architecture and how it interacts |
14 | eab73849 | Constantinos Venetsanopoulos | with multiple Ganeti clusters. We hope that after reading the Administrator's |
15 | eab73849 | Constantinos Venetsanopoulos | Guide you will be able to understand every component and all the interactions |
16 | ff6ba888 | Constantinos Venetsanopoulos | between them. It is a good idea to first go through the Administrator's Guide |
17 | ff6ba888 | Constantinos Venetsanopoulos | before proceeding. |
18 | eab73849 | Constantinos Venetsanopoulos | |
19 | 301294a9 | Constantinos Venetsanopoulos | .. image:: images/synnefo-arch2.png |
20 | 16539dad | Kostas Papadimitriou | :width: 100% |
21 | 301294a9 | Constantinos Venetsanopoulos | :target: _images/synnefo-arch2.png |
22 | eab73849 | Constantinos Venetsanopoulos | |
23 | eab73849 | Constantinos Venetsanopoulos | |
24 | a1c707c9 | Constantinos Venetsanopoulos | |
25 | eab73849 | Constantinos Venetsanopoulos | Identity Service (Astakos) |
26 | eab73849 | Constantinos Venetsanopoulos | ========================== |
27 | f846d8df | Constantinos Venetsanopoulos | |
28 | f846d8df | Constantinos Venetsanopoulos | |
29 | eab73849 | Constantinos Venetsanopoulos | Overview |
30 | eab73849 | Constantinos Venetsanopoulos | -------- |
31 | eab73849 | Constantinos Venetsanopoulos | |
32 | eab73849 | Constantinos Venetsanopoulos | Authentication methods |
33 | eab73849 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~~~~~ |
34 | eab73849 | Constantinos Venetsanopoulos | |
35 | eab73849 | Constantinos Venetsanopoulos | Local Authentication |
36 | eab73849 | Constantinos Venetsanopoulos | ```````````````````` |
37 | eab73849 | Constantinos Venetsanopoulos | |
38 | eab73849 | Constantinos Venetsanopoulos | LDAP Authentication |
39 | eab73849 | Constantinos Venetsanopoulos | ``````````````````` |
40 | eab73849 | Constantinos Venetsanopoulos | |
41 | eab73849 | Constantinos Venetsanopoulos | .. _shibboleth-auth: |
42 | eab73849 | Constantinos Venetsanopoulos | |
43 | eab73849 | Constantinos Venetsanopoulos | Shibboleth Authentication |
44 | eab73849 | Constantinos Venetsanopoulos | ````````````````````````` |
45 | eab73849 | Constantinos Venetsanopoulos | |
46 | eab73849 | Constantinos Venetsanopoulos | Astakos can delegate user authentication to a Shibboleth federation. |
47 | eab73849 | Constantinos Venetsanopoulos | |
48 | eab73849 | Constantinos Venetsanopoulos | To setup shibboleth, install package:: |
49 | eab73849 | Constantinos Venetsanopoulos | |
50 | eab73849 | Constantinos Venetsanopoulos | apt-get install libapache2-mod-shib2 |
51 | eab73849 | Constantinos Venetsanopoulos | |
52 | eab73849 | Constantinos Venetsanopoulos | Change appropriately the configuration files in ``/etc/shibboleth``. |
53 | eab73849 | Constantinos Venetsanopoulos | |
54 | eab73849 | Constantinos Venetsanopoulos | Add in ``/etc/apache2/sites-available/synnefo-ssl``:: |
55 | eab73849 | Constantinos Venetsanopoulos | |
56 | eab73849 | Constantinos Venetsanopoulos | ShibConfig /etc/shibboleth/shibboleth2.xml |
57 | eab73849 | Constantinos Venetsanopoulos | Alias /shibboleth-sp /usr/share/shibboleth |
58 | eab73849 | Constantinos Venetsanopoulos | |
59 | fb814235 | Kostas Papadimitriou | <Location /ui/login/shibboleth> |
60 | eab73849 | Constantinos Venetsanopoulos | AuthType shibboleth |
61 | eab73849 | Constantinos Venetsanopoulos | ShibRequireSession On |
62 | eab73849 | Constantinos Venetsanopoulos | ShibUseHeaders On |
63 | eab73849 | Constantinos Venetsanopoulos | require valid-user |
64 | eab73849 | Constantinos Venetsanopoulos | </Location> |
65 | eab73849 | Constantinos Venetsanopoulos | |
66 | eab73849 | Constantinos Venetsanopoulos | and before the line containing:: |
67 | eab73849 | Constantinos Venetsanopoulos | |
68 | eab73849 | Constantinos Venetsanopoulos | ProxyPass / http://localhost:8080/ retry=0 |
69 | eab73849 | Constantinos Venetsanopoulos | |
70 | eab73849 | Constantinos Venetsanopoulos | add:: |
71 | eab73849 | Constantinos Venetsanopoulos | |
72 | eab73849 | Constantinos Venetsanopoulos | ProxyPass /Shibboleth.sso ! |
73 | f846d8df | Constantinos Venetsanopoulos | |
74 | eab73849 | Constantinos Venetsanopoulos | Then, enable the shibboleth module:: |
75 | eab73849 | Constantinos Venetsanopoulos | |
76 | eab73849 | Constantinos Venetsanopoulos | a2enmod shib2 |
77 | f846d8df | Constantinos Venetsanopoulos | |
78 | eab73849 | Constantinos Venetsanopoulos | After passing through the apache module, the following tokens should be |
79 | eab73849 | Constantinos Venetsanopoulos | available at the destination:: |
80 | eab73849 | Constantinos Venetsanopoulos | |
81 | eab73849 | Constantinos Venetsanopoulos | eppn # eduPersonPrincipalName |
82 | eab73849 | Constantinos Venetsanopoulos | Shib-InetOrgPerson-givenName |
83 | eab73849 | Constantinos Venetsanopoulos | Shib-Person-surname |
84 | eab73849 | Constantinos Venetsanopoulos | Shib-Person-commonName |
85 | eab73849 | Constantinos Venetsanopoulos | Shib-InetOrgPerson-displayName |
86 | eab73849 | Constantinos Venetsanopoulos | Shib-EP-Affiliation |
87 | eab73849 | Constantinos Venetsanopoulos | Shib-Session-ID |
88 | eab73849 | Constantinos Venetsanopoulos | |
89 | eab73849 | Constantinos Venetsanopoulos | Finally, add 'shibboleth' in ``ASTAKOS_IM_MODULES`` list. The variable resides |
90 | eab73849 | Constantinos Venetsanopoulos | inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf`` |
91 | eab73849 | Constantinos Venetsanopoulos | |
92 | e0975bbf | Sofia Papagiannaki | Twitter Authentication |
93 | e0975bbf | Sofia Papagiannaki | ``````````````````````` |
94 | e0975bbf | Sofia Papagiannaki | |
95 | e0975bbf | Sofia Papagiannaki | To enable twitter authentication while signed in under a Twitter account, |
96 | e0975bbf | Sofia Papagiannaki | visit dev.twitter.com/apps. |
97 | e0975bbf | Sofia Papagiannaki | |
98 | e0975bbf | Sofia Papagiannaki | Click Create an application. |
99 | e0975bbf | Sofia Papagiannaki | |
100 | e0975bbf | Sofia Papagiannaki | Fill the necessary information and for callback URL give:: |
101 | e0975bbf | Sofia Papagiannaki | |
102 | e0975bbf | Sofia Papagiannaki | https://node1.example.com/ui/login/twitter/authenticated |
103 | e0975bbf | Sofia Papagiannaki | |
104 | e0975bbf | Sofia Papagiannaki | Finally, add 'twitter' in ``ASTAKOS_IM_MODULES`` list. The variable resides |
105 | e0975bbf | Sofia Papagiannaki | inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf`` |
106 | e0975bbf | Sofia Papagiannaki | |
107 | e0975bbf | Sofia Papagiannaki | |
108 | e0975bbf | Sofia Papagiannaki | Google Authentication |
109 | e0975bbf | Sofia Papagiannaki | ````````````````````` |
110 | e0975bbf | Sofia Papagiannaki | |
111 | e0975bbf | Sofia Papagiannaki | To enable google authentication while signed in under a Google account, |
112 | e0975bbf | Sofia Papagiannaki | visit https://code.google.com/apis/console/. |
113 | e0975bbf | Sofia Papagiannaki | |
114 | e0975bbf | Sofia Papagiannaki | Under API Access select Create another client ID, select Web application, |
115 | e0975bbf | Sofia Papagiannaki | expand more options in Your site or hostname section and in Authorized |
116 | e0975bbf | Sofia Papagiannaki | Redirect URIs add: |
117 | e0975bbf | Sofia Papagiannaki | |
118 | e0975bbf | Sofia Papagiannaki | |
119 | e0975bbf | Sofia Papagiannaki | Fill the necessary information and for callback URL give:: |
120 | e0975bbf | Sofia Papagiannaki | |
121 | e0975bbf | Sofia Papagiannaki | https://node1.example.com/ui/login/google/authenticated |
122 | e0975bbf | Sofia Papagiannaki | |
123 | e0975bbf | Sofia Papagiannaki | Finally, add 'google' in ``ASTAKOS_IM_MODULES`` list. The variable resides |
124 | e0975bbf | Sofia Papagiannaki | inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf`` |
125 | e0975bbf | Sofia Papagiannaki | |
126 | eab73849 | Constantinos Venetsanopoulos | Architecture |
127 | eab73849 | Constantinos Venetsanopoulos | ------------ |
128 | eab73849 | Constantinos Venetsanopoulos | |
129 | eab73849 | Constantinos Venetsanopoulos | Prereqs |
130 | eab73849 | Constantinos Venetsanopoulos | ------- |
131 | eab73849 | Constantinos Venetsanopoulos | |
132 | eab73849 | Constantinos Venetsanopoulos | Installation |
133 | eab73849 | Constantinos Venetsanopoulos | ------------ |
134 | eab73849 | Constantinos Venetsanopoulos | |
135 | eab73849 | Constantinos Venetsanopoulos | Configuration |
136 | eab73849 | Constantinos Venetsanopoulos | ------------- |
137 | eab73849 | Constantinos Venetsanopoulos | |
138 | eab73849 | Constantinos Venetsanopoulos | Working with Astakos |
139 | eab73849 | Constantinos Venetsanopoulos | -------------------- |
140 | eab73849 | Constantinos Venetsanopoulos | |
141 | 77186edf | Kostas Papadimitriou | User registration |
142 | 77186edf | Kostas Papadimitriou | ~~~~~~~~~~~~~~~~~ |
143 | f846d8df | Constantinos Venetsanopoulos | |
144 | 77186edf | Kostas Papadimitriou | When a new user signs up, he/she is not directly marked as active. You can see |
145 | 77186edf | Kostas Papadimitriou | his/her state by running (on the machine that runs the Astakos app): |
146 | f846d8df | Constantinos Venetsanopoulos | |
147 | f846d8df | Constantinos Venetsanopoulos | .. code-block:: console |
148 | f846d8df | Constantinos Venetsanopoulos | |
149 | 4404e02e | Ilias Tsitsimpis | $ snf-manage user-list |
150 | f846d8df | Constantinos Venetsanopoulos | |
151 | 77186edf | Kostas Papadimitriou | More detailed user status is provided in the `status` field of the `user-show` |
152 | 77186edf | Kostas Papadimitriou | command: |
153 | f846d8df | Constantinos Venetsanopoulos | |
154 | 77186edf | Kostas Papadimitriou | .. code-block:: console |
155 | f846d8df | Constantinos Venetsanopoulos | |
156 | 77186edf | Kostas Papadimitriou | $ snf-manage user-show <user-id> |
157 | f846d8df | Constantinos Venetsanopoulos | |
158 | 77186edf | Kostas Papadimitriou | id : 6 |
159 | 77186edf | Kostas Papadimitriou | uuid : 78661411-5eed-412f-a9ea-2de24f542c2e |
160 | 77186edf | Kostas Papadimitriou | status : Accepted/Active (accepted policy: manual) |
161 | 77186edf | Kostas Papadimitriou | email : user@synnefo.org |
162 | 77186edf | Kostas Papadimitriou | .... |
163 | f846d8df | Constantinos Venetsanopoulos | |
164 | f846d8df | Constantinos Venetsanopoulos | |
165 | 77186edf | Kostas Papadimitriou | Based on how your configuration of `astakos-app`, there are several ways for a |
166 | 77186edf | Kostas Papadimitriou | user to get activated and be able to login. We discuss the user activation |
167 | 77186edf | Kostas Papadimitriou | flow in the following section. |
168 | f846d8df | Constantinos Venetsanopoulos | |
169 | 77186edf | Kostas Papadimitriou | |
170 | 77186edf | Kostas Papadimitriou | User activation flow |
171 | eab73849 | Constantinos Venetsanopoulos | ```````````````````` |
172 | eab73849 | Constantinos Venetsanopoulos | |
173 | 77186edf | Kostas Papadimitriou | A user can register for an account using the astakos signup form. Once the form |
174 | 77186edf | Kostas Papadimitriou | is submited successfully a user entry is created in astakos database. That entry |
175 | 77186edf | Kostas Papadimitriou | is passed through the astakos activation backend which handles whether the user |
176 | 77186edf | Kostas Papadimitriou | should be automatically verified and activated. |
177 | 77186edf | Kostas Papadimitriou | |
178 | 77186edf | Kostas Papadimitriou | |
179 | 77186edf | Kostas Papadimitriou | Email verification |
180 | 77186edf | Kostas Papadimitriou | `````````````````` |
181 | 77186edf | Kostas Papadimitriou | |
182 | 77186edf | Kostas Papadimitriou | The verification process takes place in order to ensure that the user owns the |
183 | 77186edf | Kostas Papadimitriou | email provided during the signup process. By default, after each successful |
184 | 77186edf | Kostas Papadimitriou | signup astakos notifies user with an verification url via email. |
185 | 77186edf | Kostas Papadimitriou | |
186 | 77186edf | Kostas Papadimitriou | At this stage: |
187 | 77186edf | Kostas Papadimitriou | |
188 | 77186edf | Kostas Papadimitriou | * subsequent registrations invalidate and delete the previous registrations |
189 | 77186edf | Kostas Papadimitriou | of the same email address. |
190 | 77186edf | Kostas Papadimitriou | |
191 | 77186edf | Kostas Papadimitriou | * in case user misses the initial notification, additional emails can be |
192 | 77186edf | Kostas Papadimitriou | send either via the url which is prompted to the user if he tries to |
193 | 77186edf | Kostas Papadimitriou | login, or by the administrator using the ``snf-manage user-activation-send |
194 | 77186edf | Kostas Papadimitriou | <userid>`` command. |
195 | 77186edf | Kostas Papadimitriou | |
196 | 77186edf | Kostas Papadimitriou | * administrator may also enforce a user to get verified using the |
197 | 77186edf | Kostas Papadimitriou | ``snf-manage user-modify --verify <userid>`` command. |
198 | 77186edf | Kostas Papadimitriou | |
199 | 77186edf | Kostas Papadimitriou | |
200 | 77186edf | Kostas Papadimitriou | Account activation |
201 | 77186edf | Kostas Papadimitriou | `````````````````` |
202 | 77186edf | Kostas Papadimitriou | |
203 | 77186edf | Kostas Papadimitriou | Once user gets verified it is time for astakos to decide whether or not to |
204 | 77186edf | Kostas Papadimitriou | proceed through user activation process. If ``ASTAKOS_MODERATION_ENABLED`` |
205 | 77186edf | Kostas Papadimitriou | setting is set to ``False`` (default value) user gets activated automatically. |
206 | 77186edf | Kostas Papadimitriou | |
207 | 77186edf | Kostas Papadimitriou | In case the moderation is enabled astakos may still automatically activate the |
208 | 77186edf | Kostas Papadimitriou | user in the following cases: |
209 | 77186edf | Kostas Papadimitriou | |
210 | 77186edf | Kostas Papadimitriou | * User email matches any of the regular expressions defined in |
211 | 77186edf | Kostas Papadimitriou | ``ASTAKOS_RE_USER_EMAIL_PATTERNS`` (defaults to ``[]``) |
212 | 77186edf | Kostas Papadimitriou | * User used a signup method (e.g. ``shibboleth``) for which automatic |
213 | 77186edf | Kostas Papadimitriou | activation is enabled (see |
214 | 77186edf | Kostas Papadimitriou | :ref:`authentication methods policies <auth_methods_policies>`). |
215 | 77186edf | Kostas Papadimitriou | |
216 | 77186edf | Kostas Papadimitriou | If all of the above fail to trigger automatic activation, an email is sent |
217 | 77186edf | Kostas Papadimitriou | to the persons listed in ``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings, |
218 | 77186edf | Kostas Papadimitriou | notifing that there is a new user pending for moderation and that it's |
219 | 77186edf | Kostas Papadimitriou | up to the administrator to decide if the user should be activated, using the |
220 | 77186edf | Kostas Papadimitriou | ``user-modify`` command. |
221 | 77186edf | Kostas Papadimitriou | |
222 | 77186edf | Kostas Papadimitriou | .. code-block:: console |
223 | 77186edf | Kostas Papadimitriou | |
224 | 77186edf | Kostas Papadimitriou | # command to activate a pending user |
225 | 77186edf | Kostas Papadimitriou | $ snf-manage user-modify --accept <userid> |
226 | 77186edf | Kostas Papadimitriou | |
227 | 77186edf | Kostas Papadimitriou | # command to reject a pending user |
228 | 77186edf | Kostas Papadimitriou | $ snf-manage user-modify --reject --reject-reason="spammer" <userid> |
229 | 77186edf | Kostas Papadimitriou | |
230 | 77186edf | Kostas Papadimitriou | Once activation process finish, a greeting message is sent to the user email |
231 | 77186edf | Kostas Papadimitriou | address and a notification for the activation to the persons listed in |
232 | 77186edf | Kostas Papadimitriou | ``HELPDESK``, ``MANAGERS`` and ``ADMINS`` settings. Once activated the user is |
233 | 77186edf | Kostas Papadimitriou | able to login and access the synnefo services. |
234 | 77186edf | Kostas Papadimitriou | |
235 | 77186edf | Kostas Papadimitriou | |
236 | 77186edf | Kostas Papadimitriou | Additional authentication methods |
237 | 77186edf | Kostas Papadimitriou | ````````````````````````````````` |
238 | 77186edf | Kostas Papadimitriou | |
239 | 77186edf | Kostas Papadimitriou | Astakos supports third party logins from external identity providers. This |
240 | 77186edf | Kostas Papadimitriou | can be usefull since it allows users to use their existing credentials to |
241 | 77186edf | Kostas Papadimitriou | login to astakos service. |
242 | 77186edf | Kostas Papadimitriou | |
243 | 77186edf | Kostas Papadimitriou | Currently astakos supports the following identity providers: |
244 | 77186edf | Kostas Papadimitriou | |
245 | 77186edf | Kostas Papadimitriou | * `Shibboleth <http://www.internet2.edu/shibboleth>`_ (module name |
246 | 77186edf | Kostas Papadimitriou | ``shibboleth``) |
247 | 77186edf | Kostas Papadimitriou | * `Google <https://developers.google.com/accounts/docs/OAuth2>`_ (module |
248 | 77186edf | Kostas Papadimitriou | name ``google``) |
249 | 77186edf | Kostas Papadimitriou | * `Twitter <https://dev.twitter.com/docs/auth>`_ (module name ``twitter``) |
250 | 77186edf | Kostas Papadimitriou | * `LinkedIn <http://developer.linkedin.com/documents/authentication>`_ |
251 | 77186edf | Kostas Papadimitriou | (module name ``linkedin``) |
252 | 77186edf | Kostas Papadimitriou | |
253 | 77186edf | Kostas Papadimitriou | To enable any of the above modules (by default only ``local`` accounts are |
254 | 77186edf | Kostas Papadimitriou | allowed), retrieve and set the required provider settings and append the |
255 | 77186edf | Kostas Papadimitriou | module name in ``ASTAKOS_IM_MODULES``. |
256 | 77186edf | Kostas Papadimitriou | |
257 | 77186edf | Kostas Papadimitriou | .. code-block:: python |
258 | 77186edf | Kostas Papadimitriou | |
259 | 77186edf | Kostas Papadimitriou | # settings from https://code.google.com/apis/console/ |
260 | 77186edf | Kostas Papadimitriou | ASTAKOS_GOOGLE_CLIENT_ID = '1111111111-epi60tvimgha63qqnjo40cljkojcann3.apps.googleusercontent.com' |
261 | 77186edf | Kostas Papadimitriou | ASTAKOS_GOOGLE_SECRET = 'tNDQqTDKlTf7_LaeUcWTWwZM' |
262 | 77186edf | Kostas Papadimitriou | |
263 | 77186edf | Kostas Papadimitriou | # let users signup and login using their google account |
264 | 77186edf | Kostas Papadimitriou | ASTAKOS_IM_MODULES = ['local', 'google'] |
265 | 77186edf | Kostas Papadimitriou | |
266 | 77186edf | Kostas Papadimitriou | |
267 | 77186edf | Kostas Papadimitriou | .. _auth_methods_policies: |
268 | 77186edf | Kostas Papadimitriou | |
269 | 77186edf | Kostas Papadimitriou | Authentication method policies |
270 | 77186edf | Kostas Papadimitriou | `````````````````````````````` |
271 | 77186edf | Kostas Papadimitriou | |
272 | 77186edf | Kostas Papadimitriou | Astakos allows you to override the default policies for each enabled provider |
273 | 77186edf | Kostas Papadimitriou | separately by adding the approriate settings in your ``.conf`` files in the |
274 | 77186edf | Kostas Papadimitriou | following format: |
275 | 77186edf | Kostas Papadimitriou | |
276 | 77186edf | Kostas Papadimitriou | **ASTAKOS_AUTH_PROVIDER_<module>_<policy>_POLICY** |
277 | 77186edf | Kostas Papadimitriou | |
278 | 77186edf | Kostas Papadimitriou | Available policies are: |
279 | 77186edf | Kostas Papadimitriou | |
280 | 77186edf | Kostas Papadimitriou | * **CREATE** Users can signup using that provider (default: ``True``) |
281 | 77186edf | Kostas Papadimitriou | * **REMOVE/ADD** Users can remove/add login method from their profile |
282 | 77186edf | Kostas Papadimitriou | (default: ``True``) |
283 | 77186edf | Kostas Papadimitriou | * **AUTOMODERATE** Automatically activate users that signup using that |
284 | 77186edf | Kostas Papadimitriou | provider (default: ``False``) |
285 | 77186edf | Kostas Papadimitriou | * **LOGIN** Whether or not users can use the provider to login (default: |
286 | 77186edf | Kostas Papadimitriou | ``True``). |
287 | 77186edf | Kostas Papadimitriou | |
288 | 77186edf | Kostas Papadimitriou | e.g. to enable automatic activation for your academic users, while keeping |
289 | 77186edf | Kostas Papadimitriou | locally signed up users under moderation you can apply the following settings. |
290 | 77186edf | Kostas Papadimitriou | |
291 | 77186edf | Kostas Papadimitriou | .. code-block:: python |
292 | 77186edf | Kostas Papadimitriou | |
293 | 77186edf | Kostas Papadimitriou | ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_AUTOMODERATE_POLICY = True |
294 | 77186edf | Kostas Papadimitriou | ASTAKOS_AUTH_PROVIDER_SHIBBOLETH_REMOVE_POLICY = False |
295 | 77186edf | Kostas Papadimitriou | |
296 | eab73849 | Constantinos Venetsanopoulos | |
297 | 2988ae31 | Georgios D. Tsoukalas | Setting quota limits |
298 | 2988ae31 | Georgios D. Tsoukalas | ~~~~~~~~~~~~~~~~~~~~ |
299 | 2988ae31 | Georgios D. Tsoukalas | |
300 | 075b91de | Giorgos Korfiatis | Set default quota |
301 | 075b91de | Giorgos Korfiatis | ````````````````` |
302 | 2988ae31 | Georgios D. Tsoukalas | |
303 | 2988ae31 | Georgios D. Tsoukalas | In 20-snf-astakos-app-settings.conf, |
304 | 2988ae31 | Georgios D. Tsoukalas | uncomment the default setting ``ASTAKOS_SERVICES`` |
305 | 2988ae31 | Georgios D. Tsoukalas | and customize the ``'uplimit'`` values. |
306 | 075b91de | Giorgos Korfiatis | These are the default base quota for all users. |
307 | 2988ae31 | Georgios D. Tsoukalas | |
308 | 2988ae31 | Georgios D. Tsoukalas | To apply your configuration run:: |
309 | 2988ae31 | Georgios D. Tsoukalas | |
310 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage astakos-init --load-service-resources |
311 | 075b91de | Giorgos Korfiatis | # snf-manage quota --sync |
312 | 2988ae31 | Georgios D. Tsoukalas | |
313 | 075b91de | Giorgos Korfiatis | Set base quota for individual users |
314 | 075b91de | Giorgos Korfiatis | ``````````````````````````````````` |
315 | 2988ae31 | Georgios D. Tsoukalas | |
316 | 075b91de | Giorgos Korfiatis | For individual users that need different quota than the default |
317 | 2988ae31 | Georgios D. Tsoukalas | you can set it for each resource like this:: |
318 | 2988ae31 | Georgios D. Tsoukalas | |
319 | 075b91de | Giorgos Korfiatis | # use this to display quota / uuid |
320 | 075b91de | Giorgos Korfiatis | # snf-manage user-show 'uuid or email' --quota |
321 | 2988ae31 | Georgios D. Tsoukalas | |
322 | 075b91de | Giorgos Korfiatis | # snf-manage user-modify 'user-uuid' --set-base-quota 'cyclades.vm' 10 |
323 | 2988ae31 | Georgios D. Tsoukalas | |
324 | 2988ae31 | Georgios D. Tsoukalas | |
325 | 2988ae31 | Georgios D. Tsoukalas | Enable the Projects feature |
326 | 2988ae31 | Georgios D. Tsoukalas | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
327 | 2988ae31 | Georgios D. Tsoukalas | |
328 | 2988ae31 | Georgios D. Tsoukalas | If you want to enable the projects feature so that users may apply |
329 | 2988ae31 | Georgios D. Tsoukalas | on their own for resources by creating and joining projects, |
330 | 2988ae31 | Georgios D. Tsoukalas | in ``20-snf-astakos-app-settings.conf`` set:: |
331 | 2988ae31 | Georgios D. Tsoukalas | |
332 | 2988ae31 | Georgios D. Tsoukalas | # this will make the 'projects' page visible in the dashboard |
333 | 2988ae31 | Georgios D. Tsoukalas | ASTAKOS_PROJECTS_VISIBLE = True |
334 | 2988ae31 | Georgios D. Tsoukalas | |
335 | 9770ba6c | Giorgos Korfiatis | You can change the maximum allowed number of pending project applications |
336 | 9770ba6c | Giorgos Korfiatis | per user with:: |
337 | 827de879 | Giorgos Korfiatis | |
338 | 9770ba6c | Giorgos Korfiatis | # snf-manage resource-modify astakos.pending_app --limit <number> |
339 | 9770ba6c | Giorgos Korfiatis | |
340 | 9770ba6c | Giorgos Korfiatis | You can also set a user-specific limit with:: |
341 | 9770ba6c | Giorgos Korfiatis | |
342 | 075b91de | Giorgos Korfiatis | # snf-manage user-modify 'user-uuid' --set-base-quota 'astakos.pending_app' 5 |
343 | 827de879 | Giorgos Korfiatis | |
344 | 2988ae31 | Georgios D. Tsoukalas | When users apply for projects they are not automatically granted |
345 | 2988ae31 | Georgios D. Tsoukalas | the resources. They must first be approved by the administrator. |
346 | 2988ae31 | Georgios D. Tsoukalas | |
347 | 2988ae31 | Georgios D. Tsoukalas | To list pending project applications in astakos:: |
348 | 2988ae31 | Georgios D. Tsoukalas | |
349 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage project-list --pending |
350 | 2988ae31 | Georgios D. Tsoukalas | |
351 | 2988ae31 | Georgios D. Tsoukalas | Note the last column, the application id. To approve it:: |
352 | 2988ae31 | Georgios D. Tsoukalas | |
353 | 2988ae31 | Georgios D. Tsoukalas | # <app id> from the last column of project-list |
354 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage project-control --approve <app id> |
355 | 2988ae31 | Georgios D. Tsoukalas | |
356 | 2988ae31 | Georgios D. Tsoukalas | To deny an application:: |
357 | 2988ae31 | Georgios D. Tsoukalas | |
358 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage project-control --deny <app id> |
359 | 2988ae31 | Georgios D. Tsoukalas | |
360 | 827de879 | Giorgos Korfiatis | Users designated as *project admins* can approve, deny, or modify |
361 | 827de879 | Giorgos Korfiatis | an application through the web interface. In |
362 | 827de879 | Giorgos Korfiatis | ``20-snf-astakos-app-settings.conf`` set:: |
363 | 827de879 | Giorgos Korfiatis | |
364 | 827de879 | Giorgos Korfiatis | # UUIDs of users that can approve or deny project applications from the web. |
365 | 827de879 | Giorgos Korfiatis | ASTAKOS_PROJECT_ADMINS = [<uuid>, ...] |
366 | 2988ae31 | Georgios D. Tsoukalas | |
367 | 2988ae31 | Georgios D. Tsoukalas | |
368 | eab73849 | Constantinos Venetsanopoulos | Astakos advanced operations |
369 | eab73849 | Constantinos Venetsanopoulos | --------------------------- |
370 | eab73849 | Constantinos Venetsanopoulos | |
371 | eab73849 | Constantinos Venetsanopoulos | Adding "Terms of Use" |
372 | eab73849 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~~~~ |
373 | eab73849 | Constantinos Venetsanopoulos | |
374 | eab73849 | Constantinos Venetsanopoulos | Astakos supports versioned terms-of-use. First of all you need to create an |
375 | eab73849 | Constantinos Venetsanopoulos | html file that will contain your terms. For example, create the file |
376 | eab73849 | Constantinos Venetsanopoulos | ``/usr/share/synnefo/sample-terms.html``, which contains the following: |
377 | eab73849 | Constantinos Venetsanopoulos | |
378 | eab73849 | Constantinos Venetsanopoulos | .. code-block:: console |
379 | eab73849 | Constantinos Venetsanopoulos | |
380 | eab73849 | Constantinos Venetsanopoulos | <h1>~okeanos terms</h1> |
381 | eab73849 | Constantinos Venetsanopoulos | |
382 | eab73849 | Constantinos Venetsanopoulos | These are the example terms for ~okeanos |
383 | eab73849 | Constantinos Venetsanopoulos | |
384 | eab73849 | Constantinos Venetsanopoulos | Then, add those terms-of-use with the snf-manage command: |
385 | eab73849 | Constantinos Venetsanopoulos | |
386 | eab73849 | Constantinos Venetsanopoulos | .. code-block:: console |
387 | eab73849 | Constantinos Venetsanopoulos | |
388 | eab73849 | Constantinos Venetsanopoulos | $ snf-manage term-add /usr/share/synnefo/sample-terms.html |
389 | eab73849 | Constantinos Venetsanopoulos | |
390 | eab73849 | Constantinos Venetsanopoulos | Your terms have been successfully added and you will see the corresponding link |
391 | eab73849 | Constantinos Venetsanopoulos | appearing in the Astakos web pages' footer. |
392 | eab73849 | Constantinos Venetsanopoulos | |
393 | a23251f4 | Constantinos Venetsanopoulos | Enabling reCAPTCHA |
394 | a23251f4 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~ |
395 | a23251f4 | Constantinos Venetsanopoulos | |
396 | a23251f4 | Constantinos Venetsanopoulos | Astakos supports the `reCAPTCHA <http://www.google.com/recaptcha>`_ feature. |
397 | a23251f4 | Constantinos Venetsanopoulos | If enabled, it protects the Astakos forms from bots. To enable the feature, go |
398 | a23251f4 | Constantinos Venetsanopoulos | to https://www.google.com/recaptcha/admin/create and create your own reCAPTCHA |
399 | a23251f4 | Constantinos Venetsanopoulos | key pair. Then edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` and set |
400 | a23251f4 | Constantinos Venetsanopoulos | the corresponding variables to reflect your newly created key pair. Finally, set |
401 | a23251f4 | Constantinos Venetsanopoulos | the ``ASTAKOS_RECAPTCHA_ENABLED`` variable to ``True``: |
402 | a23251f4 | Constantinos Venetsanopoulos | |
403 | a23251f4 | Constantinos Venetsanopoulos | .. code-block:: console |
404 | a23251f4 | Constantinos Venetsanopoulos | |
405 | a23251f4 | Constantinos Venetsanopoulos | ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*(' |
406 | a23251f4 | Constantinos Venetsanopoulos | ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*(' |
407 | a23251f4 | Constantinos Venetsanopoulos | |
408 | a23251f4 | Constantinos Venetsanopoulos | ASTAKOS_RECAPTCHA_ENABLED = True |
409 | a23251f4 | Constantinos Venetsanopoulos | |
410 | a23251f4 | Constantinos Venetsanopoulos | Restart the service on the Astakos node(s) and you are ready: |
411 | a23251f4 | Constantinos Venetsanopoulos | |
412 | a23251f4 | Constantinos Venetsanopoulos | .. code-block:: console |
413 | a23251f4 | Constantinos Venetsanopoulos | |
414 | a23251f4 | Constantinos Venetsanopoulos | # /etc/init.d/gunicorn restart |
415 | a23251f4 | Constantinos Venetsanopoulos | |
416 | a23251f4 | Constantinos Venetsanopoulos | Checkout your new Sign up page. If you see the reCAPTCHA box, you have setup |
417 | a23251f4 | Constantinos Venetsanopoulos | everything correctly. |
418 | a23251f4 | Constantinos Venetsanopoulos | |
419 | eab73849 | Constantinos Venetsanopoulos | |
420 | eab73849 | Constantinos Venetsanopoulos | |
421 | eab73849 | Constantinos Venetsanopoulos | File Storage Service (Pithos) |
422 | eab73849 | Constantinos Venetsanopoulos | ============================= |
423 | eab73849 | Constantinos Venetsanopoulos | |
424 | eab73849 | Constantinos Venetsanopoulos | Overview |
425 | eab73849 | Constantinos Venetsanopoulos | -------- |
426 | eab73849 | Constantinos Venetsanopoulos | |
427 | eab73849 | Constantinos Venetsanopoulos | Architecture |
428 | eab73849 | Constantinos Venetsanopoulos | ------------ |
429 | eab73849 | Constantinos Venetsanopoulos | |
430 | eab73849 | Constantinos Venetsanopoulos | Prereqs |
431 | eab73849 | Constantinos Venetsanopoulos | ------- |
432 | eab73849 | Constantinos Venetsanopoulos | |
433 | eab73849 | Constantinos Venetsanopoulos | Installation |
434 | eab73849 | Constantinos Venetsanopoulos | ------------ |
435 | eab73849 | Constantinos Venetsanopoulos | |
436 | eab73849 | Constantinos Venetsanopoulos | Configuration |
437 | eab73849 | Constantinos Venetsanopoulos | ------------- |
438 | eab73849 | Constantinos Venetsanopoulos | |
439 | eab73849 | Constantinos Venetsanopoulos | Working with Pithos |
440 | eab73849 | Constantinos Venetsanopoulos | ------------------- |
441 | eab73849 | Constantinos Venetsanopoulos | |
442 | eab73849 | Constantinos Venetsanopoulos | Pithos advanced operations |
443 | eab73849 | Constantinos Venetsanopoulos | -------------------------- |
444 | eab73849 | Constantinos Venetsanopoulos | |
445 | eab73849 | Constantinos Venetsanopoulos | |
446 | eab73849 | Constantinos Venetsanopoulos | |
447 | eab73849 | Constantinos Venetsanopoulos | Compute/Network/Image Service (Cyclades) |
448 | eab73849 | Constantinos Venetsanopoulos | ======================================== |
449 | eab73849 | Constantinos Venetsanopoulos | |
450 | eab73849 | Constantinos Venetsanopoulos | Compute Overview |
451 | eab73849 | Constantinos Venetsanopoulos | ---------------- |
452 | eab73849 | Constantinos Venetsanopoulos | |
453 | eab73849 | Constantinos Venetsanopoulos | Network Overview |
454 | eab73849 | Constantinos Venetsanopoulos | ---------------- |
455 | eab73849 | Constantinos Venetsanopoulos | |
456 | eab73849 | Constantinos Venetsanopoulos | Image Overview |
457 | eab73849 | Constantinos Venetsanopoulos | -------------- |
458 | eab73849 | Constantinos Venetsanopoulos | |
459 | eab73849 | Constantinos Venetsanopoulos | Architecture |
460 | eab73849 | Constantinos Venetsanopoulos | ------------ |
461 | eab73849 | Constantinos Venetsanopoulos | |
462 | b9fd8ed3 | Christos Stavrakakis | Asynchronous communication with Ganeti backends |
463 | 90c74332 | Christos Stavrakakis | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
464 | a1c707c9 | Constantinos Venetsanopoulos | Synnefo uses Google Ganeti backends for VM cluster management. In order for |
465 | a1c707c9 | Constantinos Venetsanopoulos | Cyclades to be able to handle thousands of user requests, Cyclades and Ganeti |
466 | a1c707c9 | Constantinos Venetsanopoulos | communicate asynchronously. Briefly, requests are submitted to Ganeti through |
467 | a1c707c9 | Constantinos Venetsanopoulos | Ganeti's RAPI/HTTP interface, and then asynchronous notifications about the |
468 | a1c707c9 | Constantinos Venetsanopoulos | progress of Ganeti jobs are being created and pushed upwards to Cyclades. The |
469 | a1c707c9 | Constantinos Venetsanopoulos | architecture and communication with a Ganeti backend is shown in the graph |
470 | a1c707c9 | Constantinos Venetsanopoulos | below: |
471 | 9b659162 | Christos Stavrakakis | |
472 | 9b659162 | Christos Stavrakakis | .. image:: images/cyclades-ganeti-communication.png |
473 | 9b659162 | Christos Stavrakakis | :width: 50% |
474 | 9b659162 | Christos Stavrakakis | :target: _images/cyclades-ganeti-communication.png |
475 | b9fd8ed3 | Christos Stavrakakis | |
476 | a1c707c9 | Constantinos Venetsanopoulos | The Cyclades API server is responsible for handling user requests. Read-only |
477 | b9fd8ed3 | Christos Stavrakakis | requests are directly served by looking up the Cyclades DB. If the request |
478 | b9fd8ed3 | Christos Stavrakakis | needs an action in the Ganeti backend, Cyclades submit jobs to the Ganeti |
479 | a1c707c9 | Constantinos Venetsanopoulos | master using the `Ganeti RAPI interface |
480 | a1c707c9 | Constantinos Venetsanopoulos | <http://docs.ganeti.org/ganeti/2.2/html/rapi.html>`_. |
481 | b9fd8ed3 | Christos Stavrakakis | |
482 | a1c707c9 | Constantinos Venetsanopoulos | While Ganeti executes the job, `snf-ganeti-eventd`, `snf-ganeti-hook` and |
483 | a1c707c9 | Constantinos Venetsanopoulos | `snf-progress-monitor` are monitoring the progress of the job and send |
484 | c84d78c8 | Christos Stavrakakis | corresponding messages to the RabbitMQ servers. These components are part |
485 | a1c707c9 | Constantinos Venetsanopoulos | of `snf-cyclades-gtools` and must be installed on all Ganeti nodes. Specially: |
486 | b9fd8ed3 | Christos Stavrakakis | |
487 | b9fd8ed3 | Christos Stavrakakis | * *snf-ganeti-eventd* sends messages about operations affecting the operating |
488 | a1c707c9 | Constantinos Venetsanopoulos | state of instances and networks. Works by monitoring the Ganeti job queue. |
489 | a1c707c9 | Constantinos Venetsanopoulos | * *snf-ganeti_hook* sends messages about the NICs of instances. It includes a |
490 | 90c74332 | Christos Stavrakakis | number of `Ganeti hooks <http://docs.ganeti.org/ganeti/2.2/html/hooks.html>`_ |
491 | 90c74332 | Christos Stavrakakis | for customisation of operations. |
492 | a1c707c9 | Constantinos Venetsanopoulos | * *snf-progress_monitor* sends messages about the progress of the Image deployment |
493 | a1c707c9 | Constantinos Venetsanopoulos | phase which is done by the Ganeti OS Definition `snf-image`. |
494 | b9fd8ed3 | Christos Stavrakakis | |
495 | a1c707c9 | Constantinos Venetsanopoulos | Finally, `snf-dispatcher` consumes messages from the RabbitMQ queues, processes |
496 | a1c707c9 | Constantinos Venetsanopoulos | these messages and properly updates the state of the Cyclades DB. Subsequent |
497 | a1c707c9 | Constantinos Venetsanopoulos | requests to the Cyclades API, will retrieve the updated state from the DB. |
498 | b9fd8ed3 | Christos Stavrakakis | |
499 | b9fd8ed3 | Christos Stavrakakis | |
500 | eab73849 | Constantinos Venetsanopoulos | Prereqs |
501 | eab73849 | Constantinos Venetsanopoulos | ------- |
502 | eab73849 | Constantinos Venetsanopoulos | |
503 | ff6ba888 | Constantinos Venetsanopoulos | Work in progress. Please refer to :ref:`administrator's install quide <quick-install-admin-guide>`. |
504 | c84d78c8 | Christos Stavrakakis | |
505 | eab73849 | Constantinos Venetsanopoulos | Installation |
506 | eab73849 | Constantinos Venetsanopoulos | ------------ |
507 | eab73849 | Constantinos Venetsanopoulos | |
508 | ff6ba888 | Constantinos Venetsanopoulos | Work in progress. Please refer to :ref:`administrator's install quide <quick-install-admin-guide>`. |
509 | c84d78c8 | Christos Stavrakakis | |
510 | eab73849 | Constantinos Venetsanopoulos | Configuration |
511 | eab73849 | Constantinos Venetsanopoulos | ------------- |
512 | eab73849 | Constantinos Venetsanopoulos | |
513 | ff6ba888 | Constantinos Venetsanopoulos | Work in progress. Please refer to :ref:`administrator's install quide <quick-install-admin-guide>`. |
514 | c84d78c8 | Christos Stavrakakis | |
515 | eab73849 | Constantinos Venetsanopoulos | Working with Cyclades |
516 | eab73849 | Constantinos Venetsanopoulos | --------------------- |
517 | eab73849 | Constantinos Venetsanopoulos | |
518 | 0768a1df | Christos Stavrakakis | Managing Ganeti Backends |
519 | 90c74332 | Christos Stavrakakis | ~~~~~~~~~~~~~~~~~~~~~~~~ |
520 | 0768a1df | Christos Stavrakakis | |
521 | a1c707c9 | Constantinos Venetsanopoulos | Since v0.11, Synnefo is able to manage multiple Ganeti clusters (backends) |
522 | a1c707c9 | Constantinos Venetsanopoulos | making it capable to scale linearly to tens of thousands of VMs. Backends |
523 | a1c707c9 | Constantinos Venetsanopoulos | can be dynamically added or removed via `snf-manage` commands. |
524 | 0768a1df | Christos Stavrakakis | |
525 | a1c707c9 | Constantinos Venetsanopoulos | Each newly created VM is allocated to a Ganeti backend by the Cyclades backend |
526 | a1c707c9 | Constantinos Venetsanopoulos | allocator. The VM is "pinned" to this backend, and can not change through its |
527 | a1c707c9 | Constantinos Venetsanopoulos | lifetime. The backend allocator decides in which backend to spawn the VM based |
528 | a1c707c9 | Constantinos Venetsanopoulos | on the available resources of each backend, trying to balance the load between |
529 | a1c707c9 | Constantinos Venetsanopoulos | them. |
530 | 0768a1df | Christos Stavrakakis | |
531 | 0768a1df | Christos Stavrakakis | Handling of Networks, as far as backends are concerned, is based on whether the |
532 | a1c707c9 | Constantinos Venetsanopoulos | network is public or not. Public networks are created through the `snf-manage |
533 | a1c707c9 | Constantinos Venetsanopoulos | network-create` command, and are only created on one backend. Private networks |
534 | a1c707c9 | Constantinos Venetsanopoulos | are created on all backends, in order to ensure that VMs residing on different |
535 | a1c707c9 | Constantinos Venetsanopoulos | backends can be connected to the same private network. |
536 | 0768a1df | Christos Stavrakakis | |
537 | a1c707c9 | Constantinos Venetsanopoulos | Listing existing backends |
538 | a1c707c9 | Constantinos Venetsanopoulos | ````````````````````````` |
539 | a1c707c9 | Constantinos Venetsanopoulos | To list all the Ganeti backends known to Synnefo, we run: |
540 | 90c74332 | Christos Stavrakakis | |
541 | 0768a1df | Christos Stavrakakis | .. code-block:: console |
542 | 0768a1df | Christos Stavrakakis | |
543 | 0768a1df | Christos Stavrakakis | $ snf-manage backend-list |
544 | 0768a1df | Christos Stavrakakis | |
545 | 0768a1df | Christos Stavrakakis | Adding a new Ganeti backend |
546 | 90c74332 | Christos Stavrakakis | ``````````````````````````` |
547 | 0768a1df | Christos Stavrakakis | Backends are dynamically added under the control of Synnefo with `snf-manage |
548 | 0768a1df | Christos Stavrakakis | backend-add` command. In this section it is assumed that a Ganeti cluster, |
549 | a1c707c9 | Constantinos Venetsanopoulos | named ``cluster.example.com`` is already up and running and configured to be |
550 | a1c707c9 | Constantinos Venetsanopoulos | able to host Synnefo VMs. |
551 | 0768a1df | Christos Stavrakakis | |
552 | a1c707c9 | Constantinos Venetsanopoulos | To add this Ganeti cluster, we run: |
553 | 90c74332 | Christos Stavrakakis | |
554 | 0768a1df | Christos Stavrakakis | .. code-block:: console |
555 | 0768a1df | Christos Stavrakakis | |
556 | 0768a1df | Christos Stavrakakis | $ snf-manage backend-add --clustername=cluster.example.com --user="synnefo_user" --pass="synnefo_pass" |
557 | 0768a1df | Christos Stavrakakis | |
558 | 46ccbcb9 | Christos Stavrakakis | where ``clustername`` is the Cluster hostname of the Ganeti cluster, and |
559 | 46ccbcb9 | Christos Stavrakakis | ``user`` and ``pass`` are the credentials for the `Ganeti RAPI user |
560 | 46ccbcb9 | Christos Stavrakakis | <http://docs.ganeti.org/ganeti/2.2/html/rapi.html#users-and-passwords>`_. All |
561 | 46ccbcb9 | Christos Stavrakakis | backend attributes can be also changed dynamically using the `snf-manage |
562 | 46ccbcb9 | Christos Stavrakakis | backend-modify` command. |
563 | 0768a1df | Christos Stavrakakis | |
564 | a1c707c9 | Constantinos Venetsanopoulos | ``snf-manage backend-add`` will also create all existing private networks to |
565 | a1c707c9 | Constantinos Venetsanopoulos | the new backend. You can verify that the backend is added, by running |
566 | a1c707c9 | Constantinos Venetsanopoulos | `snf-manage backend-list`. |
567 | 0768a1df | Christos Stavrakakis | |
568 | a1c707c9 | Constantinos Venetsanopoulos | Note that no VMs will be spawned to this backend, since by default it is in a |
569 | a1c707c9 | Constantinos Venetsanopoulos | ``drained`` state after addition and also it has no public network assigned to |
570 | a1c707c9 | Constantinos Venetsanopoulos | it. |
571 | a1c707c9 | Constantinos Venetsanopoulos | |
572 | a1c707c9 | Constantinos Venetsanopoulos | So, first you need to create its public network, make sure everything works as |
573 | a1c707c9 | Constantinos Venetsanopoulos | expected and finally make it active by un-setting the ``drained`` flag. You can |
574 | a1c707c9 | Constantinos Venetsanopoulos | do this by running: |
575 | a1c707c9 | Constantinos Venetsanopoulos | |
576 | a1c707c9 | Constantinos Venetsanopoulos | .. code-block:: console |
577 | a1c707c9 | Constantinos Venetsanopoulos | |
578 | 46ccbcb9 | Christos Stavrakakis | $ snf-manage backend-modify --drained=False <backend_id> |
579 | 0768a1df | Christos Stavrakakis | |
580 | 0768a1df | Christos Stavrakakis | Removing an existing Ganeti backend |
581 | 90c74332 | Christos Stavrakakis | ``````````````````````````````````` |
582 | a1c707c9 | Constantinos Venetsanopoulos | In order to remove an existing backend from Synnefo, we run: |
583 | 90c74332 | Christos Stavrakakis | |
584 | 0768a1df | Christos Stavrakakis | .. code-block:: console |
585 | 0768a1df | Christos Stavrakakis | |
586 | a1c707c9 | Constantinos Venetsanopoulos | # snf-manage backend-remove <backend_id> |
587 | 0768a1df | Christos Stavrakakis | |
588 | a1c707c9 | Constantinos Venetsanopoulos | This command will fail if there are active VMs on the backend. Also, the |
589 | a1c707c9 | Constantinos Venetsanopoulos | backend is not cleaned before removal, so all the Synnefo private networks |
590 | a1c707c9 | Constantinos Venetsanopoulos | will be left on the Ganeti nodes. You need to remove them manually. |
591 | 0768a1df | Christos Stavrakakis | |
592 | a1c707c9 | Constantinos Venetsanopoulos | Allocation of VMs in Ganeti backends |
593 | a1c707c9 | Constantinos Venetsanopoulos | ```````````````````````````````````` |
594 | a1c707c9 | Constantinos Venetsanopoulos | As already mentioned, the Cyclades backend allocator is responsible for |
595 | a1c707c9 | Constantinos Venetsanopoulos | allocating new VMs to backends. This allocator does not choose the exact Ganeti |
596 | a1c707c9 | Constantinos Venetsanopoulos | node that will host the VM but just the Ganeti backend. The exact node is |
597 | a1c707c9 | Constantinos Venetsanopoulos | chosen by the Ganeti cluster's allocator (hail). |
598 | 0768a1df | Christos Stavrakakis | |
599 | 0768a1df | Christos Stavrakakis | The decision about which backend will host a VM is based on the available |
600 | 0768a1df | Christos Stavrakakis | resources. The allocator computes a score for each backend, that shows its load |
601 | 0768a1df | Christos Stavrakakis | factor, and the one with the minimum score is chosen. The admin can exclude |
602 | a1c707c9 | Constantinos Venetsanopoulos | backends from the allocation phase by marking them as ``drained`` by running: |
603 | 90c74332 | Christos Stavrakakis | |
604 | 0768a1df | Christos Stavrakakis | .. code-block:: console |
605 | 0768a1df | Christos Stavrakakis | |
606 | 46ccbcb9 | Christos Stavrakakis | $ snf-manage backend-modify --drained=True <backend_id> |
607 | 0768a1df | Christos Stavrakakis | |
608 | 0768a1df | Christos Stavrakakis | The backend resources are periodically updated, at a period defined by |
609 | a1c707c9 | Constantinos Venetsanopoulos | the ``BACKEND_REFRESH_MIN`` setting, or by running `snf-manage backend-update-status` |
610 | 0768a1df | Christos Stavrakakis | command. It is advised to have a cron job running this command at a smaller |
611 | a1c707c9 | Constantinos Venetsanopoulos | interval than ``BACKEND_REFRESH_MIN`` in order to remove the load of refreshing |
612 | 0768a1df | Christos Stavrakakis | the backends stats from the VM creation phase. |
613 | 0768a1df | Christos Stavrakakis | |
614 | a1c707c9 | Constantinos Venetsanopoulos | Finally, the admin can decide to have a user's VMs being allocated to a |
615 | a1c707c9 | Constantinos Venetsanopoulos | specific backend, with the ``BACKEND_PER_USER`` setting. This is a mapping |
616 | a1c707c9 | Constantinos Venetsanopoulos | between users and backends. If the user is found in ``BACKEND_PER_USER``, then |
617 | 46ccbcb9 | Christos Stavrakakis | Synnefo allocates all his/hers VMs to the specific backend in the variable, |
618 | 46ccbcb9 | Christos Stavrakakis | even if is marked as drained (useful for testing). |
619 | 0768a1df | Christos Stavrakakis | |
620 | 9492387a | Christos Stavrakakis | |
621 | 9492387a | Christos Stavrakakis | Managing Virtual Machines |
622 | 9492387a | Christos Stavrakakis | ~~~~~~~~~~~~~~~~~~~~~~~~~ |
623 | 9492387a | Christos Stavrakakis | |
624 | a1c707c9 | Constantinos Venetsanopoulos | As mentioned, Cyclades uses Ganeti for management of VMs. The administrator can |
625 | a1c707c9 | Constantinos Venetsanopoulos | handle Cyclades VMs just like any other Ganeti instance, via `gnt-instance` |
626 | a1c707c9 | Constantinos Venetsanopoulos | commands. All Ganeti instances that belong to Synnefo, are separated from |
627 | a1c707c9 | Constantinos Venetsanopoulos | others, by a prefix in their names. This prefix is defined in |
628 | a1c707c9 | Constantinos Venetsanopoulos | ``BACKEND_PREFIX_ID`` setting in |
629 | a1c707c9 | Constantinos Venetsanopoulos | ``/etc/synnefo/20-snf-cyclades-app-backend.conf``. |
630 | 9492387a | Christos Stavrakakis | |
631 | a1c707c9 | Constantinos Venetsanopoulos | Apart from handling instances directly in the Ganeti level, a number of `snf-manage` |
632 | 9492387a | Christos Stavrakakis | commands are available: |
633 | 9492387a | Christos Stavrakakis | |
634 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-list``: List servers |
635 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-show``: Show information about a server in the Cyclades DB |
636 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-inspect``: Inspect the state of a server both in DB and Ganeti |
637 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-modify``: Modify the state of a server in the Cycldes DB |
638 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-create``: Create a new server |
639 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-import``: Import an existing Ganeti instance to Cyclades |
640 | 9492387a | Christos Stavrakakis | |
641 | 9492387a | Christos Stavrakakis | |
642 | 9492387a | Christos Stavrakakis | Managing Virtual Networks |
643 | 9492387a | Christos Stavrakakis | ~~~~~~~~~~~~~~~~~~~~~~~~~ |
644 | 9492387a | Christos Stavrakakis | |
645 | a1c707c9 | Constantinos Venetsanopoulos | Cyclades is able to create and manage Virtual Networks. Networking is |
646 | a1c707c9 | Constantinos Venetsanopoulos | desployment specific and must be customized based on the specific needs of the |
647 | a1c707c9 | Constantinos Venetsanopoulos | system administrator. For better understanding of networking please refer to |
648 | a1c707c9 | Constantinos Venetsanopoulos | the :ref:`Network <networks>` section. |
649 | a653c059 | Christos Stavrakakis | |
650 | a653c059 | Christos Stavrakakis | Exactly as Cyclades VMs can be handled like Ganeti instances, Cyclades Networks |
651 | a653c059 | Christos Stavrakakis | can also by handled as Ganeti networks, via `gnt-network commands`. All Ganeti |
652 | a1c707c9 | Constantinos Venetsanopoulos | networks that belong to Synnefo are named with the prefix |
653 | a653c059 | Christos Stavrakakis | `${BACKEND_PREFIX_ID}-net-`. |
654 | a653c059 | Christos Stavrakakis | |
655 | a1c707c9 | Constantinos Venetsanopoulos | There are also the following `snf-manage` commands for managing networks: |
656 | a653c059 | Christos Stavrakakis | |
657 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-list``: List networks |
658 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-show``: Show information about a network in the Cyclades DB |
659 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-inspect``: Inspect the state of the network in DB and Ganeti backends |
660 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-modify``: Modify the state of a network in the Cycldes DB |
661 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-create``: Create a new network |
662 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-remove``: Remove an existing network |
663 | a653c059 | Christos Stavrakakis | |
664 | 9c98c6fc | Christos Stavrakakis | Managing Network Resources |
665 | a1c707c9 | Constantinos Venetsanopoulos | `````````````````````````` |
666 | 9c98c6fc | Christos Stavrakakis | |
667 | a1c707c9 | Constantinos Venetsanopoulos | Proper operation of the Cyclades Network Service depends on the unique |
668 | a1c707c9 | Constantinos Venetsanopoulos | assignment of specific resources to each type of virtual network. Specifically, |
669 | a1c707c9 | Constantinos Venetsanopoulos | these resources are: |
670 | 9c98c6fc | Christos Stavrakakis | |
671 | a1c707c9 | Constantinos Venetsanopoulos | * IP addresses. Cyclades creates a Pool of IPs for each Network, and assigns a |
672 | a1c707c9 | Constantinos Venetsanopoulos | unique IP address to each VM, thus connecting it to this Network. You can see |
673 | a1c707c9 | Constantinos Venetsanopoulos | the IP pool of each network by running `snf-manage network-inspect |
674 | a1c707c9 | Constantinos Venetsanopoulos | <network_ID>`. IP pools are automatically created and managed by Cyclades, |
675 | a1c707c9 | Constantinos Venetsanopoulos | depending on the subnet of the Network. |
676 | 9492387a | Christos Stavrakakis | * Bridges corresponding to physical VLANs, which are required for networks of |
677 | a1c707c9 | Constantinos Venetsanopoulos | type `PRIVATE_PHYSICAL_VLAN`. |
678 | a1c707c9 | Constantinos Venetsanopoulos | * One Bridge corresponding to one physical VLAN which is required for networks of |
679 | a1c707c9 | Constantinos Venetsanopoulos | type `PRIVATE_MAC_PREFIX`. |
680 | 9c98c6fc | Christos Stavrakakis | |
681 | a1c707c9 | Constantinos Venetsanopoulos | Cyclades allocates those resources from pools that are created by the |
682 | a1c707c9 | Constantinos Venetsanopoulos | administrator with the `snf-manage pool-create` management command. |
683 | 9c98c6fc | Christos Stavrakakis | |
684 | a1c707c9 | Constantinos Venetsanopoulos | Pool Creation |
685 | a1c707c9 | Constantinos Venetsanopoulos | ````````````` |
686 | a1c707c9 | Constantinos Venetsanopoulos | Pools are created using the `snf-manage pool-create` command: |
687 | 90c74332 | Christos Stavrakakis | |
688 | 9c98c6fc | Christos Stavrakakis | .. code-block:: console |
689 | 9c98c6fc | Christos Stavrakakis | |
690 | 9c98c6fc | Christos Stavrakakis | # snf-manage pool-create --type=bridge --base=prv --size=20 |
691 | 9c98c6fc | Christos Stavrakakis | |
692 | 9c98c6fc | Christos Stavrakakis | will create a pool of bridges, containing bridges prv1, prv2,..prv21. |
693 | 9c98c6fc | Christos Stavrakakis | |
694 | 9c98c6fc | Christos Stavrakakis | You can verify the creation of the pool, and check its contents by running: |
695 | 90c74332 | Christos Stavrakakis | |
696 | 9c98c6fc | Christos Stavrakakis | .. code-block:: console |
697 | 9c98c6fc | Christos Stavrakakis | |
698 | 9c98c6fc | Christos Stavrakakis | # snf-manage pool-list |
699 | 9c98c6fc | Christos Stavrakakis | # snf-manage pool-show --type=bridge 1 |
700 | 9c98c6fc | Christos Stavrakakis | |
701 | a1c707c9 | Constantinos Venetsanopoulos | With the same commands you can handle a pool of MAC prefixes. For example: |
702 | 90c74332 | Christos Stavrakakis | |
703 | 9c98c6fc | Christos Stavrakakis | .. code-block:: console |
704 | 9c98c6fc | Christos Stavrakakis | |
705 | 9c98c6fc | Christos Stavrakakis | # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536 |
706 | 9c98c6fc | Christos Stavrakakis | |
707 | a1c707c9 | Constantinos Venetsanopoulos | will create a pool of MAC prefixes from ``aa:00:1`` to ``b9:ff:f``. The MAC |
708 | a1c707c9 | Constantinos Venetsanopoulos | prefix pool is responsible for providing only unicast and locally administered |
709 | a1c707c9 | Constantinos Venetsanopoulos | MAC addresses, so many of these prefixes will be externally reserved, to |
710 | a1c707c9 | Constantinos Venetsanopoulos | exclude from allocation. |
711 | 0768a1df | Christos Stavrakakis | |
712 | eab73849 | Constantinos Venetsanopoulos | Cyclades advanced operations |
713 | eab73849 | Constantinos Venetsanopoulos | ---------------------------- |
714 | eab73849 | Constantinos Venetsanopoulos | |
715 | eab73849 | Constantinos Venetsanopoulos | Reconciliation mechanism |
716 | eab73849 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~~~~~~~ |
717 | a1c707c9 | Constantinos Venetsanopoulos | |
718 | 06e23415 | Christos Stavrakakis | On certain occasions, such as a Ganeti or RabbitMQ failure, the state of |
719 | 06e23415 | Christos Stavrakakis | Cyclades database may differ from the real state of VMs and networks in the |
720 | 06e23415 | Christos Stavrakakis | Ganeti backends. The reconciliation process is designed to synchronize |
721 | 06e23415 | Christos Stavrakakis | the state of the Cyclades DB with Ganeti. There are two management commands |
722 | 06e23415 | Christos Stavrakakis | for reconciling VMs and Networks |
723 | 06e23415 | Christos Stavrakakis | |
724 | a1c707c9 | Constantinos Venetsanopoulos | Reconciling Virtual Machines |
725 | a1c707c9 | Constantinos Venetsanopoulos | ```````````````````````````` |
726 | a1c707c9 | Constantinos Venetsanopoulos | |
727 | 06e23415 | Christos Stavrakakis | Reconciliation of VMs detects the following conditions: |
728 | a1c707c9 | Constantinos Venetsanopoulos | |
729 | eab73849 | Constantinos Venetsanopoulos | * Stale DB servers without corresponding Ganeti instances |
730 | eab73849 | Constantinos Venetsanopoulos | * Orphan Ganeti instances, without corresponding DB entries |
731 | 06e23415 | Christos Stavrakakis | * Out-of-sync state for DB entries wrt to Ganeti instances |
732 | eab73849 | Constantinos Venetsanopoulos | |
733 | 06e23415 | Christos Stavrakakis | To detect all inconsistencies you can just run: |
734 | 90c74332 | Christos Stavrakakis | |
735 | eab73849 | Constantinos Venetsanopoulos | .. code-block:: console |
736 | a1c707c9 | Constantinos Venetsanopoulos | |
737 | a1c707c9 | Constantinos Venetsanopoulos | $ snf-manage reconcile-servers |
738 | eab73849 | Constantinos Venetsanopoulos | |
739 | 06e23415 | Christos Stavrakakis | Adding the `--fix-all` option, will do the actual synchronization: |
740 | 90c74332 | Christos Stavrakakis | |
741 | 06e23415 | Christos Stavrakakis | .. code-block:: console |
742 | a1c707c9 | Constantinos Venetsanopoulos | |
743 | a1c707c9 | Constantinos Venetsanopoulos | $ snf-manage reconcile --fix-all |
744 | eab73849 | Constantinos Venetsanopoulos | |
745 | eab73849 | Constantinos Venetsanopoulos | Please see ``snf-manage reconcile --help`` for all the details. |
746 | eab73849 | Constantinos Venetsanopoulos | |
747 | eab73849 | Constantinos Venetsanopoulos | |
748 | 06e23415 | Christos Stavrakakis | Reconciling Networks |
749 | 90c74332 | Christos Stavrakakis | ```````````````````` |
750 | a1c707c9 | Constantinos Venetsanopoulos | |
751 | 06e23415 | Christos Stavrakakis | Reconciliation of Networks detects the following conditions: |
752 | a1c707c9 | Constantinos Venetsanopoulos | |
753 | 06e23415 | Christos Stavrakakis | * Stale DB networks without corresponding Ganeti networks |
754 | 06e23415 | Christos Stavrakakis | * Orphan Ganeti networks, without corresponding DB entries |
755 | 06e23415 | Christos Stavrakakis | * Private networks that are not created to all Ganeti backends |
756 | 06e23415 | Christos Stavrakakis | * Unsynchronized IP pools |
757 | 06e23415 | Christos Stavrakakis | |
758 | 06e23415 | Christos Stavrakakis | To detect all inconsistencies you can just run: |
759 | 90c74332 | Christos Stavrakakis | |
760 | 06e23415 | Christos Stavrakakis | .. code-block:: console |
761 | a1c707c9 | Constantinos Venetsanopoulos | |
762 | 06e23415 | Christos Stavrakakis | $ snf-manage reconcile-networks |
763 | 06e23415 | Christos Stavrakakis | |
764 | 06e23415 | Christos Stavrakakis | Adding the `--fix-all` option, will do the actual synchronization: |
765 | 90c74332 | Christos Stavrakakis | |
766 | 06e23415 | Christos Stavrakakis | .. code-block:: console |
767 | a1c707c9 | Constantinos Venetsanopoulos | |
768 | 06e23415 | Christos Stavrakakis | $ snf-manage reconcile-networks --fix-all |
769 | 06e23415 | Christos Stavrakakis | |
770 | 06e23415 | Christos Stavrakakis | Please see ``snf-manage reconcile-networks --help`` for all the details. |
771 | eab73849 | Constantinos Venetsanopoulos | |
772 | eab73849 | Constantinos Venetsanopoulos | |
773 | a1c707c9 | Constantinos Venetsanopoulos | |
774 | eab73849 | Constantinos Venetsanopoulos | Block Storage Service (Archipelago) |
775 | eab73849 | Constantinos Venetsanopoulos | =================================== |
776 | eab73849 | Constantinos Venetsanopoulos | |
777 | eab73849 | Constantinos Venetsanopoulos | Overview |
778 | eab73849 | Constantinos Venetsanopoulos | -------- |
779 | 5226b38f | Filippos Giannakos | Archipelago offers Copy-On-Write snapshotable volumes. Pithos images can be used |
780 | 5226b38f | Filippos Giannakos | to provision a volume with Copy-On-Write semantics (i.e. a clone). Snapshots |
781 | 5226b38f | Filippos Giannakos | offer a unique deduplicated image of a volume, that reflects the volume state |
782 | 5226b38f | Filippos Giannakos | during snapshot creation and are indistinguishable from a Pithos image. |
783 | 5226b38f | Filippos Giannakos | |
784 | 5226b38f | Filippos Giannakos | Archipelago is used by Cyclades and Ganeti for fast provisioning of VMs based on |
785 | 4bb29566 | Filippos Giannakos | CoW volumes. Moreover, it enables live migration of thinly-provisioned VMs with |
786 | 4bb29566 | Filippos Giannakos | no physically shared storage. |
787 | eab73849 | Constantinos Venetsanopoulos | |
788 | 301294a9 | Constantinos Venetsanopoulos | Archipelago Architecture |
789 | 301294a9 | Constantinos Venetsanopoulos | ------------------------ |
790 | 301294a9 | Constantinos Venetsanopoulos | |
791 | e3f3abdc | Filippos Giannakos | .. image:: images/archipelago-architecture.png |
792 | e3f3abdc | Filippos Giannakos | :width: 50% |
793 | e3f3abdc | Filippos Giannakos | :target: _images/archipelago-architecture.png |
794 | eab73849 | Constantinos Venetsanopoulos | |
795 | 301294a9 | Constantinos Venetsanopoulos | .. _syn+archip+rados: |
796 | 301294a9 | Constantinos Venetsanopoulos | |
797 | 301294a9 | Constantinos Venetsanopoulos | Overview of Synnefo + Archipelago + RADOS |
798 | 301294a9 | Constantinos Venetsanopoulos | ----------------------------------------- |
799 | 301294a9 | Constantinos Venetsanopoulos | |
800 | 301294a9 | Constantinos Venetsanopoulos | .. image:: images/synnefo-arch3.png |
801 | 301294a9 | Constantinos Venetsanopoulos | :width: 100% |
802 | 301294a9 | Constantinos Venetsanopoulos | :target: _images/synnefo-arch3.png |
803 | 301294a9 | Constantinos Venetsanopoulos | |
804 | eab73849 | Constantinos Venetsanopoulos | Prereqs |
805 | eab73849 | Constantinos Venetsanopoulos | ------- |
806 | 301294a9 | Constantinos Venetsanopoulos | |
807 | 902d661c | Filippos Giannakos | The administrator must initialize the storage backend where archipelago volume |
808 | 902d661c | Filippos Giannakos | blocks will reside. |
809 | 902d661c | Filippos Giannakos | |
810 | 902d661c | Filippos Giannakos | In case of a files backend, the administrator must create two directories. One |
811 | 902d661c | Filippos Giannakos | for the archipelago data blocks and one for the archipelago map blocks. These |
812 | 902d661c | Filippos Giannakos | should probably be over shared storage to enable sharing archipelago volumes |
813 | 902d661c | Filippos Giannakos | between multiple nodes. He or she, must also be able to supply a directory where |
814 | 902d661c | Filippos Giannakos | the pithos data and map blocks reside. |
815 | 902d661c | Filippos Giannakos | |
816 | 902d661c | Filippos Giannakos | In case of a RADOS backend, the administrator must create two rados pools, one |
817 | 902d661c | Filippos Giannakos | for data blocks, and one for the map blocks. These pools, must be the same pools |
818 | 902d661c | Filippos Giannakos | used in pithos, in order to enable volume creation based on pithos images. |
819 | 902d661c | Filippos Giannakos | |
820 | eab73849 | Constantinos Venetsanopoulos | Installation |
821 | eab73849 | Constantinos Venetsanopoulos | ------------ |
822 | 301294a9 | Constantinos Venetsanopoulos | |
823 | 902d661c | Filippos Giannakos | Archipelago consists of |
824 | 902d661c | Filippos Giannakos | |
825 | 902d661c | Filippos Giannakos | * ``libxseg0``: libxseg used to communicate over shared memory segments |
826 | 5226b38f | Filippos Giannakos | * ``python-xseg``: python bindings for libxseg |
827 | 902d661c | Filippos Giannakos | * ``archipelago-kernel-dkms``: contains archipelago kernel modules to provide |
828 | 902d661c | Filippos Giannakos | block devices to be used as vm disks |
829 | 5226b38f | Filippos Giannakos | * ``python-archipelago``: archipelago python module. Includes archipelago and |
830 | 5226b38f | Filippos Giannakos | vlmc functionality. |
831 | 902d661c | Filippos Giannakos | * ``archipelago``: user space tools and peers for the archipelago management and |
832 | 902d661c | Filippos Giannakos | volume composition |
833 | 902d661c | Filippos Giannakos | * ``archipelago-ganeti``: ganeti ext storage scripts, that enable ganeti to |
834 | 902d661c | Filippos Giannakos | provision VMs over archipelago |
835 | 902d661c | Filippos Giannakos | |
836 | 902d661c | Filippos Giannakos | Performing |
837 | 902d661c | Filippos Giannakos | |
838 | 902d661c | Filippos Giannakos | .. code-block:: console |
839 | 902d661c | Filippos Giannakos | |
840 | 902d661c | Filippos Giannakos | $ apt-get install archipelago-ganeti |
841 | 902d661c | Filippos Giannakos | |
842 | 902d661c | Filippos Giannakos | should fetch all the required packages and get you up 'n going with archipelago |
843 | 902d661c | Filippos Giannakos | |
844 | 902d661c | Filippos Giannakos | Bare in mind, that custom librados is required, which is provided in the apt |
845 | 5226b38f | Filippos Giannakos | repo of GRNet. |
846 | 902d661c | Filippos Giannakos | |
847 | 902d661c | Filippos Giannakos | |
848 | 5226b38f | Filippos Giannakos | For now, librados is a dependency of archipelago, even if you do not intend to |
849 | 5226b38f | Filippos Giannakos | use archipelago over RADOS. |
850 | eab73849 | Constantinos Venetsanopoulos | |
851 | eab73849 | Constantinos Venetsanopoulos | Configuration |
852 | eab73849 | Constantinos Venetsanopoulos | ------------- |
853 | 902d661c | Filippos Giannakos | Archipelago should work out of the box with a RADOS backend, but basic |
854 | 902d661c | Filippos Giannakos | configuration can be done in ``/etc/default/archipelago`` . |
855 | 902d661c | Filippos Giannakos | |
856 | 902d661c | Filippos Giannakos | If you wish to change the storage backend to files, set |
857 | 902d661c | Filippos Giannakos | |
858 | 902d661c | Filippos Giannakos | .. code-block:: console |
859 | 902d661c | Filippos Giannakos | |
860 | 902d661c | Filippos Giannakos | STORAGE="files" |
861 | 902d661c | Filippos Giannakos | |
862 | 902d661c | Filippos Giannakos | and provide the appropriate settings for files storage backend in the conf file. |
863 | 902d661c | Filippos Giannakos | |
864 | 902d661c | Filippos Giannakos | These are: |
865 | 902d661c | Filippos Giannakos | |
866 | 902d661c | Filippos Giannakos | * ``FILED_IMAGES``: directory for archipelago data blocks. |
867 | 902d661c | Filippos Giannakos | * ``FILED_MAPS``: directory for archipelago map blocks. |
868 | 902d661c | Filippos Giannakos | * ``PITHOS``: directory of pithos data blocks. |
869 | 902d661c | Filippos Giannakos | * ``PITHOSMAPS``: directory of pithos map blocks. |
870 | 902d661c | Filippos Giannakos | |
871 | 902d661c | Filippos Giannakos | The settings for RADOS storage backend are: |
872 | 902d661c | Filippos Giannakos | |
873 | 902d661c | Filippos Giannakos | * ``RADOS_POOL_MAPS``: The pool where archipelago and pithos map blocks reside. |
874 | 902d661c | Filippos Giannakos | * ``RADOS_POOL_BLOCKS``: The pool where archipelago and pithos data blocks |
875 | 5226b38f | Filippos Giannakos | reside. |
876 | 902d661c | Filippos Giannakos | |
877 | 902d661c | Filippos Giannakos | Examples can be found in the conf file. |
878 | 902d661c | Filippos Giannakos | |
879 | 902d661c | Filippos Giannakos | Be aware that archipelago infrastructure doesn't provide default values for this |
880 | 902d661c | Filippos Giannakos | settings. If they are not set in the conf file, archipelago will not be able to |
881 | 902d661c | Filippos Giannakos | function. |
882 | 902d661c | Filippos Giannakos | |
883 | 902d661c | Filippos Giannakos | Archipelago also provides ``VERBOSITY`` config options to control the output |
884 | 902d661c | Filippos Giannakos | generated by the userspace peers. |
885 | 902d661c | Filippos Giannakos | |
886 | 902d661c | Filippos Giannakos | The available options are: |
887 | 902d661c | Filippos Giannakos | |
888 | 902d661c | Filippos Giannakos | * ``VERBOSITY_BLOCKERB`` |
889 | 902d661c | Filippos Giannakos | * ``VERBOSITY_BLOCKERM`` |
890 | 902d661c | Filippos Giannakos | * ``VERBOSITY_MAPPER`` |
891 | 902d661c | Filippos Giannakos | * ``VERBOSITY_VLMC`` |
892 | 902d661c | Filippos Giannakos | |
893 | 902d661c | Filippos Giannakos | and the available values are: |
894 | 902d661c | Filippos Giannakos | |
895 | 902d661c | Filippos Giannakos | * 0 : Error only logging. |
896 | 902d661c | Filippos Giannakos | * 1 : Warning logging. |
897 | 902d661c | Filippos Giannakos | * 2 : Info logging. |
898 | 902d661c | Filippos Giannakos | * 3 : Debug logging. WARNING: This options produces tons of output, but the |
899 | 902d661c | Filippos Giannakos | logrotate daemon should take care of it. |
900 | eab73849 | Constantinos Venetsanopoulos | |
901 | eab73849 | Constantinos Venetsanopoulos | Working with Archipelago |
902 | eab73849 | Constantinos Venetsanopoulos | ------------------------ |
903 | eab73849 | Constantinos Venetsanopoulos | |
904 | 902d661c | Filippos Giannakos | ``archipelago`` provides basic functionality for archipelago. |
905 | 5226b38f | Filippos Giannakos | |
906 | 5226b38f | Filippos Giannakos | Usage: |
907 | 5226b38f | Filippos Giannakos | |
908 | 5226b38f | Filippos Giannakos | .. code-block:: console |
909 | 5226b38f | Filippos Giannakos | |
910 | 5226b38f | Filippos Giannakos | $ archipelago [-u] command |
911 | 5226b38f | Filippos Giannakos | |
912 | 5226b38f | Filippos Giannakos | |
913 | 902d661c | Filippos Giannakos | Currently it supports the following commands: |
914 | 902d661c | Filippos Giannakos | |
915 | 5226b38f | Filippos Giannakos | * ``start [peer]`` |
916 | 5226b38f | Filippos Giannakos | Starts archipelago or the specified peer. |
917 | 5226b38f | Filippos Giannakos | * ``stop [peer]`` |
918 | 5226b38f | Filippos Giannakos | Stops archipelago or the specified peer. |
919 | 5226b38f | Filippos Giannakos | * ``restart [peer]`` |
920 | 5226b38f | Filippos Giannakos | Restarts archipelago or the specified peer. |
921 | 902d661c | Filippos Giannakos | * ``status`` |
922 | 5226b38f | Filippos Giannakos | Show the status of archipelago. |
923 | 5226b38f | Filippos Giannakos | |
924 | 5226b38f | Filippos Giannakos | Available peers: ``blockerm``, ``blockerb``, ``mapperd``, ``vlmcd``. |
925 | 5226b38f | Filippos Giannakos | |
926 | 902d661c | Filippos Giannakos | |
927 | 902d661c | Filippos Giannakos | ``start``, ``stop``, ``restart`` can be combined with the ``-u / --user`` option |
928 | 902d661c | Filippos Giannakos | to affect only the userspace peers supporting archipelago. |
929 | 902d661c | Filippos Giannakos | |
930 | 902d661c | Filippos Giannakos | |
931 | 902d661c | Filippos Giannakos | |
932 | eab73849 | Constantinos Venetsanopoulos | Archipelago advanced operations |
933 | eab73849 | Constantinos Venetsanopoulos | ------------------------------- |
934 | 902d661c | Filippos Giannakos | The ``vlmc`` tool provides a way to interact with archipelago volumes |
935 | 902d661c | Filippos Giannakos | |
936 | 902d661c | Filippos Giannakos | * ``vlmc map <volumename>``: maps the volume to a xsegbd device. |
937 | 6c900328 | Filippos Giannakos | |
938 | 5226b38f | Filippos Giannakos | * ``vlmc unmap </dev/xsegbd[1-..]>``: unmaps the specified device from the |
939 | 902d661c | Filippos Giannakos | system. |
940 | 6c900328 | Filippos Giannakos | |
941 | 902d661c | Filippos Giannakos | * ``vlmc create <volumename> --snap <snapname> --size <size>``: creates a new |
942 | 902d661c | Filippos Giannakos | volume named <volumename> from snapshot name <snapname> with size <size>. |
943 | 301294a9 | Constantinos Venetsanopoulos | The ``--snap`` and ``--size`` are optional, but at least one of them is |
944 | 301294a9 | Constantinos Venetsanopoulos | mandatory. e.g: |
945 | 902d661c | Filippos Giannakos | |
946 | 301294a9 | Constantinos Venetsanopoulos | ``vlmc create <volumename> --snap <snapname>`` creates a volume named |
947 | 301294a9 | Constantinos Venetsanopoulos | volumename from snapshot snapname. The size of the volume is the same as |
948 | 301294a9 | Constantinos Venetsanopoulos | the size of the snapshot. |
949 | 902d661c | Filippos Giannakos | |
950 | 301294a9 | Constantinos Venetsanopoulos | ``vlmc create <volumename> --size <size>`` creates an empty volume of size |
951 | 301294a9 | Constantinos Venetsanopoulos | <size> named <volumename>. |
952 | 6c900328 | Filippos Giannakos | |
953 | 902d661c | Filippos Giannakos | * ``vlmc remove <volumename>``: removes the volume and all the related |
954 | 902d661c | Filippos Giannakos | archipelago blocks from storage. |
955 | 6c900328 | Filippos Giannakos | |
956 | 902d661c | Filippos Giannakos | * ``vlmc list``: provides a list of archipelago volumes. Currently only works |
957 | 902d661c | Filippos Giannakos | with RADOS storage backend. |
958 | f846d8df | Constantinos Venetsanopoulos | |
959 | 5226b38f | Filippos Giannakos | * ``vlmc info <volumename>``: shows volume information. Currently returns only |
960 | 5226b38f | Filippos Giannakos | volume size. |
961 | 5226b38f | Filippos Giannakos | |
962 | 6c900328 | Filippos Giannakos | * ``vlmc open <volumename>``: opens an archipelago volume. That is, taking all |
963 | 6c900328 | Filippos Giannakos | the necessary locks and also make the rest of the infrastructure aware of the |
964 | 6c900328 | Filippos Giannakos | operation. |
965 | 6c900328 | Filippos Giannakos | |
966 | 6c900328 | Filippos Giannakos | This operation succeeds if the volume is alread opened. |
967 | 6c900328 | Filippos Giannakos | |
968 | 6c900328 | Filippos Giannakos | * ``vlmc close <volumename>``: closes an archipelago volume. That is, performing |
969 | 6c900328 | Filippos Giannakos | all the necessary functions in the insfrastrure to successfully release the |
970 | 6c900328 | Filippos Giannakos | volume. Also releases all the acquired locks. |
971 | 6c900328 | Filippos Giannakos | |
972 | 6c900328 | Filippos Giannakos | ``vlmc close`` should be performed after a ``vlmc open`` operation. |
973 | 6c900328 | Filippos Giannakos | |
974 | 6c900328 | Filippos Giannakos | * ``vlmc lock <volumename>``: locks a volume. This step allow the administrator |
975 | 6c900328 | Filippos Giannakos | to lock an archipelago volume, independently from the rest of the |
976 | 5226b38f | Filippos Giannakos | infrastrure. |
977 | 6c900328 | Filippos Giannakos | |
978 | 6c900328 | Filippos Giannakos | * ``vlmc unlock [-f] <volumename>``: unlocks a volume. This allow the |
979 | 6c900328 | Filippos Giannakos | administrator to unlock a volume, independently from the rest of the |
980 | 6c900328 | Filippos Giannakos | infrastructure. |
981 | 6c900328 | Filippos Giannakos | The unlock option can be performed only by the blocker that acquired the lock |
982 | 6c900328 | Filippos Giannakos | in the first place. To unlock a volume from another blocker, ``-f`` option |
983 | 6c900328 | Filippos Giannakos | must be used to break the lock. |
984 | 6c900328 | Filippos Giannakos | |
985 | a1c707c9 | Constantinos Venetsanopoulos | |
986 | e81ad3e5 | Constantinos Venetsanopoulos | Synnefo management commands ("snf-manage") |
987 | e81ad3e5 | Constantinos Venetsanopoulos | ========================================== |
988 | e81ad3e5 | Constantinos Venetsanopoulos | |
989 | e81ad3e5 | Constantinos Venetsanopoulos | Each Synnefo service, Astakos, Pithos and Cyclades are controlled by the |
990 | e81ad3e5 | Constantinos Venetsanopoulos | administrator using the "snf-manage" admin tool. This tool is an extension of |
991 | e81ad3e5 | Constantinos Venetsanopoulos | the Django command-line management utility. It is run on the host that runs |
992 | e81ad3e5 | Constantinos Venetsanopoulos | each service and provides different types of commands depending the services |
993 | e81ad3e5 | Constantinos Venetsanopoulos | running on the host. If you are running more than one service on the same host |
994 | e81ad3e5 | Constantinos Venetsanopoulos | "snf-manage" adds all the corresponding commands for each service dynamically, |
995 | e81ad3e5 | Constantinos Venetsanopoulos | providing a unified admin environment. |
996 | e81ad3e5 | Constantinos Venetsanopoulos | |
997 | e81ad3e5 | Constantinos Venetsanopoulos | To run "snf-manage" you just type: |
998 | e81ad3e5 | Constantinos Venetsanopoulos | |
999 | e81ad3e5 | Constantinos Venetsanopoulos | .. code-block:: console |
1000 | e81ad3e5 | Constantinos Venetsanopoulos | |
1001 | e81ad3e5 | Constantinos Venetsanopoulos | # snf-manage <command> [arguments] |
1002 | e81ad3e5 | Constantinos Venetsanopoulos | |
1003 | e81ad3e5 | Constantinos Venetsanopoulos | on the corresponding host that runs the service. For example, if you have all |
1004 | e81ad3e5 | Constantinos Venetsanopoulos | services running on different physical hosts you would do: |
1005 | e81ad3e5 | Constantinos Venetsanopoulos | |
1006 | e81ad3e5 | Constantinos Venetsanopoulos | .. code-block:: console |
1007 | e81ad3e5 | Constantinos Venetsanopoulos | |
1008 | e81ad3e5 | Constantinos Venetsanopoulos | root@astakos-host # snf-manage <astakos-command> [argument] |
1009 | e81ad3e5 | Constantinos Venetsanopoulos | root@pithos-host # snf-manage <pithos-command> [argument] |
1010 | e81ad3e5 | Constantinos Venetsanopoulos | root@cyclades-host # snf-manage <cyclades-command> [argument] |
1011 | e81ad3e5 | Constantinos Venetsanopoulos | |
1012 | e81ad3e5 | Constantinos Venetsanopoulos | If you have all services running on the same host you would do: |
1013 | e81ad3e5 | Constantinos Venetsanopoulos | |
1014 | e81ad3e5 | Constantinos Venetsanopoulos | .. code-block:: console |
1015 | e81ad3e5 | Constantinos Venetsanopoulos | |
1016 | e81ad3e5 | Constantinos Venetsanopoulos | root@synnefo-host # snf-manage <{astakos,pithos,cyclades}-command> [argument] |
1017 | e81ad3e5 | Constantinos Venetsanopoulos | |
1018 | e81ad3e5 | Constantinos Venetsanopoulos | Note that you cannot execute a service's command on a host that is not running |
1019 | e81ad3e5 | Constantinos Venetsanopoulos | this service. For example, the following will return an error if Astakos and |
1020 | e81ad3e5 | Constantinos Venetsanopoulos | Cyclades are installed on different physical hosts: |
1021 | e81ad3e5 | Constantinos Venetsanopoulos | |
1022 | e81ad3e5 | Constantinos Venetsanopoulos | .. code-block:: console |
1023 | e81ad3e5 | Constantinos Venetsanopoulos | |
1024 | e81ad3e5 | Constantinos Venetsanopoulos | root@astakos-host # snf-manage <cyclades-command> [argument] |
1025 | e81ad3e5 | Constantinos Venetsanopoulos | Unknown command: 'cyclades-command' |
1026 | e81ad3e5 | Constantinos Venetsanopoulos | Type 'snf-manage help' for usage. |
1027 | e81ad3e5 | Constantinos Venetsanopoulos | |
1028 | e81ad3e5 | Constantinos Venetsanopoulos | This is the complete list of "snf-manage" commands for each service. |
1029 | e81ad3e5 | Constantinos Venetsanopoulos | |
1030 | e81ad3e5 | Constantinos Venetsanopoulos | Astakos snf-manage commands |
1031 | e81ad3e5 | Constantinos Venetsanopoulos | --------------------------- |
1032 | e81ad3e5 | Constantinos Venetsanopoulos | |
1033 | e81ad3e5 | Constantinos Venetsanopoulos | ============================ =========================== |
1034 | e81ad3e5 | Constantinos Venetsanopoulos | Name Description |
1035 | e81ad3e5 | Constantinos Venetsanopoulos | ============================ =========================== |
1036 | e81ad3e5 | Constantinos Venetsanopoulos | fix-superusers Transform superusers created by syncdb into AstakosUser instances |
1037 | 5e1ea6f7 | Giorgos Korfiatis | cleanup-full Cleanup sessions and session catalog |
1038 | a4451f59 | Giorgos Korfiatis | commission-list List pending commissions |
1039 | a4451f59 | Giorgos Korfiatis | commission-show Show details for a pending commission |
1040 | 252eef38 | Giorgos Korfiatis | component-add Register a component |
1041 | 252eef38 | Giorgos Korfiatis | component-list List components |
1042 | 252eef38 | Giorgos Korfiatis | component-modify Modify component attributes |
1043 | e81ad3e5 | Constantinos Venetsanopoulos | project-control Manage projects and applications |
1044 | e81ad3e5 | Constantinos Venetsanopoulos | project-list List projects |
1045 | e81ad3e5 | Constantinos Venetsanopoulos | project-show Show project details |
1046 | e81ad3e5 | Constantinos Venetsanopoulos | quota List and check the integrity of user quota |
1047 | e81ad3e5 | Constantinos Venetsanopoulos | reconcile-resources-astakos Reconcile resource usage of Quotaholder with Astakos DB |
1048 | e81ad3e5 | Constantinos Venetsanopoulos | resource-export-astakos Export astakos resources in json format |
1049 | bd1f667b | Giorgos Korfiatis | resource-import Register resources |
1050 | e81ad3e5 | Constantinos Venetsanopoulos | resource-list List resources |
1051 | 5e1ea6f7 | Giorgos Korfiatis | resource-modify Modify a resource's default base quota and boolean flags |
1052 | bd1f667b | Giorgos Korfiatis | service-import Register services |
1053 | e81ad3e5 | Constantinos Venetsanopoulos | service-list List services |
1054 | 5e1ea6f7 | Giorgos Korfiatis | service-show Show service details |
1055 | e81ad3e5 | Constantinos Venetsanopoulos | term-add Add approval terms |
1056 | e81ad3e5 | Constantinos Venetsanopoulos | user-activation-send Send user activation |
1057 | e81ad3e5 | Constantinos Venetsanopoulos | user-add Add user |
1058 | 9577d199 | Kostas Papadimitriou | authpolicy-add Create a new authentication provider policy profile |
1059 | 9577d199 | Kostas Papadimitriou | authpolicy-list List existing authentication provider policy profiles |
1060 | 9577d199 | Kostas Papadimitriou | authpolicy-remove Remove an authentication provider policy |
1061 | 9577d199 | Kostas Papadimitriou | authpolicy-set Assign an existing authentication provider policy profile to a user or group |
1062 | 9577d199 | Kostas Papadimitriou | authpolicy-show Show authentication provider profile details |
1063 | 9577d199 | Kostas Papadimitriou | group-add Create a group with the given name |
1064 | 9577d199 | Kostas Papadimitriou | group-list List available groups |
1065 | e81ad3e5 | Constantinos Venetsanopoulos | user-list List users |
1066 | e81ad3e5 | Constantinos Venetsanopoulos | user-modify Modify user |
1067 | e81ad3e5 | Constantinos Venetsanopoulos | user-show Show user details |
1068 | e81ad3e5 | Constantinos Venetsanopoulos | ============================ =========================== |
1069 | e81ad3e5 | Constantinos Venetsanopoulos | |
1070 | e81ad3e5 | Constantinos Venetsanopoulos | Pithos snf-manage commands |
1071 | e81ad3e5 | Constantinos Venetsanopoulos | -------------------------- |
1072 | e81ad3e5 | Constantinos Venetsanopoulos | |
1073 | e81ad3e5 | Constantinos Venetsanopoulos | ============================ =========================== |
1074 | e81ad3e5 | Constantinos Venetsanopoulos | Name Description |
1075 | e81ad3e5 | Constantinos Venetsanopoulos | ============================ =========================== |
1076 | 3506f6a0 | Sofia Papagiannaki | reconcile-commissions-pithos Display unresolved commissions and trigger their recovery |
1077 | 3506f6a0 | Sofia Papagiannaki | resource-export-pithos Export pithos resources in json format |
1078 | 3506f6a0 | Sofia Papagiannaki | reconcile-resources-pithos Detect unsynchronized usage between Astakos and Pithos DB resources and synchronize them if specified so. |
1079 | e81ad3e5 | Constantinos Venetsanopoulos | ============================ =========================== |
1080 | e81ad3e5 | Constantinos Venetsanopoulos | |
1081 | e81ad3e5 | Constantinos Venetsanopoulos | Cyclades snf-manage commands |
1082 | e81ad3e5 | Constantinos Venetsanopoulos | ---------------------------- |
1083 | e81ad3e5 | Constantinos Venetsanopoulos | |
1084 | bf57a264 | Christos Stavrakakis | ============================== =========================== |
1085 | bf57a264 | Christos Stavrakakis | Name Description |
1086 | bf57a264 | Christos Stavrakakis | ============================== =========================== |
1087 | bf57a264 | Christos Stavrakakis | backend-add Add a new Ganeti backend |
1088 | bf57a264 | Christos Stavrakakis | backend-list List backends |
1089 | bf57a264 | Christos Stavrakakis | backend-modify Modify a backend |
1090 | bf57a264 | Christos Stavrakakis | backend-update-status Update backend statistics for instance allocation |
1091 | bf57a264 | Christos Stavrakakis | backend-remove Remove a Ganeti backend |
1092 | bf57a264 | Christos Stavrakakis | server-create Create a new server |
1093 | bf57a264 | Christos Stavrakakis | server-show Show server details |
1094 | bf57a264 | Christos Stavrakakis | server-list List servers |
1095 | bf57a264 | Christos Stavrakakis | server-modify Modify a server |
1096 | bf57a264 | Christos Stavrakakis | server-import Import an existing Ganeti VM into synnefo |
1097 | bf57a264 | Christos Stavrakakis | server-inspect Inspect a server in DB and Ganeti |
1098 | bf57a264 | Christos Stavrakakis | network-create Create a new network |
1099 | bf57a264 | Christos Stavrakakis | network-list List networks |
1100 | bf57a264 | Christos Stavrakakis | network-modify Modify a network |
1101 | bf57a264 | Christos Stavrakakis | network-inspect Inspect network state in DB and Ganeti |
1102 | bf57a264 | Christos Stavrakakis | network-remove Delete a network |
1103 | bf57a264 | Christos Stavrakakis | flavor-create Create a new flavor |
1104 | bf57a264 | Christos Stavrakakis | flavor-list List flavors |
1105 | bf57a264 | Christos Stavrakakis | flavor-modify Modify a flavor |
1106 | bf57a264 | Christos Stavrakakis | image-list List images |
1107 | bf57a264 | Christos Stavrakakis | image-show Show image details |
1108 | bf57a264 | Christos Stavrakakis | pool-create Create a bridge or mac-prefix pool |
1109 | bf57a264 | Christos Stavrakakis | pool-show Show pool details |
1110 | bf57a264 | Christos Stavrakakis | pool-list List pools |
1111 | bf57a264 | Christos Stavrakakis | pool-modify Modify a pool |
1112 | bf57a264 | Christos Stavrakakis | pool-remove Delete a pool |
1113 | bf57a264 | Christos Stavrakakis | queue-inspect Inspect the messages of a RabbitMQ queue |
1114 | bf57a264 | Christos Stavrakakis | queue-retry Resend messages from Dead Letter queues to original exchanges |
1115 | bf57a264 | Christos Stavrakakis | resource-export-cyclades Export Cyclades resources in JSON format. |
1116 | bf57a264 | Christos Stavrakakis | service-export-cyclades Export Cyclades services in JSON format. |
1117 | bf57a264 | Christos Stavrakakis | reconcile-servers Reconcile servers of Synnefo DB with state of Ganeti backend |
1118 | bf57a264 | Christos Stavrakakis | reconcile-networks Reconcile networks of Synnefo DB with state of Ganeti backend |
1119 | bf57a264 | Christos Stavrakakis | reconcile-pools Check consistency of pool resources |
1120 | bf57a264 | Christos Stavrakakis | reconcile-commissions-cyclades Detect and resolve pending commissions to Quotaholder |
1121 | bf57a264 | Christos Stavrakakis | reconcile-resources-cyclades Reconcile resource usage of Astakos with Cyclades DB. |
1122 | bf57a264 | Christos Stavrakakis | ============================== =========================== |
1123 | e81ad3e5 | Constantinos Venetsanopoulos | |
1124 | 11716a83 | Giorgos Korfiatis | Astakos helper scripts |
1125 | 11716a83 | Giorgos Korfiatis | ====================== |
1126 | 11716a83 | Giorgos Korfiatis | |
1127 | 11716a83 | Giorgos Korfiatis | Astakos includes two scripts to facilitate the installation procedure. |
1128 | 11716a83 | Giorgos Korfiatis | Running: |
1129 | 11716a83 | Giorgos Korfiatis | |
1130 | 11716a83 | Giorgos Korfiatis | .. code-block:: console |
1131 | 11716a83 | Giorgos Korfiatis | |
1132 | b4109758 | Giorgos Korfiatis | snf-component-register [<component_name>] |
1133 | 11716a83 | Giorgos Korfiatis | |
1134 | 11716a83 | Giorgos Korfiatis | automates the registration of the standard Synnefo components (astakos, |
1135 | 11716a83 | Giorgos Korfiatis | cyclades, and pithos) in astakos database. It internally uses the script: |
1136 | 11716a83 | Giorgos Korfiatis | |
1137 | 11716a83 | Giorgos Korfiatis | .. code-block:: console |
1138 | 11716a83 | Giorgos Korfiatis | |
1139 | 11716a83 | Giorgos Korfiatis | snf-service-export <component_name> <base_url> |
1140 | 11716a83 | Giorgos Korfiatis | |
1141 | 11716a83 | Giorgos Korfiatis | which simulates the export of service and resource definitions of the |
1142 | 11716a83 | Giorgos Korfiatis | standard Synnefo components. |
1143 | 11716a83 | Giorgos Korfiatis | |
1144 | 3506f6a0 | Sofia Papagiannaki | Pithos managing accounts |
1145 | 3506f6a0 | Sofia Papagiannaki | ======================== |
1146 | 3506f6a0 | Sofia Papagiannaki | |
1147 | 3506f6a0 | Sofia Papagiannaki | Pithos provides a utility tool for managing accounts. |
1148 | 3506f6a0 | Sofia Papagiannaki | To run you just type: |
1149 | 3506f6a0 | Sofia Papagiannaki | |
1150 | 3506f6a0 | Sofia Papagiannaki | .. code-block:: console |
1151 | 3506f6a0 | Sofia Papagiannaki | |
1152 | 3506f6a0 | Sofia Papagiannaki | # pithos-manage-accounts <command> [arguments] |
1153 | 3506f6a0 | Sofia Papagiannaki | |
1154 | 3506f6a0 | Sofia Papagiannaki | This is the list of the available commands: |
1155 | 3506f6a0 | Sofia Papagiannaki | |
1156 | 3506f6a0 | Sofia Papagiannaki | ============================ =========================== |
1157 | 3506f6a0 | Sofia Papagiannaki | Name Description |
1158 | 3506f6a0 | Sofia Papagiannaki | ============================ =========================== |
1159 | 3506f6a0 | Sofia Papagiannaki | delete Remove an account from the Pithos DB |
1160 | 3506f6a0 | Sofia Papagiannaki | export-quota Export account quota in a file |
1161 | 3506f6a0 | Sofia Papagiannaki | list List existing/dublicate accounts |
1162 | 3506f6a0 | Sofia Papagiannaki | merge Move an account contents in another account |
1163 | 3506f6a0 | Sofia Papagiannaki | set-container-quota Set container quota for all or a specific account |
1164 | 3506f6a0 | Sofia Papagiannaki | ============================ =========================== |
1165 | 3506f6a0 | Sofia Papagiannaki | |
1166 | e81ad3e5 | Constantinos Venetsanopoulos | |
1167 | bc055d09 | Constantinos Venetsanopoulos | The "kamaki" API client |
1168 | eab73849 | Constantinos Venetsanopoulos | ======================= |
1169 | bc055d09 | Constantinos Venetsanopoulos | |
1170 | bc055d09 | Constantinos Venetsanopoulos | To upload, register or modify an image you will need the **kamaki** tool. |
1171 | bc055d09 | Constantinos Venetsanopoulos | Before proceeding make sure that it is configured properly. Verify that |
1172 | dd04efac | Stavros Sachtouris | *image.url*, *file.url*, *user.url* and *token* are set as needed: |
1173 | bc055d09 | Constantinos Venetsanopoulos | |
1174 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
1175 | bc055d09 | Constantinos Venetsanopoulos | |
1176 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki config list |
1177 | bc055d09 | Constantinos Venetsanopoulos | |
1178 | 73ebcd68 | Constantinos Venetsanopoulos | To change a setting use ``kamaki config set``: |
1179 | bc055d09 | Constantinos Venetsanopoulos | |
1180 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
1181 | bc055d09 | Constantinos Venetsanopoulos | |
1182 | 73ebcd68 | Constantinos Venetsanopoulos | $ kamaki config set image.url https://cyclades.example.com/image |
1183 | dd04efac | Stavros Sachtouris | $ kamaki config set file.url https://pithos.example.com/v1 |
1184 | dd04efac | Stavros Sachtouris | $ kamaki config set user.url https://accounts.example.com |
1185 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki config set token ... |
1186 | bc055d09 | Constantinos Venetsanopoulos | |
1187 | c0d88cf3 | Constantinos Venetsanopoulos | To test that everything works, try authenticating the current account with |
1188 | c0d88cf3 | Constantinos Venetsanopoulos | kamaki: |
1189 | dd04efac | Stavros Sachtouris | |
1190 | dd04efac | Stavros Sachtouris | .. code-block:: console |
1191 | dd04efac | Stavros Sachtouris | |
1192 | dd04efac | Stavros Sachtouris | $ kamaki user authenticate |
1193 | dd04efac | Stavros Sachtouris | |
1194 | c0d88cf3 | Constantinos Venetsanopoulos | This will output user information. |
1195 | dd04efac | Stavros Sachtouris | |
1196 | bc055d09 | Constantinos Venetsanopoulos | Upload Image |
1197 | bc055d09 | Constantinos Venetsanopoulos | ------------ |
1198 | bc055d09 | Constantinos Venetsanopoulos | |
1199 | c0d88cf3 | Constantinos Venetsanopoulos | By convention, images are stored in a container called ``images``. Check if the |
1200 | c0d88cf3 | Constantinos Venetsanopoulos | container exists, by listing all containers in your account: |
1201 | bc055d09 | Constantinos Venetsanopoulos | |
1202 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
1203 | bc055d09 | Constantinos Venetsanopoulos | |
1204 | dd04efac | Stavros Sachtouris | $ kamaki file list |
1205 | bc055d09 | Constantinos Venetsanopoulos | |
1206 | dd04efac | Stavros Sachtouris | If the container ``images`` does not exist, create it: |
1207 | bc055d09 | Constantinos Venetsanopoulos | |
1208 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
1209 | bc055d09 | Constantinos Venetsanopoulos | |
1210 | dd04efac | Stavros Sachtouris | $ kamaki file create images |
1211 | bc055d09 | Constantinos Venetsanopoulos | |
1212 | c0d88cf3 | Constantinos Venetsanopoulos | You are now ready to upload an image to container ``images``. You can upload it |
1213 | e5d8df8c | Constantinos Venetsanopoulos | with a Pithos client, or use kamaki directly: |
1214 | bc055d09 | Constantinos Venetsanopoulos | |
1215 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
1216 | bc055d09 | Constantinos Venetsanopoulos | |
1217 | dd04efac | Stavros Sachtouris | $ kamaki file upload ubuntu.iso images |
1218 | bc055d09 | Constantinos Venetsanopoulos | |
1219 | e5d8df8c | Constantinos Venetsanopoulos | You can use any Pithos client to verify that the image was uploaded correctly, |
1220 | c0d88cf3 | Constantinos Venetsanopoulos | or you can list the contents of the container with kamaki: |
1221 | bc055d09 | Constantinos Venetsanopoulos | |
1222 | dd04efac | Stavros Sachtouris | .. code-block:: console |
1223 | dd04efac | Stavros Sachtouris | |
1224 | dd04efac | Stavros Sachtouris | $ kamaki file list images |
1225 | bc055d09 | Constantinos Venetsanopoulos | |
1226 | bc055d09 | Constantinos Venetsanopoulos | The full Pithos URL for the previous example will be |
1227 | c0d88cf3 | Constantinos Venetsanopoulos | ``pithos://u53r-un1qu3-1d/images/ubuntu.iso`` where ``u53r-un1qu3-1d`` is the |
1228 | c0d88cf3 | Constantinos Venetsanopoulos | unique user id (uuid). |
1229 | bc055d09 | Constantinos Venetsanopoulos | |
1230 | bc055d09 | Constantinos Venetsanopoulos | Register Image |
1231 | bc055d09 | Constantinos Venetsanopoulos | -------------- |
1232 | bc055d09 | Constantinos Venetsanopoulos | |
1233 | e5d8df8c | Constantinos Venetsanopoulos | To register an image you will need to use the full Pithos URL. To register as |
1234 | bc055d09 | Constantinos Venetsanopoulos | a public image the one from the previous example use: |
1235 | bc055d09 | Constantinos Venetsanopoulos | |
1236 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
1237 | bc055d09 | Constantinos Venetsanopoulos | |
1238 | dd04efac | Stavros Sachtouris | $ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso --public |
1239 | bc055d09 | Constantinos Venetsanopoulos | |
1240 | bc055d09 | Constantinos Venetsanopoulos | The ``--public`` flag is important, if missing the registered image will not |
1241 | dd04efac | Stavros Sachtouris | be listed by ``kamaki image list``. |
1242 | bc055d09 | Constantinos Venetsanopoulos | |
1243 | dd04efac | Stavros Sachtouris | Use ``kamaki image register`` with no arguments to see a list of available |
1244 | bc055d09 | Constantinos Venetsanopoulos | options. A more complete example would be the following: |
1245 | bc055d09 | Constantinos Venetsanopoulos | |
1246 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
1247 | bc055d09 | Constantinos Venetsanopoulos | |
1248 | dd04efac | Stavros Sachtouris | $ kamaki image register Ubuntu pithos://u53r-un1qu3-1d/images/ubuntu.iso \ |
1249 | bc055d09 | Constantinos Venetsanopoulos | --public --disk-format diskdump --property kernel=3.1.2 |
1250 | bc055d09 | Constantinos Venetsanopoulos | |
1251 | bc055d09 | Constantinos Venetsanopoulos | To verify that the image was registered successfully use: |
1252 | bc055d09 | Constantinos Venetsanopoulos | |
1253 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
1254 | bc055d09 | Constantinos Venetsanopoulos | |
1255 | dd04efac | Stavros Sachtouris | $ kamaki image list --name-like=ubuntu |
1256 | bc055d09 | Constantinos Venetsanopoulos | |
1257 | eab73849 | Constantinos Venetsanopoulos | |
1258 | eab73849 | Constantinos Venetsanopoulos | Miscellaneous |
1259 | eab73849 | Constantinos Venetsanopoulos | ============= |
1260 | eab73849 | Constantinos Venetsanopoulos | |
1261 | 08ad5b01 | Constantinos Venetsanopoulos | .. _branding: |
1262 | 9c2edbf1 | Olga Brani | |
1263 | 42e767d4 | Olga Brani | Branding |
1264 | 08ad5b01 | Constantinos Venetsanopoulos | -------- |
1265 | 9c2edbf1 | Olga Brani | |
1266 | 08ad5b01 | Constantinos Venetsanopoulos | Since Synnefo v0.14, you are able to adapt the Astakos, Pithos and Cyclades Web |
1267 | 08ad5b01 | Constantinos Venetsanopoulos | UI to your company’s visual identity. This is possible using the snf-branding |
1268 | 08ad5b01 | Constantinos Venetsanopoulos | component, which is automatically installed on the nodes running the API |
1269 | 08ad5b01 | Constantinos Venetsanopoulos | servers for Astakos, Pithos and Cyclades. |
1270 | 9c2edbf1 | Olga Brani | |
1271 | 9c2edbf1 | Olga Brani | Configuration |
1272 | 9c2edbf1 | Olga Brani | ~~~~~~~~~~~~~ |
1273 | 9c2edbf1 | Olga Brani | |
1274 | 77186edf | Kostas Papadimitriou | This can be done by modifing the settings provided by the snf-branding component |
1275 | 77186edf | Kostas Papadimitriou | to match your service identity. The settings for the snf-branding application |
1276 | 77186edf | Kostas Papadimitriou | can be found inside the configuration file ``/etc/synnefo/15-snf-branding.conf`` |
1277 | 77186edf | Kostas Papadimitriou | on the nodes that have Astakos, Pithos and Cyclades installed. |
1278 | 9c2edbf1 | Olga Brani | |
1279 | 08ad5b01 | Constantinos Venetsanopoulos | By default, the global service name is "Synnefo" and the company name is |
1280 | 08ad5b01 | Constantinos Venetsanopoulos | "GRNET". These names and their respective logos and URLs are used throughout |
1281 | 08ad5b01 | Constantinos Venetsanopoulos | the Astakos, Pithos and Cyclades UI. |
1282 | 9c2edbf1 | Olga Brani | |
1283 | 08ad5b01 | Constantinos Venetsanopoulos | **Names and URLs:** |
1284 | 9c2edbf1 | Olga Brani | |
1285 | 08ad5b01 | Constantinos Venetsanopoulos | The first group of branding customization refers to the service's and company's |
1286 | 08ad5b01 | Constantinos Venetsanopoulos | information. |
1287 | 9c2edbf1 | Olga Brani | |
1288 | 08ad5b01 | Constantinos Venetsanopoulos | You can overwrite the company and the service name and URL respectively by |
1289 | 08ad5b01 | Constantinos Venetsanopoulos | uncommenting and setting the following: |
1290 | 9c2edbf1 | Olga Brani | |
1291 | 77186edf | Kostas Papadimitriou | .. code-block:: python |
1292 | 77186edf | Kostas Papadimitriou | |
1293 | 77186edf | Kostas Papadimitriou | # setting used in Astakos Dashboard/Projects pages |
1294 | 77186edf | Kostas Papadimitriou | BRANDING_SERVICE_NAME = 'My cloud' |
1295 | 77186edf | Kostas Papadimitriou | BRANDING_SERVICE_URL = 'http://www.mycloud.synnefo.org/' |
1296 | 9c2edbf1 | Olga Brani | |
1297 | 77186edf | Kostas Papadimitriou | # settings used in Astakos, Pithos, Cyclades footer only if |
1298 | 77186edf | Kostas Papadimitriou | # BRANDING_SHOW_COPYRIGHT is set to True |
1299 | 77186edf | Kostas Papadimitriou | BRANDING_SHOW_COPYRIGHT = True |
1300 | 77186edf | Kostas Papadimitriou | BRANDING_COMPANY_NAME = 'Company LTD' |
1301 | 77186edf | Kostas Papadimitriou | BRANDING_COMPANY_URL = 'https://www.company-ltd.synnefo.org/' |
1302 | 9c2edbf1 | Olga Brani | |
1303 | 9c2edbf1 | Olga Brani | |
1304 | 9c2edbf1 | Olga Brani | **Copyright options:** |
1305 | 9c2edbf1 | Olga Brani | |
1306 | 08ad5b01 | Constantinos Venetsanopoulos | By default, no Copyright message is shown in the UI footer. If you want to make |
1307 | 08ad5b01 | Constantinos Venetsanopoulos | it visible in the footer of Astakos, Pithos and Cyclades UI, you can uncomment |
1308 | 77186edf | Kostas Papadimitriou | and set to ``True`` the ``BRANDING_SHOW_COPYRIGHT`` setting: |
1309 | 9c2edbf1 | Olga Brani | |
1310 | 77186edf | Kostas Papadimitriou | .. code-block:: python |
1311 | 9c2edbf1 | Olga Brani | |
1312 | 9c2edbf1 | Olga Brani | #BRANDING_SHOW_COPYRIGHT = False |
1313 | 9c2edbf1 | Olga Brani | |
1314 | 77186edf | Kostas Papadimitriou | Copyright message defaults to 'Copyright (c) 2011-<current_year> |
1315 | 08ad5b01 | Constantinos Venetsanopoulos | <BRANDING_COMPANY_NAME>.' but you can overwrite it to a completely custom one by |
1316 | 08ad5b01 | Constantinos Venetsanopoulos | setting the following option: |
1317 | 9c2edbf1 | Olga Brani | |
1318 | 77186edf | Kostas Papadimitriou | .. code-block:: python |
1319 | 77186edf | Kostas Papadimitriou | |
1320 | 77186edf | Kostas Papadimitriou | BRANDING_COPYRIGHT_MESSAGE = 'Copyright (c) 2011-2013 GRNET' |
1321 | 9c2edbf1 | Olga Brani | |
1322 | 9c2edbf1 | Olga Brani | |
1323 | 9c2edbf1 | Olga Brani | **Images:** |
1324 | 9c2edbf1 | Olga Brani | |
1325 | 08ad5b01 | Constantinos Venetsanopoulos | The Astakos, Pithos and Cyclades Web UI has some logos and images. |
1326 | 08ad5b01 | Constantinos Venetsanopoulos | |
1327 | 9c2edbf1 | Olga Brani | The branding-related images are presented in the following table: |
1328 | 9c2edbf1 | Olga Brani | |
1329 | 9c2edbf1 | Olga Brani | =============== ============================ ========= |
1330 | 9c2edbf1 | Olga Brani | Image Name/extension convention Usage |
1331 | 9c2edbf1 | Olga Brani | =============== ============================ ========= |
1332 | 9c2edbf1 | Olga Brani | Favicon favicon.ico Favicon for all services |
1333 | 9c2edbf1 | Olga Brani | Dashboard logo dashboard_logo.png Visible in all Astakos UI pages |
1334 | 9c2edbf1 | Olga Brani | Compute logo compute_logo.png Visible in all Cyclades UI pages |
1335 | 08ad5b01 | Constantinos Venetsanopoulos | Console logo console_logo.png Visible in the Cyclades Console Window |
1336 | 08ad5b01 | Constantinos Venetsanopoulos | Storage logo storage_logo.png Visible in all Pithos UI pages |
1337 | 9c2edbf1 | Olga Brani | =============== ============================ ========= |
1338 | 9c2edbf1 | Olga Brani | |
1339 | 08ad5b01 | Constantinos Venetsanopoulos | There are two methods available for replacing all, or individual, |
1340 | 263cb456 | Olga Brani | branding-related images: |
1341 | 9c2edbf1 | Olga Brani | |
1342 | 77186edf | Kostas Papadimitriou | 1. Create a new directory inside ``/usr/share/synnefo/static/`` (e.g. |
1343 | 77186edf | Kostas Papadimitriou | ``mybranding``) and place there some or all of your images. |
1344 | 9c2edbf1 | Olga Brani | |
1345 | 08ad5b01 | Constantinos Venetsanopoulos | If you want to replace all of your images, keep the name/extension |
1346 | 77186edf | Kostas Papadimitriou | conventions as indicated in the above table and change the |
1347 | 77186edf | Kostas Papadimitriou | ``BRANDING_IMAGE_MEDIA_URL`` setting accordingly: |
1348 | 9c2edbf1 | Olga Brani | |
1349 | 77186edf | Kostas Papadimitriou | .. code-block:: python |
1350 | 77186edf | Kostas Papadimitriou | |
1351 | 77186edf | Kostas Papadimitriou | # using relative path |
1352 | 77186edf | Kostas Papadimitriou | BRANDING_IMAGE_MEDIA_URL= '/static/mybranding/images/' |
1353 | 9c2edbf1 | Olga Brani | |
1354 | 77186edf | Kostas Papadimitriou | # or if you already host them in a separate domain (e.g. cdn) |
1355 | 77186edf | Kostas Papadimitriou | BRANDING_IMAGE_MEDIA_URL= 'https://cdn.synnefo.org/branding/images/' |
1356 | 9c2edbf1 | Olga Brani | |
1357 | 9c2edbf1 | Olga Brani | |
1358 | 08ad5b01 | Constantinos Venetsanopoulos | If you wish to replace individual images, **do not uncomment** |
1359 | 08ad5b01 | Constantinos Venetsanopoulos | ``BRANDING_IMAGE_MEDIA_URL``, but instead provide a relative path, pointing to |
1360 | 08ad5b01 | Constantinos Venetsanopoulos | the file inside your directory for each ``BRANDING_<image>_URL`` that you wish |
1361 | 08ad5b01 | Constantinos Venetsanopoulos | to replace. |
1362 | 9c2edbf1 | Olga Brani | |
1363 | 263cb456 | Olga Brani | 2. Upload some or all of your images to a server and replace each |
1364 | 08ad5b01 | Constantinos Venetsanopoulos | ``BRANDING_<image>_URL`` with the absolute url of the image (i.e. |
1365 | 77186edf | Kostas Papadimitriou | ``BRANDING_DASHBOARD_URL = 'https://www.synnefo.com/images/my_dashboard.jpg'``). |
1366 | 9c2edbf1 | Olga Brani | |
1367 | 77186edf | Kostas Papadimitriou | Note that the alternative text for each image tag inside html documents is |
1368 | 08ad5b01 | Constantinos Venetsanopoulos | alt=“BRANDING_SERVICE_NAME {Dashboard, Compute. Console, Storage}” respectively. |
1369 | 9c2edbf1 | Olga Brani | |
1370 | 42e767d4 | Olga Brani | .. note:: Retina optimized images: |
1371 | 9c2edbf1 | Olga Brani | |
1372 | 08ad5b01 | Constantinos Venetsanopoulos | Synnefo UI is optimized for Retina displays. As far as images are concerned, |
1373 | 08ad5b01 | Constantinos Venetsanopoulos | `retina.js <http://retinajs.com/>`_ is used. |
1374 | 9c2edbf1 | Olga Brani | |
1375 | 77186edf | Kostas Papadimitriou | Retina.js checks each image on a page to see if there is a high-resolution |
1376 | 08ad5b01 | Constantinos Venetsanopoulos | version of that image on your server. If a high-resolution variant exists, |
1377 | 08ad5b01 | Constantinos Venetsanopoulos | the script will swap in that image in-place. |
1378 | 9c2edbf1 | Olga Brani | |
1379 | 08ad5b01 | Constantinos Venetsanopoulos | The script assumes you use `Apple's prescribed high-resolution modifier (@2x) |
1380 | 08ad5b01 | Constantinos Venetsanopoulos | <http://developer.apple.com/library/ios/#documentation/2DDrawing/Conceptual/ |
1381 | 08ad5b01 | Constantinos Venetsanopoulos | DrawingPrintingiOS/SupportingHiResScreensInViews/SupportingHiResScreensInViews |
1382 | 08ad5b01 | Constantinos Venetsanopoulos | .html#//apple_ref/doc/uid/TP40010156-CH15-SW1>`_ to denote high-resolution |
1383 | 08ad5b01 | Constantinos Venetsanopoulos | image variants on your server. |
1384 | 9c2edbf1 | Olga Brani | |
1385 | 08ad5b01 | Constantinos Venetsanopoulos | For each of the images that you wish the script to replace, you must have a |
1386 | 08ad5b01 | Constantinos Venetsanopoulos | high-resolution variant in the same folder named correctly and it will be |
1387 | 08ad5b01 | Constantinos Venetsanopoulos | detected automatically. For example if your image is in <my_directory> and is |
1388 | 08ad5b01 | Constantinos Venetsanopoulos | named "my_image.jpg" the script will look in the same directory for an image |
1389 | 08ad5b01 | Constantinos Venetsanopoulos | named "my_image@2x.jpg". |
1390 | 9c2edbf1 | Olga Brani | |
1391 | 08ad5b01 | Constantinos Venetsanopoulos | In case that you don’t want to use a high-resolution image, the |
1392 | 08ad5b01 | Constantinos Venetsanopoulos | normal-resolution image will be visible. |
1393 | 9c2edbf1 | Olga Brani | |
1394 | 08ad5b01 | Constantinos Venetsanopoulos | More branding |
1395 | 08ad5b01 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~ |
1396 | 9c2edbf1 | Olga Brani | |
1397 | 08ad5b01 | Constantinos Venetsanopoulos | Although, it is not 100% branding-related, further verbal customization is |
1398 | 08ad5b01 | Constantinos Venetsanopoulos | feasible. |
1399 | 9c2edbf1 | Olga Brani | |
1400 | 9c2edbf1 | Olga Brani | **EMAILS** |
1401 | 9c2edbf1 | Olga Brani | |
1402 | 77186edf | Kostas Papadimitriou | The output of all email `*`.txt files will be already customized to contain your |
1403 | 77186edf | Kostas Papadimitriou | company and service names but you can further alter their content if you feel it |
1404 | 77186edf | Kostas Papadimitriou | best fits your needs as simple as creasynnefo template. |
1405 | 017b7d20 | Olga Brani | |
1406 | 77186edf | Kostas Papadimitriou | In order to overwrite one or more email-templates you need to place your |
1407 | 77186edf | Kostas Papadimitriou | modified <email-file>.txt files respecting the following structure: |
1408 | 017b7d20 | Olga Brani | |
1409 | 77186edf | Kostas Papadimitriou | **/etc/synnefo/templates/** |
1410 | 77186edf | Kostas Papadimitriou | **im/** |
1411 | 017b7d20 | Olga Brani | | activation_email.txt |
1412 | 017b7d20 | Olga Brani | | email.txt |
1413 | 017b7d20 | Olga Brani | | invitation.txt |
1414 | 017b7d20 | Olga Brani | | switch_accounts_email.txt |
1415 | 017b7d20 | Olga Brani | | welcome_email.txt |
1416 | 77186edf | Kostas Papadimitriou | **projects/** |
1417 | 017b7d20 | Olga Brani | | project_approval_notification.txt |
1418 | 017b7d20 | Olga Brani | | project_denial_notification.txt |
1419 | 017b7d20 | Olga Brani | | project_membership_change_notification.txt |
1420 | 017b7d20 | Olga Brani | | project_membership_enroll_notification.txt |
1421 | 017b7d20 | Olga Brani | | project_membership_leave_request_notification.txt |
1422 | 017b7d20 | Olga Brani | | project_membership_request_notification.txt |
1423 | 017b7d20 | Olga Brani | | project_suspension_notification.txt |
1424 | 017b7d20 | Olga Brani | | project_termination_notification.txt |
1425 | 77186edf | Kostas Papadimitriou | **registration/** |
1426 | 017b7d20 | Olga Brani | | email_change_email.txt |
1427 | 017b7d20 | Olga Brani | | password_email.txt |
1428 | 017b7d20 | Olga Brani | |
1429 | 017b7d20 | Olga Brani | Feel free to omit any of the above files you do not wish to overwrite. |
1430 | 017b7d20 | Olga Brani | |
1431 | 9c2edbf1 | Olga Brani | Below is a list of all emails sent by Synnefo to users along with a short |
1432 | 9c2edbf1 | Olga Brani | description and a link to their content: |
1433 | 9c2edbf1 | Olga Brani | |
1434 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/email.txt`` |
1435 | 08ad5b01 | Constantinos Venetsanopoulos | Base email template. Contains a contact email and a “thank you” message. |
1436 | 08ad5b01 | Constantinos Venetsanopoulos | (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/email.txt>`_) |
1437 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/activation_email.txt`` Email sent to |
1438 | 08ad5b01 | Constantinos Venetsanopoulos | user that prompts him/her to click on a link provided to activate the account. |
1439 | 08ad5b01 | Constantinos Venetsanopoulos | Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/activation_email.txt>`_) |
1440 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/invitation.txt`` Email sent to an |
1441 | 08ad5b01 | Constantinos Venetsanopoulos | invited user. He/she has to click on a link provided to activate the account. |
1442 | 08ad5b01 | Constantinos Venetsanopoulos | Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/invitation.txt>`_) |
1443 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt`` Email |
1444 | 08ad5b01 | Constantinos Venetsanopoulos | sent to user upon his/her request to associate this email address with a |
1445 | 08ad5b01 | Constantinos Venetsanopoulos | shibboleth account. He/she has to click on a link provided to activate the |
1446 | 08ad5b01 | Constantinos Venetsanopoulos | association. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/switch_accounts_email.txt>`_) |
1447 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/welcome_email.txt`` Email sent to |
1448 | 08ad5b01 | Constantinos Venetsanopoulos | inform the user that his/ her account has been activated. Extends “email.txt” |
1449 | 08ad5b01 | Constantinos Venetsanopoulos | (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/welcome_email.txt>`_) |
1450 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/registration/email_change_email.txt`` |
1451 | 08ad5b01 | Constantinos Venetsanopoulos | Email sent to user when he/she has requested new email address assignment. The |
1452 | 08ad5b01 | Constantinos Venetsanopoulos | user has to click on a link provided to validate this action. Extends |
1453 | 08ad5b01 | Constantinos Venetsanopoulos | “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/email_change_email.txt>`_) |
1454 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/registration/password_email.txt`` Email |
1455 | 08ad5b01 | Constantinos Venetsanopoulos | sent for resetting password purpose. The user has to click on a link provided |
1456 | 08ad5b01 | Constantinos Venetsanopoulos | to validate this action. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/registration/password_email.txt>`_) |
1457 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt`` |
1458 | 08ad5b01 | Constantinos Venetsanopoulos | Informs the project owner that his/her project has been approved. Extends |
1459 | 08ad5b01 | Constantinos Venetsanopoulos | “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_approval_notification.txt>`_) |
1460 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt`` |
1461 | 08ad5b01 | Constantinos Venetsanopoulos | Informs the project owner that his/her project application has been denied |
1462 | 08ad5b01 | Constantinos Venetsanopoulos | explaining the reasons. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_denial_notification.txt>`_) |
1463 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt`` |
1464 | 08ad5b01 | Constantinos Venetsanopoulos | An email is sent to a user containing information about his project membership |
1465 | 08ad5b01 | Constantinos Venetsanopoulos | (whether he has been accepted, rejected or removed). Extends “email.txt” (`Link |
1466 | 08ad5b01 | Constantinos Venetsanopoulos | <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_change_notification.txt>`_) |
1467 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt`` |
1468 | 08ad5b01 | Constantinos Venetsanopoulos | Informs a user that he/she has been enrolled to a project. Extends |
1469 | 08ad5b01 | Constantinos Venetsanopoulos | “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_enroll_notification.txt>`_) |
1470 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt`` |
1471 | 08ad5b01 | Constantinos Venetsanopoulos | An email is sent to the project owner to make him aware of a user having |
1472 | 08ad5b01 | Constantinos Venetsanopoulos | requested to leave his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_leave_request_notification.txt>`_) |
1473 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt`` |
1474 | 08ad5b01 | Constantinos Venetsanopoulos | An email is sent to the project owner to make him/her aware of a user having |
1475 | 08ad5b01 | Constantinos Venetsanopoulos | requested to join his project. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_membership_request_notification.txt>`_) |
1476 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt`` |
1477 | 08ad5b01 | Constantinos Venetsanopoulos | An email is sent to the project owner to make him/her aware of his/her project |
1478 | 08ad5b01 | Constantinos Venetsanopoulos | having been suspended. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_suspension_notification.txt>`_) |
1479 | 08ad5b01 | Constantinos Venetsanopoulos | * ``snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt`` |
1480 | 08ad5b01 | Constantinos Venetsanopoulos | An email is sent to the project owner to make him/her aware of his/her project |
1481 | 08ad5b01 | Constantinos Venetsanopoulos | having been terminated. Extends “email.txt” (`Link <https://code.grnet.gr/projects/synnefo/repository/revisions/master/changes/snf-astakos-app/astakos/im/templates/im/projects/project_termination_notification.txt>`_) |
1482 | 9c2edbf1 | Olga Brani | |
1483 | 9c2edbf1 | Olga Brani | .. warning:: Django templates language: |
1484 | 9c2edbf1 | Olga Brani | |
1485 | 9c2edbf1 | Olga Brani | If you choose to overwrite these email templates, be mindful of the necessary |
1486 | 263cb456 | Olga Brani | information contained in django template variables that must not be omitted, |
1487 | 263cb456 | Olga Brani | such as the activation link for activating one’s account and many more. |
1488 | 263cb456 | Olga Brani | These variables are contained into {{}} inside the templates. |
1489 | 9c2edbf1 | Olga Brani | |
1490 | 9c2edbf1 | Olga Brani | |
1491 | 9b659162 | Christos Stavrakakis | .. RabbitMQ |
1492 | a1c707c9 | Constantinos Venetsanopoulos | |
1493 | 9b659162 | Christos Stavrakakis | RabbitMQ Broker |
1494 | 9b659162 | Christos Stavrakakis | --------------- |
1495 | 9b659162 | Christos Stavrakakis | |
1496 | 9b659162 | Christos Stavrakakis | Queue nodes run the RabbitMQ sofware, which provides AMQP functionality. To |
1497 | 9b659162 | Christos Stavrakakis | guarantee high-availability, more than one Queue nodes should be deployed, each |
1498 | 9b659162 | Christos Stavrakakis | of them belonging to the same `RabbitMQ cluster |
1499 | a1c707c9 | Constantinos Venetsanopoulos | <http://www.rabbitmq.com/clustering.html>`_. Synnefo uses the RabbitMQ |
1500 | a1c707c9 | Constantinos Venetsanopoulos | active/active `High Available Queues <http://www.rabbitmq.com/ha.html>`_ which |
1501 | a1c707c9 | Constantinos Venetsanopoulos | are mirrored between two nodes within a RabbitMQ cluster. |
1502 | a1c707c9 | Constantinos Venetsanopoulos | |
1503 | a1c707c9 | Constantinos Venetsanopoulos | The RabbitMQ nodes that form the cluster, are declared to Synnefo through the |
1504 | a1c707c9 | Constantinos Venetsanopoulos | `AMQP_HOSTS` setting. Each time a Synnefo component needs to connect to |
1505 | a1c707c9 | Constantinos Venetsanopoulos | RabbitMQ, one of these nodes is chosen in a random way. The client that Synnefo |
1506 | a1c707c9 | Constantinos Venetsanopoulos | uses to connect to RabbitMQ, handles connection failures transparently and |
1507 | a1c707c9 | Constantinos Venetsanopoulos | tries to reconnect to a different node. As long as one of these nodes are up |
1508 | a1c707c9 | Constantinos Venetsanopoulos | and running, functionality of Synnefo should not be downgraded by the RabbitMQ |
1509 | a1c707c9 | Constantinos Venetsanopoulos | node failures. |
1510 | 9b659162 | Christos Stavrakakis | |
1511 | 9b659162 | Christos Stavrakakis | All the queues that are being used are declared as durable, meaning that |
1512 | a1c707c9 | Constantinos Venetsanopoulos | messages are persistently stored to RabbitMQ, until they get successfully |
1513 | a1c707c9 | Constantinos Venetsanopoulos | processed by a client. |
1514 | 9b659162 | Christos Stavrakakis | |
1515 | 9b659162 | Christos Stavrakakis | Currently, RabbitMQ is used by the following components: |
1516 | 9b659162 | Christos Stavrakakis | |
1517 | a1c707c9 | Constantinos Venetsanopoulos | * `snf-ganeti-eventd`, `snf-ganeti-hook` and `snf-progress-monitor`: |
1518 | 9b659162 | Christos Stavrakakis | These components send messages concerning the status and progress of |
1519 | 9b659162 | Christos Stavrakakis | jobs in the Ganeti backend. |
1520 | a1c707c9 | Constantinos Venetsanopoulos | * `snf-dispatcher`: This daemon, consumes the messages that are sent from |
1521 | a1c707c9 | Constantinos Venetsanopoulos | the above components, and updates the Cyclades DB accordingly. |
1522 | 9b659162 | Christos Stavrakakis | |
1523 | e4404297 | Christos Stavrakakis | |
1524 | 9b659162 | Christos Stavrakakis | Installation |
1525 | 301294a9 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~ |
1526 | 301294a9 | Constantinos Venetsanopoulos | |
1527 | a1c707c9 | Constantinos Venetsanopoulos | Please check the RabbitMQ documentation which covers extensively the |
1528 | a1c707c9 | Constantinos Venetsanopoulos | `installation of RabbitMQ server <http://www.rabbitmq.com/download.html>`_ and |
1529 | a1c707c9 | Constantinos Venetsanopoulos | the setup of a `RabbitMQ cluster <http://www.rabbitmq.com/clustering.html>`_. |
1530 | a1c707c9 | Constantinos Venetsanopoulos | Also, check out the `web management plugin |
1531 | a1c707c9 | Constantinos Venetsanopoulos | <http://www.rabbitmq.com/management.html>`_ that can be useful for managing and |
1532 | a1c707c9 | Constantinos Venetsanopoulos | monitoring RabbitMQ. |
1533 | 9b659162 | Christos Stavrakakis | |
1534 | 9b659162 | Christos Stavrakakis | For a basic installation of RabbitMQ on two nodes (node1 and node2) you can do |
1535 | a1c707c9 | Constantinos Venetsanopoulos | the following: |
1536 | 9b659162 | Christos Stavrakakis | |
1537 | a1c707c9 | Constantinos Venetsanopoulos | On both nodes, install rabbitmq-server and create a Synnefo user: |
1538 | 9b659162 | Christos Stavrakakis | |
1539 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
1540 | 9b659162 | Christos Stavrakakis | |
1541 | 9b659162 | Christos Stavrakakis | $ apt-get install rabbitmq-server |
1542 | 9b659162 | Christos Stavrakakis | $ rabbitmqctl add_user synnefo "example_pass" |
1543 | 9b659162 | Christos Stavrakakis | $ rabbitmqctl set_permissions synnefo ".*" ".*" ".*" |
1544 | 9b659162 | Christos Stavrakakis | |
1545 | a1c707c9 | Constantinos Venetsanopoulos | Also guarantee that both nodes share the same cookie, by running: |
1546 | 9b659162 | Christos Stavrakakis | |
1547 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
1548 | 9b659162 | Christos Stavrakakis | |
1549 | 9b659162 | Christos Stavrakakis | $ scp node1:/var/lib/rabbitmq/.erlang.cookie node2:/var/lib/rabbitmq/.erlang.cookie |
1550 | 9b659162 | Christos Stavrakakis | |
1551 | 9b659162 | Christos Stavrakakis | and restart the nodes: |
1552 | 9b659162 | Christos Stavrakakis | |
1553 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
1554 | 9b659162 | Christos Stavrakakis | |
1555 | 9b659162 | Christos Stavrakakis | $ /etc/init.d/rabbitmq-server restart |
1556 | 9b659162 | Christos Stavrakakis | |
1557 | 9b659162 | Christos Stavrakakis | |
1558 | 9b659162 | Christos Stavrakakis | To setup the RabbitMQ cluster run: |
1559 | 9b659162 | Christos Stavrakakis | |
1560 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
1561 | 9b659162 | Christos Stavrakakis | |
1562 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl stop_app |
1563 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl reset |
1564 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl cluster rabbit@node1 rabbit@node2 |
1565 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl start_app |
1566 | 9b659162 | Christos Stavrakakis | |
1567 | 9b659162 | Christos Stavrakakis | You can verify that the cluster is set up correctly by running: |
1568 | 9b659162 | Christos Stavrakakis | |
1569 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
1570 | 9b659162 | Christos Stavrakakis | |
1571 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl cluster_status |
1572 | 9b659162 | Christos Stavrakakis | |
1573 | 9b659162 | Christos Stavrakakis | |
1574 | bc055d09 | Constantinos Venetsanopoulos | Logging |
1575 | bc055d09 | Constantinos Venetsanopoulos | ------- |
1576 | bc055d09 | Constantinos Venetsanopoulos | |
1577 | bc055d09 | Constantinos Venetsanopoulos | Logging in Synnefo is using Python's logging module. The module is configured |
1578 | bc055d09 | Constantinos Venetsanopoulos | using dictionary configuration, whose format is described here: |
1579 | bc055d09 | Constantinos Venetsanopoulos | |
1580 | bc055d09 | Constantinos Venetsanopoulos | http://docs.python.org/release/2.7.1/library/logging.html#logging-config-dictschema |
1581 | bc055d09 | Constantinos Venetsanopoulos | |
1582 | bc055d09 | Constantinos Venetsanopoulos | Note that this is a feature of Python 2.7 that we have backported for use in |
1583 | bc055d09 | Constantinos Venetsanopoulos | Python 2.6. |
1584 | bc055d09 | Constantinos Venetsanopoulos | |
1585 | 4be65f1f | Christos Stavrakakis | The logging configuration dictionary is defined in |
1586 | a1c707c9 | Constantinos Venetsanopoulos | ``/etc/synnefo/10-snf-webproject-logging.conf`` |
1587 | bc055d09 | Constantinos Venetsanopoulos | |
1588 | a1c707c9 | Constantinos Venetsanopoulos | The administrator can have finer logging control by modifying the |
1589 | a1c707c9 | Constantinos Venetsanopoulos | ``LOGGING_SETUP`` dictionary, and defining subloggers with different handlers |
1590 | a1c707c9 | Constantinos Venetsanopoulos | and log levels. e.g. To enable debug messages only for the API set the level |
1591 | a1c707c9 | Constantinos Venetsanopoulos | of 'synnefo.api' to ``DEBUG`` |
1592 | 4be65f1f | Christos Stavrakakis | |
1593 | a1c707c9 | Constantinos Venetsanopoulos | By default, the Django webapp and snf-manage logs to syslog, while |
1594 | a1c707c9 | Constantinos Venetsanopoulos | `snf-dispatcher` logs to `/var/log/synnefo/dispatcher.log`. |
1595 | bc055d09 | Constantinos Venetsanopoulos | |
1596 | d189d11c | Constantinos Venetsanopoulos | |
1597 | 21d3d487 | Constantinos Venetsanopoulos | .. _scale-up: |
1598 | 21d3d487 | Constantinos Venetsanopoulos | |
1599 | bc055d09 | Constantinos Venetsanopoulos | Scaling up to multiple nodes |
1600 | bc055d09 | Constantinos Venetsanopoulos | ============================ |
1601 | bc055d09 | Constantinos Venetsanopoulos | |
1602 | 301294a9 | Constantinos Venetsanopoulos | Here we will describe how should a large scale Synnefo deployment look like. Make |
1603 | 301294a9 | Constantinos Venetsanopoulos | sure you are familiar with Synnefo and Ganeti before proceeding with this section. |
1604 | 301294a9 | Constantinos Venetsanopoulos | This means you should at least have already set up successfully a working Synnefo |
1605 | ff6ba888 | Constantinos Venetsanopoulos | deployment as described in the :ref:`Admin's Installation Guide |
1606 | 301294a9 | Constantinos Venetsanopoulos | <quick-install-admin-guide>` and also read the Administrator's Guide until this |
1607 | 301294a9 | Constantinos Venetsanopoulos | section. |
1608 | 2f6143c9 | Constantinos Venetsanopoulos | |
1609 | 301294a9 | Constantinos Venetsanopoulos | Graph of a scale-out Synnefo deployment |
1610 | 301294a9 | Constantinos Venetsanopoulos | --------------------------------------- |
1611 | 2f6143c9 | Constantinos Venetsanopoulos | |
1612 | 301294a9 | Constantinos Venetsanopoulos | Each box in the following graph corresponds to a distinct physical node: |
1613 | 2f6143c9 | Constantinos Venetsanopoulos | |
1614 | 301294a9 | Constantinos Venetsanopoulos | .. image:: images/synnefo-arch2-roles.png |
1615 | 301294a9 | Constantinos Venetsanopoulos | :width: 100% |
1616 | 301294a9 | Constantinos Venetsanopoulos | :target: _images/synnefo-arch2-roles.png |
1617 | 301294a9 | Constantinos Venetsanopoulos | |
1618 | 301294a9 | Constantinos Venetsanopoulos | The above graph is actually the same with the one at the beginning of this |
1619 | 301294a9 | Constantinos Venetsanopoulos | :ref:`guide <admin-guide>`, with the only difference that here we show the |
1620 | 301294a9 | Constantinos Venetsanopoulos | Synnefo roles of each physical node. These roles are described in the |
1621 | 301294a9 | Constantinos Venetsanopoulos | following section. |
1622 | 301294a9 | Constantinos Venetsanopoulos | |
1623 | 21d3d487 | Constantinos Venetsanopoulos | .. _physical-node-roles: |
1624 | 21d3d487 | Constantinos Venetsanopoulos | |
1625 | 301294a9 | Constantinos Venetsanopoulos | Physical Node roles |
1626 | 301294a9 | Constantinos Venetsanopoulos | ------------------- |
1627 | 301294a9 | Constantinos Venetsanopoulos | |
1628 | 301294a9 | Constantinos Venetsanopoulos | As appears in the previous graph, a scale-out Synnefo deployment consists of |
1629 | 301294a9 | Constantinos Venetsanopoulos | multiple physical nodes that have the following roles: |
1630 | 301294a9 | Constantinos Venetsanopoulos | |
1631 | 301294a9 | Constantinos Venetsanopoulos | * **WEBSERVER**: A web server running in front of gunicorn (e.g.: Apache, nginx) |
1632 | 301294a9 | Constantinos Venetsanopoulos | * **ASTAKOS**: The Astakos application (gunicorn) |
1633 | 301294a9 | Constantinos Venetsanopoulos | * **ASTAKOS_DB**: The Astakos database (postgresql) |
1634 | 301294a9 | Constantinos Venetsanopoulos | * **PITHOS**: The Pithos application (gunicorn) |
1635 | 301294a9 | Constantinos Venetsanopoulos | * **PITHOS_DB**: The Pithos database (postgresql) |
1636 | 301294a9 | Constantinos Venetsanopoulos | * **CYCLADES**: The Cyclades application (gunicorn) |
1637 | 301294a9 | Constantinos Venetsanopoulos | * **CYCLADES_DB**: The Cyclades database (postgresql) |
1638 | 301294a9 | Constantinos Venetsanopoulos | * **MQ**: The message queue (RabbitMQ) |
1639 | 301294a9 | Constantinos Venetsanopoulos | * **GANETI_MASTER**: The Ganeti master of a Ganeti cluster |
1640 | 301294a9 | Constantinos Venetsanopoulos | * **GANETI_NODE** : A VM-capable Ganeti node of a Ganeti cluster |
1641 | 2f6143c9 | Constantinos Venetsanopoulos | |
1642 | 301294a9 | Constantinos Venetsanopoulos | You will probably also have: |
1643 | 2f6143c9 | Constantinos Venetsanopoulos | |
1644 | 301294a9 | Constantinos Venetsanopoulos | * **CMS**: The CMS used as a frotend portal for the Synnefo services |
1645 | 21d3d487 | Constantinos Venetsanopoulos | * **NS**: A nameserver serving all other Synnefo nodes and resolving Synnefo FQDNs |
1646 | 301294a9 | Constantinos Venetsanopoulos | * **CLIENT**: A machine that runs the Synnefo clients (e.g.: kamaki, Web UI), |
1647 | 301294a9 | Constantinos Venetsanopoulos | most of the times, the end user's local machine |
1648 | 301294a9 | Constantinos Venetsanopoulos | |
1649 | 301294a9 | Constantinos Venetsanopoulos | From this point we will also refer to the following groups of roles: |
1650 | 301294a9 | Constantinos Venetsanopoulos | |
1651 | 301294a9 | Constantinos Venetsanopoulos | * **SYNNEFO**: [ **ASTAKOS**, **ASTAKOS_DB**, **PITHOS**, **PITHOS_DB**, **CYCLADES**, **CYCLADES_DB**, **MQ**, **CMS**] |
1652 | 301294a9 | Constantinos Venetsanopoulos | * **G_BACKEND**: [**GANETI_MASTER**, **GANETI_NODE**] |
1653 | 301294a9 | Constantinos Venetsanopoulos | |
1654 | 301294a9 | Constantinos Venetsanopoulos | Of course, when deploying Synnefo you can combine multiple of the above roles on a |
1655 | 301294a9 | Constantinos Venetsanopoulos | single physical node, but if you are trying to scale out, the above separation |
1656 | 301294a9 | Constantinos Venetsanopoulos | gives you significant advantages. |
1657 | 301294a9 | Constantinos Venetsanopoulos | |
1658 | 301294a9 | Constantinos Venetsanopoulos | So, in the next section we will take a look on what components you will have to |
1659 | 301294a9 | Constantinos Venetsanopoulos | install on each physical node depending on its Synnefo role. We assume the graph's |
1660 | 301294a9 | Constantinos Venetsanopoulos | architecture. |
1661 | 301294a9 | Constantinos Venetsanopoulos | |
1662 | 301294a9 | Constantinos Venetsanopoulos | Components for each role |
1663 | 301294a9 | Constantinos Venetsanopoulos | ------------------------ |
1664 | 2f6143c9 | Constantinos Venetsanopoulos | |
1665 | 301294a9 | Constantinos Venetsanopoulos | When deploying Synnefo in large scale, you need to install different Synnefo |
1666 | 301294a9 | Constantinos Venetsanopoulos | or/and third party components on different physical nodes according to their |
1667 | 301294a9 | Constantinos Venetsanopoulos | Synnefo role, as stated in the previous section. |
1668 | 301294a9 | Constantinos Venetsanopoulos | |
1669 | 301294a9 | Constantinos Venetsanopoulos | Specifically: |
1670 | 301294a9 | Constantinos Venetsanopoulos | |
1671 | 301294a9 | Constantinos Venetsanopoulos | Role **WEBSERVER** |
1672 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1673 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Apache |
1674 | 301294a9 | Constantinos Venetsanopoulos | Role **ASTAKOS** |
1675 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-webproject`, `snf-astakos-app` |
1676 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Django, Gunicorn |
1677 | 301294a9 | Constantinos Venetsanopoulos | Role **ASTAKOS_DB** |
1678 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1679 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: PostgreSQL |
1680 | 301294a9 | Constantinos Venetsanopoulos | Role **PITHOS** |
1681 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-webproject`, `snf-pithos-app`, `snf-pithos-webclient` |
1682 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Django, Gunicorn |
1683 | 301294a9 | Constantinos Venetsanopoulos | Role **PITHOS_DB** |
1684 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1685 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: PostgreSQL |
1686 | 301294a9 | Constantinos Venetsanopoulos | Role **CYCLADES** |
1687 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-webproject`, `snf-cyclades-app`, `snf-vncauthproxy` |
1688 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Django Gunicorn |
1689 | 301294a9 | Constantinos Venetsanopoulos | Role **CYCLADES_DB** |
1690 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1691 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: PostgreSQL |
1692 | 301294a9 | Constantinos Venetsanopoulos | Role **MQ** |
1693 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1694 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: RabbitMQ |
1695 | 301294a9 | Constantinos Venetsanopoulos | Role **GANETI_MASTER** |
1696 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-cyclades-gtools` |
1697 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Ganeti |
1698 | 301294a9 | Constantinos Venetsanopoulos | Role **GANETI_NODE** |
1699 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-cyclades-gtools`, `snf-network`, `snf-image`, `nfdhcpd` |
1700 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Ganeti |
1701 | 301294a9 | Constantinos Venetsanopoulos | Role **CMS** |
1702 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-webproject`, `snf-cloudcms` |
1703 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Django, Gunicorn |
1704 | 301294a9 | Constantinos Venetsanopoulos | Role **NS** |
1705 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1706 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: BIND |
1707 | 301294a9 | Constantinos Venetsanopoulos | Role **CLIENT** |
1708 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `kamaki`, `snf-image-creator` |
1709 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: `None` |
1710 | 301294a9 | Constantinos Venetsanopoulos | |
1711 | 301294a9 | Constantinos Venetsanopoulos | Example scale out installation |
1712 | 301294a9 | Constantinos Venetsanopoulos | ------------------------------ |
1713 | 301294a9 | Constantinos Venetsanopoulos | |
1714 | 301294a9 | Constantinos Venetsanopoulos | In this section we describe an example of a medium scale installation which |
1715 | 301294a9 | Constantinos Venetsanopoulos | combines multiple roles on 10 different physical nodes. We also provide a |
1716 | 301294a9 | Constantinos Venetsanopoulos | :ref:`guide <i-synnefo>` to help with such an install. |
1717 | 301294a9 | Constantinos Venetsanopoulos | |
1718 | 301294a9 | Constantinos Venetsanopoulos | We assume that we have the following 10 physical nodes with the corresponding |
1719 | 301294a9 | Constantinos Venetsanopoulos | roles: |
1720 | 301294a9 | Constantinos Venetsanopoulos | |
1721 | 301294a9 | Constantinos Venetsanopoulos | Node1: |
1722 | 301294a9 | Constantinos Venetsanopoulos | **WEBSERVER**, **ASTAKOS** |
1723 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1724 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1725 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`gunicorn <i-gunicorn>` |
1726 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apache <i-apache>` |
1727 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-webproject <i-webproject>` |
1728 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-astakos-app <i-astakos>` |
1729 | 301294a9 | Constantinos Venetsanopoulos | Node2: |
1730 | 301294a9 | Constantinos Venetsanopoulos | **WEBSERVER**, **PITHOS** |
1731 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1732 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1733 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`gunicorn <i-gunicorn>` |
1734 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apache <i-apache>` |
1735 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-webproject <i-webproject>` |
1736 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-pithos-app <i-pithos>` |
1737 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-pithos-webclient <i-pithos>` |
1738 | 301294a9 | Constantinos Venetsanopoulos | Node3: |
1739 | 301294a9 | Constantinos Venetsanopoulos | **WEBSERVER**, **CYCLADES** |
1740 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1741 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1742 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`gunicorn <i-gunicorn>` |
1743 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apache <i-apache>` |
1744 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-webproject <i-webproject>` |
1745 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-cyclades-app <i-cyclades>` |
1746 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-vncauthproxy <i-cyclades>` |
1747 | 301294a9 | Constantinos Venetsanopoulos | Node4: |
1748 | 301294a9 | Constantinos Venetsanopoulos | **WEBSERVER**, **CMS** |
1749 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1750 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1751 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`gunicorn <i-gunicorn>` |
1752 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apache <i-apache>` |
1753 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-webproject <i-webproject>` |
1754 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-cloudcms <i-cms>` |
1755 | 301294a9 | Constantinos Venetsanopoulos | Node5: |
1756 | 301294a9 | Constantinos Venetsanopoulos | **ASTAKOS_DB**, **PITHOS_DB**, **CYCLADES_DB** |
1757 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1758 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1759 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`postgresql <i-db>` |
1760 | 301294a9 | Constantinos Venetsanopoulos | Node6: |
1761 | 301294a9 | Constantinos Venetsanopoulos | **MQ** |
1762 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1763 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1764 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`rabbitmq <i-mq>` |
1765 | 301294a9 | Constantinos Venetsanopoulos | Node7: |
1766 | 301294a9 | Constantinos Venetsanopoulos | **GANETI_MASTER**, **GANETI_NODE** |
1767 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1768 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1769 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`general <i-backends>` |
1770 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`ganeti <i-ganeti>` |
1771 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-cyclades-gtools <i-gtools>` |
1772 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-network <i-network>` |
1773 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-image <i-image>` |
1774 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`nfdhcpd <i-network>` |
1775 | 301294a9 | Constantinos Venetsanopoulos | Node8: |
1776 | 301294a9 | Constantinos Venetsanopoulos | **GANETI_NODE** |
1777 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1778 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1779 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`general <i-backends>` |
1780 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`ganeti <i-ganeti>` |
1781 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-cyclades-gtools <i-gtools>` |
1782 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-network <i-network>` |
1783 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-image <i-image>` |
1784 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`nfdhcpd <i-network>` |
1785 | 301294a9 | Constantinos Venetsanopoulos | Node9: |
1786 | 301294a9 | Constantinos Venetsanopoulos | **GANETI_NODE** |
1787 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1788 | 301294a9 | Constantinos Venetsanopoulos | `Same as Node8` |
1789 | 301294a9 | Constantinos Venetsanopoulos | Node10: |
1790 | 301294a9 | Constantinos Venetsanopoulos | **GANETI_NODE** |
1791 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1792 | 301294a9 | Constantinos Venetsanopoulos | `Same as Node8` |
1793 | 301294a9 | Constantinos Venetsanopoulos | |
1794 | 301294a9 | Constantinos Venetsanopoulos | All sections: :ref:`Scale out Guide <i-synnefo>` |
1795 | 547c78f6 | Constantinos Venetsanopoulos | |
1796 | bc055d09 | Constantinos Venetsanopoulos | |
1797 | bbcd3dd1 | Constantinos Venetsanopoulos | Upgrade Notes |
1798 | bbcd3dd1 | Constantinos Venetsanopoulos | ============= |
1799 | bc055d09 | Constantinos Venetsanopoulos | |
1800 | 457fe4fc | Georgios D. Tsoukalas | .. toctree:: |
1801 | 457fe4fc | Georgios D. Tsoukalas | :maxdepth: 1 |
1802 | 457fe4fc | Georgios D. Tsoukalas | |
1803 | 1cd3daa1 | Constantinos Venetsanopoulos | v0.12 -> v0.13 <upgrade/upgrade-0.13> |
1804 | 09974606 | Georgios D. Tsoukalas | v0.13 -> v0.14 <upgrade/upgrade-0.14> |
1805 | 15151f83 | Christos Stavrakakis | v0.14 -> v0.14.2 <upgrade/upgrade-0.14.2> |
1806 | 457fe4fc | Georgios D. Tsoukalas | |
1807 | bbcd3dd1 | Constantinos Venetsanopoulos | |
1808 | 3045e739 | Constantinos Venetsanopoulos | Changelog, NEWS |
1809 | bbcd3dd1 | Constantinos Venetsanopoulos | =============== |
1810 | bbcd3dd1 | Constantinos Venetsanopoulos | |
1811 | 15151f83 | Christos Stavrakakis | |
1812 | 30fe9049 | Christos Stavrakakis | * v0.14.4 :ref:`Changelog <Changelog-0.14.5>`, :ref:`NEWS <NEWS-0.14.5>` |
1813 | 38be3b6e | Christos Stavrakakis | * v0.14.4 :ref:`Changelog <Changelog-0.14.4>`, :ref:`NEWS <NEWS-0.14.4>` |
1814 | cda13b21 | Christos Stavrakakis | * v0.14.3 :ref:`Changelog <Changelog-0.14.3>`, :ref:`NEWS <NEWS-0.14.3>` |
1815 | 15151f83 | Christos Stavrakakis | * v0.14.2 :ref:`Changelog <Changelog-0.14.2>`, :ref:`NEWS <NEWS-0.14.2>` |
1816 | a0f672f1 | Stratos Psomadakis | * v0.14 :ref:`Changelog <Changelog-0.14>`, :ref:`NEWS <NEWS-0.14>` |
1817 | bbcd3dd1 | Constantinos Venetsanopoulos | * v0.13 :ref:`Changelog <Changelog-0.13>`, :ref:`NEWS <NEWS-0.13>` |