root / docs / admin-guide.rst @ bbcd3dd1
History | View | Annotate | Download (38.9 kB)
1 | bc055d09 | Constantinos Venetsanopoulos | .. _admin-guide: |
---|---|---|---|
2 | bc055d09 | Constantinos Venetsanopoulos | |
3 | bc055d09 | Constantinos Venetsanopoulos | Synnefo Administrator's Guide |
4 | bc055d09 | Constantinos Venetsanopoulos | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
5 | bc055d09 | Constantinos Venetsanopoulos | |
6 | bc055d09 | Constantinos Venetsanopoulos | This is the complete Synnefo Administrator's Guide. |
7 | bc055d09 | Constantinos Venetsanopoulos | |
8 | bc055d09 | Constantinos Venetsanopoulos | |
9 | bc055d09 | Constantinos Venetsanopoulos | |
10 | eab73849 | Constantinos Venetsanopoulos | General Synnefo Architecture |
11 | eab73849 | Constantinos Venetsanopoulos | ============================ |
12 | eab73849 | Constantinos Venetsanopoulos | |
13 | eab73849 | Constantinos Venetsanopoulos | The following graph shows the whole Synnefo architecture and how it interacts |
14 | eab73849 | Constantinos Venetsanopoulos | with multiple Ganeti clusters. We hope that after reading the Administrator's |
15 | eab73849 | Constantinos Venetsanopoulos | Guide you will be able to understand every component and all the interactions |
16 | eab73849 | Constantinos Venetsanopoulos | between them. It is a good idea to first go through the Quick Administrator's |
17 | eab73849 | Constantinos Venetsanopoulos | Guide before proceeding. |
18 | eab73849 | Constantinos Venetsanopoulos | |
19 | 301294a9 | Constantinos Venetsanopoulos | .. image:: images/synnefo-arch2.png |
20 | 16539dad | Kostas Papadimitriou | :width: 100% |
21 | 301294a9 | Constantinos Venetsanopoulos | :target: _images/synnefo-arch2.png |
22 | eab73849 | Constantinos Venetsanopoulos | |
23 | eab73849 | Constantinos Venetsanopoulos | |
24 | a1c707c9 | Constantinos Venetsanopoulos | |
25 | eab73849 | Constantinos Venetsanopoulos | Identity Service (Astakos) |
26 | eab73849 | Constantinos Venetsanopoulos | ========================== |
27 | f846d8df | Constantinos Venetsanopoulos | |
28 | f846d8df | Constantinos Venetsanopoulos | |
29 | eab73849 | Constantinos Venetsanopoulos | Overview |
30 | eab73849 | Constantinos Venetsanopoulos | -------- |
31 | eab73849 | Constantinos Venetsanopoulos | |
32 | eab73849 | Constantinos Venetsanopoulos | Authentication methods |
33 | eab73849 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~~~~~ |
34 | eab73849 | Constantinos Venetsanopoulos | |
35 | eab73849 | Constantinos Venetsanopoulos | Local Authentication |
36 | eab73849 | Constantinos Venetsanopoulos | ```````````````````` |
37 | eab73849 | Constantinos Venetsanopoulos | |
38 | eab73849 | Constantinos Venetsanopoulos | LDAP Authentication |
39 | eab73849 | Constantinos Venetsanopoulos | ``````````````````` |
40 | eab73849 | Constantinos Venetsanopoulos | |
41 | eab73849 | Constantinos Venetsanopoulos | .. _shibboleth-auth: |
42 | eab73849 | Constantinos Venetsanopoulos | |
43 | eab73849 | Constantinos Venetsanopoulos | Shibboleth Authentication |
44 | eab73849 | Constantinos Venetsanopoulos | ````````````````````````` |
45 | eab73849 | Constantinos Venetsanopoulos | |
46 | eab73849 | Constantinos Venetsanopoulos | Astakos can delegate user authentication to a Shibboleth federation. |
47 | eab73849 | Constantinos Venetsanopoulos | |
48 | eab73849 | Constantinos Venetsanopoulos | To setup shibboleth, install package:: |
49 | eab73849 | Constantinos Venetsanopoulos | |
50 | eab73849 | Constantinos Venetsanopoulos | apt-get install libapache2-mod-shib2 |
51 | eab73849 | Constantinos Venetsanopoulos | |
52 | eab73849 | Constantinos Venetsanopoulos | Change appropriately the configuration files in ``/etc/shibboleth``. |
53 | eab73849 | Constantinos Venetsanopoulos | |
54 | eab73849 | Constantinos Venetsanopoulos | Add in ``/etc/apache2/sites-available/synnefo-ssl``:: |
55 | eab73849 | Constantinos Venetsanopoulos | |
56 | eab73849 | Constantinos Venetsanopoulos | ShibConfig /etc/shibboleth/shibboleth2.xml |
57 | eab73849 | Constantinos Venetsanopoulos | Alias /shibboleth-sp /usr/share/shibboleth |
58 | eab73849 | Constantinos Venetsanopoulos | |
59 | eab73849 | Constantinos Venetsanopoulos | <Location /im/login/shibboleth> |
60 | eab73849 | Constantinos Venetsanopoulos | AuthType shibboleth |
61 | eab73849 | Constantinos Venetsanopoulos | ShibRequireSession On |
62 | eab73849 | Constantinos Venetsanopoulos | ShibUseHeaders On |
63 | eab73849 | Constantinos Venetsanopoulos | require valid-user |
64 | eab73849 | Constantinos Venetsanopoulos | </Location> |
65 | eab73849 | Constantinos Venetsanopoulos | |
66 | eab73849 | Constantinos Venetsanopoulos | and before the line containing:: |
67 | eab73849 | Constantinos Venetsanopoulos | |
68 | eab73849 | Constantinos Venetsanopoulos | ProxyPass / http://localhost:8080/ retry=0 |
69 | eab73849 | Constantinos Venetsanopoulos | |
70 | eab73849 | Constantinos Venetsanopoulos | add:: |
71 | eab73849 | Constantinos Venetsanopoulos | |
72 | eab73849 | Constantinos Venetsanopoulos | ProxyPass /Shibboleth.sso ! |
73 | f846d8df | Constantinos Venetsanopoulos | |
74 | eab73849 | Constantinos Venetsanopoulos | Then, enable the shibboleth module:: |
75 | eab73849 | Constantinos Venetsanopoulos | |
76 | eab73849 | Constantinos Venetsanopoulos | a2enmod shib2 |
77 | f846d8df | Constantinos Venetsanopoulos | |
78 | eab73849 | Constantinos Venetsanopoulos | After passing through the apache module, the following tokens should be |
79 | eab73849 | Constantinos Venetsanopoulos | available at the destination:: |
80 | eab73849 | Constantinos Venetsanopoulos | |
81 | eab73849 | Constantinos Venetsanopoulos | eppn # eduPersonPrincipalName |
82 | eab73849 | Constantinos Venetsanopoulos | Shib-InetOrgPerson-givenName |
83 | eab73849 | Constantinos Venetsanopoulos | Shib-Person-surname |
84 | eab73849 | Constantinos Venetsanopoulos | Shib-Person-commonName |
85 | eab73849 | Constantinos Venetsanopoulos | Shib-InetOrgPerson-displayName |
86 | eab73849 | Constantinos Venetsanopoulos | Shib-EP-Affiliation |
87 | eab73849 | Constantinos Venetsanopoulos | Shib-Session-ID |
88 | eab73849 | Constantinos Venetsanopoulos | |
89 | eab73849 | Constantinos Venetsanopoulos | Finally, add 'shibboleth' in ``ASTAKOS_IM_MODULES`` list. The variable resides |
90 | eab73849 | Constantinos Venetsanopoulos | inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf`` |
91 | eab73849 | Constantinos Venetsanopoulos | |
92 | eab73849 | Constantinos Venetsanopoulos | Architecture |
93 | eab73849 | Constantinos Venetsanopoulos | ------------ |
94 | eab73849 | Constantinos Venetsanopoulos | |
95 | eab73849 | Constantinos Venetsanopoulos | Prereqs |
96 | eab73849 | Constantinos Venetsanopoulos | ------- |
97 | eab73849 | Constantinos Venetsanopoulos | |
98 | eab73849 | Constantinos Venetsanopoulos | Installation |
99 | eab73849 | Constantinos Venetsanopoulos | ------------ |
100 | eab73849 | Constantinos Venetsanopoulos | |
101 | eab73849 | Constantinos Venetsanopoulos | Configuration |
102 | eab73849 | Constantinos Venetsanopoulos | ------------- |
103 | eab73849 | Constantinos Venetsanopoulos | |
104 | eab73849 | Constantinos Venetsanopoulos | Working with Astakos |
105 | eab73849 | Constantinos Venetsanopoulos | -------------------- |
106 | eab73849 | Constantinos Venetsanopoulos | |
107 | eab73849 | Constantinos Venetsanopoulos | User activation methods |
108 | eab73849 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~~~~~~ |
109 | f846d8df | Constantinos Venetsanopoulos | |
110 | f846d8df | Constantinos Venetsanopoulos | When a new user signs up, he/she is not marked as active. You can see his/her |
111 | f846d8df | Constantinos Venetsanopoulos | state by running (on the machine that runs the Astakos app): |
112 | f846d8df | Constantinos Venetsanopoulos | |
113 | f846d8df | Constantinos Venetsanopoulos | .. code-block:: console |
114 | f846d8df | Constantinos Venetsanopoulos | |
115 | 4404e02e | Ilias Tsitsimpis | $ snf-manage user-list |
116 | f846d8df | Constantinos Venetsanopoulos | |
117 | f846d8df | Constantinos Venetsanopoulos | There are two different ways to activate a new user. Both need access to a |
118 | 4404e02e | Ilias Tsitsimpis | running :ref:`mail server <mail-server>`. |
119 | f846d8df | Constantinos Venetsanopoulos | |
120 | f846d8df | Constantinos Venetsanopoulos | Manual activation |
121 | eab73849 | Constantinos Venetsanopoulos | ````````````````` |
122 | f846d8df | Constantinos Venetsanopoulos | |
123 | f846d8df | Constantinos Venetsanopoulos | You can manually activate a new user that has already signed up, by sending |
124 | f846d8df | Constantinos Venetsanopoulos | him/her an activation email. The email will contain an approriate activation |
125 | f846d8df | Constantinos Venetsanopoulos | link, which will complete the activation process if followed. You can send the |
126 | f846d8df | Constantinos Venetsanopoulos | email by running: |
127 | f846d8df | Constantinos Venetsanopoulos | |
128 | f846d8df | Constantinos Venetsanopoulos | .. code-block:: console |
129 | f846d8df | Constantinos Venetsanopoulos | |
130 | 4404e02e | Ilias Tsitsimpis | $ snf-manage user-activation-send <user ID or email> |
131 | f846d8df | Constantinos Venetsanopoulos | |
132 | eab73849 | Constantinos Venetsanopoulos | Be sure to have already setup your mail server and defined it in your Synnefo |
133 | f846d8df | Constantinos Venetsanopoulos | settings, before running the command. |
134 | f846d8df | Constantinos Venetsanopoulos | |
135 | f846d8df | Constantinos Venetsanopoulos | Automatic activation |
136 | eab73849 | Constantinos Venetsanopoulos | ```````````````````` |
137 | eab73849 | Constantinos Venetsanopoulos | |
138 | eab73849 | Constantinos Venetsanopoulos | FIXME: Describe Regex activation method |
139 | eab73849 | Constantinos Venetsanopoulos | |
140 | 2988ae31 | Georgios D. Tsoukalas | Setting quota limits |
141 | 2988ae31 | Georgios D. Tsoukalas | ~~~~~~~~~~~~~~~~~~~~ |
142 | 2988ae31 | Georgios D. Tsoukalas | |
143 | 2988ae31 | Georgios D. Tsoukalas | Set default quotas |
144 | 2988ae31 | Georgios D. Tsoukalas | `````````````````` |
145 | 2988ae31 | Georgios D. Tsoukalas | |
146 | 2988ae31 | Georgios D. Tsoukalas | In 20-snf-astakos-app-settings.conf, |
147 | 2988ae31 | Georgios D. Tsoukalas | uncomment the default setting ``ASTAKOS_SERVICES`` |
148 | 2988ae31 | Georgios D. Tsoukalas | and customize the ``'uplimit'`` values. |
149 | 2988ae31 | Georgios D. Tsoukalas | These are the default base quotas for all users. |
150 | 2988ae31 | Georgios D. Tsoukalas | |
151 | 2988ae31 | Georgios D. Tsoukalas | To apply your configuration run:: |
152 | 2988ae31 | Georgios D. Tsoukalas | |
153 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage astakos-init --load-service-resources |
154 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage astakos-quota --sync |
155 | 2988ae31 | Georgios D. Tsoukalas | |
156 | 2988ae31 | Georgios D. Tsoukalas | Set base quotas for individual users |
157 | 2988ae31 | Georgios D. Tsoukalas | ```````````````````````````````````` |
158 | 2988ae31 | Georgios D. Tsoukalas | |
159 | 2988ae31 | Georgios D. Tsoukalas | For individual users that need different quotas than the default |
160 | 2988ae31 | Georgios D. Tsoukalas | you can set it for each resource like this:: |
161 | 2988ae31 | Georgios D. Tsoukalas | |
162 | 2988ae31 | Georgios D. Tsoukalas | # use this to display quotas / uuid |
163 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage user-show 'uuid or email' |
164 | 2988ae31 | Georgios D. Tsoukalas | |
165 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage user-set-initial-quota --set-capacity 'user-uuid' 'cyclades.vm' 10 |
166 | 2988ae31 | Georgios D. Tsoukalas | |
167 | 2988ae31 | Georgios D. Tsoukalas | # this applies the configuration |
168 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage astakos-quota --sync --user 'user-uuid' |
169 | 2988ae31 | Georgios D. Tsoukalas | |
170 | 2988ae31 | Georgios D. Tsoukalas | |
171 | 2988ae31 | Georgios D. Tsoukalas | Enable the Projects feature |
172 | 2988ae31 | Georgios D. Tsoukalas | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
173 | 2988ae31 | Georgios D. Tsoukalas | |
174 | 2988ae31 | Georgios D. Tsoukalas | If you want to enable the projects feature so that users may apply |
175 | 2988ae31 | Georgios D. Tsoukalas | on their own for resources by creating and joining projects, |
176 | 2988ae31 | Georgios D. Tsoukalas | in ``20-snf-astakos-app-settings.conf`` set:: |
177 | 2988ae31 | Georgios D. Tsoukalas | |
178 | 2988ae31 | Georgios D. Tsoukalas | # this will allow at most one pending project application per user |
179 | 2988ae31 | Georgios D. Tsoukalas | ASTAKOS_PENDING_APPLICATION_LIMIT = 1 |
180 | 2988ae31 | Georgios D. Tsoukalas | # this will make the 'projects' page visible in the dashboard |
181 | 2988ae31 | Georgios D. Tsoukalas | ASTAKOS_PROJECTS_VISIBLE = True |
182 | 2988ae31 | Georgios D. Tsoukalas | |
183 | 2988ae31 | Georgios D. Tsoukalas | When users apply for projects they are not automatically granted |
184 | 2988ae31 | Georgios D. Tsoukalas | the resources. They must first be approved by the administrator. |
185 | 2988ae31 | Georgios D. Tsoukalas | |
186 | 2988ae31 | Georgios D. Tsoukalas | To list pending project applications in astakos:: |
187 | 2988ae31 | Georgios D. Tsoukalas | |
188 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage project-list --pending |
189 | 2988ae31 | Georgios D. Tsoukalas | |
190 | 2988ae31 | Georgios D. Tsoukalas | Note the last column, the application id. To approve it:: |
191 | 2988ae31 | Georgios D. Tsoukalas | |
192 | 2988ae31 | Georgios D. Tsoukalas | # <app id> from the last column of project-list |
193 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage project-control --approve <app id> |
194 | 2988ae31 | Georgios D. Tsoukalas | |
195 | 2988ae31 | Georgios D. Tsoukalas | To deny an application:: |
196 | 2988ae31 | Georgios D. Tsoukalas | |
197 | 2988ae31 | Georgios D. Tsoukalas | # snf-manage project-control --deny <app id> |
198 | 2988ae31 | Georgios D. Tsoukalas | |
199 | 2988ae31 | Georgios D. Tsoukalas | |
200 | 2988ae31 | Georgios D. Tsoukalas | |
201 | eab73849 | Constantinos Venetsanopoulos | Astakos advanced operations |
202 | eab73849 | Constantinos Venetsanopoulos | --------------------------- |
203 | eab73849 | Constantinos Venetsanopoulos | |
204 | eab73849 | Constantinos Venetsanopoulos | Adding "Terms of Use" |
205 | eab73849 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~~~~ |
206 | eab73849 | Constantinos Venetsanopoulos | |
207 | eab73849 | Constantinos Venetsanopoulos | Astakos supports versioned terms-of-use. First of all you need to create an |
208 | eab73849 | Constantinos Venetsanopoulos | html file that will contain your terms. For example, create the file |
209 | eab73849 | Constantinos Venetsanopoulos | ``/usr/share/synnefo/sample-terms.html``, which contains the following: |
210 | eab73849 | Constantinos Venetsanopoulos | |
211 | eab73849 | Constantinos Venetsanopoulos | .. code-block:: console |
212 | eab73849 | Constantinos Venetsanopoulos | |
213 | eab73849 | Constantinos Venetsanopoulos | <h1>~okeanos terms</h1> |
214 | eab73849 | Constantinos Venetsanopoulos | |
215 | eab73849 | Constantinos Venetsanopoulos | These are the example terms for ~okeanos |
216 | eab73849 | Constantinos Venetsanopoulos | |
217 | eab73849 | Constantinos Venetsanopoulos | Then, add those terms-of-use with the snf-manage command: |
218 | eab73849 | Constantinos Venetsanopoulos | |
219 | eab73849 | Constantinos Venetsanopoulos | .. code-block:: console |
220 | eab73849 | Constantinos Venetsanopoulos | |
221 | eab73849 | Constantinos Venetsanopoulos | $ snf-manage term-add /usr/share/synnefo/sample-terms.html |
222 | eab73849 | Constantinos Venetsanopoulos | |
223 | eab73849 | Constantinos Venetsanopoulos | Your terms have been successfully added and you will see the corresponding link |
224 | eab73849 | Constantinos Venetsanopoulos | appearing in the Astakos web pages' footer. |
225 | eab73849 | Constantinos Venetsanopoulos | |
226 | a23251f4 | Constantinos Venetsanopoulos | Enabling reCAPTCHA |
227 | a23251f4 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~ |
228 | a23251f4 | Constantinos Venetsanopoulos | |
229 | a23251f4 | Constantinos Venetsanopoulos | Astakos supports the `reCAPTCHA <http://www.google.com/recaptcha>`_ feature. |
230 | a23251f4 | Constantinos Venetsanopoulos | If enabled, it protects the Astakos forms from bots. To enable the feature, go |
231 | a23251f4 | Constantinos Venetsanopoulos | to https://www.google.com/recaptcha/admin/create and create your own reCAPTCHA |
232 | a23251f4 | Constantinos Venetsanopoulos | key pair. Then edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` and set |
233 | a23251f4 | Constantinos Venetsanopoulos | the corresponding variables to reflect your newly created key pair. Finally, set |
234 | a23251f4 | Constantinos Venetsanopoulos | the ``ASTAKOS_RECAPTCHA_ENABLED`` variable to ``True``: |
235 | a23251f4 | Constantinos Venetsanopoulos | |
236 | a23251f4 | Constantinos Venetsanopoulos | .. code-block:: console |
237 | a23251f4 | Constantinos Venetsanopoulos | |
238 | a23251f4 | Constantinos Venetsanopoulos | ASTAKOS_RECAPTCHA_PUBLIC_KEY = 'example_recaptcha_public_key!@#$%^&*(' |
239 | a23251f4 | Constantinos Venetsanopoulos | ASTAKOS_RECAPTCHA_PRIVATE_KEY = 'example_recaptcha_private_key!@#$%^&*(' |
240 | a23251f4 | Constantinos Venetsanopoulos | |
241 | a23251f4 | Constantinos Venetsanopoulos | ASTAKOS_RECAPTCHA_ENABLED = True |
242 | a23251f4 | Constantinos Venetsanopoulos | |
243 | a23251f4 | Constantinos Venetsanopoulos | Restart the service on the Astakos node(s) and you are ready: |
244 | a23251f4 | Constantinos Venetsanopoulos | |
245 | a23251f4 | Constantinos Venetsanopoulos | .. code-block:: console |
246 | a23251f4 | Constantinos Venetsanopoulos | |
247 | a23251f4 | Constantinos Venetsanopoulos | # /etc/init.d/gunicorn restart |
248 | a23251f4 | Constantinos Venetsanopoulos | |
249 | a23251f4 | Constantinos Venetsanopoulos | Checkout your new Sign up page. If you see the reCAPTCHA box, you have setup |
250 | a23251f4 | Constantinos Venetsanopoulos | everything correctly. |
251 | a23251f4 | Constantinos Venetsanopoulos | |
252 | eab73849 | Constantinos Venetsanopoulos | |
253 | eab73849 | Constantinos Venetsanopoulos | |
254 | eab73849 | Constantinos Venetsanopoulos | File Storage Service (Pithos) |
255 | eab73849 | Constantinos Venetsanopoulos | ============================= |
256 | eab73849 | Constantinos Venetsanopoulos | |
257 | eab73849 | Constantinos Venetsanopoulos | Overview |
258 | eab73849 | Constantinos Venetsanopoulos | -------- |
259 | eab73849 | Constantinos Venetsanopoulos | |
260 | eab73849 | Constantinos Venetsanopoulos | Architecture |
261 | eab73849 | Constantinos Venetsanopoulos | ------------ |
262 | eab73849 | Constantinos Venetsanopoulos | |
263 | eab73849 | Constantinos Venetsanopoulos | Prereqs |
264 | eab73849 | Constantinos Venetsanopoulos | ------- |
265 | eab73849 | Constantinos Venetsanopoulos | |
266 | eab73849 | Constantinos Venetsanopoulos | Installation |
267 | eab73849 | Constantinos Venetsanopoulos | ------------ |
268 | eab73849 | Constantinos Venetsanopoulos | |
269 | eab73849 | Constantinos Venetsanopoulos | Configuration |
270 | eab73849 | Constantinos Venetsanopoulos | ------------- |
271 | eab73849 | Constantinos Venetsanopoulos | |
272 | eab73849 | Constantinos Venetsanopoulos | Working with Pithos |
273 | eab73849 | Constantinos Venetsanopoulos | ------------------- |
274 | eab73849 | Constantinos Venetsanopoulos | |
275 | eab73849 | Constantinos Venetsanopoulos | Pithos advanced operations |
276 | eab73849 | Constantinos Venetsanopoulos | -------------------------- |
277 | eab73849 | Constantinos Venetsanopoulos | |
278 | eab73849 | Constantinos Venetsanopoulos | |
279 | eab73849 | Constantinos Venetsanopoulos | |
280 | eab73849 | Constantinos Venetsanopoulos | Compute/Network/Image Service (Cyclades) |
281 | eab73849 | Constantinos Venetsanopoulos | ======================================== |
282 | eab73849 | Constantinos Venetsanopoulos | |
283 | eab73849 | Constantinos Venetsanopoulos | Compute Overview |
284 | eab73849 | Constantinos Venetsanopoulos | ---------------- |
285 | eab73849 | Constantinos Venetsanopoulos | |
286 | eab73849 | Constantinos Venetsanopoulos | Network Overview |
287 | eab73849 | Constantinos Venetsanopoulos | ---------------- |
288 | eab73849 | Constantinos Venetsanopoulos | |
289 | eab73849 | Constantinos Venetsanopoulos | Image Overview |
290 | eab73849 | Constantinos Venetsanopoulos | -------------- |
291 | eab73849 | Constantinos Venetsanopoulos | |
292 | eab73849 | Constantinos Venetsanopoulos | Architecture |
293 | eab73849 | Constantinos Venetsanopoulos | ------------ |
294 | eab73849 | Constantinos Venetsanopoulos | |
295 | b9fd8ed3 | Christos Stavrakakis | Asynchronous communication with Ganeti backends |
296 | 90c74332 | Christos Stavrakakis | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
297 | a1c707c9 | Constantinos Venetsanopoulos | Synnefo uses Google Ganeti backends for VM cluster management. In order for |
298 | a1c707c9 | Constantinos Venetsanopoulos | Cyclades to be able to handle thousands of user requests, Cyclades and Ganeti |
299 | a1c707c9 | Constantinos Venetsanopoulos | communicate asynchronously. Briefly, requests are submitted to Ganeti through |
300 | a1c707c9 | Constantinos Venetsanopoulos | Ganeti's RAPI/HTTP interface, and then asynchronous notifications about the |
301 | a1c707c9 | Constantinos Venetsanopoulos | progress of Ganeti jobs are being created and pushed upwards to Cyclades. The |
302 | a1c707c9 | Constantinos Venetsanopoulos | architecture and communication with a Ganeti backend is shown in the graph |
303 | a1c707c9 | Constantinos Venetsanopoulos | below: |
304 | 9b659162 | Christos Stavrakakis | |
305 | 9b659162 | Christos Stavrakakis | .. image:: images/cyclades-ganeti-communication.png |
306 | 9b659162 | Christos Stavrakakis | :width: 50% |
307 | 9b659162 | Christos Stavrakakis | :target: _images/cyclades-ganeti-communication.png |
308 | b9fd8ed3 | Christos Stavrakakis | |
309 | a1c707c9 | Constantinos Venetsanopoulos | The Cyclades API server is responsible for handling user requests. Read-only |
310 | b9fd8ed3 | Christos Stavrakakis | requests are directly served by looking up the Cyclades DB. If the request |
311 | b9fd8ed3 | Christos Stavrakakis | needs an action in the Ganeti backend, Cyclades submit jobs to the Ganeti |
312 | a1c707c9 | Constantinos Venetsanopoulos | master using the `Ganeti RAPI interface |
313 | a1c707c9 | Constantinos Venetsanopoulos | <http://docs.ganeti.org/ganeti/2.2/html/rapi.html>`_. |
314 | b9fd8ed3 | Christos Stavrakakis | |
315 | a1c707c9 | Constantinos Venetsanopoulos | While Ganeti executes the job, `snf-ganeti-eventd`, `snf-ganeti-hook` and |
316 | a1c707c9 | Constantinos Venetsanopoulos | `snf-progress-monitor` are monitoring the progress of the job and send |
317 | c84d78c8 | Christos Stavrakakis | corresponding messages to the RabbitMQ servers. These components are part |
318 | a1c707c9 | Constantinos Venetsanopoulos | of `snf-cyclades-gtools` and must be installed on all Ganeti nodes. Specially: |
319 | b9fd8ed3 | Christos Stavrakakis | |
320 | b9fd8ed3 | Christos Stavrakakis | * *snf-ganeti-eventd* sends messages about operations affecting the operating |
321 | a1c707c9 | Constantinos Venetsanopoulos | state of instances and networks. Works by monitoring the Ganeti job queue. |
322 | a1c707c9 | Constantinos Venetsanopoulos | * *snf-ganeti_hook* sends messages about the NICs of instances. It includes a |
323 | 90c74332 | Christos Stavrakakis | number of `Ganeti hooks <http://docs.ganeti.org/ganeti/2.2/html/hooks.html>`_ |
324 | 90c74332 | Christos Stavrakakis | for customisation of operations. |
325 | a1c707c9 | Constantinos Venetsanopoulos | * *snf-progress_monitor* sends messages about the progress of the Image deployment |
326 | a1c707c9 | Constantinos Venetsanopoulos | phase which is done by the Ganeti OS Definition `snf-image`. |
327 | b9fd8ed3 | Christos Stavrakakis | |
328 | a1c707c9 | Constantinos Venetsanopoulos | Finally, `snf-dispatcher` consumes messages from the RabbitMQ queues, processes |
329 | a1c707c9 | Constantinos Venetsanopoulos | these messages and properly updates the state of the Cyclades DB. Subsequent |
330 | a1c707c9 | Constantinos Venetsanopoulos | requests to the Cyclades API, will retrieve the updated state from the DB. |
331 | b9fd8ed3 | Christos Stavrakakis | |
332 | b9fd8ed3 | Christos Stavrakakis | |
333 | eab73849 | Constantinos Venetsanopoulos | Prereqs |
334 | eab73849 | Constantinos Venetsanopoulos | ------- |
335 | eab73849 | Constantinos Venetsanopoulos | |
336 | c84d78c8 | Christos Stavrakakis | Work in progress. Please refer to :ref:`quick administrator quide <quick-install-admin-guide>`. |
337 | c84d78c8 | Christos Stavrakakis | |
338 | eab73849 | Constantinos Venetsanopoulos | Installation |
339 | eab73849 | Constantinos Venetsanopoulos | ------------ |
340 | eab73849 | Constantinos Venetsanopoulos | |
341 | c84d78c8 | Christos Stavrakakis | Work in progress. Please refer to :ref:`quick administrator quide <quick-install-admin-guide>`. |
342 | c84d78c8 | Christos Stavrakakis | |
343 | eab73849 | Constantinos Venetsanopoulos | Configuration |
344 | eab73849 | Constantinos Venetsanopoulos | ------------- |
345 | eab73849 | Constantinos Venetsanopoulos | |
346 | c84d78c8 | Christos Stavrakakis | Work in progress. Please refer to :ref:`quick administrator quide <quick-install-admin-guide>`. |
347 | c84d78c8 | Christos Stavrakakis | |
348 | eab73849 | Constantinos Venetsanopoulos | Working with Cyclades |
349 | eab73849 | Constantinos Venetsanopoulos | --------------------- |
350 | eab73849 | Constantinos Venetsanopoulos | |
351 | 0768a1df | Christos Stavrakakis | Managing Ganeti Backends |
352 | 90c74332 | Christos Stavrakakis | ~~~~~~~~~~~~~~~~~~~~~~~~ |
353 | 0768a1df | Christos Stavrakakis | |
354 | a1c707c9 | Constantinos Venetsanopoulos | Since v0.11, Synnefo is able to manage multiple Ganeti clusters (backends) |
355 | a1c707c9 | Constantinos Venetsanopoulos | making it capable to scale linearly to tens of thousands of VMs. Backends |
356 | a1c707c9 | Constantinos Venetsanopoulos | can be dynamically added or removed via `snf-manage` commands. |
357 | 0768a1df | Christos Stavrakakis | |
358 | a1c707c9 | Constantinos Venetsanopoulos | Each newly created VM is allocated to a Ganeti backend by the Cyclades backend |
359 | a1c707c9 | Constantinos Venetsanopoulos | allocator. The VM is "pinned" to this backend, and can not change through its |
360 | a1c707c9 | Constantinos Venetsanopoulos | lifetime. The backend allocator decides in which backend to spawn the VM based |
361 | a1c707c9 | Constantinos Venetsanopoulos | on the available resources of each backend, trying to balance the load between |
362 | a1c707c9 | Constantinos Venetsanopoulos | them. |
363 | 0768a1df | Christos Stavrakakis | |
364 | 0768a1df | Christos Stavrakakis | Handling of Networks, as far as backends are concerned, is based on whether the |
365 | a1c707c9 | Constantinos Venetsanopoulos | network is public or not. Public networks are created through the `snf-manage |
366 | a1c707c9 | Constantinos Venetsanopoulos | network-create` command, and are only created on one backend. Private networks |
367 | a1c707c9 | Constantinos Venetsanopoulos | are created on all backends, in order to ensure that VMs residing on different |
368 | a1c707c9 | Constantinos Venetsanopoulos | backends can be connected to the same private network. |
369 | 0768a1df | Christos Stavrakakis | |
370 | a1c707c9 | Constantinos Venetsanopoulos | Listing existing backends |
371 | a1c707c9 | Constantinos Venetsanopoulos | ````````````````````````` |
372 | a1c707c9 | Constantinos Venetsanopoulos | To list all the Ganeti backends known to Synnefo, we run: |
373 | 90c74332 | Christos Stavrakakis | |
374 | 0768a1df | Christos Stavrakakis | .. code-block:: console |
375 | 0768a1df | Christos Stavrakakis | |
376 | 0768a1df | Christos Stavrakakis | $ snf-manage backend-list |
377 | 0768a1df | Christos Stavrakakis | |
378 | 0768a1df | Christos Stavrakakis | Adding a new Ganeti backend |
379 | 90c74332 | Christos Stavrakakis | ``````````````````````````` |
380 | 0768a1df | Christos Stavrakakis | Backends are dynamically added under the control of Synnefo with `snf-manage |
381 | 0768a1df | Christos Stavrakakis | backend-add` command. In this section it is assumed that a Ganeti cluster, |
382 | a1c707c9 | Constantinos Venetsanopoulos | named ``cluster.example.com`` is already up and running and configured to be |
383 | a1c707c9 | Constantinos Venetsanopoulos | able to host Synnefo VMs. |
384 | 0768a1df | Christos Stavrakakis | |
385 | a1c707c9 | Constantinos Venetsanopoulos | To add this Ganeti cluster, we run: |
386 | 90c74332 | Christos Stavrakakis | |
387 | 0768a1df | Christos Stavrakakis | .. code-block:: console |
388 | 0768a1df | Christos Stavrakakis | |
389 | 0768a1df | Christos Stavrakakis | $ snf-manage backend-add --clustername=cluster.example.com --user="synnefo_user" --pass="synnefo_pass" |
390 | 0768a1df | Christos Stavrakakis | |
391 | 46ccbcb9 | Christos Stavrakakis | where ``clustername`` is the Cluster hostname of the Ganeti cluster, and |
392 | 46ccbcb9 | Christos Stavrakakis | ``user`` and ``pass`` are the credentials for the `Ganeti RAPI user |
393 | 46ccbcb9 | Christos Stavrakakis | <http://docs.ganeti.org/ganeti/2.2/html/rapi.html#users-and-passwords>`_. All |
394 | 46ccbcb9 | Christos Stavrakakis | backend attributes can be also changed dynamically using the `snf-manage |
395 | 46ccbcb9 | Christos Stavrakakis | backend-modify` command. |
396 | 0768a1df | Christos Stavrakakis | |
397 | a1c707c9 | Constantinos Venetsanopoulos | ``snf-manage backend-add`` will also create all existing private networks to |
398 | a1c707c9 | Constantinos Venetsanopoulos | the new backend. You can verify that the backend is added, by running |
399 | a1c707c9 | Constantinos Venetsanopoulos | `snf-manage backend-list`. |
400 | 0768a1df | Christos Stavrakakis | |
401 | a1c707c9 | Constantinos Venetsanopoulos | Note that no VMs will be spawned to this backend, since by default it is in a |
402 | a1c707c9 | Constantinos Venetsanopoulos | ``drained`` state after addition and also it has no public network assigned to |
403 | a1c707c9 | Constantinos Venetsanopoulos | it. |
404 | a1c707c9 | Constantinos Venetsanopoulos | |
405 | a1c707c9 | Constantinos Venetsanopoulos | So, first you need to create its public network, make sure everything works as |
406 | a1c707c9 | Constantinos Venetsanopoulos | expected and finally make it active by un-setting the ``drained`` flag. You can |
407 | a1c707c9 | Constantinos Venetsanopoulos | do this by running: |
408 | a1c707c9 | Constantinos Venetsanopoulos | |
409 | a1c707c9 | Constantinos Venetsanopoulos | .. code-block:: console |
410 | a1c707c9 | Constantinos Venetsanopoulos | |
411 | 46ccbcb9 | Christos Stavrakakis | $ snf-manage backend-modify --drained=False <backend_id> |
412 | 0768a1df | Christos Stavrakakis | |
413 | 0768a1df | Christos Stavrakakis | Removing an existing Ganeti backend |
414 | 90c74332 | Christos Stavrakakis | ``````````````````````````````````` |
415 | a1c707c9 | Constantinos Venetsanopoulos | In order to remove an existing backend from Synnefo, we run: |
416 | 90c74332 | Christos Stavrakakis | |
417 | 0768a1df | Christos Stavrakakis | .. code-block:: console |
418 | 0768a1df | Christos Stavrakakis | |
419 | a1c707c9 | Constantinos Venetsanopoulos | # snf-manage backend-remove <backend_id> |
420 | 0768a1df | Christos Stavrakakis | |
421 | a1c707c9 | Constantinos Venetsanopoulos | This command will fail if there are active VMs on the backend. Also, the |
422 | a1c707c9 | Constantinos Venetsanopoulos | backend is not cleaned before removal, so all the Synnefo private networks |
423 | a1c707c9 | Constantinos Venetsanopoulos | will be left on the Ganeti nodes. You need to remove them manually. |
424 | 0768a1df | Christos Stavrakakis | |
425 | a1c707c9 | Constantinos Venetsanopoulos | Allocation of VMs in Ganeti backends |
426 | a1c707c9 | Constantinos Venetsanopoulos | ```````````````````````````````````` |
427 | a1c707c9 | Constantinos Venetsanopoulos | As already mentioned, the Cyclades backend allocator is responsible for |
428 | a1c707c9 | Constantinos Venetsanopoulos | allocating new VMs to backends. This allocator does not choose the exact Ganeti |
429 | a1c707c9 | Constantinos Venetsanopoulos | node that will host the VM but just the Ganeti backend. The exact node is |
430 | a1c707c9 | Constantinos Venetsanopoulos | chosen by the Ganeti cluster's allocator (hail). |
431 | 0768a1df | Christos Stavrakakis | |
432 | 0768a1df | Christos Stavrakakis | The decision about which backend will host a VM is based on the available |
433 | 0768a1df | Christos Stavrakakis | resources. The allocator computes a score for each backend, that shows its load |
434 | 0768a1df | Christos Stavrakakis | factor, and the one with the minimum score is chosen. The admin can exclude |
435 | a1c707c9 | Constantinos Venetsanopoulos | backends from the allocation phase by marking them as ``drained`` by running: |
436 | 90c74332 | Christos Stavrakakis | |
437 | 0768a1df | Christos Stavrakakis | .. code-block:: console |
438 | 0768a1df | Christos Stavrakakis | |
439 | 46ccbcb9 | Christos Stavrakakis | $ snf-manage backend-modify --drained=True <backend_id> |
440 | 0768a1df | Christos Stavrakakis | |
441 | 0768a1df | Christos Stavrakakis | The backend resources are periodically updated, at a period defined by |
442 | a1c707c9 | Constantinos Venetsanopoulos | the ``BACKEND_REFRESH_MIN`` setting, or by running `snf-manage backend-update-status` |
443 | 0768a1df | Christos Stavrakakis | command. It is advised to have a cron job running this command at a smaller |
444 | a1c707c9 | Constantinos Venetsanopoulos | interval than ``BACKEND_REFRESH_MIN`` in order to remove the load of refreshing |
445 | 0768a1df | Christos Stavrakakis | the backends stats from the VM creation phase. |
446 | 0768a1df | Christos Stavrakakis | |
447 | a1c707c9 | Constantinos Venetsanopoulos | Finally, the admin can decide to have a user's VMs being allocated to a |
448 | a1c707c9 | Constantinos Venetsanopoulos | specific backend, with the ``BACKEND_PER_USER`` setting. This is a mapping |
449 | a1c707c9 | Constantinos Venetsanopoulos | between users and backends. If the user is found in ``BACKEND_PER_USER``, then |
450 | 46ccbcb9 | Christos Stavrakakis | Synnefo allocates all his/hers VMs to the specific backend in the variable, |
451 | 46ccbcb9 | Christos Stavrakakis | even if is marked as drained (useful for testing). |
452 | 0768a1df | Christos Stavrakakis | |
453 | 9492387a | Christos Stavrakakis | |
454 | 9492387a | Christos Stavrakakis | Managing Virtual Machines |
455 | 9492387a | Christos Stavrakakis | ~~~~~~~~~~~~~~~~~~~~~~~~~ |
456 | 9492387a | Christos Stavrakakis | |
457 | a1c707c9 | Constantinos Venetsanopoulos | As mentioned, Cyclades uses Ganeti for management of VMs. The administrator can |
458 | a1c707c9 | Constantinos Venetsanopoulos | handle Cyclades VMs just like any other Ganeti instance, via `gnt-instance` |
459 | a1c707c9 | Constantinos Venetsanopoulos | commands. All Ganeti instances that belong to Synnefo, are separated from |
460 | a1c707c9 | Constantinos Venetsanopoulos | others, by a prefix in their names. This prefix is defined in |
461 | a1c707c9 | Constantinos Venetsanopoulos | ``BACKEND_PREFIX_ID`` setting in |
462 | a1c707c9 | Constantinos Venetsanopoulos | ``/etc/synnefo/20-snf-cyclades-app-backend.conf``. |
463 | 9492387a | Christos Stavrakakis | |
464 | a1c707c9 | Constantinos Venetsanopoulos | Apart from handling instances directly in the Ganeti level, a number of `snf-manage` |
465 | 9492387a | Christos Stavrakakis | commands are available: |
466 | 9492387a | Christos Stavrakakis | |
467 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-list``: List servers |
468 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-show``: Show information about a server in the Cyclades DB |
469 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-inspect``: Inspect the state of a server both in DB and Ganeti |
470 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-modify``: Modify the state of a server in the Cycldes DB |
471 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-create``: Create a new server |
472 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage server-import``: Import an existing Ganeti instance to Cyclades |
473 | 9492387a | Christos Stavrakakis | |
474 | 9492387a | Christos Stavrakakis | |
475 | 9492387a | Christos Stavrakakis | Managing Virtual Networks |
476 | 9492387a | Christos Stavrakakis | ~~~~~~~~~~~~~~~~~~~~~~~~~ |
477 | 9492387a | Christos Stavrakakis | |
478 | a1c707c9 | Constantinos Venetsanopoulos | Cyclades is able to create and manage Virtual Networks. Networking is |
479 | a1c707c9 | Constantinos Venetsanopoulos | desployment specific and must be customized based on the specific needs of the |
480 | a1c707c9 | Constantinos Venetsanopoulos | system administrator. For better understanding of networking please refer to |
481 | a1c707c9 | Constantinos Venetsanopoulos | the :ref:`Network <networks>` section. |
482 | a653c059 | Christos Stavrakakis | |
483 | a653c059 | Christos Stavrakakis | Exactly as Cyclades VMs can be handled like Ganeti instances, Cyclades Networks |
484 | a653c059 | Christos Stavrakakis | can also by handled as Ganeti networks, via `gnt-network commands`. All Ganeti |
485 | a1c707c9 | Constantinos Venetsanopoulos | networks that belong to Synnefo are named with the prefix |
486 | a653c059 | Christos Stavrakakis | `${BACKEND_PREFIX_ID}-net-`. |
487 | a653c059 | Christos Stavrakakis | |
488 | a1c707c9 | Constantinos Venetsanopoulos | There are also the following `snf-manage` commands for managing networks: |
489 | a653c059 | Christos Stavrakakis | |
490 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-list``: List networks |
491 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-show``: Show information about a network in the Cyclades DB |
492 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-inspect``: Inspect the state of the network in DB and Ganeti backends |
493 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-modify``: Modify the state of a network in the Cycldes DB |
494 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-create``: Create a new network |
495 | a1c707c9 | Constantinos Venetsanopoulos | * ``snf-manage network-remove``: Remove an existing network |
496 | a653c059 | Christos Stavrakakis | |
497 | 9c98c6fc | Christos Stavrakakis | Managing Network Resources |
498 | a1c707c9 | Constantinos Venetsanopoulos | `````````````````````````` |
499 | 9c98c6fc | Christos Stavrakakis | |
500 | a1c707c9 | Constantinos Venetsanopoulos | Proper operation of the Cyclades Network Service depends on the unique |
501 | a1c707c9 | Constantinos Venetsanopoulos | assignment of specific resources to each type of virtual network. Specifically, |
502 | a1c707c9 | Constantinos Venetsanopoulos | these resources are: |
503 | 9c98c6fc | Christos Stavrakakis | |
504 | a1c707c9 | Constantinos Venetsanopoulos | * IP addresses. Cyclades creates a Pool of IPs for each Network, and assigns a |
505 | a1c707c9 | Constantinos Venetsanopoulos | unique IP address to each VM, thus connecting it to this Network. You can see |
506 | a1c707c9 | Constantinos Venetsanopoulos | the IP pool of each network by running `snf-manage network-inspect |
507 | a1c707c9 | Constantinos Venetsanopoulos | <network_ID>`. IP pools are automatically created and managed by Cyclades, |
508 | a1c707c9 | Constantinos Venetsanopoulos | depending on the subnet of the Network. |
509 | 9492387a | Christos Stavrakakis | * Bridges corresponding to physical VLANs, which are required for networks of |
510 | a1c707c9 | Constantinos Venetsanopoulos | type `PRIVATE_PHYSICAL_VLAN`. |
511 | a1c707c9 | Constantinos Venetsanopoulos | * One Bridge corresponding to one physical VLAN which is required for networks of |
512 | a1c707c9 | Constantinos Venetsanopoulos | type `PRIVATE_MAC_PREFIX`. |
513 | 9c98c6fc | Christos Stavrakakis | |
514 | a1c707c9 | Constantinos Venetsanopoulos | Cyclades allocates those resources from pools that are created by the |
515 | a1c707c9 | Constantinos Venetsanopoulos | administrator with the `snf-manage pool-create` management command. |
516 | 9c98c6fc | Christos Stavrakakis | |
517 | a1c707c9 | Constantinos Venetsanopoulos | Pool Creation |
518 | a1c707c9 | Constantinos Venetsanopoulos | ````````````` |
519 | a1c707c9 | Constantinos Venetsanopoulos | Pools are created using the `snf-manage pool-create` command: |
520 | 90c74332 | Christos Stavrakakis | |
521 | 9c98c6fc | Christos Stavrakakis | .. code-block:: console |
522 | 9c98c6fc | Christos Stavrakakis | |
523 | 9c98c6fc | Christos Stavrakakis | # snf-manage pool-create --type=bridge --base=prv --size=20 |
524 | 9c98c6fc | Christos Stavrakakis | |
525 | 9c98c6fc | Christos Stavrakakis | will create a pool of bridges, containing bridges prv1, prv2,..prv21. |
526 | 9c98c6fc | Christos Stavrakakis | |
527 | 9c98c6fc | Christos Stavrakakis | You can verify the creation of the pool, and check its contents by running: |
528 | 90c74332 | Christos Stavrakakis | |
529 | 9c98c6fc | Christos Stavrakakis | .. code-block:: console |
530 | 9c98c6fc | Christos Stavrakakis | |
531 | 9c98c6fc | Christos Stavrakakis | # snf-manage pool-list |
532 | 9c98c6fc | Christos Stavrakakis | # snf-manage pool-show --type=bridge 1 |
533 | 9c98c6fc | Christos Stavrakakis | |
534 | a1c707c9 | Constantinos Venetsanopoulos | With the same commands you can handle a pool of MAC prefixes. For example: |
535 | 90c74332 | Christos Stavrakakis | |
536 | 9c98c6fc | Christos Stavrakakis | .. code-block:: console |
537 | 9c98c6fc | Christos Stavrakakis | |
538 | 9c98c6fc | Christos Stavrakakis | # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536 |
539 | 9c98c6fc | Christos Stavrakakis | |
540 | a1c707c9 | Constantinos Venetsanopoulos | will create a pool of MAC prefixes from ``aa:00:1`` to ``b9:ff:f``. The MAC |
541 | a1c707c9 | Constantinos Venetsanopoulos | prefix pool is responsible for providing only unicast and locally administered |
542 | a1c707c9 | Constantinos Venetsanopoulos | MAC addresses, so many of these prefixes will be externally reserved, to |
543 | a1c707c9 | Constantinos Venetsanopoulos | exclude from allocation. |
544 | 0768a1df | Christos Stavrakakis | |
545 | eab73849 | Constantinos Venetsanopoulos | Cyclades advanced operations |
546 | eab73849 | Constantinos Venetsanopoulos | ---------------------------- |
547 | eab73849 | Constantinos Venetsanopoulos | |
548 | eab73849 | Constantinos Venetsanopoulos | Reconciliation mechanism |
549 | eab73849 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~~~~~~~ |
550 | a1c707c9 | Constantinos Venetsanopoulos | |
551 | 06e23415 | Christos Stavrakakis | On certain occasions, such as a Ganeti or RabbitMQ failure, the state of |
552 | 06e23415 | Christos Stavrakakis | Cyclades database may differ from the real state of VMs and networks in the |
553 | 06e23415 | Christos Stavrakakis | Ganeti backends. The reconciliation process is designed to synchronize |
554 | 06e23415 | Christos Stavrakakis | the state of the Cyclades DB with Ganeti. There are two management commands |
555 | 06e23415 | Christos Stavrakakis | for reconciling VMs and Networks |
556 | 06e23415 | Christos Stavrakakis | |
557 | a1c707c9 | Constantinos Venetsanopoulos | Reconciling Virtual Machines |
558 | a1c707c9 | Constantinos Venetsanopoulos | ```````````````````````````` |
559 | a1c707c9 | Constantinos Venetsanopoulos | |
560 | 06e23415 | Christos Stavrakakis | Reconciliation of VMs detects the following conditions: |
561 | a1c707c9 | Constantinos Venetsanopoulos | |
562 | eab73849 | Constantinos Venetsanopoulos | * Stale DB servers without corresponding Ganeti instances |
563 | eab73849 | Constantinos Venetsanopoulos | * Orphan Ganeti instances, without corresponding DB entries |
564 | 06e23415 | Christos Stavrakakis | * Out-of-sync state for DB entries wrt to Ganeti instances |
565 | eab73849 | Constantinos Venetsanopoulos | |
566 | 06e23415 | Christos Stavrakakis | To detect all inconsistencies you can just run: |
567 | 90c74332 | Christos Stavrakakis | |
568 | eab73849 | Constantinos Venetsanopoulos | .. code-block:: console |
569 | a1c707c9 | Constantinos Venetsanopoulos | |
570 | a1c707c9 | Constantinos Venetsanopoulos | $ snf-manage reconcile-servers |
571 | eab73849 | Constantinos Venetsanopoulos | |
572 | 06e23415 | Christos Stavrakakis | Adding the `--fix-all` option, will do the actual synchronization: |
573 | 90c74332 | Christos Stavrakakis | |
574 | 06e23415 | Christos Stavrakakis | .. code-block:: console |
575 | a1c707c9 | Constantinos Venetsanopoulos | |
576 | a1c707c9 | Constantinos Venetsanopoulos | $ snf-manage reconcile --fix-all |
577 | eab73849 | Constantinos Venetsanopoulos | |
578 | eab73849 | Constantinos Venetsanopoulos | Please see ``snf-manage reconcile --help`` for all the details. |
579 | eab73849 | Constantinos Venetsanopoulos | |
580 | eab73849 | Constantinos Venetsanopoulos | |
581 | 06e23415 | Christos Stavrakakis | Reconciling Networks |
582 | 90c74332 | Christos Stavrakakis | ```````````````````` |
583 | a1c707c9 | Constantinos Venetsanopoulos | |
584 | 06e23415 | Christos Stavrakakis | Reconciliation of Networks detects the following conditions: |
585 | a1c707c9 | Constantinos Venetsanopoulos | |
586 | 06e23415 | Christos Stavrakakis | * Stale DB networks without corresponding Ganeti networks |
587 | 06e23415 | Christos Stavrakakis | * Orphan Ganeti networks, without corresponding DB entries |
588 | 06e23415 | Christos Stavrakakis | * Private networks that are not created to all Ganeti backends |
589 | 06e23415 | Christos Stavrakakis | * Unsynchronized IP pools |
590 | 06e23415 | Christos Stavrakakis | |
591 | 06e23415 | Christos Stavrakakis | To detect all inconsistencies you can just run: |
592 | 90c74332 | Christos Stavrakakis | |
593 | 06e23415 | Christos Stavrakakis | .. code-block:: console |
594 | a1c707c9 | Constantinos Venetsanopoulos | |
595 | 06e23415 | Christos Stavrakakis | $ snf-manage reconcile-networks |
596 | 06e23415 | Christos Stavrakakis | |
597 | 06e23415 | Christos Stavrakakis | Adding the `--fix-all` option, will do the actual synchronization: |
598 | 90c74332 | Christos Stavrakakis | |
599 | 06e23415 | Christos Stavrakakis | .. code-block:: console |
600 | a1c707c9 | Constantinos Venetsanopoulos | |
601 | 06e23415 | Christos Stavrakakis | $ snf-manage reconcile-networks --fix-all |
602 | 06e23415 | Christos Stavrakakis | |
603 | 06e23415 | Christos Stavrakakis | Please see ``snf-manage reconcile-networks --help`` for all the details. |
604 | eab73849 | Constantinos Venetsanopoulos | |
605 | eab73849 | Constantinos Venetsanopoulos | |
606 | a1c707c9 | Constantinos Venetsanopoulos | |
607 | eab73849 | Constantinos Venetsanopoulos | Block Storage Service (Archipelago) |
608 | eab73849 | Constantinos Venetsanopoulos | =================================== |
609 | eab73849 | Constantinos Venetsanopoulos | |
610 | eab73849 | Constantinos Venetsanopoulos | Overview |
611 | eab73849 | Constantinos Venetsanopoulos | -------- |
612 | 5226b38f | Filippos Giannakos | Archipelago offers Copy-On-Write snapshotable volumes. Pithos images can be used |
613 | 5226b38f | Filippos Giannakos | to provision a volume with Copy-On-Write semantics (i.e. a clone). Snapshots |
614 | 5226b38f | Filippos Giannakos | offer a unique deduplicated image of a volume, that reflects the volume state |
615 | 5226b38f | Filippos Giannakos | during snapshot creation and are indistinguishable from a Pithos image. |
616 | 5226b38f | Filippos Giannakos | |
617 | 5226b38f | Filippos Giannakos | Archipelago is used by Cyclades and Ganeti for fast provisioning of VMs based on |
618 | 4bb29566 | Filippos Giannakos | CoW volumes. Moreover, it enables live migration of thinly-provisioned VMs with |
619 | 4bb29566 | Filippos Giannakos | no physically shared storage. |
620 | eab73849 | Constantinos Venetsanopoulos | |
621 | 301294a9 | Constantinos Venetsanopoulos | Archipelago Architecture |
622 | 301294a9 | Constantinos Venetsanopoulos | ------------------------ |
623 | 301294a9 | Constantinos Venetsanopoulos | |
624 | e3f3abdc | Filippos Giannakos | .. image:: images/archipelago-architecture.png |
625 | e3f3abdc | Filippos Giannakos | :width: 50% |
626 | e3f3abdc | Filippos Giannakos | :target: _images/archipelago-architecture.png |
627 | eab73849 | Constantinos Venetsanopoulos | |
628 | 301294a9 | Constantinos Venetsanopoulos | .. _syn+archip+rados: |
629 | 301294a9 | Constantinos Venetsanopoulos | |
630 | 301294a9 | Constantinos Venetsanopoulos | Overview of Synnefo + Archipelago + RADOS |
631 | 301294a9 | Constantinos Venetsanopoulos | ----------------------------------------- |
632 | 301294a9 | Constantinos Venetsanopoulos | |
633 | 301294a9 | Constantinos Venetsanopoulos | .. image:: images/synnefo-arch3.png |
634 | 301294a9 | Constantinos Venetsanopoulos | :width: 100% |
635 | 301294a9 | Constantinos Venetsanopoulos | :target: _images/synnefo-arch3.png |
636 | 301294a9 | Constantinos Venetsanopoulos | |
637 | eab73849 | Constantinos Venetsanopoulos | Prereqs |
638 | eab73849 | Constantinos Venetsanopoulos | ------- |
639 | 301294a9 | Constantinos Venetsanopoulos | |
640 | 902d661c | Filippos Giannakos | The administrator must initialize the storage backend where archipelago volume |
641 | 902d661c | Filippos Giannakos | blocks will reside. |
642 | 902d661c | Filippos Giannakos | |
643 | 902d661c | Filippos Giannakos | In case of a files backend, the administrator must create two directories. One |
644 | 902d661c | Filippos Giannakos | for the archipelago data blocks and one for the archipelago map blocks. These |
645 | 902d661c | Filippos Giannakos | should probably be over shared storage to enable sharing archipelago volumes |
646 | 902d661c | Filippos Giannakos | between multiple nodes. He or she, must also be able to supply a directory where |
647 | 902d661c | Filippos Giannakos | the pithos data and map blocks reside. |
648 | 902d661c | Filippos Giannakos | |
649 | 902d661c | Filippos Giannakos | In case of a RADOS backend, the administrator must create two rados pools, one |
650 | 902d661c | Filippos Giannakos | for data blocks, and one for the map blocks. These pools, must be the same pools |
651 | 902d661c | Filippos Giannakos | used in pithos, in order to enable volume creation based on pithos images. |
652 | 902d661c | Filippos Giannakos | |
653 | eab73849 | Constantinos Venetsanopoulos | Installation |
654 | eab73849 | Constantinos Venetsanopoulos | ------------ |
655 | 301294a9 | Constantinos Venetsanopoulos | |
656 | 902d661c | Filippos Giannakos | Archipelago consists of |
657 | 902d661c | Filippos Giannakos | |
658 | 902d661c | Filippos Giannakos | * ``libxseg0``: libxseg used to communicate over shared memory segments |
659 | 5226b38f | Filippos Giannakos | * ``python-xseg``: python bindings for libxseg |
660 | 902d661c | Filippos Giannakos | * ``archipelago-kernel-dkms``: contains archipelago kernel modules to provide |
661 | 902d661c | Filippos Giannakos | block devices to be used as vm disks |
662 | 5226b38f | Filippos Giannakos | * ``python-archipelago``: archipelago python module. Includes archipelago and |
663 | 5226b38f | Filippos Giannakos | vlmc functionality. |
664 | 902d661c | Filippos Giannakos | * ``archipelago``: user space tools and peers for the archipelago management and |
665 | 902d661c | Filippos Giannakos | volume composition |
666 | 902d661c | Filippos Giannakos | * ``archipelago-ganeti``: ganeti ext storage scripts, that enable ganeti to |
667 | 902d661c | Filippos Giannakos | provision VMs over archipelago |
668 | 902d661c | Filippos Giannakos | |
669 | 902d661c | Filippos Giannakos | Performing |
670 | 902d661c | Filippos Giannakos | |
671 | 902d661c | Filippos Giannakos | .. code-block:: console |
672 | 902d661c | Filippos Giannakos | |
673 | 902d661c | Filippos Giannakos | $ apt-get install archipelago-ganeti |
674 | 902d661c | Filippos Giannakos | |
675 | 902d661c | Filippos Giannakos | should fetch all the required packages and get you up 'n going with archipelago |
676 | 902d661c | Filippos Giannakos | |
677 | 902d661c | Filippos Giannakos | Bare in mind, that custom librados is required, which is provided in the apt |
678 | 5226b38f | Filippos Giannakos | repo of GRNet. |
679 | 902d661c | Filippos Giannakos | |
680 | 902d661c | Filippos Giannakos | |
681 | 5226b38f | Filippos Giannakos | For now, librados is a dependency of archipelago, even if you do not intend to |
682 | 5226b38f | Filippos Giannakos | use archipelago over RADOS. |
683 | eab73849 | Constantinos Venetsanopoulos | |
684 | eab73849 | Constantinos Venetsanopoulos | Configuration |
685 | eab73849 | Constantinos Venetsanopoulos | ------------- |
686 | 902d661c | Filippos Giannakos | Archipelago should work out of the box with a RADOS backend, but basic |
687 | 902d661c | Filippos Giannakos | configuration can be done in ``/etc/default/archipelago`` . |
688 | 902d661c | Filippos Giannakos | |
689 | 902d661c | Filippos Giannakos | If you wish to change the storage backend to files, set |
690 | 902d661c | Filippos Giannakos | |
691 | 902d661c | Filippos Giannakos | .. code-block:: console |
692 | 902d661c | Filippos Giannakos | |
693 | 902d661c | Filippos Giannakos | STORAGE="files" |
694 | 902d661c | Filippos Giannakos | |
695 | 902d661c | Filippos Giannakos | and provide the appropriate settings for files storage backend in the conf file. |
696 | 902d661c | Filippos Giannakos | |
697 | 902d661c | Filippos Giannakos | These are: |
698 | 902d661c | Filippos Giannakos | |
699 | 902d661c | Filippos Giannakos | * ``FILED_IMAGES``: directory for archipelago data blocks. |
700 | 902d661c | Filippos Giannakos | * ``FILED_MAPS``: directory for archipelago map blocks. |
701 | 902d661c | Filippos Giannakos | * ``PITHOS``: directory of pithos data blocks. |
702 | 902d661c | Filippos Giannakos | * ``PITHOSMAPS``: directory of pithos map blocks. |
703 | 902d661c | Filippos Giannakos | |
704 | 902d661c | Filippos Giannakos | The settings for RADOS storage backend are: |
705 | 902d661c | Filippos Giannakos | |
706 | 902d661c | Filippos Giannakos | * ``RADOS_POOL_MAPS``: The pool where archipelago and pithos map blocks reside. |
707 | 902d661c | Filippos Giannakos | * ``RADOS_POOL_BLOCKS``: The pool where archipelago and pithos data blocks |
708 | 5226b38f | Filippos Giannakos | reside. |
709 | 902d661c | Filippos Giannakos | |
710 | 902d661c | Filippos Giannakos | Examples can be found in the conf file. |
711 | 902d661c | Filippos Giannakos | |
712 | 902d661c | Filippos Giannakos | Be aware that archipelago infrastructure doesn't provide default values for this |
713 | 902d661c | Filippos Giannakos | settings. If they are not set in the conf file, archipelago will not be able to |
714 | 902d661c | Filippos Giannakos | function. |
715 | 902d661c | Filippos Giannakos | |
716 | 902d661c | Filippos Giannakos | Archipelago also provides ``VERBOSITY`` config options to control the output |
717 | 902d661c | Filippos Giannakos | generated by the userspace peers. |
718 | 902d661c | Filippos Giannakos | |
719 | 902d661c | Filippos Giannakos | The available options are: |
720 | 902d661c | Filippos Giannakos | |
721 | 902d661c | Filippos Giannakos | * ``VERBOSITY_BLOCKERB`` |
722 | 902d661c | Filippos Giannakos | * ``VERBOSITY_BLOCKERM`` |
723 | 902d661c | Filippos Giannakos | * ``VERBOSITY_MAPPER`` |
724 | 902d661c | Filippos Giannakos | * ``VERBOSITY_VLMC`` |
725 | 902d661c | Filippos Giannakos | |
726 | 902d661c | Filippos Giannakos | and the available values are: |
727 | 902d661c | Filippos Giannakos | |
728 | 902d661c | Filippos Giannakos | * 0 : Error only logging. |
729 | 902d661c | Filippos Giannakos | * 1 : Warning logging. |
730 | 902d661c | Filippos Giannakos | * 2 : Info logging. |
731 | 902d661c | Filippos Giannakos | * 3 : Debug logging. WARNING: This options produces tons of output, but the |
732 | 902d661c | Filippos Giannakos | logrotate daemon should take care of it. |
733 | eab73849 | Constantinos Venetsanopoulos | |
734 | eab73849 | Constantinos Venetsanopoulos | Working with Archipelago |
735 | eab73849 | Constantinos Venetsanopoulos | ------------------------ |
736 | eab73849 | Constantinos Venetsanopoulos | |
737 | 902d661c | Filippos Giannakos | ``archipelago`` provides basic functionality for archipelago. |
738 | 5226b38f | Filippos Giannakos | |
739 | 5226b38f | Filippos Giannakos | Usage: |
740 | 5226b38f | Filippos Giannakos | |
741 | 5226b38f | Filippos Giannakos | .. code-block:: console |
742 | 5226b38f | Filippos Giannakos | |
743 | 5226b38f | Filippos Giannakos | $ archipelago [-u] command |
744 | 5226b38f | Filippos Giannakos | |
745 | 5226b38f | Filippos Giannakos | |
746 | 902d661c | Filippos Giannakos | Currently it supports the following commands: |
747 | 902d661c | Filippos Giannakos | |
748 | 5226b38f | Filippos Giannakos | * ``start [peer]`` |
749 | 5226b38f | Filippos Giannakos | Starts archipelago or the specified peer. |
750 | 5226b38f | Filippos Giannakos | * ``stop [peer]`` |
751 | 5226b38f | Filippos Giannakos | Stops archipelago or the specified peer. |
752 | 5226b38f | Filippos Giannakos | * ``restart [peer]`` |
753 | 5226b38f | Filippos Giannakos | Restarts archipelago or the specified peer. |
754 | 902d661c | Filippos Giannakos | * ``status`` |
755 | 5226b38f | Filippos Giannakos | Show the status of archipelago. |
756 | 5226b38f | Filippos Giannakos | |
757 | 5226b38f | Filippos Giannakos | Available peers: ``blockerm``, ``blockerb``, ``mapperd``, ``vlmcd``. |
758 | 5226b38f | Filippos Giannakos | |
759 | 902d661c | Filippos Giannakos | |
760 | 902d661c | Filippos Giannakos | ``start``, ``stop``, ``restart`` can be combined with the ``-u / --user`` option |
761 | 902d661c | Filippos Giannakos | to affect only the userspace peers supporting archipelago. |
762 | 902d661c | Filippos Giannakos | |
763 | 902d661c | Filippos Giannakos | |
764 | 902d661c | Filippos Giannakos | |
765 | eab73849 | Constantinos Venetsanopoulos | Archipelago advanced operations |
766 | eab73849 | Constantinos Venetsanopoulos | ------------------------------- |
767 | 902d661c | Filippos Giannakos | The ``vlmc`` tool provides a way to interact with archipelago volumes |
768 | 902d661c | Filippos Giannakos | |
769 | 902d661c | Filippos Giannakos | * ``vlmc map <volumename>``: maps the volume to a xsegbd device. |
770 | 6c900328 | Filippos Giannakos | |
771 | 5226b38f | Filippos Giannakos | * ``vlmc unmap </dev/xsegbd[1-..]>``: unmaps the specified device from the |
772 | 902d661c | Filippos Giannakos | system. |
773 | 6c900328 | Filippos Giannakos | |
774 | 902d661c | Filippos Giannakos | * ``vlmc create <volumename> --snap <snapname> --size <size>``: creates a new |
775 | 902d661c | Filippos Giannakos | volume named <volumename> from snapshot name <snapname> with size <size>. |
776 | 301294a9 | Constantinos Venetsanopoulos | The ``--snap`` and ``--size`` are optional, but at least one of them is |
777 | 301294a9 | Constantinos Venetsanopoulos | mandatory. e.g: |
778 | 902d661c | Filippos Giannakos | |
779 | 301294a9 | Constantinos Venetsanopoulos | ``vlmc create <volumename> --snap <snapname>`` creates a volume named |
780 | 301294a9 | Constantinos Venetsanopoulos | volumename from snapshot snapname. The size of the volume is the same as |
781 | 301294a9 | Constantinos Venetsanopoulos | the size of the snapshot. |
782 | 902d661c | Filippos Giannakos | |
783 | 301294a9 | Constantinos Venetsanopoulos | ``vlmc create <volumename> --size <size>`` creates an empty volume of size |
784 | 301294a9 | Constantinos Venetsanopoulos | <size> named <volumename>. |
785 | 6c900328 | Filippos Giannakos | |
786 | 902d661c | Filippos Giannakos | * ``vlmc remove <volumename>``: removes the volume and all the related |
787 | 902d661c | Filippos Giannakos | archipelago blocks from storage. |
788 | 6c900328 | Filippos Giannakos | |
789 | 902d661c | Filippos Giannakos | * ``vlmc list``: provides a list of archipelago volumes. Currently only works |
790 | 902d661c | Filippos Giannakos | with RADOS storage backend. |
791 | f846d8df | Constantinos Venetsanopoulos | |
792 | 5226b38f | Filippos Giannakos | * ``vlmc info <volumename>``: shows volume information. Currently returns only |
793 | 5226b38f | Filippos Giannakos | volume size. |
794 | 5226b38f | Filippos Giannakos | |
795 | 6c900328 | Filippos Giannakos | * ``vlmc open <volumename>``: opens an archipelago volume. That is, taking all |
796 | 6c900328 | Filippos Giannakos | the necessary locks and also make the rest of the infrastructure aware of the |
797 | 6c900328 | Filippos Giannakos | operation. |
798 | 6c900328 | Filippos Giannakos | |
799 | 6c900328 | Filippos Giannakos | This operation succeeds if the volume is alread opened. |
800 | 6c900328 | Filippos Giannakos | |
801 | 6c900328 | Filippos Giannakos | * ``vlmc close <volumename>``: closes an archipelago volume. That is, performing |
802 | 6c900328 | Filippos Giannakos | all the necessary functions in the insfrastrure to successfully release the |
803 | 6c900328 | Filippos Giannakos | volume. Also releases all the acquired locks. |
804 | 6c900328 | Filippos Giannakos | |
805 | 6c900328 | Filippos Giannakos | ``vlmc close`` should be performed after a ``vlmc open`` operation. |
806 | 6c900328 | Filippos Giannakos | |
807 | 6c900328 | Filippos Giannakos | * ``vlmc lock <volumename>``: locks a volume. This step allow the administrator |
808 | 6c900328 | Filippos Giannakos | to lock an archipelago volume, independently from the rest of the |
809 | 5226b38f | Filippos Giannakos | infrastrure. |
810 | 6c900328 | Filippos Giannakos | |
811 | 6c900328 | Filippos Giannakos | * ``vlmc unlock [-f] <volumename>``: unlocks a volume. This allow the |
812 | 6c900328 | Filippos Giannakos | administrator to unlock a volume, independently from the rest of the |
813 | 6c900328 | Filippos Giannakos | infrastructure. |
814 | 6c900328 | Filippos Giannakos | The unlock option can be performed only by the blocker that acquired the lock |
815 | 6c900328 | Filippos Giannakos | in the first place. To unlock a volume from another blocker, ``-f`` option |
816 | 6c900328 | Filippos Giannakos | must be used to break the lock. |
817 | 6c900328 | Filippos Giannakos | |
818 | a1c707c9 | Constantinos Venetsanopoulos | |
819 | bc055d09 | Constantinos Venetsanopoulos | The "kamaki" API client |
820 | eab73849 | Constantinos Venetsanopoulos | ======================= |
821 | bc055d09 | Constantinos Venetsanopoulos | |
822 | bc055d09 | Constantinos Venetsanopoulos | To upload, register or modify an image you will need the **kamaki** tool. |
823 | bc055d09 | Constantinos Venetsanopoulos | Before proceeding make sure that it is configured properly. Verify that |
824 | bc055d09 | Constantinos Venetsanopoulos | *image_url*, *storage_url*, and *token* are set as needed: |
825 | bc055d09 | Constantinos Venetsanopoulos | |
826 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
827 | bc055d09 | Constantinos Venetsanopoulos | |
828 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki config list |
829 | bc055d09 | Constantinos Venetsanopoulos | |
830 | bc055d09 | Constantinos Venetsanopoulos | To chage a setting use ``kamaki config set``: |
831 | bc055d09 | Constantinos Venetsanopoulos | |
832 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
833 | bc055d09 | Constantinos Venetsanopoulos | |
834 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki config set image_url https://cyclades.example.com/plankton |
835 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki config set storage_url https://pithos.example.com/v1 |
836 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki config set token ... |
837 | bc055d09 | Constantinos Venetsanopoulos | |
838 | bc055d09 | Constantinos Venetsanopoulos | Upload Image |
839 | bc055d09 | Constantinos Venetsanopoulos | ------------ |
840 | bc055d09 | Constantinos Venetsanopoulos | |
841 | bc055d09 | Constantinos Venetsanopoulos | As a shortcut, you can configure a default account and container that will be |
842 | bc055d09 | Constantinos Venetsanopoulos | used by the ``kamaki store`` commands: |
843 | bc055d09 | Constantinos Venetsanopoulos | |
844 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
845 | bc055d09 | Constantinos Venetsanopoulos | |
846 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki config set storage_account images@example.com |
847 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki config set storage_container images |
848 | bc055d09 | Constantinos Venetsanopoulos | |
849 | bc055d09 | Constantinos Venetsanopoulos | If the container does not exist, you will have to create it before uploading |
850 | bc055d09 | Constantinos Venetsanopoulos | any images: |
851 | bc055d09 | Constantinos Venetsanopoulos | |
852 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
853 | bc055d09 | Constantinos Venetsanopoulos | |
854 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki store create images |
855 | bc055d09 | Constantinos Venetsanopoulos | |
856 | bc055d09 | Constantinos Venetsanopoulos | You are now ready to upload an image. You can upload it with a Pithos+ client, |
857 | bc055d09 | Constantinos Venetsanopoulos | or use kamaki directly: |
858 | bc055d09 | Constantinos Venetsanopoulos | |
859 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
860 | bc055d09 | Constantinos Venetsanopoulos | |
861 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki store upload ubuntu.iso |
862 | bc055d09 | Constantinos Venetsanopoulos | |
863 | bc055d09 | Constantinos Venetsanopoulos | You can use any Pithos+ client to verify that the image was uploaded correctly. |
864 | bc055d09 | Constantinos Venetsanopoulos | The full Pithos URL for the previous example will be |
865 | bc055d09 | Constantinos Venetsanopoulos | ``pithos://images@example.com/images/ubuntu.iso``. |
866 | bc055d09 | Constantinos Venetsanopoulos | |
867 | bc055d09 | Constantinos Venetsanopoulos | |
868 | bc055d09 | Constantinos Venetsanopoulos | Register Image |
869 | bc055d09 | Constantinos Venetsanopoulos | -------------- |
870 | bc055d09 | Constantinos Venetsanopoulos | |
871 | bc055d09 | Constantinos Venetsanopoulos | To register an image you will need to use the full Pithos+ URL. To register as |
872 | bc055d09 | Constantinos Venetsanopoulos | a public image the one from the previous example use: |
873 | bc055d09 | Constantinos Venetsanopoulos | |
874 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
875 | bc055d09 | Constantinos Venetsanopoulos | |
876 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki glance register Ubuntu pithos://images@example.com/images/ubuntu.iso --public |
877 | bc055d09 | Constantinos Venetsanopoulos | |
878 | bc055d09 | Constantinos Venetsanopoulos | The ``--public`` flag is important, if missing the registered image will not |
879 | bc055d09 | Constantinos Venetsanopoulos | be listed by ``kamaki glance list``. |
880 | bc055d09 | Constantinos Venetsanopoulos | |
881 | bc055d09 | Constantinos Venetsanopoulos | Use ``kamaki glance register`` with no arguments to see a list of available |
882 | bc055d09 | Constantinos Venetsanopoulos | options. A more complete example would be the following: |
883 | bc055d09 | Constantinos Venetsanopoulos | |
884 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
885 | bc055d09 | Constantinos Venetsanopoulos | |
886 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki glance register Ubuntu pithos://images@example.com/images/ubuntu.iso \ |
887 | bc055d09 | Constantinos Venetsanopoulos | --public --disk-format diskdump --property kernel=3.1.2 |
888 | bc055d09 | Constantinos Venetsanopoulos | |
889 | bc055d09 | Constantinos Venetsanopoulos | To verify that the image was registered successfully use: |
890 | bc055d09 | Constantinos Venetsanopoulos | |
891 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
892 | bc055d09 | Constantinos Venetsanopoulos | |
893 | bc055d09 | Constantinos Venetsanopoulos | $ kamaki glance list -l |
894 | bc055d09 | Constantinos Venetsanopoulos | |
895 | bc055d09 | Constantinos Venetsanopoulos | |
896 | eab73849 | Constantinos Venetsanopoulos | |
897 | eab73849 | Constantinos Venetsanopoulos | Miscellaneous |
898 | eab73849 | Constantinos Venetsanopoulos | ============= |
899 | eab73849 | Constantinos Venetsanopoulos | |
900 | 9b659162 | Christos Stavrakakis | .. RabbitMQ |
901 | a1c707c9 | Constantinos Venetsanopoulos | |
902 | 9b659162 | Christos Stavrakakis | RabbitMQ Broker |
903 | 9b659162 | Christos Stavrakakis | --------------- |
904 | 9b659162 | Christos Stavrakakis | |
905 | 9b659162 | Christos Stavrakakis | Queue nodes run the RabbitMQ sofware, which provides AMQP functionality. To |
906 | 9b659162 | Christos Stavrakakis | guarantee high-availability, more than one Queue nodes should be deployed, each |
907 | 9b659162 | Christos Stavrakakis | of them belonging to the same `RabbitMQ cluster |
908 | a1c707c9 | Constantinos Venetsanopoulos | <http://www.rabbitmq.com/clustering.html>`_. Synnefo uses the RabbitMQ |
909 | a1c707c9 | Constantinos Venetsanopoulos | active/active `High Available Queues <http://www.rabbitmq.com/ha.html>`_ which |
910 | a1c707c9 | Constantinos Venetsanopoulos | are mirrored between two nodes within a RabbitMQ cluster. |
911 | a1c707c9 | Constantinos Venetsanopoulos | |
912 | a1c707c9 | Constantinos Venetsanopoulos | The RabbitMQ nodes that form the cluster, are declared to Synnefo through the |
913 | a1c707c9 | Constantinos Venetsanopoulos | `AMQP_HOSTS` setting. Each time a Synnefo component needs to connect to |
914 | a1c707c9 | Constantinos Venetsanopoulos | RabbitMQ, one of these nodes is chosen in a random way. The client that Synnefo |
915 | a1c707c9 | Constantinos Venetsanopoulos | uses to connect to RabbitMQ, handles connection failures transparently and |
916 | a1c707c9 | Constantinos Venetsanopoulos | tries to reconnect to a different node. As long as one of these nodes are up |
917 | a1c707c9 | Constantinos Venetsanopoulos | and running, functionality of Synnefo should not be downgraded by the RabbitMQ |
918 | a1c707c9 | Constantinos Venetsanopoulos | node failures. |
919 | 9b659162 | Christos Stavrakakis | |
920 | 9b659162 | Christos Stavrakakis | All the queues that are being used are declared as durable, meaning that |
921 | a1c707c9 | Constantinos Venetsanopoulos | messages are persistently stored to RabbitMQ, until they get successfully |
922 | a1c707c9 | Constantinos Venetsanopoulos | processed by a client. |
923 | 9b659162 | Christos Stavrakakis | |
924 | 9b659162 | Christos Stavrakakis | Currently, RabbitMQ is used by the following components: |
925 | 9b659162 | Christos Stavrakakis | |
926 | a1c707c9 | Constantinos Venetsanopoulos | * `snf-ganeti-eventd`, `snf-ganeti-hook` and `snf-progress-monitor`: |
927 | 9b659162 | Christos Stavrakakis | These components send messages concerning the status and progress of |
928 | 9b659162 | Christos Stavrakakis | jobs in the Ganeti backend. |
929 | a1c707c9 | Constantinos Venetsanopoulos | * `snf-dispatcher`: This daemon, consumes the messages that are sent from |
930 | a1c707c9 | Constantinos Venetsanopoulos | the above components, and updates the Cyclades DB accordingly. |
931 | 9b659162 | Christos Stavrakakis | |
932 | e4404297 | Christos Stavrakakis | |
933 | 9b659162 | Christos Stavrakakis | Installation |
934 | 301294a9 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~ |
935 | 301294a9 | Constantinos Venetsanopoulos | |
936 | a1c707c9 | Constantinos Venetsanopoulos | Please check the RabbitMQ documentation which covers extensively the |
937 | a1c707c9 | Constantinos Venetsanopoulos | `installation of RabbitMQ server <http://www.rabbitmq.com/download.html>`_ and |
938 | a1c707c9 | Constantinos Venetsanopoulos | the setup of a `RabbitMQ cluster <http://www.rabbitmq.com/clustering.html>`_. |
939 | a1c707c9 | Constantinos Venetsanopoulos | Also, check out the `web management plugin |
940 | a1c707c9 | Constantinos Venetsanopoulos | <http://www.rabbitmq.com/management.html>`_ that can be useful for managing and |
941 | a1c707c9 | Constantinos Venetsanopoulos | monitoring RabbitMQ. |
942 | 9b659162 | Christos Stavrakakis | |
943 | 9b659162 | Christos Stavrakakis | For a basic installation of RabbitMQ on two nodes (node1 and node2) you can do |
944 | a1c707c9 | Constantinos Venetsanopoulos | the following: |
945 | 9b659162 | Christos Stavrakakis | |
946 | a1c707c9 | Constantinos Venetsanopoulos | On both nodes, install rabbitmq-server and create a Synnefo user: |
947 | 9b659162 | Christos Stavrakakis | |
948 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
949 | 9b659162 | Christos Stavrakakis | |
950 | 9b659162 | Christos Stavrakakis | $ apt-get install rabbitmq-server |
951 | 9b659162 | Christos Stavrakakis | $ rabbitmqctl add_user synnefo "example_pass" |
952 | 9b659162 | Christos Stavrakakis | $ rabbitmqctl set_permissions synnefo ".*" ".*" ".*" |
953 | 9b659162 | Christos Stavrakakis | |
954 | a1c707c9 | Constantinos Venetsanopoulos | Also guarantee that both nodes share the same cookie, by running: |
955 | 9b659162 | Christos Stavrakakis | |
956 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
957 | 9b659162 | Christos Stavrakakis | |
958 | 9b659162 | Christos Stavrakakis | $ scp node1:/var/lib/rabbitmq/.erlang.cookie node2:/var/lib/rabbitmq/.erlang.cookie |
959 | 9b659162 | Christos Stavrakakis | |
960 | 9b659162 | Christos Stavrakakis | and restart the nodes: |
961 | 9b659162 | Christos Stavrakakis | |
962 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
963 | 9b659162 | Christos Stavrakakis | |
964 | 9b659162 | Christos Stavrakakis | $ /etc/init.d/rabbitmq-server restart |
965 | 9b659162 | Christos Stavrakakis | |
966 | 9b659162 | Christos Stavrakakis | |
967 | 9b659162 | Christos Stavrakakis | To setup the RabbitMQ cluster run: |
968 | 9b659162 | Christos Stavrakakis | |
969 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
970 | 9b659162 | Christos Stavrakakis | |
971 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl stop_app |
972 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl reset |
973 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl cluster rabbit@node1 rabbit@node2 |
974 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl start_app |
975 | 9b659162 | Christos Stavrakakis | |
976 | 9b659162 | Christos Stavrakakis | You can verify that the cluster is set up correctly by running: |
977 | 9b659162 | Christos Stavrakakis | |
978 | 9b659162 | Christos Stavrakakis | .. code-block:: console |
979 | 9b659162 | Christos Stavrakakis | |
980 | 9b659162 | Christos Stavrakakis | root@node2: rabbitmqctl cluster_status |
981 | 9b659162 | Christos Stavrakakis | |
982 | 9b659162 | Christos Stavrakakis | |
983 | 9b659162 | Christos Stavrakakis | |
984 | 9b659162 | Christos Stavrakakis | |
985 | 9b659162 | Christos Stavrakakis | |
986 | bc055d09 | Constantinos Venetsanopoulos | Admin tool: snf-manage |
987 | bc055d09 | Constantinos Venetsanopoulos | ---------------------- |
988 | bc055d09 | Constantinos Venetsanopoulos | |
989 | bc055d09 | Constantinos Venetsanopoulos | ``snf-manage`` is a tool used to perform various administrative tasks. It needs |
990 | bc055d09 | Constantinos Venetsanopoulos | to be able to access the django database, so the following should be able to |
991 | bc055d09 | Constantinos Venetsanopoulos | import the Django settings. |
992 | bc055d09 | Constantinos Venetsanopoulos | |
993 | bc055d09 | Constantinos Venetsanopoulos | Additionally, administrative tasks can be performed via the admin web interface |
994 | e18edd26 | Constantinos Venetsanopoulos | located in /admin. Only users of type ADMIN can access the admin pages. To |
995 | 998f872c | Christos Stavrakakis | change the type of a user to ADMIN, snf-manage can be used: |
996 | bc055d09 | Constantinos Venetsanopoulos | |
997 | bc055d09 | Constantinos Venetsanopoulos | .. code-block:: console |
998 | bc055d09 | Constantinos Venetsanopoulos | |
999 | 7a3439cf | Constantinos Venetsanopoulos | $ snf-manage user-modify 42 --type ADMIN |
1000 | bc055d09 | Constantinos Venetsanopoulos | |
1001 | bc055d09 | Constantinos Venetsanopoulos | Logging |
1002 | bc055d09 | Constantinos Venetsanopoulos | ------- |
1003 | bc055d09 | Constantinos Venetsanopoulos | |
1004 | bc055d09 | Constantinos Venetsanopoulos | Logging in Synnefo is using Python's logging module. The module is configured |
1005 | bc055d09 | Constantinos Venetsanopoulos | using dictionary configuration, whose format is described here: |
1006 | bc055d09 | Constantinos Venetsanopoulos | |
1007 | bc055d09 | Constantinos Venetsanopoulos | http://docs.python.org/release/2.7.1/library/logging.html#logging-config-dictschema |
1008 | bc055d09 | Constantinos Venetsanopoulos | |
1009 | bc055d09 | Constantinos Venetsanopoulos | Note that this is a feature of Python 2.7 that we have backported for use in |
1010 | bc055d09 | Constantinos Venetsanopoulos | Python 2.6. |
1011 | bc055d09 | Constantinos Venetsanopoulos | |
1012 | 4be65f1f | Christos Stavrakakis | The logging configuration dictionary is defined in |
1013 | a1c707c9 | Constantinos Venetsanopoulos | ``/etc/synnefo/10-snf-webproject-logging.conf`` |
1014 | bc055d09 | Constantinos Venetsanopoulos | |
1015 | a1c707c9 | Constantinos Venetsanopoulos | The administrator can have finer logging control by modifying the |
1016 | a1c707c9 | Constantinos Venetsanopoulos | ``LOGGING_SETUP`` dictionary, and defining subloggers with different handlers |
1017 | a1c707c9 | Constantinos Venetsanopoulos | and log levels. e.g. To enable debug messages only for the API set the level |
1018 | a1c707c9 | Constantinos Venetsanopoulos | of 'synnefo.api' to ``DEBUG`` |
1019 | 4be65f1f | Christos Stavrakakis | |
1020 | a1c707c9 | Constantinos Venetsanopoulos | By default, the Django webapp and snf-manage logs to syslog, while |
1021 | a1c707c9 | Constantinos Venetsanopoulos | `snf-dispatcher` logs to `/var/log/synnefo/dispatcher.log`. |
1022 | bc055d09 | Constantinos Venetsanopoulos | |
1023 | d189d11c | Constantinos Venetsanopoulos | |
1024 | 21d3d487 | Constantinos Venetsanopoulos | .. _scale-up: |
1025 | 21d3d487 | Constantinos Venetsanopoulos | |
1026 | bc055d09 | Constantinos Venetsanopoulos | Scaling up to multiple nodes |
1027 | bc055d09 | Constantinos Venetsanopoulos | ============================ |
1028 | bc055d09 | Constantinos Venetsanopoulos | |
1029 | 301294a9 | Constantinos Venetsanopoulos | Here we will describe how should a large scale Synnefo deployment look like. Make |
1030 | 301294a9 | Constantinos Venetsanopoulos | sure you are familiar with Synnefo and Ganeti before proceeding with this section. |
1031 | 301294a9 | Constantinos Venetsanopoulos | This means you should at least have already set up successfully a working Synnefo |
1032 | 301294a9 | Constantinos Venetsanopoulos | deployment as described in the :ref:`Admin's Quick Installation Guide |
1033 | 301294a9 | Constantinos Venetsanopoulos | <quick-install-admin-guide>` and also read the Administrator's Guide until this |
1034 | 301294a9 | Constantinos Venetsanopoulos | section. |
1035 | 2f6143c9 | Constantinos Venetsanopoulos | |
1036 | 301294a9 | Constantinos Venetsanopoulos | Graph of a scale-out Synnefo deployment |
1037 | 301294a9 | Constantinos Venetsanopoulos | --------------------------------------- |
1038 | 2f6143c9 | Constantinos Venetsanopoulos | |
1039 | 301294a9 | Constantinos Venetsanopoulos | Each box in the following graph corresponds to a distinct physical node: |
1040 | 2f6143c9 | Constantinos Venetsanopoulos | |
1041 | 301294a9 | Constantinos Venetsanopoulos | .. image:: images/synnefo-arch2-roles.png |
1042 | 301294a9 | Constantinos Venetsanopoulos | :width: 100% |
1043 | 301294a9 | Constantinos Venetsanopoulos | :target: _images/synnefo-arch2-roles.png |
1044 | 301294a9 | Constantinos Venetsanopoulos | |
1045 | 301294a9 | Constantinos Venetsanopoulos | The above graph is actually the same with the one at the beginning of this |
1046 | 301294a9 | Constantinos Venetsanopoulos | :ref:`guide <admin-guide>`, with the only difference that here we show the |
1047 | 301294a9 | Constantinos Venetsanopoulos | Synnefo roles of each physical node. These roles are described in the |
1048 | 301294a9 | Constantinos Venetsanopoulos | following section. |
1049 | 301294a9 | Constantinos Venetsanopoulos | |
1050 | 21d3d487 | Constantinos Venetsanopoulos | .. _physical-node-roles: |
1051 | 21d3d487 | Constantinos Venetsanopoulos | |
1052 | 301294a9 | Constantinos Venetsanopoulos | Physical Node roles |
1053 | 301294a9 | Constantinos Venetsanopoulos | ------------------- |
1054 | 301294a9 | Constantinos Venetsanopoulos | |
1055 | 301294a9 | Constantinos Venetsanopoulos | As appears in the previous graph, a scale-out Synnefo deployment consists of |
1056 | 301294a9 | Constantinos Venetsanopoulos | multiple physical nodes that have the following roles: |
1057 | 301294a9 | Constantinos Venetsanopoulos | |
1058 | 301294a9 | Constantinos Venetsanopoulos | * **WEBSERVER**: A web server running in front of gunicorn (e.g.: Apache, nginx) |
1059 | 301294a9 | Constantinos Venetsanopoulos | * **ASTAKOS**: The Astakos application (gunicorn) |
1060 | 301294a9 | Constantinos Venetsanopoulos | * **ASTAKOS_DB**: The Astakos database (postgresql) |
1061 | 301294a9 | Constantinos Venetsanopoulos | * **PITHOS**: The Pithos application (gunicorn) |
1062 | 301294a9 | Constantinos Venetsanopoulos | * **PITHOS_DB**: The Pithos database (postgresql) |
1063 | 301294a9 | Constantinos Venetsanopoulos | * **CYCLADES**: The Cyclades application (gunicorn) |
1064 | 301294a9 | Constantinos Venetsanopoulos | * **CYCLADES_DB**: The Cyclades database (postgresql) |
1065 | 301294a9 | Constantinos Venetsanopoulos | * **MQ**: The message queue (RabbitMQ) |
1066 | 301294a9 | Constantinos Venetsanopoulos | * **GANETI_MASTER**: The Ganeti master of a Ganeti cluster |
1067 | 301294a9 | Constantinos Venetsanopoulos | * **GANETI_NODE** : A VM-capable Ganeti node of a Ganeti cluster |
1068 | 2f6143c9 | Constantinos Venetsanopoulos | |
1069 | 301294a9 | Constantinos Venetsanopoulos | You will probably also have: |
1070 | 2f6143c9 | Constantinos Venetsanopoulos | |
1071 | 301294a9 | Constantinos Venetsanopoulos | * **CMS**: The CMS used as a frotend portal for the Synnefo services |
1072 | 21d3d487 | Constantinos Venetsanopoulos | * **NS**: A nameserver serving all other Synnefo nodes and resolving Synnefo FQDNs |
1073 | 301294a9 | Constantinos Venetsanopoulos | * **CLIENT**: A machine that runs the Synnefo clients (e.g.: kamaki, Web UI), |
1074 | 301294a9 | Constantinos Venetsanopoulos | most of the times, the end user's local machine |
1075 | 301294a9 | Constantinos Venetsanopoulos | |
1076 | 301294a9 | Constantinos Venetsanopoulos | From this point we will also refer to the following groups of roles: |
1077 | 301294a9 | Constantinos Venetsanopoulos | |
1078 | 301294a9 | Constantinos Venetsanopoulos | * **SYNNEFO**: [ **ASTAKOS**, **ASTAKOS_DB**, **PITHOS**, **PITHOS_DB**, **CYCLADES**, **CYCLADES_DB**, **MQ**, **CMS**] |
1079 | 301294a9 | Constantinos Venetsanopoulos | * **G_BACKEND**: [**GANETI_MASTER**, **GANETI_NODE**] |
1080 | 301294a9 | Constantinos Venetsanopoulos | |
1081 | 301294a9 | Constantinos Venetsanopoulos | Of course, when deploying Synnefo you can combine multiple of the above roles on a |
1082 | 301294a9 | Constantinos Venetsanopoulos | single physical node, but if you are trying to scale out, the above separation |
1083 | 301294a9 | Constantinos Venetsanopoulos | gives you significant advantages. |
1084 | 301294a9 | Constantinos Venetsanopoulos | |
1085 | 301294a9 | Constantinos Venetsanopoulos | So, in the next section we will take a look on what components you will have to |
1086 | 301294a9 | Constantinos Venetsanopoulos | install on each physical node depending on its Synnefo role. We assume the graph's |
1087 | 301294a9 | Constantinos Venetsanopoulos | architecture. |
1088 | 301294a9 | Constantinos Venetsanopoulos | |
1089 | 301294a9 | Constantinos Venetsanopoulos | Components for each role |
1090 | 301294a9 | Constantinos Venetsanopoulos | ------------------------ |
1091 | 2f6143c9 | Constantinos Venetsanopoulos | |
1092 | 301294a9 | Constantinos Venetsanopoulos | When deploying Synnefo in large scale, you need to install different Synnefo |
1093 | 301294a9 | Constantinos Venetsanopoulos | or/and third party components on different physical nodes according to their |
1094 | 301294a9 | Constantinos Venetsanopoulos | Synnefo role, as stated in the previous section. |
1095 | 301294a9 | Constantinos Venetsanopoulos | |
1096 | 301294a9 | Constantinos Venetsanopoulos | Specifically: |
1097 | 301294a9 | Constantinos Venetsanopoulos | |
1098 | 301294a9 | Constantinos Venetsanopoulos | Role **WEBSERVER** |
1099 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1100 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Apache |
1101 | 301294a9 | Constantinos Venetsanopoulos | Role **ASTAKOS** |
1102 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-webproject`, `snf-astakos-app` |
1103 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Django, Gunicorn |
1104 | 301294a9 | Constantinos Venetsanopoulos | Role **ASTAKOS_DB** |
1105 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1106 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: PostgreSQL |
1107 | 301294a9 | Constantinos Venetsanopoulos | Role **PITHOS** |
1108 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-webproject`, `snf-pithos-app`, `snf-pithos-webclient` |
1109 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Django, Gunicorn |
1110 | 301294a9 | Constantinos Venetsanopoulos | Role **PITHOS_DB** |
1111 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1112 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: PostgreSQL |
1113 | 301294a9 | Constantinos Venetsanopoulos | Role **CYCLADES** |
1114 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-webproject`, `snf-cyclades-app`, `snf-vncauthproxy` |
1115 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Django Gunicorn |
1116 | 301294a9 | Constantinos Venetsanopoulos | Role **CYCLADES_DB** |
1117 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1118 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: PostgreSQL |
1119 | 301294a9 | Constantinos Venetsanopoulos | Role **MQ** |
1120 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1121 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: RabbitMQ |
1122 | 301294a9 | Constantinos Venetsanopoulos | Role **GANETI_MASTER** |
1123 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-cyclades-gtools` |
1124 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Ganeti |
1125 | 301294a9 | Constantinos Venetsanopoulos | Role **GANETI_NODE** |
1126 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-cyclades-gtools`, `snf-network`, `snf-image`, `nfdhcpd` |
1127 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Ganeti |
1128 | 301294a9 | Constantinos Venetsanopoulos | Role **CMS** |
1129 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `snf-webproject`, `snf-cloudcms` |
1130 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: Django, Gunicorn |
1131 | 301294a9 | Constantinos Venetsanopoulos | Role **NS** |
1132 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `None` |
1133 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: BIND |
1134 | 301294a9 | Constantinos Venetsanopoulos | Role **CLIENT** |
1135 | 301294a9 | Constantinos Venetsanopoulos | * Synnefo components: `kamaki`, `snf-image-creator` |
1136 | 301294a9 | Constantinos Venetsanopoulos | * 3rd party components: `None` |
1137 | 301294a9 | Constantinos Venetsanopoulos | |
1138 | 301294a9 | Constantinos Venetsanopoulos | Example scale out installation |
1139 | 301294a9 | Constantinos Venetsanopoulos | ------------------------------ |
1140 | 301294a9 | Constantinos Venetsanopoulos | |
1141 | 301294a9 | Constantinos Venetsanopoulos | In this section we describe an example of a medium scale installation which |
1142 | 301294a9 | Constantinos Venetsanopoulos | combines multiple roles on 10 different physical nodes. We also provide a |
1143 | 301294a9 | Constantinos Venetsanopoulos | :ref:`guide <i-synnefo>` to help with such an install. |
1144 | 301294a9 | Constantinos Venetsanopoulos | |
1145 | 301294a9 | Constantinos Venetsanopoulos | We assume that we have the following 10 physical nodes with the corresponding |
1146 | 301294a9 | Constantinos Venetsanopoulos | roles: |
1147 | 301294a9 | Constantinos Venetsanopoulos | |
1148 | 301294a9 | Constantinos Venetsanopoulos | Node1: |
1149 | 301294a9 | Constantinos Venetsanopoulos | **WEBSERVER**, **ASTAKOS** |
1150 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1151 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1152 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`gunicorn <i-gunicorn>` |
1153 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apache <i-apache>` |
1154 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-webproject <i-webproject>` |
1155 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-astakos-app <i-astakos>` |
1156 | 301294a9 | Constantinos Venetsanopoulos | Node2: |
1157 | 301294a9 | Constantinos Venetsanopoulos | **WEBSERVER**, **PITHOS** |
1158 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1159 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1160 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`gunicorn <i-gunicorn>` |
1161 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apache <i-apache>` |
1162 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-webproject <i-webproject>` |
1163 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-pithos-app <i-pithos>` |
1164 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-pithos-webclient <i-pithos>` |
1165 | 301294a9 | Constantinos Venetsanopoulos | Node3: |
1166 | 301294a9 | Constantinos Venetsanopoulos | **WEBSERVER**, **CYCLADES** |
1167 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1168 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1169 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`gunicorn <i-gunicorn>` |
1170 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apache <i-apache>` |
1171 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-webproject <i-webproject>` |
1172 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-cyclades-app <i-cyclades>` |
1173 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-vncauthproxy <i-cyclades>` |
1174 | 301294a9 | Constantinos Venetsanopoulos | Node4: |
1175 | 301294a9 | Constantinos Venetsanopoulos | **WEBSERVER**, **CMS** |
1176 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1177 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1178 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`gunicorn <i-gunicorn>` |
1179 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apache <i-apache>` |
1180 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-webproject <i-webproject>` |
1181 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-cloudcms <i-cms>` |
1182 | 301294a9 | Constantinos Venetsanopoulos | Node5: |
1183 | 301294a9 | Constantinos Venetsanopoulos | **ASTAKOS_DB**, **PITHOS_DB**, **CYCLADES_DB** |
1184 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1185 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1186 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`postgresql <i-db>` |
1187 | 301294a9 | Constantinos Venetsanopoulos | Node6: |
1188 | 301294a9 | Constantinos Venetsanopoulos | **MQ** |
1189 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1190 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1191 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`rabbitmq <i-mq>` |
1192 | 301294a9 | Constantinos Venetsanopoulos | Node7: |
1193 | 301294a9 | Constantinos Venetsanopoulos | **GANETI_MASTER**, **GANETI_NODE** |
1194 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1195 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1196 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`general <i-backends>` |
1197 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`ganeti <i-ganeti>` |
1198 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-cyclades-gtools <i-gtools>` |
1199 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-network <i-network>` |
1200 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-image <i-image>` |
1201 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`nfdhcpd <i-network>` |
1202 | 301294a9 | Constantinos Venetsanopoulos | Node8: |
1203 | 301294a9 | Constantinos Venetsanopoulos | **GANETI_NODE** |
1204 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1205 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`apt <i-apt>` |
1206 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`general <i-backends>` |
1207 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`ganeti <i-ganeti>` |
1208 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-cyclades-gtools <i-gtools>` |
1209 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-network <i-network>` |
1210 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`snf-image <i-image>` |
1211 | 301294a9 | Constantinos Venetsanopoulos | * :ref:`nfdhcpd <i-network>` |
1212 | 301294a9 | Constantinos Venetsanopoulos | Node9: |
1213 | 301294a9 | Constantinos Venetsanopoulos | **GANETI_NODE** |
1214 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1215 | 301294a9 | Constantinos Venetsanopoulos | `Same as Node8` |
1216 | 301294a9 | Constantinos Venetsanopoulos | Node10: |
1217 | 301294a9 | Constantinos Venetsanopoulos | **GANETI_NODE** |
1218 | 301294a9 | Constantinos Venetsanopoulos | Guide sections: |
1219 | 301294a9 | Constantinos Venetsanopoulos | `Same as Node8` |
1220 | 301294a9 | Constantinos Venetsanopoulos | |
1221 | 301294a9 | Constantinos Venetsanopoulos | All sections: :ref:`Scale out Guide <i-synnefo>` |
1222 | 547c78f6 | Constantinos Venetsanopoulos | |
1223 | bc055d09 | Constantinos Venetsanopoulos | |
1224 | bbcd3dd1 | Constantinos Venetsanopoulos | Upgrade Notes |
1225 | bbcd3dd1 | Constantinos Venetsanopoulos | ============= |
1226 | bc055d09 | Constantinos Venetsanopoulos | |
1227 | 457fe4fc | Georgios D. Tsoukalas | .. toctree:: |
1228 | 457fe4fc | Georgios D. Tsoukalas | :maxdepth: 1 |
1229 | 457fe4fc | Georgios D. Tsoukalas | |
1230 | 1cd3daa1 | Constantinos Venetsanopoulos | v0.12 -> v0.13 <upgrade/upgrade-0.13> |
1231 | 457fe4fc | Georgios D. Tsoukalas | |
1232 | bbcd3dd1 | Constantinos Venetsanopoulos | |
1233 | bbcd3dd1 | Constantinos Venetsanopoulos | Changelog, News |
1234 | bbcd3dd1 | Constantinos Venetsanopoulos | =============== |
1235 | bbcd3dd1 | Constantinos Venetsanopoulos | |
1236 | bbcd3dd1 | Constantinos Venetsanopoulos | * v0.13 :ref:`Changelog <Changelog-0.13>`, :ref:`NEWS <NEWS-0.13>` |
1237 | bbcd3dd1 | Constantinos Venetsanopoulos | |
1238 | bbcd3dd1 | Constantinos Venetsanopoulos | |
1239 | 1cd3daa1 | Constantinos Venetsanopoulos | Older Cyclades Upgrade Notes |
1240 | 457fe4fc | Georgios D. Tsoukalas | ============================ |
1241 | bc055d09 | Constantinos Venetsanopoulos | |
1242 | bc055d09 | Constantinos Venetsanopoulos | .. toctree:: |
1243 | bc055d09 | Constantinos Venetsanopoulos | :maxdepth: 2 |
1244 | bc055d09 | Constantinos Venetsanopoulos | |
1245 | 45aa2f06 | Constantinos Venetsanopoulos | Upgrade <upgrade/cyclades-upgrade> |