root / docs / upgrade-0.13.rst @ 7a882417
History | View | Annotate | Download (21.2 kB)
1 |
Upgrade to Synnefo v0.13 |
---|---|
2 |
^^^^^^^^^^^^^^^^^^^^^^^^ |
3 |
|
4 |
The bulk of the upgrade to v0.13 is about user and quota migrations. |
5 |
In summary, the migration process has 3 steps: |
6 |
|
7 |
1. Run some commands and scripts to diagnose and extract some migration data |
8 |
while the OLD code is running, and BEFORE any changes are made. |
9 |
|
10 |
2. Bring down services, upgrade packages, configure services, and perform |
11 |
django database migrations. These migrations do not need any interaction |
12 |
between services. |
13 |
|
14 |
3. Initialize the Astakos quota system and bring the Astakos service up, since |
15 |
it will be needed during a second-phase of UUID and quota migrations, that |
16 |
also uses data extracted from step 1. |
17 |
|
18 |
|
19 |
.. warning:: |
20 |
|
21 |
It is strongly suggested that you keep separate database backups |
22 |
for each service after the completion of each of step. |
23 |
|
24 |
1. Bring web services down, backup databases |
25 |
============================================ |
26 |
|
27 |
1. All web services must be brought down so that the database maintains a |
28 |
predictable and consistent state during the migration process:: |
29 |
|
30 |
# service gunicorn stop |
31 |
# service snf-dispatcher stop |
32 |
# etc. |
33 |
|
34 |
2. Backup databases for recovery to a pre-migration state. |
35 |
|
36 |
3. Keep the database servers running during the migration process |
37 |
|
38 |
|
39 |
2. Prepare astakos user migration to case insensitive emails |
40 |
============================================================ |
41 |
|
42 |
It is possible that two or more users have been registered with emails that |
43 |
differ only in the case of its letters. There can only be one of those |
44 |
accounts after the migration, so the rest must be deleted. |
45 |
|
46 |
Note that even if the users are deleted in Astakos, there still are duplicate |
47 |
entries in Cyclades and Pithos. For each service we need to reduce those |
48 |
multiple accounts into one, either merging them together, or deleting and |
49 |
discarding data from all but one. |
50 |
|
51 |
.. _find_duplicate_emails: |
52 |
|
53 |
2.1 Find duplicate email entries in Astakos |
54 |
------------------------------------------- |
55 |
(script: ``find_astakos_users_with_conflicting_emails.py``):: |
56 |
|
57 |
astakos-host$ cat << EOF > find_astakos_users_with_conflicting_emails.py |
58 |
#!/usr/bin/env python |
59 |
import os |
60 |
import sys |
61 |
|
62 |
os.environ['DJANGO_SETTINGS_MODULE'] = 'synnefo.settings' |
63 |
|
64 |
import astakos |
65 |
from astakos.im.models import AstakosUser as A |
66 |
|
67 |
|
68 |
def user_filter(user): |
69 |
return A.objects.filter(email__iexact=user.email).count() > 1 |
70 |
|
71 |
all_users = list(A.objects.all()) |
72 |
userlist = [(str(u.pk) + ': ' + str(u.email) + ' (' + str(u.is_active) + ', ' + |
73 |
str(u.date_joined) + ')') for u in filter(user_filter, all_users)] |
74 |
|
75 |
sys.stderr.write("id email (is_active, creation date)\n") |
76 |
print "\n".join(userlist) |
77 |
EOF |
78 |
|
79 |
astakos-host$ python ./find_astakos_users_with_conflicting_emails.py |
80 |
|
81 |
.. _remove_astakos_duplicate: |
82 |
|
83 |
2.1 Remove duplicate users in Astakos by their id |
84 |
------------------------------------------------- |
85 |
(script: ``delete_astakos_users.py``):: |
86 |
|
87 |
astakos-host$ cat << EOF > delete_astakos_users.py |
88 |
#!/usr/bin/env python |
89 |
|
90 |
import os |
91 |
import sys |
92 |
from time import sleep |
93 |
|
94 |
os.environ['DJANGO_SETTINGS_MODULE'] = 'synnefo.settings' |
95 |
|
96 |
import astakos |
97 |
from astakos.im.models import AstakosUser as A |
98 |
|
99 |
|
100 |
def user_filter(user): |
101 |
return A.objects.filter(email__iexact=user.email).count() > 1 |
102 |
|
103 |
argv = sys.argv |
104 |
argc = len(sys.argv) |
105 |
|
106 |
if argc < 2: |
107 |
print "Usage: ./delete_astakos_users.py <id>..." |
108 |
raise SystemExit() |
109 |
|
110 |
id_list = [int(x) for x in argv[1:]] |
111 |
|
112 |
print "" |
113 |
print "This will permanently delete the following users:\n" |
114 |
print "id email (is_active, creation date)" |
115 |
print "-- --------------------------------" |
116 |
|
117 |
users = A.objects.filter(id__in=id_list) |
118 |
for user in users: |
119 |
print "%s: %s (%s, %s)" % (user.id, user.email, user.is_active, |
120 |
user.date_joined) |
121 |
|
122 |
print "\nExecute? (yes/no): ", |
123 |
line = raw_input().rstrip() |
124 |
if line != 'yes': |
125 |
print "\nCancelled" |
126 |
raise SystemExit() |
127 |
|
128 |
print "\nConfirmed." |
129 |
sleep(2) |
130 |
for user in users: |
131 |
print "deleting %s: %s" % (user.id, user.email) |
132 |
user.delete() |
133 |
|
134 |
EOF |
135 |
|
136 |
astakos-host$ python ./delete_astakos_users.py 30 40 |
137 |
|
138 |
.. warning:: |
139 |
|
140 |
After deleting users with the ``delete_astakos_users.py`` script, |
141 |
check again with ``find_astakos_users_with_conflicting_emails.py`` |
142 |
(as in :ref:`find_duplicate_emails`) |
143 |
to make sure that no duplicate email conflicts remain. |
144 |
|
145 |
|
146 |
3. Upgrade Synnefo and configure settings |
147 |
========================================= |
148 |
|
149 |
3.1 Install the new versions of packages |
150 |
---------------------------------------- |
151 |
|
152 |
:: |
153 |
|
154 |
astakos.host$ apt-get install \ |
155 |
kamaki \ |
156 |
snf-common \ |
157 |
snf-webproject \ |
158 |
snf-quotaholder-app \ |
159 |
snf-astakos-app \ |
160 |
|
161 |
|
162 |
cyclades.host$ apt-get install \ |
163 |
kamaki \ |
164 |
snf-common \ |
165 |
snf-webproject |
166 |
snf-pithos-backend \ |
167 |
snf-cyclades-app \ |
168 |
|
169 |
|
170 |
pithos.host$ apt-get install \ |
171 |
kamaki \ |
172 |
snf-common \ |
173 |
snf-webproject |
174 |
snf-pithos-backend \ |
175 |
snf-pithos-app \ |
176 |
snf-pithos-webclient \ |
177 |
|
178 |
|
179 |
ganeti.node$ apt-get install \ |
180 |
kamaki \ |
181 |
snf-common \ |
182 |
snf-cyclades-gtools \ |
183 |
snf-pithos-backend \ |
184 |
|
185 |
.. note:: |
186 |
|
187 |
Installing the packages will cause services to start. Make sure you bring |
188 |
them down again (at least ``gunicorn``, ``snf-dispatcher``) |
189 |
|
190 |
3.2 Sync and migrate Django DB |
191 |
------------------------------ |
192 |
|
193 |
.. note:: |
194 |
|
195 |
If you are asked about stale content types during the migration process, |
196 |
answer 'no' and let the migration finish. |
197 |
|
198 |
:: |
199 |
|
200 |
astakos-host$ snf-manage syncdb |
201 |
astakos-host$ snf-manage migrate |
202 |
|
203 |
cyclades-host$ snf-manage syncdb |
204 |
cyclades-host$ snf-manage migrate |
205 |
|
206 |
.. note:: |
207 |
|
208 |
After the migration, Astakos has created uuids for all users, |
209 |
and has set the uuid as the public identifier of a user. |
210 |
This uuid is to be used both at other services (Cyclades, Pithos) |
211 |
and at the clientside (kamaki client settings). |
212 |
|
213 |
Duplicate-email users have been deleted earlier in |
214 |
:ref:`remove_astakos_duplicate` |
215 |
|
216 |
3.3 Setup quota settings for all services |
217 |
----------------------------------------- |
218 |
|
219 |
Generally: |
220 |
|
221 |
:: |
222 |
|
223 |
# Service Setting Value |
224 |
# quotaholder: QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v |
225 |
# quotaholder: QUOTAHOLDER_TOKEN = <random string> |
226 |
|
227 |
# astakos: ASTAKOS_QUOTAHOLDER_TOKEN = <the same random string> |
228 |
# astakos: ASTAKOS_QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v |
229 |
|
230 |
# cyclades: CYCLADES_QUOTAHOLDER_TOKEN = <the same random string> |
231 |
# cyclades: CYCLADES_QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v |
232 |
# cyclades: CYCLADES_USE_QUOTAHOLDER = True |
233 |
|
234 |
|
235 |
# pithos: PITHOS_QUOTAHOLDER_TOKEN = <the same random string> |
236 |
# pithos: PITHOS_QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v |
237 |
# pithos: PITHOS_USE_QUOTAHOLDER = True |
238 |
# All services must match the quotaholder token and url configured for quotaholder. |
239 |
|
240 |
Specifically: |
241 |
|
242 |
On the Astakos host, edit ``/etc/synnefo/20-snf-astakos-app-settings.conf``: |
243 |
|
244 |
:: |
245 |
|
246 |
QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v' |
247 |
QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w' |
248 |
ASTAKOS_QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w' |
249 |
ASTAKOS_QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v' |
250 |
|
251 |
On the Cyclades host, edit ``/etc/synnefo/20-snf-cyclades-app-quotas.conf``: |
252 |
|
253 |
:: |
254 |
|
255 |
CYCLADES_USE_QUOTAHOLDER = True |
256 |
CYCLADES_QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v' |
257 |
CYCLADES_QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w' |
258 |
|
259 |
# Set to False if astakos & cyclades are on the same host |
260 |
#CYCLADES_PROXY_USER_SERVICES = True |
261 |
|
262 |
.. note:: |
263 |
|
264 |
If Cylcades and Astakos are installed on the same server, |
265 |
set ``CYCLADES_PROXY_USER_SERVICES = False`` |
266 |
|
267 |
|
268 |
On the Pithos host, edit ``/etc/synnefo/20-snf-pithos-app-settings.conf``: |
269 |
|
270 |
:: |
271 |
|
272 |
PITHOS_QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v' |
273 |
PITHOS_QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w' |
274 |
PITHOS_USE_QUOTAHOLDER = False # will set to True after migration |
275 |
|
276 |
.. note:: |
277 |
|
278 |
During the migration it must be set, ``PITHOS_USE_QUOTAHOLDER = False``. |
279 |
Set to ``True`` once the migration is over. |
280 |
|
281 |
3.4 Setup astakos |
282 |
----------------- |
283 |
|
284 |
- **Remove** this redirection from astakos front-end web server :: |
285 |
|
286 |
RewriteRule ^/login(.*) /im/login/redirect$1 [PT,NE] |
287 |
|
288 |
(see `<http://docs.dev.grnet.gr/synnefo/latest/quick-install-admin-guide.html#apache2-setup>`_) |
289 |
|
290 |
- Enable users to change their contact email. Edit |
291 |
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :: |
292 |
|
293 |
ASTAKOS_EMAILCHANGE_ENABLED = True |
294 |
|
295 |
- Rename the following (Astakos-specific) setting:: |
296 |
|
297 |
ASTAKOS_DEFAULT_FROM_EMAIL |
298 |
|
299 |
to this (Django-specific) name:: |
300 |
|
301 |
SERVER_EMAIL |
302 |
|
303 |
- Instead of using the following (Astakos-specific) setting:: |
304 |
|
305 |
ASTAKOS_DEFAULT_ADMIN_EMAIL |
306 |
|
307 |
include one or more entries in this (Django-specific) setting:: |
308 |
|
309 |
ADMINS = ( |
310 |
('Joe Doe', 'doe@example.net'), |
311 |
('Mary Jean', 'mary@example.net'), |
312 |
) |
313 |
|
314 |
.. note:: |
315 |
|
316 |
The ``SERVER_EMAIL`` and ``ADMINS`` settings are Django-specific. |
317 |
As such they will be the shared for any two (or more) services that happen |
318 |
to be collocated within the same application server (e.g. astakos & |
319 |
cyclades within the same gunicorn) |
320 |
|
321 |
3.5 Setup Cyclades |
322 |
------------------ |
323 |
|
324 |
- Run on the Astakos host :: |
325 |
|
326 |
# snf-manage service-list |
327 |
|
328 |
- Set the Cyclades service token in |
329 |
``/etc/synnefo/20-snf-cyclades-app-api.conf`` :: |
330 |
|
331 |
CYCLADES_ASTAKOS_SERVICE_TOKEN = 'asfasdf_CycladesServiceToken_iknl' |
332 |
|
333 |
- Since version 0.13, Synnefo uses **VMAPI** in order to prevent sensitive data |
334 |
needed by 'snf-image' to be stored in Ganeti configuration (e.g. VM |
335 |
password). This is achieved by storing all sensitive information to a CACHE |
336 |
backend and exporting it via VMAPI. The cache entries are invalidated after |
337 |
the first request. Synnefo uses **memcached** as a django cache backend. |
338 |
To install, run on the Cyclades host:: |
339 |
|
340 |
apt-get install memcached |
341 |
apt-get install python-memcache |
342 |
|
343 |
You will also need to configure Cyclades to use the memcached cache backend. |
344 |
Namely, you need to set IP address and port of the memcached daemon, and the |
345 |
default timeout (seconds tha value is stored in the cache). Edit |
346 |
``/etc/synnefo/20-snf-cyclades-app-vmapi.conf`` :: |
347 |
|
348 |
VMAPI_CACHE_BACKEND = "memcached://127.0.0.1:11211/?timeout=3600" |
349 |
|
350 |
|
351 |
Finally, set the BASE_URL for the VMAPI, which is actually the base URL of |
352 |
Cyclades, again in ``/etc/synnefo/20-snf-cyclades-app-vmapi.conf``. Make sure |
353 |
the domain is exaclty the same, so that no re-directs happen :: |
354 |
|
355 |
VMAPI_BASE_URL = "https://cyclades.example.synnefo.org" |
356 |
|
357 |
.. note:: |
358 |
|
359 |
- These settings are needed in all Cyclades workers. |
360 |
|
361 |
- VMAPI_CACHE_BACKEND just overrides django's CACHE_BACKEND setting |
362 |
|
363 |
- memcached must be reachable from all Cyclades workers. |
364 |
|
365 |
- For more information about configuring django to use memcached: |
366 |
https://docs.djangoproject.com/en/1.2/topics/cache |
367 |
|
368 |
3.6 Setup Pithos |
369 |
---------------- |
370 |
|
371 |
- Pithos forwards user catalog services to Astakos so that web clients may |
372 |
access them for uuid-displayname translations. Edit on the Pithos host |
373 |
``/etc/synnefo/20-snf-pithos-app-settings.conf`` :: |
374 |
|
375 |
PITHOS_USER_CATALOG_URL = https://accounts.example.synnefo.org/user_catalogs/ |
376 |
PITHOS_USER_FEEDBACK_URL = https://accounts.example.synnefo.org/feedback/ |
377 |
PITHOS_USER_LOGIN_URL = https://accounts.example.synnefo.org/login/ |
378 |
|
379 |
# Set to False if astakos & pithos are on the same host |
380 |
#PITHOS_PROXY_USER_SERVICES = True |
381 |
|
382 |
.. note:: |
383 |
|
384 |
If Pithos and Astakos are installed on the same server, |
385 |
set ``PITHOS_PROXY_USER_SERVICES = False`` |
386 |
|
387 |
4. Start astakos and quota services |
388 |
=================================== |
389 |
.. warning:: |
390 |
|
391 |
To ensure consistency, prevent public access to astakos during migrations. |
392 |
This can be done via firewall or webserver access control. |
393 |
|
394 |
Start (or restart, if running) the webserver and gunicorn on the Astakos host. |
395 |
E.g.:: |
396 |
|
397 |
# service apache2 start |
398 |
# service gunicorn start |
399 |
|
400 |
.. _astakos-load-resources: |
401 |
|
402 |
5. Load resource definitions into Astakos |
403 |
========================================= |
404 |
|
405 |
First, set the corresponding values on the following dict in |
406 |
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :: |
407 |
|
408 |
# Set the cloud service properties |
409 |
ASTAKOS_SERVICES = { |
410 |
'cyclades': { |
411 |
# # Specifying the key 'url' will overwrite it. |
412 |
# # Use this to (re)set service URL. |
413 |
# 'url': 'https://cyclades.example.synnefo.org/ui/', |
414 |
# # order services in listings, cloudbar, etc. |
415 |
# 'order' : 1 |
416 |
'resources': [{ |
417 |
'name': 'disk', |
418 |
'group': 'compute', |
419 |
'uplimit': 30*1024*1024*1024, |
420 |
'unit': 'bytes', |
421 |
'desc': 'Virtual machine disk size' |
422 |
}, { |
423 |
'name': 'cpu', |
424 |
'group': 'compute', |
425 |
'uplimit': 6, |
426 |
'desc': 'Number of virtual machine processors' |
427 |
}, { |
428 |
'name': 'ram', |
429 |
'group': 'compute', |
430 |
'uplimit': 6*1024*1024*1024, |
431 |
'unit': 'bytes', |
432 |
'desc': 'Virtual machines' |
433 |
}, { |
434 |
'name': 'vm', |
435 |
'group': 'compute', |
436 |
'uplimit': 2, |
437 |
'desc': 'Number of virtual machines' |
438 |
}, { |
439 |
'name': 'network.private', |
440 |
'group': 'network', |
441 |
'uplimit': 1, |
442 |
'desc': 'Private networks' |
443 |
} |
444 |
] |
445 |
}, |
446 |
'pithos+': { |
447 |
# # Use this to (re)set service URL. |
448 |
# 'url': 'https://pithos.example.synnefo.org/ui/', |
449 |
# # order services in listings, cloudbar, etc. |
450 |
# 'order' : 2 |
451 |
'resources':[{ |
452 |
'name': 'diskspace', |
453 |
'group': 'storage', |
454 |
'uplimit': 5*1024*1024*1024, |
455 |
'unit': 'bytes', |
456 |
'desc': 'Pithos account diskspace' |
457 |
}] |
458 |
} |
459 |
} |
460 |
|
461 |
.. note:: |
462 |
|
463 |
The name of the Pithos service is ``pithos+``. |
464 |
If you have named your pithos service ``pithos``, without ``+``, |
465 |
then you must rename it:: |
466 |
|
467 |
$ snf-manage service-list | grep pithos # find service id |
468 |
$ snf-manage service-update --name='pithos+' <service id> |
469 |
|
470 |
Then, configure and load the available resources per service |
471 |
and associated default limits into Astakos. On the Astakos host run :: |
472 |
|
473 |
# snf-manage astakos-init --load-service-resources |
474 |
|
475 |
|
476 |
.. note:: |
477 |
|
478 |
Before v0.13, only `cyclades.vm`, `cyclades.network.private`, |
479 |
and `pithos+.diskspace` existed (not with these names, |
480 |
there were per-service settings). |
481 |
However, limits to the new resources must also be set. |
482 |
|
483 |
If the intetion is to keep a resource unlimited, (counting on that VM |
484 |
creation will be limited by other resources' limit) it is best to calculate |
485 |
a value that is too large to be reached because of other limits (and |
486 |
available flavours), but not much larger than needed because this might |
487 |
confuse users who do not readily understand that multiple limits apply and |
488 |
flavors are limited. |
489 |
|
490 |
|
491 |
6. Migrate Services user names to uuids |
492 |
======================================= |
493 |
|
494 |
|
495 |
6.1 Double-check cyclades before user case/uuid migration |
496 |
--------------------------------------------------------- |
497 |
|
498 |
:: |
499 |
|
500 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --validate |
501 |
|
502 |
Duplicate user found? |
503 |
|
504 |
- either *merge* (merge will merge all resources to one user):: |
505 |
|
506 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --merge-user=kpap@grnet.gr |
507 |
|
508 |
- or *delete* :: |
509 |
|
510 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --delete-user=KPap@grnet.gr |
511 |
# (only KPap will be deleted not kpap) |
512 |
|
513 |
6.2 Double-check pithos before user case/uuid migration |
514 |
--------------------------------------------------------- |
515 |
|
516 |
:: |
517 |
|
518 |
pithos.host$ snf-manage pithos-manage-accounts --list-duplicate |
519 |
|
520 |
Duplicate user found? |
521 |
|
522 |
If you want to migrate files first: |
523 |
|
524 |
- *merge* (merge will merge all resources to one user):: |
525 |
|
526 |
pithos.host$ snf-manage pithos-manage-accounts --merge-accounts --src-account=SPapagian@grnet.gr --dest-account=spapagian@grnet.gr |
527 |
# (SPapagian@grnet.gr's contents will be merged into spapagian@grnet.gr, but SPapagian@grnet.gr account will still exist) |
528 |
|
529 |
- and then *delete* :: |
530 |
|
531 |
pithos.host$ snf-manage pithos-manage-accounts --delete-account=SPapagian@grnet.gr |
532 |
# (only SPapagian@grnet.gr will be deleted not spapagian@grnet.gr) |
533 |
|
534 |
If you do *NOT* want to migrate files just run the second step and delete |
535 |
the duplicate account. |
536 |
|
537 |
6.3 Migrate Cyclades users (email case/uuid) |
538 |
-------------------------------------------- |
539 |
|
540 |
:: |
541 |
|
542 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --migrate-users |
543 |
|
544 |
- if invalid usernames are found, verify that they do not exist in astakos:: |
545 |
|
546 |
astakos.host$ snf-manage user-list |
547 |
|
548 |
- if no user exists:: |
549 |
|
550 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --delete-user=<userid> |
551 |
|
552 |
Finally, if you have set manually quotas for specific users inside |
553 |
``/etc/synnefo/20-snf-cyclades-app-api.conf`` (in ``VMS_USER_QUOTA``, |
554 |
``NETWORKS_USER_QUOTA`` make sure to update them so that: |
555 |
|
556 |
1. There are no double entries wrt case sensitivity |
557 |
2. Replace all user email addresses with the corresponding UUIDs |
558 |
|
559 |
To find the UUIDs for step 2 run on the Astakos host :: |
560 |
|
561 |
# snf-manage user-list |
562 |
|
563 |
6.4 Migrate Pithos user names |
564 |
----------------------------- |
565 |
|
566 |
Check if alembic has not been initialized :: |
567 |
|
568 |
pithos.host$ pithos-migrate current |
569 |
|
570 |
- If alembic current is None (e.g. okeanos.io) :: |
571 |
|
572 |
pithos.host$ pithos-migrate stamp 3dd56e750a3 |
573 |
|
574 |
Then, migrate pithos account name to uuid:: |
575 |
|
576 |
pithos.host$ pithos-migrate upgrade head |
577 |
|
578 |
Finally, set this setting to ``True``:: |
579 |
|
580 |
PITHOS_USE_QUOTAHOLDER = True |
581 |
|
582 |
|
583 |
7. Migrate old quota limits |
584 |
=========================== |
585 |
|
586 |
7.1 Migrate Pithos quota limits to Astakos |
587 |
------------------------------------------ |
588 |
|
589 |
Migrate from pithos native to astakos/quotaholder. |
590 |
This requires a file to be transfered from Cyclades to Astakos:: |
591 |
|
592 |
pithos.host$ snf-manage pithos-export-quota --location=pithos-quota.txt |
593 |
pithos.host$ scp pithos-quota.txt astakos.host: |
594 |
astakos.host$ snf-manage user-set-initial-quota pithos-quota.txt |
595 |
|
596 |
.. _export-quota-note: |
597 |
|
598 |
.. note:: |
599 |
|
600 |
`pithos-export-quota` will only export quotas that are not equal to the |
601 |
defaults in Pithos. Therefore, it is possible to both change or maintain |
602 |
the default quotas across the migration. To maintain quotas the new default |
603 |
pithos+.diskpace limit in Astakos must be equal to the (old) default quota |
604 |
limit in Pithos. Change either one of them make them equal. |
605 |
|
606 |
see :ref:`astakos-load-resources` on how to set the (new) default quotas in Astakos. |
607 |
|
608 |
7.2 Migrate Cyclades quota limits to Astakos |
609 |
-------------------------------------------- |
610 |
|
611 |
:: |
612 |
|
613 |
cyclades.host$ snf-manage cyclades-export-quota --location=cyclades-quota.txt |
614 |
cyclades.host$ scp cyclades-quota.txt astakos.host: |
615 |
astakos.host$ snf-manage user-set-initial-quota cyclades-quota.txt |
616 |
|
617 |
`cyclades-export-quota` will only export quotas that are not equal to the defaults. |
618 |
See :ref:`note above <export-quota-note>`. |
619 |
|
620 |
8. Enforce the new quota limits migrated to Astakos |
621 |
=================================================== |
622 |
The following should report all users not having quota limits set |
623 |
because the effective quota database has not been initialized yet. :: |
624 |
|
625 |
astakos.host$ snf-manage astakos-quota --verify |
626 |
|
627 |
Initialize the effective quota database:: |
628 |
|
629 |
astakos.host$ snf-manage astakos-quota --sync |
630 |
|
631 |
This procedure may be used to verify and re-synchronize the effective quota |
632 |
database with the quota limits that are derived from policies in Astakos |
633 |
(initial quotas, project memberships, etc.) |
634 |
|
635 |
9. Initialize resource usage |
636 |
============================ |
637 |
|
638 |
The effective quota database (quotaholder) has just been initialized and knows |
639 |
nothing of the current resource usage. Therefore, each service must send it in. |
640 |
|
641 |
9.1 Initialize Pithos resource usage |
642 |
------------------------------------ |
643 |
|
644 |
:: |
645 |
|
646 |
pithos.host$ snf-manage pithos-reset-usage |
647 |
|
648 |
9.2 Initialize Cyclades resource usage |
649 |
-------------------------------------- |
650 |
|
651 |
:: |
652 |
|
653 |
cyclades.host$ snf-manage cyclades-reset-usage |
654 |
|
655 |
10. Install periodic project maintainance checks |
656 |
================================================ |
657 |
In order to detect and effect project expiration, |
658 |
a management command has to be run periodically |
659 |
(depending on the required granularity, e.g. once a day/hour):: |
660 |
|
661 |
astakos.host$ snf-manage project-control --terminate-expired |
662 |
|
663 |
A list of expired projects can be extracted with:: |
664 |
|
665 |
astakos.host$ snf-manage project-control --list-expired |
666 |
|
667 |
|
668 |
11. Restart all services |
669 |
======================== |
670 |
|
671 |
Start (or restart, if running) all Synnefo services on all hosts. |
672 |
|
673 |
:: |
674 |
|
675 |
# service gunicorn restart |
676 |
# service snf-dispatcher restart |
677 |
# etc. |