root / docs / upgrade / upgrade-0.13.rst @ 36fea6f9
History | View | Annotate | Download (21.1 kB)
1 |
Upgrade to Synnefo v0.13 |
---|---|
2 |
^^^^^^^^^^^^^^^^^^^^^^^^ |
3 |
|
4 |
The bulk of the upgrade to v0.13 is about user and quota migrations. |
5 |
In summary, the migration process has 3 steps: |
6 |
|
7 |
1. Run some commands and scripts to diagnose and extract some migration data |
8 |
while the OLD code is running, and BEFORE any changes are made. |
9 |
|
10 |
2. Bring down services, upgrade packages, configure services, and perform |
11 |
django database migrations. These migrations do not need any interaction |
12 |
between services. |
13 |
|
14 |
3. Initialize the Astakos quota system and bring the Astakos service up, since |
15 |
it will be needed during a second-phase of UUID and quota migrations, that |
16 |
also uses data extracted from step 1. |
17 |
|
18 |
|
19 |
.. warning:: |
20 |
|
21 |
It is strongly suggested that you keep separate database backups |
22 |
for each service after the completion of each of step. |
23 |
|
24 |
1. Bring web services down, backup databases |
25 |
============================================ |
26 |
|
27 |
1. All web services must be brought down so that the database maintains a |
28 |
predictable and consistent state during the migration process:: |
29 |
|
30 |
# service gunicorn stop |
31 |
# service snf-dispatcher stop |
32 |
# etc. |
33 |
|
34 |
2. Backup databases for recovery to a pre-migration state. |
35 |
|
36 |
3. Keep the database servers running during the migration process |
37 |
|
38 |
|
39 |
2. Prepare astakos user migration to case insensitive emails |
40 |
============================================================ |
41 |
|
42 |
It is possible that two or more users have been registered with emails that |
43 |
differ only in the case of its letters. There can only be one of those |
44 |
accounts after the migration, so the rest must be deleted. |
45 |
|
46 |
Note that even if the users are deleted in Astakos, there still are duplicate |
47 |
entries in Cyclades and Pithos. For each service we need to reduce those |
48 |
multiple accounts into one, either merging them together, or deleting and |
49 |
discarding data from all but one. |
50 |
|
51 |
.. _find_duplicate_emails: |
52 |
|
53 |
2.1 Find duplicate email entries in Astakos |
54 |
------------------------------------------- |
55 |
(script: ``find_astakos_users_with_conflicting_emails.py``):: |
56 |
|
57 |
astakos-host$ cat << EOF > find_astakos_users_with_conflicting_emails.py |
58 |
#!/usr/bin/env python |
59 |
import os |
60 |
import sys |
61 |
|
62 |
os.environ['DJANGO_SETTINGS_MODULE'] = 'synnefo.settings' |
63 |
|
64 |
import astakos |
65 |
from astakos.im.models import AstakosUser as A |
66 |
|
67 |
|
68 |
def user_filter(user): |
69 |
return A.objects.filter(email__iexact=user.email).count() > 1 |
70 |
|
71 |
all_users = list(A.objects.all()) |
72 |
userlist = [(str(u.pk) + ': ' + str(u.email) + ' (' + str(u.is_active) + ', ' + |
73 |
str(u.date_joined) + ')') for u in filter(user_filter, all_users)] |
74 |
|
75 |
sys.stderr.write("id email (is_active, creation date)\n") |
76 |
print "\n".join(userlist) |
77 |
EOF |
78 |
|
79 |
astakos-host$ python ./find_astakos_users_with_conflicting_emails.py |
80 |
|
81 |
.. _remove_astakos_duplicate: |
82 |
|
83 |
2.1 Remove duplicate users in Astakos by their id |
84 |
------------------------------------------------- |
85 |
(script: ``delete_astakos_users.py``):: |
86 |
|
87 |
astakos-host$ cat << EOF > delete_astakos_users.py |
88 |
#!/usr/bin/env python |
89 |
|
90 |
import os |
91 |
import sys |
92 |
from time import sleep |
93 |
|
94 |
os.environ['DJANGO_SETTINGS_MODULE'] = 'synnefo.settings' |
95 |
|
96 |
import astakos |
97 |
from astakos.im.models import AstakosUser as A |
98 |
|
99 |
|
100 |
def user_filter(user): |
101 |
return A.objects.filter(email__iexact=user.email).count() > 1 |
102 |
|
103 |
argv = sys.argv |
104 |
argc = len(sys.argv) |
105 |
|
106 |
if argc < 2: |
107 |
print "Usage: ./delete_astakos_users.py <id>..." |
108 |
raise SystemExit() |
109 |
|
110 |
id_list = [int(x) for x in argv[1:]] |
111 |
|
112 |
print "" |
113 |
print "This will permanently delete the following users:\n" |
114 |
print "id email (is_active, creation date)" |
115 |
print "-- --------------------------------" |
116 |
|
117 |
users = A.objects.filter(id__in=id_list) |
118 |
for user in users: |
119 |
print "%s: %s (%s, %s)" % (user.id, user.email, user.is_active, |
120 |
user.date_joined) |
121 |
|
122 |
print "\nExecute? (yes/no): ", |
123 |
line = raw_input().rstrip() |
124 |
if line != 'yes': |
125 |
print "\nCancelled" |
126 |
raise SystemExit() |
127 |
|
128 |
print "\nConfirmed." |
129 |
sleep(2) |
130 |
for user in users: |
131 |
print "deleting %s: %s" % (user.id, user.email) |
132 |
user.delete() |
133 |
|
134 |
EOF |
135 |
|
136 |
astakos-host$ python ./delete_astakos_users.py 30 40 |
137 |
|
138 |
.. warning:: |
139 |
|
140 |
After deleting users with the ``delete_astakos_users.py`` script, |
141 |
check again with ``find_astakos_users_with_conflicting_emails.py`` |
142 |
(as in :ref:`find_duplicate_emails`) |
143 |
to make sure that no duplicate email conflicts remain. |
144 |
|
145 |
|
146 |
3. Upgrade Synnefo and configure settings |
147 |
========================================= |
148 |
|
149 |
3.1 Install the new versions of packages |
150 |
---------------------------------------- |
151 |
|
152 |
:: |
153 |
|
154 |
astakos.host$ apt-get install \ |
155 |
kamaki \ |
156 |
snf-common \ |
157 |
snf-webproject \ |
158 |
snf-quotaholder-app \ |
159 |
snf-astakos-app |
160 |
|
161 |
|
162 |
cyclades.host$ apt-get install \ |
163 |
kamaki \ |
164 |
snf-common \ |
165 |
snf-webproject \ |
166 |
snf-pithos-backend \ |
167 |
snf-cyclades-app |
168 |
|
169 |
pithos.host$ apt-get install \ |
170 |
kamaki \ |
171 |
snf-common \ |
172 |
snf-webproject \ |
173 |
snf-pithos-backend \ |
174 |
snf-pithos-app \ |
175 |
snf-pithos-webclient |
176 |
|
177 |
|
178 |
ganeti.node$ apt-get install \ |
179 |
kamaki \ |
180 |
snf-common \ |
181 |
snf-cyclades-gtools \ |
182 |
snf-pithos-backend |
183 |
|
184 |
.. note:: |
185 |
|
186 |
Installing the packages will cause services to start. Make sure you bring |
187 |
them down again (at least ``gunicorn``, ``snf-dispatcher``) |
188 |
|
189 |
3.2 Sync and migrate Django DB |
190 |
------------------------------ |
191 |
|
192 |
.. note:: |
193 |
|
194 |
If you are asked about stale content types during the migration process, |
195 |
answer 'no' and let the migration finish. |
196 |
|
197 |
:: |
198 |
|
199 |
astakos-host$ snf-manage syncdb |
200 |
astakos-host$ snf-manage migrate |
201 |
|
202 |
cyclades-host$ snf-manage syncdb |
203 |
cyclades-host$ snf-manage migrate |
204 |
|
205 |
.. note:: |
206 |
|
207 |
After the migration, Astakos has created uuids for all users, |
208 |
and has set the uuid as the public identifier of a user. |
209 |
This uuid is to be used both at other services (Cyclades, Pithos) |
210 |
and at the clientside (kamaki client settings). |
211 |
|
212 |
Duplicate-email users have been deleted earlier in |
213 |
:ref:`remove_astakos_duplicate` |
214 |
|
215 |
3.3 Setup quota settings for all services |
216 |
----------------------------------------- |
217 |
|
218 |
Generally: |
219 |
|
220 |
:: |
221 |
|
222 |
# Service Setting Value |
223 |
# quotaholder: QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v |
224 |
# quotaholder: QUOTAHOLDER_TOKEN = <random string> |
225 |
|
226 |
# astakos: ASTAKOS_QUOTAHOLDER_TOKEN = <the same random string> |
227 |
# astakos: ASTAKOS_QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v |
228 |
|
229 |
# cyclades: CYCLADES_QUOTAHOLDER_TOKEN = <the same random string> |
230 |
# cyclades: CYCLADES_QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v |
231 |
# cyclades: CYCLADES_USE_QUOTAHOLDER = True |
232 |
|
233 |
|
234 |
# pithos: PITHOS_QUOTAHOLDER_TOKEN = <the same random string> |
235 |
# pithos: PITHOS_QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v |
236 |
# pithos: PITHOS_USE_QUOTAHOLDER = True |
237 |
# All services must match the quotaholder token and url configured for quotaholder. |
238 |
|
239 |
Specifically: |
240 |
|
241 |
On the Astakos host, edit ``/etc/synnefo/20-snf-astakos-app-settings.conf``: |
242 |
|
243 |
:: |
244 |
|
245 |
QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v' |
246 |
QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w' |
247 |
ASTAKOS_QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w' |
248 |
ASTAKOS_QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v' |
249 |
|
250 |
On the Cyclades host, edit ``/etc/synnefo/20-snf-cyclades-app-quotas.conf``: |
251 |
|
252 |
:: |
253 |
|
254 |
CYCLADES_USE_QUOTAHOLDER = True |
255 |
CYCLADES_QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v' |
256 |
CYCLADES_QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w' |
257 |
|
258 |
# Set to False if astakos & cyclades are on the same host |
259 |
#CYCLADES_PROXY_USER_SERVICES = True |
260 |
|
261 |
.. note:: |
262 |
|
263 |
If Cylcades and Astakos are installed on the same server, |
264 |
set ``CYCLADES_PROXY_USER_SERVICES = False`` |
265 |
|
266 |
|
267 |
On the Pithos host, edit ``/etc/synnefo/20-snf-pithos-app-settings.conf``: |
268 |
|
269 |
:: |
270 |
|
271 |
PITHOS_QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v' |
272 |
PITHOS_QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w' |
273 |
PITHOS_USE_QUOTAHOLDER = False # will set to True after migration |
274 |
|
275 |
.. note:: |
276 |
|
277 |
During the migration it must be set, ``PITHOS_USE_QUOTAHOLDER = False``. |
278 |
Set to ``True`` once the migration is over. |
279 |
|
280 |
3.4 Setup astakos |
281 |
----------------- |
282 |
|
283 |
- **Remove** this redirection from astakos front-end web server :: |
284 |
|
285 |
RewriteRule ^/login(.*) /im/login/redirect$1 [PT,NE] |
286 |
|
287 |
(see `<http://www.synnefo.org/docs/synnefo/latest/quick-install-admin-guide.html#apache2-setup>`_) |
288 |
|
289 |
- Enable users to change their contact email. Edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` :: |
290 |
|
291 |
ASTAKOS_EMAILCHANGE_ENABLED = True |
292 |
|
293 |
- Rename the following (Astakos-specific) setting:: |
294 |
|
295 |
ASTAKOS_DEFAULT_FROM_EMAIL |
296 |
|
297 |
to this (Django-specific) name:: |
298 |
|
299 |
SERVER_EMAIL |
300 |
|
301 |
- Instead of using the following (Astakos-specific) setting:: |
302 |
|
303 |
ASTAKOS_DEFAULT_ADMIN_EMAIL |
304 |
|
305 |
include one or more entries in this (Django-specific) setting:: |
306 |
|
307 |
ADMINS = ( |
308 |
('Joe Doe', 'doe@example.net'), |
309 |
('Mary Jean', 'mary@example.net'), |
310 |
) |
311 |
|
312 |
.. note:: |
313 |
|
314 |
The ``SERVER_EMAIL`` and ``ADMINS`` settings are Django-specific. |
315 |
As such they will be shared among any two (or more) services that happen |
316 |
to be collocated within the same application server (e.g. astakos & |
317 |
cyclades within the same gunicorn) |
318 |
|
319 |
3.5 Setup Cyclades |
320 |
------------------ |
321 |
|
322 |
- Run on the Astakos host :: |
323 |
|
324 |
# snf-manage service-list |
325 |
|
326 |
- Set the Cyclades service token in |
327 |
``/etc/synnefo/20-snf-cyclades-app-api.conf`` :: |
328 |
|
329 |
CYCLADES_ASTAKOS_SERVICE_TOKEN = 'asfasdf_CycladesServiceToken_iknl' |
330 |
|
331 |
- Since version 0.13, Synnefo uses **VMAPI** in order to prevent sensitive data |
332 |
needed by 'snf-image' to be stored in Ganeti configuration (e.g. VM |
333 |
password). This is achieved by storing all sensitive information to a CACHE |
334 |
backend and exporting it via VMAPI. The cache entries are invalidated after |
335 |
the first request. Synnefo uses **memcached** as a django cache backend. |
336 |
To install, run on the Cyclades host:: |
337 |
|
338 |
apt-get install memcached |
339 |
apt-get install python-memcache |
340 |
|
341 |
You will also need to configure Cyclades to use the memcached cache backend. |
342 |
Namely, you need to set IP address and port of the memcached daemon, and the |
343 |
default timeout (seconds tha value is stored in the cache). Edit |
344 |
``/etc/synnefo/20-snf-cyclades-app-vmapi.conf`` :: |
345 |
|
346 |
VMAPI_CACHE_BACKEND = "memcached://127.0.0.1:11211/?timeout=3600" |
347 |
|
348 |
|
349 |
Finally, set the BASE_URL for the VMAPI, which is actually the base URL of |
350 |
Cyclades, again in ``/etc/synnefo/20-snf-cyclades-app-vmapi.conf``. Make sure |
351 |
the domain is exaclty the same, so that no re-directs happen :: |
352 |
|
353 |
VMAPI_BASE_URL = "https://cyclades.example.synnefo.org" |
354 |
|
355 |
.. note:: |
356 |
|
357 |
- These settings are needed in all Cyclades workers. |
358 |
|
359 |
- VMAPI_CACHE_BACKEND just overrides django's CACHE_BACKEND setting |
360 |
|
361 |
- memcached must be reachable from all Cyclades workers. |
362 |
|
363 |
- For more information about configuring django to use memcached: |
364 |
https://docs.djangoproject.com/en/1.2/topics/cache |
365 |
|
366 |
3.6 Setup Pithos |
367 |
---------------- |
368 |
|
369 |
- Pithos forwards user catalog services to Astakos so that web clients may |
370 |
access them for uuid-displayname translations. Edit on the Pithos host |
371 |
``/etc/synnefo/20-snf-pithos-app-settings.conf`` :: |
372 |
|
373 |
PITHOS_USER_CATALOG_URL = https://accounts.example.synnefo.org/user_catalogs/ |
374 |
PITHOS_USER_FEEDBACK_URL = https://accounts.example.synnefo.org/feedback/ |
375 |
PITHOS_USER_LOGIN_URL = https://accounts.example.synnefo.org/login/ |
376 |
|
377 |
# Set to False if astakos & pithos are on the same host |
378 |
#PITHOS_PROXY_USER_SERVICES = True |
379 |
|
380 |
.. note:: |
381 |
|
382 |
If Pithos and Astakos are installed on the same server, |
383 |
set ``PITHOS_PROXY_USER_SERVICES = False`` |
384 |
|
385 |
4. Start astakos and quota services |
386 |
=================================== |
387 |
.. warning:: |
388 |
|
389 |
To ensure consistency, prevent public access to astakos during migrations. |
390 |
This can be done via firewall or webserver access control. |
391 |
|
392 |
Start (or restart, if running) the webserver and gunicorn on the Astakos host. |
393 |
E.g.:: |
394 |
|
395 |
# service apache2 start |
396 |
# service gunicorn start |
397 |
|
398 |
.. _astakos-load-resources: |
399 |
|
400 |
5. Load resource definitions into Astakos |
401 |
========================================= |
402 |
|
403 |
First, set the corresponding values on the following dict in |
404 |
``/etc/synnefo/20-snf-astakos-app-settings.conf`` :: |
405 |
|
406 |
# Set the cloud service properties |
407 |
ASTAKOS_SERVICES = { |
408 |
'cyclades': { |
409 |
# # Specifying the key 'url' will overwrite it. |
410 |
# # Use this to (re)set service URL. |
411 |
# 'url': 'https://cyclades.example.synnefo.org/ui/', |
412 |
# # order services in listings, cloudbar, etc. |
413 |
# 'order' : 1 |
414 |
'resources': [{ |
415 |
'name': 'disk', |
416 |
'group': 'compute', |
417 |
'uplimit': 30*1024*1024*1024, |
418 |
'unit': 'bytes', |
419 |
'desc': 'Virtual machine disk size' |
420 |
}, { |
421 |
'name': 'cpu', |
422 |
'group': 'compute', |
423 |
'uplimit': 6, |
424 |
'desc': 'Number of virtual machine processors' |
425 |
}, { |
426 |
'name': 'ram', |
427 |
'group': 'compute', |
428 |
'uplimit': 6*1024*1024*1024, |
429 |
'unit': 'bytes', |
430 |
'desc': 'Virtual machines' |
431 |
}, { |
432 |
'name': 'vm', |
433 |
'group': 'compute', |
434 |
'uplimit': 2, |
435 |
'desc': 'Number of virtual machines' |
436 |
}, { |
437 |
'name': 'network.private', |
438 |
'group': 'network', |
439 |
'uplimit': 1, |
440 |
'desc': 'Private networks' |
441 |
} |
442 |
] |
443 |
}, |
444 |
'pithos+': { |
445 |
# # Use this to (re)set service URL. |
446 |
# 'url': 'https://pithos.example.synnefo.org/ui/', |
447 |
# # order services in listings, cloudbar, etc. |
448 |
# 'order' : 2 |
449 |
'resources':[{ |
450 |
'name': 'diskspace', |
451 |
'group': 'storage', |
452 |
'uplimit': 5*1024*1024*1024, |
453 |
'unit': 'bytes', |
454 |
'desc': 'Pithos account diskspace' |
455 |
}] |
456 |
} |
457 |
} |
458 |
|
459 |
.. note:: |
460 |
|
461 |
The name of the Pithos service is ``pithos+``. |
462 |
If you have named your pithos service ``pithos``, without ``+``, |
463 |
then you must rename it:: |
464 |
|
465 |
$ snf-manage service-list | grep pithos # find service id |
466 |
$ snf-manage service-update --name='pithos+' <service id> |
467 |
|
468 |
Then, configure and load the available resources per service |
469 |
and associated default limits into Astakos. On the Astakos host run :: |
470 |
|
471 |
# snf-manage astakos-init --load-service-resources |
472 |
|
473 |
|
474 |
.. note:: |
475 |
|
476 |
Before v0.13, only `cyclades.vm`, `cyclades.network.private`, |
477 |
and `pithos+.diskspace` existed (not with these names, |
478 |
there were per-service settings). |
479 |
However, limits to the new resources must also be set. |
480 |
|
481 |
If the intention is to keep a resource unlimited, (counting on that VM |
482 |
creation will be limited by other resources' limit) it is best to calculate |
483 |
a value that is too large to be reached because of other limits (and |
484 |
available flavours), but not much larger than needed because this might |
485 |
confuse users who do not readily understand that multiple limits apply and |
486 |
flavors are limited. |
487 |
|
488 |
|
489 |
6. Migrate Services user names to uuids |
490 |
======================================= |
491 |
|
492 |
|
493 |
6.1 Double-check cyclades before user case/uuid migration |
494 |
--------------------------------------------------------- |
495 |
|
496 |
:: |
497 |
|
498 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --validate |
499 |
|
500 |
Duplicate user found? |
501 |
|
502 |
- either *merge* (merge will merge all resources to one user):: |
503 |
|
504 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --merge-user=kpap@grnet.gr |
505 |
|
506 |
- or *delete* :: |
507 |
|
508 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --delete-user=KPap@grnet.gr |
509 |
# (only KPap will be deleted not kpap) |
510 |
|
511 |
6.2 Double-check pithos before user case/uuid migration |
512 |
--------------------------------------------------------- |
513 |
|
514 |
:: |
515 |
|
516 |
pithos.host$ snf-manage pithos-manage-accounts --list-duplicate |
517 |
|
518 |
Duplicate user found? |
519 |
|
520 |
If you want to migrate files first: |
521 |
|
522 |
- *merge* (merge will merge all resources to one user):: |
523 |
|
524 |
pithos.host$ snf-manage pithos-manage-accounts --merge-accounts --src-account=SPapagian@grnet.gr --dest-account=spapagian@grnet.gr |
525 |
# (SPapagian@grnet.gr's contents will be merged into spapagian@grnet.gr, but SPapagian@grnet.gr account will still exist) |
526 |
|
527 |
- and then *delete* :: |
528 |
|
529 |
pithos.host$ snf-manage pithos-manage-accounts --delete-account=SPapagian@grnet.gr |
530 |
# (only SPapagian@grnet.gr will be deleted not spapagian@grnet.gr) |
531 |
|
532 |
If you do *NOT* want to migrate files just run the second step and delete |
533 |
the duplicate account. |
534 |
|
535 |
6.3 Migrate Cyclades users (email case/uuid) |
536 |
-------------------------------------------- |
537 |
|
538 |
:: |
539 |
|
540 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --migrate-users |
541 |
|
542 |
- if invalid usernames are found, verify that they do not exist in astakos:: |
543 |
|
544 |
astakos.host$ snf-manage user-list |
545 |
|
546 |
- if no user exists:: |
547 |
|
548 |
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --delete-user=<userid> |
549 |
|
550 |
Finally, if you have set manually quotas for specific users inside |
551 |
``/etc/synnefo/20-snf-cyclades-app-api.conf`` (in ``VMS_USER_QUOTA``, |
552 |
``NETWORKS_USER_QUOTA`` make sure to update them so that: |
553 |
|
554 |
1. There are no double entries wrt case sensitivity |
555 |
2. Replace all user email addresses with the corresponding UUIDs |
556 |
|
557 |
To find the UUIDs for step 2 run on the Astakos host :: |
558 |
|
559 |
# snf-manage user-list |
560 |
|
561 |
6.4 Migrate Pithos user names |
562 |
----------------------------- |
563 |
|
564 |
Check if alembic has not been initialized :: |
565 |
|
566 |
pithos.host$ pithos-migrate current |
567 |
|
568 |
- If alembic current is None (e.g. okeanos.io) :: |
569 |
|
570 |
pithos.host$ pithos-migrate stamp 3dd56e750a3 |
571 |
|
572 |
Then, migrate pithos account name to uuid:: |
573 |
|
574 |
pithos.host$ pithos-migrate upgrade head |
575 |
|
576 |
Finally, set this setting to ``True``:: |
577 |
|
578 |
PITHOS_USE_QUOTAHOLDER = True |
579 |
|
580 |
|
581 |
7. Migrate old quota limits |
582 |
=========================== |
583 |
|
584 |
7.1 Migrate Pithos quota limits to Astakos |
585 |
------------------------------------------ |
586 |
|
587 |
Migrate from pithos native to astakos/quotaholder. |
588 |
This requires a file to be transfered from Cyclades to Astakos:: |
589 |
|
590 |
pithos.host$ snf-manage pithos-export-quota --location=pithos-quota.txt |
591 |
pithos.host$ scp pithos-quota.txt astakos.host: |
592 |
astakos.host$ snf-manage user-set-initial-quota pithos-quota.txt |
593 |
|
594 |
.. _export-quota-note: |
595 |
|
596 |
.. note:: |
597 |
|
598 |
`pithos-export-quota` will only export quotas that are not equal to the |
599 |
defaults in Pithos. Therefore, it is possible to both change or maintain |
600 |
the default quotas across the migration. To maintain quotas the new default |
601 |
pithos+.diskpace limit in Astakos must be equal to the (old) default quota |
602 |
limit in Pithos. Change either one of them make them equal. |
603 |
|
604 |
see :ref:`astakos-load-resources` on how to set the (new) default quotas in Astakos. |
605 |
|
606 |
7.2 Migrate Cyclades quota limits to Astakos |
607 |
-------------------------------------------- |
608 |
|
609 |
:: |
610 |
|
611 |
cyclades.host$ snf-manage cyclades-export-quota --location=cyclades-quota.txt |
612 |
cyclades.host$ scp cyclades-quota.txt astakos.host: |
613 |
astakos.host$ snf-manage user-set-initial-quota cyclades-quota.txt |
614 |
|
615 |
`cyclades-export-quota` will only export quotas that are not equal to the defaults. |
616 |
See :ref:`note above <export-quota-note>`. |
617 |
|
618 |
8. Enforce the new quota limits migrated to Astakos |
619 |
=================================================== |
620 |
The following should report all users not having quota limits set |
621 |
because the effective quota database has not been initialized yet. :: |
622 |
|
623 |
astakos.host$ snf-manage astakos-quota --verify |
624 |
|
625 |
Initialize the effective quota database:: |
626 |
|
627 |
astakos.host$ snf-manage astakos-quota --sync |
628 |
|
629 |
This procedure may be used to verify and re-synchronize the effective quota |
630 |
database with the quota limits that are derived from policies in Astakos |
631 |
(initial quotas, project memberships, etc.) |
632 |
|
633 |
9. Initialize resource usage |
634 |
============================ |
635 |
|
636 |
The effective quota database (quotaholder) has just been initialized and knows |
637 |
nothing of the current resource usage. Therefore, each service must send it in. |
638 |
|
639 |
9.1 Initialize Pithos resource usage |
640 |
------------------------------------ |
641 |
|
642 |
:: |
643 |
|
644 |
pithos.host$ snf-manage pithos-usage --reset |
645 |
|
646 |
9.2 Initialize Cyclades resource usage |
647 |
-------------------------------------- |
648 |
|
649 |
:: |
650 |
|
651 |
cyclades.host$ snf-manage cyclades-reset-usage |
652 |
|
653 |
10. Install periodic project maintainance checks |
654 |
================================================ |
655 |
In order to detect and effect project expiration, |
656 |
a management command has to be run periodically |
657 |
(depending on the required granularity, e.g. once a day/hour):: |
658 |
|
659 |
astakos.host$ snf-manage project-control --terminate-expired |
660 |
|
661 |
A list of expired projects can be extracted with:: |
662 |
|
663 |
astakos.host$ snf-manage project-control --check-expired |
664 |
|
665 |
|
666 |
11. Restart all services |
667 |
======================== |
668 |
|
669 |
Start (or restart, if running) all Synnefo services on all hosts. |
670 |
|
671 |
:: |
672 |
|
673 |
# service gunicorn restart |
674 |
# service snf-dispatcher restart |
675 |
# etc. |