Statistics
| Branch: | Tag: | Revision:

root / docs / upgrade-0.13.rst @ 76f70858

History | View | Annotate | Download (15.1 kB)

1
Upgrade to Synnefo v0.13
2
^^^^^^^^^^^^^^^^^^^^^^^^
3

    
4
The bulk of the upgrade to v0.13 is about user and quota migrations.
5
In summary, the migration process has 3 steps:
6

    
7
1. Run some commands and scripts to diagnose and extract some migration data
8
   while the OLD code is running, and BEFORE any changes are made.
9

    
10
2. Bring down services, upgrade packages, configure services, and perform
11
   django database migrations.  These migrations do not need any interaction
12
   between services.
13

    
14
3. Initialize the Astakos quota system and bring the Astakos service up, since
15
   it will be needed during a second-phase of UUID and quota migrations, that
16
   also uses data extracted from step 1.
17

    
18

    
19
1. Bring all services down
20
==========================
21

    
22
All services must be brought down so that the database maintains a predictable
23
and consistent state as the migration is being executed.
24

    
25

    
26
2. Prepare astakos user migration to case insensitive emails
27
============================================================
28

    
29
It is possible that two or more users have been registered with emails that
30
differ only in the case of its letters.  There can only be one of those
31
accounts after the migration, so the rest must be deleted.
32

    
33
Note that even if the users are deleted in Astakos, there still are duplicate
34
entries in Cyclades and Pithos.  For each service we need to reduce those
35
multiple accounts into one, either merging them together, or deleting and
36
discarding data from all but one.
37

    
38
2.1 Find duplicate email entries in Astakos
39
-------------------------------------------
40
(script: ``find_astakos_users_with_conflicting_emails.py``)::
41

    
42
    astakos-host$ cat << EOF > find_astakos_users_with_conflicting_emails.py
43
    import os
44
    import sys
45
    
46
    os.environ['DJANGO_SETTINGS_MODULE'] = 'synnefo.settings'
47
    
48
    import astakos
49
    from astakos.im.models import AstakosUser as A
50
    
51
    def user_filter(user):
52
        return A.objects.filter(email__iexact=user.email).count() > 1
53
    
54
    all_users = list(A.objects.all())
55
    userlist = [(str(u.pk) + ': ' + str(u.email))
56
                for u in filter(user_filter, all_users)]
57
    sys.stderr.write("id: email\n")
58
    print "\n".join(userlist)
59
    
60
    EOF
61

    
62
    astakos-host$ python ./find_astakos_users_with_conflicting_emails.py
63

    
64

    
65
.. _remove_astakos_duplicate:
66

    
67
2.1 Remove duplicate users in Astakos by their id
68
-------------------------------------------------
69
(script: ``delete_astakos_users.py``)::
70

    
71
    astakos-host$ cat << EOF > delete_astakos_users.py
72
    import os
73
    import sys
74
    from time import sleep
75

    
76
    os.environ['DJANGO_SETTINGS_MODULE'] = 'synnefo.settings'
77

    
78
    import astakos
79
    from astakos.im.models import AstakosUser as A
80

    
81
    def user_filter(user):
82
        return A.objects.filter(email__iexact=user.email).count() > 1
83

    
84
    argv = sys.argv
85
    argc = len(sys.argv)
86

    
87
    if argc < 2:
88
        print "Usage: ./delete_astakos_users.py <id>..."
89
        raise SystemExit()
90

    
91
    id_list = [int(x) for x in argv[1:]]
92

    
93
    print ""
94
    print "This will permanently delete the following users:\n"
95
    print "id: email"
96
    print "--  -----"
97

    
98
    users = A.objects.filter(id__in=id_list)
99
    for user in users:
100
        print "%s: %s" % (user.id, user.email)
101

    
102
    print "\nExecute? (yes/no): ",
103
    line = raw_input().rstrip()
104
    if line != 'yes':
105
        print "\nCancelled"
106
        raise SystemExit()
107

    
108
    print "\nConfirmed."
109
    sleep(2)
110
    for user in users:
111
        print "deleting %s: %s" % (user.id, user.email)
112
        user.delete()
113

    
114
    EOF
115

    
116
    astakos-host$ python ./delete_astakos_users.py 30 40
117

    
118
.. warning::
119

    
120
    MAKE SURE THAT YOU HAVE RESOLVED ALL CONFLICTS
121

    
122

    
123
3. Upgrade Synnefo and configure settings
124
=========================================
125

    
126
3.1 Install the new versions of packages
127
----------------------------------------
128

    
129
::
130

    
131
    astakos.host$ apt-get install \
132
                            snf-common \
133
                            snf-webproject \
134
                            snf-quotaholder-app \
135
                            snf-astakos-app \
136
                            kamaki \
137

    
138

    
139
    cyclades.host$ apt-get install \
140
                            snf-common \
141
                            snf-webproject
142
                            snf-pithos-backend \
143
                            snf-cyclades-app \
144
                            kamaki \
145

    
146
                           
147
    pithos.host$ apt-get install \
148
                            snf-common \
149
                            snf-webproject
150
                            snf-pithos-backend \
151
                            snf-pithos-app \
152
                            snf-pithos-webclient \
153
                            kamaki \
154

    
155
3.2 Sync and migrate Django DB
156
------------------------------
157

    
158
::
159

    
160
    astakos-host$ snf-manage syncdb
161
    astakos-host$ snf-manage migrate
162

    
163
    cyclades-host$ snf-manage syncdb
164
    cyclades-host$ snf-manage migrate
165

    
166
.. note::
167

    
168
    After the migration, Astakos has created uuids for all users,
169
    and has set the uuid as the public identifier of a user.
170
    This uuid is to be used both at other services (Cyclades, Pithos)
171
    and at the clientside (kamaki client settings).
172

    
173
    Duplicate-email users have been deleted earlier in
174
    :ref:`remove_astakos_duplicate`
175

    
176
3.3 Setup quota settings for all services
177
-----------------------------------------
178

    
179
::
180

    
181
    # Service       Setting                       Value
182
    # quotaholder:  QUOTAHOLDER_TOKEN          = <random string>
183

    
184
    # astakos:      ASTAKOS_QUOTAHOLDER_TOKEN  = <the same random string>
185
    # astakos:      ASTAKOS_QUOTAHOLDER_URL    = https://quotaholder.host/quotaholder/v
186

    
187
    # cyclades:     CYCLADES_QUOTAHOLDER_TOKEN = <the same random string>
188
    # cyclades:     CYCLADES_QUOTAHOLDER_URL   = http://quotaholder.host/quotaholder/v
189
    # cyclades:     CYCLADES_USE_QUOTAHOLDER   = True
190

    
191

    
192
    # pithos:       PITHOS_QUOTAHOLDER_TOKEN   = <the same random string>
193
    # pithos:       PITHOS_QUOTAHOLDER_URL     = http://quotaholder.host/quotaholder/v
194
    # All services must match the quotaholder token and url configured for quotaholder.
195

    
196
3.4 Setup astakos
197
-----------------
198

    
199
- **Remove** this redirection from astakos front-end web server::
200

    
201
        RewriteRule ^/login(.*) /im/login/redirect$1 [PT,NE]
202

    
203
    (see `<http://docs.dev.grnet.gr/synnefo/latest/quick-install-admin-guide.html#apache2-setup>`_)
204

    
205
- Enable users to change their contact email with the setting::
206

    
207
      # astakos:        ASTAKOS_EMAILCHANGE_ENABLED = True
208

    
209
3.6 Setup Cyclades
210
------------------
211

    
212
- Make sure this setting is set::
213

    
214
    # cyclades:     CYCLADES_ASTAKOS_SERVICE_TOKEN = 'secretstring'
215

    
216
  from the value in::
217

    
218
    astakos.host$ snf-manage service-list
219

    
220
- The Cyclades user interface needs to translate uuids to displaynames::
221

    
222
    # cyclades:     UI_USER_CATALOG_URL = 'https://astakos.host/user_catalogs/'
223

    
224
- Since version 0.13, Synnefo uses **VMAPI** in order to prevent sensitive data
225
  needed by 'snf-image' to be stored in Ganeti configuration (e.g. VM
226
  password). This is achieved by storing all sensitive information to a CACHE
227
  backend and exporting it via VMAPI. The cache entries are invalidated after
228
  the first request. Synnefo uses **memcached** as a django cache backend.
229
  To install::
230

    
231
        apt-get install memcached
232
        apt-get install python-memcache
233

    
234

    
235
  You will also need to configure Cyclades to use the memcached cache backend.
236
  Namely, you need to set IP address and port of the memcached daemon, and the
237
  default timeout (seconds tha value is stored in the cache)::
238

    
239
    VMAPI_CACHE_BACKEND = "memcached://127.0.0.1:11211/?timeout=3600"
240

    
241

    
242
  Finally, set the BASE_URL for the VMAPI, which is actually the base URL of
243
  Cyclades::
244

    
245
    VMAPI_BASE_URL = "https://cyclades.okeanos.grnet.gr/"
246

    
247
  .. note::
248

    
249
    - These settings are needed in all Cyclades workers.
250

    
251
    - VMAPI_CACHE_BACKEND just overrides django's CACHE_BACKEND setting
252

    
253
    - memcached must be reachable from all Cyclades workers.
254

    
255
    - For more information about configuring django to use memcached:
256
      https://docs.djangoproject.com/en/1.2/topics/cache
257

    
258
3.6 Setup Pithos
259
----------------
260

    
261
- Pithos forwards user catalog services to Astakos so that web clients may
262
  access them for uuid-displayname translations::
263

    
264
    # pithos:       PITHOS_USER_CATALOG_URL    = https://astakos.host/user_catalogs/
265
    # pithos:       PITHOS_USER_FEEDBACK_URL   = https://astakos.host/feedback/
266
    # pithos:       PITHOS_USER_LOGIN_URL      = https://astakos.host/login/
267
    # pithos:       #PITHOS_PROXY_USER_SERVICES = True # Set False if astakos & pithos are on the same host
268

    
269

    
270
4. Start astakos and quota services
271
===================================
272
E.g.::
273

    
274
    astakos.host$ service gunicorn restart
275

    
276

    
277
.. _astakos-load-resources:
278

    
279
5. Load resource definitions into Astakos
280
=========================================
281

    
282
Configure and load the available resources per service
283
and associated default limits into Astakos::
284

    
285
    astakos.host$ snf-manage astakos-load-service-resources
286

    
287
Example astakos settings (from `okeanos.io <https://okeanos.io/>`_)::
288

    
289
    # Set the cloud service properties
290
    ASTAKOS_SERVICES = {
291
        'cyclades': {
292
            #This can also be set from a management command
293
            'url': 'https://cyclades.host/ui/',
294
            'order': 0,
295
            'resources': [{
296
                'name':'disk',
297
                'group':'compute',
298
                'uplimit':300*1024*1024*1024,
299
                'unit':'bytes',
300
                'desc': 'Virtual machine disk size'
301
                },{
302
                'name':'cpu',
303
                'group':'compute',
304
                'uplimit':24,
305
                'desc': 'Number of virtual machine processors'
306
                },{
307
                'name':'ram',
308
                'group':'compute',
309
                'uplimit':40*1024*1024*1024,
310
                'unit':'bytes',
311
                'desc': 'Virtual machines'
312
                },{
313
                'name':'vm',
314
                'group':'compute',
315
                'uplimit':5,
316
                'desc': 'Number of virtual machines'
317
                },{
318
                'name':'network.private',
319
                'group':'network',
320
                'uplimit':5,
321
                'desc': 'Private networks'
322
                }
323
            ]
324
        },
325
        'pithos+': {
326
            'url': 'https://pithos.host/ui/',
327
            'order': 1,
328
            'resources':[{
329
                'name':'diskspace',
330
                'group':'storage',
331
                'uplimit':20 * 1024 * 1024 * 1024,
332
                'unit':'bytes',
333
                'desc': 'Pithos account diskspace'
334
                }]
335
        }
336
    }
337

    
338
.. note::
339

    
340
    Before v0.13, only `cyclades.vm`, `cyclades.network.private`,
341
    and `pithos+.diskspace` existed (not with this names, of course).
342
    However, limits to the new resources must also be set.
343

    
344
    If the intetion is to keep a resource unlimited, (counting on that VM
345
    creation will be limited by other resources' limit) it is best to calculate
346
    a value that is too large to be reached because of other limits (and
347
    available flavours), but not much larger than needed because this might
348
    confuse users who do not readily understand that multiple limits apply and
349
    flavors are limited.
350

    
351

    
352
6. Migrate Services user names to uuids
353
=======================================
354

    
355

    
356

    
357
6.1 Double-check cyclades before user case/uuid migration
358
---------------------------------------------------------
359

    
360
::
361

    
362
    cyclades.host$ snf-manage cyclades-astakos-migrate-0.13 --validate
363

    
364
Duplicate user found?
365

    
366
- either *merge* (merge will merge all resources to one user)::
367

    
368
    cyclades.host$ snf-manage cyclades-astakos-migrate-0.13 --merge-user=kpap@grnet.gr
369

    
370
- or *delete* ::
371

    
372
    cyclades.host$ snf-manage cyclades-astakos-migrate-0.13 --delete-user=KPap@grnet.gr
373
    # (only KPap will be deleted not kpap)
374

    
375
6.2 Migrate Cyclades users (email case/uuid)
376
--------------------------------------------
377

    
378
::
379

    
380
    cyclades.host$ snf-manage cyclades-astakos-migrate-0.13 --migrate-users
381

    
382
- if invalid usernames are found, verify that they do not exist in astakos::
383

    
384
    astakos.host$ snf-manage user-list
385

    
386
- if no user exists::
387

    
388
    cyclades.host$ snf-manage cyclades-astakos-migrate-0.13 --delete-user=<userid>
389

    
390
6.3 Migrate Pithos user names
391
-----------------------------
392

    
393
Check if alembic has not been initialized ::
394

    
395
    pithos.host$ pithos-migrate-0.13 current
396

    
397
- If alembic current is None (e.g. okeanos.io) ::
398

    
399
    pithos.host$ pithos-migrate-0.13 stamp 3dd56e750a3
400

    
401
Finally, migrate pithos account name to uuid::
402

    
403
    pithos.host$ pithos-migrate-0.13 upgrade head
404

    
405
7. Migrate old quota limits
406
===========================
407

    
408
7.1 Migrate Pithos quota limits to Astakos
409
------------------------------------------
410

    
411
Migrate from pithos native to astakos/quotaholder.
412
This requires a file to be transfered from Cyclades to Astakos::
413

    
414
    pithos.host$ snf-manage pithos-export-quota --location=pithos-quota.txt
415
    pithos.host$ rsync -avP pithos-quota.txt astakos.host:
416
    astakos.host$ snf-manage user-set-initial-quota pithos-quota.txt
417

    
418
.. _export-quota-note:
419

    
420
.. note::
421

    
422
    `pithos-export-quota` will only export quotas that are not equal to the
423
    defaults in Pithos. Therefore, it is possible to both change or maintain
424
    the default quotas across the migration. To maintain quotas the new default
425
    pithos+.diskpace limit in Astakos must be equal to the (old) default quota
426
    limit in Pithos. Change either one of them make them equal.
427

    
428
    see :ref:`astakos-load-resources` on how to set the (new) default quotas in Astakos.
429

    
430
7.2 Migrate Cyclades quota limits to Astakos
431
--------------------------------------------
432

    
433
::
434

    
435
    cyclades.host$ snf-manage cyclades-export-quota --location=cyclades-quota.txt
436
    cyclades.host$ rsync -avP cyclades-quota.txt astakos.host:
437
    astakos.host$ snf-manage user-set-initial-quota cyclades-quota.txt
438

    
439
`cyclades-export-quota` will only export quotas that are not equal to the defaults.
440
See :ref:`note above <export-quota-note>`.
441

    
442
8. Enforce the new quota limits migrated to Astakos
443
===================================================
444
The following should report all users not having quota limits set
445
because the effective quota database has not been initialized yet. ::
446

    
447
    astakos.host$ snf-manage astakos-quota --verify
448

    
449
Initialize the effective quota database::
450

    
451
    astakos.host$ snf-manage astakos-quota --sync
452

    
453
This procedure may be used to verify and re-synchronize the effective quota
454
database with the quota limits that are derived from policies in Astakos
455
(initial quotas, project memberships, etc.)
456

    
457
9. Initialize resource usage
458
============================
459

    
460
The effective quota database (quotaholder) has just been initialized and knows
461
nothing of the current resource usage. Therefore, each service must send it in.
462

    
463
9.1 Initialize Pithos resource usage
464
------------------------------------
465

    
466
::
467

    
468
    cyclades.host$ snf-manage pithos-reset-usage
469

    
470
9.2 Initialize Cyclades resource usage
471
--------------------------------------
472

    
473
::
474

    
475
    cyclades.host$ snf-manage cyclades-reset-usage
476

    
477
10. Install periodic project maintainance checks
478
================================================
479
In order to detect and effect project expiration,
480
a management command has to be run periodically
481
(depending on the required granularity, e.g. once a day/hour)::
482

    
483
    astakos.host$ snf-manage project-control --terminate-expired
484

    
485
A list of expired projects can be extracted with::
486

    
487
    astakos.host$ snf-manage project-control --list-expired
488