Revision c7f29a98

b/docs/upgrade/upgrade-0.15.rst
1 1
Upgrade to Synnefo v0.15
2 2
^^^^^^^^^^^^^^^^^^^^^^^^
3 3

  
4

  
4 5
Prerequisites
5 6
==============
6 7

  
7 8
Before upgrading to v0.15 there are three steps that must be performed, relative
8 9
to the Cyclades networking service.
9 10

  
10
Add unique names to the NICs of all Ganeti instances
11
---------------------------------------------------
11
Add unique names to all NICs of all Ganeti instances
12
----------------------------------------------------
12 13

  
13
Since Ganeti 2.8, it is supported to give a name to NICs of Ganeti instances
14
and refer to them by their name, and not only by their index. Synnefo v0.15
15
assigns a unique name to each NIC and refers to them by their unique name.
16
Before upgrading to v0.15, Synnefo must assign names to all existing NICs.
17
This can easily be performed with a helper script that is shipped with Synnefo
18
v0.14.10:
14
Since Ganeti 2.8, it is supported to give a name to a NIC of a Ganeti instance
15
and then refer to the NIC by that name (and not only by its index). Synnefo
16
v0.15 assigns a unique name to each NIC and refers to it by that unique name.
17
Before upgrading to v0.15, Synnefo must assign names to all existing NICs. This
18
can be easily performed with a helper script that is already shipped with
19
Synnefo v0.14.10:
19 20

  
20 21
.. code-block:: console
21 22

  
......
24 25
.. note:: If you are not upgrading from v0.14.10, you can find the migration
25 26
 script :ref:`here <add_names>`.
26 27

  
27

  
28 28
Extend public networks to all Ganeti backends
29 29
---------------------------------------------
30 30

  
31 31
Before v0.15, each public network of Cyclades existed in one of the Ganeti
32 32
backends. In order to support dynamic addition and removal of public IPv4
33
address across VMs, each public network must exist in all Ganeti backends.
33
addresses across VMs, each public network must exist in all Ganeti backends.
34 34

  
35 35
If you are using more than one Ganeti backends, before upgrading to v0.15 you
36
must ensure that the network configuration to all Ganeti backends is identical
36
must ensure that the network configuration of all Ganeti backends is identical
37 37
and appropriate to support all public networks of Cyclades.
38 38

  
39 39
Update Ganeti allocation policy
......
41 41

  
42 42
Minimum number of NICs
43 43
``````````````````````
44

  
45
Until v0.14 all Cyclades VM were forced to be connected to the public network.
46
Synnefo v0.15 supports dynamic addition and removal of public IPv4 addresses,
47
which may result in a VM holding no NICs. However, Ganeti's default allocation
48
policy will not allow instance's without NICs. You will have to override
49
allocation policy to set the minimum number of NICs to zero. Todo this,
50
first get the current allocation policy:
44
Before v0.15, all Cyclades VMs were forced to be connected to the public
45
network. Synnefo v0.15 supports more flexible configurations and dynamic
46
addition/removal of public IPv4 addresses, which can result in a VMs with no
47
NICs at all. However, Ganeti's default allocation policy will not allow
48
instances without NICs. You will have to override Ganeti's default allocation
49
policy to set the minimum number of NICs to zero. To do this, first get the
50
current allocation policy:
51 51

  
52 52
.. code-block:: console
53 53

  
......
57 57
   ganeti1.example.synnefo.org
58 58

  
59 59
And replace `min:nic-count=1` with `min:nic-count=0`. Also, set
60
`max:nic-count=32` to avoid reaching default limit of 8.
60
`max:nic-count=32` to avoid reaching the default limit of 8.
61 61

  
62 62

  
63 63
.. code-block:: console
......
66 66

  
67 67
Enabled and allowed disk templates
68 68
``````````````````````````````````
69

  
70
In v0.15, the ``ARCHIPELAGO_BACKENDS`` settings, that was used to seperate
69
In v0.15, the ``ARCHIPELAGO_BACKENDS`` setting, that was used to separate
71 70
backends that were using Archipelago from the ones that were using all other
72 71
disk templates, has been removed. Instead, allocation of instances to Ganeti
73 72
backends is based on which disk templates are enabled and allowed in each
74
Ganeti backend (see section in :ref:`admin guide <alloc_disk_templates>`).
75
You can see the enabled/allowed disk templates by inspecting
76
the corresponding fields in `gnt-cluster info` output. For example, to have
77
a backend holding only instances with archipelago disk templates, you can set
78
the `--ipolicy-disk-templates` to include only `ext` disk template.
73
Ganeti backend (see section in :ref:`admin guide <alloc_disk_templates>`). You
74
can see the enabled/allowed disk templates by inspecting the corresponding
75
fields in the `gnt-cluster info` output. For example, to have a backend holding
76
only instances with archipelago disk templates, you can set the
77
`--ipolicy-disk-templates` to include only the `ext` disk template.
79 78

  
80 79
.. code-block:: console
81 80

  
82 81
 gnt-cluster modify --ipolicy-disk-templates=ext
83 82

  
83

  
84 84
Upgrade Steps
85 85
=============
86 86

  
......
92 92

  
93 93
3. Create floating IP pools
94 94

  
95
4. Register services and resources.
95
4. Re-register services and resources.
96 96

  
97 97
5. Bring up all services.
98 98

  
......
190 190

  
191 191
.. _pithos_view_registration:
192 192

  
193
2.3 Register pithos view as an OAuth 2.0 client in astakos
193
2.3 Register Pithos view as an OAuth 2.0 client in Astakos
194 194
----------------------------------------------------------
195 195

  
196
Starting from synnefo version 0.15, the pithos view, in order to get access to
197
the data of a protect pithos resource, has to be granted authorization for the
198
specific resource by astakos.
196
Starting from Synnefo version 0.15, the Pithos view, in order to get access to
197
the data of a protected Pithos resource, has to be granted authorization for
198
the specific resource by Astakos.
199 199

  
200 200
During the authorization grant procedure, it has to authenticate itself with
201
astakos since the later has to prevent serving requests by unknown/unauthorized
202
clients.
201
Astakos, since the latter has to prevent serving requests by
202
unknown/unauthorized clients.
203 203

  
204
To register the pithos view as an OAuth 2.0 client in astakos, use the
205
following command::
204
To register the Pithos view as an OAuth 2.0 client in Astakos, use the
205
following command (with the corresponding URL to reflect your deployment)::
206 206

  
207 207
    snf-manage oauth2-client-add pithos-view --secret=<secret> --is-trusted --url https://pithos.synnefo.live/pithos/ui/view
208 208

  
209 209
2.4 Update configuration files
210 210
------------------------------
211 211

  
212
The ``ASTAKOS_BASE_URL`` setting has been replaced (both in Cyclades and
213
Pithos services) with the ``ASTAKOS_AUTH_URL`` setting.
212
The ``ASTAKOS_BASE_URL`` setting has been replaced (both in Cyclades and Pithos
213
services) with the ``ASTAKOS_AUTH_URL`` setting.
214 214

  
215 215
For Cyclades service we have to change the ``20-snf-cyclades-app-api.conf``
216 216
file, remove the ``ASTAKOS_BASE_URL`` setting and replace it with
217
``ASTAKOS_AUTH_URL``. Typically it is sufficient to add ``/identity/v2.0``
218
at the end of base url to get the auth url. For example if base url had the
219
value of 'https://accounts.example.synnefo.org/' then the ``ASTAKOS_AUTH_URL``
217
``ASTAKOS_AUTH_URL``. Typically it is sufficient to add ``/identity/v2.0`` at
218
the end of base URL to get the auth URL. For example, if base URL had the value
219
of 'https://accounts.example.synnefo.org/' then the ``ASTAKOS_AUTH_URL``
220 220
setting will have the value of
221 221
'https://accounts.example.synnefo.org/identity/v2.0'.
222 222

  
223
For Pithos service we have to change the ``20-snf-pithos-app-settings.conf``
223
For the Pithos service we have to change the ``20-snf-pithos-app-settings.conf``
224 224
file in the same way as above. In addition to this, we have to change the
225 225
``PITHOS_OAUTH2_CLIENT_CREDENTIALS`` setting in the same configuration file
226 226
to set the credentials issued for the pithos view in `the previous step`__.
227 227

  
228 228
__ pithos_view_registration_
229 229

  
230

  
231 230
2.5 Upgrade vncauthproxy and configure snf-cyclades-app
232 231
-------------------------------------------------------
233 232

  
......
235 234
older versions. You will have to upgrade snf-vncauthproxy to v1.5 and
236 235
configure the authentication (users) file (``/var/lib/vncauthproxy/users``).
237 236

  
238
In case you're upgrading from an older snf-vncauthproxy version or if it's the
237
In case you are upgrading from an older snf-vncauthproxy version or if it's the
239 238
first time you're installing snf-vncauthproxy, you will need to add a
240 239
vncauthproxy user (see below for more information on user management) and
241
restart vncauthproxy daemon.
240
restart the vncauthproxy daemon.
242 241

  
243
To manage the authentication file, you can use the vncauthproxy-passwd tool,
242
To manage the authentication file, you can use the ``vncauthproxy-passwd`` tool,
244 243
to easily add, update and delete users.
245 244

  
246 245
To add a user:
......
253 252

  
254 253
You should also configure the new ``CYCLADES_VNCAUTHPROXY_OPTS`` setting in
255 254
``snf-cyclades-app``, to provide the user and password configured for
256
``Synnefo`` in the vncauthproxy authentication file and enable SSL support if
255
``synnefo`` in the vncauthproxy authentication file and enable SSL support if
257 256
snf-vncauthproxy is configured to run with SSL enabled for the control socket.
258 257

  
259 258
.. warning:: The vncauthproxy daemon requires a restart for the changes in the
......
267 266
Finally, snf-vncauthproxy-1.5 adds a dedicated user and group to be used by the
268 267
vncauthproxy daemon. The Debian default file has changed accordingly (``CHUID``
269 268
option in ``/etc/default/vncauthproxy``). The Debian default file now also
270
includes a ``DAEMON_OPTS`` variable which is used to pass any necessary / extra
269
includes a ``DAEMON_OPTS`` variable which is used to pass any necessary/extra
271 270
options to the vncauthproxy daemon. In case you're ugprading from an older
272 271
version of vncauthproxy, you should make sure to 'merge' the new default file
273 272
with the older one.
......
281 280

  
282 281
snf-cyclades-gtools comes with a collectd plugin to collect CPU and network
283 282
stats for Ganeti VMs and an example collectd configuration. snf-stats-app is a
284
Django (snf-webproject) app that serves the VM stats graphsmm by reading the VM
285
stats (from RRD files) and serves graphs.
283
Django (snf-webproject) app that serves the VM stats graphs by reading the VM
284
stats (from RRD files).
286 285

  
287
To enable / deploy VM stats collecting and snf-stats-app see the relevant
286
To enable/deploy the VM stats collecting and snf-stats-app, see the relevant
288 287
documentation in the :ref:`admin guide <admin-guide-stats>`.
289 288

  
290
If you were using collectd to collect VM stats on Debian squeeze and you are
289
If you were using collectd to collect VM stats on Debian Squeeze and you are
291 290
upgrading to Wheezy, you will need to upgrade your RRD files. Follow the
292 291
instructions on the collectd v4-to-v5 migration `guide
293 292
<https://collectd.org/wiki/index.php/V4_to_v5_migration_guide>`_.
294
You will proabably just need to run the `migration script
293
You will probably just need to run the `migration script
295 294
<https://collectd.org/wiki/index.php/V4_to_v5_migration_guide#Migration_script>`_
296 295
provided.
297 296

  
......
304 303
``STATS_SECRET_KEY`` settings. ``CYCLADES_STATS_SECRET_KEY`` in
305 304
``20-snf-cyclades-app-api.conf`` is used by Cyclades to encrypt the instance id
306 305
/ hostname  in the URLs serving the VM stats. You should set it to a random
307
value / string and make sure that it's the same as the ``STATS_SECRET_KEY``
306
value/string and make sure that it's the same as the ``STATS_SECRET_KEY``
308 307
setting (used to decrypt the instance hostname) in
309 308
``20-snf-stats-settings.conf`` on your Stats host.
310 309

  
......
313 312

  
314 313
.. note::
315 314

  
316
  Skip this step unless you have ``shibboleth`` enabled in astakos
315
  Skip this step unless you have ``shibboleth`` enabled in Astakos
317 316
  ``IM_MODULES`` setting.
318 317

  
319
As of v0.15 astakos uses the ``REMOTE_USER`` header provided by apache's
320
``mod_shib2`` service in order to resolve the unique identifier which is used to
321
associate a shibboleth account to a local astakos user. Prior to this version
322
astakos adhered to the presence of the ``MOD_SHIB_EPPN`` header which although
323
safe enough on most of the ``SP`` deployment scenarios, it may cause issues in
324
certain cases, such as global wide IdP support or inability of supported IdPs
325
to release the ``eduPersonPrincipalName`` attribute. The ``REMOTE_USER`` header
326
can be set by administrators to match any of the available shibboleth
327
attributes.
318
As of v0.15 Astakos uses the ``REMOTE_USER`` header provided by Apache's
319
``mod_shib2`` service in order to resolve the unique identifier which is used
320
to associate a shibboleth account to a local Astakos user. Prior to this
321
version, Astakos adhered to the presence of the ``MOD_SHIB_EPPN`` header which
322
although safe enough on most of the ``SP`` deployment scenarios, it may cause
323
issues in certain cases, such as global wide IdP support or inability of
324
supported IdPs to release the ``eduPersonPrincipalName`` attribute. The
325
``REMOTE_USER`` header can be set by administrators to match any of the
326
available shibboleth attributes.
328 327

  
329 328
If ``EPPN`` matches the service provider needs and you want to continue using
330 329
it as the unique identifier, you need to ensure that the ``REMOTE_USER``
......
350 349

  
351 350
  $ service shibd restart
352 351

  
353
**notice** that every time you alter the ``REMOTE_USER`` attribute, all
354
existing shibboleth enabled astakos users will be invalidated and no longer be
355
able to login to their existing account using shibboleth. Specifically for the
356
case of switching from *eppn* to another attribute, astakos is able to prevent
352
**Note** that every time you alter the ``REMOTE_USER`` attribute, all existing
353
shibboleth enabled Astakos users will be invalidated and no longer be able to
354
login to their existing account using shibboleth. Specifically, for the case of
355
switching from *eppn* to another attribute, Astakos is able to prevent
357 356
invalidation and automatically migrate existing *eppn* accounts. In order to do
358 357
that, set the ``ASTAKOS_SHIBBOLETH_MIGRATE_EPPN`` setting to ``True`` in
359 358
``20-snf-astakos-app-settings.conf`` configuration file. Now every time an
360
existing *eppn* user logs in using shibboleth, astakos will update the associated 
361
*eppn* identifier to the contents of the ``REMOTE_USER`` header.
359
existing *eppn* user logs in using shibboleth, Astakos will update the
360
associated *eppn* identifier to the contents of the ``REMOTE_USER`` header.
362 361

  
363 362
.. warning::
364 363
  
365 364
  IdPs should keep releasing the ``EPPN`` attribute in order for the migration
366 365
  to work.
367 366

  
367

  
368 368
3. Create floating IP pools
369 369
===========================
370 370

  
371 371
Synnefo v0.15 introduces floating IPs, which are public IPv4 addresses that can
372
dynamically be added/removed to/from VMs and are quotable via the
373
'cyclades.floating_ip' resource. Connecting a VM to a public network is only
374
allowed if the user has firstly created a floating IP from this network.
372
be dynamically added/removed to/from VMs and are quotable via the
373
``cyclades.floating_ip`` resource. Connecting a VM to a public network is only
374
allowed if the user has first allocated a floating IP from this network.
375 375

  
376 376
Floating IPs are created from networks that are marked as Floating IP pools.
377 377
Creation of floating IP pools is done with the `snf-manage network-create`
......
386 386

  
387 387
Already allocated public IPv4 addresses are not automatically converted to
388 388
floating IPs. Existing VMs can keep their IPv4 addresses which will be
389
automatically be released when these VMs will be destroyed. In order to
390
convert existing public IPs to floating IPs run the following command:
389
automatically released when these VMs get destroyed. If the admin wants to
390
convert existing public IPs to floating IPs, he/she can do so by running the
391
following provided tool:
391 392

  
392 393
.. code-block:: console
393 394

  
394 395
 cyclades.host$ /usr/lib/synnefo/tools/update_to_floating_ips
395 396

  
396
or for just one network:
397
or just for one network:
397 398

  
398 399
.. code-block:: console
399 400

  
400 401
 cyclades.host$ /usr/lib/synnefo/tools/update_to_floating_ips --network-id=<network_ID>
401 402

  
403

  
402 404
4. Register services and resources
403 405
==================================
404 406

  
......
406 408
------------------------------------------------
407 409

  
408 410
You will need to register again all Synnefo components, updating the
409
service and resource definitions. On the astakos node, run::
411
service and resource definitions. On the Astakos node, run::
410 412

  
411 413
    astakos-host$ snf-component-register
412 414

  
......
420 422
   current registered UI URL. In the default installation, the base URL can
421 423
   be found by stripping ``/ui`` from the UI URL.
422 424

  
423
The meaning of resources ``cyclades.cpu`` and ``cyclades.ram`` has changed:
424
they now denote the number of CPUs and, respectively, RAM of *active* VMs
425
rather than all VMs. To represent total CPUs and total RAM, as previously,
426
new resources ``cyclades.total_cpu`` and ``cyclades.total_ram`` are
427
introduced. We now also control the usage of floating IPs through resource
428
``cyclades.floating_ip``.
425
The meaning of resources ``cyclades.cpu`` and ``cyclades.ram`` has changed in
426
v0.15: they now denote the number of CPUs/RAM of *active* VMs (VMs that are not
427
shutdown) rather than all VMs as happened until now. To represent total CPUs
428
and total RAM, as previously, two new resources ``cyclades.total_cpu`` and
429
``cyclades.total_ram`` are introduced. We now also control the usage of
430
floating IPs through the resource ``cyclades.floating_ip``.
429 431

  
430 432
4.2 Tweek resource settings
431 433
---------------------------
432 434

  
433
New resources (``cyclades.total_cpu``, ``cyclades.total_ram``, and
434
``cyclades.floating_ip``) are registered with infinite default base quota.
435
You will probably need to restrict them, especially
436
``cyclades.floating_ip``. In order to change the default for all *future*
437
users, for instance restricting floating IPs to 2, run::
435
The new resources (``cyclades.total_cpu``, ``cyclades.total_ram``, and
436
``cyclades.floating_ip``) are registered with infinite default base quota
437
(meaning that they are not restricted at all). You will probably need to
438
restrict them, especially ``cyclades.floating_ip``. In order to change the
439
default limit of a resource for all *future* users, for instance restricting
440
floating IPs to 2, run::
438 441

  
439 442
    astakos-host$ snf-manage resource-modify cyclades.floating_ip --default-quota 2
440 443

  
......
442 445
still have infinite floating IPs. You can update base quota of existing
443 446
users in bulk, possibly excluding some users, with::
444 447

  
445
    astakos-host$ snf-manage user-modify --all --base-quota cyclades.floating_ip 2 --exclude uuid1,uuid2
448
    astakos-host$ snf-manage user-modify --all --base-quota cyclades.floating_ip 2 --exclude userid1,userid2
446 449

  
447 450
.. note::
448 451

  
449
   You can inspect base quota with ``snf-manage quota-list`` before applying
452
   You can inspect base quota with ``snf-manage quota-list``, before applying
450 453
   any changes, for example::
451 454

  
452 455
     # Get users with cyclades.vm base quota that differ from the default value
......
455 458
     # Get users with cyclades.vm base quota greater than 3
456 459
     astakos-host$ snf-manage quota-list --filter-by "resource=cyclades.vm,base_quota>3"
457 460

  
458
It is now possible to control whether a resource is visible for the users
459
through the API or the UI. Note that the system always checks resource
460
quota, regardless of their visibility. By default, ``cyclades.total_cpu``,
461
``cyclades.total_ram`` and ``astakos.pending_app`` are not visible. You can
462
change this behavior with::
461
Furthermore in v0.15, it is possible to control whether a resource is visible
462
to the users via the API or the Web UI. The default value for these options is
463
denoted inside the default resource definitions. Note that the system always
464
checks and enforces resource quota, regardless of their visibility. By default,
465
the new resources ``cyclades.total_cpu``, ``cyclades.total_ram`` and
466
``astakos.pending_app`` are not visible neither via the API nor via the Web UI.
467
You can change this behavior with::
463 468

  
464 469
    astakos-host$ snf-manage resource-modify <resource> --api-visible=True (or --ui-visible=True)
465 470

  

Also available in: Unified diff