Revision c469ca86
| b/Changelog | ||
|---|---|---|
| 27 | 27 |
* Fix Enter/Esc on machine rename not working in the UI (#1501) |
| 28 | 28 |
* Fix UI barfs if a VM is based on an Image with state=DELETED (#1494) |
| 29 | 29 |
* Fix UI user logout action does not work (#1497) |
| 30 |
Images: |
|
| 31 |
* Fix erroneous reference to $(HELPER_DIR) in /etc/default /snf-image |
|
| 32 |
(#1498) |
|
| 33 |
* Fix typo in snf-image-helper ChangePassword task (#1500) |
|
| 34 |
* Show timestamps on execution of helper tasks in snf-image helper VM |
|
| 35 |
(#1499) |
|
| 36 |
Deployment: |
|
| 37 |
* Improve check for Ganeti master in snf-ganeti-eventd init script |
|
| 38 |
(#1323) |
|
| 30 |
Images: |
|
| 31 |
* Fix erroneous reference to $(HELPER_DIR) in /etc/default /snf-image (#1498) |
|
| 32 |
* Fix typo in snf-image-helper ChangePassword task (#1500) |
|
| 33 |
* Show timestamps on execution of helper tasks in snf-image helper VM (#1499) |
|
| 34 |
Deployment: |
|
| 35 |
* Improve check for Ganeti master in snf-ganeti-eventd init script (#1323) |
|
| 39 | 36 |
* Fix snf-ganeti-eventd init script ignored defaults file (#1505) |
| 40 | 37 |
|
| 41 | 38 |
|
| b/docs/conf.py | ||
|---|---|---|
| 242 | 242 |
|
| 243 | 243 |
# todo extension configuration |
| 244 | 244 |
todo_include_todos = True |
| 245 |
|
|
| 246 |
# autodoc config |
|
| 247 |
autodoc_default_flags = ['members'] |
|
| b/docs/index.rst | ||
|---|---|---|
| 12 | 12 |
src/install |
| 13 | 13 |
src/configuration |
| 14 | 14 |
src/deployment |
| 15 |
src/ci |
|
| 15 | 16 |
src/settings |
| 16 | 17 |
src/admin |
| 17 | 18 |
src/develop |
| 18 |
src/ci |
|
| 19 | 19 |
src/storage |
| 20 | 20 |
src/upgrade |
| 21 | 21 |
src/changelog |
| b/docs/src/ci.rst | ||
|---|---|---|
| 8 | 8 |
problem we currently mirror the central Git repository locally on the jenkins |
| 9 | 9 |
installation machine. To setup such a mirror do the following: |
| 10 | 10 |
|
| 11 |
-edit .netrc
|
|
| 11 |
edit .netrc::
|
|
| 12 | 12 |
|
| 13 |
machine code.grnet.gr |
|
| 14 |
login accountname |
|
| 15 |
password accountpasswd |
|
| 13 |
machine code.grnet.gr
|
|
| 14 |
login accountname
|
|
| 15 |
password accountpasswd
|
|
| 16 | 16 |
|
| 17 |
-Create the mirror
|
|
| 17 |
Create the mirror::
|
|
| 18 | 18 |
|
| 19 |
git clone --mirror https://code.grnet.gr/git/synnefo synnefo |
|
| 19 |
git clone --mirror https://code.grnet.gr/git/synnefo synnefo
|
|
| 20 | 20 |
|
| 21 |
-Setup cron to pull from the mirror periodically. Ideally, Git mirror updates
|
|
| 22 |
should run just before Jenkins jobs check the mirror for changes.
|
|
| 21 |
Setup cron to pull from the mirror periodically. Ideally, Git mirror updates |
|
| 22 |
should run just before Jenkins jobs check the mirror for changes::
|
|
| 23 | 23 |
|
| 24 |
4,14,24,34,44,54 * * * * cd /path/to/mirror && git fetch && git remote prune origin |
|
| 24 |
4,14,24,34,44,54 * * * * cd /path/to/mirror && git fetch && git remote prune origin
|
|
| 25 | 25 |
|
| 26 | 26 |
Jenkins setup |
| 27 | 27 |
------------- |
| ... | ... | |
| 66 | 66 |
4. Make sure that all system-level dependencies specified in README.develop |
| 67 | 67 |
are correctly installed |
| 68 | 68 |
|
| 69 |
5. Create a new "free-style software" job and set the following values: |
|
| 70 |
|
|
| 71 |
-Project name: synnefo |
|
| 72 |
-Source Code Management: Git |
|
| 73 |
-URL of repository: Jenkins Git does not support HTTPS for checking out directly |
|
| 74 |
from the repository. The temporary solution is to checkout |
|
| 75 |
with a cron script in a directory and set the checkout path |
|
| 76 |
in this field |
|
| 77 |
-Branches to build: master and perhaps others |
|
| 78 |
-Git->Advanced->Local subdirectory for repo (optional): synnefo |
|
| 79 |
-Git->Advanced->Prune remote branches before build: check |
|
| 80 |
-Repository browser: redmineweb, |
|
| 81 |
URL: https://code.grnet.gr/projects/synnefo/repository/ |
|
| 82 |
-Build Triggers->Poll SCM: check |
|
| 83 |
Schedule: # every five minutes |
|
| 84 |
0,5,10,15,20,25,30,35,40,45,50,55 * * * * |
|
| 85 |
|
|
| 86 |
-Build -> Add build step-> Execute shell |
|
| 87 |
|
|
| 88 |
Command: |
|
| 89 |
|
|
| 90 |
#!/bin/bash -ex |
|
| 91 |
cd synnefo |
|
| 92 |
mkdir -p reports |
|
| 93 |
/usr/bin/sloccount --duplicates --wide --details api util ui logic auth > reports/sloccount.sc |
|
| 94 |
cp conf/ci/manage.py . |
|
| 95 |
if [ ! -e requirements.pip ]; then cp conf/ci/pip-1.2.conf requirements.pip; fi |
|
| 96 |
cat settings.py.dist conf/ci/settings.py.sqlite > settings.py |
|
| 97 |
python manage.py update_ve |
|
| 98 |
python manage.py hudson api db logic |
|
| 99 |
|
|
| 100 |
-Post-build Actions->Publish JUnit test result report: check |
|
| 101 |
Test report XMLs: synnefo/reports/TEST-*.xml |
|
| 102 |
|
|
| 103 |
-Post-build Actions->Publish Cobertura Coverage Report: check |
|
| 104 |
Cobertura xml report pattern: synnefo/reports/coverage.xml |
|
| 105 |
|
|
| 106 |
-Post-build Actions->Report Violations: check |
|
| 107 |
pylint[XML filename pattern]: synnefo/reports/pylint.report |
|
| 108 |
|
|
| 109 |
-Post-build Actions->Publish SLOCCount analysis results |
|
| 110 |
SLOCCount reports: synnefo/reports/sloccount.sc |
|
| 111 |
(also, remember to install sloccount at /usr/bin) |
|
| 112 |
--------------- |
|
| 113 |
See also: |
|
| 114 |
|
|
| 115 |
http://sites.google.com/site/kmmbvnr/home/django-hudson-tutorial |
|
| 69 |
5. Create a new "free-style software" job and set the following values:: |
|
| 70 |
|
|
| 71 |
Project name: synnefo |
|
| 72 |
Source Code Management: Git |
|
| 73 |
URL of repository: Jenkins Git does not support HTTPS for checking out directly |
|
| 74 |
from the repository. The temporary solution is to checkout |
|
| 75 |
with a cron script in a directory and set the checkout path |
|
| 76 |
in this field |
|
| 77 |
Branches to build: master and perhaps others |
|
| 78 |
Git->Advanced->Local subdirectory for repo (optional): synnefo |
|
| 79 |
Git->Advanced->Prune remote branches before build: check |
|
| 80 |
Repository browser: redmineweb, |
|
| 81 |
URL: https://code.grnet.gr/projects/synnefo/repository/ |
|
| 82 |
Build Triggers->Poll SCM: check |
|
| 83 |
Schedule: # every five minutes |
|
| 84 |
0,5,10,15,20,25,30,35,40,45,50,55 * * * * |
|
| 85 |
|
|
| 86 |
Build -> Add build step-> Execute shell |
|
| 87 |
|
|
| 88 |
Command:: |
|
| 89 |
|
|
| 90 |
#!/bin/bash -ex |
|
| 91 |
cd synnefo |
|
| 92 |
mkdir -p reports |
|
| 93 |
/usr/bin/sloccount --duplicates --wide --details api util ui logic auth > reports/sloccount.sc |
|
| 94 |
cp conf/ci/manage.py . |
|
| 95 |
if [ ! -e requirements.pip ]; then cp conf/ci/pip-1.2.conf requirements.pip; fi |
|
| 96 |
cat settings.py.dist conf/ci/settings.py.sqlite > settings.py |
|
| 97 |
python manage.py update_ve |
|
| 98 |
python manage.py hudson api db logic |
|
| 99 |
|
|
| 100 |
Post-build Actions->Publish JUnit test result report: check |
|
| 101 |
Test report XMLs: synnefo/reports/TEST-*.xml |
|
| 102 |
|
|
| 103 |
Post-build Actions->Publish Cobertura Coverage Report: check |
|
| 104 |
Cobertura xml report pattern: synnefo/reports/coverage.xml |
|
| 105 |
|
|
| 106 |
Post-build Actions->Report Violations: check |
|
| 107 |
pylint[XML filename pattern]: synnefo/reports/pylint.report |
|
| 108 |
|
|
| 109 |
Post-build Actions->Publish SLOCCount analysis results |
|
| 110 |
SLOCCount reports: synnefo/reports/sloccount.sc |
|
| 111 |
(also, remember to install sloccount at /usr/bin) |
|
| 112 |
|
|
| 113 |
.. seealso:: |
|
| 114 |
http://sites.google.com/site/kmmbvnr/home/django-hudson-tutorial |
|
| b/docs/src/configuration.rst | ||
|---|---|---|
| 101 | 101 |
sample users.json file: |
| 102 | 102 |
|
| 103 | 103 |
.. literalinclude:: ../../synnefo/db/fixtures/users.json |
| 104 |
:language: json |
|
| 105 | 104 |
|
| 106 | 105 |
`download <../_static/users.json>`_ |
| 107 | 106 |
|
| ... | ... | |
| 114 | 113 |
on LOGIC nodes. |
| 115 | 114 |
|
| 116 | 115 |
The dispatcher retrieves messages from the queue and calls the appropriate |
| 117 |
handler function as defined in the queue configuration in `/etc/synnefo/*.conf'
|
|
| 116 |
handler function as defined in the queue configuration in `/etc/synnefo/*.conf`
|
|
| 118 | 117 |
files. |
| 119 | 118 |
|
| 120 | 119 |
The default configuration should work directly without any modifications. |
| b/docs/src/deployment.rst | ||
|---|---|---|
| 67 | 67 |
the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py]. |
| 68 | 68 |
on each GANETI_NODE: |
| 69 | 69 |
a deployment-specific KVM ifup script |
| 70 |
properly configured `NFDHCPD`_
|
|
| 70 |
properly configured :ref:`NFDHCPD <nfdhcpd-setup>`
|
|
| 71 | 71 |
|
| 72 | 72 |
|
| 73 | 73 |
.. _WEBAPP_NODE: |
| ... | ... | |
| 77 | 77 |
Synnefo WEBAPP node is the server that runs the web application contained within |
| 78 | 78 |
the synnefo package. At the current state Synnefo provides two web frontends. |
| 79 | 79 |
|
| 80 |
.. webapp-deploy: |
|
| 80 |
|
|
| 81 |
.. _webapp-deploy: |
|
| 81 | 82 |
|
| 82 | 83 |
Web application deployment |
| 83 | 84 |
-------------------------- |
| b/docs/src/develop.rst | ||
|---|---|---|
| 4 | 4 |
Information on how to setup a development environment. |
| 5 | 5 |
|
| 6 | 6 |
This file documents the installation of a development environment for Synnefo. |
| 7 |
It should be read alongside README.deploy.
|
|
| 7 |
It should be read alongside :ref:`installation guide <installation>`.
|
|
| 8 | 8 |
|
| 9 |
It contains development-specific ammendments to the basic deployment steps
|
|
| 10 |
outlined in README.deploy, and development-specific notes.
|
|
| 9 |
It contains development-specific ammendments to the basic installation steps
|
|
| 10 |
outlined in `installation guide <installation>`, and development-specific notes.
|
|
| 11 | 11 |
|
| 12 | 12 |
Installing the development environment |
| 13 | 13 |
-------------------------------------- |
| 14 | 14 |
|
| 15 |
For a basic development environment you need to follow steps 0-15 |
|
| 16 |
of README.deploy, which should be read in its entirety *before* this document. |
|
| 15 |
For a basic development environment you need to follow steps |
|
| 16 |
of `installation guide <installation>`, which should be read in its |
|
| 17 |
entirety *before* this document. |
|
| 17 | 18 |
|
| 18 | 19 |
Development-specific guidelines on each step: |
| 19 | 20 |
|
| ... | ... | |
| 28 | 29 |
|
| 29 | 30 |
|
| 30 | 31 |
1. You do not need to install your own Ganeti installation. |
| 31 |
Use the RAPI endpoint as contained in settings.py.dist.
|
|
| 32 |
Use the RAPI endpoint as contained in common settings.
|
|
| 32 | 33 |
|
| 33 | 34 |
|
| 34 | 35 |
2. You do not need to setup your own RabbitMQ nodes, use the AMQP endpoints |
| 35 | 36 |
contained in settings.py.dist. |
| 36 | 37 |
|
| 37 | 38 |
3. For development purposes, Django's own development |
| 38 |
`server, ./manage.py runserver' will suffice.
|
|
| 39 |
server, `./manage.py runserver` will suffice.
|
|
| 39 | 40 |
|
| 40 | 41 |
|
| 41 | 42 |
4. Use a virtual environment to install the Django project, or packages provided |
| ... | ... | |
| 103 | 104 |
|
| 104 | 105 |
|
| 105 | 106 |
17.[OPTIONAL] Create settings.d/99-local.conf and insert local overrides for |
| 106 |
settings.d/*. This will allow pulling new files without needing to reapply |
|
| 107 |
settings.d/\*. This will allow pulling new files without needing to reapply
|
|
| 107 | 108 |
local any local modifications. |
| 108 | 109 |
|
| 109 | 110 |
|
| 110 | 111 |
South Database Migrations |
| 111 |
------------------------=
|
|
| 112 |
-------------------------
|
|
| 112 | 113 |
|
| 113 | 114 |
* Initial Migration |
| 114 | 115 |
|
| ... | ... | |
| 159 | 160 |
|
| 160 | 161 |
$ ./bin/python manage.py schemamigration db --auto |
| 161 | 162 |
+ Added field new_south_test_field on db.SynnefoUser |
| 163 |
|
|
| 162 | 164 |
Created 0002_auto__add_field_synnefouser_new_south_test_field.py. |
| 163 | 165 |
|
| 164 | 166 |
You can now apply this migration with: ./manage.py migrate db |
| ... | ... | |
| 168 | 170 |
- Migrating forwards to 0002_auto__add_field_synnefouser_new_south_test_field. |
| 169 | 171 |
> db:0002_auto__add_field_synnefouser_new_south_test_field |
| 170 | 172 |
- Loading initial data for db. |
| 173 |
|
|
| 171 | 174 |
Installing json fixture 'initial_data' from '/home/bkarak/devel/synnefo/../synnefo/db/fixtures'. |
| 172 | 175 |
Installed 1 object(s) from 1 fixture(s) |
| 173 | 176 |
|
| b/docs/src/i18n.rst | ||
|---|---|---|
| 11 | 11 |
|
| 12 | 12 |
LANGUAGES = ( |
| 13 | 13 |
('el', u'Ελληνικά'),
|
| 14 |
('en', 'English'),
|
|
| 15 |
) |
|
| 14 |
('en', 'English'),)
|
|
| 16 | 15 |
|
| 17 | 16 |
1) For each language we want to add, we run makemessages, from our project's |
| 18 | 17 |
base: |
| 19 | 18 |
|
| 20 | 19 |
$ ./bin/django-admin.py makemessages -l el -e html,txt,py |
| 21 |
(./bin/django-admin.py makemessages -l el -e html,txt,py --ignore=lib/*) |
|
| 20 |
(./bin/django-admin.py makemessages -l el -e html,txt,py --ignore=lib/\*)
|
|
| 22 | 21 |
|
| 23 | 22 |
This will add the Greek language, and we specify that html, txt and python |
| 24 | 23 |
files contain translatable strings |
| b/docs/src/install.rst | ||
|---|---|---|
| 1 |
.. _installation: |
|
| 2 |
|
|
| 1 | 3 |
Installation |
| 2 | 4 |
============ |
| 3 | 5 |
|
| ... | ... | |
| 59 | 61 |
|
| 60 | 62 |
.. note:: |
| 61 | 63 |
The link name may only contain "upper and lower case, digits, |
| 62 |
underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9_-]+$." |
|
| 64 |
underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9\_-]+$."
|
|
| 63 | 65 |
|
| 64 | 66 |
.. seealso:: |
| 65 | 67 |
`Ganeti customisation using hooks <http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming>`_ |
| ... | ... | |
| 112 | 114 |
cd $SYNNEFO |
| 113 | 115 |
sudo pip install -e git+https://code.grnet.gr/git/vncauthproxy@5a196d8481e171a#egg=vncauthproxy |
| 114 | 116 |
|
| 117 |
|
|
| 118 |
.. _nfdhcpd-setup: |
|
| 119 |
|
|
| 115 | 120 |
NFDHCPD installation |
| 116 | 121 |
******************** |
| 117 | 122 |
Setup Synnefo-specific networking on the Ganeti backend. |
| ... | ... | |
| 148 | 153 |
$ rabbitmqctl set_permissions -p / <username> "^.*" ".*" ".*" |
| 149 | 154 |
|
| 150 | 155 |
The values set for the user and password must be mirrored in the |
| 151 |
`RABBIT_*` variables in your `settings`_ (see step 6)
|
|
| 156 |
`RABBIT_*` variables in your :ref:`settings <configuration>`.
|
|
| 152 | 157 |
|
| 153 | 158 |
|
| 154 | 159 |
snf-image installation |
| b/docs/src/intro.rst | ||
|---|---|---|
| 1 |
Intro |
|
| 2 |
===== |
|
| 3 |
|
|
| 4 |
.. todo:: write introductory documentation |
|
| b/docs/src/settings.rst | ||
|---|---|---|
| 1 |
.. automodule:: synnefo.settings.common.aai |
|
| 1 |
Available settings |
|
| 2 |
================== |
|
| 3 |
|
|
| 4 |
ADMINS |
|
| 5 |
------ |
|
| 6 |
.. automodule:: synnefo.settings.common.admins |
|
| 2 | 7 |
:members: |
| 3 |
:undoc-members: |
|
| 8 |
:no-undoc-members: |
|
| 9 |
|
|
| 10 |
APPS |
|
| 11 |
---- |
|
| 12 |
.. literalinclude:: ../../synnefo/settings/common/apps.py |
|
| b/docs/src/storage.rst | ||
|---|---|---|
| 286 | 286 |
components in a RADOS cluster. |
| 287 | 287 |
|
| 288 | 288 |
0. Add Ceph Debian repository in /etc/apt/sources.list on every node (mon, osd, |
| 289 |
clients): |
|
| 289 |
clients):: |
|
| 290 |
|
|
| 290 | 291 |
deb http://ceph.newdream.net/debian/ squeeze main |
| 291 | 292 |
deb-src http://ceph.newdream.net/debian/ squeeze main |
| 292 | 293 |
|
| ... | ... | |
| 362 | 363 |
After the splitting is complete, the number of PGs in the system must be |
| 363 | 364 |
changed. Warning: this is not considered safe on PGs in use (with objects), |
| 364 | 365 |
and should be changed only when the PG is created, and before being used: |
| 365 |
ceph osd pool set $poolname pgp_num $num
|
|
| 366 |
ceph osd pool set $poolname pgp_num $num |
|
| 366 | 367 |
|
| 367 | 368 |
4. Replacing the journal for osd.$id: |
| 368 | 369 |
Edit the osd.$id journal configration section |
| ... | ... | |
| 446 | 447 |
|
| 447 | 448 |
It can be configured via libvirt, and the configuration looks like this: |
| 448 | 449 |
|
| 450 |
.. code-block:: xml |
|
| 451 |
|
|
| 449 | 452 |
<disk type='network' device='disk'> |
| 450 | 453 |
<driver name='qemu' type='raw'/> |
| 451 | 454 |
<source protocol='rbd' name='[pool]/[image]:rbd_writeback_window=8000000'/> |
Also available in: Unified diff