root / docs / quick-install-intgrt-guide.rst @ 95e64ecd
History | View | Annotate | Download (17.4 kB)
1 | bc055d09 | Constantinos Venetsanopoulos | .. _quick-install-intgrt-guide: |
---|---|---|---|
2 | 99644884 | Vangelis Koukis | |
3 | bc055d09 | Constantinos Venetsanopoulos | Integrator's Quick Installation Guide |
4 | bc055d09 | Constantinos Venetsanopoulos | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
5 | 99644884 | Vangelis Koukis | |
6 | bc055d09 | Constantinos Venetsanopoulos | This is the Integrator's quick installation guide. |
7 | 99644884 | Vangelis Koukis | |
8 | 99644884 | Vangelis Koukis | It is intended for developers, wishing to implement new functionality |
9 | bc055d09 | Constantinos Venetsanopoulos | inside Synnefo. It assumes thorough familiarity with the |
10 | bc055d09 | Constantinos Venetsanopoulos | :ref:`Synnefo Administrator's Guide <admin-guide>`. |
11 | 99644884 | Vangelis Koukis | |
12 | bc055d09 | Constantinos Venetsanopoulos | It describes how to install the whole synnefo stack on two (2) physical nodes, |
13 | bc055d09 | Constantinos Venetsanopoulos | with minimum configuration. It installs synnefo in a ``virtualenv`` using ``pip |
14 | bc055d09 | Constantinos Venetsanopoulos | install``, and assumes the nodes run Debian Squeeze. After successful |
15 | bc055d09 | Constantinos Venetsanopoulos | installation, you will have the following services running: |
16 | bc055d09 | Constantinos Venetsanopoulos | |
17 | bc055d09 | Constantinos Venetsanopoulos | * Identity Management (Astakos) |
18 | dad708b4 | Antony Chazapis | * Object Storage Service (Pithos+) |
19 | bc055d09 | Constantinos Venetsanopoulos | * Compute Service (Cyclades) |
20 | bc055d09 | Constantinos Venetsanopoulos | * Image Registry Service (Plankton) |
21 | bc055d09 | Constantinos Venetsanopoulos | |
22 | bc055d09 | Constantinos Venetsanopoulos | and a single unified Web UI to manage them all. |
23 | bc055d09 | Constantinos Venetsanopoulos | |
24 | bc055d09 | Constantinos Venetsanopoulos | The Volume Storage Service (Archipelago) and the Billing Service (Aquarium) are |
25 | bc055d09 | Constantinos Venetsanopoulos | not released yet. |
26 | bc055d09 | Constantinos Venetsanopoulos | |
27 | dad708b4 | Antony Chazapis | If you just want to install the Object Storage Service (Pithos+), follow the guide |
28 | bc055d09 | Constantinos Venetsanopoulos | and just stop after the "Testing of Pithos+" section. |
29 | 99644884 | Vangelis Koukis | |
30 | 99644884 | Vangelis Koukis | Building a dev environment |
31 | 99644884 | Vangelis Koukis | -------------------------- |
32 | 99644884 | Vangelis Koukis | |
33 | 99644884 | Vangelis Koukis | virtualenv |
34 | 99644884 | Vangelis Koukis | ********** |
35 | 99644884 | Vangelis Koukis | |
36 | 99644884 | Vangelis Koukis | The easiest method to deploy a development environment is using |
37 | 99644884 | Vangelis Koukis | :command:`virtualenv`. Alternatively, you can use your system's package manager |
38 | 99644884 | Vangelis Koukis | to install any dependencies of synnefo components (e.g. Macports has them all). |
39 | 99644884 | Vangelis Koukis | |
40 | 99644884 | Vangelis Koukis | .. code-block:: console |
41 | 99644884 | Vangelis Koukis | |
42 | 99644884 | Vangelis Koukis | $ virtualenv ~/synnefo-env |
43 | 4c099520 | Kostas Papadimitriou | $ source ~/synnefo-env/bin/activate |
44 | 99644884 | Vangelis Koukis | (synnefo-env)$ |
45 | 4c099520 | Kostas Papadimitriou | |
46 | 4c099520 | Kostas Papadimitriou | Virtualenv creates an isolated python environment to the path you pass as the |
47 | 4c099520 | Kostas Papadimitriou | first argument of the command. That means that all packages you install using |
48 | 4c099520 | Kostas Papadimitriou | :command:`pip` or :command:`easy_install` will be placed in |
49 | 4c099520 | Kostas Papadimitriou | ``ENV/lib/pythonX.X/site-packages`` and their console scripts in ``ENV/bin/``. |
50 | 4c099520 | Kostas Papadimitriou | |
51 | 4c099520 | Kostas Papadimitriou | This allows you to develop against multiple versions of packages that your |
52 | 4c099520 | Kostas Papadimitriou | software depends on without messing with system python packages that may be |
53 | 4c099520 | Kostas Papadimitriou | needed in specific versions for other software you have installed on your |
54 | 4c099520 | Kostas Papadimitriou | system. |
55 | 99644884 | Vangelis Koukis | |
56 | 99644884 | Vangelis Koukis | * It is also recommended to install development helpers: |
57 | 99644884 | Vangelis Koukis | |
58 | 99644884 | Vangelis Koukis | .. code-block:: console |
59 | 99644884 | Vangelis Koukis | |
60 | 4c099520 | Kostas Papadimitriou | (synnefo-env)$ pip install django_extensions fabric>=1.3 |
61 | 99644884 | Vangelis Koukis | |
62 | 99644884 | Vangelis Koukis | * Create a custom settings directory for :ref:`snf-common <snf-common>` and set |
63 | 99644884 | Vangelis Koukis | the ``SYNNEFO_SETTINGS_DIR`` environment variable to use development-specific |
64 | 99644884 | Vangelis Koukis | file:`*.conf` files inside this directory. |
65 | 99644884 | Vangelis Koukis | |
66 | 99644884 | Vangelis Koukis | (synnefo-env)$ mkdir ~/synnefo-settings-dir |
67 | 99644884 | Vangelis Koukis | (synnefo-env)$ export SYNNEFO_SETTINGS_DIR=~/synnefo-settings-dir |
68 | 99644884 | Vangelis Koukis | |
69 | 99644884 | Vangelis Koukis | Insert your custom settings in a file such as :file:`$SYNNEFO_SETTINGS_DIR/99-local.conf`: |
70 | 99644884 | Vangelis Koukis | |
71 | 99644884 | Vangelis Koukis | .. code-block:: python |
72 | 99644884 | Vangelis Koukis | |
73 | 99644884 | Vangelis Koukis | # uncomment this if have django-extensions installed (pip install django_extensions) |
74 | 99644884 | Vangelis Koukis | #INSTALLED_APPS = list(INSTALLED_APPS) + ['django_extensions'] |
75 | 99644884 | Vangelis Koukis | |
76 | 99644884 | Vangelis Koukis | DEV_PATH = os.path.abspath(os.path.dirname(__file__)) |
77 | 99644884 | Vangelis Koukis | DATABASES['default']['NAME'] = os.path.join(DEV_PATH, "synnefo.sqlite") |
78 | 99644884 | Vangelis Koukis | |
79 | 99644884 | Vangelis Koukis | # development rabitmq configuration |
80 | 99644884 | Vangelis Koukis | RABBIT_HOST = "<RabbitMQ_host>" |
81 | 99644884 | Vangelis Koukis | RABBIT_USERNAME = "<RabbitMQ_username>" |
82 | 99644884 | Vangelis Koukis | RABBIT_PASSWORD = "<RabbitMQ_password>" |
83 | 99644884 | Vangelis Koukis | RABBIT_VHOST = "/" |
84 | 99644884 | Vangelis Koukis | |
85 | 99644884 | Vangelis Koukis | # development ganeti settings |
86 | 99644884 | Vangelis Koukis | GANETI_MASTER_IP = "<Ganeti_master_IP>" |
87 | 99644884 | Vangelis Koukis | GANETI_CLUSTER_INFO = (GANETI_MASTER_IP, 5080, "<username>", "<password>") |
88 | 99644884 | Vangelis Koukis | GANETI_CREATEINSTANCE_KWARGS['disk_template'] = 'plain' |
89 | 99644884 | Vangelis Koukis | |
90 | 99644884 | Vangelis Koukis | # This prefix gets used when determining the instance names |
91 | 99644884 | Vangelis Koukis | # of Synnefo VMs at the Ganeti backend. |
92 | 99644884 | Vangelis Koukis | # The dash must always appear in the name! |
93 | 99644884 | Vangelis Koukis | BACKEND_PREFIX_ID = "<your_commit_name>-" |
94 | 99644884 | Vangelis Koukis | |
95 | 99644884 | Vangelis Koukis | IGNORE_FLAVOR_DISK_SIZES = True |
96 | 99644884 | Vangelis Koukis | |
97 | 99644884 | Vangelis Koukis | # do not actually send emails |
98 | 99644884 | Vangelis Koukis | # save them as files in /tmp/synnefo-mails |
99 | 99644884 | Vangelis Koukis | EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend' |
100 | 99644884 | Vangelis Koukis | EMAIL_FILE_PATH = '/tmp/synnefo-mails' |
101 | 99644884 | Vangelis Koukis | |
102 | 99644884 | Vangelis Koukis | # for UI developers |
103 | 99644884 | Vangelis Koukis | UI_HANDLE_WINDOW_EXCEPTIONS = False |
104 | 99644884 | Vangelis Koukis | |
105 | 99644884 | Vangelis Koukis | # allow login using /?test url |
106 | 99644884 | Vangelis Koukis | BYPASS_AUTHENTICATION = True |
107 | 99644884 | Vangelis Koukis | |
108 | 99644884 | Vangelis Koukis | synnefo source |
109 | 99644884 | Vangelis Koukis | ************** |
110 | 99644884 | Vangelis Koukis | |
111 | 99644884 | Vangelis Koukis | * Clone the repository of the synnefo software components you wish |
112 | 99644884 | Vangelis Koukis | to work on, e.g.: |
113 | 99644884 | Vangelis Koukis | |
114 | 442b6550 | Vangelis Koukis | .. code-block:: console |
115 | 442b6550 | Vangelis Koukis | |
116 | 442b6550 | Vangelis Koukis | (synnefo-env)$ git clone https://code.grnet.gr/git/synnefo synnefo |
117 | 4c099520 | Kostas Papadimitriou | (synnefo-env)$ git clone https://code.grnet.gr/git/pithos pithos |
118 | 442b6550 | Vangelis Koukis | |
119 | 99644884 | Vangelis Koukis | * Install the software components you wish to work on inside the |
120 | 99644884 | Vangelis Koukis | virtualenv, in development mode: |
121 | 99644884 | Vangelis Koukis | |
122 | 442b6550 | Vangelis Koukis | .. code-block:: console |
123 | 442b6550 | Vangelis Koukis | |
124 | b1996c81 | Kostas Papadimitriou | (synnefo-env)$ cd snf-cyclades-app |
125 | 442b6550 | Vangelis Koukis | (synnefo-env)$ python setup.py develop -N |
126 | 442b6550 | Vangelis Koukis | |
127 | 442b6550 | Vangelis Koukis | * Initialize database: |
128 | 99644884 | Vangelis Koukis | |
129 | 442b6550 | Vangelis Koukis | .. code-block:: console |
130 | 442b6550 | Vangelis Koukis | |
131 | 442b6550 | Vangelis Koukis | (synnefo-env)$ snf-manage syndb |
132 | 442b6550 | Vangelis Koukis | (synnefo-env)$ snf-manage migrate |
133 | 442b6550 | Vangelis Koukis | (synnefo-env)$ snf-manage loaddata users flavors images |
134 | 442b6550 | Vangelis Koukis | |
135 | 99644884 | Vangelis Koukis | Development tips |
136 | 99644884 | Vangelis Koukis | **************** |
137 | 99644884 | Vangelis Koukis | |
138 | 99644884 | Vangelis Koukis | * Running a development web server: |
139 | 99644884 | Vangelis Koukis | |
140 | 99644884 | Vangelis Koukis | .. code-block:: console |
141 | 99644884 | Vangelis Koukis | |
142 | 99644884 | Vangelis Koukis | (synnefo-env)$ snf-manage runserver |
143 | 99644884 | Vangelis Koukis | |
144 | 4c099520 | Kostas Papadimitriou | or, if you have the ``django_extensions`` and ``werkzeug`` packages installed: |
145 | 99644884 | Vangelis Koukis | |
146 | 99644884 | Vangelis Koukis | .. code-block:: console |
147 | 99644884 | Vangelis Koukis | |
148 | 99644884 | Vangelis Koukis | (synnefo-env)$ snf-manage runserver_plus |
149 | 99644884 | Vangelis Koukis | |
150 | 99644884 | Vangelis Koukis | * Opening a python console with the synnefo environment initialized: |
151 | 99644884 | Vangelis Koukis | |
152 | 99644884 | Vangelis Koukis | .. code-block:: console |
153 | 99644884 | Vangelis Koukis | |
154 | 99644884 | Vangelis Koukis | (synnefo-env)$ snf-manage shell |
155 | 99644884 | Vangelis Koukis | |
156 | 99644884 | Vangelis Koukis | or, with the django_extensions package installed: |
157 | 99644884 | Vangelis Koukis | |
158 | 99644884 | Vangelis Koukis | .. code-block:: console |
159 | 99644884 | Vangelis Koukis | |
160 | 99644884 | Vangelis Koukis | (synnefo-env)$ snf-manage shell_plus |
161 | 99644884 | Vangelis Koukis | |
162 | 99644884 | Vangelis Koukis | |
163 | 99644884 | Vangelis Koukis | South Database Migrations |
164 | 99644884 | Vangelis Koukis | ------------------------- |
165 | 99644884 | Vangelis Koukis | |
166 | b1996c81 | Kostas Papadimitriou | .. _cyclades-dev-initialmigration: |
167 | 99644884 | Vangelis Koukis | |
168 | 99644884 | Vangelis Koukis | Initial Migration |
169 | 99644884 | Vangelis Koukis | ***************** |
170 | 99644884 | Vangelis Koukis | |
171 | 99644884 | Vangelis Koukis | To initialize south migrations in your database the following commands must be |
172 | 99644884 | Vangelis Koukis | executed: |
173 | 99644884 | Vangelis Koukis | |
174 | 99644884 | Vangelis Koukis | .. code-block:: console |
175 | 99644884 | Vangelis Koukis | |
176 | 4c099520 | Kostas Papadimitriou | $ snf-manage syncdb --all # Create / update the database with the south tables |
177 | 4c099520 | Kostas Papadimitriou | $ snf-manage migrate --fake # Perform migration in the database |
178 | 4c099520 | Kostas Papadimitriou | |
179 | 4c099520 | Kostas Papadimitriou | |
180 | 4c099520 | Kostas Papadimitriou | Note that ``--all`` and ``--fake`` arguments are only needed when you are |
181 | 4c099520 | Kostas Papadimitriou | initializing your database. If you want to migrate a previously create databse |
182 | 4c099520 | Kostas Papadimitriou | to the latest db scheme just run the same commands without those arguments. |
183 | 99644884 | Vangelis Koukis | |
184 | 4c099520 | Kostas Papadimitriou | If you are trying to migrate a database that already contains the changes that |
185 | 4c099520 | Kostas Papadimitriou | applied from a specific migration script, ``south`` will probably notify you for |
186 | 4c099520 | Kostas Papadimitriou | inconsistent db scheme, a workaround for that issue is to use ``--fake`` option |
187 | 4c099520 | Kostas Papadimitriou | for a specific migration. |
188 | 99644884 | Vangelis Koukis | |
189 | 99644884 | Vangelis Koukis | For example: |
190 | 99644884 | Vangelis Koukis | |
191 | 99644884 | Vangelis Koukis | |
192 | 99644884 | Vangelis Koukis | .. code-block:: console |
193 | 99644884 | Vangelis Koukis | |
194 | 99644884 | Vangelis Koukis | $ snf-manage migrate db 0001 --fake |
195 | 99644884 | Vangelis Koukis | |
196 | 99644884 | Vangelis Koukis | To be sure that all migrations are applied use: |
197 | 99644884 | Vangelis Koukis | |
198 | 99644884 | Vangelis Koukis | .. code-block:: console |
199 | 99644884 | Vangelis Koukis | |
200 | 99644884 | Vangelis Koukis | $ snf-manage migrate db --list |
201 | 99644884 | Vangelis Koukis | |
202 | 99644884 | Vangelis Koukis | All starred migrations are applied. |
203 | 99644884 | Vangelis Koukis | |
204 | 99644884 | Vangelis Koukis | Schema migrations |
205 | 99644884 | Vangelis Koukis | ***************** |
206 | 99644884 | Vangelis Koukis | |
207 | 99644884 | Vangelis Koukis | Do not use the syncdb management command. It can only be used the first time |
208 | 99644884 | Vangelis Koukis | and/or if you drop the database and must recreate it from scratch. See |
209 | b1996c81 | Kostas Papadimitriou | :ref:`cyclades-dev-initialmigration`. |
210 | 99644884 | Vangelis Koukis | |
211 | 99644884 | Vangelis Koukis | |
212 | 99644884 | Vangelis Koukis | Every time you make changes to the database and data migration is not required |
213 | 99644884 | Vangelis Koukis | (WARNING: always perform this with extreme care): |
214 | 99644884 | Vangelis Koukis | |
215 | 99644884 | Vangelis Koukis | .. code-block:: console |
216 | 99644884 | Vangelis Koukis | |
217 | 99644884 | Vangelis Koukis | $ snf-manage schemamigration db --auto |
218 | 99644884 | Vangelis Koukis | |
219 | 99644884 | Vangelis Koukis | The above will create the migration script. Now this must be applied to the live |
220 | 99644884 | Vangelis Koukis | database: |
221 | 99644884 | Vangelis Koukis | |
222 | 99644884 | Vangelis Koukis | .. code-block:: console |
223 | 99644884 | Vangelis Koukis | |
224 | 99644884 | Vangelis Koukis | $ snf-manage migrate db |
225 | 99644884 | Vangelis Koukis | |
226 | 99644884 | Vangelis Koukis | Consider this example (adding a field to the ``SynnefoUser`` model): |
227 | 99644884 | Vangelis Koukis | |
228 | 99644884 | Vangelis Koukis | .. code-block:: console |
229 | 99644884 | Vangelis Koukis | |
230 | 99644884 | Vangelis Koukis | $ ./bin/python manage.py schemamigration db --auto |
231 | 99644884 | Vangelis Koukis | + Added field new_south_test_field on db.SynnefoUser |
232 | 99644884 | Vangelis Koukis | |
233 | 99644884 | Vangelis Koukis | Created 0002_auto__add_field_synnefouser_new_south_test_field.py. |
234 | 99644884 | Vangelis Koukis | |
235 | 99644884 | Vangelis Koukis | You can now apply this migration with: |
236 | 99644884 | Vangelis Koukis | |
237 | 99644884 | Vangelis Koukis | .. code-block:: console |
238 | 99644884 | Vangelis Koukis | |
239 | 99644884 | Vangelis Koukis | $ ./manage.py migrate db |
240 | 99644884 | Vangelis Koukis | Running migrations for db: |
241 | 99644884 | Vangelis Koukis | - Migrating forwards to 0002_auto__add_field_synnefouser_new_south_test_field. |
242 | 99644884 | Vangelis Koukis | > db:0002_auto__add_field_synnefouser_new_south_test_field |
243 | 99644884 | Vangelis Koukis | - Loading initial data for db. |
244 | 99644884 | Vangelis Koukis | |
245 | 99644884 | Vangelis Koukis | Installing json fixture 'initial_data' from '/home/bkarak/devel/synnefo/../synnefo/db/fixtures'. |
246 | 99644884 | Vangelis Koukis | Installed 1 object(s) from 1 fixture(s) |
247 | 99644884 | Vangelis Koukis | |
248 | 99644884 | Vangelis Koukis | South needs some extra definitions to the model to preserve and migrate the |
249 | 99644884 | Vangelis Koukis | existing data, for example, if we add a field in a model, we should declare its |
250 | 99644884 | Vangelis Koukis | default value. If not, South will propably fail, after indicating the error: |
251 | 99644884 | Vangelis Koukis | |
252 | 99644884 | Vangelis Koukis | .. code-block:: console |
253 | 99644884 | Vangelis Koukis | |
254 | 99644884 | Vangelis Koukis | $ ./bin/python manage.py schemamigration db --auto |
255 | 99644884 | Vangelis Koukis | ? The field 'SynnefoUser.new_south_field_2' does not have a default specified, yet is NOT NULL. |
256 | 99644884 | Vangelis Koukis | ? Since you are adding or removing this field, you MUST specify a default |
257 | 99644884 | Vangelis Koukis | ? value to use for existing rows. Would you like to: |
258 | 99644884 | Vangelis Koukis | ? 1. Quit now, and add a default to the field in models.py |
259 | 99644884 | Vangelis Koukis | ? 2. Specify a one-off value to use for existing columns now |
260 | 99644884 | Vangelis Koukis | ? Please select a choice: 1 |
261 | 99644884 | Vangelis Koukis | |
262 | 99644884 | Vangelis Koukis | Data migrations |
263 | 99644884 | Vangelis Koukis | *************** |
264 | 99644884 | Vangelis Koukis | |
265 | 99644884 | Vangelis Koukis | To do data migration as well, for example rename a field, use the |
266 | 99644884 | Vangelis Koukis | ``datamigration`` management command. |
267 | 99644884 | Vangelis Koukis | |
268 | 99644884 | Vangelis Koukis | In contrast with ``schemamigration``, to perform complex data migration, we |
269 | 99644884 | Vangelis Koukis | must write the script manually. The process is the following: |
270 | 99644884 | Vangelis Koukis | |
271 | 99644884 | Vangelis Koukis | 1. Introduce the changes in the code and fixtures (initial data). |
272 | 99644884 | Vangelis Koukis | 2. Execute: |
273 | 99644884 | Vangelis Koukis | |
274 | 99644884 | Vangelis Koukis | .. code-block:: console |
275 | 99644884 | Vangelis Koukis | |
276 | 99644884 | Vangelis Koukis | $ snf-manage datamigration <migration_name_here> |
277 | 99644884 | Vangelis Koukis | |
278 | 99644884 | Vangelis Koukis | For example: |
279 | 99644884 | Vangelis Koukis | |
280 | 99644884 | Vangelis Koukis | .. code-block:: console |
281 | 99644884 | Vangelis Koukis | |
282 | 99644884 | Vangelis Koukis | $ ./bin/python manage.py datamigration db rename_credit_wallet |
283 | 99644884 | Vangelis Koukis | Created 0003_rename_credit_wallet.py. |
284 | 99644884 | Vangelis Koukis | |
285 | 99644884 | Vangelis Koukis | 3. Edit the generated script. It contains two methods, ``forwards`` and |
286 | 99644884 | Vangelis Koukis | ``backwards``. |
287 | 99644884 | Vangelis Koukis | |
288 | 99644884 | Vangelis Koukis | For database operations (column additions, alter tables etc), use the |
289 | 99644884 | Vangelis Koukis | South database API (http://south.aeracode.org/docs/databaseapi.html). |
290 | 99644884 | Vangelis Koukis | |
291 | 99644884 | Vangelis Koukis | To access the data, use the database reference (``orm``) provided as |
292 | 99644884 | Vangelis Koukis | parameter in ``forwards``, ``backwards`` method declarations in the |
293 | 99644884 | Vangelis Koukis | migration script. For example: |
294 | 99644884 | Vangelis Koukis | |
295 | 99644884 | Vangelis Koukis | .. code-block:: python |
296 | 99644884 | Vangelis Koukis | |
297 | 99644884 | Vangelis Koukis | class Migration(DataMigration): |
298 | 99644884 | Vangelis Koukis | |
299 | 99644884 | Vangelis Koukis | def forwards(self, orm): |
300 | 99644884 | Vangelis Koukis | orm.SynnefoUser.objects.all() |
301 | 99644884 | Vangelis Koukis | |
302 | 99644884 | Vangelis Koukis | 4. To migrate the database to the latest version, run: |
303 | 99644884 | Vangelis Koukis | |
304 | 99644884 | Vangelis Koukis | .. code-block:: console |
305 | 99644884 | Vangelis Koukis | |
306 | 99644884 | Vangelis Koukis | $ snf-manage migrate db |
307 | 99644884 | Vangelis Koukis | |
308 | 99644884 | Vangelis Koukis | To see which migrations are applied: |
309 | 99644884 | Vangelis Koukis | |
310 | 99644884 | Vangelis Koukis | .. code-block:: console |
311 | 99644884 | Vangelis Koukis | |
312 | 99644884 | Vangelis Koukis | $ snf-manage migrate db --list |
313 | 99644884 | Vangelis Koukis | |
314 | 99644884 | Vangelis Koukis | db |
315 | 99644884 | Vangelis Koukis | (*) 0001_initial |
316 | 99644884 | Vangelis Koukis | (*) 0002_auto__add_field_synnefouser_new_south_test_field |
317 | 99644884 | Vangelis Koukis | (*) 0003_rename_credit_wallet |
318 | 99644884 | Vangelis Koukis | |
319 | 99644884 | Vangelis Koukis | .. seealso:: |
320 | 442b6550 | Vangelis Koukis | More information and more thorough examples can be found in the South web site, |
321 | 99644884 | Vangelis Koukis | http://south.aeracode.org/ |
322 | 99644884 | Vangelis Koukis | |
323 | 99644884 | Vangelis Koukis | Test coverage |
324 | 99644884 | Vangelis Koukis | ------------- |
325 | 49f513ad | Vangelis Koukis | |
326 | 49f513ad | Vangelis Koukis | .. warning:: This section may be out of date. |
327 | 99644884 | Vangelis Koukis | |
328 | 99644884 | Vangelis Koukis | In order to get code coverage reports you need to install django-test-coverage |
329 | 99644884 | Vangelis Koukis | |
330 | 99644884 | Vangelis Koukis | .. code-block:: console |
331 | 99644884 | Vangelis Koukis | |
332 | 99644884 | Vangelis Koukis | $ pip install django-test-coverage |
333 | 99644884 | Vangelis Koukis | |
334 | 99644884 | Vangelis Koukis | Then configure the test runner inside Django settings: |
335 | 99644884 | Vangelis Koukis | |
336 | 99644884 | Vangelis Koukis | .. code-block:: python |
337 | 99644884 | Vangelis Koukis | |
338 | 99644884 | Vangelis Koukis | TEST_RUNNER = 'django-test-coverage.runner.run_tests' |
339 | 99644884 | Vangelis Koukis | |
340 | 99644884 | Vangelis Koukis | |
341 | 49f513ad | Vangelis Koukis | Internationalization |
342 | 49f513ad | Vangelis Koukis | -------------------- |
343 | 49f513ad | Vangelis Koukis | |
344 | 49f513ad | Vangelis Koukis | This section describes how to translate static strings in Django projects: |
345 | 49f513ad | Vangelis Koukis | |
346 | 49f513ad | Vangelis Koukis | 0. From our project's base, we add directory locale |
347 | 49f513ad | Vangelis Koukis | |
348 | 49f513ad | Vangelis Koukis | .. code-block:: console |
349 | 49f513ad | Vangelis Koukis | |
350 | 49f513ad | Vangelis Koukis | $ mkdir locale |
351 | 49f513ad | Vangelis Koukis | |
352 | 49f513ad | Vangelis Koukis | then we add on the settings.py the language code e.g., |
353 | 49f513ad | Vangelis Koukis | |
354 | 49f513ad | Vangelis Koukis | .. code-block:: python |
355 | 49f513ad | Vangelis Koukis | |
356 | 49f513ad | Vangelis Koukis | LANGUAGES = ( |
357 | 49f513ad | Vangelis Koukis | ('el', u'Greek'), |
358 | 49f513ad | Vangelis Koukis | ('en', u'English'),) |
359 | 49f513ad | Vangelis Koukis | |
360 | 49f513ad | Vangelis Koukis | 1. For each language we want to add, we run ``makemessages`` from the project's |
361 | 49f513ad | Vangelis Koukis | base: |
362 | 49f513ad | Vangelis Koukis | |
363 | 49f513ad | Vangelis Koukis | .. code-block:: python |
364 | 49f513ad | Vangelis Koukis | |
365 | 49f513ad | Vangelis Koukis | $ ./bin/django-admin.py makemessages -l el -e html,txt,py |
366 | 49f513ad | Vangelis Koukis | (./bin/django-admin.py makemessages -l el -e html,txt,py --ignore=lib/\*) |
367 | 49f513ad | Vangelis Koukis | |
368 | 49f513ad | Vangelis Koukis | This will add the Greek language, and we specify that :file:`*.html`, |
369 | 49f513ad | Vangelis Koukis | :file:`*.txt` and :file:`*.py` files contain translatable strings |
370 | 49f513ad | Vangelis Koukis | |
371 | 49f513ad | Vangelis Koukis | 2. We translate our strings: |
372 | 49f513ad | Vangelis Koukis | |
373 | 49f513ad | Vangelis Koukis | On :file:`.py` files, e.g., :file:`views.py`, first import ``gettext``: |
374 | 49f513ad | Vangelis Koukis | |
375 | 49f513ad | Vangelis Koukis | .. code-block:: python |
376 | 49f513ad | Vangelis Koukis | |
377 | 49f513ad | Vangelis Koukis | from django.utils.translation import gettext_lazy as _ |
378 | 49f513ad | Vangelis Koukis | |
379 | 49f513ad | Vangelis Koukis | Then every ``string`` to be translated becomes: ``_('string')`` |
380 | 49f513ad | Vangelis Koukis | e.g.: |
381 | 49f513ad | Vangelis Koukis | |
382 | 49f513ad | Vangelis Koukis | .. code-block:: python |
383 | 49f513ad | Vangelis Koukis | |
384 | 49f513ad | Vangelis Koukis | help_text=_("letters and numbers only")) |
385 | 49f513ad | Vangelis Koukis | 'title': _('Ubuntu 10.10 server 64bit'), |
386 | 49f513ad | Vangelis Koukis | |
387 | 49f513ad | Vangelis Koukis | On django templates (``html`` files), on the beggining of the file we add |
388 | 49f513ad | Vangelis Koukis | ``{% load i18n %}`` then rewrite every string that needs to be translated, |
389 | 49f513ad | Vangelis Koukis | as ``{% trans "string" %}``. For example: ``{% trans "Home" %}`` |
390 | 49f513ad | Vangelis Koukis | |
391 | 49f513ad | Vangelis Koukis | 3. When all strings have been translated, run: |
392 | 49f513ad | Vangelis Koukis | |
393 | 49f513ad | Vangelis Koukis | .. code-block:: console |
394 | 49f513ad | Vangelis Koukis | |
395 | 49f513ad | Vangelis Koukis | $ django-admin.py makemessages -l el -e html,txt,py |
396 | 49f513ad | Vangelis Koukis | |
397 | 49f513ad | Vangelis Koukis | processing language ``el``. This creates (or updates) the :file:`po` file |
398 | 49f513ad | Vangelis Koukis | for the Greek language. We run this command each time we add new strings to |
399 | 49f513ad | Vangelis Koukis | be translated. After that, we can translate our strings in the :file:`po` |
400 | 49f513ad | Vangelis Koukis | file (:file:`locale/el/LC_MESSAGES/django.po`) |
401 | 99644884 | Vangelis Koukis | |
402 | 49f513ad | Vangelis Koukis | 4. When the :file:`po` file is ready, run |
403 | 49f513ad | Vangelis Koukis | |
404 | 49f513ad | Vangelis Koukis | .. code-block:: console |
405 | 49f513ad | Vangelis Koukis | |
406 | 49f513ad | Vangelis Koukis | $ ./bin/django-admin.py compilemessages |
407 | 49f513ad | Vangelis Koukis | |
408 | 49f513ad | Vangelis Koukis | This compiles the ``po`` files to ``mo``. Our strings will appear translated |
409 | 49f513ad | Vangelis Koukis | once we change the language (e.g., from a dropdown menu in the page) |
410 | 49f513ad | Vangelis Koukis | |
411 | 49f513ad | Vangelis Koukis | .. seealso:: |
412 | 49f513ad | Vangelis Koukis | http://docs.djangoproject.com/en/dev/topics/i18n/internationalization/ |
413 | 49f513ad | Vangelis Koukis | |
414 | 49f513ad | Vangelis Koukis | |
415 | 49f513ad | Vangelis Koukis | Building source packages |
416 | 99644884 | Vangelis Koukis | ------------------------ |
417 | 49f513ad | Vangelis Koukis | |
418 | 49f513ad | Vangelis Koukis | .. warning:: This section may be out of date. |
419 | 99644884 | Vangelis Koukis | |
420 | 99644884 | Vangelis Koukis | To create a python package from the Synnefo source code run |
421 | 99644884 | Vangelis Koukis | |
422 | 99644884 | Vangelis Koukis | .. code-block:: bash |
423 | 99644884 | Vangelis Koukis | |
424 | 9c0ac5af | Kostas Papadimitriou | $ cd snf-cyclades-app |
425 | 99644884 | Vangelis Koukis | $ python setup.py sdist |
426 | 99644884 | Vangelis Koukis | |
427 | 99644884 | Vangelis Koukis | this command will create a ``tar.gz`` python source package inside ``dist`` directory. |
428 | 99644884 | Vangelis Koukis | |
429 | 99644884 | Vangelis Koukis | |
430 | 49f513ad | Vangelis Koukis | Building documentation |
431 | 49f513ad | Vangelis Koukis | ---------------------- |
432 | 99644884 | Vangelis Koukis | |
433 | 99644884 | Vangelis Koukis | Make sure you have ``sphinx`` installed. |
434 | 99644884 | Vangelis Koukis | |
435 | 99644884 | Vangelis Koukis | .. code-block:: bash |
436 | 99644884 | Vangelis Koukis | |
437 | 9c0ac5af | Kostas Papadimitriou | $ cd snf-cyclades-app/docs |
438 | 99644884 | Vangelis Koukis | $ make html |
439 | 99644884 | Vangelis Koukis | |
440 | 99644884 | Vangelis Koukis | .. note:: |
441 | 99644884 | Vangelis Koukis | |
442 | 99644884 | Vangelis Koukis | The theme define in the Sphinx configuration file ``conf.py`` is ``nature``, |
443 | 99644884 | Vangelis Koukis | not available in the version of Sphinx shipped with Debian Squeeze. Replace |
444 | 99644884 | Vangelis Koukis | it with ``default`` to build with a Squeeze-provided Sphinx. |
445 | 99644884 | Vangelis Koukis | |
446 | 9c0ac5af | Kostas Papadimitriou | html files are generated in the ``snf-cyclades-app/docs/_build/html`` directory. |
447 | 99644884 | Vangelis Koukis | |
448 | 49f513ad | Vangelis Koukis | |
449 | 49f513ad | Vangelis Koukis | Continuous integration with Jenkins |
450 | 49f513ad | Vangelis Koukis | ----------------------------------- |
451 | 49f513ad | Vangelis Koukis | .. warning:: This section may be out of date. |
452 | 49f513ad | Vangelis Koukis | |
453 | 49f513ad | Vangelis Koukis | Preparing a GIT mirror |
454 | 49f513ad | Vangelis Koukis | ********************** |
455 | 49f513ad | Vangelis Koukis | |
456 | 49f513ad | Vangelis Koukis | Jenkins cannot currently work with Git over encrypted HTTP. To solve this |
457 | 49f513ad | Vangelis Koukis | problem we currently mirror the central Git repository locally on the jenkins |
458 | 49f513ad | Vangelis Koukis | installation machine. To setup such a mirror do the following: |
459 | 49f513ad | Vangelis Koukis | |
460 | 49f513ad | Vangelis Koukis | edit .netrc:: |
461 | 49f513ad | Vangelis Koukis | |
462 | 49f513ad | Vangelis Koukis | machine code.grnet.gr |
463 | 49f513ad | Vangelis Koukis | login accountname |
464 | 49f513ad | Vangelis Koukis | password accountpasswd |
465 | 49f513ad | Vangelis Koukis | |
466 | 49f513ad | Vangelis Koukis | Create the mirror:: |
467 | 49f513ad | Vangelis Koukis | |
468 | 49f513ad | Vangelis Koukis | git clone --mirror https://code.grnet.gr/git/synnefo synnefo |
469 | 49f513ad | Vangelis Koukis | |
470 | 49f513ad | Vangelis Koukis | Setup cron to pull from the mirror periodically. Ideally, Git mirror updates |
471 | 49f513ad | Vangelis Koukis | should run just before Jenkins jobs check the mirror for changes:: |
472 | 49f513ad | Vangelis Koukis | |
473 | 49f513ad | Vangelis Koukis | 4,14,24,34,44,54 * * * * cd /path/to/mirror && git fetch && git remote prune origin |
474 | 49f513ad | Vangelis Koukis | |
475 | 49f513ad | Vangelis Koukis | Jenkins setup |
476 | 49f513ad | Vangelis Koukis | ************* |
477 | 49f513ad | Vangelis Koukis | |
478 | 49f513ad | Vangelis Koukis | The following instructions will setup Jenkins to run synnefo tests with the |
479 | 49f513ad | Vangelis Koukis | SQLite database. To run the tests on MySQL and/or Postgres, step 5 must be |
480 | 49f513ad | Vangelis Koukis | replicated. Also, the correct configuration file must be copied (line 6 of the |
481 | 49f513ad | Vangelis Koukis | build script). |
482 | 49f513ad | Vangelis Koukis | |
483 | 49f513ad | Vangelis Koukis | 1. Install and start Jenkins. On Debian Squeeze: |
484 | 49f513ad | Vangelis Koukis | |
485 | 49f513ad | Vangelis Koukis | wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | apt-key add - |
486 | 49f513ad | Vangelis Koukis | echo "deb http://pkg.jenkins-ci.org/debian binary/" >>/etc/apt/sources.list |
487 | 49f513ad | Vangelis Koukis | echo "deb http://ppa.launchpad.net/chris-lea/zeromq/ubuntu lucid main" >> /etc/apt/sources.list |
488 | 49f513ad | Vangelis Koukis | sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C7917B12 |
489 | 49f513ad | Vangelis Koukis | sudo apt-get update |
490 | 49f513ad | Vangelis Koukis | sudo apt-get install jenkins |
491 | 49f513ad | Vangelis Koukis | |
492 | 49f513ad | Vangelis Koukis | Also install the following packages: |
493 | 49f513ad | Vangelis Koukis | |
494 | 49f513ad | Vangelis Koukis | apt-get install python-virtualenv libcurl3-gnutls libcurl3-gnutls-dev |
495 | 49f513ad | Vangelis Koukis | uuid-dev libmysqlclient-dev libpq-dev libsqlite-dev |
496 | 49f513ad | Vangelis Koukis | python-dev libzmq-dev |
497 | 49f513ad | Vangelis Koukis | |
498 | 49f513ad | Vangelis Koukis | 2. After Jenkins starts, go to |
499 | 49f513ad | Vangelis Koukis | |
500 | 49f513ad | Vangelis Koukis | http://$HOST:8080/pluginManager/ |
501 | 49f513ad | Vangelis Koukis | |
502 | 49f513ad | Vangelis Koukis | and install the following plug-ins at |
503 | 49f513ad | Vangelis Koukis | |
504 | 49f513ad | Vangelis Koukis | -Jenkins Cobertura Plugin |
505 | 49f513ad | Vangelis Koukis | -Jenkins Email Extension Plugin |
506 | 49f513ad | Vangelis Koukis | -Jenkins GIT plugin |
507 | 49f513ad | Vangelis Koukis | -Jenkins SLOCCount Plug-in |
508 | 49f513ad | Vangelis Koukis | -Hudson/Jenkins Violations plugin |
509 | 49f513ad | Vangelis Koukis | |
510 | 49f513ad | Vangelis Koukis | 3. Configure the Jenkins user's Git details: |
511 | 49f513ad | Vangelis Koukis | su jenkins |
512 | 49f513ad | Vangelis Koukis | git config --global user.email "buildbot@lists.grnet.gr" |
513 | 49f513ad | Vangelis Koukis | git config --global user.name "Buildbot" |
514 | 49f513ad | Vangelis Koukis | |
515 | 49f513ad | Vangelis Koukis | 4. Make sure that all system-level dependencies specified in README.develop |
516 | 49f513ad | Vangelis Koukis | are correctly installed |
517 | 49f513ad | Vangelis Koukis | |
518 | 49f513ad | Vangelis Koukis | 5. Create a new "free-style software" job and set the following values:: |
519 | 49f513ad | Vangelis Koukis | |
520 | 49f513ad | Vangelis Koukis | Project name: synnefo |
521 | 49f513ad | Vangelis Koukis | Source Code Management: Git |
522 | 49f513ad | Vangelis Koukis | URL of repository: Jenkins Git does not support HTTPS for checking out directly |
523 | 49f513ad | Vangelis Koukis | from the repository. The temporary solution is to checkout |
524 | 49f513ad | Vangelis Koukis | with a cron script in a directory and set the checkout path |
525 | 49f513ad | Vangelis Koukis | in this field |
526 | 49f513ad | Vangelis Koukis | Branches to build: master and perhaps others |
527 | 49f513ad | Vangelis Koukis | Git->Advanced->Local subdirectory for repo (optional): synnefo |
528 | 49f513ad | Vangelis Koukis | Git->Advanced->Prune remote branches before build: check |
529 | 49f513ad | Vangelis Koukis | Repository browser: redmineweb, |
530 | 49f513ad | Vangelis Koukis | URL: https://code.grnet.gr/projects/synnefo/repository/ |
531 | 49f513ad | Vangelis Koukis | Build Triggers->Poll SCM: check |
532 | 49f513ad | Vangelis Koukis | Schedule: # every five minutes |
533 | 49f513ad | Vangelis Koukis | 0,5,10,15,20,25,30,35,40,45,50,55 * * * * |
534 | 49f513ad | Vangelis Koukis | |
535 | 49f513ad | Vangelis Koukis | Build -> Add build step-> Execute shell |
536 | 49f513ad | Vangelis Koukis | |
537 | 49f513ad | Vangelis Koukis | Command:: |
538 | 49f513ad | Vangelis Koukis | |
539 | 49f513ad | Vangelis Koukis | #!/bin/bash -ex |
540 | 49f513ad | Vangelis Koukis | cd synnefo |
541 | 49f513ad | Vangelis Koukis | mkdir -p reports |
542 | 49f513ad | Vangelis Koukis | /usr/bin/sloccount --duplicates --wide --details api util ui logic auth > reports/sloccount.sc |
543 | 49f513ad | Vangelis Koukis | cp conf/ci/manage.py . |
544 | 49f513ad | Vangelis Koukis | if [ ! -e requirements.pip ]; then cp conf/ci/pip-1.2.conf requirements.pip; fi |
545 | 49f513ad | Vangelis Koukis | cat settings.py.dist conf/ci/settings.py.sqlite > settings.py |
546 | 49f513ad | Vangelis Koukis | python manage.py update_ve |
547 | 49f513ad | Vangelis Koukis | python manage.py hudson api db logic |
548 | 49f513ad | Vangelis Koukis | |
549 | 49f513ad | Vangelis Koukis | Post-build Actions->Publish JUnit test result report: check |
550 | 49f513ad | Vangelis Koukis | Test report XMLs: synnefo/reports/TEST-*.xml |
551 | 49f513ad | Vangelis Koukis | |
552 | 49f513ad | Vangelis Koukis | Post-build Actions->Publish Cobertura Coverage Report: check |
553 | 49f513ad | Vangelis Koukis | Cobertura xml report pattern: synnefo/reports/coverage.xml |
554 | 49f513ad | Vangelis Koukis | |
555 | 49f513ad | Vangelis Koukis | Post-build Actions->Report Violations: check |
556 | 49f513ad | Vangelis Koukis | pylint[XML filename pattern]: synnefo/reports/pylint.report |
557 | 49f513ad | Vangelis Koukis | |
558 | 49f513ad | Vangelis Koukis | Post-build Actions->Publish SLOCCount analysis results |
559 | 49f513ad | Vangelis Koukis | SLOCCount reports: synnefo/reports/sloccount.sc |
560 | 49f513ad | Vangelis Koukis | (also, remember to install sloccount at /usr/bin) |
561 | 49f513ad | Vangelis Koukis | |
562 | 49f513ad | Vangelis Koukis | .. seealso:: |
563 | 49f513ad | Vangelis Koukis | http://sites.google.com/site/kmmbvnr/home/django-hudson-tutorial |