root / docs / src / develop.rst @ c469ca86
History | View | Annotate | Download (8.9 kB)
1 |
Developers guide |
---|---|
2 |
================ |
3 |
|
4 |
Information on how to setup a development environment. |
5 |
|
6 |
This file documents the installation of a development environment for Synnefo. |
7 |
It should be read alongside :ref:`installation guide <installation>`. |
8 |
|
9 |
It contains development-specific ammendments to the basic installation steps |
10 |
outlined in `installation guide <installation>`, and development-specific notes. |
11 |
|
12 |
Installing the development environment |
13 |
-------------------------------------- |
14 |
|
15 |
For a basic development environment you need to follow steps |
16 |
of `installation guide <installation>`, which should be read in its |
17 |
entirety *before* this document. |
18 |
|
19 |
Development-specific guidelines on each step: |
20 |
|
21 |
|
22 |
0. Allocation of physical nodes: |
23 |
Node types DB, APISERVER, LOGIC may all be run on the same physical machine, |
24 |
usually, your development workstation. |
25 |
|
26 |
Nodes of type GANETI-MASTER, GANETI-NODES and QUEUE are already provided |
27 |
by the development Ganeti backend. Access credentials are provided in |
28 |
settings.py.dist. |
29 |
|
30 |
|
31 |
1. You do not need to install your own Ganeti installation. |
32 |
Use the RAPI endpoint as contained in common settings. |
33 |
|
34 |
|
35 |
2. You do not need to setup your own RabbitMQ nodes, use the AMQP endpoints |
36 |
contained in settings.py.dist. |
37 |
|
38 |
3. For development purposes, Django's own development |
39 |
server, `./manage.py runserver` will suffice. |
40 |
|
41 |
|
42 |
4. Use a virtual environment to install the Django project, or packages provided |
43 |
by your distribution. |
44 |
|
45 |
|
46 |
5. Install a DB of your own, or use the PostgreSQL instance available on the |
47 |
development backend. |
48 |
|
49 |
|
50 |
6. As is. |
51 |
|
52 |
|
53 |
7. The following fixtures can be loaded optionally depending on |
54 |
testing/development requirements, and are not needed in a production setup: |
55 |
|
56 |
$ ./bin/python manage.py loaddata db/fixtures/vms.json |
57 |
$ ./bin/python manage.py loaddata db/fixtures/disks.json |
58 |
|
59 |
|
60 |
8. MAKE SURE you setup a distinct BACKEND_PREFIX_ID, e.g., use your commit |
61 |
username. |
62 |
|
63 |
|
64 |
9. The Ganeti monitoring daemon from the latest Synnefo release is already |
65 |
running on the development Ganeti master. You may also run your own, on your |
66 |
own Ganeti backend if you so wish. |
67 |
|
68 |
|
69 |
10.As is. |
70 |
|
71 |
11.The Synnefo Ganeti hook is already running on the development backend, |
72 |
sending notifications over AMQP. |
73 |
|
74 |
|
75 |
12.The VNC authentication proxy is already running on the Ganeti development |
76 |
backend. You *cannot* run your own, unless you install your own Ganeti |
77 |
backend, because it needs direct access to the hypervisor's VNC port on |
78 |
GANETI-NODEs. |
79 |
|
80 |
Note: You still need to install the vncauthproxy package to satisfy |
81 |
the dependency of the API on the vncauthproxy client. See Synnefo #807 |
82 |
for more details. |
83 |
|
84 |
|
85 |
13.The development Ganeti backend already has a number of OS Images available. |
86 |
|
87 |
|
88 |
14.The development Ganeti backend already has a number of pre-provisioned |
89 |
bridges available, per each BACKEND_PREFIX_ID. |
90 |
|
91 |
To setup simple NAT-based networking on a Ganeti backend on your own, |
92 |
please see the provided patches under contrib/patches/. |
93 |
You will need minor patches to the sample KVM ifup hook, kvm-vif-bridge, |
94 |
and a small patch to NFDHCPD to enable it to work with bridged tap+ |
95 |
interfaces. To support bridged tap interfaces you also need to patch the |
96 |
python-nfqueue package, patches against python-nfqueue-0.3 [part of Debian |
97 |
Sid] are also provided under contrib/patches/. |
98 |
|
99 |
|
100 |
15.As is. |
101 |
|
102 |
|
103 |
16.As is. |
104 |
|
105 |
|
106 |
17.[OPTIONAL] Create settings.d/99-local.conf and insert local overrides for |
107 |
settings.d/\*. This will allow pulling new files without needing to reapply |
108 |
local any local modifications. |
109 |
|
110 |
|
111 |
South Database Migrations |
112 |
------------------------- |
113 |
|
114 |
* Initial Migration |
115 |
|
116 |
First, remember to add the south app to settings.py (it is already included in |
117 |
the settings.py.dist). |
118 |
|
119 |
To initialise south migrations in your database the following commands must be |
120 |
executed: |
121 |
|
122 |
$ ./bin/python manage.py syncdb # Create / update the database with the south tables |
123 |
$ ./bin/python manage.py migrate db # Perform migration in the database |
124 |
|
125 |
Note that syncdb will create the latest models that exist in the db app, so some |
126 |
migrations may fail. If you are sure a migration has already taken place you |
127 |
must use the "--fake" option, to apply it. |
128 |
|
129 |
For example: |
130 |
|
131 |
$ ./bin/python manage.py migrate db 0001 --fake |
132 |
|
133 |
To be sure that all migrations are applied type: |
134 |
|
135 |
$ ./bin/python manage.py migrate db --list |
136 |
|
137 |
All starred migrations are applied. |
138 |
|
139 |
Remember, the migration is performed mainly for the data, not for the database |
140 |
schema. If you do not want to migrate the data, a syncdb and fake migrations for |
141 |
all the migration versions will suffice. |
142 |
|
143 |
* Schema migrations: |
144 |
|
145 |
Do not use the syncdb management command. It can only be used the first time |
146 |
and/or if you drop the database and must recreate it from scratch. See |
147 |
"Initial Migration" section. |
148 |
|
149 |
Every time you make changes to the database and data migration is not required |
150 |
(WARNING: always perform this with extreme care): |
151 |
|
152 |
$ ./bin/python manage.py schemamigration db --auto |
153 |
|
154 |
The above will create the migration script. Now this must be applied to the live |
155 |
database. |
156 |
|
157 |
$ ./bin/python migrate db |
158 |
|
159 |
Consider this example (adding a field to the SynnefoUser model): |
160 |
|
161 |
$ ./bin/python manage.py schemamigration db --auto |
162 |
+ Added field new_south_test_field on db.SynnefoUser |
163 |
|
164 |
Created 0002_auto__add_field_synnefouser_new_south_test_field.py. |
165 |
|
166 |
You can now apply this migration with: ./manage.py migrate db |
167 |
|
168 |
$ ./manage.py migrate db |
169 |
Running migrations for db: |
170 |
- Migrating forwards to 0002_auto__add_field_synnefouser_new_south_test_field. |
171 |
> db:0002_auto__add_field_synnefouser_new_south_test_field |
172 |
- Loading initial data for db. |
173 |
|
174 |
Installing json fixture 'initial_data' from '/home/bkarak/devel/synnefo/../synnefo/db/fixtures'. |
175 |
Installed 1 object(s) from 1 fixture(s) |
176 |
|
177 |
South needs some extra definitions to the model to preserve and migrate the |
178 |
existing data, for example, if we add a field in a model, we should declare its |
179 |
default value. If not, South will propably fail, after indicating the error. |
180 |
|
181 |
$ ./bin/python manage.py schemamigration db --auto |
182 |
? The field 'SynnefoUser.new_south_field_2' does not have a default specified, yet is NOT NULL. |
183 |
? Since you are adding or removing this field, you MUST specify a default |
184 |
? value to use for existing rows. Would you like to: |
185 |
? 1. Quit now, and add a default to the field in models.py |
186 |
? 2. Specify a one-off value to use for existing columns now |
187 |
? Please select a choice: 1 |
188 |
|
189 |
* Data migrations: |
190 |
|
191 |
If we need to do data migration as well, for example rename a field, we use the |
192 |
'datamigration' management command. |
193 |
|
194 |
In contrast with schemamigration, to perform complex data migration, we must |
195 |
write the script manually. The process is the following: |
196 |
|
197 |
1. Introduce the changes in the code and fixtures (initial data). |
198 |
2. Execute: |
199 |
|
200 |
$ ./bin/python manage.py datamigration <migration_name_here> |
201 |
|
202 |
For example: |
203 |
|
204 |
$ ./bin/python manage.py datamigration db rename_credit_wallet |
205 |
Created 0003_rename_credit_wallet.py. |
206 |
|
207 |
3. We edit the generated script. It contains two methods: forwards and |
208 |
backwards. |
209 |
|
210 |
For database operations (column additions, alter tables etc) we use the |
211 |
South database API (http://south.aeracode.org/docs/databaseapi.html). |
212 |
|
213 |
To access the data, we use the database reference (orm) provided as |
214 |
parameter in forwards, backwards method declarations in the migration |
215 |
script. For example: |
216 |
|
217 |
class Migration(DataMigration): |
218 |
|
219 |
def forwards(self, orm): |
220 |
orm.SynnefoUser.objects.all() |
221 |
|
222 |
4. To migrate the database to the latest version, we execute: |
223 |
|
224 |
./manage.py migrate db |
225 |
|
226 |
To see which migrations are applied: |
227 |
|
228 |
$ ./bin/python manage.py migrate db --list |
229 |
|
230 |
db |
231 |
(*) 0001_initial |
232 |
(*) 0002_auto__add_field_synnefouser_new_south_test_field |
233 |
(*) 0003_rename_credit_wallet |
234 |
|
235 |
More information and more thorough examples can be found in the South web site. |
236 |
|
237 |
http://south.aeracode.org/ |
238 |
|
239 |
|
240 |
UI Testing |
241 |
---------- |
242 |
The functional ui tests require the Selenium server and the synnefo app to |
243 |
be running. |
244 |
|
245 |
$ wget http://selenium.googlecode.com/files/selenium-server-standalone-2.0b2.jar |
246 |
$ java -jar selenium-server-standalone-2.0b2.jar & |
247 |
$ ./bin/python manage.py runserver & |
248 |
$ ./bin/python manage.py test ui |
249 |
|
250 |
|
251 |
Test coverage |
252 |
------------- |
253 |
|
254 |
In order to get code coverage reports you need to install django-test-coverage |
255 |
|
256 |
$ ./bin/pip install django-test-coverage |
257 |
|
258 |
Then edit your settings.py and configure the test runner: |
259 |
|
260 |
TEST_RUNNER = 'django-test-coverage.runner.run_tests' |
261 |
|
262 |
|
263 |
.. include:: i18n.rst |
264 |
|
265 |
Building Synnefo package |
266 |
------------------------ |
267 |
|
268 |
To create a python package from the Synnefo source code run:: |
269 |
|
270 |
$ python setup.py sdist |
271 |
|
272 |
this command will create a ``tar.gz`` python source package using |
273 |
the version number provided in ``setup.py``. |
274 |
|
275 |
Building Synnefo documentation |
276 |
------------------------------ |
277 |
|
278 |
Make sure you have ``sphinx`` installed. |
279 |
|
280 |
.. code-block:: bash |
281 |
|
282 |
$ cd docs |
283 |
$ make html |
284 |
|
285 |
html files are generated in ``docs/_build/html`` directory |