root / docs / src / develop.rst @ c9a968e8
History | View | Annotate | Download (8.5 kB)
1 |
Developers guide |
---|---|
2 |
================ |
3 |
|
4 |
Information on how to setup a development environment. |
5 |
|
6 |
This file documents the installation of a development environment for Synnefo. |
7 |
It should be read alongside README.deploy. |
8 |
|
9 |
It contains development-specific ammendments to the basic deployment steps |
10 |
outlined in README.deploy, and development-specific notes. |
11 |
|
12 |
Installing the development environment |
13 |
-------------------------------------- |
14 |
|
15 |
For a basic development environment you need to follow steps 0-15 |
16 |
of README.deploy, which should be read in its entirety *before* this document. |
17 |
|
18 |
Development-specific guidelines on each step: |
19 |
|
20 |
|
21 |
0. Allocation of physical nodes: |
22 |
Node types DB, APISERVER, LOGIC may all be run on the same physical machine, |
23 |
usually, your development workstation. |
24 |
|
25 |
Nodes of type GANETI-MASTER, GANETI-NODES and QUEUE are already provided |
26 |
by the development Ganeti backend. Access credentials are provided in |
27 |
settings.py.dist. |
28 |
|
29 |
|
30 |
1. You do not need to install your own Ganeti installation. |
31 |
Use the RAPI endpoint as contained in settings.py.dist. |
32 |
|
33 |
|
34 |
2. You do not need to setup your own RabbitMQ nodes, use the AMQP endpoints |
35 |
contained in settings.py.dist. |
36 |
|
37 |
3. For development purposes, Django's own development |
38 |
`server, ./manage.py runserver' will suffice. |
39 |
|
40 |
|
41 |
4. Use a virtual environment to install the Django project, or packages provided |
42 |
by your distribution. |
43 |
|
44 |
|
45 |
5. Install a DB of your own, or use the PostgreSQL instance available on the |
46 |
development backend. |
47 |
|
48 |
|
49 |
6. As is. |
50 |
|
51 |
|
52 |
7. The following fixtures can be loaded optionally depending on |
53 |
testing/development requirements, and are not needed in a production setup: |
54 |
|
55 |
$ ./bin/python manage.py loaddata db/fixtures/vms.json |
56 |
$ ./bin/python manage.py loaddata db/fixtures/disks.json |
57 |
|
58 |
|
59 |
8. MAKE SURE you setup a distinct BACKEND_PREFIX_ID, e.g., use your commit |
60 |
username. |
61 |
|
62 |
|
63 |
9. The Ganeti monitoring daemon from the latest Synnefo release is already |
64 |
running on the development Ganeti master. You may also run your own, on your |
65 |
own Ganeti backend if you so wish. |
66 |
|
67 |
|
68 |
10.As is. |
69 |
|
70 |
11.The Synnefo Ganeti hook is already running on the development backend, |
71 |
sending notifications over AMQP. |
72 |
|
73 |
|
74 |
12.The VNC authentication proxy is already running on the Ganeti development |
75 |
backend. You *cannot* run your own, unless you install your own Ganeti |
76 |
backend, because it needs direct access to the hypervisor's VNC port on |
77 |
GANETI-NODEs. |
78 |
|
79 |
Note: You still need to install the vncauthproxy package to satisfy |
80 |
the dependency of the API on the vncauthproxy client. See Synnefo #807 |
81 |
for more details. |
82 |
|
83 |
|
84 |
13.The development Ganeti backend already has a number of OS Images available. |
85 |
|
86 |
|
87 |
14.The development Ganeti backend already has a number of pre-provisioned |
88 |
bridges available, per each BACKEND_PREFIX_ID. |
89 |
|
90 |
To setup simple NAT-based networking on a Ganeti backend on your own, |
91 |
please see the provided patches under contrib/patches/. |
92 |
You will need minor patches to the sample KVM ifup hook, kvm-vif-bridge, |
93 |
and a small patch to NFDHCPD to enable it to work with bridged tap+ |
94 |
interfaces. To support bridged tap interfaces you also need to patch the |
95 |
python-nfqueue package, patches against python-nfqueue-0.3 [part of Debian |
96 |
Sid] are also provided under contrib/patches/. |
97 |
|
98 |
|
99 |
15.As is. |
100 |
|
101 |
|
102 |
16.As is. |
103 |
|
104 |
|
105 |
17.[OPTIONAL] Create settings.d/99-local.conf and insert local overrides for |
106 |
settings.d/*. This will allow pulling new files without needing to reapply |
107 |
local any local modifications. |
108 |
|
109 |
|
110 |
South Database Migrations |
111 |
------------------------= |
112 |
|
113 |
* Initial Migration |
114 |
|
115 |
First, remember to add the south app to settings.py (it is already included in |
116 |
the settings.py.dist). |
117 |
|
118 |
To initialise south migrations in your database the following commands must be |
119 |
executed: |
120 |
|
121 |
$ ./bin/python manage.py syncdb # Create / update the database with the south tables |
122 |
$ ./bin/python manage.py migrate db # Perform migration in the database |
123 |
|
124 |
Note that syncdb will create the latest models that exist in the db app, so some |
125 |
migrations may fail. If you are sure a migration has already taken place you |
126 |
must use the "--fake" option, to apply it. |
127 |
|
128 |
For example: |
129 |
|
130 |
$ ./bin/python manage.py migrate db 0001 --fake |
131 |
|
132 |
To be sure that all migrations are applied type: |
133 |
|
134 |
$ ./bin/python manage.py migrate db --list |
135 |
|
136 |
All starred migrations are applied. |
137 |
|
138 |
Remember, the migration is performed mainly for the data, not for the database |
139 |
schema. If you do not want to migrate the data, a syncdb and fake migrations for |
140 |
all the migration versions will suffice. |
141 |
|
142 |
* Schema migrations: |
143 |
|
144 |
Do not use the syncdb management command. It can only be used the first time |
145 |
and/or if you drop the database and must recreate it from scratch. See |
146 |
"Initial Migration" section. |
147 |
|
148 |
Every time you make changes to the database and data migration is not required |
149 |
(WARNING: always perform this with extreme care): |
150 |
|
151 |
$ ./bin/python manage.py schemamigration db --auto |
152 |
|
153 |
The above will create the migration script. Now this must be applied to the live |
154 |
database. |
155 |
|
156 |
$ ./bin/python migrate db |
157 |
|
158 |
Consider this example (adding a field to the SynnefoUser model): |
159 |
|
160 |
$ ./bin/python manage.py schemamigration db --auto |
161 |
+ Added field new_south_test_field on db.SynnefoUser |
162 |
Created 0002_auto__add_field_synnefouser_new_south_test_field.py. |
163 |
|
164 |
You can now apply this migration with: ./manage.py migrate db |
165 |
|
166 |
$ ./manage.py migrate db |
167 |
Running migrations for db: |
168 |
- Migrating forwards to 0002_auto__add_field_synnefouser_new_south_test_field. |
169 |
> db:0002_auto__add_field_synnefouser_new_south_test_field |
170 |
- Loading initial data for db. |
171 |
Installing json fixture 'initial_data' from '/home/bkarak/devel/synnefo/../synnefo/db/fixtures'. |
172 |
Installed 1 object(s) from 1 fixture(s) |
173 |
|
174 |
South needs some extra definitions to the model to preserve and migrate the |
175 |
existing data, for example, if we add a field in a model, we should declare its |
176 |
default value. If not, South will propably fail, after indicating the error. |
177 |
|
178 |
$ ./bin/python manage.py schemamigration db --auto |
179 |
? The field 'SynnefoUser.new_south_field_2' does not have a default specified, yet is NOT NULL. |
180 |
? Since you are adding or removing this field, you MUST specify a default |
181 |
? value to use for existing rows. Would you like to: |
182 |
? 1. Quit now, and add a default to the field in models.py |
183 |
? 2. Specify a one-off value to use for existing columns now |
184 |
? Please select a choice: 1 |
185 |
|
186 |
* Data migrations: |
187 |
|
188 |
If we need to do data migration as well, for example rename a field, we use the |
189 |
'datamigration' management command. |
190 |
|
191 |
In contrast with schemamigration, to perform complex data migration, we must |
192 |
write the script manually. The process is the following: |
193 |
|
194 |
1. Introduce the changes in the code and fixtures (initial data). |
195 |
2. Execute: |
196 |
|
197 |
$ ./bin/python manage.py datamigration <migration_name_here> |
198 |
|
199 |
For example: |
200 |
|
201 |
$ ./bin/python manage.py datamigration db rename_credit_wallet |
202 |
Created 0003_rename_credit_wallet.py. |
203 |
|
204 |
3. We edit the generated script. It contains two methods: forwards and |
205 |
backwards. |
206 |
|
207 |
For database operations (column additions, alter tables etc) we use the |
208 |
South database API (http://south.aeracode.org/docs/databaseapi.html). |
209 |
|
210 |
To access the data, we use the database reference (orm) provided as |
211 |
parameter in forwards, backwards method declarations in the migration |
212 |
script. For example: |
213 |
|
214 |
class Migration(DataMigration): |
215 |
|
216 |
def forwards(self, orm): |
217 |
orm.SynnefoUser.objects.all() |
218 |
|
219 |
4. To migrate the database to the latest version, we execute: |
220 |
|
221 |
./manage.py migrate db |
222 |
|
223 |
To see which migrations are applied: |
224 |
|
225 |
$ ./bin/python manage.py migrate db --list |
226 |
|
227 |
db |
228 |
(*) 0001_initial |
229 |
(*) 0002_auto__add_field_synnefouser_new_south_test_field |
230 |
(*) 0003_rename_credit_wallet |
231 |
|
232 |
More information and more thorough examples can be found in the South web site. |
233 |
|
234 |
http://south.aeracode.org/ |
235 |
|
236 |
|
237 |
UI Testing |
238 |
---------- |
239 |
The functional ui tests require the Selenium server and the synnefo app to |
240 |
be running. |
241 |
|
242 |
$ wget http://selenium.googlecode.com/files/selenium-server-standalone-2.0b2.jar |
243 |
$ java -jar selenium-server-standalone-2.0b2.jar & |
244 |
$ ./bin/python manage.py runserver & |
245 |
$ ./bin/python manage.py test ui |
246 |
|
247 |
|
248 |
Test coverage |
249 |
------------- |
250 |
|
251 |
In order to get code coverage reports you need to install django-test-coverage |
252 |
|
253 |
$ ./bin/pip install django-test-coverage |
254 |
|
255 |
Then edit your settings.py and configure the test runner: |
256 |
|
257 |
TEST_RUNNER = 'django-test-coverage.runner.run_tests' |
258 |
|
259 |
|
260 |
.. include:: i18n.rst |
261 |
|
262 |
Building synnefo package |
263 |
------------------------ |
264 |
|
265 |
.. todo:: document building instructions |