Statistics
| Branch: | Tag: | Revision:

root / README.develop @ c0f6fb49

History | View | Annotate | Download (8.3 kB)

1
DEVELOP.txt - Information on how to setup a development environment.
2

    
3
This file documents the installation of a development environment for Synnefo.
4
It should be read alongside README.deploy.
5

    
6
It contains development-specific ammendments to the basic deployment steps
7
outlined in README.deploy, and development-specific notes.
8

    
9

    
10
Installing the development environment
11
======================================
12

    
13
For a basic development environment you need to follow steps 0-15
14
of README.deploy, which should be read in its entirety *before* this document.
15

    
16
Development-specific guidelines on each step:
17

    
18

    
19
0. Allocation of physical nodes:
20
   Node types DB, APISERVER, LOGIC may all be run on the same physical machine,
21
   usually, your development workstation.
22

    
23
   Nodes of type GANETI-MASTER, GANETI-NODES and QUEUE are already provided
24
   by the development Ganeti backend. Access credentials are provided in
25
   settings.py.dist.
26

    
27

    
28
1. You do not need to install your own Ganeti installation.
29
   Use the RAPI endpoint as contained in settings.py.dist.
30

    
31

    
32
2. You do not need to setup your own RabbitMQ nodes, use the AMQP endpoints
33
   contained in settings.py.dist. MAKE SURE to call fix_amqp_settings() after
34
   settings a custom BACKEND_PREFIX_ID.
35

    
36

    
37
3. For development purposes, Django's own development
38
   `server, ./manage.py runserver' will suffice.
39

    
40

    
41
4. Use a virtual environment to install the Django project, or packages provided
42
   by your distribution.
43

    
44

    
45
5. Install a DB of your own, or use the PostgreSQL instance available on the
46
   development backend.
47

    
48

    
49
6. As is.
50

    
51

    
52
7. The following fixtures can be loaded optionally depending on
53
   testing/development requirements, and are not needed in a production setup:
54

    
55
	$ ./bin/python manage.py loaddata db/fixtures/vms.json
56
	$ ./bin/python manage.py loaddata db/fixtures/disks.json
57

    
58

    
59
8. MAKE SURE you setup a distinct BACKEND_PREFIX_ID, e.g., use your commit
60
   username. Make sure you call fix_amqp_settings() to setup use of individual
61
   queues on RabbiMQ.
62

    
63

    
64
9. The Ganeti monitoring daemon from the latest Synnefo release is already
65
   running on the development Ganeti master. You may also run your own, on your
66
   own Ganeti backend if you so wish.
67

    
68

    
69
10.As is.
70

    
71
11.The Synnefo Ganeti hook is already running on the development backend,
72
   sending notifications over AMQP.
73

    
74

    
75
12.The VNC authentication proxy is already running on the Ganeti development
76
   backend. You *cannot* run your own, unless you install your own Ganeti
77
   backend, because it needs direct access to the hypervisor's VNC port on
78
   GANETI-NODEs.
79

    
80

    
81
13.The development Ganeti backend already has a number of OS Images available.
82

    
83

    
84
14.The development Ganeti backend already has a number of pre-provisioned
85
   bridges available, per each BACKEND_PREFIX_ID.
86

    
87
   To setup simple NAT-based networking on a Ganeti backend on your own,
88
   please see the provided patches under contrib/patches/.
89
   You will need minor patches to the sample KVM ifup hook, kvm-vif-bridge,
90
   and a small patch to NFDHCPD to enable it to work with bridged tap+
91
   interfaces. To support bridged tap interfaces you also need to patch the
92
   python-nfqueue package, patches against python-nfqueue-0.3 [part of Debian
93
   Sid] are also provided under contrib/patches/.
94

    
95

    
96
15.As is.
97

    
98

    
99
16.As is.
100

    
101

    
102
17.[OPTIONAL] Create settings.d/99-local.conf and insert local overrides for
103
   settings.d/*.  This will allow pulling new files without needing to reapply
104
   local any local modifications.
105

    
106

    
107
South Database Migrations
108
=========================
109

    
110
* Initial Migration
111

    
112
First, remember to add the south app to settings.py (it is already included in
113
the settings.py.dist).
114

    
115
To initialise south migrations in your database the following commands must be
116
executed:
117

    
118
    $ ./bin/python manage.py syncdb       # Create / update the database with the south tables
119
    $ ./bin/python manage.py migrate db   # Perform migration in the database
120

    
121
Note that syncdb will create the latest models that exist in the db app, so some
122
migrations may fail.  If you are sure a migration has already taken place you
123
must use the "--fake" option, to apply it.
124

    
125
For example:
126

    
127
    $ ./bin/python manage.py migrate db 0001 --fake
128

    
129
To be sure that all migrations are applied type:
130

    
131
    $ ./bin/python manage.py migrate db --list
132

    
133
All starred migrations are applied.
134

    
135
Remember, the migration is performed mainly for the data, not for the database
136
schema. If you do not want to migrate the data, a syncdb and fake migrations for
137
all the migration versions will suffice.
138

    
139
* Schema migrations:
140

    
141
Do not use the syncdb management command. It can only be used the first time
142
and/or if you drop the database and must recreate it from scratch. See
143
"Initial Migration" section.
144

    
145
Every time you make changes to the database and data migration is not required
146
(WARNING: always perform this with extreme care):
147

    
148
    $ ./bin/python manage.py schemamigration db --auto
149

    
150
The above will create the migration script. Now this must be applied to the live
151
database.
152

    
153
    $ ./bin/python migrate db
154

    
155
Consider this example (adding a field to the SynnefoUser model):
156

    
157
    $ ./bin/python manage.py schemamigration db --auto
158
     + Added field new_south_test_field on db.SynnefoUser
159
     Created 0002_auto__add_field_synnefouser_new_south_test_field.py.
160

    
161
  You can now apply this migration with: ./manage.py migrate db
162

    
163
    $ ./manage.py migrate db
164
     Running migrations for db:
165
     - Migrating forwards to 0002_auto__add_field_synnefouser_new_south_test_field.
166
     > db:0002_auto__add_field_synnefouser_new_south_test_field
167
     - Loading initial data for db.
168
    Installing json fixture 'initial_data' from '/home/bkarak/devel/synnefo/../synnefo/db/fixtures'.
169
    Installed 1 object(s) from 1 fixture(s)
170

    
171
South needs some extra definitions to the model to preserve and migrate the
172
existing data, for example, if we add a field in a model, we should declare its
173
default value. If not, South will propably fail, after indicating the error.
174

    
175
    $ ./bin/python manage.py schemamigration db --auto
176
     ? The field 'SynnefoUser.new_south_field_2' does not have a default specified, yet is NOT NULL.
177
     ? Since you are adding or removing this field, you MUST specify a default
178
     ? value to use for existing rows. Would you like to:
179
     ?  1. Quit now, and add a default to the field in models.py
180
     ?  2. Specify a one-off value to use for existing columns now
181
     ? Please select a choice: 1
182

    
183
* Data migrations:
184

    
185
If we need to do data migration as well, for example rename a field, we use the
186
'datamigration' management command.
187

    
188
In contrast with schemamigration, to perform complex data migration, we must
189
write the script manually. The process is the following:
190

    
191
    1. Introduce the changes in the code and fixtures (initial data).
192
    2. Execute:
193

    
194
    $ ./bin/python manage.py datamigration <migration_name_here>
195

    
196
    For example:
197

    
198
    $ ./bin/python manage.py datamigration db rename_credit_wallet
199
    Created 0003_rename_credit_wallet.py.
200

    
201
    3. We edit the generated script. It contains two methods: forwards and
202
    backwards.
203

    
204
    For database operations (column additions, alter tables etc) we use the
205
    South database API (http://south.aeracode.org/docs/databaseapi.html).
206

    
207
    To access the data, we use the database reference (orm) provided as
208
    parameter in forwards, backwards method declarations in the migration
209
    script. For example:
210

    
211
    class Migration(DataMigration):
212

    
213
    def forwards(self, orm):
214
        orm.SynnefoUser.objects.all()
215

    
216
    4. To migrate the database to the latest version, we execute:
217

    
218
    ./manage.py migrate db
219

    
220
To see which migrations are applied:
221

    
222
    $ ./bin/python manage.py migrate db --list
223

    
224
      db
225
        (*) 0001_initial
226
        (*) 0002_auto__add_field_synnefouser_new_south_test_field
227
        (*) 0003_rename_credit_wallet
228

    
229
More information and more thorough examples can be found in the South web site.
230

    
231
http://south.aeracode.org/
232

    
233

    
234
UI Testing
235
==========
236
The functional ui tests require the Selenium server and the synnefo app to
237
be running.
238

    
239
    $ wget http://selenium.googlecode.com/files/selenium-server-standalone-2.0b2.jar
240
    $ java -jar selenium-server-standalone-2.0b2.jar &
241
    $ ./bin/python manage.py runserver &
242
    $ ./bin/python manage.py test ui
243

    
244

    
245
Test coverage
246
=============
247

    
248
In order to get code coverage reports you need to install django-test-coverage
249

    
250
   $ ./bin/pip install django-test-coverage
251

    
252
Then edit your settings.py and configure the test runner:
253

    
254
   TEST_RUNNER = 'django-test-coverage.runner.run_tests'