Revision fc2afa67
b/README.deploy | ||
---|---|---|
5 | 5 |
infrastructure and the dependencies among them. This document applies |
6 | 6 |
to Synnefo v0.3. |
7 | 7 |
|
8 |
|
|
8 |
|
|
9 | 9 |
- Node types |
10 | 10 |
|
11 | 11 |
Nodes in a Synnefo deployment belong in one of the following types: |
... | ... | |
44 | 44 |
Any WEBSERVER can issue commands to the GANETI-MASTER, over RAPI, to effect |
45 | 45 |
changes in the state of the VMs. The GANETI-MASTER runs the Ganeti request |
46 | 46 |
queue. |
47 |
Services: only on GANETI-MASTER: |
|
47 |
Services: |
|
48 |
only on GANETI-MASTER: |
|
48 | 49 |
the Synnefo Ganeti monitoring daemon [/ganeti/ganeti-eventd], |
49 | 50 |
and the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py]. |
51 |
on each GANETI_NODE: |
|
52 |
a deployment-specific KVM ifup script |
|
53 |
properly configured NFDHCPD |
|
54 |
|
|
50 | 55 |
|
51 |
As of v0.3, the Synnefo Django project needs to be installed on nodes
|
|
56 |
As of v0.4, the Synnefo Django project needs to be installed on nodes
|
|
52 | 57 |
of type WEBSERVER, LOGIC and on the GANETI-MASTER, with a properly configured |
53 | 58 |
settings.py. In later revisions, the specific parts of the Django project |
54 | 59 |
which need to run on each node type will be identified. |
55 | 60 |
|
56 |
The settings.py file for Django is derived by concatenating the
|
|
61 |
The settings.py file for Django may be derived by concatenating the
|
|
57 | 62 |
settings.py.dist file contained in the Synnefo distribution with a file |
58 | 63 |
containing custom modifications. This is recommended to minimize the load |
59 | 64 |
of reconstructing settings.py from scratch, since each release currently |
... | ... | |
64 | 69 |
|
65 | 70 |
This section describes each of the required service dependencies. |
66 | 71 |
|
72 |
* API implementation |
|
73 |
|
|
74 |
You need to load fixtures db/fixtures/{flavors,images}.json, |
|
75 |
which make the API usable by end users by defining a sample |
|
76 |
set of hardware configurations (flavors) and OS images. |
|
77 |
|
|
67 | 78 |
* RabbitMQ |
68 | 79 |
|
69 | 80 |
RabbitMQ is used as a generic message broker for the system. It should |
... | ... | |
111 | 122 |
hooks in Ganeti. It resides in the ganeti/ directory of the Synnefo |
112 | 123 |
project root. |
113 | 124 |
|
114 |
The hook needs to be enabled for the post-start phase, by *symlinking* |
|
115 |
in /etc/ganeti/hooks/instance-start-post.d on GANETI-MASTER: |
|
125 |
The hook needs to be enabled for phases |
|
126 |
post-{add,modify,reboot,start,stop} by *symlinking* |
|
127 |
in /etc/ganeti/hooks/instance-{add,modify,reboot,start,stop}-post.d |
|
128 |
on GANETI-MASTER, e.g.: |
|
116 | 129 |
|
117 | 130 |
root@ganeti-master:/etc/ganeti/hooks/instance-start-post.d# ls -l |
118 | 131 |
lrwxrwxrwx 1 root root 45 May 3 13:45 00-snf-ganeti-hook -> /home/devel/synnefo/ganeti/snf-ganeti-hook.py* |
... | ... | |
150 | 163 |
repository, at https://code.grnet.gr/git/gnt-instance-image |
151 | 164 |
|
152 | 165 |
After installing gnt-instance-image do the following: |
153 |
1.root@ganeti-master:/path-to-repo# cp ./defaults /etc/default/ganeti-instance-image |
|
154 |
2.Uncomment the following in /etc/default/ganeti-instance-image: |
|
155 |
SWAP=no |
|
156 |
ARCH="x86_64" |
|
157 |
3.Add to /etc/default/ganeti-instance-image (so that make-dump makes no /boot partition): |
|
158 |
KERNEL_PATH="True" |
|
159 |
4.Change the path in make-dump line 22 according to your installation |
|
160 |
(/usr/share/ganeti/os/image/common.sh --> /srv/ganeti/os/image/common.sh) |
|
161 |
5.In common.sh, comment out the KERNEL_PATH variable initialization. |
|
162 |
(#KERNEL_PATH="$INSTANCE_HV_kernel_path") |
|
163 |
6.In /etc/ganeti/instance-image/hooks, make sure the hooks you want to |
|
164 |
run during instance creation process have execute permission. At least |
|
165 |
`grub' and `root_passwd' should be triggered to make the instance |
|
166 |
usable: |
|
167 |
chmod +x /etc/ganeti/instance-image/hooks/{grub,root_passwd} |
|
166 |
1. root@ganeti-master:/path-to-repo# cp ./defaults /etc/default/ganeti-instance-image
|
|
167 |
2. Uncomment the following in /etc/default/ganeti-instance-image:
|
|
168 |
SWAP=no
|
|
169 |
ARCH="x86_64"
|
|
170 |
3. Add to /etc/default/ganeti-instance-image (so that make-dump makes no /boot partition):
|
|
171 |
KERNEL_PATH="True"
|
|
172 |
4. Change the path in make-dump line 22 according to your installation
|
|
173 |
(/usr/share/ganeti/os/image/common.sh --> /srv/ganeti/os/image/common.sh)
|
|
174 |
5. In common.sh, comment out the KERNEL_PATH variable initialization.
|
|
175 |
(#KERNEL_PATH="$INSTANCE_HV_kernel_path")
|
|
176 |
6. In /etc/ganeti/instance-image/hooks, make sure the hooks you want to
|
|
177 |
run during instance creation process have execute permission. At least
|
|
178 |
`grub' and `root_passwd' should be triggered to make the instance
|
|
179 |
usable:
|
|
180 |
chmod +x /etc/ganeti/instance-image/hooks/{grub,root_passwd}
|
|
168 | 181 |
|
169 | 182 |
Your Custom Images should be stored in a dump format under /var/cache/ganeti-instance-image |
170 | 183 |
and their filenames should have the following format: |
... | ... | |
172 | 185 |
e.g. debian-6.0.1a-x86_64-root.dump (backend_id = "debian-6.0.1a") |
173 | 186 |
|
174 | 187 |
|
175 |
-Administration |
|
188 |
|
|
189 |
- Administration |
|
190 |
|
|
176 | 191 |
* Reconciliation process: On certain occasions, such as a Ganeti or |
177 | 192 |
RabbitMQ failure, the VM state in the system's database may differ from |
178 | 193 |
that in the Ganeti installation. The reconciliation process is designed |
... | ... | |
197 | 212 |
|
198 | 213 |
On each invocation of the reconcile command, the system will trigger a |
199 | 214 |
reconciliation for ((num_all_vms/RECONCILIATION_MIN) * interval) |
200 |
machines. Obviously the less the interval value and the more the
|
|
201 |
RECONCILIATION_MIN setting, the less load is going to be put on Ganeti.
|
|
215 |
machines. Obviously the lower the interval value and the higher the
|
|
216 |
setting of RECONCILIATION_MIN, the less load is going to be put on Ganeti.
|
|
202 | 217 |
|
203 | 218 |
|
204 | 219 |
- OS Specific instructions |
b/README.develop | ||
---|---|---|
23 | 23 |
if the invitations application is deployed, the following |
24 | 24 |
dependencies should be installed |
25 | 25 |
|
26 |
-pycrypto==2.1.0 |
|
26 |
- pycrypto==2.1.0
|
|
27 | 27 |
|
28 |
if you want to test the ganeti |
|
28 |
if you want to test the ganeti (FIXME)
|
|
29 | 29 |
|
30 | 30 |
to run the user interface tests, selenium must be installed |
31 | 31 |
- selenium [?] |
... | ... | |
33 | 33 |
Preparing the development environment |
34 | 34 |
------------------------------------- |
35 | 35 |
|
36 |
1. Prepare the system |
|
37 |
The easiest method is to setup a working environment through virtualenv. |
|
38 |
Alternatively, you can use your system's package manager to install |
|
39 |
the dependencies (e.g. Macports has them all). |
|
36 |
1. Prepare the system: The easiest method is to setup a working environment |
|
37 |
through virtualenv. Alternatively, you can use your system's package manager |
|
38 |
to install the dependencies (e.g. Macports has them all). |
|
40 | 39 |
|
41 |
*On Snow Leopard and linux (64-bit), you have to set the following environment |
|
42 |
variable for pip to compile the dependencies correctly. |
|
40 |
* On Snow Leopard and linux (64-bit), you have to set the following environment
|
|
41 |
variable for pip to compile the dependencies correctly.
|
|
43 | 42 |
|
44 |
$export ARCHFLAGS="-arch x86_64" |
|
43 |
$ export ARCHFLAGS="-arch x86_64"
|
|
45 | 44 |
|
46 |
*On Ubuntu, a few more packages must be installed before installing the |
|
47 |
prerequisite Python libraries |
|
45 |
* On Ubuntu, a few more packages must be installed before installing the
|
|
46 |
prerequisite Python libraries
|
|
48 | 47 |
|
49 |
$sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev |
|
48 |
$ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
|
|
50 | 49 |
|
51 | 50 |
2. Checkout the code and install the Python prerequisites. This assumes |
52 | 51 |
that python is already installed on the host. |
... | ... | |
61 | 60 |
3. At this point you should have all required dependencies installed. Now you |
62 | 61 |
have to select a database engine. The choices are: postgres, mysql and sqlite. |
63 | 62 |
|
64 |
-SQLite |
|
63 |
- SQLite
|
|
65 | 64 |
The python sqlite driver is available by default with Python so no additional |
66 | 65 |
configuration is required. Also, most self-respecting systems have the sqlite |
67 | 66 |
library installed by default. |
68 | 67 |
|
69 |
-MySQL |
|
68 |
- MySQL
|
|
70 | 69 |
MySQL must be installed first |
71 | 70 |
|
72 |
*Ubuntu - Debian |
|
73 |
$sudo apt-get install libmysqlclient-dev |
|
71 |
* Ubuntu - Debian
|
|
72 |
$ sudo apt-get install libmysqlclient-dev
|
|
74 | 73 |
|
75 |
*MacPorts |
|
74 |
* MacPorts
|
|
76 | 75 |
$sudo port install mysql5 |
77 | 76 |
|
78 | 77 |
Install the MySQL python library |
... | ... | |
93 | 92 |
mysql> show databases; |
94 | 93 |
mysql> GRANT ALL on synnefo.* TO username IDENTIFIED BY 'password'; |
95 | 94 |
|
96 |
-Postgres |
|
95 |
- Postgres
|
|
97 | 96 |
|
98 |
#Ubuntu - Debian |
|
97 |
# Ubuntu - Debian
|
|
99 | 98 |
$ sudo apt-get install postgresql-8.4 libpq-dev |
100 | 99 |
|
101 |
#MacPorts |
|
100 |
# MacPorts
|
|
102 | 101 |
$ sudo port install postgresql84 |
103 | 102 |
|
104 | 103 |
Install the postgres Python library |
... | ... | |
130 | 129 |
|
131 | 130 |
and then copy/edit according to the database used: |
132 | 131 |
|
133 |
-SQLite |
|
132 |
- SQLite
|
|
134 | 133 |
|
135 | 134 |
PROJECT_PATH = os.path.dirname(os.path.abspath(__file__)) + '/' |
136 | 135 |
|
... | ... | |
141 | 140 |
} |
142 | 141 |
} |
143 | 142 |
|
144 |
-MySQL |
|
143 |
- MySQL
|
|
145 | 144 |
DATABASES = { |
146 | 145 |
'default': { |
147 | 146 |
'ENGINE': 'django.db.backends.mysql', |
... | ... | |
156 | 155 |
} |
157 | 156 |
} |
158 | 157 |
|
159 |
-Postgres |
|
158 |
- Postgres
|
|
160 | 159 |
|
161 | 160 |
DATABASES = { |
162 | 161 |
'default': { |
... | ... | |
174 | 173 |
|
175 | 174 |
$ ./bin/python manage.py sql db |
176 | 175 |
|
177 |
6. Create the DB and (optionally) load test data
|
|
176 |
6. Create the DB and load initial data:
|
|
178 | 177 |
|
179 | 178 |
$ ./bin/python manage.py syncdb |
180 | 179 |
$ ./bin/python manage.py migrate db |
... | ... | |
204 | 203 |
|
205 | 204 |
10. (Hopefully) Done |
206 | 205 |
|
206 |
|
|
207 | 207 |
South Database Migrations |
208 | 208 |
------------------------ |
209 | 209 |
|
210 |
*Initial Migration |
|
210 |
* Initial Migration
|
|
211 | 211 |
|
212 |
First, remember to add the south app to settings.py (it is already included in the
|
|
213 |
settings.py.dist). |
|
212 |
First, remember to add the south app to settings.py (it is already included in |
|
213 |
the settings.py.dist).
|
|
214 | 214 |
|
215 |
To initialise south migrations in your database the following commands must be executed: |
|
215 |
To initialise south migrations in your database the following commands must be |
|
216 |
executed: |
|
216 | 217 |
|
217 | 218 |
$ ./bin/python manage.py syncdb # Create / update the database with the south tables |
218 | 219 |
$ ./bin/python manage.py migrate db # Perform migration in the database |
219 | 220 |
|
220 |
Note that syncdb will create the latest models that exist in the db app, so some migrations may fail. |
|
221 |
If you are sure a migration has already taken place you must use the "--fake" option, to apply it. |
|
221 |
Note that syncdb will create the latest models that exist in the db app, so |
|
222 |
some migrations may fail. If you are sure a migration has already taken place |
|
223 |
you must use the "--fake" option, to apply it. |
|
222 | 224 |
|
223 | 225 |
For example: |
224 | 226 |
|
... | ... | |
230 | 232 |
|
231 | 233 |
All starred migrations are applied. |
232 | 234 |
|
233 |
Remember, the migration is performed mainly for the data, not for the database schema. If you do not want to migrate the |
|
234 |
data, a syncdb and fake migrations for all the migration versions will suffice. |
|
235 |
Remember, the migration is performed mainly for the data, not for the database |
|
236 |
schema. If you do not want to migrate the data, a syncdb and fake migrations |
|
237 |
for all the migration versions will suffice. |
|
235 | 238 |
|
236 |
*Schema migrations: |
|
239 |
* Schema migrations:
|
|
237 | 240 |
|
238 | 241 |
Do not use the syncdb management command. It can only be used |
239 |
the first time and/or if you drop the database and must recreate it from scratch. |
|
242 |
the first time and/or if you drop the database and must recreate it from |
|
243 |
scratch. |
|
240 | 244 |
See "Initial Migration" section. |
241 | 245 |
|
242 |
Each time you make changes to the database and data migration is not required (WARNING: always
|
|
243 |
perform this with extreme care): |
|
246 |
Every time you make changes to the database and data migration is not required
|
|
247 |
(WARNING: always perform this with extreme care):
|
|
244 | 248 |
|
245 | 249 |
$ ./bin/python manage.py schemamigration db --auto |
246 | 250 |
|
247 |
The above will create the migration script. Now this must be applied to the live database. |
|
251 |
The above will create the migration script. Now this must be applied to the |
|
252 |
live database. |
|
248 | 253 |
|
249 | 254 |
$ ./bin/python migrate db |
250 | 255 |
|
... | ... | |
264 | 269 |
Installing json fixture 'initial_data' from '/home/bkarak/devel/synnefo/../synnefo/db/fixtures'. |
265 | 270 |
Installed 1 object(s) from 1 fixture(s) |
266 | 271 |
|
267 |
South needs some extra definitions to the model to preserve and migrate the existing data, for example, if we add a field |
|
268 |
in a model, we should declare its default value. If not, South will propably fail, after indicating the error. |
|
272 |
South needs some extra definitions to the model to preserve and migrate the |
|
273 |
existing data, for example, if we add a field in a model, we should declare its |
|
274 |
default value. If not, South will propably fail, after indicating the error. |
|
269 | 275 |
|
270 | 276 |
$ ./bin/python manage.py schemamigration db --auto |
271 | 277 |
? The field 'SynnefoUser.new_south_field_2' does not have a default specified, yet is NOT NULL. |
... | ... | |
275 | 281 |
? 2. Specify a one-off value to use for existing columns now |
276 | 282 |
? Please select a choice: 1 |
277 | 283 |
|
278 |
*Data migrations: |
|
284 |
* Data migrations:
|
|
279 | 285 |
|
280 |
If we need to do data migration as well, for example rename a field, we use tha 'datamigration' management command. |
|
286 |
If we need to do data migration as well, for example rename a field, we use tha |
|
287 |
'datamigration' management command. |
|
281 | 288 |
|
282 |
In contrast with schemamigration, to perform complex data migration, we must write the script manually. The process is
|
|
283 |
the following: |
|
289 |
In contrast with schemamigration, to perform complex data migration, we must |
|
290 |
write the script manually. The process is the following:
|
|
284 | 291 |
|
285 | 292 |
1. Introduce the changes in the code and fixtures (initial data). |
286 | 293 |
2. Execute: |
... | ... | |
292 | 299 |
$ ./bin/python manage.py datamigration db rename_credit_wallet |
293 | 300 |
Created 0003_rename_credit_wallet.py. |
294 | 301 |
|
295 |
3. We edit the generated script. It contains two methods: forwards and backwards. |
|
302 |
3. We edit the generated script. It contains two methods: forwards and |
|
303 |
backwards. |
|
296 | 304 |
|
297 |
For database operations (column additions, alter tables etc) we use the South database API
|
|
298 |
(http://south.aeracode.org/docs/databaseapi.html). |
|
305 |
For database operations (column additions, alter tables etc) we use the |
|
306 |
South database API (http://south.aeracode.org/docs/databaseapi.html).
|
|
299 | 307 |
|
300 |
To access the data, we use the database reference (orm) provided as parameter in forwards, backwards method declarations |
|
301 |
in the migration script. For example: |
|
308 |
To access the data, we use the database reference (orm) provided as |
|
309 |
parameter in forwards, backwards method declarations in the migration |
|
310 |
script. For example: |
|
302 | 311 |
|
303 | 312 |
class Migration(DataMigration): |
304 | 313 |
|
... | ... | |
322 | 331 |
|
323 | 332 |
http://south.aeracode.org/ |
324 | 333 |
|
334 |
|
|
325 | 335 |
UI Testing |
326 | 336 |
---------- |
327 | 337 |
The functional ui tests require the Selenium server and the synnefo app to |
... | ... | |
332 | 342 |
$ ./bin/python manage.py runserver & |
333 | 343 |
$ ./bin/python manage.py test ui |
334 | 344 |
|
345 |
|
|
335 | 346 |
Test coverage |
336 | 347 |
------------- |
337 | 348 |
|
Also available in: Unified diff