Synnefo

DEVELOP.txt - Information on how to setup a development environment. 

Dependencies

------------

Synnefo is written in Python 2.6 and depends on the following Python modules

[package versions confirmed to be compatible are in braces]

- django 1.2 [Django==1.2.4]

- simplejson [simplejson==2.1.3]

- selenium [?]

- pyzmq-static [pyzmq==2.0.10.1]

- pycurl [pycurl==7.19.0]

- python-dateutil  [python-dateutil==1.4.1]

  WARNING: version python-dateutil==2.0 downloaded by pip known *not* to work with Python 2.6

- south [south==0.7.1]

also, depending on the database engine of choice, on one of the following:

- MySQL-python [MySQL-python==1.2.3]

- psycopg2 [psycopg2==2.4]

Preparing the development environment

-------------------------------------

1. Prepare the system 

The easiest method is to setup a working environment through virtualenv. 

Alternatively, you can use your system's package manager to install

the dependencies (e.g. Macports has them all).

*On Snow Leopard and linux (64-bit), you have to set the following environment variable for

pip to compile the dependencies correctly. 

	$export ARCHFLAGS="-arch x86_64"

*On Ubuntu, a few more packages must be installed before installing the

prerequisite Python libraries

	$sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev 

2. Checkout the code and install the Python prerequisites. This assumes

that python is already installed on the host.

    $ sudo easy_install virtualenv

    $ git clone https://user@code.grnet.gr/git/synnefo synnefo

    $ virtualenv --python=python2.6 synnefo --no-site-packages

    ...

    $ cd synnefo

    $ ./bin/pip install <list_of_prerequisites>

3. At this point you should have all required dependencies installed. Now you

have to select a database engine. The choices are: postgres, mysql and sqlite.

-SQLite

The python sqlite driver is available by default with Python so no additional 

configuration is required. Also, most self-respecting systems have the sqlite 

library installed by default.   

-MySQL

MySQL must be installed first

*Ubuntu - Debian

	$sudo apt-get install libmysqlclient-dev

*MacPorts

	$sudo port install mysql5

Install the MySQL python library

	$ bin/pip install MySQL-python

Note: On MacOSX with Mysql install from MacPorts the above command will fail

complaining that it cannot find the mysql_config command. Do the following and

restart the installation

	$ echo "mysql_config = /opt/local/bin/mysql_config5" >> ./build/MySQL-python/site.cfg

Configure a MySQL db/account for synnefo

	$ mysql -u root -p

	mysql> create database synnefo;

	mysql> show databases;

	mysql> GRANT ALL on synnefo.* TO username IDENTIFIED BY 'password';

-Postgres

#Ubuntu - Debian

	$ sudo apt-get install postgresql-8.4 libpq-dev

#MacPorts

	$ sudo port install postgresql84

Install the postgres Python library

	$ bin/pip install psycopg2

Configure a postgres db/account for synnefo

	Become the postgres user, connect to PostgreSQL:

	$ sudo su - postgres 

	$ psql

	Run the following commands:

	DROP DATABASE synnefo;

	DROP USER username;

	CREATE USER username WITH PASSWORD 'password';

	CREATE DATABASE synnefo;

	GRANT ALL PRIVILEGES ON DATABASE synnefo TO username;

	ALTER DATABASE synnefo OWNER TO username;

	ALTER USER username CREATEDB;

The last line enables the newly created user to create own databases. This is

needed for Django to create and drop the test_synnefo database for unit

testing.

4. At this point you should have a working DB. Now configure Django to access it: 

Copy the default configuration file 

    $ cp settings.py.dist settings.py

and then copy/edit according to the database used:  

-SQLite

	PROJECT_PATH = os.path.dirname(os.path.abspath(__file__)) + '/'

	DATABASES = {

	    'default': {

		'ENGINE': 'django.db.backends.sqlite3',

		'NAME': PROJECT_PATH + 'synnefo.db' #WARN: This must be an absolute path

	    }

	}

-MySQL

	DATABASES = {

	    'default': {

		'ENGINE': 'django.db.backends.mysql', 

		'NAME': 'synnefo',

		'USER': 'USERNAME'

		'PASSWORD': 'PASSWORD',

		'HOST': 'HOST',

		'PORT': 'PORT',

		'OPTIONS': {

		    'init_command': 'SET storage_engine=INNODB',

		}

	    }

	}

-Postgres    

    DATABASES = {

	    'default': {

		'ENGINE': 'django.db.backends.postgresql_psycopg2',

		'NAME': 'DATABASE',

		'USER': 'USERNAME',

		'PASSWORD': 'PASSWORD',

		'HOST': 'HOST',

		'PORT': 'PORT',

	    }

	}

5. Try it out. The following command will attempt to connect to the DB and 

print out DDL statements. It should not fail.

	$ ./bin/python manage.py sql db

6. Create the DB and (optionally) load test data

    $ ./bin/python manage.py syncdb

	$ ./bin/python manage.py loaddata db/fixtures/flavors.json

	$ ./bin/python manage.py loaddata db/fixtures/images.json

The following fictures can be loaded optionally depending on

testing requirements:

	$ ./bin/python manage.py loaddata db/fixtures/vms.json

	$ ./bin/python manage.py loaddata db/fixtures/initial_data.json

	$ ./bin/python manage.py loaddata db/fixtures/disks.json

7. Set the BACKEND_PREFIX_ID variable to some unique prefix, e.g. your commit

username 

8. Start the system

    $ ./bin/python db/db_controller.py  #DB synch daemon

    $ ./bin/python manage.py runserver  #Django

9. (Hopefully) Done

South Database Migration

------------------------

First, remember to add the south app to settings.py (it is already included in the

settings.py.dist).

In addition, do not use the syncdb management command. Only the first time

The first time (is not required, since when you read this documentation the migration

is already active) the migration of the schema will initialised with the following

command:

    $ ./bin/python schemamigration db --initial

Each time you make changes to the database and data migration is not required (WARNING: always

perform this with extreme care):

    $ ./bin/python schemamigration db --auto

The above will create the migration script. Now this must be applied to the live database.

    $ ./bin/python migrate db

Consider this example (adding a field to the SynnefoUser model):

    bkarak@nefarian:~/devel/synnefo$ ./bin/python manage.py schemamigration db --auto

     + Added field new_south_test_field on db.SynnefoUser

     Created 0002_auto__add_field_synnefouser_new_south_test_field.py. You can now apply this migration with: ./manage.py migrate db

    bkarak@nefarian:~/devel/synnefo$ ./manage.py migrate db

     Running migrations for db:

     - Migrating forwards to 0002_auto__add_field_synnefouser_new_south_test_field.

     > db:0002_auto__add_field_synnefouser_new_south_test_field

     - Loading initial data for db.

    Installing json fixture 'initial_data' from '/home/bkarak/devel/synnefo/../synnefo/db/fixtures'.

    Installed 1 object(s) from 1 fixture(s)

South needs some extra definitions to the model to preserve and migrate the existing data, for example, if we add a field

in a model, we should declare its default value. If not, south management will propably fail, after indicating the error.

If we need to do data migration as well, for example rename a field, we use tha 'datamigration' management command.

UI Testing

----------

The functional ui tests require the Selenium server and the synnefo app to 

be running.

    $ wget http://selenium.googlecode.com/files/selenium-server-standalone-2.0b2.jar

    $ java -jar selenium-server-standalone-2.0b2.jar &

    $ ./bin/python manage.py runserver &

    $ ./bin/python manage.py test ui

Test coverage

-------------

In order to get code coverage reports you need to install django-test-coverage

   $ ./bin/pip install django-test-coverage

Then edit your settings.py and configure the test runner:

   TEST_RUNNER = 'django-test-coverage.runner.run_tests'