Synnefo

Continuous integration with Jenkins

-----------------------------------

Preparing a GIT mirror

**********************

Jenkins cannot currently work with Git over encrypted HTTP. To solve this

problem we currently mirror the central Git repository locally on the jenkins

installation machine. To setup such a mirror do the following:

edit .netrc::

    machine code.grnet.gr

    login accountname

    password accountpasswd

Create the mirror::

    git clone --mirror https://code.grnet.gr/git/synnefo synnefo

Setup cron to pull from the mirror periodically. Ideally, Git mirror updates

should run just before Jenkins jobs check the mirror for changes::

    4,14,24,34,44,54 * * * * cd /path/to/mirror && git fetch && git remote prune origin

Jenkins setup

*************

The following instructions will setup Jenkins to run synnefo tests with the

SQLite database. To run the tests on MySQL and/or Postgres, step 5 must be

replicated. Also, the correct configuration file must be copied (line 6 of the

build script).

1. Install and start Jenkins. On Debian Squeeze:

   wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | apt-key add -

   echo "deb http://pkg.jenkins-ci.org/debian binary/" >>/etc/apt/sources.list

   echo "deb http://ppa.launchpad.net/chris-lea/zeromq/ubuntu lucid main" >> /etc/apt/sources.list

   sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C7917B12  

   sudo apt-get update

   sudo apt-get install jenkins

   Also install the following packages:

   apt-get install python-virtualenv libcurl3-gnutls libcurl3-gnutls-dev

                   uuid-dev libmysqlclient-dev libpq-dev libsqlite-dev

                   python-dev libzmq-dev

2. After Jenkins starts, go to

   http://$HOST:8080/pluginManager/

   and install the following plug-ins at

   -Jenkins Cobertura Plugin

   -Jenkins Email Extension Plugin

   -Jenkins GIT plugin

   -Jenkins SLOCCount Plug-in

   -Hudson/Jenkins Violations plugin

3. Configure the Jenkins user's Git details:

   su jenkins

   git config --global user.email "buildbot@lists.grnet.gr"

   git config --global user.name "Buildbot"

4. Make sure that all system-level dependencies specified in README.develop

   are correctly installed

5. Create a new "free-style software" job and set the following values::

    Project name: synnefo

    Source Code Management: Git

    URL of repository: Jenkins Git does not support HTTPS for checking out directly

                        from the repository. The temporary solution is to checkout

                        with a cron script in a directory and set the checkout path

                        in this field

    Branches to build: master and perhaps others

    Git->Advanced->Local subdirectory for repo (optional): synnefo

    Git->Advanced->Prune remote branches before build: check

    Repository browser: redmineweb,

                         URL: https://code.grnet.gr/projects/synnefo/repository/

    Build Triggers->Poll SCM: check

                     Schedule: # every five minutes

                   0,5,10,15,20,25,30,35,40,45,50,55 * * * * 

    Build -> Add build step-> Execute shell

    Command::

        #!/bin/bash -ex

        cd synnefo

        mkdir -p reports

        /usr/bin/sloccount --duplicates --wide --details api util ui logic auth > reports/sloccount.sc

        cp conf/ci/manage.py .

        if [ ! -e requirements.pip ]; then cp conf/ci/pip-1.2.conf requirements.pip; fi

        cat settings.py.dist conf/ci/settings.py.sqlite > settings.py

        python manage.py update_ve

        python manage.py hudson api db logic 

    Post-build Actions->Publish JUnit test result report: check

                         Test report XMLs: synnefo/reports/TEST-*.xml

    Post-build Actions->Publish Cobertura Coverage Report: check

                         Cobertura xml report pattern: synnefo/reports/coverage.xml

    Post-build Actions->Report Violations: check

                         pylint[XML filename pattern]: synnefo/reports/pylint.report

    Post-build Actions->Publish SLOCCount analysis results

                         SLOCCount reports: synnefo/reports/sloccount.sc

                         (also, remember to install sloccount at /usr/bin)

.. seealso::

    http://sites.google.com/site/kmmbvnr/home/django-hudson-tutorial