8 Ubuntu 12.04.3 (64) - Django 1.3.x
9 ==================================
11 This guide assumes that installation is carried out in /srv/flowspy directory. If other directory is to be used, please change the corresponding configuration files. It is also assumed that the root user will perform every action.
13 Required system packages
14 ------------------------
16 Update and install the required packages::
20 apt-get install mysql-server apache2 memcached libapache2-mod-proxy-html gunicorn beanstalkd python-django python-django-extensions python-django-south python-django-tinymce tinymce python-mysqldb python-yaml python-memcache python-django-registration python-ipaddr python-lxml mysql-client git python-django-celery python-paramiko python-gevent vim
23 Set username and password for mysql if used
26 If you wish to deploy an outgoing mail server, now it is time to do it. Otherwise you could set FoD to send out mails via a third party account
30 If you are using mysql, you should create a database::
32 mysql -u root -p -e 'create database fod'
35 Required application packages
36 -----------------------------
37 Get the required packages and install them
39 - ncclient: NETCONF python client::
42 git clone https://github.com/leopoul/ncclient.git
44 python setup.py install
46 - nxpy: Python Objects from/to XML proxy::
49 git clone https://code.grnet.gr/git/nxpy
51 python setup.py install
53 - flowspy: core application. Installation is done at /srv/flowspy::
56 git clone https://code.grnet.gr/git/flowspy
59 Application configuration
60 =========================
62 Copy settings.py.dist to settings.py::
64 cp settings.py.dist settings.py
66 Edit settings.py file and set the following according to your configuration::
68 ADMINS: set your admin name and email (assuming that your server can send notifications)
69 DATABASES (to point to your local database). You could use views instead of tables for models: peer, peercontacts, peernetworks. For this to work we suggest MySQL with MyISAM db engine
70 SECRET_KEY : Make this unique, and don't share it with anybody
71 STATIC_URL (static media directory) . If you have followed the above this should be: /srv/flowspy/static
72 TEMPLATE_DIRS : If you have followed the above this should be: /srv/flowspy/templates
73 CACHE_BACKEND: If you have followed the above this should be: memcached://127.0.0.1:11211/?timeout=3600
74 Alternatively you could go for redis with the corresponding Django client lib.
75 NETCONF_DEVICE (tested with Juniper EX4200 but any BGP enabled Juniper should work). This is the flowspec capable device
76 NETCONF_USER (enable ssh and netconf on device)
78 If beanstalk is selected the following should be left intact.
79 BROKER_HOST (beanstalk host)
80 BROKER_PORT (beanstalk port)
83 If beanstalk is selected the following should be left intact.
84 BROKER_URL (beanstalk url)
85 SHIB_AUTH_ENTITLEMENT (if you go for Shibboleth authentication)
86 NOTIFY_ADMIN_MAILS (bcc mail addresses)
87 PROTECTED_SUBNETS (subnets for which source or destination address will prevent rule creation and notify the NOTIFY_ADMIN_MAILS)
88 The whois client is meant to be used in case you have inserted peers with their ASes in the peers table and wish to get network info for each one in an automated manner.
91 If you wish to deploy FoD with Shibboleth change the following attributes according to your setup:
92 SHIB_AUTH_ENTITLEMENT = 'urn:mace'
93 SHIB_ADMIN_DOMAIN = 'example.com'
94 SHIB_LOGOUT_URL = 'https://example.com/Shibboleth.sso/Logout'
95 SHIB_USERNAME = ['HTTP_EPPN']
96 SHIB_MAIL = ['mail', 'HTTP_MAIL', 'HTTP_SHIB_INETORGPERSON_MAIL']
97 SHIB_FIRSTNAME = ['HTTP_SHIB_INETORGPERSON_GIVENNAME']
98 SHIB_LASTNAME = ['HTTP_SHIB_PERSON_SURNAME']
99 SHIB_ENTITLEMENT = ['HTTP_SHIB_EP_ENTITLEMENT']
101 If you have not installed an outgoing mail server you can always use your own account (either corporate or gmail, hotmail ,etc) by adding the following lines in settings.py::
103 EMAIL_USE_TLS = True #(or False)
104 EMAIL_HOST = 'smtp.example.com'
105 EMAIL_HOST_USER = 'username'
106 EMAIL_HOST_PASSWORD = 'yourpassword'
107 EMAIL_PORT = 587 #(outgoing)
111 Soon we will release a version with django-registration as a means to add users and Shibboleth as an alternative
113 Let's move on with some copies and dir creations::
115 cp urls.py.dist urls.py
117 chown -R root:www-data log/
122 Apache operates as a gunicorn Proxy with WSGI and Shibboleth modules enabled.
123 Depending on the setup the apache configuration may vary::
130 If shibboleth is to be used::
132 apt-get install libapache2-mod-shib2
135 Now it is time to configure beanstalk, gunicorn, celery and apache.
140 Enable beanstalk by editting /etc/default/beanstalkd::
142 vim /etc/default/beanstalkd
144 Uncomment the line **START=yes** to enable beanstalk
148 service beanstalkd start
153 create and edit /etc/gunicorn.d/fod::
155 vim /etc/gunicorn.d/fod
157 FoD is served via gunicorn and is then proxied by Apache. If the above directory conventions have been followed so far, then your configuration should be::
161 'working_dir': '/srv/flowspy',
163 '--bind=127.0.0.1:8081',
166 '--worker-class=egg:gunicorn#gevent',
175 Celery is used over beanstalkd to apply firewall rules in a serial manner so that locks are avoided on the flowspec capable device. In our setup celery runs via django. That is why the python-django-celery package was installed.
177 Create the celeryd daemon at /etc/init.d/celeryd::
179 vim /etc/init.d/celeryd
181 The configuration should be::
184 # ============================================
185 # celeryd - Starts the Celery worker daemon.
186 # ============================================
188 # :Usage: /etc/init.d/celeryd {start|stop|force-reload|restart|try-restart|status}
189 # :Configuration file: /etc/default/celeryd
191 # See http://docs.celeryq.org/en/latest/cookbook/daemonizing.html#init-script-celeryd
196 # Required-Start: $network $local_fs $remote_fs
197 # Required-Stop: $network $local_fs $remote_fs
198 # Default-Start: 2 3 4 5
199 # Default-Stop: 0 1 6
200 # Short-Description: celery task worker daemon
205 DEFAULT_PID_FILE="/var/run/celeryd@%n.pid"
206 DEFAULT_LOG_FILE="/var/log/celeryd@%n.log"
207 DEFAULT_LOG_LEVEL="INFO"
208 DEFAULT_NODES="celery"
209 DEFAULT_CELERYD="-m celery.bin.celeryd_detach"
211 # /etc/init.d/celeryd: start and stop the celery task worker daemon.
213 CELERY_DEFAULTS=${CELERY_DEFAULTS:-"/etc/default/celeryd"}
215 test -f "$CELERY_DEFAULTS" && . "$CELERY_DEFAULTS"
216 if [ -f "/etc/default/celeryd" ]; then
217 . /etc/default/celeryd
220 CELERYD_PID_FILE=${CELERYD_PID_FILE:-${CELERYD_PIDFILE:-$DEFAULT_PID_FILE}}
221 CELERYD_LOG_FILE=${CELERYD_LOG_FILE:-${CELERYD_LOGFILE:-$DEFAULT_LOG_FILE}}
222 CELERYD_LOG_LEVEL=${CELERYD_LOG_LEVEL:-${CELERYD_LOGLEVEL:-$DEFAULT_LOG_LEVEL}}
223 CELERYD_MULTI=${CELERYD_MULTI:-"celeryd-multi"}
224 CELERYD=${CELERYD:-$DEFAULT_CELERYD}
225 CELERYCTL=${CELERYCTL:="celeryctl"}
226 CELERYD_NODES=${CELERYD_NODES:-$DEFAULT_NODES}
231 CELERYD_OPTS="$CELERYD_OPTS $2"
234 CELERYD_LOG_DIR=`dirname $CELERYD_LOG_FILE`
235 CELERYD_PID_DIR=`dirname $CELERYD_PID_FILE`
236 if [ ! -d "$CELERYD_LOG_DIR" ]; then
237 mkdir -p $CELERYD_LOG_DIR
239 if [ ! -d "$CELERYD_PID_DIR" ]; then
240 mkdir -p $CELERYD_PID_DIR
243 # Extra start-stop-daemon options, like user/group.
244 if [ -n "$CELERYD_USER" ]; then
245 DAEMON_OPTS="$DAEMON_OPTS --uid=$CELERYD_USER"
246 chown "$CELERYD_USER" $CELERYD_LOG_DIR $CELERYD_PID_DIR
248 if [ -n "$CELERYD_GROUP" ]; then
249 DAEMON_OPTS="$DAEMON_OPTS --gid=$CELERYD_GROUP"
250 chgrp "$CELERYD_GROUP" $CELERYD_LOG_DIR $CELERYD_PID_DIR
253 if [ -n "$CELERYD_CHDIR" ]; then
254 DAEMON_OPTS="$DAEMON_OPTS --workdir=\"$CELERYD_CHDIR\""
259 if [ ! -c /dev/null ]; then
260 echo "/dev/null is not a character device!"
266 export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
270 $CELERYD_MULTI stop $CELERYD_NODES --pidfile="$CELERYD_PID_FILE"
275 $CELERYD_MULTI start $CELERYD_NODES $DAEMON_OPTS \
276 --pidfile="$CELERYD_PID_FILE" \
277 --logfile="$CELERYD_LOG_FILE" \
278 --loglevel="$CELERYD_LOG_LEVEL" \
285 $CELERYD_MULTI restart $CELERYD_NODES $DAEMON_OPTS \
286 --pidfile="$CELERYD_PID_FILE" \
287 --logfile="$CELERYD_LOG_FILE" \
288 --loglevel="$CELERYD_LOG_LEVEL" \
311 $CELERYCTL status $CELERYCTL_OPTS
325 echo "Usage: /etc/init.d/celeryd {start|stop|restart|try-restart|kill}"
332 and make it executable::
334 chmod +x /etc/init.d/celeryd
336 celeryd requires a /etc/default/celeryd file to be in place.
337 Thus we are going to create this file (/etc/default/celeryd)::
339 vim /etc/default/celeryd
341 Again if the directory conventions have been followed the file should be::
343 # Name of nodes to start, here we have a single node
345 # or we could have three nodes:
346 #CELERYD_NODES="w1 w2 w3"
348 # Where to chdir at start.
349 CELERYD_CHDIR="/srv/flowspy/"
350 # How to call "manage.py celeryd_multi"
351 CELERYD_MULTI="$CELERYD_CHDIR/manage.py celeryd_multi"
353 # How to call "manage.py celeryctl"
354 CELERYCTL="$CELERYD_CHDIR/manage.py celeryctl"
356 # Extra arguments to celeryd
357 #CELERYD_OPTS="--time-limit=300 --concurrency=8"
359 # Name of the celery config module.
360 CELERY_CONFIG_MODULE="celeryconfig"
362 # %n will be replaced with the nodename.
363 CELERYD_LOG_FILE="$CELERYD_CHDIR/celery_var/log/celery/%n.log"
364 CELERYD_PID_FILE="$CELERYD_CHDIR/celery_var/run/celery/%n.pid"
366 # Workers should run as an unprivileged user.
370 # Name of the projects settings module.
371 export DJANGO_SETTINGS_MODULE="settings"
375 Apache proxies gunicorn. Things are more flexible here as you may follow your own configuration and conventions. Create and edit /etc/apache2/sites-available/fod. You should set <server_name> and <admin_mail> along with your certificates. If under testing environment, you can use the provided snakeoil certs. If you do not intent to use Shibboleth delete or comment the corresponding configuration parts inside **Shibboleth configuration** ::
377 vim /etc/apache2/sites-available/fod
379 Again if the directory conventions have been followed the file should be::
382 ServerAdmin webmaster@localhost
383 ServerName <server_name>
384 DocumentRoot /var/www
386 Options FollowSymLinks
389 <Directory /var/www/>
390 Options Indexes FollowSymLinks MultiViews
396 ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
397 <Directory "/usr/lib/cgi-bin">
399 Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
404 ErrorLog ${APACHE_LOG_DIR}/error.log
406 # Possible values include: debug, info, notice, warn, error, crit,
410 CustomLog ${APACHE_LOG_DIR}/access.log combined
412 Alias /doc/ "/usr/share/doc/"
413 <Directory "/usr/share/doc/">
414 Options Indexes MultiViews FollowSymLinks
418 Allow from 127.0.0.0/255.0.0.0 ::1/128
422 RewriteCond %{HTTPS} off
423 RewriteRule ^/(.*) https://<server_name>/$1 [L,R]
427 ServerName <server_name>
428 ServerAdmin <admin_mail>
432 SSLCertificateFile /etc/ssl/certs/example.com.crt
433 SSLCertificateChainFile /etc/ssl/certs/example.com.crt
434 SSLCertificateKeyFile /etc/ssl/private/example.com.key
436 AddDefaultCharset UTF-8
437 IndexOptions +Charset=UTF-8
439 # Shibboleth configuration
440 ShibConfig /etc/shibboleth/shibboleth2.xml
441 Alias /shibboleth-sp /usr/share/shibboleth
443 <Location /fod/login>
445 ShibRequireSession On
450 # Shibboleth debugging CGI script
451 ScriptAlias /shibboleth/test /usr/lib/cgi-bin/shibtest.cgi
452 <Location /shibboleth/test>
454 ShibRequireSession On
459 <Location /Shibboleth.sso>
463 # End of Shibboleth configuration
465 <Location /admin/media/>
469 Alias /admin/media /usr/share/pyshared/django/contrib/admin/media
470 Alias /media /usr/share/pyshared/django/contrib/admin/media
471 DocumentRoot /var/www
472 <Directory /var/www/>
473 Options Indexes FollowSymLinks MultiViews
487 ProxyErrorOverride off
489 ProxyPass /fod http://localhost:8081/fod retry=0
490 ProxyPassReverse /fod http://localhost:8081/fod
493 ErrorLog /var/log/apache2/ssl-error.log
494 CustomLog /var/log/apache2/ssl-access.log combined
499 Alias /fodstatic /srv/flowspy/static
503 Now, enable your site. You might want to disable the default site if fod is the only site you host on your server::
508 You are not far away from deploying FoD. When asked for a super user, create one::
511 python manage.py syncdb
512 python manage.py migrate
514 Restart, gunicorn and apache::
516 service gunicorn restart && service apache2 restart
520 Log in to the admin interface via https://<your ip>/fod/admin. Go to Peer ranges and add a new range (part of/or a complete subnet), eg. 83.212.0.0/19
521 Go to Peers and add a new peer, eg. id: 1, name: Test, AS: 16503, tag: TEST and move the network you have crteated from Avalable to Chosen. From the admin front, go to User, and edit your user. From the bottom of the page, select the TEST peer and save.
522 Last but not least, modify as required the existing (example.com) Site instance. You are done. As you are logged-n via the admin, there is no need for Shibboleth. Go to https://<your ip>/fod/ and create a new rule. Your rule should be applied on the flowspec capable device after aprox. 10 seconds.
526 Via the admin interface you can modify flatpages to suit your needs
530 Inside the static folder you will find two empty png files: fod_logo.xcf (Gimp file) and shib_login.dist.png. Edit those two with your favourite image processing software and save them as fod_logo.png (under static/img/) and shib_login.png (under static/). Image sizes are optimized to operate without any other code changes. In case you want to incorporate images of different sizes you have to fine tune css and/or html as well.
534 Under the templates folder (templates), you can alter the footer.html file to include your own footer messages, badges, etc.
538 Under the templates folder (templates), you can alter the welcome page - welcome.html with your own images, carousel, videos, etc.