7 Firewall on Demand applies, via Netconf, flow rules to a network device.
8 These rules are then propagated via e-bgp to peering routers. Each user
9 is authenticated against shibboleth. Authorization is performed via a
10 combination of a Shibboleth attribute and the peer network address range
11 that the user originates from. FoD is meant to operate over this
14 +-----------+ +------------+ +------------+
15 | FoD | NETCONF | flowspec | ebgp | router |
16 | web app +----------> device +--------> |
17 +-----------+ +------+-----+ +------------+
25 NETCONF is chosen as the mgmt protocol to apply rules to a single
26 flowspec capable device. Rules are then propagated via igbp to all
27 flowspec capable routers. Of course FoD could apply rules directly (via
28 NETCONF always) to a router and then ibgp would do the rest. In GRNET’s
29 case the flowspec capable device is an EX4200.
33 > Make sure your FoD server has ssh access to your flowspec device.
37 > Installation instructions assume a clean Debian Wheezy with Django 1.4
42 You can find more about FoD or raise your issues at [GRNET FoD
45 You can contact us directly at staurosk{at}noc[dot]grnet(.)gr
50 [GRNET FoD repository]: https://code.grnet.gr/projects/flowspy
51 [Github FoD repository]: https://github.com/grnet/flowspy
56 Debian Wheezy (x64) - Django 1.4.x
57 ----------------------------------
59 This guide assumes that installation is carried out in /srv/flowspy
60 directory. If other directory is to be used, please change the
61 corresponding configuration files. It is also assumed that the root user
62 will perform every action.
64 ### Upgrading from v\<1.1.x
68 > If PEER\_\*\_TABLE tables are set to FALSE in settings.py, you need to
69 > perform the south migrations per application:
71 > ./manage.py migrate longerusername
72 > ./manage.py migrate flowspec
73 > ./manage.py migrate accounts
75 If upgrading from flowspy version \<1.1.x pay attention to settings.py
76 changes. Also, do not forget to run if PEER\_\*\_TABLE tables are set to
81 to catch-up with latest database changes.
83 ### Upgrading from v\<1.0.x
85 If upgrading from flowspy version \<1.0.x pay attention to settings.py
86 changes. Also, do not forget to run:
90 to catch-up with latest database changes.
92 ### Required system packages
94 Update and install the required packages:
98 apt-get install mysql-server apache2 memcached libapache2-mod-proxy-html gunicorn beanstalkd python-django python-django-south python-django-tinymce tinymce python-mysqldb python-yaml python-memcache python-django-registration python-ipaddr python-lxml mysql-client git python-django-celery python-paramiko python-gevent vim
100 Also, django rest framework package is required. In debian Wheezy it is
101 not available, but one can install it via pip.
105 > Set username and password for mysql if used
109 > If you wish to deploy an outgoing mail server, now it is time to do
110 > it. Otherwise you could set FoD to send out mails via a third party
113 ### Create a database
115 If you are using mysql, you should create a database:
117 mysql -u root -p -e 'create database fod'
119 ### Required application packages
121 Get the required packages and their dependencies and install them:
123 apt-get install libxml2-dev libxslt-dev gcc python-dev
125 - ncclient: NETCONF python client:
128 git clone https://github.com/leopoul/ncclient.git
130 python setup.py install
132 - nxpy: Python Objects from/to XML proxy:
135 git clone https://code.grnet.gr/git/nxpy
137 python setup.py install
139 - flowspy: core application. Installation is done at /srv/flowspy:
142 git clone https://code.grnet.gr/git/flowspy
145 Application configuration
146 =========================
148 Copy settings.py.dist to settings.py:
151 cp settings.py.dist settings.py
153 Edit settings.py file and set the following according to your
156 ADMINS: set your admin name and email (assuming that your server can send notifications)
157 DATABASES (to point to your local database). You could use views instead of tables for models: peer, peercontacts, peernetworks. For this to work we suggest MySQL with MyISAM db engine
158 SECRET_KEY : Make this unique, and don't share it with anybody
159 STATIC_ROOT: /srv/flowspy/static (or your installation directory)
160 STATIC_URL (static media directory) . If you have followed the above this should be: /srv/flowspy/static
161 TEMPLATE_DIRS : If you have followed the above this should be: /srv/flowspy/templates
162 CACHE_BACKEND: Enable Memcached for production or leave to DummyCache for development environments
163 Alternatively you could go for redis with the corresponding Django client lib.
164 NETCONF_DEVICE (tested with Juniper EX4200 but any BGP enabled Juniper should work). This is the flowspec capable device
165 NETCONF_USER (enable ssh and netconf on device)
167 If beanstalk is selected the following should be left intact.
168 BROKER_HOST (beanstalk host)
169 BROKER_PORT (beanstalk port)
172 If beanstalk is selected the following should be left intact.
173 BROKER_URL (beanstalk url)
174 SHIB_AUTH_ENTITLEMENT (if you go for Shibboleth authentication)
175 NOTIFY_ADMIN_MAILS (bcc mail addresses)
176 PROTECTED_SUBNETS (subnets for which source or destination address will prevent rule creation and notify the NOTIFY_ADMIN_MAILS)
177 The whois client is meant to be used in case you have inserted peers with their ASes in the peers table and wish to get network info for each one in an automated manner.
180 If you wish to deploy FoD with Shibboleth change the following attributes according to your setup:
181 SHIB_AUTH_ENTITLEMENT = 'urn:mace'
182 SHIB_ADMIN_DOMAIN = 'example.com'
183 SHIB_LOGOUT_URL = 'https://example.com/Shibboleth.sso/Logout'
184 SHIB_USERNAME = ['HTTP_EPPN']
185 SHIB_MAIL = ['mail', 'HTTP_MAIL', 'HTTP_SHIB_INETORGPERSON_MAIL']
186 SHIB_FIRSTNAME = ['HTTP_SHIB_INETORGPERSON_GIVENNAME']
187 SHIB_LASTNAME = ['HTTP_SHIB_PERSON_SURNAME']
188 SHIB_ENTITLEMENT = ['HTTP_SHIB_EP_ENTITLEMENT']
190 If you have not installed an outgoing mail server you can always use
191 your own account (either corporate or gmail, hotmail ,etc) by adding the
192 following lines in settings.py:
194 EMAIL_USE_TLS = True #(or False)
195 EMAIL_HOST = 'smtp.example.com'
196 EMAIL_HOST_USER = 'username'
197 EMAIL_HOST_PASSWORD = 'yourpassword'
198 EMAIL_PORT = 587 #(outgoing)
200 It is strongly advised that you do not change the following to False
201 values unless, you want to integrate FoD with you CRM or members
202 database. This implies that you are able/have the rights to create
203 database views between the two databases:
205 PEER_MANAGED_TABLE = True
206 PEER_RANGE_MANAGED_TABLE = True
207 PEER_TECHC_MANAGED_TABLE = True
209 By doing that the corresponding tables as defined in peers/models will
210 not be created. As noted above, you have to create the views that the
215 > Soon we will release a version with django-registration as a means to
216 > add users and Shibboleth will become an alternative
218 Let’s move on with some copies and dir creations:
221 chown www-data.www-data /var/log/fod
222 cp urls.py.dist urls.py
227 > LOG\_FILE\_LOCATION in settings.py is set to **/var/log/fod**. Adjust
228 > the chown command above to your selected dir.
233 Apache operates as a gunicorn Proxy with WSGI and Shibboleth modules
234 enabled. Depending on the setup the apache configuration may vary:
241 If shibboleth is to be used:
243 apt-get install libapache2-mod-shib2
246 Now it is time to configure beanstalk, gunicorn, celery and apache.
251 Enable beanstalk by editting /etc/default/beanstalkd:
253 vim /etc/default/beanstalkd
255 Uncomment the line **START=yes** to enable beanstalk
259 service beanstalkd start
264 Create and edit /etc/gunicorn.d/fod:
266 vim /etc/gunicorn.d/fod
268 FoD is served via gunicorn and is then proxied by Apache. If the above
269 directory conventions have been followed so far, then your configuration
274 'working_dir': '/srv/flowspy',
276 '--bind=127.0.0.1:8081',
278 '--worker-class=egg:gunicorn#gevent',
282 '--log-file=/var/log/gunicorn/fod.log',
289 Celery is used over beanstalkd to apply firewall rules in a serial
290 manner so that locks are avoided on the flowspec capable device. In our
291 setup celery runs via django. That is why the python-django-celery
292 package was installed.
294 Create the celeryd daemon at /etc/init.d/celeryd **if it does not
297 vim /etc/init.d/celeryd
299 The configuration should be:
302 # ============================================
303 # celeryd - Starts the Celery worker daemon.
304 # ============================================
306 # :Usage: /etc/init.d/celeryd {start|stop|force-reload|restart|try-restart|status}
307 # :Configuration file: /etc/default/celeryd
309 # See http://docs.celeryq.org/en/latest/cookbook/daemonizing.html#init-script-celeryd
314 # Required-Start: $network $local_fs $remote_fs
315 # Required-Stop: $network $local_fs $remote_fs
316 # Default-Start: 2 3 4 5
317 # Default-Stop: 0 1 6
318 # Short-Description: celery task worker daemon
319 # Description: Starts the Celery worker daemon for a single project.
324 DEFAULT_PID_FILE="/var/run/celery/%n.pid"
325 DEFAULT_LOG_FILE="/var/log/celery/%n.log"
326 DEFAULT_LOG_LEVEL="INFO"
327 DEFAULT_NODES="celery"
328 DEFAULT_CELERYD="-m celery.bin.celeryd_detach"
331 [ -r "$CELERY_DEFAULTS" ] && . "$CELERY_DEFAULTS"
333 [ -r /etc/default/celeryd ] && . /etc/default/celeryd
335 if [ "$ENABLED" != "true" ]; then
336 echo "celery daemon disabled - see /etc/default/celeryd."
341 CELERYD_PID_FILE=${CELERYD_PID_FILE:-${CELERYD_PIDFILE:-$DEFAULT_PID_FILE}}
342 CELERYD_LOG_FILE=${CELERYD_LOG_FILE:-${CELERYD_LOGFILE:-$DEFAULT_LOG_FILE}}
343 CELERYD_LOG_LEVEL=${CELERYD_LOG_LEVEL:-${CELERYD_LOGLEVEL:-$DEFAULT_LOG_LEVEL}}
344 CELERYD_MULTI=${CELERYD_MULTI:-"celeryd-multi"}
345 CELERYD=${CELERYD:-$DEFAULT_CELERYD}
346 CELERYCTL=${CELERYCTL:="celeryctl"}
347 CELERYD_NODES=${CELERYD_NODES:-$DEFAULT_NODES}
352 CELERYD_OPTS="$CELERYD_OPTS $2"
355 CELERYD_LOG_DIR=`dirname $CELERYD_LOG_FILE`
356 CELERYD_PID_DIR=`dirname $CELERYD_PID_FILE`
357 if [ ! -d "$CELERYD_LOG_DIR" ]; then
358 mkdir -p $CELERYD_LOG_DIR
360 if [ ! -d "$CELERYD_PID_DIR" ]; then
361 mkdir -p $CELERYD_PID_DIR
364 # Extra start-stop-daemon options, like user/group.
365 if [ -n "$CELERYD_USER" ]; then
366 DAEMON_OPTS="$DAEMON_OPTS --uid=$CELERYD_USER"
367 chown "$CELERYD_USER" $CELERYD_LOG_DIR $CELERYD_PID_DIR
369 if [ -n "$CELERYD_GROUP" ]; then
370 DAEMON_OPTS="$DAEMON_OPTS --gid=$CELERYD_GROUP"
371 chgrp "$CELERYD_GROUP" $CELERYD_LOG_DIR $CELERYD_PID_DIR
374 if [ -n "$CELERYD_CHDIR" ]; then
375 DAEMON_OPTS="$DAEMON_OPTS --workdir=\"$CELERYD_CHDIR\""
380 if [ ! -c /dev/null ]; then
381 echo "/dev/null is not a character device!"
387 export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
391 $CELERYD_MULTI stop $CELERYD_NODES --pidfile="$CELERYD_PID_FILE"
396 $CELERYD_MULTI start $CELERYD_NODES $DAEMON_OPTS \
397 --pidfile="$CELERYD_PID_FILE" \
398 --logfile="$CELERYD_LOG_FILE" \
399 --loglevel="$CELERYD_LOG_LEVEL" \
406 $CELERYD_MULTI restart $CELERYD_NODES $DAEMON_OPTS \
407 --pidfile="$CELERYD_PID_FILE" \
408 --logfile="$CELERYD_LOG_FILE" \
409 --loglevel="$CELERYD_LOG_LEVEL" \
432 $CELERYCTL status $CELERYCTL_OPTS
446 echo "Usage: /etc/init.d/celeryd {start|stop|restart|try-restart|kill}"
453 celeryd configuration
454 =====================
456 celeryd requires a /etc/default/celeryd file to be in place. Thus we are
457 going to create this file (/etc/default/celeryd):
459 vim /etc/default/celeryd
461 Again if the directory conventions have been followed the file is (pay
462 attention to the CELERYD\_USER, CELERYD\_GROUP and change accordingly) :
467 # Name of nodes to start, here we have a single node
469 # or we could have three nodes:
470 #CELERYD_NODES="w1 w2 w3"
472 # Where to chdir at start.
473 CELERYD_CHDIR="/srv/flowspy"
474 # How to call "manage.py celeryd_multi"
475 CELERYD_MULTI="python $CELERYD_CHDIR/manage.py celeryd_multi"
477 # How to call "manage.py celeryctl"
478 CELERYCTL="python $CELERYD_CHDIR/manage.py celeryctl"
480 # Extra arguments to celeryd
481 #CELERYD_OPTS="--time-limit=300 --concurrency=8"
482 CELERYD_OPTS="-E -B --schedule=/var/run/celery/celerybeat-schedule --concurrency=1 --soft-time-limit=180 --time-limit=1800"
483 # Name of the celery config module.
484 CELERY_CONFIG_MODULE="celeryconfig"
486 # %n will be replaced with the nodename.
487 CELERYD_LOG_FILE="/var/log/celery/fod_%n.log"
488 CELERYD_PID_FILE="/var/run/celery/%n.pid"
493 # Name of the projects settings module.
494 export DJANGO_SETTINGS_MODULE="flowspy.settings"
499 Apache proxies gunicorn. Things are more flexible here as you may follow
500 your own configuration and conventions. Create and edit
501 /etc/apache2/sites-available/fod. You should set \<server\_name\> and
502 \<admin\_mail\> along with your certificates. If under testing
503 environment, you can use the provided snakeoil certs. If you do not
504 intent to use Shibboleth delete or comment the corresponding
505 configuration parts inside **Shibboleth configuration** :
507 vim /etc/apache2/sites-available/fod
509 Again if the directory conventions have been followed the file should
513 ServerAdmin webmaster@localhost
514 ServerName fod.example.com
515 DocumentRoot /var/www
517 ErrorLog ${APACHE_LOG_DIR}/fod_error.log
519 # Possible values include: debug, info, notice, warn, error, crit,
523 CustomLog ${APACHE_LOG_DIR}/fod_access.log combined
525 Alias /static /srv/flowspy/static
527 RewriteCond %{HTTPS} off
528 RewriteRule ^/(.*) https://fod.example.com/$1 [L,R]
532 ServerName fod.example.com
533 ServerAdmin webmaster@localhost
537 SSLCertificateFile /etc/ssl/certs/fod.example.com.crt
538 SSLCertificateChainFile /etc/ssl/certs/example-chain.pem
539 SSLCertificateKeyFile /etc/ssl/private/fod.example.com.key
541 AddDefaultCharset UTF-8
542 IndexOptions +Charset=UTF-8
544 ShibConfig /etc/shibboleth/shibboleth2.xml
545 Alias /shibboleth-sp /usr/share/shibboleth
550 ShibRequireSession On
552 ShibRequestSetting entityID https://idp.example.com/idp/shibboleth
556 # Shibboleth debugging CGI script
557 ScriptAlias /shibboleth/test /usr/lib/cgi-bin/shibtest.cgi
558 <Location /shibboleth/test>
560 ShibRequireSession On
565 <Location /Shibboleth.sso>
569 # Shibboleth SP configuration
571 #SetEnv proxy-sendchunked
579 ProxyErrorOverride off
582 ProxyPass /shibboleth !
583 ProxyPass /Shibboleth.sso !
585 ProxyPass / http://localhost:8081/ retry=0
586 ProxyPassReverse / http://localhost:8081/
588 Alias /static /srv/flowspy/static
592 Now, enable your site. You might want to disable the default site if fod
593 is the only site you host on your server:
598 You are not far away from deploying FoD. When asked for a super user,
602 python manage.py syncdb
603 python manage.py migrate longerusername
604 python manage.py migrate flowspec
605 python manage.py migrate djcelery
606 python manage.py migrate accounts
607 python manage.py migrate
609 If you have not changed the values of the PEER\_\*\_TABLE variables to
610 False and thus you are going for a default installation (that is
611 PEER\_\*\_TABLE variables are set to True) , then run:
613 python manage.py migrate peers
615 If however you have set the PEER\_\*\_TABLE variables to False and by
616 accident you have ran the command above, then you have to cleanup you
617 database manually by dropping the peer\* tables plus the techc\_email
618 table. For MySQL the command is:
620 DROP TABLE `peer`, `peer_networks`, `peer_range`, `peer_techc_emails`, techc_email;
622 Restart, gunicorn and apache:
624 service gunicorn restart && service apache2 restart
626 Propagate the flatpages
627 =======================
629 Inside the initial\_data/fixtures\_manual.xml file we have placed 4
630 flatpages (2 for Greek, 2 for English) with Information and Terms of
631 Service about the service. To import the flatpages, run from root
634 python manage.py loaddata initial_data/fixtures_manual.xml
639 Log in to the admin interface via [https:\\/\\/][]\<hostname\>/admin. Go
640 to Peer ranges and add a new range (part of/or a complete subnet), eg.
641 10.20.0.0/19 Go to Peers and add a new peer, eg. id: 1, name: Test, AS:
642 16503, tag: TEST and move the network you have created from Avalable to
643 Chosen. From the admin front, go to User, and edit your user. From the
644 bottom of the page, select the TEST peer and save. Last but not least,
645 modify as required the existing (example.com) Site instance (admin
646 home-\>Sites). You are done. As you are logged-in via the admin, there
647 is no need to go through Shibboleth at this time. Go to
648 <hostname\> and create a new rule. Your rule should be
649 applied on the flowspec capable device after aprox. 10 seconds. If no
650 Shibboleth authentication is available, a
651 <hostname\> altlogin is provided.
656 Via the admin interface you can modify flatpages to suit your needs
661 Under the templates folder (templates), you can alter the footer.html
662 file to include your own footer messages, badges, etc.
667 Under the templates folder (templates), you can alter the welcome page -
668 welcome.html with your own images, carousel, videos, etc.
674 -------------------------
675 FoD comes with a web interface, in which one can edit and apply new routes.
680 There is a rest api available in /api/v1/. One can set new rules or see the applied ones by using it.