Revision b7566dcc doc/source/install.rst

b/doc/source/install.rst
5 5
.. toctree::
6 6
   :maxdepth: 2
7 7

  
8
Ubuntu 12.04.3 (64) - Django 1.3.x
8
Debian Wheezy (x64) - Django 1.4.x
9 9
==================================
10 10

  
11 11
This guide assumes that installation is carried out in /srv/flowspy directory. If other directory is to be used, please change the corresponding configuration files. It is also assumed that the root user will perform every action.
......
17 17

  
18 18
	apt-get update
19 19
	apt-get upgrade
20
	apt-get install mysql-server apache2 memcached libapache2-mod-proxy-html gunicorn beanstalkd python-django python-django-extensions python-django-south python-django-tinymce tinymce python-mysqldb python-yaml python-memcache python-django-registration python-ipaddr python-lxml mysql-client git python-django-celery python-paramiko python-gevent vim
20
	apt-get install mysql-server apache2 memcached libapache2-mod-proxy-html gunicorn beanstalkd python-django python-django-south python-django-tinymce tinymce python-mysqldb python-yaml python-memcache python-django-registration python-ipaddr python-lxml mysql-client git python-django-celery python-paramiko python-gevent vim
21 21

  
22 22
.. note::
23 23
 Set username and password for mysql if used
......
60 60
=========================
61 61

  
62 62
Copy settings.py.dist to settings.py::
63

  
63
   
64
   cd flowspy
64 65
	cp settings.py.dist settings.py
65 66

  
66 67
Edit settings.py file and set the following according to your configuration::
......
68 69
	ADMINS: set your admin name and email (assuming that your server can send notifications)
69 70
	DATABASES (to point to your local database). You could use views instead of tables for models: peer, peercontacts, peernetworks. For this to work we suggest MySQL with MyISAM db engine
70 71
	SECRET_KEY : Make this unique, and don't share it with anybody
72
   STATIC_ROOT: /srv/flowspy/static (or your installation directory)
71 73
	STATIC_URL (static media directory) . If you have followed the above this should be: /srv/flowspy/static
72 74
	TEMPLATE_DIRS : If you have followed the above this should be: /srv/flowspy/templates
73
	CACHE_BACKEND:  If you have followed the above this should be: memcached://127.0.0.1:11211/?timeout=3600
75
	CACHE_BACKEND:  Enable Memcached for production or leave to DummyCache for development environments
74 76
	Alternatively you could go for redis with the corresponding Django client lib.
75 77
	NETCONF_DEVICE (tested with Juniper EX4200 but any BGP enabled Juniper should work). This is the flowspec capable device
76 78
	NETCONF_USER (enable ssh and netconf on device)
......
108 110

  
109 111

  
110 112
.. note::
111
	Soon we will release a version with django-registration as a means to add users and Shibboleth as an alternative
113
	Soon we will release a version with django-registration as a means to add users and Shibboleth will become an alternative
112 114

  
113 115
Let's move on with some copies and dir creations::
114 116

  
115 117
	cp urls.py.dist urls.py
118
   cd ..
116 119
	mkdir log
117 120
	chown -R root:www-data log/
118 121
	chmod -R g+w log
......
156 159

  
157 160
FoD is served via gunicorn and is then proxied by Apache. If the above directory conventions have been followed so far, then your configuration should be::
158 161

  
159
	CONFIG = {
160
	    'mode': 'django',
161
	    'working_dir': '/srv/flowspy',
162
	    'args': (
163
	        '--bind=127.0.0.1:8081',
164
	        '--workers=1',
165
	        '--timeout=360',
166
	        '--worker-class=egg:gunicorn#gevent',
167
	        '--log-level=debug',
168
	        'settings.py',
169
	    ),
170
	}
162
   CONFIG = {
163
       'mode': 'django',
164
       'working_dir': '/srv/flowspy',
165
       'args': (
166
           '--bind=127.0.0.1:8081',
167
           '--workers=1',
168
           '--worker-class=egg:gunicorn#gevent',
169
           '--timeout=360',
170
           '--log-level=debug',
171
           '--log-file=/tmp/fod.log',
172
       ),
173
   }
174

  
171 175

  
172 176
celery.d
173 177
--------
174 178

  
175 179
Celery is used over beanstalkd to apply firewall rules in a serial manner so that locks are avoided on the flowspec capable device. In our setup celery runs via django. That is why the python-django-celery package was installed.
176 180

  
177
Create the celeryd daemon at /etc/init.d/celeryd::
178

  
179
	vim /etc/init.d/celeryd
180

  
181
The configuration should be::
182

  
183
	#!/bin/sh -e
184
	# ============================================
185
	#  celeryd - Starts the Celery worker daemon.
186
	# ============================================
187
	#
188
	# :Usage: /etc/init.d/celeryd {start|stop|force-reload|restart|try-restart|status}
189
	# :Configuration file: /etc/default/celeryd
190
	#
191
	# See http://docs.celeryq.org/en/latest/cookbook/daemonizing.html#init-script-celeryd
192

  
193

  
194
	### BEGIN INIT INFO
195
	# Provides:          celeryd
196
	# Required-Start:    $network $local_fs $remote_fs
197
	# Required-Stop:     $network $local_fs $remote_fs
198
	# Default-Start:     2 3 4 5
199
	# Default-Stop:      0 1 6
200
	# Short-Description: celery task worker daemon
201
	### END INIT INFO
202

  
203
	#set -e
204

  
205
	DEFAULT_PID_FILE="/var/run/celeryd@%n.pid"
206
	DEFAULT_LOG_FILE="/var/log/celeryd@%n.log"
207
	DEFAULT_LOG_LEVEL="INFO"
208
	DEFAULT_NODES="celery"
209
	DEFAULT_CELERYD="-m celery.bin.celeryd_detach"
210

  
211
	# /etc/init.d/celeryd: start and stop the celery task worker daemon.
212

  
213
	CELERY_DEFAULTS=${CELERY_DEFAULTS:-"/etc/default/celeryd"}
214

  
215
	test -f "$CELERY_DEFAULTS" && . "$CELERY_DEFAULTS"
216
	if [ -f "/etc/default/celeryd" ]; then
217
	    . /etc/default/celeryd
218
	fi
219

  
220
	CELERYD_PID_FILE=${CELERYD_PID_FILE:-${CELERYD_PIDFILE:-$DEFAULT_PID_FILE}}
221
	CELERYD_LOG_FILE=${CELERYD_LOG_FILE:-${CELERYD_LOGFILE:-$DEFAULT_LOG_FILE}}
222
	CELERYD_LOG_LEVEL=${CELERYD_LOG_LEVEL:-${CELERYD_LOGLEVEL:-$DEFAULT_LOG_LEVEL}}
223
	CELERYD_MULTI=${CELERYD_MULTI:-"celeryd-multi"}
224
	CELERYD=${CELERYD:-$DEFAULT_CELERYD}
225
	CELERYCTL=${CELERYCTL:="celeryctl"}
226
	CELERYD_NODES=${CELERYD_NODES:-$DEFAULT_NODES}
227

  
228
	export CELERY_LOADER
229

  
230
	if [ -n "$2" ]; then
231
	    CELERYD_OPTS="$CELERYD_OPTS $2"
232
	fi
233

  
234
	CELERYD_LOG_DIR=`dirname $CELERYD_LOG_FILE`
235
	CELERYD_PID_DIR=`dirname $CELERYD_PID_FILE`
236
	if [ ! -d "$CELERYD_LOG_DIR" ]; then
237
	    mkdir -p $CELERYD_LOG_DIR
238
	fi
239
	if [ ! -d "$CELERYD_PID_DIR" ]; then
240
	    mkdir -p $CELERYD_PID_DIR
241
	fi
242

  
243
	# Extra start-stop-daemon options, like user/group.
244
	if [ -n "$CELERYD_USER" ]; then
245
	    DAEMON_OPTS="$DAEMON_OPTS --uid=$CELERYD_USER"
246
	    chown "$CELERYD_USER" $CELERYD_LOG_DIR $CELERYD_PID_DIR
247
	fi
248
	if [ -n "$CELERYD_GROUP" ]; then
249
	    DAEMON_OPTS="$DAEMON_OPTS --gid=$CELERYD_GROUP"
250
	    chgrp "$CELERYD_GROUP" $CELERYD_LOG_DIR $CELERYD_PID_DIR
251
	fi
252

  
253
	if [ -n "$CELERYD_CHDIR" ]; then
254
	    DAEMON_OPTS="$DAEMON_OPTS --workdir=\"$CELERYD_CHDIR\""
255
	fi
256

  
257

  
258
	check_dev_null() {
259
	    if [ ! -c /dev/null ]; then
260
		echo "/dev/null is not a character device!"
261
		exit 1
262
	    fi
263
	}
264

  
265

  
266
	export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
267

  
268

  
269
	stop_workers () {
270
	    $CELERYD_MULTI stop $CELERYD_NODES --pidfile="$CELERYD_PID_FILE"
271
	}
272

  
273

  
274
	start_workers () {
275
	    $CELERYD_MULTI start $CELERYD_NODES $DAEMON_OPTS        \
276
		                 --pidfile="$CELERYD_PID_FILE"      \
277
		                 --logfile="$CELERYD_LOG_FILE"      \
278
		                 --loglevel="$CELERYD_LOG_LEVEL"    \
279
		                 --cmd="$CELERYD"                   \
280
		                 $CELERYD_OPTS
281
	}
282

  
283

  
284
	restart_workers () {
285
	    $CELERYD_MULTI restart $CELERYD_NODES $DAEMON_OPTS      \
286
		                   --pidfile="$CELERYD_PID_FILE"    \
287
		                   --logfile="$CELERYD_LOG_FILE"    \
288
		                   --loglevel="$CELERYD_LOG_LEVEL"  \
289
		                   --cmd="$CELERYD"                 \
290
		                   $CELERYD_OPTS
291
	}
292

  
293

  
294

  
295
	case "$1" in
296
	    start)
297
		check_dev_null
298
		start_workers
299
	    ;;
300

  
301
	    stop)
302
		check_dev_null
303
		stop_workers
304
	    ;;
305

  
306
	    reload|force-reload)
307
		echo "Use restart"
308
	    ;;
309

  
310
	    status)
311
		$CELERYCTL status $CELERYCTL_OPTS
312
	    ;;
313

  
314
	    restart)
315
		check_dev_null
316
		restart_workers
317
	    ;;
318

  
319
	    try-restart)
320
		check_dev_null
321
		restart_workers
322
	    ;;
323

  
324
	    *)
325
		echo "Usage: /etc/init.d/celeryd {start|stop|restart|try-restart|kill}"
326
		exit 1
327
	    ;;
328
	esac
329

  
330
	exit 0
331

  
332
and make it executable::
333

  
334
	chmod +x /etc/init.d/celeryd
335

  
336 181
celeryd requires a /etc/default/celeryd file to be in place.
337 182
Thus we are going to create this file (/etc/default/celeryd)::
338 183

  
......
340 185

  
341 186
Again if the directory conventions have been followed the file is (pay attention to the CELERYD_USER, CELERYD_GROUP and change accordingly)  ::
342 187

  
343
	# Name of nodes to start, here we have a single node
344
	CELERYD_NODES="w1"
345
	# or we could have three nodes:
346
	#CELERYD_NODES="w1 w2 w3"
347

  
348
	# Where to chdir at start.
349
	CELERYD_CHDIR="/srv/flowspy"
350
	# How to call "manage.py celeryd_multi"
351
	CELERYD_MULTI="$CELERYD_CHDIR/manage.py celeryd_multi"
188
   # Default: false
189
   ENABLED="true"
190
   
191
   # Name of nodes to start, here we have a single node
192
   CELERYD_NODES="w1"
193
   # or we could have three nodes:
194
   #CELERYD_NODES="w1 w2 w3"
195
   
196
   # Where to chdir at start.
197
   CELERYD_CHDIR="/srv/flowspy"
198
   # How to call "manage.py celeryd_multi"
199
   CELERYD_MULTI="python $CELERYD_CHDIR/manage.py celeryd_multi"
200
   
201
   # How to call "manage.py celeryctl"
202
   CELERYCTL="python $CELERYD_CHDIR/manage.py celeryctl"
203
   
204
   # Extra arguments to celeryd
205
   #CELERYD_OPTS="--time-limit=300 --concurrency=8"
206
   CELERYD_OPTS="-E -B"
207
   # Name of the celery config module.
208
   CELERY_CONFIG_MODULE="celeryconfig"
209
   
210
   # %n will be replaced with the nodename.
211
   CELERYD_LOG_FILE="$CELERYD_CHDIR/celery_var/log/celery/%n.log"
212
   CELERYD_PID_FILE="$CELERYD_CHDIR/celery_var/run/celery/%n.pid"
213
   
214
   # Workers should run as an unprivileged user.
215
   CELERYD_USER="user"
216
   CELERYD_GROUP="users"
217
   
218
   # Name of the projects settings module.
219
   export DJANGO_SETTINGS_MODULE="flowspy.settings"
352 220

  
353
	# How to call "manage.py celeryctl"
354
	CELERYCTL="$CELERYD_CHDIR/manage.py celeryctl"
355

  
356
	# Extra arguments to celeryd
357
	#CELERYD_OPTS="--time-limit=300 --concurrency=8"
358
	CELERYD_OPTS="-E -B"
359
	# Name of the celery config module.
360
	CELERY_CONFIG_MODULE="celeryconfig"
361

  
362
	# %n will be replaced with the nodename.
363
	CELERYD_LOG_FILE="$CELERYD_CHDIR/celery_var/log/celery/%n.log"
364
	CELERYD_PID_FILE="$CELERYD_CHDIR/celery_var/run/celery/%n.pid"
365

  
366
	# Workers should run as an unprivileged user.
367
	CELERYD_USER="user"
368
	CELERYD_GROUP="user"
369

  
370
	# Name of the projects settings module.
371
	export DJANGO_SETTINGS_MODULE="settings"
372 221

  
373 222
Apache
374 223
------
......
378 227

  
379 228
Again if the directory conventions have been followed the file should be::
380 229

  
381
	<VirtualHost *:80>
382
		ServerAdmin webmaster@localhost
383
		ServerName	<server_name>
384
		DocumentRoot /var/www
385
		<Directory />
386
			Options FollowSymLinks
387
			AllowOverride None
388
		</Directory>
389
		<Directory /var/www/>
390
			Options Indexes FollowSymLinks MultiViews
391
			AllowOverride None
392
			Order allow,deny
393
			allow from all
394
		</Directory>
395

  
396
		ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
397
		<Directory "/usr/lib/cgi-bin">
398
			AllowOverride None
399
			Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
400
			Order allow,deny
401
			Allow from all
402
		</Directory>
403

  
404
		ErrorLog ${APACHE_LOG_DIR}/error.log
405

  
406
		# Possible values include: debug, info, notice, warn, error, crit,
407
		# alert, emerg.
408
		LogLevel warn
409

  
410
		CustomLog ${APACHE_LOG_DIR}/access.log combined
411

  
412
	    Alias /doc/ "/usr/share/doc/"
413
	    <Directory "/usr/share/doc/">
414
	        Options Indexes MultiViews FollowSymLinks
415
	        AllowOverride None
416
	        Order deny,allow
417
	        Deny from all
418
	        Allow from 127.0.0.0/255.0.0.0 ::1/128
419
	    </Directory>
420

  
421
		RewriteEngine On
422
		RewriteCond %{HTTPS} off
423
		RewriteRule ^/(.*) https://<server_name>/$1 [L,R]
424

  
425
	</VirtualHost>
426
	<VirtualHost *:443>
427
		ServerName	 <server_name>
428
		ServerAdmin		<admin_mail>
429
		ServerSignature		On
430

  
431
		SSLEngine on
432
		SSLCertificateFile	/etc/ssl/certs/example.com.crt
433
		SSLCertificateChainFile	/etc/ssl/certs/example.com.crt
434
		SSLCertificateKeyFile	/etc/ssl/private/example.com.key
435

  
436
		AddDefaultCharset	UTF-8
437
		IndexOptions		+Charset=UTF-8
438

  
439
		# Shibboleth configuration
440
		ShibConfig		/etc/shibboleth/shibboleth2.xml
441
		Alias			/shibboleth-sp	/usr/share/shibboleth
442

  
443
		<Location /fod/login>
444
	        AuthType shibboleth
445
	        ShibRequireSession On
446
	        ShibUseHeaders On
447
	        require valid-user
448
	    </Location>
449

  
450
		# Shibboleth debugging CGI script
451
		ScriptAlias /shibboleth/test /usr/lib/cgi-bin/shibtest.cgi
452
		<Location /shibboleth/test>
453
			AuthType shibboleth
454
			ShibRequireSession On
455
			ShibUseHeaders On
456
			require valid-user
457
		</Location>
458

  
459
		<Location /Shibboleth.sso>
460
			SetHandler shib
461
		</Location>
462

  
463
		# End of Shibboleth configuration
464

  
465
		<Location /admin/media/>
466
			SetHandler      None
467
		</Location>
468

  
469
		Alias /admin/media	/usr/share/pyshared/django/contrib/admin/media
470
		Alias /media  /usr/share/pyshared/django/contrib/admin/media
471
			DocumentRoot /var/www
472
	        <Directory /var/www/>
473
	                Options Indexes FollowSymLinks MultiViews
474
	                AllowOverride None
475
	                Order allow,deny
476
	                allow from all
477
	        </Directory>
478

  
479

  
480

  
481
		<Proxy *>
482
	  	Order allow,deny
483
		Allow from all
484
		</Proxy>
485

  
486
		SSLProxyEngine        off
487
		ProxyErrorOverride    off
488
	    ProxyTimeout    28800
489
		ProxyPass        /fod http://localhost:8081/fod retry=0
490
		ProxyPassReverse /fod http://localhost:8081/fod
491

  
492
		LogLevel warn
493
		ErrorLog /var/log/apache2/ssl-error.log
494
		CustomLog /var/log/apache2/ssl-access.log combined
495

  
496

  
497

  
498

  
499
		Alias /fodstatic	/srv/flowspy/static
500

  
501
	</VirtualHost>
230
   <VirtualHost *:80>
231
      ServerAdmin webmaster@localhost
232
      ServerName  fod.example.com
233
      DocumentRoot /var/www
234
   
235
      ErrorLog ${APACHE_LOG_DIR}/fod_error.log
236
   
237
      # Possible values include: debug, info, notice, warn, error, crit,
238
      # alert, emerg.
239
      LogLevel debug
240
      
241
      CustomLog ${APACHE_LOG_DIR}/fod_access.log combined
242
   
243
      Alias /static     /srv/flowspy/static
244
       RewriteEngine On
245
       RewriteCond %{HTTPS} off
246
       RewriteRule ^/(.*) https://fod.example.com/$1 [L,R]
247
   </VirtualHost>
248
   
249
   <VirtualHost *:443>
250
      ServerName   fod.example.com
251
      ServerAdmin    webmaster@localhost
252
      ServerSignature      On
253
      
254
      SSLEngine on
255
      SSLCertificateFile   /etc/ssl/certs/fod.example.com.crt
256
      SSLCertificateChainFile /etc/ssl/certs/example-chain.pem
257
      SSLCertificateKeyFile   /etc/ssl/private/fod.example.com.key
258
   
259
      AddDefaultCharset UTF-8
260
      IndexOptions      +Charset=UTF-8
261
   
262
      ShibConfig     /etc/shibboleth/shibboleth2.xml
263
      Alias       /shibboleth-sp /usr/share/shibboleth
264
   
265
   
266
      <Location /login>
267
         AuthType shibboleth
268
         ShibRequireSession On
269
         ShibUseHeaders On
270
         ShibRequestSetting entityID https://idp.example.com/idp/shibboleth
271
         require valid-user
272
      </Location>
273
      
274
      # Shibboleth debugging CGI script
275
      ScriptAlias /shibboleth/test /usr/lib/cgi-bin/shibtest.cgi
276
      <Location /shibboleth/test>
277
         AuthType shibboleth
278
         ShibRequireSession On
279
         ShibUseHeaders On
280
         require valid-user
281
      </Location>
282
   
283
      <Location /Shibboleth.sso>
284
         SetHandler shib
285
      </Location>
286
   
287
      # Shibboleth SP configuration
288
   
289
      #SetEnv                proxy-sendchunked
290
      
291
          <Proxy *>
292
           Order allow,deny
293
           Allow from all
294
           </Proxy>
295
   
296
           SSLProxyEngine        off
297
           ProxyErrorOverride    off
298
       ProxyTimeout    28800
299
         ProxyPass      /static !
300
         ProxyPass       /shibboleth !
301
         ProxyPass      /Shibboleth.sso !
302
         
303
           ProxyPass        / http://localhost:8081/ retry=0
304
           ProxyPassReverse / http://localhost:8081/
305
   
306
       Alias /static       /srv/flowspy/static
307
   
308
      LogLevel warn
309
      
310
      ErrorLog ${APACHE_LOG_DIR}/fod_error.log
311
       CustomLog ${APACHE_LOG_DIR}/fod_access.log combined
312
   
313
   </VirtualHost>
502 314

  
503 315
Now, enable your site. You might want to disable the default site if fod is the only site you host on your server::
504 316

  
......
517 329

  
518 330
Testing the platform
519 331
====================
520
Log in to the admin interface via https://<your ip>/fod/admin. Go to Peer ranges and add a new range (part of/or a complete subnet), eg. 83.212.0.0/19
332
Log in to the admin interface via https://<hostname>/admin. Go to Peer ranges and add a new range (part of/or a complete subnet), eg. 10.20.0.0/19
521 333
Go to Peers and add a new peer, eg. id: 1, name: Test, AS: 16503, tag: TEST and move the network you have crteated from Avalable to Chosen. From the admin front, go to User, and edit your user. From the bottom of the page, select the TEST peer and save.
522
Last but not least, modify as required the existing (example.com) Site instance (admin home->Sites). You are done. As you are logged-in via the admin, there is no need for Shibboleth. Go to https://<your ip>/fod/ and create a new rule. Your rule should be applied on the flowspec capable device after aprox. 10 seconds.
334
Last but not least, modify as required the existing (example.com) Site instance (admin home->Sites). You are done. As you are logged-in via the admin, there is no need for Shibboleth. Go to https://<hostname>/ and create a new rule. Your rule should be applied on the flowspec capable device after aprox. 10 seconds.
523 335

  
524 336
Branding
525 337
========
526 338
Via the admin interface you can modify flatpages to suit your needs
527 339

  
528
Logos
529
-----
530
Inside the static folder you will find two empty png files: fod_logo.xcf (Gimp file) and shib_login.dist.png. Edit those two with your favourite image processing software and save them as fod_logo.png (under static/img/) and shib_login.png (under static/). Image sizes are optimized to operate without any other code changes. In case you want to incorporate images of different sizes you have to fine tune css and/or html as well.
531

  
532 340
Footer
533 341
------
534 342
Under the templates folder (templates), you can alter the footer.html file to include your own footer messages, badges, etc.

Also available in: Unified diff