Statistics
| Branch: | Tag: | Revision:

root / doc / source / install.rst @ 0bf16f7f

History | View | Annotate | Download (16.6 kB)

1
************
2
Installation
3
************
4

    
5
.. toctree::
6
   :maxdepth: 2
7

    
8
Debian Wheezy (x64) - Django 1.4.x
9
==================================
10
This guide assumes that installation is carried out in /srv/flowspy directory. If other directory is to be used, please change the corresponding configuration files. It is also assumed that the root user will perform every action.
11

    
12
Upgrading from v<1.0.x
13
----------------------
14
If upgrading from flowspy version <1.0.x pay attention to settings.py changes. Also, do not forget to run::
15
   
16
   ./manage.py migrate
17
   
18
to catch-up with latest database changes.
19

    
20
Required system packages
21
------------------------
22
Update and install the required packages::
23

    
24
	apt-get update
25
	apt-get upgrade
26
	apt-get install mysql-server apache2 memcached libapache2-mod-proxy-html gunicorn beanstalkd python-django python-django-south python-django-tinymce tinymce python-mysqldb python-yaml python-memcache python-django-registration python-ipaddr python-lxml mysql-client git python-django-celery python-paramiko python-gevent vim
27

    
28
.. note::
29
 Set username and password for mysql if used
30

    
31
.. note::
32
	If you wish to deploy an outgoing mail server, now it is time to do it. Otherwise you could set FoD to send out mails via a third party account
33

    
34
Create a database
35
-----------------
36
If you are using mysql, you should create a database::
37

    
38
	mysql -u root -p -e 'create database fod'
39

    
40

    
41
Required application packages
42
-----------------------------
43
Get the required packages and install them
44

    
45
- ncclient: NETCONF python client::
46

    
47
	cd ~
48
	git clone https://github.com/leopoul/ncclient.git
49
	cd ncclient
50
	python setup.py install
51

    
52
- nxpy: Python Objects from/to XML proxy::
53

    
54
	cd ~
55
	git clone https://code.grnet.gr/git/nxpy
56
	cd nxpy
57
	python setup.py install
58

    
59
- flowspy: core application. Installation is done at /srv/flowspy::
60

    
61
	cd /srv
62
	git clone https://code.grnet.gr/git/flowspy
63
	cd flowspy
64

    
65
Application configuration
66
=========================
67
Copy settings.py.dist to settings.py::
68
   
69
   cd flowspy
70
	cp settings.py.dist settings.py
71

    
72
Edit settings.py file and set the following according to your configuration::
73

    
74
	ADMINS: set your admin name and email (assuming that your server can send notifications)
75
	DATABASES (to point to your local database). You could use views instead of tables for models: peer, peercontacts, peernetworks. For this to work we suggest MySQL with MyISAM db engine
76
	SECRET_KEY : Make this unique, and don't share it with anybody
77
   STATIC_ROOT: /srv/flowspy/static (or your installation directory)
78
	STATIC_URL (static media directory) . If you have followed the above this should be: /srv/flowspy/static
79
	TEMPLATE_DIRS : If you have followed the above this should be: /srv/flowspy/templates
80
	CACHE_BACKEND:  Enable Memcached for production or leave to DummyCache for development environments
81
	Alternatively you could go for redis with the corresponding Django client lib.
82
	NETCONF_DEVICE (tested with Juniper EX4200 but any BGP enabled Juniper should work). This is the flowspec capable device
83
	NETCONF_USER (enable ssh and netconf on device)
84
	NETCONF_PASS
85
	If beanstalk is selected the following should be left intact.
86
	BROKER_HOST (beanstalk host)
87
	BROKER_PORT (beanstalk port)
88
	SERVER_EMAIL
89
	EMAIL_SUBJECT_PREFIX
90
	If beanstalk is selected the following should be left intact.
91
	BROKER_URL (beanstalk url)
92
	SHIB_AUTH_ENTITLEMENT (if you go for Shibboleth authentication)
93
	NOTIFY_ADMIN_MAILS (bcc mail addresses)
94
	PROTECTED_SUBNETS (subnets for which source or destination address will prevent rule creation and notify the NOTIFY_ADMIN_MAILS)
95
	The whois client is meant to be used in case you have inserted peers with their ASes in the peers table and wish to get network info for each one in an automated manner.
96
	PRIMARY_WHOIS
97
	ALTERNATE_WHOIS
98
	If you wish to deploy FoD with Shibboleth change the following attributes according to your setup:
99
	SHIB_AUTH_ENTITLEMENT = 'urn:mace'
100
	SHIB_ADMIN_DOMAIN = 'example.com'
101
	SHIB_LOGOUT_URL = 'https://example.com/Shibboleth.sso/Logout'
102
	SHIB_USERNAME = ['HTTP_EPPN']
103
	SHIB_MAIL = ['mail', 'HTTP_MAIL', 'HTTP_SHIB_INETORGPERSON_MAIL']
104
	SHIB_FIRSTNAME = ['HTTP_SHIB_INETORGPERSON_GIVENNAME']
105
	SHIB_LASTNAME = ['HTTP_SHIB_PERSON_SURNAME']
106
	SHIB_ENTITLEMENT = ['HTTP_SHIB_EP_ENTITLEMENT']
107

    
108
If you have not installed an outgoing mail server you can always use your own account (either corporate or gmail, hotmail ,etc) by adding the following lines in settings.py::
109

    
110
	EMAIL_USE_TLS = True #(or False)
111
	EMAIL_HOST = 'smtp.example.com'
112
	EMAIL_HOST_USER = 'username'
113
	EMAIL_HOST_PASSWORD = 'yourpassword'
114
	EMAIL_PORT = 587 #(outgoing)
115

    
116

    
117
.. note::
118
	Soon we will release a version with django-registration as a means to add users and Shibboleth will become an alternative
119

    
120
Let's move on with some copies and dir creations::
121

    
122
	cp urls.py.dist urls.py
123
   cd ..
124
	mkdir log
125
	chown -R root:www-data log/
126
	chmod -R g+w log
127

    
128
System configuration
129
====================
130
Apache operates as a gunicorn Proxy with WSGI and Shibboleth modules enabled.
131
Depending on the setup the apache configuration may vary::
132

    
133
	a2enmod rewrite
134
	a2enmod proxy
135
	a2enmod ssl
136
	a2enmod proxy_http
137

    
138
If shibboleth is to be used::
139

    
140
	apt-get install libapache2-mod-shib2
141
	a2enmod shib2
142

    
143
Now it is time to configure beanstalk, gunicorn, celery and apache.
144

    
145
beanstalkd
146
----------
147
Enable beanstalk by editting /etc/default/beanstalkd::
148

    
149
	vim /etc/default/beanstalkd
150

    
151
Uncomment the line **START=yes** to enable beanstalk
152

    
153
Start beanstalkd::
154

    
155
	service beanstalkd start
156

    
157
gunicorn.d
158
----------
159
Create and edit /etc/gunicorn.d/fod::
160

    
161
	vim /etc/gunicorn.d/fod
162

    
163
FoD is served via gunicorn and is then proxied by Apache. If the above directory conventions have been followed so far, then your configuration should be::
164

    
165
   CONFIG = {
166
       'mode': 'django',
167
       'working_dir': '/srv/flowspy',
168
       'args': (
169
           '--bind=127.0.0.1:8081',
170
           '--workers=1',
171
           '--worker-class=egg:gunicorn#gevent',
172
           '--timeout=30',
173
           '--log-level=debug',
174
           '--log-file=/var/log/flowspy.log',
175
       ),
176
   }
177

    
178

    
179
celeryd
180
-------
181
Celery is used over beanstalkd to apply firewall rules in a serial manner so that locks are avoided on the flowspec capable device. In our setup celery runs via django. That is why the python-django-celery package was installed.
182

    
183
Create the celeryd daemon at /etc/init.d/celeryd **if it does not already exist**::
184

    
185
   vim /etc/init.d/celeryd
186

    
187
The configuration should be::
188

    
189
   #!/bin/sh -e
190
   # ============================================
191
   #  celeryd - Starts the Celery worker daemon.
192
   # ============================================
193
   #
194
   # :Usage: /etc/init.d/celeryd {start|stop|force-reload|restart|try-restart|status}
195
   # :Configuration file: /etc/default/celeryd
196
   #
197
   # See http://docs.celeryq.org/en/latest/cookbook/daemonizing.html#init-script-celeryd
198
   
199
   
200
   ### BEGIN INIT INFO
201
   # Provides:          celeryd
202
   # Required-Start:    $network $local_fs $remote_fs
203
   # Required-Stop:     $network $local_fs $remote_fs
204
   # Default-Start:     2 3 4 5
205
   # Default-Stop:      0 1 6
206
   # Short-Description: celery task worker daemon
207
   # Description:       Starts the Celery worker daemon for a single project.
208
   ### END INIT INFO
209
   
210
   #set -e
211
   
212
   DEFAULT_PID_FILE="/var/run/celery/%n.pid"
213
   DEFAULT_LOG_FILE="/var/log/celery/%n.log"
214
   DEFAULT_LOG_LEVEL="INFO"
215
   DEFAULT_NODES="celery"
216
   DEFAULT_CELERYD="-m celery.bin.celeryd_detach"
217
   ENABLED="false"
218
   
219
   [ -r "$CELERY_DEFAULTS" ] && . "$CELERY_DEFAULTS"
220
   
221
   [ -r /etc/default/celeryd ] && . /etc/default/celeryd
222
   
223
   if [ "$ENABLED" != "true" ]; then
224
       echo "celery daemon disabled - see /etc/default/celeryd."
225
       exit 0
226
   fi
227
   
228
   
229
   CELERYD_PID_FILE=${CELERYD_PID_FILE:-${CELERYD_PIDFILE:-$DEFAULT_PID_FILE}}
230
   CELERYD_LOG_FILE=${CELERYD_LOG_FILE:-${CELERYD_LOGFILE:-$DEFAULT_LOG_FILE}}
231
   CELERYD_LOG_LEVEL=${CELERYD_LOG_LEVEL:-${CELERYD_LOGLEVEL:-$DEFAULT_LOG_LEVEL}}
232
   CELERYD_MULTI=${CELERYD_MULTI:-"celeryd-multi"}
233
   CELERYD=${CELERYD:-$DEFAULT_CELERYD}
234
   CELERYCTL=${CELERYCTL:="celeryctl"}
235
   CELERYD_NODES=${CELERYD_NODES:-$DEFAULT_NODES}
236
   
237
   export CELERY_LOADER
238
   
239
   if [ -n "$2" ]; then
240
       CELERYD_OPTS="$CELERYD_OPTS $2"
241
   fi
242
   
243
   CELERYD_LOG_DIR=`dirname $CELERYD_LOG_FILE`
244
   CELERYD_PID_DIR=`dirname $CELERYD_PID_FILE`
245
   if [ ! -d "$CELERYD_LOG_DIR" ]; then
246
       mkdir -p $CELERYD_LOG_DIR
247
   fi
248
   if [ ! -d "$CELERYD_PID_DIR" ]; then
249
       mkdir -p $CELERYD_PID_DIR
250
   fi
251
   
252
   # Extra start-stop-daemon options, like user/group.
253
   if [ -n "$CELERYD_USER" ]; then
254
       DAEMON_OPTS="$DAEMON_OPTS --uid=$CELERYD_USER"
255
       chown "$CELERYD_USER" $CELERYD_LOG_DIR $CELERYD_PID_DIR
256
   fi
257
   if [ -n "$CELERYD_GROUP" ]; then
258
       DAEMON_OPTS="$DAEMON_OPTS --gid=$CELERYD_GROUP"
259
       chgrp "$CELERYD_GROUP" $CELERYD_LOG_DIR $CELERYD_PID_DIR
260
   fi
261
   
262
   if [ -n "$CELERYD_CHDIR" ]; then
263
       DAEMON_OPTS="$DAEMON_OPTS --workdir=\"$CELERYD_CHDIR\""
264
   fi
265
   
266
   
267
   check_dev_null() {
268
       if [ ! -c /dev/null ]; then
269
           echo "/dev/null is not a character device!"
270
           exit 1
271
       fi
272
   }
273
   
274
   
275
   export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
276
   
277
   
278
   stop_workers () {
279
       $CELERYD_MULTI stop $CELERYD_NODES --pidfile="$CELERYD_PID_FILE"
280
   }
281
   
282
   
283
   start_workers () {
284
       $CELERYD_MULTI start $CELERYD_NODES $DAEMON_OPTS        \
285
                            --pidfile="$CELERYD_PID_FILE"      \
286
                            --logfile="$CELERYD_LOG_FILE"      \
287
                            --loglevel="$CELERYD_LOG_LEVEL"    \
288
                            --cmd="$CELERYD"                   \
289
                            $CELERYD_OPTS
290
   }
291
   
292
   
293
   restart_workers () {
294
       $CELERYD_MULTI restart $CELERYD_NODES $DAEMON_OPTS      \
295
                              --pidfile="$CELERYD_PID_FILE"    \
296
                              --logfile="$CELERYD_LOG_FILE"    \
297
                              --loglevel="$CELERYD_LOG_LEVEL"  \
298
                              --cmd="$CELERYD"                 \
299
                              $CELERYD_OPTS
300
   }
301
   
302
   
303
   
304
   case "$1" in
305
       start)
306
           check_dev_null
307
           start_workers
308
       ;;
309
   
310
       stop)
311
           check_dev_null
312
           stop_workers
313
       ;;
314
   
315
       reload|force-reload)
316
           echo "Use restart"
317
       ;;
318
   
319
       status)
320
           $CELERYCTL status $CELERYCTL_OPTS
321
       ;;
322
   
323
       restart)
324
           check_dev_null
325
           restart_workers
326
       ;;
327
   
328
       try-restart)
329
           check_dev_null
330
           restart_workers
331
       ;;
332
   
333
       *)
334
           echo "Usage: /etc/init.d/celeryd {start|stop|restart|try-restart|kill}"
335
           exit 1
336
       ;;
337
   esac
338
   
339
   exit 0
340

    
341
celeryd configuration
342
---------------------
343
celeryd requires a /etc/default/celeryd file to be in place.
344
Thus we are going to create this file (/etc/default/celeryd)::
345

    
346
	vim /etc/default/celeryd
347

    
348
Again if the directory conventions have been followed the file is (pay attention to the CELERYD_USER, CELERYD_GROUP and change accordingly)  ::
349

    
350
   # Default: false
351
   ENABLED="true"
352
   
353
   # Name of nodes to start, here we have a single node
354
   CELERYD_NODES="w1"
355
   # or we could have three nodes:
356
   #CELERYD_NODES="w1 w2 w3"
357
   
358
   # Where to chdir at start.
359
   CELERYD_CHDIR="/srv/flowspy"
360
   # How to call "manage.py celeryd_multi"
361
   CELERYD_MULTI="python $CELERYD_CHDIR/manage.py celeryd_multi"
362
   
363
   # How to call "manage.py celeryctl"
364
   CELERYCTL="python $CELERYD_CHDIR/manage.py celeryctl"
365
   
366
   # Extra arguments to celeryd
367
   #CELERYD_OPTS="--time-limit=300 --concurrency=8"
368
   CELERYD_OPTS="-E -B --schedule=/var/run/celery/celerybeat-schedule"
369
   # Name of the celery config module.
370
   CELERY_CONFIG_MODULE="celeryconfig"
371
   
372
   # %n will be replaced with the nodename.
373
   CELERYD_LOG_FILE="/var/log/celery/%n.log"
374
   CELERYD_PID_FILE="/var/run/celery/%n.pid"
375
   
376
   # Workers should run as an unprivileged user.
377
   CELERYD_USER="celery"
378
   CELERYD_GROUP="celery"
379
   
380
   # Name of the projects settings module.
381
   export DJANGO_SETTINGS_MODULE="flowspy.settings"
382

    
383
Apache
384
------
385
Apache proxies gunicorn. Things are more flexible here as you may follow your own configuration and conventions. Create and edit /etc/apache2/sites-available/fod. You should set <server_name> and <admin_mail> along with your certificates. If under testing environment, you can use the provided snakeoil certs. If you do not intent to use Shibboleth delete or comment the corresponding configuration parts inside **Shibboleth configuration** ::
386

    
387
	vim /etc/apache2/sites-available/fod
388

    
389
Again if the directory conventions have been followed the file should be::
390

    
391
   <VirtualHost *:80>
392
      ServerAdmin webmaster@localhost
393
      ServerName  fod.example.com
394
      DocumentRoot /var/www
395
   
396
      ErrorLog ${APACHE_LOG_DIR}/fod_error.log
397
   
398
      # Possible values include: debug, info, notice, warn, error, crit,
399
      # alert, emerg.
400
      LogLevel debug
401
      
402
      CustomLog ${APACHE_LOG_DIR}/fod_access.log combined
403
   
404
      Alias /static     /srv/flowspy/static
405
       RewriteEngine On
406
       RewriteCond %{HTTPS} off
407
       RewriteRule ^/(.*) https://fod.example.com/$1 [L,R]
408
   </VirtualHost>
409
   
410
   <VirtualHost *:443>
411
      ServerName   fod.example.com
412
      ServerAdmin    webmaster@localhost
413
      ServerSignature      On
414
      
415
      SSLEngine on
416
      SSLCertificateFile   /etc/ssl/certs/fod.example.com.crt
417
      SSLCertificateChainFile /etc/ssl/certs/example-chain.pem
418
      SSLCertificateKeyFile   /etc/ssl/private/fod.example.com.key
419
   
420
      AddDefaultCharset UTF-8
421
      IndexOptions      +Charset=UTF-8
422
   
423
      ShibConfig     /etc/shibboleth/shibboleth2.xml
424
      Alias       /shibboleth-sp /usr/share/shibboleth
425
   
426
   
427
      <Location /login>
428
         AuthType shibboleth
429
         ShibRequireSession On
430
         ShibUseHeaders On
431
         ShibRequestSetting entityID https://idp.example.com/idp/shibboleth
432
         require valid-user
433
      </Location>
434
      
435
      # Shibboleth debugging CGI script
436
      ScriptAlias /shibboleth/test /usr/lib/cgi-bin/shibtest.cgi
437
      <Location /shibboleth/test>
438
         AuthType shibboleth
439
         ShibRequireSession On
440
         ShibUseHeaders On
441
         require valid-user
442
      </Location>
443
   
444
      <Location /Shibboleth.sso>
445
         SetHandler shib
446
      </Location>
447
   
448
      # Shibboleth SP configuration
449
   
450
      #SetEnv                proxy-sendchunked
451
      
452
          <Proxy *>
453
           Order allow,deny
454
           Allow from all
455
           </Proxy>
456
   
457
           SSLProxyEngine        off
458
           ProxyErrorOverride    off
459
       ProxyTimeout    28800
460
         ProxyPass      /static !
461
         ProxyPass       /shibboleth !
462
         ProxyPass      /Shibboleth.sso !
463
         
464
           ProxyPass        / http://localhost:8081/ retry=0
465
           ProxyPassReverse / http://localhost:8081/
466
   
467
       Alias /static       /srv/flowspy/static
468
   
469
      LogLevel warn
470
      
471
      ErrorLog ${APACHE_LOG_DIR}/fod_error.log
472
       CustomLog ${APACHE_LOG_DIR}/fod_access.log combined
473
   
474
   </VirtualHost>
475

    
476
Now, enable your site. You might want to disable the default site if fod is the only site you host on your server::
477

    
478
	a2dissite default
479
	a2ensite fod
480

    
481
You are not far away from deploying FoD. When asked for a super user, create one::
482

    
483
	cd /srv/flowspy
484
	python manage.py syncdb
485
	python manage.py migrate
486

    
487
Restart, gunicorn and apache::
488

    
489
	service gunicorn restart && service apache2 restart
490

    
491

    
492
Propagate the flatpages
493
=======================
494
Inside the initial_data/fixtures_manual.xml file we have placed 4 flatpages (2 for Greek, 2 for English) with Information and Terms of Service about the service. 
495
To import the flatpages, run from root folder::
496

    
497
   python manage.py loaddata initial_data/fixtures_manual.xml
498

    
499

    
500

    
501
Testing the platform
502
====================
503
Log in to the admin interface via https://<hostname>/admin. Go to Peer ranges and add a new range (part of/or a complete subnet), eg. 10.20.0.0/19
504
Go to Peers and add a new peer, eg. id: 1, name: Test, AS: 16503, tag: TEST and move the network you have crteated from Avalable to Chosen. From the admin front, go to User, and edit your user. From the bottom of the page, select the TEST peer and save.
505
Last but not least, modify as required the existing (example.com) Site instance (admin home->Sites). You are done. As you are logged-in via the admin, there is no need for Shibboleth. Go to https://<hostname>/ and create a new rule. Your rule should be applied on the flowspec capable device after aprox. 10 seconds.
506

    
507
Branding
508
========
509
Via the admin interface you can modify flatpages to suit your needs
510

    
511
Footer
512
------
513
Under the templates folder (templates), you can alter the footer.html file to include your own footer messages, badges, etc.
514

    
515
Welcome Page
516
------------
517
Under the templates folder (templates), you can alter the welcome page - welcome.html with your own images, carousel, videos, etc.