Statistics
| Branch: | Tag: | Revision:

root / README.deploy @ 728e7d91

History | View | Annotate | Download (17.7 kB)

1
README.deploy -- Instructions for a basic Synnefo deployment
2

    
3
This document describes the basic steps to obtain a basic, working Synnefo
4
deployment. It begins by examining the different node roles, then moves to the
5
installation and setup of distinct software components.
6

    
7
It is current as of Synnefo v0.7.
8

    
9

    
10
Node types
11
===========
12

    
13
Nodes in a Synnefo deployment belong in one of the following types:
14

    
15
 * DB:
16
   A node [or more than one nodes, if using an HA configuration], running a DB
17
   engine supported by the Django ORM layer. The DB is the single source of
18
   truth for the servicing of API requests by Synnefo.
19
   Services: PostgreSQL / MySQL
20

    
21
 * APISERVER:
22
   A node running the implementation of the OpenStack API, in Django. Any number
23
   of APISERVERs can be used, in a load-balancing configuration, without any
24
   special consideration. Access to a common DB ensures consistency.
25
   Services: Web server, vncauthproxy
26

    
27
 * QUEUE:
28
   A node running the RabbitMQ software, which provides AMQP functionality. More
29
   than one QUEUE nodes may be deployed, in an HA configuration. Such
30
   deployments require shared storage, provided e.g., by DRBD.
31
   Services: RabbitMQ [rabbitmq-server]
32

    
33
 * LOGIC:
34
   A node running the business logic of Synnefo, in Django. It dequeues
35
   messages from QUEUE nodes, and provides the context in which business logic
36
   functions run. It uses Django ORM to connect to the common DB and update the
37
   state of the system, based on notifications received from the rest of the
38
   infrastructure, over AMQP.
39
   Services: the Synnefo logic dispatcher [/logic/dispatcher.py]
40

    
41
 * GANETI-MASTER and GANETI-NODE:
42
   A single GANETI-MASTER and a large number of GANETI-NODEs constitute the
43
   Ganeti backend for Synnefo, which undertakes all VM management functions.
44
   Any APISERVER can issue commands to the GANETI-MASTER, over RAPI, to effect
45
   changes in the state of the VMs. The GANETI-MASTER runs the Ganeti request
46
   queue.
47
   Services:
48
     only on GANETI-MASTER:
49
       the Synnefo Ganeti monitoring daemon [/ganeti/snf-ganeti-eventd]
50
       the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py].
51
     on each GANETI_NODE:
52
       a deployment-specific KVM ifup script
53
       properly configured NFDHCPD
54

    
55

    
56
Installation Process
57
=====================
58

    
59
This section describes the installation process of the various node roles in a
60
Synnefo deployment.
61

    
62

    
63
0. Allocation of physical nodes:
64
   Determine the role of every physical node in your deployment.
65

    
66

    
67
1. Ganeti installation:
68
   Synnefo requires a working Ganeti installation at the backend. Installation
69
   of Ganeti is not covered by this document, please refer to
70
   http://docs.ganeti.org/ganeti/current/html for all the gory details. A
71
   successful Ganeti installation concludes with a working GANETI-MASTER and a
72
   number of GANETI-NODEs.
73

    
74

    
75
2. RabbitMQ installation:
76
   RabbitMQ is used as a generic message broker for the system. It should be
77
   installed on two seperate QUEUE nodes (VMs should be enough for the moment)
78
   in a high availability configuration as described here:
79

    
80
     http://www.rabbitmq.com/pacemaker.html
81

    
82
   After installation, create a user and set its permissions
83
     rabbitmqctl add_user okeanos 0k3@n0s
84
     rabbitmqctl set_permissions -p / okeanos  "^.*" ".*" ".*"
85

    
86
   The values set for the user and password must be mirrored in the
87
   RABBIT_* variables in settings.py (see step 6)
88

    
89

    
90
3. Web server installation:
91
   A Web Server (e.g., Apache) needs to be installed on the APISERVERs,
92
   and be configured to run the Synnefo Django project appropriately. Selection
93
   and configuration of a Web server is outside the scope of this document.
94

    
95
   For testing or development purposes, Django's own development server,
96
   `./manage.py runserver' can be used.
97

    
98

    
99
4. Installation of the Synnefo Django project:
100
   As of v0.5 the Synnefo Django project needs to be installed on nodes
101
   of type APISERVER, and LOGIC, with a properly configured settings.py. In
102
   later revisions, the specific parts of the Django project which need to run
103
   on each node type will be identified.
104

    
105
   Synnefo is written in Python 2.6 and depends on the following Python modules:
106
   [package versions confirmed to be compatible are in braces]
107

    
108
    * django 1.2 [Django==1.2.4]
109
    * simplejson [simplejson==2.1.3]
110
    * pycurl [pycurl==7.19.0]
111
    * python-dateutil  [python-dateutil==1.4.1]
112
      WARNING: version python-dateutil==2.0 downloaded by pip known *not* to
113
               work with Python 2.6
114
    * python-ipy [IPy==0.75]
115
        also verified to work with python-ipy 0.70-1 as shipped with Squeeze
116
    * south [south==0.7.1]
117
      WARNING: might not work with Debian Squeeze's default south-0.7-1 package.
118
    * amqplib [amqplib==0.6.1]
119
    * lockfile [lockfile==0.8]
120
    * python-daemon [python-daemon==1.5.5]
121
    * python-prctl [python-prctl==1.3.0]
122

    
123
   also, depending on the database engine of choice, on one of the following:
124
    * MySQL-python [MySQL-python==1.2.3]
125
    * psycopg2 [psycopg2==2.4]
126

    
127
   if the invitations application is deployed, the following dependencies should
128
   be installed:
129
    * pycrypto==2.1.0
130

    
131
   for server side ssh key pair generation to work the following module is required:
132
    * M2Crypto==0.20.1
133

    
134
   The integration test suite snf-tools/snf-test depends on:
135
    * python-unittest2 [unittest2==0.5.1]
136
    * python-paramiko  [paramiko==1.7.6], version included in Debian Squeeze
137
      is broken wrt to use of RandomPool, see Debian bug #576697
138
    * python-ipy [IPy==0.75]
139
    * python-prctl [python-prctl==1.3.0]
140
    * the client component of vncauthproxy, see Step 12
141
    * the kamaki client library, please see
142
      https://code.grnet.gr/projects/kamaki for installation instructions.
143
      [FIXME: Update instructions on kamaki installation]
144

    
145
   To run the user interface tests, selenium must be installed
146
    * selenium [?]
147

    
148
   The easiest method for installation of the Django project is to setup a
149
   working environment through virtualenv. Alternatively, you can use your
150
   system's package manager to install the dependencies (e.g. Macports has them
151
   all).
152

    
153
   * On Snow Leopard and linux (64-bit), you have to set the following
154
     environment variable for pip to compile the dependencies correctly.
155

    
156
  	   $ export ARCHFLAGS="-arch x86_64"
157

    
158
   * On Ubuntu, a few more packages must be installed before installing the
159
     prerequisite Python libraries
160

    
161
	   $ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
162

    
163
   Checkout the code and install the Python prerequisites. This assumes that
164
   python is already installed on the host.
165

    
166
    $ sudo easy_install virtualenv
167
    $ git clone https://user@code.grnet.gr/git/synnefo synnefo
168
    $ virtualenv --python=python2.6 synnefo --no-site-packages
169
    ...
170
    $ cd synnefo
171
    $ ./bin/pip install <list_of_dependencies>
172

    
173
    [WARNING]: The software must be checked out in a directory named synnefo,
174
    otherwise python imports will not work. Therefore, do not change the
175
    or rename the checkout path.
176

    
177

    
178
5. Database installation:
179
   A database supported by the Django ORM layer must be installed on nodes
180
   of type DB. The choices are: SQLIte, MySQL, PostgreSQL.
181

    
182
   * SQLite:
183
     The python sqlite driver is available by default with Python so no
184
     additional configuration is required. Also, most self-respecting systems
185
     have the sqlite library installed by default.
186

    
187
   * MySQL:
188
      MySQL must be installed first:
189

    
190
      * Ubuntu - Debian
191
	      $ sudo apt-get install libmysqlclient-dev
192

    
193
      * MacPorts
194
	      $ sudo port install mysql5
195

    
196
      Install the MySQL python library on servers running the Django project:
197

    
198
	    $ bin/pip install MySQL-python
199

    
200
      Note: On MacOSX with Mysql install from MacPorts the above command will
201
            fail complaining that it cannot find the mysql_config command. Do
202
            the following and restart the installation
203
	        $ echo "mysql_config = /opt/local/bin/mysql_config5" >> \
204
                                         ./build/MySQL-python/site.cfg
205

    
206
      Configure a MySQL db/account for synnefo
207
	    $ mysql -u root -p
208

    
209
    	mysql> create database synnefo;
210
	    mysql> show databases;
211
	    mysql> GRANT ALL on synnefo.* TO username IDENTIFIED BY 'password';
212

    
213
     IMPORTANT:
214
        MySQL *must* be set in READ-COMMITED mode, e.g. by setting
215

    
216
        transaction-isolation = READ-COMMITTED
217

    
218
        in the [mysqld] section of /etc/mysql/my.cnf.
219

    
220
        Alternatively, make sure the following code fragment stays enabled
221
        in settings.d/10-database.conf:
222

    
223
            if DATABASES['default']['ENGINE'].endswith('mysql'):
224
                DATABASES['default']['OPTIONS'] = {
225
                        'init_command': 'SET storage_engine=INNODB; ' +
226
                            'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
227
                }
228
          
229
   * PostgreSQL
230
     You need to install the PostgreSQL binaries:
231
     * Ubuntu - Debian
232
	     $ sudo apt-get install postgresql-8.4 libpq-dev
233

    
234
     * MacPorts
235
	     $ sudo port install postgresql84
236

    
237
     Install the postgres Python library
238
	    $ bin/pip install psycopg2
239

    
240
     Configure a postgres db/account for synnefo:
241

    
242
     Become the postgres user, connect to PostgreSQL:
243
       $ sudo su - postgres
244
       $ psql
245
	
246
	 Run the following commands:
247
	   DROP DATABASE synnefo;
248
	   DROP USER username;
249
	   CREATE USER username WITH PASSWORD 'password';
250
	   CREATE DATABASE synnefo;
251
	   GRANT ALL PRIVILEGES ON DATABASE synnefo TO username;
252
	   ALTER DATABASE synnefo OWNER TO username;
253
	   ALTER USER username CREATEDB;
254

    
255
     The last line enables the newly created user to create own databases. This
256
     is needed for Django to create and drop the test_synnefo database for unit
257
     testing.
258

    
259

    
260
6. Setting up the Django project:
261
   The settings.py file for Django may be derived by concatenating the
262
   settings.py.dist file contained in the Synnefo distribution with a file
263
   containing custom modifications, which shall override all settings deviating
264
   from the supplied settings.py.dist. This is recommended to minimize the load
265
   of reconstructing settings.py from scratch, since each release currently
266
   brings heavy changes to settings.py.dist.
267

    
268
   Add the following to your custom settings.py, depending on your choice
269
   of DB:
270
   * SQLite
271

    
272
	 PROJECT_PATH = os.path.dirname(os.path.abspath(__file__)) + '/'
273

    
274
	 DATABASES = {
275
	     'default': {
276
		     'ENGINE': 'django.db.backends.sqlite3',
277
		     'NAME': PROJECT_PATH + 'synnefo.db' # WARN: This must be an absolute path
278
	     }
279
	 }
280

    
281
   * MySQL
282

    
283
 	 DATABASES = {
284
	     'default': {
285
             'ENGINE': 'django.db.backends.mysql',
286
             'NAME': 'synnefo',
287
             'USER': 'USERNAME',
288
             'PASSWORD': 'PASSWORD',
289
             'HOST': 'HOST',
290
             'PORT': 'PORT',
291
             'OPTIONS': {
292
                 'init_command': 'SET storage_engine=INNODB',
293
             }
294
	    }
295
	}
296

    
297
   * PostgreSQL
298

    
299
     DATABASES = {
300
	     'default': {
301
             'ENGINE': 'django.db.backends.postgresql_psycopg2',
302
             'NAME': 'DATABASE',
303
             'USER': 'USERNAME',
304
             'PASSWORD': 'PASSWORD',
305
             'HOST': 'HOST',
306
             'PORT': 'PORT',
307
	     }
308
     }
309

    
310
    Try it out. The following command will attempt to connect to the DB and
311
    print out DDL statements. It should not fail.
312

    
313
	$ ./bin/python manage.py sql db
314

    
315

    
316
7. Initialization of Synnefo DB:
317
   You need to initialize the Synnefo DB and load fixtures
318
   db/fixtures/{users,flavors,images}.json, which make the API usable by end
319
   users by defining a sample set of users, hardware configurations (flavors)
320
   and OS images.
321

    
322
   IMPORTANT: Be sure to modify db/fixtures/users.json and select
323
   a unique token for each of the initial and any other users defined in this
324
   file. DO NOT LEAVE THE SAMPLE AUTHENTICATION TOKENS enabled in deployed
325
   configurations.
326

    
327
     $ ./bin/python manage.py syncdb
328
     $ ./bin/python manage.py migrate db
329
     $ ./bin/python manage.py loaddata db/fixtures/users.json
330
     $ ./bin/python manage.py loaddata db/fixtures/flavors.json
331
     $ ./bin/python manage.py loaddata db/fixtures/images.json
332

    
333

    
334
8. Finalization of settings.py:
335
   Set the BACKEND_PREFIX_ID variable to some unique prefix, e.g. your commit
336
   username in settings.py. Several functional conventions within the system
337
   require this variable to include a dash at its end (e.g. snf-)
338

    
339

    
340
9. Installation of the Ganeti monitoring daemon, /ganeti/snf-ganeti-eventd:
341
   The Ganeti monitoring daemon must run on GANETI-MASTER.
342

    
343
   The monitoring daemon is configured through /etc/synnefo/settings.conf.
344
   An example is provided under snf-ganeti-tools/.
345

    
346
   If run from the repository directory, make sure to have snf-ganeti-tools/
347
   in the PYTHONPATH.
348

    
349
   You may also build Debian packages directly from the repository:
350
   $ cd snf-ganeti-tools
351
   $ dpkg-buildpackage -b -uc -us
352
   # dpkg -i ../snf-ganeti-tools-*deb
353

    
354
   TBD: how to handle master migration.
355

    
356

    
357
10. Installation of the Synnefo dispatcher, /logic/dispatcher.py:
358
    The logic dispatcher is part of the Synnefo Django project and must run
359
    on LOGIC nodes.
360

    
361
    The dispatcher retrieves messages from the queue and calls the appropriate
362
    handler function as defined in the queue configuration in `setttings.py'.
363
    The default configuration should work directly without any modifications.
364

    
365
    For the time being The dispatcher must be run by hand:
366
      $ ./bin/python ./logic/dispatcher.py
367

    
368
    The dispatcher should run in at least 2 instances to ensure high
369
    (actually, increased) availability.
370

    
371

    
372
11. Installation of the Synnefo Ganeti hook:
373
    The generic Synnefo Ganeti hook wrapper resides in the snf-ganeti-tools/
374
    directory of the Synnefo repository.
375

    
376
    The hook needs to be enabled for phases post-{add,modify,reboot,start,stop}
377
    by *symlinking* in
378
    /etc/ganeti/hooks/instance-{add,modify,reboot,start,stop}-post.d on
379
    GANETI-MASTER, e.g.:
380

    
381
    root@ganeti-master:/etc/ganeti/hooks/instance-start-post.d# ls -l
382
    lrwxrwxrwx 1 root root 45 May   3 13:45 00-snf-ganeti-hook -> /home/devel/synnefo/snf-ganeti-hook/snf-ganeti-hook.py
383

    
384
    IMPORTANT: The link name may only contain "upper and lower case, digits,
385
    underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9_-]+$."
386
    See:
387
    http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming
388

    
389
    If run from the repository directory, make sure to have snf-ganeti-tools/
390
    in the PYTHONPATH.
391

    
392
    Alternative, build Debian packages which take care of building, installing
393
    and activating the Ganeti hook automatically, see step. 9.
394

    
395

    
396
12. Installation of the VNC authentication proxy, vncauthproxy:
397
    To support OOB console access to the VMs over VNC, the vncauthproxy
398
    daemon must be running on every node of type APISERVER.
399

    
400
    Download and install vncauthproxy from its own repository,
401
    at https://code.grnet.gr/git/vncauthproxy (known good commit: tag v1.0).
402

    
403
    Download and install a specific repository commit:
404

    
405
    $ bin/pip install -e git+https://code.grnet.gr/git/vncauthproxy@INSERT_COMMIT_HERE#egg=vncauthproxy
406

    
407
    Create /var/log/vncauthproxy and set its permissions appropriately.
408

    
409
    Alternatively, you can build Debian packages. To do so,
410
    checkout the "debian" branch of the vncauthproxy repository
411
    (known good commit: tag debian/v1.0):
412

    
413
    $ git checkout debian
414

    
415
    Then build debian package, and install as root:
416

    
417
    $ dpkg-buildpackage -b -uc -us
418
    # dpkg -i ../vncauthproxy_1.0-1_all.deb
419

    
420
    --Failure to build the package on the Mac.
421

    
422
    libevent, a requirement for gevent which in turn is a requirement for
423
    vncauthproxy is not included in MacOSX by default and installing it with
424
    MacPorts does not lead to a version that can be found by the gevent
425
    build process. A quick workaround is to execute the following commands:
426

    
427
    cd $SYNNEFO
428
    sudo pip install -e git+https://code.grnet.gr/git/vncauthproxy@5a196d8481e171a#egg=vncauthproxy
429
    <the above fails>
430
    cd build/gevent
431
    sudo python setup.py -I/opt/local/include -L/opt/local/lib build
432
    cd $SYNNEFO
433
    sudo pip install -e git+https://code.grnet.gr/git/vncauthproxy@5a196d8481e171a#egg=vncauthproxy
434

    
435

    
436
13. Installation of the snf-image Ganeti OS provider for image deployment:
437
    For Synnefo to be able to launch VMs from specified Images, you need
438
    the snf-image OS Provider installed on *all* Ganeti nodes.
439

    
440
    Please see https://code.grnet.gr/projects/snf-image/wiki
441
    for installation instructions and documentation on the design
442
    and implementation of snf-image.
443

    
444
    Please see https://code.grnet.gr/projects/snf-image/files
445
    for the latest packages.
446

    
447
    Images should be stored under extdump format in a directory
448
    of your choice, configurable as IMAGE_DIR in /etc/default/snf-image.
449

    
450

    
451
14. Setup Synnefo-specific networking on the Ganeti backend:
452
    This part is deployment-specific and must be customized based on the
453
    specific needs of the system administrators.
454

    
455
    A reference installation will use a Synnefo-specific KVM ifup script,
456
    NFDHCPD and pre-provisioned Linux bridges to support public and private
457
    network functionality. For this:
458

    
459
    Grab NFDHCPD from its own repository (https://code.grnet.gr/git/nfdhcpd),
460
    install it, modify /etc/nfdhcpd/nfdhcpd.conf to reflect your network
461
    configuration.
462

    
463
    Install a custom KVM ifup script for use by Ganeti, as
464
    /etc/ganeti/kvm-vif-bridge, on GANETI-NODEs. A sample implementation is
465
    provided under /contrib/ganeti-hooks. Set NFDHCPD_STATE_DIR to point
466
    to NFDHCPD's state directory, usually /var/lib/nfdhcpd.
467

    
468

    
469
15. See section "Logging" in README.admin, and edit settings.d/00-logging.conf
470
    according to your OS and individual deployment characteristics.
471

    
472

    
473
16. Optionally, read the okeanos_site/README file to setup ~okeanos introductory 
474
    site (intro, video/info pages). Please see okeanos_site/90-okeanos.sample
475
    for a sample configuration file which overrides site-specific variables,
476
    to be placed under settings.d/, after customization.
477

    
478

    
479
17. (Hopefully) Done
480

    
481