Statistics
| Branch: | Tag: | Revision:

root / README.deploy @ 96b635d9

History | View | Annotate | Download (17.8 kB)

1
README.deploy -- Instructions for a basic deployment of Synnefo v0.5.1
2

    
3
This document describes the basic steps to obtain a basic, working Synnefo
4
deployment. It begins by examining the different node roles, then moves to the
5
installation and setup of distinct software components.
6

    
7
It is current as of Synnefo v0.5.1.
8

    
9

    
10
Node types
11
===========
12

    
13
Nodes in a Synnefo deployment belong in one of the following types:
14

    
15
 * DB:
16
   A node [or more than one nodes, if using an HA configuration], running a DB
17
   engine supported by the Django ORM layer. The DB is the single source of
18
   truth for the servicing of API requests by Synnefo.
19
   Services: PostgreSQL / MySQL
20

    
21
 * APISERVER:
22
   A node running the implementation of the OpenStack API, in Django. Any number
23
   of APISERVERs can be used, in a load-balancing configuration, without any
24
   special consideration. Access to a common DB ensures consistency.
25
   Services: Web server, vncauthproxy
26

    
27
 * QUEUE:
28
   A node running the RabbitMQ software, which provides AMQP functionality. More
29
   than one QUEUE nodes may be deployed, in an HA configuration. Such
30
   deployments require shared storage, provided e.g., by DRBD.
31
   Services: RabbitMQ [rabbitmq-server]
32

    
33
 * LOGIC:
34
   A node running the business logic of Synnefo, in Django. It dequeues
35
   messages from QUEUE nodes, and provides the context in which business logic
36
   functions run. It uses Django ORM to connect to the common DB and update the
37
   state of the system, based on notifications received from the rest of the
38
   infrastructure, over AMQP.
39
   Services: the Synnefo logic dispatcher [/logic/dispatcher.py]
40

    
41
 * GANETI-MASTER and GANETI-NODE:
42
   A single GANETI-MASTER and a large number of GANETI-NODEs constitute the
43
   Ganeti backend for Synnefo, which undertakes all VM management functions.
44
   Any APISERVER can issue commands to the GANETI-MASTER, over RAPI, to effect
45
   changes in the state of the VMs. The GANETI-MASTER runs the Ganeti request
46
   queue.
47
   Services:
48
     only on GANETI-MASTER:
49
       the Synnefo Ganeti monitoring daemon [/ganeti/snf-ganeti-eventd]
50
       the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py].
51
     on each GANETI_NODE:
52
       a deployment-specific KVM ifup script
53
       properly configured NFDHCPD
54

    
55

    
56
Installation Process
57
=====================
58

    
59
This section describes the installation process of the various node roles in a
60
Synnefo deployment.
61

    
62

    
63
0. Allocation of physical nodes:
64
   Determine the role of every physical node in your deployment.
65

    
66

    
67
1. Ganeti installation:
68
   Synnefo requires a working Ganeti installation at the backend. Installation
69
   of Ganeti is not covered by this document, please refer to
70
   http://docs.ganeti.org/ganeti/current/html for all the gory details. A
71
   successful Ganeti installation concludes with a working GANETI-MASTER and a
72
   number of GANETI-NODEs.
73

    
74

    
75
2. RabbitMQ installation:
76
   RabbitMQ is used as a generic message broker for the system. It should be
77
   installed on two seperate QUEUE nodes (VMs should be enough for the moment)
78
   in a high availability configuration as described here:
79

    
80
     http://www.rabbitmq.com/pacemaker.html
81

    
82
   After installation, create a user and set its permissions
83
     rabbitmqctl add_user okeanos 0k3@n0s
84
     rabbitmqctl set_permissions -p / okeanos  "^.*" ".*" ".*"
85

    
86
   The values set for the user and password must be mirrored in the
87
   RABBIT_* variables in settings.py (see step 6)
88

    
89

    
90
3. Web server installation:
91
   A Web Server (e.g., Apache) needs to be installed on the APISERVERs,
92
   and be configured to run the Synnefo Django project appropriately. Selection
93
   and configuration of a Web server is outside the scope of this document.
94

    
95
   For testing or development purposes, Django's own development server,
96
   `./manage.py runserver' can be used.
97

    
98

    
99
4. Installation of the Synnefo Django project:
100
   As of v0.5 the Synnefo Django project needs to be installed on nodes
101
   of type APISERVER, LOGIC and on the GANETI-MASTER, with a properly
102
   configured settings.py. In later revisions, the specific parts of the Django
103
   project which need to run on each node type will be identified.
104

    
105
   Synnefo is written in Python 2.6 and depends on the following Python modules:
106
   [package versions confirmed to be compatible are in braces]
107

    
108
    * django 1.2 [Django==1.2.4]
109
    * simplejson [simplejson==2.1.3]
110
    * pycurl [pycurl==7.19.0]
111
    * python-dateutil  [python-dateutil==1.4.1]
112
      WARNING: version python-dateutil==2.0 downloaded by pip known *not* to
113
               work with Python 2.6
114
    * python-ipy [IPy==0.75]
115
        also verified to work with python-ipy 0.70-1 as shipped with Squeeze
116
    * south [south==0.7.1]
117
      WARNING: might not work with Debian squeeze's default south-0.7-1 package.
118
    * amqplib [amqplib==0.6.1]
119
    * lockfile [lockfile==0.8]
120
    * python-daemon [python-daemon==1.5.5]
121

    
122
   also, depending on the database engine of choice, on one of the following:
123
    * MySQL-python [MySQL-python==1.2.3]
124
    * psycopg2 [psycopg2==2.4]
125

    
126
   if the invitations application is deployed, the following dependencies should
127
   be installed:
128
    * pycrypto==2.1.0
129

    
130
   To run the user interface tests, selenium must be installed
131
    * selenium [?]
132

    
133
   The easiest method for installation of the Django project is to setup a
134
   working environment through virtualenv. Alternatively, you can use your
135
   system's package manager to install the dependencies (e.g. Macports has them
136
   all).
137

    
138
   * On Snow Leopard and linux (64-bit), you have to set the following
139
     environment variable for pip to compile the dependencies correctly.
140

    
141
  	   $ export ARCHFLAGS="-arch x86_64"
142

    
143
   * On Ubuntu, a few more packages must be installed before installing the
144
     prerequisite Python libraries
145

    
146
	   $ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
147

    
148
   Checkout the code and install the Python prerequisites. This assumes that
149
   python is already installed on the host.
150

    
151
    $ sudo easy_install virtualenv
152
    $ git clone https://user@code.grnet.gr/git/synnefo synnefo
153
    $ virtualenv --python=python2.6 synnefo --no-site-packages
154
    ...
155
    $ cd synnefo
156
    $ ./bin/pip install <list_of_dependencies>
157

    
158
    [WARNING]: The software must be checked out in a directory named synnefo,
159
    otherwise python imports will not work. Therefore, do not change the
160
    or rename the checkout path.
161

    
162

    
163
5. Database installation:
164
   A database supported by the Django ORM layer must be installed on nodes
165
   of type DB. The choices are: SQLIte, MySQL, PostgreSQL.
166

    
167
   * SQLite:
168
     The python sqlite driver is available by default with Python so no
169
     additional configuration is required. Also, most self-respecting systems
170
     have the sqlite library installed by default.
171

    
172
   * MySQL:
173
      MySQL must be installed first:
174

    
175
      * Ubuntu - Debian
176
	      $ sudo apt-get install libmysqlclient-dev
177

    
178
      * MacPorts
179
	      $ sudo port install mysql5
180

    
181
      Install the MySQL python library on servers running the Django project:
182

    
183
	    $ bin/pip install MySQL-python
184

    
185
      Note: On MacOSX with Mysql install from MacPorts the above command will
186
            fail complaining that it cannot find the mysql_config command. Do
187
            the following and restart the installation
188
	        $ echo "mysql_config = /opt/local/bin/mysql_config5" >> \
189
                                         ./build/MySQL-python/site.cfg
190

    
191
      Configure a MySQL db/account for synnefo
192
	    $ mysql -u root -p
193

    
194
    	mysql> create database synnefo;
195
	    mysql> show databases;
196
	    mysql> GRANT ALL on synnefo.* TO username IDENTIFIED BY 'password';
197

    
198
     IMPORTANT:
199
        MySQL *must* be set in READ-COMMITED mode, e.g. by setting
200

    
201
        transaction-isolation = READ-COMMITTED
202

    
203
        in the [mysqld] section of /etc/mysql/my.cnf.
204

    
205
        Alternatively, make sure the following code fragment stays enabled
206
        in settings.d/10-database.conf:
207

    
208
            if DATABASES['default']['ENGINE'].endswith('mysql'):
209
                DATABASES['default']['OPTIONS'] = {
210
                        'init_command': 'SET storage_engine=INNODB; ' +
211
                            'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
212
                }
213
          
214
   * PostgreSQL
215
     You need to install the PostgreSQL binaries:
216
     * Ubuntu - Debian
217
	     $ sudo apt-get install postgresql-8.4 libpq-dev
218

    
219
     * MacPorts
220
	     $ sudo port install postgresql84
221

    
222
     Install the postgres Python library
223
	    $ bin/pip install psycopg2
224

    
225
     Configure a postgres db/account for synnefo:
226

    
227
     Become the postgres user, connect to PostgreSQL:
228
       $ sudo su - postgres
229
       $ psql
230
	
231
	 Run the following commands:
232
	   DROP DATABASE synnefo;
233
	   DROP USER username;
234
	   CREATE USER username WITH PASSWORD 'password';
235
	   CREATE DATABASE synnefo;
236
	   GRANT ALL PRIVILEGES ON DATABASE synnefo TO username;
237
	   ALTER DATABASE synnefo OWNER TO username;
238
	   ALTER USER username CREATEDB;
239

    
240
     The last line enables the newly created user to create own databases. This
241
     is needed for Django to create and drop the test_synnefo database for unit
242
     testing.
243

    
244

    
245
6. Setting up the Django project:
246
   The settings.py file for Django may be derived by concatenating the
247
   settings.py.dist file contained in the Synnefo distribution with a file
248
   containing custom modifications, which shall override all settings deviating
249
   from the supplied settings.py.dist. This is recommended to minimize the load
250
   of reconstructing settings.py from scratch, since each release currently
251
   brings heavy changes to settings.py.dist.
252

    
253
   Add the following to your custom settings.py, depending on your choice
254
   of DB:
255
   * SQLite
256

    
257
	 PROJECT_PATH = os.path.dirname(os.path.abspath(__file__)) + '/'
258

    
259
	 DATABASES = {
260
	     'default': {
261
		     'ENGINE': 'django.db.backends.sqlite3',
262
		     'NAME': PROJECT_PATH + 'synnefo.db' # WARN: This must be an absolute path
263
	     }
264
	 }
265

    
266
   * MySQL
267

    
268
 	 DATABASES = {
269
	     'default': {
270
             'ENGINE': 'django.db.backends.mysql',
271
             'NAME': 'synnefo',
272
             'USER': 'USERNAME',
273
             'PASSWORD': 'PASSWORD',
274
             'HOST': 'HOST',
275
             'PORT': 'PORT',
276
             'OPTIONS': {
277
                 'init_command': 'SET storage_engine=INNODB',
278
             }
279
	    }
280
	}
281

    
282
   * PostgreSQL
283

    
284
     DATABASES = {
285
	     'default': {
286
             'ENGINE': 'django.db.backends.postgresql_psycopg2',
287
             'NAME': 'DATABASE',
288
             'USER': 'USERNAME',
289
             'PASSWORD': 'PASSWORD',
290
             'HOST': 'HOST',
291
             'PORT': 'PORT',
292
	     }
293
     }
294

    
295
    Try it out. The following command will attempt to connect to the DB and
296
    print out DDL statements. It should not fail.
297

    
298
	$ ./bin/python manage.py sql db
299

    
300

    
301
7. Initialization of Synnefo DB:
302
   You need to initialize the Synnefo DB and load fixtures
303
   db/fixtures/{flavors,images}.json, which make the API usable by end users by
304
   defining a sample set of hardware configurations (flavors) and OS images.
305

    
306
     $ ./bin/python manage.py syncdb
307
     $ ./bin/python manage.py migrate db
308
     $ ./bin/python manage.py loaddata db/fixtures/flavors.json
309
     $ ./bin/python manage.py loaddata db/fixtures/images.json
310

    
311

    
312
8. Finalization of settings.py:
313
   Set the BACKEND_PREFIX_ID variable to some unique prefix, e.g. your commit
314
   username in settings.py. Several functional conventions within the system
315
   require this variable to include a dash at its end (e.g. snf-)
316

    
317
   Fix the AMQP-specific settings based on the selected BACKEND_PREFIX_ID.
318
   The fix_amqp_settings() function is called once at the end of
319
   settings.py.dist, you must call it again if you change BACKEND_PREFIX_ID at
320
   some later point.
321

    
322

    
323
9. Installation of the Ganeti monitoring daemon, /ganeti/snf-ganeti-eventd:
324
   The Ganeti monitoring daemon must run on GANETI-MASTER.
325
   The Ganeti monitoring daemon has no dependency on Django.
326

    
327
   Override all relevant settings in settings.d/99-snf-ganeti-eventd.conf,
328
   GANETI_* variables.
329
   Then, make sure PYTHONPATH contains the parent of the Django project,
330
   and start the server on the Ganeti master as root.
331

    
332
     root:~# export PYTHONPATH=$PYTHONPATH:/opt
333
     root:~# /opt/synnefo/ganeti/snf-ganeti-eventd.py
334

    
335
   TBD: how to handle master migration.
336

    
337

    
338
10. Installation of the Synnefo dispatcher, /logic/dispatcher.py:
339
    The logic dispatcher is part of the Synnefo Django project and must run
340
    on LOGIC nodes.
341

    
342
    The dispatcher retrieves messages from the queue and calls the appropriate
343
    handler function as defined in the queue configuration in `setttings.py'.
344
    The default configuration should work directly without any modifications.
345

    
346
    For the time being The dispatcher must be run by hand:
347
      $ ./bin/python ./logic/dispatcher.py
348

    
349
    The dispatcher should run in at least 2 instances to ensure high
350
    (actually, increased) availability.
351

    
352

    
353
11. Installation of the Synnefo Ganeti hook:
354
    The bash wrapper ganeti/snf-ganeti-hook is the generic launcher for
355
    Synnefo hooks in Ganeti. It resides in the ganeti/ directory under the
356
    root of the Synnefo Django project.
357

    
358
    The hook needs to be enabled for phases post-{add,modify,reboot,start,stop}
359
    by *symlinking* in
360
    /etc/ganeti/hooks/instance-{add,modify,reboot,start,stop}-post.d on
361
    GANETI-MASTER, e.g.:
362

    
363
    root@ganeti-master:/etc/ganeti/hooks/instance-start-post.d# ls -l
364
    lrwxrwxrwx 1 root root 45 May   3 13:45 00-snf-ganeti-hook -> /home/devel/synnefo/ganeti/snf-ganeti-hook
365

    
366
    IMPORTANT: The link name may only contain "upper and lower case, digits,
367
    underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9_-]+$."
368
    See:
369
     http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming
370

    
371
    You need to set SYNNEFO_PROJECT_DIR in ganeti/snf-ganeti-hook. The bash
372
    script modifies PYTHONPATH accordingly, before passing control to the
373
    relevant Python code.
374

    
375

    
376
12. Installation of the VNC authentication proxy, vncauthproxy:
377
    To support OOB console access to the VMs over VNC, the vncauthproxy
378
    daemon must be running on every node of type APISERVER.
379

    
380
    Download and install vncauthproxy from its own repository,
381
    at https://code.grnet.gr/git/vncauthproxy (known good commit: tag v1.0).
382

    
383
    Download and install a specific repository commit:
384

    
385
    $ bin/pip install -e git+https://code.grnet.gr/git/vncauthproxy@INSERT_COMMIT_HERE#egg=vncauthproxy
386

    
387
    Create /var/log/vncauthproxy and set its permissions appropriately.
388

    
389
    Alternatively, you can build Debian packages. To do so,
390
    checkout the "debian" branch of the vncauthproxy repository
391
    (known good commit: tag debian/v1.0):
392

    
393
    $ git checkout debian
394

    
395
    Then build debian package, and install as root:
396

    
397
    $ dpkg-buildpackage -b -uc -us
398
    # dpkg -i ../vncauthproxy_1.0-1_all.deb
399

    
400
    --Failure to build the package on the Mac.
401

    
402
    libevent, a requirement for gevent which in turn is a requirement for
403
    vncauthproxy is not included in MacOSX by default and installing it with
404
    MacPorts does not lead to a version that can be found by the gevent
405
    build process. A quick workaround is to execute the following commands:
406

    
407
    cd $SYNNEFO
408
    sudo pip install -e git+https://code.grnet.gr/git/vncauthproxy@5a196d8481e171a#egg=vncauthproxy
409
    <the above fails>
410
    cd build/gevent
411
    sudo python setup.py -I/opt/local/include -L/opt/local/lib build
412
    cd $SYNNEFO
413
    sudo pip install -e git+https://code.grnet.gr/git/vncauthproxy@5a196d8481e171a#egg=vncauthproxy
414

    
415
13. Installation of the customized Ganeti Instance Image for image deployment:
416
    For Synnefo to be able to launch VMs from specified Images, you need
417
    the gnt-instance-image OS Provider installed on the Ganeti backend.
418

    
419
    Download and install gnt-instance-image in all Ganeti nodes from its own
420
    repository, at https://code.grnet.gr/git/gnt-instance-image. It's
421
    recommended to use the win-support branch (known good commit:
422
    aa661a7ea6da9c11a05a3587ee61d10348eb7422).
423

    
424
    After installing gnt-instance-image do the following:
425
    1. root@ganeti-master$ cd /path-to-repo
426
       root@ganeti-master$ cp ./defaults /etc/default/ganeti-instance-image
427
    2. Uncomment the following in /etc/default/ganeti-instance-image:
428
         SWAP=no
429
         ARCH="x86_64"
430
    3. In /etc/ganeti/instance-image/hooks, make sure the hooks you want to
431
       run during instance creation process have execute permission.
432
       For linux you will need at lease `grub' and `root_passwd' to make the
433
       instance usable:
434
         chmod +x /etc/ganeti/instance-image/hooks/linux/{grub,root_passwd}
435
       For security reasons make sure `ssh' hook is also enabled.
436
       For windows you will need `mbr' and `admin_passwd':
437
         chmod +x /etc/ganeti/instance-image/hooks/windows/{mbr,admin_passwd}
438
       For both architectures it is also highly recommended to enable
439
       `hostname' hook too:
440
         chmod +x /et/ganeti/instance-image/hooks/{linux,windows}/hostname
441

    
442
    Your Custom Images should be stored in a dump format under
443
    /var/cache/ganeti-instance-image and their filenames should have the
444
    following format:
445
      {backend_id}-x86_64-root.dump
446
    e.g., debian-6.0.1a-x86_64-root.dump (backend_id = "debian-6.0.1a")
447

    
448
14. Setup Synnefo-specific networking on the Ganeti backend:
449
    This part is deployment-specific and must be customized based on the
450
    specific needs of the system administrators.
451

    
452
    A reference installation will use a Synnefo-specific KVM ifup script,
453
    NFDHCPD and pre-provisioned Linux bridges to support public and private
454
    network functionality. For this:
455

    
456
    Grab NFDHCPD from its own repository (https://code.grnet.gr/git/nfdhcpd),
457
    install it, modify /etc/nfdhcpd/nfdhcpd.conf to reflect your network
458
    configuration.
459

    
460
    Install a custom KVM ifup script for use by Ganeti, as
461
    /etc/ganeti/kvm-vif-bridge, on GANETI-NODEs. A sample implementation is
462
    provided under /contrib/ganeti-hooks. Set NFDHCPD_STATE_DIR to point
463
    to NFDHCPD's state directory, usually /var/lib/nfdhcpd.
464

    
465

    
466
15. Create log file directories for Synnefo components, set appropriate
467
    permissions. By default logic/dispatcher.py and ganeti/snf-ganeti-eventd.py
468
    use /var/log/synnefo.
469

    
470

    
471
16. (Hopefully) Done
472