Statistics
| Branch: | Tag: | Revision:

root / README.deploy @ 3d9f561d

History | View | Annotate | Download (19.3 kB)

1
README.deploy -- Instructions for a basic Synnefo deployment
2

    
3
This document describes the basic steps to obtain a basic, working Synnefo
4
deployment. It begins by examining the different node roles, then moves to the
5
installation and setup of distinct software components.
6

    
7
It is current as of Synnefo v0.5.2.
8

    
9

    
10
Node types
11
===========
12

    
13
Nodes in a Synnefo deployment belong in one of the following types:
14

    
15
 * DB:
16
   A node [or more than one nodes, if using an HA configuration], running a DB
17
   engine supported by the Django ORM layer. The DB is the single source of
18
   truth for the servicing of API requests by Synnefo.
19
   Services: PostgreSQL / MySQL
20

    
21
 * APISERVER:
22
   A node running the implementation of the OpenStack API, in Django. Any number
23
   of APISERVERs can be used, in a load-balancing configuration, without any
24
   special consideration. Access to a common DB ensures consistency.
25
   Services: Web server, vncauthproxy
26

    
27
 * QUEUE:
28
   A node running the RabbitMQ software, which provides AMQP functionality. More
29
   than one QUEUE nodes may be deployed, in an HA configuration. Such
30
   deployments require shared storage, provided e.g., by DRBD.
31
   Services: RabbitMQ [rabbitmq-server]
32

    
33
 * LOGIC:
34
   A node running the business logic of Synnefo, in Django. It dequeues
35
   messages from QUEUE nodes, and provides the context in which business logic
36
   functions run. It uses Django ORM to connect to the common DB and update the
37
   state of the system, based on notifications received from the rest of the
38
   infrastructure, over AMQP.
39
   Services: the Synnefo logic dispatcher [/logic/dispatcher.py]
40

    
41
 * GANETI-MASTER and GANETI-NODE:
42
   A single GANETI-MASTER and a large number of GANETI-NODEs constitute the
43
   Ganeti backend for Synnefo, which undertakes all VM management functions.
44
   Any APISERVER can issue commands to the GANETI-MASTER, over RAPI, to effect
45
   changes in the state of the VMs. The GANETI-MASTER runs the Ganeti request
46
   queue.
47
   Services:
48
     only on GANETI-MASTER:
49
       the Synnefo Ganeti monitoring daemon [/ganeti/snf-ganeti-eventd]
50
       the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py].
51
     on each GANETI_NODE:
52
       a deployment-specific KVM ifup script
53
       properly configured NFDHCPD
54

    
55

    
56
Installation Process
57
=====================
58

    
59
This section describes the installation process of the various node roles in a
60
Synnefo deployment.
61

    
62

    
63
0. Allocation of physical nodes:
64
   Determine the role of every physical node in your deployment.
65

    
66

    
67
1. Ganeti installation:
68
   Synnefo requires a working Ganeti installation at the backend. Installation
69
   of Ganeti is not covered by this document, please refer to
70
   http://docs.ganeti.org/ganeti/current/html for all the gory details. A
71
   successful Ganeti installation concludes with a working GANETI-MASTER and a
72
   number of GANETI-NODEs.
73

    
74

    
75
2. RabbitMQ installation:
76
   RabbitMQ is used as a generic message broker for the system. It should be
77
   installed on two seperate QUEUE nodes (VMs should be enough for the moment)
78
   in a high availability configuration as described here:
79

    
80
     http://www.rabbitmq.com/pacemaker.html
81

    
82
   After installation, create a user and set its permissions
83
     rabbitmqctl add_user okeanos 0k3@n0s
84
     rabbitmqctl set_permissions -p / okeanos  "^.*" ".*" ".*"
85

    
86
   The values set for the user and password must be mirrored in the
87
   RABBIT_* variables in settings.py (see step 6)
88

    
89

    
90
3. Web server installation:
91
   A Web Server (e.g., Apache) needs to be installed on the APISERVERs,
92
   and be configured to run the Synnefo Django project appropriately. Selection
93
   and configuration of a Web server is outside the scope of this document.
94

    
95
   For testing or development purposes, Django's own development server,
96
   `./manage.py runserver' can be used.
97

    
98

    
99
4. Installation of the Synnefo Django project:
100
   As of v0.5 the Synnefo Django project needs to be installed on nodes
101
   of type APISERVER, and LOGIC, with a properly configured settings.py. In
102
   later revisions, the specific parts of the Django project which need to run
103
   on each node type will be identified.
104

    
105
   Synnefo is written in Python 2.6 and depends on the following Python modules:
106
   [package versions confirmed to be compatible are in braces]
107

    
108
    * django 1.2 [Django==1.2.4]
109
    * simplejson [simplejson==2.1.3]
110
    * pycurl [pycurl==7.19.0]
111
    * python-dateutil  [python-dateutil==1.4.1]
112
      WARNING: version python-dateutil==2.0 downloaded by pip known *not* to
113
               work with Python 2.6
114
    * python-ipy [IPy==0.75]
115
        also verified to work with python-ipy 0.70-1 as shipped with Squeeze
116
    * south [south==0.7.1]
117
      WARNING: might not work with Debian Squeeze's default south-0.7-1 package.
118
    * amqplib [amqplib==0.6.1]
119
    * lockfile [lockfile==0.8]
120
    * python-daemon [python-daemon==1.5.5]
121
    * python-prctl [python-prctl==1.3.0]
122

    
123
   also, depending on the database engine of choice, on one of the following:
124
    * MySQL-python [MySQL-python==1.2.3]
125
    * psycopg2 [psycopg2==2.4]
126

    
127
   if the invitations application is deployed, the following dependencies should
128
   be installed:
129
    * pycrypto==2.1.0
130

    
131
   To run the user interface tests, selenium must be installed
132
    * selenium [?]
133

    
134
   The easiest method for installation of the Django project is to setup a
135
   working environment through virtualenv. Alternatively, you can use your
136
   system's package manager to install the dependencies (e.g. Macports has them
137
   all).
138

    
139
   * On Snow Leopard and linux (64-bit), you have to set the following
140
     environment variable for pip to compile the dependencies correctly.
141

    
142
  	   $ export ARCHFLAGS="-arch x86_64"
143

    
144
   * On Ubuntu, a few more packages must be installed before installing the
145
     prerequisite Python libraries
146

    
147
	   $ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
148

    
149
   Checkout the code and install the Python prerequisites. This assumes that
150
   python is already installed on the host.
151

    
152
    $ sudo easy_install virtualenv
153
    $ git clone https://user@code.grnet.gr/git/synnefo synnefo
154
    $ virtualenv --python=python2.6 synnefo --no-site-packages
155
    ...
156
    $ cd synnefo
157
    $ ./bin/pip install <list_of_dependencies>
158

    
159
    [WARNING]: The software must be checked out in a directory named synnefo,
160
    otherwise python imports will not work. Therefore, do not change the
161
    or rename the checkout path.
162

    
163

    
164
5. Database installation:
165
   A database supported by the Django ORM layer must be installed on nodes
166
   of type DB. The choices are: SQLIte, MySQL, PostgreSQL.
167

    
168
   * SQLite:
169
     The python sqlite driver is available by default with Python so no
170
     additional configuration is required. Also, most self-respecting systems
171
     have the sqlite library installed by default.
172

    
173
   * MySQL:
174
      MySQL must be installed first:
175

    
176
      * Ubuntu - Debian
177
	      $ sudo apt-get install libmysqlclient-dev
178

    
179
      * MacPorts
180
	      $ sudo port install mysql5
181

    
182
      Install the MySQL python library on servers running the Django project:
183

    
184
	    $ bin/pip install MySQL-python
185

    
186
      Note: On MacOSX with Mysql install from MacPorts the above command will
187
            fail complaining that it cannot find the mysql_config command. Do
188
            the following and restart the installation
189
	        $ echo "mysql_config = /opt/local/bin/mysql_config5" >> \
190
                                         ./build/MySQL-python/site.cfg
191

    
192
      Configure a MySQL db/account for synnefo
193
	    $ mysql -u root -p
194

    
195
    	mysql> create database synnefo;
196
	    mysql> show databases;
197
	    mysql> GRANT ALL on synnefo.* TO username IDENTIFIED BY 'password';
198

    
199
     IMPORTANT:
200
        MySQL *must* be set in READ-COMMITED mode, e.g. by setting
201

    
202
        transaction-isolation = READ-COMMITTED
203

    
204
        in the [mysqld] section of /etc/mysql/my.cnf.
205

    
206
        Alternatively, make sure the following code fragment stays enabled
207
        in settings.d/10-database.conf:
208

    
209
            if DATABASES['default']['ENGINE'].endswith('mysql'):
210
                DATABASES['default']['OPTIONS'] = {
211
                        'init_command': 'SET storage_engine=INNODB; ' +
212
                            'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
213
                }
214
          
215
   * PostgreSQL
216
     You need to install the PostgreSQL binaries:
217
     * Ubuntu - Debian
218
	     $ sudo apt-get install postgresql-8.4 libpq-dev
219

    
220
     * MacPorts
221
	     $ sudo port install postgresql84
222

    
223
     Install the postgres Python library
224
	    $ bin/pip install psycopg2
225

    
226
     Configure a postgres db/account for synnefo:
227

    
228
     Become the postgres user, connect to PostgreSQL:
229
       $ sudo su - postgres
230
       $ psql
231
	
232
	 Run the following commands:
233
	   DROP DATABASE synnefo;
234
	   DROP USER username;
235
	   CREATE USER username WITH PASSWORD 'password';
236
	   CREATE DATABASE synnefo;
237
	   GRANT ALL PRIVILEGES ON DATABASE synnefo TO username;
238
	   ALTER DATABASE synnefo OWNER TO username;
239
	   ALTER USER username CREATEDB;
240

    
241
     The last line enables the newly created user to create own databases. This
242
     is needed for Django to create and drop the test_synnefo database for unit
243
     testing.
244

    
245

    
246
6. Setting up the Django project:
247
   The settings.py file for Django may be derived by concatenating the
248
   settings.py.dist file contained in the Synnefo distribution with a file
249
   containing custom modifications, which shall override all settings deviating
250
   from the supplied settings.py.dist. This is recommended to minimize the load
251
   of reconstructing settings.py from scratch, since each release currently
252
   brings heavy changes to settings.py.dist.
253

    
254
   Add the following to your custom settings.py, depending on your choice
255
   of DB:
256
   * SQLite
257

    
258
	 PROJECT_PATH = os.path.dirname(os.path.abspath(__file__)) + '/'
259

    
260
	 DATABASES = {
261
	     'default': {
262
		     'ENGINE': 'django.db.backends.sqlite3',
263
		     'NAME': PROJECT_PATH + 'synnefo.db' # WARN: This must be an absolute path
264
	     }
265
	 }
266

    
267
   * MySQL
268

    
269
 	 DATABASES = {
270
	     'default': {
271
             'ENGINE': 'django.db.backends.mysql',
272
             'NAME': 'synnefo',
273
             'USER': 'USERNAME',
274
             'PASSWORD': 'PASSWORD',
275
             'HOST': 'HOST',
276
             'PORT': 'PORT',
277
             'OPTIONS': {
278
                 'init_command': 'SET storage_engine=INNODB',
279
             }
280
	    }
281
	}
282

    
283
   * PostgreSQL
284

    
285
     DATABASES = {
286
	     'default': {
287
             'ENGINE': 'django.db.backends.postgresql_psycopg2',
288
             'NAME': 'DATABASE',
289
             'USER': 'USERNAME',
290
             'PASSWORD': 'PASSWORD',
291
             'HOST': 'HOST',
292
             'PORT': 'PORT',
293
	     }
294
     }
295

    
296
    Try it out. The following command will attempt to connect to the DB and
297
    print out DDL statements. It should not fail.
298

    
299
	$ ./bin/python manage.py sql db
300

    
301

    
302
7. Initialization of Synnefo DB:
303
   You need to initialize the Synnefo DB and load fixtures
304
   db/fixtures/{users,flavors,images}.json, which make the API usable by end
305
   users by defining a sample set of users, hardware configurations (flavors)
306
   and OS images.
307

    
308
   IMPORTANT: Be sure to modify db/fixtures/users.json and select
309
   a unique token for each of the initial and any other users defined in this
310
   file. DO NOT LEAVE THE SAMPLE AUTHENTICATION TOKENS enabled in deployed
311
   configurations.
312

    
313
     $ ./bin/python manage.py syncdb
314
     $ ./bin/python manage.py migrate db
315
     $ ./bin/python manage.py loaddata db/fixtures/users.json
316
     $ ./bin/python manage.py loaddata db/fixtures/flavors.json
317
     $ ./bin/python manage.py loaddata db/fixtures/images.json
318

    
319

    
320
8. Finalization of settings.py:
321
   Set the BACKEND_PREFIX_ID variable to some unique prefix, e.g. your commit
322
   username in settings.py. Several functional conventions within the system
323
   require this variable to include a dash at its end (e.g. snf-)
324

    
325

    
326
9. Installation of the Ganeti monitoring daemon, /ganeti/snf-ganeti-eventd:
327
   The Ganeti monitoring daemon must run on GANETI-MASTER.
328

    
329
   The monitoring daemon is configured through /etc/synnefo/settings.conf.
330
   An example is provided under snf-ganeti-tools/.
331

    
332
   If run from the repository directory, make sure to have snf-ganeti-tools/
333
   in the PYTHONPATH.
334

    
335
   You may also build Debian packages directly from the repository:
336
   $ cd snf-ganeti-tools
337
   $ dpkg-buildpackage -b -uc -us
338
   # dpkg -i ../snf-ganeti-tools-*deb
339

    
340
   TBD: how to handle master migration.
341

    
342

    
343
10. Installation of the Synnefo dispatcher, /logic/dispatcher.py:
344
    The logic dispatcher is part of the Synnefo Django project and must run
345
    on LOGIC nodes.
346

    
347
    The dispatcher retrieves messages from the queue and calls the appropriate
348
    handler function as defined in the queue configuration in `setttings.py'.
349
    The default configuration should work directly without any modifications.
350

    
351
    For the time being The dispatcher must be run by hand:
352
      $ ./bin/python ./logic/dispatcher.py
353

    
354
    The dispatcher should run in at least 2 instances to ensure high
355
    (actually, increased) availability.
356

    
357

    
358
11. Installation of the Synnefo Ganeti hook:
359
    The generic Synnefo Ganeti hook wrapper resides in the snf-ganeti-tools/
360
    directory of the Synnefo repository.
361

    
362
    The hook needs to be enabled for phases post-{add,modify,reboot,start,stop}
363
    by *symlinking* in
364
    /etc/ganeti/hooks/instance-{add,modify,reboot,start,stop}-post.d on
365
    GANETI-MASTER, e.g.:
366

    
367
    root@ganeti-master:/etc/ganeti/hooks/instance-start-post.d# ls -l
368
    lrwxrwxrwx 1 root root 45 May   3 13:45 00-snf-ganeti-hook -> /home/devel/synnefo/snf-ganeti-hook/snf-ganeti-hook.py
369

    
370
    IMPORTANT: The link name may only contain "upper and lower case, digits,
371
    underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9_-]+$."
372
    See:
373
    http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming
374

    
375
    If run from the repository directory, make sure to have snf-ganeti-tools/
376
    in the PYTHONPATH.
377

    
378
    Alternative, build Debian packages which take care of building, installing
379
    and activating the Ganeti hook automatically, see step. 9.
380

    
381

    
382
12. Installation of the VNC authentication proxy, vncauthproxy:
383
    To support OOB console access to the VMs over VNC, the vncauthproxy
384
    daemon must be running on every node of type APISERVER.
385

    
386
    Download and install vncauthproxy from its own repository,
387
    at https://code.grnet.gr/git/vncauthproxy (known good commit: tag v1.0).
388

    
389
    Download and install a specific repository commit:
390

    
391
    $ bin/pip install -e git+https://code.grnet.gr/git/vncauthproxy@INSERT_COMMIT_HERE#egg=vncauthproxy
392

    
393
    Create /var/log/vncauthproxy and set its permissions appropriately.
394

    
395
    Alternatively, you can build Debian packages. To do so,
396
    checkout the "debian" branch of the vncauthproxy repository
397
    (known good commit: tag debian/v1.0):
398

    
399
    $ git checkout debian
400

    
401
    Then build debian package, and install as root:
402

    
403
    $ dpkg-buildpackage -b -uc -us
404
    # dpkg -i ../vncauthproxy_1.0-1_all.deb
405

    
406
    --Failure to build the package on the Mac.
407

    
408
    libevent, a requirement for gevent which in turn is a requirement for
409
    vncauthproxy is not included in MacOSX by default and installing it with
410
    MacPorts does not lead to a version that can be found by the gevent
411
    build process. A quick workaround is to execute the following commands:
412

    
413
    cd $SYNNEFO
414
    sudo pip install -e git+https://code.grnet.gr/git/vncauthproxy@5a196d8481e171a#egg=vncauthproxy
415
    <the above fails>
416
    cd build/gevent
417
    sudo python setup.py -I/opt/local/include -L/opt/local/lib build
418
    cd $SYNNEFO
419
    sudo pip install -e git+https://code.grnet.gr/git/vncauthproxy@5a196d8481e171a#egg=vncauthproxy
420

    
421

    
422
13. Installation of the customized Ganeti Instance Image for image deployment:
423
    For Synnefo to be able to launch VMs from specified Images, you need
424
    the gnt-instance-image OS Provider installed on *all* Ganeti nodes.
425
    There are 2 different ways to install gnt-instance-image on a node:
426

    
427
    1.As a debian package (recommended)
428
        
429
        Download and install the debian package found here:
430
        https://code.grnet.gr/attachments/download/351/ganeti-instance-image_0.5.1-1-snf1_all.deb
431
        You can do this by running the following:
432
        
433
        $ wget https://code.grnet.gr/attachments/download/351/ganeti-instance-image_0.5.1-1-snf1_all.deb
434
        # dpkg -i ganeti-instance-image_0.5.1-1-snf1_all.deb
435

    
436
        Make any additional configuration changes in 
437
        /etc/default/ganeti-instance-image.
438
        It is recommended to change the default IMAGE_DIR from 
439
        /var/cache/ganeti-instance-image to something like /srv/example_images_repo
440

    
441
	Be sure to have all the package's needed dependencies installed 
442
        on your system.
443

    
444
    2.From source    
445

    
446
        Download and install gnt-instance-image from its own repository,
447
        at https://code.grnet.gr/git/gnt-instance-image. It's
448
        recommended to use the win-support branch (known good commit:
449
        a7fb0e606623e7ac10d06ac935b0afe7f03d496c).
450

    
451
        Make sure to enable progress monitoring, using the --with-progress-monitor
452
        argument to configure. This requires the snf-progress-monitor tool,
453
        provided in snf-ganeti-tools/ and also as part of the snf-ganeti-tools
454
        Debian package.
455

    
456
        After installing gnt-instance-image do the following:
457
        1. $ cd /path-to-repo
458
           # cp ./defaults /etc/default/ganeti-instance-image
459
        2. In /etc/ganeti/instance-image/hooks, make sure the hooks you want to
460
           run during the instance creation process have execute permission.
461
           For linux you will need at lease `grub' and `root_passwd' to make the
462
           instance usable:
463
           chmod +x /etc/ganeti/instance-image/hooks/linux/{grub,root_passwd}
464
           For security reasons make sure `ssh' hook is also enabled.
465
           For windows you will need `mbr' and `admin_passwd':
466
           chmod +x /etc/ganeti/instance-image/hooks/windows/{mbr,admin_passwd}
467
           For both architectures it is also highly recommended to enable the
468
           `hostname' hook too:
469
           chmod +x /et/ganeti/instance-image/hooks/{linux,windows}/hostname
470

    
471
    Your custom Images should be stored in a dump format under
472
    /var/cache/ganeti-instance-image (default) or a different directory of your
473
    choice, accordingly set in /etc/default/ganeti-instance-image. The latter
474
    is recommended. Their filenames should have the following format:
475
      {backend_id}-x86_64-root.dump
476
    e.g., debian-6.0.1a-x86_64-root.dump (backend_id = "debian-6.0.1a")
477

    
478

    
479
14. Setup Synnefo-specific networking on the Ganeti backend:
480
    This part is deployment-specific and must be customized based on the
481
    specific needs of the system administrators.
482

    
483
    A reference installation will use a Synnefo-specific KVM ifup script,
484
    NFDHCPD and pre-provisioned Linux bridges to support public and private
485
    network functionality. For this:
486

    
487
    Grab NFDHCPD from its own repository (https://code.grnet.gr/git/nfdhcpd),
488
    install it, modify /etc/nfdhcpd/nfdhcpd.conf to reflect your network
489
    configuration.
490

    
491
    Install a custom KVM ifup script for use by Ganeti, as
492
    /etc/ganeti/kvm-vif-bridge, on GANETI-NODEs. A sample implementation is
493
    provided under /contrib/ganeti-hooks. Set NFDHCPD_STATE_DIR to point
494
    to NFDHCPD's state directory, usually /var/lib/nfdhcpd.
495

    
496

    
497
15. See section "Logging" in README.admin, and edit logic/logging.conf
498
    according to your OS and individual deployment characteristics.
499

    
500

    
501
16. Optionally, read the okeanos_site/README file to setup ~okeanos introductory 
502
    site (intro, video/info pages). Please see okeanos_site/90-okeanos.sample
503
    for a sample configuration file which overrides site-specific variables,
504
    to be placed under settings.d/, after customization.
505

    
506

    
507
17. (Hopefully) Done
508

    
509