Statistics
| Branch: | Tag: | Revision:

root / README.deploy @ 271baf11

History | View | Annotate | Download (15.8 kB)

1
README.deploy -- Instructions for a basic deployment of Synnefo v0.4
2

    
3
This document describes the basic steps to obtain a basic, working Synnefo
4
deployment. It begins by examining the different node roles, then moves to the
5
installation and setup of distinct software components.
6

    
7
It applies to Synnefo v0.4.
8

    
9

    
10
Node types
11
===========
12

    
13
Nodes in a Synnefo deployment belong in one of the following types:
14

    
15
 * DB:
16
   A node [or more than one nodes, if using an HA configuration], running a DB
17
   engine supported by the Django ORM layer. The DB is the single source of
18
   truth for the servicing of API requests by Synnefo.
19
   Services: PostgreSQL / MySQL
20

    
21
 * APISERVER:
22
   A node running the implementation of the OpenStack API, in Django. Any number
23
   of APISERVERs can be used, in a load-balancing configuration, without any
24
   special consideration. Access to a common DB ensures consistency.
25
   Services: Web server, vncauthproxy
26

    
27
 * QUEUE:
28
   A node running the RabbitMQ software, which provides AMQP functionality. More
29
   than one QUEUE nodes may be deployed, in an HA configuration. Such
30
   deployments require shared storage, provided e.g., by DRBD.
31
   Services: RabbitMQ [rabbitmq-server]
32

    
33
 * LOGIC:
34
   A node running the business logic of Synnefo, in Django. It dequeues
35
   messages from QUEUE nodes, and provides the context in which business logic
36
   functions run. It uses Django ORM to connect to the common DB and update the
37
   state of the system, based on notifications received from the rest of the
38
   infrastructure, over AMQP.
39
   Services: the Synnefo logic dispatcher [/logic/dispatcher.py]
40

    
41
 * GANETI-MASTER and GANETI-NODE:
42
   A single GANETI-MASTER and a large number of GANETI-NODEs constitute the
43
   Ganeti backend for Synnefo, which undertakes all VM management functions.
44
   Any APISERVER can issue commands to the GANETI-MASTER, over RAPI, to effect
45
   changes in the state of the VMs. The GANETI-MASTER runs the Ganeti request
46
   queue.
47
   Services:
48
     only on GANETI-MASTER:
49
       the Synnefo Ganeti monitoring daemon [/ganeti/ganeti-eventd]
50
       the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py].
51
     on each GANETI_NODE:
52
       a deployment-specific KVM ifup script
53
       properly configured NFDHCPD
54

    
55

    
56
Installation Process
57
=====================
58

    
59
This section describes the installation process of the various node roles in a
60
Synnefo deployment.
61

    
62

    
63
0. Allocation of physical nodes:
64
   Determine the role of every physical node in your deployment.
65

    
66

    
67
1. Ganeti installation:
68
   Synnefo requires a working Ganeti installation at the backend. Installation
69
   of Ganeti is not covered by this document, please refer to
70
   http://docs.ganeti.org/ganeti/current/html for all the gory details. A
71
   successful Ganeti installation concludes with a working GANETI-MASTER and a
72
   number of GANETI-NODEs.
73

    
74

    
75
2. RabbitMQ installation:
76
   RabbitMQ is used as a generic message broker for the system. It should be
77
   installed on two seperate QUEUE nodes (VMs should be enough for the moment)
78
   in a high availability configuration as described here:
79

    
80
     http://www.rabbitmq.com/pacemaker.html
81

    
82
   After installation, create a user and set its permissions
83
     rabbitmqctl add_user okeanos 0k3@n0s
84
     rabbitmqctl set_permissions -p / okeanos  "^.*" ".*" ".*"
85

    
86
   The values set for the user and password must be mirrored in the
87
   RABBIT_* variables in settings.py (see step 6)
88

    
89

    
90
3. Web server installation:
91
   A Web Server (e.g., Apache) needs to be installed on the APISERVERs,
92
   and be configured to run the Synnefo Django project appropriately. Selection
93
   and configuration of a Web server is outside the scope of this document.
94

    
95
   For testing or development purposes, Django's own development server,
96
   `./manage.py runserver' can be used.
97

    
98

    
99
4. Installation of the Synnefo Django project:
100
   As of v0.4, the Synnefo Django project needs to be installed on nodes of type
101
   APISERVER, LOGIC and on the GANETI-MASTER, with a properly configured
102
   settings.py. In later revisions, the specific parts of the Django project
103
   which need to run on each node type will be identified.
104

    
105
   Synnefo is written in Python 2.6 and depends on the following Python modules:
106
   [package versions confirmed to be compatible are in braces]
107

    
108
    * django 1.2 [Django==1.2.4]
109
    * simplejson [simplejson==2.1.3]
110
    * pycurl [pycurl==7.19.0]
111
    * python-dateutil  [python-dateutil==1.4.1]
112
      WARNING: version python-dateutil==2.0 downloaded by pip known *not* to
113
      work with Python 2.6
114
    * south [south==0.7.1]
115
      WARNING: not working with Debian squeeze's default south-0.7-1 package.
116
    * amqplib [amqplib==0.6.1]
117

    
118
   also, depending on the database engine of choice, on one of the following:
119
    * MySQL-python [MySQL-python==1.2.3]
120
    * psycopg2 [psycopg2==2.4]
121

    
122
   if the invitations application is deployed, the following dependencies should
123
   be installed:
124
    * pycrypto==2.1.0
125

    
126
   if you want to test the ganeti (FIXME)
127

    
128
   to run the user interface tests, selenium must be installed:
129
    * selenium [?]
130

    
131
   The easiest method for installation of the Django project is to setup a
132
   working environment through virtualenv. Alternatively, you can use your
133
   system's package manager to install the dependencies (e.g. Macports has them
134
   all).
135

    
136
   * On Snow Leopard and linux (64-bit), you have to set the following
137
     environment variable for pip to compile the dependencies correctly.
138

    
139
  	   $ export ARCHFLAGS="-arch x86_64"
140

    
141
   * On Ubuntu, a few more packages must be installed before installing the
142
     prerequisite Python libraries
143

    
144
	   $ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
145

    
146
   Checkout the code and install the Python prerequisites. This assumes that
147
   python is already installed on the host.
148

    
149
    $ sudo easy_install virtualenv
150
    $ git clone https://user@code.grnet.gr/git/synnefo synnefo
151
    $ virtualenv --python=python2.6 synnefo --no-site-packages
152
    ...
153
    $ cd synnefo
154
    $ ./bin/pip install <list_of_dependencies>
155

    
156

    
157
5. Database installation:
158
   A database supported by the Django ORM layer must be installed on nodes
159
   of type DB. The choices are: SQLIte, MySQL, PostgreSQL.
160

    
161
   * SQLite:
162
     The python sqlite driver is available by default with Python so no
163
     additional configuration is required. Also, most self-respecting systems
164
     have the sqlite library installed by default.
165

    
166
   * MySQL:
167
      MySQL must be installed first:
168

    
169
      * Ubuntu - Debian
170
	      $ sudo apt-get install libmysqlclient-dev
171

    
172
      * MacPorts
173
	      $ sudo port install mysql5
174

    
175
      Install the MySQL python library on servers running the Django project:
176

    
177
	    $ bin/pip install MySQL-python
178

    
179
      Note: On MacOSX with Mysql install from MacPorts the above command will
180
            fail complaining that it cannot find the mysql_config command. Do
181
            the following and restart the installation
182
	        $ echo "mysql_config = /opt/local/bin/mysql_config5" >> \
183
                                         ./build/MySQL-python/site.cfg
184

    
185
      Configure a MySQL db/account for synnefo
186
	    $ mysql -u root -p
187

    
188
    	mysql> create database synnefo;
189
	    mysql> show databases;
190
	    mysql> GRANT ALL on synnefo.* TO username IDENTIFIED BY 'password';
191

    
192
   * PostgreSQL
193
     You need to install the PostgreSQL binaries:
194
     * Ubuntu - Debian
195
	     $ sudo apt-get install postgresql-8.4 libpq-dev
196

    
197
     * MacPorts
198
	     $ sudo port install postgresql84
199

    
200
     Install the postgres Python library
201
	    $ bin/pip install psycopg2
202

    
203
     Configure a postgres db/account for synnefo:
204

    
205
     Become the postgres user, connect to PostgreSQL:
206
       $ sudo su - postgres
207
       $ psql
208
	
209
	 Run the following commands:
210
	   DROP DATABASE synnefo;
211
	   DROP USER username;
212
	   CREATE USER username WITH PASSWORD 'password';
213
	   CREATE DATABASE synnefo;
214
	   GRANT ALL PRIVILEGES ON DATABASE synnefo TO username;
215
	   ALTER DATABASE synnefo OWNER TO username;
216
	   ALTER USER username CREATEDB;
217

    
218
     The last line enables the newly created user to create own databases. This
219
     is needed for Django to create and drop the test_synnefo database for unit
220
     testing.
221

    
222

    
223
6. Setting up the Django project:
224
   The settings.py file for Django may be derived by concatenating the
225
   settings.py.dist file contained in the Synnefo distribution with a file
226
   containing custom modifications, which shall override all settings deviating
227
   from the supplied settings.py.dist. This is recommended to minimize the load
228
   of reconstructing settings.py from scratch, since each release currently
229
   brings heavy changes to settings.py.dist.
230

    
231
   Add the following to your custom settings.py, depending on your choice of DB:
232
   * SQLite
233

    
234
	 PROJECT_PATH = os.path.dirname(os.path.abspath(__file__)) + '/'
235

    
236
	 DATABASES = {
237
	     'default': {
238
		     'ENGINE': 'django.db.backends.sqlite3',
239
		     'NAME': PROJECT_PATH + 'synnefo.db' #WARN: This must be an absolute path
240
	     }
241
	 }
242

    
243
   * MySQL
244

    
245
 	 DATABASES = {
246
	     'default': {
247
             'ENGINE': 'django.db.backends.mysql',
248
             'NAME': 'synnefo',
249
             'USER': 'USERNAME',
250
             'PASSWORD': 'PASSWORD',
251
             'HOST': 'HOST',
252
             'PORT': 'PORT',
253
             'OPTIONS': {
254
                 'init_command': 'SET storage_engine=INNODB',
255
             }
256
	    }
257
	}
258

    
259
   * PostgreSQL
260

    
261
     DATABASES = {
262
	     'default': {
263
             'ENGINE': 'django.db.backends.postgresql_psycopg2',
264
             'NAME': 'DATABASE',
265
             'USER': 'USERNAME',
266
             'PASSWORD': 'PASSWORD',
267
             'HOST': 'HOST',
268
             'PORT': 'PORT',
269
	     }
270
     }
271

    
272
    Try it out. The following command will attempt to connect to the DB and
273
    print out DDL statements. It should not fail.
274

    
275
	$ ./bin/python manage.py sql db
276

    
277

    
278
7. Initialization of Synnefo DB:
279
   You need to initialize the Synnefo DB and load fixtures
280
   db/fixtures/{flavors,images}.json, which make the API usable by end users by
281
   defining a sample set of hardware configurations (flavors) and OS images.
282

    
283
     $ ./bin/python manage.py syncdb
284
     $ ./bin/python manage.py migrate db
285
     $ ./bin/python manage.py loaddata db/fixtures/flavors.json
286
     $ ./bin/python manage.py loaddata db/fixtures/images.json
287

    
288

    
289
8. Finalization of settings.py:
290
   Set the BACKEND_PREFIX_ID variable to some unique prefix, e.g. your commit
291
   username in settings.py. Several functional conventions within the system
292
   require this variable to include a dash at its end (e.g. snf-)
293

    
294
   Fix the AMQP-specific settings based on the selected BACKEND_PREFIX_ID.
295
   The fix_amqp_settings() function is called once at the end of
296
   settings.py.dist, you must call it again if you change BACKEND_PREFIX_ID at
297
   some later point.
298

    
299

    
300
9. Installation of the Ganeti monitoring daemon, /ganeti/ganeti-eventd:
301
   The Ganeti monitoring daemon is part of the Django project and must run on
302
   GANETI-MASTER.
303

    
304
   Override all relevant settings in settings.py.dist, GANETI_* variables.
305
   Then start the server on the Ganeti master as root
306

    
307
     # cd synnefo && python ./ganeti/ganeti-eventd
308

    
309
   FIXME: The server must be started from the project root directory.
310
   TBD: how to handle master migration.
311

    
312

    
313
10. Installation of the Synnefo dispatcher, /logic/dispatcher.py:
314
    The logic dispatcher is part of the Synnefo Django project and must run
315
    on LOGIC nodes.
316

    
317
    The dispatcher retrieves messages from the queue and calls the appropriate
318
    handler function as defined in the queue configuration in `setttings.py'.
319
    The default configuration should work directly without any modifications.
320

    
321
    For the time being The dispatcher must be run by hand:
322
      $ ./bin/python ./logic/dispatcher.py
323

    
324
    The dispatcher should run in at least 2 instances to ensure high
325
    (actually, increased) availability.
326

    
327

    
328
11. Installation of the Synnefo Ganeti hook:
329
    The Python script ganeti/snf-ganeti-hook.py is the generic launcher for
330
    Synnefo hooks in Ganeti.  It resides in the ganeti/ directory under the root
331
    of Synnefo Django project root.
332

    
333
    The hook needs to be enabled for phases post-{add,modify,reboot,start,stop}
334
    by *symlinking* in
335
    /etc/ganeti/hooks/instance-{add,modify,reboot,start,stop}-post.d on
336
    GANETI-MASTER, e.g.:
337

    
338
    root@ganeti-master:/etc/ganeti/hooks/instance-start-post.d# ls -l
339
    lrwxrwxrwx 1 root root 45 May   3 13:45 00-snf-ganeti-hook -> /home/devel/synnefo/ganeti/snf-ganeti-hook.py*
340

    
341
    IMPORTANT: The link name may only contain "upper and lower case, digits,
342
    underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9_-]+$."
343
    See:
344
     http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming
345

    
346
    The script uses the location of the link target to determine the Synnefo
347
    Project root, before passing control to the relevant Python code.
348

    
349
    FIXME: Perhaps require a SYNNEFO_PROJECT_ROOT environment variable?
350

    
351

    
352
12. Installation of the VNC authentication proxy, vncauthproxy:
353
    To support OOB console access to the VMs over VNC, the vncauthproxy
354
    daemon must be running on every node of type APISERVER.
355

    
356
    Download and install vncauthproxy from its own repository,
357
    at https://code.grnet.gr/git/vncauthproxy (known good commit: 8799ab6d6e).
358

    
359
    Edit default settings on top of vncauthproxy.py.
360
    Set CTRL_SOCKET in util/vapclient.py to point to its control socket.
361

    
362
    FIXME: The CTRL_SOCKET setting will be moved to settings.py as
363
    VNCAUTHPROXY_CTRL_SOCKET.
364

    
365
    Create /var/log/vncauthproxy and set its permissions appropriately.
366

    
367

    
368
13. Installation of the customized Ganeti Instance Image for image deployment:
369
    For Synnefo to be able to launch VMs from specified Images, you need
370
    the gnt-instance-image OS Provider installed on the Ganeti backend.
371

    
372
    Download and install gnt-instance-image in all Ganeti nodes from its own
373
    repository, at https://code.grnet.gr/git/gnt-instance-image.
374

    
375
    After installing gnt-instance-image do the following:
376
    1. root@ganeti-master$ cd /path-to-repocp
377
       root@ganeti-master$ ./defaults /etc/default/ganeti-instance-image
378
    2. Uncomment the following in /etc/default/ganeti-instance-image:
379
         SWAP=no
380
         ARCH="x86_64"
381
    3. Add to /etc/default/ganeti-instance-image (so that make-dump makes no
382
       /boot partition):
383
         KERNEL_PATH="True"
384
    4. Change the path in make-dump line 22 according to your installation
385
       (/usr/share/ganeti/os/image/common.sh --> /srv/ganeti/os/image/common.sh)
386
    5. In common.sh, comment out the KERNEL_PATH variable initialization:
387
         #KERNEL_PATH="$INSTANCE_HV_kernel_path"
388
    6. In /etc/ganeti/instance-image/hooks, make sure the hooks you want to
389
       run during instance creation process have execute permission. At least
390
       `grub' and `root_passwd' should be triggered to make the instance usable:
391
         chmod +x /etc/ganeti/instance-image/hooks/{grub,root_passwd}
392

    
393
    Your Custom Images should be stored in a dump format under
394
    /var/cache/ganeti-instance-image and their filenames should have the
395
    following format:
396
      {backend_id}-x86_64-root.dump
397
    e.g., debian-6.0.1a-x86_64-root.dump (backend_id = "debian-6.0.1a")
398

    
399

    
400
14. Setup Synnefo-specific networking on the Ganeti backend:
401
    This part is deployment-specific and must be customized based on the
402
    specific needs of the system administrators.
403

    
404
    A reference installation will use a Synnefo-specific KVM ifup script,
405
    NFDHCPD and pre-provisioned Linux bridges to support public and private
406
    network functionality. For this:
407

    
408
    Grab NFDHCPD from its own repository (https://code.grnet.gr/git/nfdhcpd),
409
    install it, modify /etc/nfdhcpd/nfdhcpd.conf to reflect your network
410
    configuration.
411

    
412
    Install a custom KVM ifup script for use by Ganeti, as
413
    /etc/ganeti/kvm-vif-bridge, on GANETI-NODEs. A sample implementation is
414
    provided under /contrib/ganeti-hooks. Set NFDHCPD_STATE_DIR to point
415
    to NFDHCPD's state directory, usually /var/lib/nfdhcpd.
416

    
417

    
418
15. Create log file directories for Synnefo components, set appropriate
419
    permissions. By default, logic/dispatcher.py and ganeti/ganeti-eventd.py
420
    use /var/log/synnefo.
421

    
422

    
423
16. (Hopefully) Done
424