Statistics
| Branch: | Tag: | Revision:

root / README.deploy @ 33f3103d

History | View | Annotate | Download (16 kB)

1
README.deploy -- Instructions for a basic deployment of Synnefo v0.4
2

    
3
This document describes the basic steps to obtain a basic, working Synnefo
4
deployment. It begins by examining the different node roles, then moves to the
5
installation and setup of distinct software components.
6

    
7
It applies to Synnefo v0.4.
8

    
9

    
10
Node types
11
===========
12

    
13
Nodes in a Synnefo deployment belong in one of the following types:
14

    
15
 * DB:
16
   A node [or more than one nodes, if using an HA configuration],
17
   running a DB engine supported by the Django ORM layer. The DB
18
   is the single source of truth for the servicing of API requests by
19
   Synnefo.
20
   Services: PostgreSQL / MySQL
21

    
22
 * APISERVER:
23
   A node running the implementation of the OpenStack API, in Django.
24
   Any number of APISERVERs can be used, in a load-balancing configuration,
25
   without any special consideration. Access to a common DB ensures
26
   consistency.
27
   Services: Web server, vncauthproxy
28

    
29
 * QUEUE:
30
   A node running the RabbitMQ software, which provides AMQP functionality.
31
   More than one QUEUE nodes may be deployed, in an HA configuration. Such
32
   deployments require shared storage, provided e.g., by DRBD.
33
   Services: RabbitMQ [rabbitmq-server]
34

    
35
 * LOGIC:
36
   A node running the business logic of Synnefo, in Django. It dequeues
37
   messages from QUEUE nodes, and provides the context in which business
38
   logic functions run. It uses Django ORM to connect to the common DB and
39
   update the state of the system, based on notifications received from the
40
   rest of the infrastructure, over AMQP.
41
   Services: the Synnefo logic dispatcher [/logic/dispatcher.py]
42

    
43
 * GANETI-MASTER and GANETI-NODE:
44
   A single GANETI-MASTER and a large number of GANETI-NODEs constitute the
45
   Ganeti backend for Synnefo, which undertakes all VM management functions.
46
   Any APISERVER can issue commands to the GANETI-MASTER, over RAPI, to effect
47
   changes in the state of the VMs. The GANETI-MASTER runs the Ganeti request
48
   queue.
49
   Services:
50
   only on GANETI-MASTER:
51
     the Synnefo Ganeti monitoring daemon [/ganeti/snf-ganeti-eventd],
52
     and the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py].
53
   on each GANETI_NODE:
54
     a deployment-specific KVM ifup script
55
     properly configured NFDHCPD
56

    
57

    
58
Installation Process
59
=====================
60

    
61
This section describes the installation process of the various node roles
62
in a Synnefo deployment.
63

    
64

    
65
0. Allocation of physical nodes:
66
   Determine the role of every physical node in your deployment.
67

    
68

    
69
1. Ganeti installation:
70
   Synnefo requires a working Ganeti installation at the backend. Installation
71
   of Ganeti is not covered by this document, please refer to
72
   http://docs.ganeti.org/ganeti/current/html for all the gory details. A
73
   successful Ganeti installation concludes with a working GANETI-MASTER and a
74
   number of GANETI-NODEs.
75

    
76

    
77
2. RabbitMQ installation:
78
   RabbitMQ is used as a generic message broker for the system. It should
79
   be installed on two seperate QUEUE nodes (VMs should be enough for the
80
   moment) in a high availability configuration as described here:
81

    
82
     http://www.rabbitmq.com/pacemaker.html
83

    
84
   After installation, create a user and set its permissions
85
     rabbitmqctl add_user okeanos 0k3@n0s
86
     rabbitmqctl set_permissions -p / okeanos  "^.*" ".*" ".*"
87

    
88
   The values set for the user and password must be mirrored in the
89
   RABBIT_* variables in settings.py (see step 6)
90

    
91

    
92
3. Web server installation:
93
   A Web Server (e.g., Apache) needs to be installed on the APISERVERs,
94
   and be configured to run the Synnefo Django project appropriately.
95
   Selection and configuration of a Web server is outside the scope of this
96
   document.
97

    
98
   For testing or development purposes, Django's own development server,
99
   ./manage.py runserver can be used.
100

    
101

    
102
4. Installation of the Synnefo Django project:
103
   As of v0.4, the Synnefo Django project needs to be installed on nodes
104
   of type APISERVER, LOGIC and on the GANETI-MASTER, with a properly
105
   configured settings.py. In later revisions, the specific parts of the Django
106
   project which need to run on each node type will be identified.
107

    
108
   Synnefo is written in Python 2.6 and depends on the following Python
109
   modules: [package versions confirmed to be compatible are in braces]
110

    
111
    * django 1.2 [Django==1.2.4]
112
    * simplejson [simplejson==2.1.3]
113
    * pycurl [pycurl==7.19.0]
114
    * python-dateutil  [python-dateutil==1.4.1]
115
      WARNING: version python-dateutil==2.0 downloaded by pip known *not* to
116
               work with Python 2.6
117
    * python-ipy [IPy==0.72]
118
    * south [south==0.7.1]
119
      WARNING: might not work with Debian squeeze's default south-0.7-1 package.
120
    * amqplib [amqplib==0.6.1]
121
    * lockfile [lockfile==0.8]
122
    * python-daemon [python-daemon==1.5.5]
123

    
124
   also, depending on the database engine of choice, on one of the following:
125
    * MySQL-python [MySQL-python==1.2.3]
126
    * psycopg2 [psycopg2==2.4]
127

    
128
   if the invitations application is deployed, the following
129
   dependencies should be installed:
130

    
131
    * pycrypto==2.1.0
132

    
133
   To run the user interface tests, selenium must be installed
134
    * selenium [?]
135

    
136
   The easiest method for installation of the Django project is to setup a
137
   working environment through virtualenv. Alternatively, you can use your
138
   system's package manager to install the dependencies (e.g. Macports has them
139
   all).
140

    
141
   * On Snow Leopard and linux (64-bit), you have to set the following
142
     environment variable for pip to compile the dependencies correctly.
143

    
144
  	   $ export ARCHFLAGS="-arch x86_64"
145

    
146
   * On Ubuntu, a few more packages must be installed before installing the
147
     prerequisite Python libraries
148

    
149
	   $ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
150

    
151
   Checkout the code and install the Python prerequisites. This assumes
152
   that python is already installed on the host.
153

    
154
    $ sudo easy_install virtualenv
155
    $ git clone https://user@code.grnet.gr/git/synnefo synnefo
156
    $ virtualenv --python=python2.6 synnefo --no-site-packages
157
    ...
158
    $ cd synnefo
159
    $ ./bin/pip install <list_of_dependencies>
160

    
161
    [WARNING]: The software must be checked out in a directory named synnefo,
162
    otherwise python imports will not work. Therefore, do not change the
163
    or rename the checkout path.
164

    
165
5. Database installation:
166
   A database supported by the Django ORM layer must be installed on nodes
167
   of type DB. The choices are: SQLIte, MySQL, PostgreSQL.
168

    
169
   * SQLite:
170
     The python sqlite driver is available by default with Python so no
171
     additional configuration is required. Also, most self-respecting systems
172
     have the sqlite library installed by default.
173

    
174
   *  MySQL:
175
      MySQL must be installed first:
176

    
177
      * Ubuntu - Debian
178
	      $ sudo apt-get install libmysqlclient-dev
179

    
180
      * MacPorts
181
	      $ sudo port install mysql5
182

    
183
      Install the MySQL python library on servers running the Django project:
184

    
185
	    $ bin/pip install MySQL-python
186

    
187
      Note: On MacOSX with Mysql install from MacPorts the above command will
188
      fail complaining that it cannot find the mysql_config command. Do the
189
      following and restart the installation
190
	    $ echo "mysql_config = /opt/local/bin/mysql_config5" >> ./build/MySQL-python/site.cfg
191

    
192
      Configure a MySQL db/account for synnefo
193
	    $ mysql -u root -p
194

    
195
    	mysql> create database synnefo;
196
	    mysql> show databases;
197
	    mysql> GRANT ALL on synnefo.* TO username IDENTIFIED BY 'password';
198

    
199
   * PostgreSQL
200
     You need to install the PostgreSQL binaries:
201
     * Ubuntu - Debian
202
	     $ sudo apt-get install postgresql-8.4 libpq-dev
203

    
204
     * MacPorts
205
	     $ sudo port install postgresql84
206

    
207
     Install the postgres Python library
208
	    $ bin/pip install psycopg2
209

    
210
     Configure a postgres db/account for synnefo:
211

    
212
     Become the postgres user, connect to PostgreSQL:
213
       $ sudo su - postgres
214
       $ psql
215
	
216
	 Run the following commands:
217
	   DROP DATABASE synnefo;
218
	   DROP USER username;
219
	   CREATE USER username WITH PASSWORD 'password';
220
	   CREATE DATABASE synnefo;
221
	   GRANT ALL PRIVILEGES ON DATABASE synnefo TO username;
222
	   ALTER DATABASE synnefo OWNER TO username;
223
	   ALTER USER username CREATEDB;
224

    
225
     The last line enables the newly created user to create own databases. This
226
     is needed for Django to create and drop the test_synnefo database for unit
227
     testing.
228

    
229

    
230
6. Setting up the Django project:
231
   The settings.py file for Django may be derived by concatenating the
232
   settings.py.dist file contained in the Synnefo distribution with a file
233
   containing custom modifications, which shall override all settings deviating
234
   from the supplied settings.py.dist. This is recommended to minimize the load
235
   of reconstructing settings.py from scratch, since each release currently
236
   brings heavy changes to settings.py.dist.
237

    
238
   Add the following to your custom settings.py, depending on your choice
239
   of DB:
240
   * SQLite
241

    
242
	 PROJECT_PATH = os.path.dirname(os.path.abspath(__file__)) + '/'
243

    
244
	 DATABASES = {
245
	     'default': {
246
		     'ENGINE': 'django.db.backends.sqlite3',
247
		     'NAME': PROJECT_PATH + 'synnefo.db' # WARN: This must be an absolute path
248
	     }
249
	 }
250

    
251
   * MySQL
252

    
253
 	 DATABASES = {
254
	     'default': {
255
             'ENGINE': 'django.db.backends.mysql',
256
             'NAME': 'synnefo',
257
             'USER': 'USERNAME',
258
             'PASSWORD': 'PASSWORD',
259
             'HOST': 'HOST',
260
             'PORT': 'PORT',
261
             'OPTIONS': {
262
                 'init_command': 'SET storage_engine=INNODB',
263
             }
264
	    }
265
	}
266

    
267
   * PostgreSQL
268

    
269
     DATABASES = {
270
	     'default': {
271
             'ENGINE': 'django.db.backends.postgresql_psycopg2',
272
             'NAME': 'DATABASE',
273
             'USER': 'USERNAME',
274
             'PASSWORD': 'PASSWORD',
275
             'HOST': 'HOST',
276
             'PORT': 'PORT',
277
	     }
278
     }
279

    
280
    Try it out. The following command will attempt to connect to the DB and
281
    print out DDL statements. It should not fail.
282

    
283
	$ ./bin/python manage.py sql db
284

    
285

    
286
7. Initialization of Synnefo DB:
287
   You need to initialize the Synnefo DB and load fixtures
288
   db/fixtures/{flavors,images}.json, which make the API usable by end users by
289
   defining a sample set of hardware configurations (flavors) and OS images.
290

    
291
     $ ./bin/python manage.py syncdb
292
     $ ./bin/python manage.py migrate db
293
     $ ./bin/python manage.py loaddata db/fixtures/flavors.json
294
     $ ./bin/python manage.py loaddata db/fixtures/images.json
295

    
296

    
297
8. Finalization of settings.py:
298
   Set the BACKEND_PREFIX_ID variable to some unique prefix, e.g. your commit
299
   username in settings.py. Several functional conventions within the system
300
   require this variable to include a dash at its end (e.g. snf-)
301

    
302
   Fix the AMQP-specific settings based on the selected BACKEND_PREFIX_ID.
303
   The fix_amqp_settings() function is called once at the end of
304
   settings.py.dist, you must call it again if you change BACKEND_PREFIX_ID
305
   at some later point.
306

    
307

    
308
9. Installation of the Ganeti monitoring daemon, /ganeti/snf-ganeti-eventd:
309
   The Ganeti monitoring daemon must run on GANETI-MASTER.
310
   The Ganeti monitoring daemon has no dependency on Django.
311
   
312
   Override all relevant settings in settings.d/99-snf-ganeti-eventd.conf,
313
   GANETI_* variables.
314
   Then, make sure PYTHONPATH contains the parent of the Django project,
315
   and start the server on the Ganeti master as root.
316

    
317
     root:~# export PYTHONPATH=$PYTHONPATH:/opt
318
     root:~# /opt/synnefo/ganeti/snf-ganeti-eventd.py
319

    
320
   TBD: how to handle master migration.
321

    
322

    
323
10. Installation of the Synnefo dispatcher, /logic/dispatcher.py:
324
    The logic dispatcher is part of the Synnefo Django project and must run
325
    on LOGIC nodes.
326

    
327
    The dispatcher retrieves messages from the queue and calls the
328
    appropriate handler function as defined in the queue configuration in
329
    setttings.py. The default configuration should work directly without
330
    any modifications.
331

    
332
    For the time being The dispatcher must be run by hand:
333
      $ ./bin/python ./logic/dispatcher.py
334

    
335
    The dispatcher should run in at least 2 instances to ensure high
336
    (actually, increased) availability.
337

    
338

    
339
11. Installation of the Synnefo Ganeti hook:
340
    The bash wrapper ganeti/snf-ganeti-hook is the generic launcher for
341
    Synnefo hooks in Ganeti. It resides in the ganeti/ directory under the
342
    root of the Synnefo Django project.
343

    
344
    The hook needs to be enabled for phases
345
    post-{add,modify,reboot,start,stop} by *symlinking*
346
    in  /etc/ganeti/hooks/instance-{add,modify,reboot,start,stop}-post.d
347
    on GANETI-MASTER, e.g.:
348

    
349
    root@ganeti-master:/etc/ganeti/hooks/instance-start-post.d# ls -l
350
    lrwxrwxrwx 1 root root 45 May   3 13:45 00-snf-ganeti-hook -> /home/devel/synnefo/ganeti/snf-ganeti-hook
351

    
352
    IMPORTANT: The link name may only contain "upper and lower case, digits,
353
    underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9_-]+$."
354
    See http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming.
355

    
356
    You need to set SYNNEFO_PROJECT_DIR in ganeti/snf-ganeti-hook. The bash
357
    script modifies PYTHONPATH accordingly, before passing control to the
358
    relevant Python code.
359

    
360

    
361
12. Installation of the VNC authentication proxy, vncauthproxy:
362
    To support OOB console access to the VMs over VNC, the vncauthproxy
363
    daemon must be running on every node of type APISERVER.
364

    
365
    Download and install vncauthproxy from its own repository,
366
    at https://code.grnet.gr/git/vncauthproxy (known good commit: 8799ab6d6e).
367

    
368
    Edit default settings on top of vncauthproxy.py.
369
    Set CTRL_SOCKET in util/vapclient.py to point to its control socket.
370

    
371
    FIXME: The CTRL_SOCKET setting will be moved to settings.py as
372
    VNCAUTHPROXY_CTRL_SOCKET.
373

    
374
    Create /var/log/vncauthproxy and set its permissions appropriately.
375

    
376

    
377
13. Installation of the customized Ganeti Instance Image for image deployment:
378
    For Synnefo to be able to launch VMs from specified Images, you need
379
    the gnt-instance-image OS Provider installed on the Ganeti backend.
380

    
381
    Download and install gnt-instance-image in all Ganeti nodes from its own
382
    repository, at https://code.grnet.gr/git/gnt-instance-image.
383

    
384
    After installing gnt-instance-image do the following:
385
    1. root@ganeti-master:/path-to-repo# cp ./defaults /etc/default/ganeti-instance-image
386
    2. Uncomment the following in /etc/default/ganeti-instance-image:
387
       SWAP=no
388
       ARCH="x86_64"
389
    3. Add to /etc/default/ganeti-instance-image, so that make-dump makes
390
       no /boot partition:
391
       KERNEL_PATH="True"
392
    4. Change the path in make-dump line 22 according to your installation
393
       (/usr/share/ganeti/os/image/common.sh --> /srv/ganeti/os/image/common.sh)
394
    5. In common.sh, comment out the KERNEL_PATH variable initialization.
395
       (#KERNEL_PATH="$INSTANCE_HV_kernel_path")
396
    6. In /etc/ganeti/instance-image/hooks, make sure the hooks you want to
397
       run during instance creation process have execute permission. At least
398
       `grub' and `root_passwd' should be triggered to make the instance
399
       usable:
400
         chmod +x /etc/ganeti/instance-image/hooks/{grub,root_passwd}
401

    
402
    Your custom mmages should be stored in a dump format under
403
    /var/cache/ganeti-instance-image
404
    and their filenames should have the following format:
405
      {backend_id}-x86_64-root.dump
406
    e.g., debian-6.0.1a-x86_64-root.dump (backend_id = "debian-6.0.1a")
407

    
408

    
409
14. Setup Synnefo-specific networking on the Ganeti backend:
410
    This part is deployment-specific and must be customized based on the
411
    specific needs of the system administrators.
412

    
413
    A reference installation will use a Synnefo-specific KVM ifup script,
414
    NFDHCPD and pre-provisioned Linux bridges to support public and private
415
    network functionality. For this:
416

    
417
    Grab NFDHCPD from its own repository (https://code.grnet.gr/git/nfdhcpd),
418
    install it, modify /etc/nfdhcpd/nfdhcpd.conf to reflect your network
419
    configuration.
420

    
421
    Install a custom KVM ifup script for use by Ganeti, as
422
    /etc/ganeti/kvm-vif-bridge, on GANETI-NODEs. A sample implementation is
423
    provided under /contrib/ganeti-hooks. Set NFDHCPD_STATE_DIR to point
424
    to NFDHCPD's state directory, usually /var/lib/nfdhcpd.
425

    
426

    
427
15. Create log file directories for Synnefo components, set appropriate
428
    permissions. By default logic/dispatcher.py and ganeti/snf-ganeti-eventd.py
429
    use /var/log/synnefo.
430

    
431

    
432
16. (Hopefully) Done
433