Statistics
| Branch: | Tag: | Revision:

root / README.deploy @ 45e3d1ac

History | View | Annotate | Download (15.8 kB)

1
README.deploy -- Instructions for a basic deployment of Synnefo v0.4
2

    
3
This document describes the basic steps to obtain a basic, working Synnefo
4
deployment. It begins by examining the different node roles, then moves to the
5
installation and setup of distinct software components.
6

    
7
It applies to Synnefo v0.4.
8

    
9

    
10
Node types
11
===========
12

    
13
Nodes in a Synnefo deployment belong in one of the following types:
14

    
15
 * DB:
16
   A node [or more than one nodes, if using an HA configuration],
17
   running a DB engine supported by the Django ORM layer. The DB
18
   is the single source of truth for the servicing of API requests by
19
   Synnefo.
20
   Services: PostgreSQL / MySQL
21

    
22
 * APISERVER:
23
   A node running the implementation of the OpenStack API, in Django.
24
   Any number of APISERVERs can be used, in a load-balancing configuration,
25
   without any special consideration. Access to a common DB ensures
26
   consistency.
27
   Services: Web server, vncauthproxy
28

    
29
 * QUEUE:
30
   A node running the RabbitMQ software, which provides AMQP functionality.
31
   More than one QUEUE nodes may be deployed, in an HA configuration. Such
32
   deployments require shared storage, provided e.g., by DRBD.
33
   Services: RabbitMQ [rabbitmq-server]
34

    
35
 * LOGIC:
36
   A node running the business logic of Synnefo, in Django. It dequeues
37
   messages from QUEUE nodes, and provides the context in which business
38
   logic functions run. It uses Django ORM to connect to the common DB and
39
   update the state of the system, based on notifications received from the
40
   rest of the infrastructure, over AMQP.
41
   Services: the Synnefo logic dispatcher [/logic/dispatcher.py]
42

    
43
 * GANETI-MASTER and GANETI-NODE:
44
   A single GANETI-MASTER and a large number of GANETI-NODEs constitute the
45
   Ganeti backend for Synnefo, which undertakes all VM management functions.
46
   Any APISERVER can issue commands to the GANETI-MASTER, over RAPI, to effect
47
   changes in the state of the VMs. The GANETI-MASTER runs the Ganeti request
48
   queue.
49
   Services:
50
   only on GANETI-MASTER:
51
     the Synnefo Ganeti monitoring daemon [/ganeti/snf-ganeti-eventd],
52
     and the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py].
53
   on each GANETI_NODE:
54
     a deployment-specific KVM ifup script
55
     properly configured NFDHCPD
56

    
57

    
58
Installation Process
59
=====================
60

    
61
This section describes the installation process of the various node roles
62
in a Synnefo deployment.
63

    
64

    
65
0. Allocation of physical nodes:
66
   Determine the role of every physical node in your deployment.
67

    
68

    
69
1. Ganeti installation:
70
   Synnefo requires a working Ganeti installation at the backend. Installation
71
   of Ganeti is not covered by this document, please refer to
72
   http://docs.ganeti.org/ganeti/current/html for all the gory details. A
73
   successful Ganeti installation concludes with a working GANETI-MASTER and a
74
   number of GANETI-NODEs.
75

    
76

    
77
2. RabbitMQ installation:
78
   RabbitMQ is used as a generic message broker for the system. It should
79
   be installed on two seperate QUEUE nodes (VMs should be enough for the
80
   moment) in a high availability configuration as described here:
81

    
82
     http://www.rabbitmq.com/pacemaker.html
83

    
84
   After installation, create a user and set its permissions
85
     rabbitmqctl add_user okeanos 0k3@n0s
86
     rabbitmqctl set_permissions -p / okeanos  "^.*" ".*" ".*"
87

    
88
   The values set for the user and password must be mirrored in the
89
   RABBIT_* variables in settings.py (see step 6)
90

    
91

    
92
3. Web server installation:
93
   A Web Server (e.g., Apache) needs to be installed on the APISERVERs,
94
   and be configured to run the Synnefo Django project appropriately.
95
   Selection and configuration of a Web server is outside the scope of this
96
   document.
97

    
98
   For testing or development purposes, Django's own development server,
99
   ./manage.py runserver can be used.
100

    
101

    
102
4. Installation of the Synnefo Django project:
103
   As of v0.4, the Synnefo Django project needs to be installed on nodes
104
   of type APISERVER, LOGIC and on the GANETI-MASTER, with a properly
105
   configured settings.py. In later revisions, the specific parts of the Django
106
   project which need to run on each node type will be identified.
107

    
108
   Synnefo is written in Python 2.6 and depends on the following Python
109
   modules: [package versions confirmed to be compatible are in braces]
110

    
111
    * django 1.2 [Django==1.2.4]
112
    * simplejson [simplejson==2.1.3]
113
    * pycurl [pycurl==7.19.0]
114
    * python-dateutil  [python-dateutil==1.4.1]
115
      WARNING: version python-dateutil==2.0 downloaded by pip known *not* to
116
               work with Python 2.6
117
    * python-ipy [IPy==0.72]
118
    * south [south==0.7.1]
119
      WARNING: not working with Debian squeeze's default south-0.7-1 package.
120
    * amqplib [amqplib==0.6.1]
121

    
122
   also, depending on the database engine of choice, on one of the following:
123
    * MySQL-python [MySQL-python==1.2.3]
124
    * psycopg2 [psycopg2==2.4]
125

    
126
   if the invitations application is deployed, the following
127
   dependencies should be installed:
128

    
129
    * pycrypto==2.1.0
130

    
131
   To run the user interface tests, selenium must be installed
132
    * selenium [?]
133

    
134
   The easiest method for installation of the Django project is to setup a
135
   working environment through virtualenv. Alternatively, you can use your
136
   system's package manager to install the dependencies (e.g. Macports has them
137
   all).
138

    
139
   * On Snow Leopard and linux (64-bit), you have to set the following
140
     environment variable for pip to compile the dependencies correctly.
141

    
142
  	   $ export ARCHFLAGS="-arch x86_64"
143

    
144
   * On Ubuntu, a few more packages must be installed before installing the
145
     prerequisite Python libraries
146

    
147
	   $ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
148

    
149
   Checkout the code and install the Python prerequisites. This assumes
150
   that python is already installed on the host.
151

    
152
    $ sudo easy_install virtualenv
153
    $ git clone https://user@code.grnet.gr/git/synnefo synnefo
154
    $ virtualenv --python=python2.6 synnefo --no-site-packages
155
    ...
156
    $ cd synnefo
157
    $ ./bin/pip install <list_of_dependencies>
158

    
159

    
160
5. Database installation:
161
   A database supported by the Django ORM layer must be installed on nodes
162
   of type DB. The choices are: SQLIte, MySQL, PostgreSQL.
163

    
164
   * SQLite:
165
     The python sqlite driver is available by default with Python so no
166
     additional configuration is required. Also, most self-respecting systems
167
     have the sqlite library installed by default.
168

    
169
   *  MySQL:
170
      MySQL must be installed first:
171

    
172
      * Ubuntu - Debian
173
	      $ sudo apt-get install libmysqlclient-dev
174

    
175
      * MacPorts
176
	      $ sudo port install mysql5
177

    
178
      Install the MySQL python library on servers running the Django project:
179

    
180
	    $ bin/pip install MySQL-python
181

    
182
      Note: On MacOSX with Mysql install from MacPorts the above command will
183
      fail complaining that it cannot find the mysql_config command. Do the
184
      following and restart the installation
185
	    $ echo "mysql_config = /opt/local/bin/mysql_config5" >> ./build/MySQL-python/site.cfg
186

    
187
      Configure a MySQL db/account for synnefo
188
	    $ mysql -u root -p
189

    
190
    	mysql> create database synnefo;
191
	    mysql> show databases;
192
	    mysql> GRANT ALL on synnefo.* TO username IDENTIFIED BY 'password';
193

    
194
   * PostgreSQL
195
     You need to install the PostgreSQL binaries:
196
     * Ubuntu - Debian
197
	     $ sudo apt-get install postgresql-8.4 libpq-dev
198

    
199
     * MacPorts
200
	     $ sudo port install postgresql84
201

    
202
     Install the postgres Python library
203
	    $ bin/pip install psycopg2
204

    
205
     Configure a postgres db/account for synnefo:
206

    
207
     Become the postgres user, connect to PostgreSQL:
208
       $ sudo su - postgres
209
       $ psql
210
	
211
	 Run the following commands:
212
	   DROP DATABASE synnefo;
213
	   DROP USER username;
214
	   CREATE USER username WITH PASSWORD 'password';
215
	   CREATE DATABASE synnefo;
216
	   GRANT ALL PRIVILEGES ON DATABASE synnefo TO username;
217
	   ALTER DATABASE synnefo OWNER TO username;
218
	   ALTER USER username CREATEDB;
219

    
220
     The last line enables the newly created user to create own databases. This
221
     is needed for Django to create and drop the test_synnefo database for unit
222
     testing.
223

    
224

    
225
6. Setting up the Django project:
226
   The settings.py file for Django may be derived by concatenating the
227
   settings.py.dist file contained in the Synnefo distribution with a file
228
   containing custom modifications, which shall override all settings deviating
229
   from the supplied settings.py.dist. This is recommended to minimize the load
230
   of reconstructing settings.py from scratch, since each release currently
231
   brings heavy changes to settings.py.dist.
232

    
233
   Add the following to your custom settings.py, depending on your choice
234
   of DB:
235
   * SQLite
236

    
237
	 PROJECT_PATH = os.path.dirname(os.path.abspath(__file__)) + '/'
238

    
239
	 DATABASES = {
240
	     'default': {
241
		     'ENGINE': 'django.db.backends.sqlite3',
242
		     'NAME': PROJECT_PATH + 'synnefo.db' # WARN: This must be an absolute path
243
	     }
244
	 }
245

    
246
   * MySQL
247

    
248
 	 DATABASES = {
249
	     'default': {
250
             'ENGINE': 'django.db.backends.mysql',
251
             'NAME': 'synnefo',
252
             'USER': 'USERNAME',
253
             'PASSWORD': 'PASSWORD',
254
             'HOST': 'HOST',
255
             'PORT': 'PORT',
256
             'OPTIONS': {
257
                 'init_command': 'SET storage_engine=INNODB',
258
             }
259
	    }
260
	}
261

    
262
   * PostgreSQL
263

    
264
     DATABASES = {
265
	     'default': {
266
             'ENGINE': 'django.db.backends.postgresql_psycopg2',
267
             'NAME': 'DATABASE',
268
             'USER': 'USERNAME',
269
             'PASSWORD': 'PASSWORD',
270
             'HOST': 'HOST',
271
             'PORT': 'PORT',
272
	     }
273
     }
274

    
275
    Try it out. The following command will attempt to connect to the DB and
276
    print out DDL statements. It should not fail.
277

    
278
	$ ./bin/python manage.py sql db
279

    
280

    
281
7. Initialization of Synnefo DB:
282
   You need to initialize the Synnefo DB and load fixtures
283
   db/fixtures/{flavors,images}.json, which make the API usable by end users by
284
   defining a sample set of hardware configurations (flavors) and OS images.
285

    
286
     $ ./bin/python manage.py syncdb
287
     $ ./bin/python manage.py migrate db
288
     $ ./bin/python manage.py loaddata db/fixtures/flavors.json
289
     $ ./bin/python manage.py loaddata db/fixtures/images.json
290

    
291

    
292
8. Finalization of settings.py:
293
   Set the BACKEND_PREFIX_ID variable to some unique prefix, e.g. your commit
294
   username in settings.py. Several functional conventions within the system
295
   require this variable to include a dash at its end (e.g. snf-)
296

    
297
   Fix the AMQP-specific settings based on the selected BACKEND_PREFIX_ID.
298
   The fix_amqp_settings() function is called once at the end of
299
   settings.py.dist, you must call it again if you change BACKEND_PREFIX_ID
300
   at some later point.
301

    
302

    
303
9. Installation of the Ganeti monitoring daemon, /ganeti/snf-ganeti-eventd:
304
   The Ganeti monitoring daemon must run on GANETI-MASTER.
305
   The Ganeti monitoring daemon has no dependency on Django.
306
   
307
   Override all relevant settings in settings.d/99-snf-ganeti-eventd.conf,
308
   GANETI_* variables.
309
   Then, make sure PYTHONPATH contains the parent of the Django project,
310
   and start the server on the Ganeti master as root.
311

    
312
     root:~# export PYTHONPATH=$PYTHONPATH:/opt
313
     root:~# /opt/synnefo/ganeti/snf-ganeti-eventd.py
314

    
315
   TBD: how to handle master migration.
316

    
317

    
318
10. Installation of the Synnefo dispatcher, /logic/dispatcher.py:
319
    The logic dispatcher is part of the Synnefo Django project and must run
320
    on LOGIC nodes.
321

    
322
    The dispatcher retrieves messages from the queue and calls the
323
    appropriate handler function as defined in the queue configuration in
324
    setttings.py. The default configuration should work directly without
325
    any modifications.
326

    
327
    For the time being The dispatcher must be run by hand:
328
      $ ./bin/python ./logic/dispatcher.py
329

    
330
    The dispatcher should run in at least 2 instances to ensure high
331
    (actually, increased) availability.
332

    
333

    
334
11. Installation of the Synnefo Ganeti hook:
335
    The bash wrapper ganeti/snf-ganeti-hook is the generic launcher for
336
    Synnefo hooks in Ganeti. It resides in the ganeti/ directory under the
337
    root of the Synnefo Django project.
338

    
339
    The hook needs to be enabled for phases
340
    post-{add,modify,reboot,start,stop} by *symlinking*
341
    in  /etc/ganeti/hooks/instance-{add,modify,reboot,start,stop}-post.d
342
    on GANETI-MASTER, e.g.:
343

    
344
    root@ganeti-master:/etc/ganeti/hooks/instance-start-post.d# ls -l
345
    lrwxrwxrwx 1 root root 45 May   3 13:45 00-snf-ganeti-hook -> /home/devel/synnefo/ganeti/snf-ganeti-hook
346

    
347
    IMPORTANT: The link name may only contain "upper and lower case, digits,
348
    underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9_-]+$."
349
    See http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming.
350

    
351
    You need to set SYNNEFO_PROJECT_DIR in ganeti/snf-ganeti-hook. The bash
352
    script modifies PYTHONPATH accordingly, before passing control to the
353
    relevant Python code.
354

    
355

    
356
12. Installation of the VNC authentication proxy, vncauthproxy:
357
    To support OOB console access to the VMs over VNC, the vncauthproxy
358
    daemon must be running on every node of type APISERVER.
359

    
360
    Download and install vncauthproxy from its own repository,
361
    at https://code.grnet.gr/git/vncauthproxy (known good commit: 8799ab6d6e).
362

    
363
    Edit default settings on top of vncauthproxy.py.
364
    Set CTRL_SOCKET in util/vapclient.py to point to its control socket.
365

    
366
    FIXME: The CTRL_SOCKET setting will be moved to settings.py as
367
    VNCAUTHPROXY_CTRL_SOCKET.
368

    
369
    Create /var/log/vncauthproxy and set its permissions appropriately.
370

    
371

    
372
13. Installation of the customized Ganeti Instance Image for image deployment:
373
    For Synnefo to be able to launch VMs from specified Images, you need
374
    the gnt-instance-image OS Provider installed on the Ganeti backend.
375

    
376
    Download and install gnt-instance-image in all Ganeti nodes from its own
377
    repository, at https://code.grnet.gr/git/gnt-instance-image.
378

    
379
    After installing gnt-instance-image do the following:
380
    1. root@ganeti-master:/path-to-repo# cp ./defaults /etc/default/ganeti-instance-image
381
    2. Uncomment the following in /etc/default/ganeti-instance-image:
382
       SWAP=no
383
       ARCH="x86_64"
384
    3. Add to /etc/default/ganeti-instance-image, so that make-dump makes
385
       no /boot partition:
386
       KERNEL_PATH="True"
387
    4. Change the path in make-dump line 22 according to your installation
388
       (/usr/share/ganeti/os/image/common.sh --> /srv/ganeti/os/image/common.sh)
389
    5. In common.sh, comment out the KERNEL_PATH variable initialization.
390
       (#KERNEL_PATH="$INSTANCE_HV_kernel_path")
391
    6. In /etc/ganeti/instance-image/hooks, make sure the hooks you want to
392
       run during instance creation process have execute permission. At least
393
       `grub' and `root_passwd' should be triggered to make the instance
394
       usable:
395
         chmod +x /etc/ganeti/instance-image/hooks/{grub,root_passwd}
396

    
397
    Your custom mmages should be stored in a dump format under
398
    /var/cache/ganeti-instance-image
399
    and their filenames should have the following format:
400
      {backend_id}-x86_64-root.dump
401
    e.g., debian-6.0.1a-x86_64-root.dump (backend_id = "debian-6.0.1a")
402

    
403

    
404
14. Setup Synnefo-specific networking on the Ganeti backend:
405
    This part is deployment-specific and must be customized based on the
406
    specific needs of the system administrators.
407

    
408
    A reference installation will use a Synnefo-specific KVM ifup script,
409
    NFDHCPD and pre-provisioned Linux bridges to support public and private
410
    network functionality. For this:
411

    
412
    Grab NFDHCPD from its own repository (https://code.grnet.gr/git/nfdhcpd),
413
    install it, modify /etc/nfdhcpd/nfdhcpd.conf to reflect your network
414
    configuration.
415

    
416
    Install a custom KVM ifup script for use by Ganeti, as
417
    /etc/ganeti/kvm-vif-bridge, on GANETI-NODEs. A sample implementation is
418
    provided under /contrib/ganeti-hooks. Set NFDHCPD_STATE_DIR to point
419
    to NFDHCPD's state directory, usually /var/lib/nfdhcpd.
420

    
421

    
422
15. Create log file directories for Synnefo components, set appropriate
423
    permissions. By default logic/dispatcher.py and ganeti/snf-ganeti-eventd.py
424
    use /var/log/synnefo.
425

    
426

    
427
16. (Hopefully) Done
428