Revision 271baf11 README.deploy

b/README.deploy
13 13
Nodes in a Synnefo deployment belong in one of the following types:
14 14

  
15 15
 * DB:
16
   A node [or more than one nodes, if using an HA configuration],
17
   running a DB engine supported by the Django ORM layer. The DB
18
   is the single source of truth for the servicing of API requests by
19
   Synnefo.
16
   A node [or more than one nodes, if using an HA configuration], running a DB
17
   engine supported by the Django ORM layer. The DB is the single source of
18
   truth for the servicing of API requests by Synnefo.
20 19
   Services: PostgreSQL / MySQL
21 20

  
22 21
 * APISERVER:
23
   A node running the implementation of the OpenStack API, in Django.
24
   Any number of APISERVERs can be used, in a load-balancing configuration,
25
   without any special consideration. Access to a common DB ensures
26
   consistency.
22
   A node running the implementation of the OpenStack API, in Django. Any number
23
   of APISERVERs can be used, in a load-balancing configuration, without any
24
   special consideration. Access to a common DB ensures consistency.
27 25
   Services: Web server, vncauthproxy
28 26

  
29 27
 * QUEUE:
30
   A node running the RabbitMQ software, which provides AMQP functionality.
31
   More than one QUEUE nodes may be deployed, in an HA configuration. Such
28
   A node running the RabbitMQ software, which provides AMQP functionality. More
29
   than one QUEUE nodes may be deployed, in an HA configuration. Such
32 30
   deployments require shared storage, provided e.g., by DRBD.
33 31
   Services: RabbitMQ [rabbitmq-server]
34 32

  
35 33
 * LOGIC:
36 34
   A node running the business logic of Synnefo, in Django. It dequeues
37
   messages from QUEUE nodes, and provides the context in which business
38
   logic functions run. It uses Django ORM to connect to the common DB and
39
   update the state of the system, based on notifications received from the
40
   rest of the infrastructure, over AMQP.
35
   messages from QUEUE nodes, and provides the context in which business logic
36
   functions run. It uses Django ORM to connect to the common DB and update the
37
   state of the system, based on notifications received from the rest of the
38
   infrastructure, over AMQP.
41 39
   Services: the Synnefo logic dispatcher [/logic/dispatcher.py]
42 40

  
43 41
 * GANETI-MASTER and GANETI-NODE:
......
47 45
   changes in the state of the VMs. The GANETI-MASTER runs the Ganeti request
48 46
   queue.
49 47
   Services:
50
   only on GANETI-MASTER:
51
     the Synnefo Ganeti monitoring daemon [/ganeti/ganeti-eventd],
52
     and the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py].
53
   on each GANETI_NODE:
54
     a deployment-specific KVM ifup script
55
     properly configured NFDHCPD
48
     only on GANETI-MASTER:
49
       the Synnefo Ganeti monitoring daemon [/ganeti/ganeti-eventd]
50
       the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py].
51
     on each GANETI_NODE:
52
       a deployment-specific KVM ifup script
53
       properly configured NFDHCPD
56 54

  
57 55

  
58 56
Installation Process
59 57
=====================
60 58

  
61
This section describes the installation process of the various node roles
62
in a Synnefo deployment.
59
This section describes the installation process of the various node roles in a
60
Synnefo deployment.
63 61

  
64 62

  
65 63
0. Allocation of physical nodes:
......
75 73

  
76 74

  
77 75
2. RabbitMQ installation:
78
   RabbitMQ is used as a generic message broker for the system. It should
79
   be installed on two seperate QUEUE nodes (VMs should be enough for the
80
   moment) in a high availability configuration as described here:
76
   RabbitMQ is used as a generic message broker for the system. It should be
77
   installed on two seperate QUEUE nodes (VMs should be enough for the moment)
78
   in a high availability configuration as described here:
81 79

  
82 80
     http://www.rabbitmq.com/pacemaker.html
83 81

  
......
91 89

  
92 90
3. Web server installation:
93 91
   A Web Server (e.g., Apache) needs to be installed on the APISERVERs,
94
   and be configured to run the Synnefo Django project appropriately.
95
   Selection and configuration of a Web server is outside the scope of this
96
   document.
92
   and be configured to run the Synnefo Django project appropriately. Selection
93
   and configuration of a Web server is outside the scope of this document.
97 94

  
98 95
   For testing or development purposes, Django's own development server,
99
   ./manage.py runserver can be used.
96
   `./manage.py runserver' can be used.
100 97

  
101 98

  
102 99
4. Installation of the Synnefo Django project:
103
   As of v0.4, the Synnefo Django project needs to be installed on nodes
104
   of type APISERVER, LOGIC and on the GANETI-MASTER, with a properly configured
100
   As of v0.4, the Synnefo Django project needs to be installed on nodes of type
101
   APISERVER, LOGIC and on the GANETI-MASTER, with a properly configured
105 102
   settings.py. In later revisions, the specific parts of the Django project
106 103
   which need to run on each node type will be identified.
107 104

  
108
   Synnefo is written in Python 2.6 and depends on the following Python
109
   modules: [package versions confirmed to be compatible are in braces]
105
   Synnefo is written in Python 2.6 and depends on the following Python modules:
106
   [package versions confirmed to be compatible are in braces]
110 107

  
111 108
    * django 1.2 [Django==1.2.4]
112 109
    * simplejson [simplejson==2.1.3]
113 110
    * pycurl [pycurl==7.19.0]
114 111
    * python-dateutil  [python-dateutil==1.4.1]
115
      WARNING: version python-dateutil==2.0 downloaded by pip known *not* to work
116
               with Python 2.6
112
      WARNING: version python-dateutil==2.0 downloaded by pip known *not* to
113
      work with Python 2.6
117 114
    * south [south==0.7.1]
118 115
      WARNING: not working with Debian squeeze's default south-0.7-1 package.
119 116
    * amqplib [amqplib==0.6.1]
......
122 119
    * MySQL-python [MySQL-python==1.2.3]
123 120
    * psycopg2 [psycopg2==2.4]
124 121

  
125
   if the invitations application is deployed, the following
126
   dependencies should be installed:
127

  
122
   if the invitations application is deployed, the following dependencies should
123
   be installed:
128 124
    * pycrypto==2.1.0
129 125

  
130 126
   if you want to test the ganeti (FIXME)
131 127

  
132
   to run the user interface tests, selenium must be installed
128
   to run the user interface tests, selenium must be installed:
133 129
    * selenium [?]
134 130

  
135 131
   The easiest method for installation of the Django project is to setup a
......
137 133
   system's package manager to install the dependencies (e.g. Macports has them
138 134
   all).
139 135

  
140
   * On Snow Leopard and linux (64-bit), you have to set the following environment
141
     variable for pip to compile the dependencies correctly.
136
   * On Snow Leopard and linux (64-bit), you have to set the following
137
     environment variable for pip to compile the dependencies correctly.
142 138

  
143 139
  	   $ export ARCHFLAGS="-arch x86_64"
144 140

  
......
147 143

  
148 144
	   $ sudo aptitude install libcurl3-gnutls libcurl3-gnutls-dev uuid-dev
149 145

  
150
   Checkout the code and install the Python prerequisites. This assumes
151
   that python is already installed on the host.
146
   Checkout the code and install the Python prerequisites. This assumes that
147
   python is already installed on the host.
152 148

  
153 149
    $ sudo easy_install virtualenv
154 150
    $ git clone https://user@code.grnet.gr/git/synnefo synnefo
......
163 159
   of type DB. The choices are: SQLIte, MySQL, PostgreSQL.
164 160

  
165 161
   * SQLite:
166
     The python sqlite driver is available by default with Python so no additional
167
     configuration is required. Also, most self-respecting systems have the sqlite
168
     library installed by default.
162
     The python sqlite driver is available by default with Python so no
163
     additional configuration is required. Also, most self-respecting systems
164
     have the sqlite library installed by default.
169 165

  
170
   *  MySQL:
166
   * MySQL:
171 167
      MySQL must be installed first:
172 168

  
173 169
      * Ubuntu - Debian
......
180 176

  
181 177
	    $ bin/pip install MySQL-python
182 178

  
183
      Note: On MacOSX with Mysql install from MacPorts the above command will fail
184
      complaining that it cannot find the mysql_config command. Do the following and
185
      restart the installation
186
	    $ echo "mysql_config = /opt/local/bin/mysql_config5" >> ./build/MySQL-python/site.cfg
179
      Note: On MacOSX with Mysql install from MacPorts the above command will
180
            fail complaining that it cannot find the mysql_config command. Do
181
            the following and restart the installation
182
	        $ echo "mysql_config = /opt/local/bin/mysql_config5" >> \
183
                                         ./build/MySQL-python/site.cfg
187 184

  
188 185
      Configure a MySQL db/account for synnefo
189 186
	    $ mysql -u root -p
......
218 215
	   ALTER DATABASE synnefo OWNER TO username;
219 216
	   ALTER USER username CREATEDB;
220 217

  
221
     The last line enables the newly created user to create own databases. This is
222
     needed for Django to create and drop the test_synnefo database for unit
218
     The last line enables the newly created user to create own databases. This
219
     is needed for Django to create and drop the test_synnefo database for unit
223 220
     testing.
224 221

  
225 222

  
......
296 293

  
297 294
   Fix the AMQP-specific settings based on the selected BACKEND_PREFIX_ID.
298 295
   The fix_amqp_settings() function is called once at the end of
299
   settings.py.dist, you must call it again if you change BACKEND_PREFIX_ID
300
   at some later point.
296
   settings.py.dist, you must call it again if you change BACKEND_PREFIX_ID at
297
   some later point.
301 298

  
302 299

  
303 300
9. Installation of the Ganeti monitoring daemon, /ganeti/ganeti-eventd:
......
317 314
    The logic dispatcher is part of the Synnefo Django project and must run
318 315
    on LOGIC nodes.
319 316

  
320
    The dispatcher retrieves messages from the queue and calls the
321
    appropriate handler function as defined in the queue configuration in
322
    setttings.py. The default configuration should work directly without
323
    any modifications.
317
    The dispatcher retrieves messages from the queue and calls the appropriate
318
    handler function as defined in the queue configuration in `setttings.py'.
319
    The default configuration should work directly without any modifications.
324 320

  
325 321
    For the time being The dispatcher must be run by hand:
326 322
      $ ./bin/python ./logic/dispatcher.py
......
334 330
    Synnefo hooks in Ganeti.  It resides in the ganeti/ directory under the root
335 331
    of Synnefo Django project root.
336 332

  
337
    The hook needs to be enabled for phases
338
    post-{add,modify,reboot,start,stop} by *symlinking*
339
    in  /etc/ganeti/hooks/instance-{add,modify,reboot,start,stop}-post.d
340
    on GANETI-MASTER, e.g.:
333
    The hook needs to be enabled for phases post-{add,modify,reboot,start,stop}
334
    by *symlinking* in
335
    /etc/ganeti/hooks/instance-{add,modify,reboot,start,stop}-post.d on
336
    GANETI-MASTER, e.g.:
341 337

  
342 338
    root@ganeti-master:/etc/ganeti/hooks/instance-start-post.d# ls -l
343 339
    lrwxrwxrwx 1 root root 45 May   3 13:45 00-snf-ganeti-hook -> /home/devel/synnefo/ganeti/snf-ganeti-hook.py*
344 340

  
345 341
    IMPORTANT: The link name may only contain "upper and lower case, digits,
346 342
    underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9_-]+$."
347
    See http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming.
343
    See:
344
     http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming
348 345

  
349 346
    The script uses the location of the link target to determine the Synnefo
350 347
    Project root, before passing control to the relevant Python code.
......
376 373
    repository, at https://code.grnet.gr/git/gnt-instance-image.
377 374

  
378 375
    After installing gnt-instance-image do the following:
379
    1. root@ganeti-master:/path-to-repo# cp ./defaults /etc/default/ganeti-instance-image
376
    1. root@ganeti-master$ cd /path-to-repocp
377
       root@ganeti-master$ ./defaults /etc/default/ganeti-instance-image
380 378
    2. Uncomment the following in /etc/default/ganeti-instance-image:
381
       SWAP=no
382
       ARCH="x86_64"
383
    3. Add to /etc/default/ganeti-instance-image (so that make-dump makes no /boot partition):
384
       KERNEL_PATH="True"
379
         SWAP=no
380
         ARCH="x86_64"
381
    3. Add to /etc/default/ganeti-instance-image (so that make-dump makes no
382
       /boot partition):
383
         KERNEL_PATH="True"
385 384
    4. Change the path in make-dump line 22 according to your installation
386 385
       (/usr/share/ganeti/os/image/common.sh --> /srv/ganeti/os/image/common.sh)
387
    5. In common.sh, comment out the KERNEL_PATH variable initialization.
388
       (#KERNEL_PATH="$INSTANCE_HV_kernel_path")
386
    5. In common.sh, comment out the KERNEL_PATH variable initialization:
387
         #KERNEL_PATH="$INSTANCE_HV_kernel_path"
389 388
    6. In /etc/ganeti/instance-image/hooks, make sure the hooks you want to
390 389
       run during instance creation process have execute permission. At least
391
       `grub' and `root_passwd' should be triggered to make the instance
392
       usable:
390
       `grub' and `root_passwd' should be triggered to make the instance usable:
393 391
         chmod +x /etc/ganeti/instance-image/hooks/{grub,root_passwd}
394 392

  
395
    Your Custom Images should be stored in a dump format under /var/cache/ganeti-instance-image
396
    and their filenames should have the following format:
393
    Your Custom Images should be stored in a dump format under
394
    /var/cache/ganeti-instance-image and their filenames should have the
395
    following format:
397 396
      {backend_id}-x86_64-root.dump
398 397
    e.g., debian-6.0.1a-x86_64-root.dump (backend_id = "debian-6.0.1a")
399 398

  

Also available in: Unified diff