root / README.deploy @ 76a429fb
History | View | Annotate | Download (7.2 kB)
1 |
DEPLOYMENT notes |
---|---|
2 |
================= |
3 |
|
4 |
This document describes the services that comprise the Synnefo software |
5 |
infrastructure and the dependencies among them. This document applies |
6 |
to Synnefo v0.3. |
7 |
|
8 |
|
9 |
- Node types |
10 |
|
11 |
Nodes in a Synnefo deployment belong in one of the following types: |
12 |
|
13 |
* DB: |
14 |
A node [or more than one nodes, if using an HA configuration], |
15 |
running a DB engine supported by the Django ORM layer. The DB |
16 |
is the single source of truth for the servicing of API requests by |
17 |
Synnefo. |
18 |
Services: PostgreSQL / MySQL |
19 |
|
20 |
* WEBSERVER: |
21 |
A node running the implementation of the OpenStack API, in Django. |
22 |
Any number of WEBSERVERs can be used, in a load-balancing configuration, |
23 |
without any special consideration. Access to a common DB ensures |
24 |
consistency. |
25 |
Services: Web server, vncauthproxy |
26 |
|
27 |
* QUEUE: |
28 |
A node running the RabbitMQ software, which provides AMQP functionality. |
29 |
More than one QUEUE nodes may be deployed, in an HA configuration. Such |
30 |
deployments require shared storage, provided e.g., by DRBD. |
31 |
Services: RabbitMQ [rabbitmq-server] |
32 |
|
33 |
* LOGIC: |
34 |
A node running the business logic of Synnefo, in Django. It dequeues |
35 |
messages from QUEUE nodes, and provides the context in which business |
36 |
logic functions run. It uses Django ORM to connect to the common DB and |
37 |
update the state of the system, based on notifications received from the |
38 |
rest of the infrastructure, over AMQP. |
39 |
Services: the Synnefo logic dispatcher [/logic/dispatcher.py] |
40 |
|
41 |
* GANETI-MASTER and GANETI-NODE: |
42 |
A single GANETI-MASTER and a large number of GANETI-NODEs constitute the |
43 |
Ganeti backend for Synnefo, which undertakes all VM management functions. |
44 |
Any WEBSERVER can issue commands to the GANETI-MASTER, over RAPI, to effect |
45 |
changes in the state of the VMs. The GANETI-MASTER runs the Ganeti request |
46 |
queue. |
47 |
Services: only on GANETI-MASTER: |
48 |
the Synnefo Ganeti monitoring daemon [/ganeti/ganeti-eventd], |
49 |
and the Synnefo Ganeti hook [/ganeti/snf-ganeti-hook.py]. |
50 |
|
51 |
As of v0.3, the Synnefo Django project needs to be installed on nodes |
52 |
of type WEBSERVER, LOGIC and on the GANETI-MASTER, with a properly configured |
53 |
settings.py. In later revisions, the specific parts of the Django project |
54 |
which need to run on each node type will be identified. |
55 |
|
56 |
The settings.py file for Django is derived by concatenating the |
57 |
settings.py.dist file contained in the Synnefo distribution with a file |
58 |
containing custom modifications. This is recommended to minimize the load |
59 |
of reconstructing settings.py from scratch, since each release currently |
60 |
brings heavy changes to settings.py.dist. |
61 |
|
62 |
|
63 |
- Service dependencies |
64 |
|
65 |
This section describes each of the required service dependencies. |
66 |
|
67 |
* RabbitMQ |
68 |
|
69 |
RabbitMQ is used as a generic message broker for the system. It should |
70 |
be installed on two seperate machines (VMs should be enough for the |
71 |
moment) in a high availability configuration as described here: |
72 |
|
73 |
http://www.rabbitmq.com/pacemaker.html |
74 |
|
75 |
After installation, create a user and set its permissions |
76 |
|
77 |
rabbitmqctl add_user okeanos 0k3@n0s |
78 |
rabbitmqctl set_permissions -p / okeanos "^.*" ".*" ".*" |
79 |
|
80 |
The values set for the user and password must be mirrored in the |
81 |
RABBIT_* variables in settings.py |
82 |
|
83 |
* /ganeti/ganeti-eventd: the Ganeti monitoring daemon |
84 |
|
85 |
The Ganeti monitoring daemon must run on GANETI-MASTER. |
86 |
|
87 |
Override all relevant settings in settings.py.dist, GANETI_* variables. |
88 |
Then start the server on the Ganeti master as root |
89 |
|
90 |
# cd synnefo && python ./ganeti/ganeti-eventd |
91 |
|
92 |
FIXME: The server must be started from the project root directory. |
93 |
TBD: how to handle master migration. |
94 |
|
95 |
* /logic/dispatcher.py: the dispatcher |
96 |
|
97 |
The dispatcher retrieves messages from the queue and calls the |
98 |
appropriate handler function as defined in the queue configuration in |
99 |
setttings.py. The default configuration should work directly without |
100 |
any modifications. |
101 |
|
102 |
The dispatcher must be run by hand: |
103 |
$ ./bin/python ./logic/dispatcher.py |
104 |
|
105 |
The dispatcher should run in at least 2 instances to ensure high |
106 |
(actually, increased) availability. |
107 |
|
108 |
* /ganeti/snf-ganeti-hook.py: Ganeti hook for synnefo |
109 |
|
110 |
The script ganeti/snf-ganeti-hook.py is the generic launcher for Synnefo |
111 |
hooks in Ganeti. It resides in the ganeti/ directory of the Synnefo |
112 |
project root. |
113 |
|
114 |
The hook needs to be enabled for the post-start phase, by *symlinking* |
115 |
in /etc/ganeti/hooks/instance-start-post.d on GANETI-MASTER: |
116 |
|
117 |
root@ganeti-master:/etc/ganeti/hooks/instance-start-post.d# ls -l |
118 |
lrwxrwxrwx 1 root root 45 May 3 13:45 00-snf-ganeti-hook -> /home/devel/synnefo/ganeti/snf-ganeti-hook.py* |
119 |
|
120 |
IMPORTANT: The link name may only contain "upper and lower case, digits, |
121 |
underscores and hyphens. In other words, the regexp ^[a-zA-Z0-9_-]+$." |
122 |
See http://docs.ganeti.org/ganeti/master/html/hooks.html?highlight=hooks#naming. |
123 |
|
124 |
The script uses the location of the link target to determine the Synnefo |
125 |
Project root, before passing control to the relevant Python code. |
126 |
|
127 |
FIXME: Perhaps require a SYNNEFO_PROJECT_ROOT environment variable? |
128 |
|
129 |
* vncauthproxy |
130 |
|
131 |
To support OOB console access to the VMs over VNC, the vncauthproxy |
132 |
daemon must be running on every node of type WEBSERVER. |
133 |
|
134 |
Download and install vncauthproxy from its own repository, |
135 |
at https://code.grnet.gr/git/vncauthproxy |
136 |
(known good commit [FIXME for v0.3]: 48b1400e). |
137 |
|
138 |
Edit default settings on top of vncauthproxy.py. |
139 |
Set CTRL_SOCKET in util/vapclient.py to point to its control socket. |
140 |
|
141 |
FIXME: The CTRL_SOCKET setting will be moved to settings.py as |
142 |
VNCAUTHPROXY_CTRL_SOCKET. |
143 |
|
144 |
|
145 |
-Administration |
146 |
* Reconciliation process: On certain occasions, such as a Ganeti or |
147 |
RabbitMQ failure, the VM state in the system's database may differ from |
148 |
that in the Ganeti installation. The reconciliation process is designed |
149 |
to bring the system's database in sync with what Ganeti knows about |
150 |
each VM. |
151 |
|
152 |
The reconciliation process can be triggered for all VMs using the command |
153 |
|
154 |
./manage.py reconcile --all |
155 |
|
156 |
It is advised, though not strictly necessary, to run the reconciliation |
157 |
process periodically, through cron. To avoid overloading the Ganeti |
158 |
master, the periodic reconciliation process takes a staggered approach |
159 |
to updating the VMs, which is configured through the following |
160 |
parameters: |
161 |
|
162 |
* The settings.py parameter: RECONCILIATION_MIN, which specifies the |
163 |
maximum time a VM can remain ``non-reconciled''. (default: 30 mins) |
164 |
|
165 |
* The --interval option to the reconcile command, which declares the |
166 |
interval time between reconciliation attempts (default: 1 min) |
167 |
|
168 |
On each invocation of the reconcile command, the system will trigger a |
169 |
reconciliation for ((num_all_vms/RECONCILIATION_MIN) * interval) |
170 |
machines. Obviously the less the interval value and the more the |
171 |
RECONCILIATION_MIN setting, the less load is going to be put on Ganeti. |
172 |
|
173 |
|
174 |
- OS Specific instructions |
175 |
|
176 |
* Debian Squeeze |
177 |
|
178 |
TBD. |
179 |
|