root / docs / snf-deploy.rst @ c3ff5c37
History | View | Annotate | Download (16.9 kB)
1 | 77180645 | Dimitris Aragiorgis | .. _snf-deploy: |
---|---|---|---|
2 | 77180645 | Dimitris Aragiorgis | |
3 | 77180645 | Dimitris Aragiorgis | snf-deploy tool |
4 | 77180645 | Dimitris Aragiorgis | ^^^^^^^^^^^^^^^ |
5 | 77180645 | Dimitris Aragiorgis | |
6 | 21d3d487 | Constantinos Venetsanopoulos | The `snf-deploy` tool allows you to automatically deploy Synnefo. |
7 | 21d3d487 | Constantinos Venetsanopoulos | You can use `snf-deploy` to deploy Synnefo, in two ways: |
8 | 77180645 | Dimitris Aragiorgis | |
9 | 21d3d487 | Constantinos Venetsanopoulos | 1. Create a virtual cluster on your local machine and then deploy on that cluster. |
10 | bb535968 | Constantinos Venetsanopoulos | 2. Deploy on a pre-existent cluster of physical nodes running Debian Squeeze. |
11 | bb535968 | Constantinos Venetsanopoulos | |
12 | bb535968 | Constantinos Venetsanopoulos | Currently, `snf-deploy` is mostly useful for testing/demo installations and is |
13 | bb535968 | Constantinos Venetsanopoulos | not recommended for production environment Synnefo deployments. If you want to |
14 | ff6ba888 | Constantinos Venetsanopoulos | deploy Synnefo in production, please read first the :ref:`Admin's installation |
15 | ff6ba888 | Constantinos Venetsanopoulos | guide <quick-install-admin-guide>` and then the :ref:`Admin's guide |
16 | ff6ba888 | Constantinos Venetsanopoulos | <admin-guide>`. |
17 | bb535968 | Constantinos Venetsanopoulos | |
18 | bb535968 | Constantinos Venetsanopoulos | If you use `snf-deploy` you will setup an up-and-running Synnefo installation, |
19 | bb535968 | Constantinos Venetsanopoulos | but the end-to-end functionality will depend on your underlying infrastracture |
20 | bb535968 | Constantinos Venetsanopoulos | (e.g. is nested virtualization enabled in your PC, is the router properly |
21 | bb535968 | Constantinos Venetsanopoulos | configured, do nodes have fully qualified domain names, etc.). In any way, it |
22 | bb535968 | Constantinos Venetsanopoulos | will enable you to get a grasp of the Web UI/API and base funtionality of |
23 | bb535968 | Constantinos Venetsanopoulos | Synnefo and also provide a proper configuration that you can afterwards consult |
24 | bb535968 | Constantinos Venetsanopoulos | while reading the Admin guides to set up a production environment that will |
25 | bb535968 | Constantinos Venetsanopoulos | scale up and use all available features (e.g. RADOS, Archipelago, etc). |
26 | bb535968 | Constantinos Venetsanopoulos | |
27 | bb535968 | Constantinos Venetsanopoulos | `snf-deploy` is a debian package that should be installed locally and allows |
28 | bb535968 | Constantinos Venetsanopoulos | you to install Synnefo on remote nodes (if you go for (2)), or spawn a cluster |
29 | bb535968 | Constantinos Venetsanopoulos | of VMs on your local machine using KVM and then install Synnefo on this cluster |
30 | bb535968 | Constantinos Venetsanopoulos | (if you go for (1)). To this end, here we will break down our description into |
31 | bb535968 | Constantinos Venetsanopoulos | three sections: |
32 | bb535968 | Constantinos Venetsanopoulos | |
33 | bb535968 | Constantinos Venetsanopoulos | a. :ref:`snf-deploy configuration <conf>` |
34 | bb535968 | Constantinos Venetsanopoulos | b. :ref:`Creating a virtual cluster <vcluster>` (needed for (1)) |
35 | bb535968 | Constantinos Venetsanopoulos | c. :ref:`Synnefo deployment <inst>` (either on virtual nodes created on section b, |
36 | bb535968 | Constantinos Venetsanopoulos | or on remote physical nodes) |
37 | 77180645 | Dimitris Aragiorgis | |
38 | 21d3d487 | Constantinos Venetsanopoulos | If you go for (1) you will need to walk through all the sections. If you go for |
39 | bb535968 | Constantinos Venetsanopoulos | (2), you should skip section `(b) <vcluster>`, since you only need sections |
40 | bb535968 | Constantinos Venetsanopoulos | `(a) <conf>` and `(c) <inst>`. |
41 | 77180645 | Dimitris Aragiorgis | |
42 | 21d3d487 | Constantinos Venetsanopoulos | Before getting any further we should mention the roles that `snf-deploy` refers |
43 | bb535968 | Constantinos Venetsanopoulos | to. The Synnefo roles are described in detail :ref:`here |
44 | bb535968 | Constantinos Venetsanopoulos | <physical-node-roles>`. Note that multiple roles can co-exist in the same node |
45 | bb535968 | Constantinos Venetsanopoulos | (virtual or physical). |
46 | 77180645 | Dimitris Aragiorgis | |
47 | 21d3d487 | Constantinos Venetsanopoulos | Currently, `snf-deploy` recognizes the following combined roles: |
48 | 77180645 | Dimitris Aragiorgis | |
49 | bb535968 | Constantinos Venetsanopoulos | * **accounts** = **WEBSERVER** + **ASTAKOS** |
50 | 21d3d487 | Constantinos Venetsanopoulos | * **pithos** = **WEBSERVER** + **PITHOS** |
51 | 21d3d487 | Constantinos Venetsanopoulos | * **cyclades** = **WEBSERVER** + **CYCLADES** |
52 | 21d3d487 | Constantinos Venetsanopoulos | * **db** = **ASTAKOS_DB** + **PITHOS_DB** + **CYCLADES_DB** |
53 | 77180645 | Dimitris Aragiorgis | |
54 | 21d3d487 | Constantinos Venetsanopoulos | the following independent roles: |
55 | 77180645 | Dimitris Aragiorgis | |
56 | bb535968 | Constantinos Venetsanopoulos | * **qh** = **QHOLDER** |
57 | 21d3d487 | Constantinos Venetsanopoulos | * **cms** = **CMS** |
58 | 21d3d487 | Constantinos Venetsanopoulos | * **mq** = **MQ** |
59 | 21d3d487 | Constantinos Venetsanopoulos | * **ns** = **NS** |
60 | 21d3d487 | Constantinos Venetsanopoulos | * **client** = **CLIENT** |
61 | 21d3d487 | Constantinos Venetsanopoulos | * **router**: The node to do any routing and NAT needed |
62 | 21d3d487 | Constantinos Venetsanopoulos | |
63 | 21d3d487 | Constantinos Venetsanopoulos | The above define the roles relative to the Synnefo components. However, in |
64 | 21d3d487 | Constantinos Venetsanopoulos | order to have instances up-and-running, at least one backend must be associated |
65 | bb535968 | Constantinos Venetsanopoulos | with Cyclades. Backends are Ganeti clusters, each with multiple **GANETI_NODE** |
66 | bb535968 | Constantinos Venetsanopoulos | s. Please note that these nodes may be the same as the ones used for the |
67 | bb535968 | Constantinos Venetsanopoulos | previous roles. To this end, `snf-deploy` also recognizes: |
68 | 21d3d487 | Constantinos Venetsanopoulos | |
69 | bb535968 | Constantinos Venetsanopoulos | * **cluster_nodes** = **G_BACKEND** = All available nodes of a specific backend |
70 | bb535968 | Constantinos Venetsanopoulos | * **master_node** = **GANETI_MASTER** |
71 | 21d3d487 | Constantinos Venetsanopoulos | |
72 | 21d3d487 | Constantinos Venetsanopoulos | Finally, it recognizes the group role: |
73 | 21d3d487 | Constantinos Venetsanopoulos | |
74 | 21d3d487 | Constantinos Venetsanopoulos | * **existing_nodes** = **SYNNEFO** + (N x **G_BACKEND**) |
75 | 21d3d487 | Constantinos Venetsanopoulos | |
76 | 21d3d487 | Constantinos Venetsanopoulos | In the future, `snf-deploy` will recognize all the independent roles of a scale |
77 | 21d3d487 | Constantinos Venetsanopoulos | out deployment as stated in the :ref:`scale up section <scale-up>`. When that's |
78 | bb535968 | Constantinos Venetsanopoulos | done, it won't need to introduce its own roles (stated here with lowercase) but |
79 | 21d3d487 | Constantinos Venetsanopoulos | rather use the scale out ones (stated with uppercase on the admin guide). |
80 | 21d3d487 | Constantinos Venetsanopoulos | |
81 | 21d3d487 | Constantinos Venetsanopoulos | |
82 | bb535968 | Constantinos Venetsanopoulos | .. _conf: |
83 | bb535968 | Constantinos Venetsanopoulos | |
84 | 21d3d487 | Constantinos Venetsanopoulos | Configuration (a) |
85 | 21d3d487 | Constantinos Venetsanopoulos | ================= |
86 | 21d3d487 | Constantinos Venetsanopoulos | |
87 | 21d3d487 | Constantinos Venetsanopoulos | All configuration of `snf-deploy` happens by editting the following files under |
88 | 21d3d487 | Constantinos Venetsanopoulos | ``/etc/snf-deploy``: |
89 | 77180645 | Dimitris Aragiorgis | |
90 | bb535968 | Constantinos Venetsanopoulos | ``nodes.conf`` |
91 | bb535968 | Constantinos Venetsanopoulos | -------------- |
92 | bb535968 | Constantinos Venetsanopoulos | |
93 | bb535968 | Constantinos Venetsanopoulos | This file reflects the hardware infrastucture on which Synnefo is going to be |
94 | bb535968 | Constantinos Venetsanopoulos | deployed and is the first to be set before running `snf-deploy`. |
95 | bb535968 | Constantinos Venetsanopoulos | |
96 | bb535968 | Constantinos Venetsanopoulos | Defines the nodes' hostnames and their IPs. Currently `snf-deploy` expects all |
97 | bb535968 | Constantinos Venetsanopoulos | nodes to reside in the same network subnet and domain, and share the same |
98 | bb535968 | Constantinos Venetsanopoulos | gateway and nameserver. Since Synnefo requires FQDNs to operate, a nameserver |
99 | bb535968 | Constantinos Venetsanopoulos | is going to be automatically setup in the cluster by `snf-deploy`. Thus, the |
100 | bb535968 | Constantinos Venetsanopoulos | nameserver's IP should appear among the defined node IPs. From now on, we will |
101 | bb535968 | Constantinos Venetsanopoulos | refer to the nodes with their hostnames. This implies their FQDN and their IP. |
102 | bb535968 | Constantinos Venetsanopoulos | |
103 | bb535968 | Constantinos Venetsanopoulos | Also, defines the nodes' authentication credentials (username, password). |
104 | bb535968 | Constantinos Venetsanopoulos | Furthermore, whether nodes have an extra disk (used for LVM/DRBD storage in |
105 | bb535968 | Constantinos Venetsanopoulos | Ganeti backends) or not. The VM container nodes should have three separate |
106 | bb535968 | Constantinos Venetsanopoulos | network interfaces (either physical or vlans) each in the same collision |
107 | bb535968 | Constantinos Venetsanopoulos | domain; one for the node's public network, one for VMs' public network and one |
108 | bb535968 | Constantinos Venetsanopoulos | for VMs' private networks. In order to support the most common case, a router |
109 | bb535968 | Constantinos Venetsanopoulos | is setup on the VMs' public interface and does NAT (hoping the node has itself |
110 | bb535968 | Constantinos Venetsanopoulos | internet access). |
111 | bb535968 | Constantinos Venetsanopoulos | |
112 | bb535968 | Constantinos Venetsanopoulos | The nodes defined in this file can reflect a number of physical nodes, on which |
113 | bb535968 | Constantinos Venetsanopoulos | you will deploy Synnefo (option (2)), or a number of virtual nodes which will |
114 | bb535968 | Constantinos Venetsanopoulos | get created by `snf-deploy` using KVM (option (1)), before deploying Synnefo. |
115 | bb535968 | Constantinos Venetsanopoulos | As we will see in the next sections, one should first set up this file and then |
116 | bb535968 | Constantinos Venetsanopoulos | tell `snf-deploy` whether the nodes on this file should be created, or treated |
117 | bb535968 | Constantinos Venetsanopoulos | as pre-existing. |
118 | bb535968 | Constantinos Venetsanopoulos | |
119 | bb535968 | Constantinos Venetsanopoulos | An example ``nodes.conf`` file looks like this: |
120 | bb535968 | Constantinos Venetsanopoulos | |
121 | bb535968 | Constantinos Venetsanopoulos | FIXME: example file here |
122 | bb535968 | Constantinos Venetsanopoulos | |
123 | bb535968 | Constantinos Venetsanopoulos | ``synnefo.conf`` |
124 | bb535968 | Constantinos Venetsanopoulos | ---------------- |
125 | bb535968 | Constantinos Venetsanopoulos | |
126 | bb535968 | Constantinos Venetsanopoulos | This file reflects the way Synnefo will be deployed on the nodes defined at |
127 | bb535968 | Constantinos Venetsanopoulos | ``nodes.conf``. |
128 | bb535968 | Constantinos Venetsanopoulos | |
129 | bb535968 | Constantinos Venetsanopoulos | The important section here is the roles. In this file we assing each of the |
130 | bb535968 | Constantinos Venetsanopoulos | roles described in the :ref:`introduction <snf-deploy>` to a specific node. The |
131 | bb535968 | Constantinos Venetsanopoulos | node is one of the nodes defined at ``nodes.conf``. Note that we refer to nodes |
132 | bb535968 | Constantinos Venetsanopoulos | with their short hostnames. |
133 | bb535968 | Constantinos Venetsanopoulos | |
134 | bb535968 | Constantinos Venetsanopoulos | Here we also define all credentials related to users needed by the various |
135 | bb535968 | Constantinos Venetsanopoulos | Synnefo services (database, RAPI, RabbitMQ) and the credentials of a test |
136 | bb535968 | Constantinos Venetsanopoulos | end-user (`snf-deploy` simulates a user signing up). |
137 | bb535968 | Constantinos Venetsanopoulos | |
138 | bb535968 | Constantinos Venetsanopoulos | Furthermore, define the Pithos shared directory which will hold all the Pithos |
139 | bb535968 | Constantinos Venetsanopoulos | related data (maps and blocks). |
140 | bb535968 | Constantinos Venetsanopoulos | |
141 | bb535968 | Constantinos Venetsanopoulos | Finally, define the name of the bridge interfaces controlled by Synnefo, and a |
142 | bb535968 | Constantinos Venetsanopoulos | testing Image to register after everything is up and running. |
143 | bb535968 | Constantinos Venetsanopoulos | |
144 | bb535968 | Constantinos Venetsanopoulos | An example ``setup.conf`` file (based on the previous ``nodes.conf`` example) |
145 | bb535968 | Constantinos Venetsanopoulos | looks like this: |
146 | bb535968 | Constantinos Venetsanopoulos | |
147 | bb535968 | Constantinos Venetsanopoulos | FIXME: example file here |
148 | bb535968 | Constantinos Venetsanopoulos | |
149 | bb535968 | Constantinos Venetsanopoulos | ``ganeti.conf`` |
150 | bb535968 | Constantinos Venetsanopoulos | --------------- |
151 | bb535968 | Constantinos Venetsanopoulos | |
152 | bb535968 | Constantinos Venetsanopoulos | This file reflects the way Ganeti clusters will be deployed on the nodes |
153 | bb535968 | Constantinos Venetsanopoulos | defined at ``nodes.conf``. |
154 | bb535968 | Constantinos Venetsanopoulos | |
155 | bb535968 | Constantinos Venetsanopoulos | Here we include all info with regard to Ganeti backends. That is: the master |
156 | bb535968 | Constantinos Venetsanopoulos | node, its floating IP, the volume group name (in case of LVM support) and the |
157 | bb535968 | Constantinos Venetsanopoulos | VMs' public network associated to it. Please note that currently Synnefo |
158 | bb535968 | Constantinos Venetsanopoulos | expects different public networks per backend but still can support multiple |
159 | bb535968 | Constantinos Venetsanopoulos | public networks per backend. |
160 | bb535968 | Constantinos Venetsanopoulos | |
161 | bb535968 | Constantinos Venetsanopoulos | FIXME: example file here |
162 | bb535968 | Constantinos Venetsanopoulos | |
163 | bb535968 | Constantinos Venetsanopoulos | ``deploy.conf`` |
164 | bb535968 | Constantinos Venetsanopoulos | --------------- |
165 | bb535968 | Constantinos Venetsanopoulos | |
166 | bb535968 | Constantinos Venetsanopoulos | This file customizes `snf-deploy` itself. |
167 | bb535968 | Constantinos Venetsanopoulos | |
168 | bb535968 | Constantinos Venetsanopoulos | It defines some needed directories and also includes options that have to do |
169 | bb535968 | Constantinos Venetsanopoulos | with the source of the packages to be deployed. Specifically, whether to deploy |
170 | bb535968 | Constantinos Venetsanopoulos | using local packages found under a local directory or deploy using an apt |
171 | bb535968 | Constantinos Venetsanopoulos | repository. If deploying from local packages, there is also an option to first |
172 | bb535968 | Constantinos Venetsanopoulos | download the packages from a custom URL and save them under the local directory |
173 | bb535968 | Constantinos Venetsanopoulos | for later use. |
174 | bb535968 | Constantinos Venetsanopoulos | |
175 | bb535968 | Constantinos Venetsanopoulos | FIXME: example file here |
176 | bb535968 | Constantinos Venetsanopoulos | |
177 | bb535968 | Constantinos Venetsanopoulos | ``vcluster.conf`` |
178 | bb535968 | Constantinos Venetsanopoulos | ----------------- |
179 | bb535968 | Constantinos Venetsanopoulos | |
180 | bb535968 | Constantinos Venetsanopoulos | This file defines options that are relevant to the virtual cluster creationi, if |
181 | bb535968 | Constantinos Venetsanopoulos | one chooses to create one. |
182 | bb535968 | Constantinos Venetsanopoulos | |
183 | bb535968 | Constantinos Venetsanopoulos | There is an option to define the URL of the Image that will be used as the host |
184 | bb535968 | Constantinos Venetsanopoulos | OS for the VMs of the virtual cluster. Also, options for defining an LVM space |
185 | bb535968 | Constantinos Venetsanopoulos | or a plain file to be used as a second disk. Finally, networking options to |
186 | bb535968 | Constantinos Venetsanopoulos | define where to bridge the virtual cluster. |
187 | 77180645 | Dimitris Aragiorgis | |
188 | 77180645 | Dimitris Aragiorgis | |
189 | bb535968 | Constantinos Venetsanopoulos | .. _vcluster: |
190 | bb535968 | Constantinos Venetsanopoulos | |
191 | 21d3d487 | Constantinos Venetsanopoulos | Virtual Cluster Creation (b) |
192 | 21d3d487 | Constantinos Venetsanopoulos | ============================ |
193 | 77180645 | Dimitris Aragiorgis | |
194 | bb535968 | Constantinos Venetsanopoulos | As stated in the introduction, `snf-deploy` gives you the ability to create a |
195 | bb535968 | Constantinos Venetsanopoulos | local virtual cluster using KVM and then deploy Synnefo on top of this cluster. |
196 | bb535968 | Constantinos Venetsanopoulos | The number of cluster nodes is arbitrary and is defined in ``nodes.conf``. |
197 | bb535968 | Constantinos Venetsanopoulos | |
198 | bb535968 | Constantinos Venetsanopoulos | This section describes the creation of the virtual cluster, on which Synnefo |
199 | bb535968 | Constantinos Venetsanopoulos | will be deployed in the :ref:`next section <inst>`. If you want to deploy |
200 | bb535968 | Constantinos Venetsanopoulos | Synnefo on existing physical nodes, you should skip this section. |
201 | bb535968 | Constantinos Venetsanopoulos | |
202 | bb535968 | Constantinos Venetsanopoulos | The first thing you need to deploy a virtual cluster, is a Debian Base image, |
203 | bb535968 | Constantinos Venetsanopoulos | which will be used to spawn the VMs. We already provide an 8GB Debian Squeeze |
204 | bb535968 | Constantinos Venetsanopoulos | Base image with preinstalled keys and network-manager hostname hooks. This |
205 | bb535968 | Constantinos Venetsanopoulos | resides on our production Pithos service. Please see the corresponding |
206 | bb535968 | Constantinos Venetsanopoulos | ``squeeze_image_url`` variable in ``vcluster.conf``. The image can be fetched |
207 | bb535968 | Constantinos Venetsanopoulos | by running: |
208 | 77180645 | Dimitris Aragiorgis | |
209 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
210 | 77180645 | Dimitris Aragiorgis | |
211 | 9bd540f5 | Dimitris Aragiorgis | snf-deploy vcluster image |
212 | 77180645 | Dimitris Aragiorgis | |
213 | bb535968 | Constantinos Venetsanopoulos | This will download the image from the URL defined at ``squeeez_image_url`` |
214 | bb535968 | Constantinos Venetsanopoulos | (Pithos by default) and save it locally under ``/var/lib/snf-deploy/images``. |
215 | 77180645 | Dimitris Aragiorgis | |
216 | bb535968 | Constantinos Venetsanopoulos | TODO: mention related options: --img-dir, --extra-disk, --lvg, --os |
217 | bb535968 | Constantinos Venetsanopoulos | |
218 | bb535968 | Constantinos Venetsanopoulos | Once you have the image, then you need to setup the local machine's networking |
219 | bb535968 | Constantinos Venetsanopoulos | appropriately. You can do this by running: |
220 | 77180645 | Dimitris Aragiorgis | |
221 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
222 | 77180645 | Dimitris Aragiorgis | |
223 | 9bd540f5 | Dimitris Aragiorgis | snf-deploy vcluster network |
224 | 77180645 | Dimitris Aragiorgis | |
225 | bb535968 | Constantinos Venetsanopoulos | This will add a bridge (defined with the ``bridge`` option inside |
226 | bb535968 | Constantinos Venetsanopoulos | ``vcluster.conf``), iptables to allow traffic from/to the cluster, and enable |
227 | bb535968 | Constantinos Venetsanopoulos | forwarding and NAT for the selected network subnet (defined inside |
228 | bb535968 | Constantinos Venetsanopoulos | ``nodes.conf`` in the ``subnet`` option). |
229 | 77180645 | Dimitris Aragiorgis | |
230 | bb535968 | Constantinos Venetsanopoulos | To complete the preparation, you need a DHCP server that will provide the |
231 | bb535968 | Constantinos Venetsanopoulos | selected hostnames and IPs to the cluster (defined under ``[ips]`` in |
232 | bb535968 | Constantinos Venetsanopoulos | ``nodes.conf``). To do so, run: |
233 | 77180645 | Dimitris Aragiorgis | |
234 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
235 | 77180645 | Dimitris Aragiorgis | |
236 | 9bd540f5 | Dimitris Aragiorgis | snf-deploy vcluster dhcp |
237 | 77180645 | Dimitris Aragiorgis | |
238 | bb535968 | Constantinos Venetsanopoulos | This will launch a dnsmasq instance, acting only as DHCP server and listening |
239 | bb535968 | Constantinos Venetsanopoulos | only on the cluster's bridge. Every time you make changes inside ``nodes.conf`` |
240 | bb535968 | Constantinos Venetsanopoulos | you should re-create the dnsmasq related files (under ``/etc/snf-deploy``) by |
241 | bb535968 | Constantinos Venetsanopoulos | passing --save-config option. |
242 | 77180645 | Dimitris Aragiorgis | |
243 | bb535968 | Constantinos Venetsanopoulos | After running all the above preparation tasks we can finally create the cluster |
244 | bb535968 | Constantinos Venetsanopoulos | defined in ``nodes.conf`` by running: |
245 | 77180645 | Dimitris Aragiorgis | |
246 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
247 | 77180645 | Dimitris Aragiorgis | |
248 | bb535968 | Constantinos Venetsanopoulos | snf-deploy vcluster create |
249 | bb535968 | Constantinos Venetsanopoulos | |
250 | bb535968 | Constantinos Venetsanopoulos | This will launch all the needed KVM virtual machines, snapshotting the image we |
251 | bb535968 | Constantinos Venetsanopoulos | fetched before. Their taps will be connected with the already created bridge |
252 | bb535968 | Constantinos Venetsanopoulos | and their primary interface will get the given address. |
253 | 77180645 | Dimitris Aragiorgis | |
254 | bb535968 | Constantinos Venetsanopoulos | Now that we have the nodes ready, we can move on and deploy Synnefo on them. |
255 | 77180645 | Dimitris Aragiorgis | |
256 | 77180645 | Dimitris Aragiorgis | |
257 | bb535968 | Constantinos Venetsanopoulos | .. _inst: |
258 | bb535968 | Constantinos Venetsanopoulos | |
259 | 21d3d487 | Constantinos Venetsanopoulos | Synnefo Installation (c) |
260 | 21d3d487 | Constantinos Venetsanopoulos | ======================== |
261 | 21d3d487 | Constantinos Venetsanopoulos | |
262 | bb535968 | Constantinos Venetsanopoulos | At this point you should have an up-and-running cluster, either virtual |
263 | 22a8a5e1 | Dimitris Aragiorgis | (created in the :ref:`previous section <vcluster>` on your local machine) or |
264 | bb535968 | Constantinos Venetsanopoulos | physical on remote nodes. The cluster should also have valid hostnames and IPs. |
265 | bb535968 | Constantinos Venetsanopoulos | And all its nodes should be defined in ``nodes.conf``. |
266 | bb535968 | Constantinos Venetsanopoulos | |
267 | bb535968 | Constantinos Venetsanopoulos | You should also have set up ``synnefo.conf`` to reflect which Synnefo component |
268 | bb535968 | Constantinos Venetsanopoulos | will reside in which node. |
269 | bb535968 | Constantinos Venetsanopoulos | |
270 | 22a8a5e1 | Dimitris Aragiorgis | Node Requirements |
271 | 22a8a5e1 | Dimitris Aragiorgis | ----------------- |
272 | 22a8a5e1 | Dimitris Aragiorgis | |
273 | 22a8a5e1 | Dimitris Aragiorgis | - OS: Debian Squeeze |
274 | 22a8a5e1 | Dimitris Aragiorgis | - authentication: `root` with same password for all nodes |
275 | 22a8a5e1 | Dimitris Aragiorgis | - primary network interface: `eth0` |
276 | 22a8a5e1 | Dimitris Aragiorgis | - primary IP in the same IPv4 subnet and network domain |
277 | 22a8a5e1 | Dimitris Aragiorgis | - spare network interfaces: `eth1`, `eth2` (or vlans on `eth0`) |
278 | 22a8a5e1 | Dimitris Aragiorgis | |
279 | 22a8a5e1 | Dimitris Aragiorgis | In case you have created a virtual cluster as described in the :ref:`section |
280 | 22a8a5e1 | Dimitris Aragiorgis | (b) <vcluster>`, the above requirements are already taken care of. In case of a |
281 | 22a8a5e1 | Dimitris Aragiorgis | physical cluster, you need to set them up manually by yourself, before |
282 | 22a8a5e1 | Dimitris Aragiorgis | proceeding with the Synnefo installation. |
283 | 22a8a5e1 | Dimitris Aragiorgis | |
284 | 22a8a5e1 | Dimitris Aragiorgis | Preparing the Synnefo deployment |
285 | 22a8a5e1 | Dimitris Aragiorgis | -------------------------------- |
286 | 22a8a5e1 | Dimitris Aragiorgis | |
287 | 22a8a5e1 | Dimitris Aragiorgis | The following actions are mandatory and must run before the actual deployment. |
288 | 22a8a5e1 | Dimitris Aragiorgis | In the following we refer to the sub commands of ``snf-deploy prepare`` and |
289 | 22a8a5e1 | Dimitris Aragiorgis | what they actually do. |
290 | 77180645 | Dimitris Aragiorgis | |
291 | bb535968 | Constantinos Venetsanopoulos | Synnefo expects FQDNs and therefore a nameserver (BIND) should be setup in a |
292 | bb535968 | Constantinos Venetsanopoulos | node inside the cluster. All nodes along with your local machine should use |
293 | bb535968 | Constantinos Venetsanopoulos | this nameserver and search in the corresponding network domain. To this end, |
294 | bb535968 | Constantinos Venetsanopoulos | add to your local ``resolv.conf`` (please change the default values with the |
295 | bb535968 | Constantinos Venetsanopoulos | ones of your custom configuration): |
296 | 77180645 | Dimitris Aragiorgis | |
297 | bb535968 | Constantinos Venetsanopoulos | .. code-block:: console |
298 | bb535968 | Constantinos Venetsanopoulos | |
299 | bb535968 | Constantinos Venetsanopoulos | search <your_domain> synnefo.deploy.local |
300 | bb535968 | Constantinos Venetsanopoulos | nameserver 192.168.0.1 |
301 | 77180645 | Dimitris Aragiorgis | |
302 | 22a8a5e1 | Dimitris Aragiorgis | WARNING: In case you are running the installation on physical nodes please |
303 | 22a8a5e1 | Dimitris Aragiorgis | ensure that they have the same `resolv.conf` and it does not change during |
304 | 22a8a5e1 | Dimitris Aragiorgis | and after installation (because of NetworkManager hooks or something..) |
305 | 22a8a5e1 | Dimitris Aragiorgis | |
306 | bb535968 | Constantinos Venetsanopoulos | To actually setup the nameserver in the node specified as ``ns`` in |
307 | bb535968 | Constantinos Venetsanopoulos | ``synnefo.conf`` run: |
308 | 77180645 | Dimitris Aragiorgis | |
309 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
310 | 77180645 | Dimitris Aragiorgis | |
311 | 22a8a5e1 | Dimitris Aragiorgis | snf-deploy prepare ns |
312 | 77180645 | Dimitris Aragiorgis | |
313 | 22a8a5e1 | Dimitris Aragiorgis | To do some node tweaking and install correct `id_rsa/dsa` keys and `authorized_keys` |
314 | 22a8a5e1 | Dimitris Aragiorgis | needed for password-less intra-node communication run: |
315 | 77180645 | Dimitris Aragiorgis | |
316 | 22a8a5e1 | Dimitris Aragiorgis | .. code-block:: console |
317 | 77180645 | Dimitris Aragiorgis | |
318 | 22a8a5e1 | Dimitris Aragiorgis | snf-deploy prepare hosts |
319 | 77180645 | Dimitris Aragiorgis | |
320 | 22a8a5e1 | Dimitris Aragiorgis | At this point you should have a cluster with FQDNs and reverse DNS lookups |
321 | 22a8a5e1 | Dimitris Aragiorgis | ready for the Synnefo deployment. To sum up, we mention all the node |
322 | 22a8a5e1 | Dimitris Aragiorgis | requirements for a successful Synnefo installation, before proceeding. |
323 | 77180645 | Dimitris Aragiorgis | |
324 | bb535968 | Constantinos Venetsanopoulos | To check the network configuration (FQDNs, connectivity): |
325 | 77180645 | Dimitris Aragiorgis | |
326 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
327 | 77180645 | Dimitris Aragiorgis | |
328 | 22a8a5e1 | Dimitris Aragiorgis | snf-deploy prepare check |
329 | bb535968 | Constantinos Venetsanopoulos | |
330 | bb535968 | Constantinos Venetsanopoulos | WARNING: In case ping fails check ``/etc/nsswitch.conf`` hosts entry and put dns |
331 | bb535968 | Constantinos Venetsanopoulos | after files!!! |
332 | 77180645 | Dimitris Aragiorgis | |
333 | 0c29b5ba | Dimitris Aragiorgis | To setup the apt repository and update each nodes' package index files: |
334 | 0c29b5ba | Dimitris Aragiorgis | |
335 | 0c29b5ba | Dimitris Aragiorgis | .. code-block:: console |
336 | 0c29b5ba | Dimitris Aragiorgis | |
337 | 22a8a5e1 | Dimitris Aragiorgis | snf-deploy prepare apt |
338 | 22a8a5e1 | Dimitris Aragiorgis | |
339 | 22a8a5e1 | Dimitris Aragiorgis | Finally Synnefo needs a shared file system, so we need to setup the NFS server |
340 | 22a8a5e1 | Dimitris Aragiorgis | on node ``pithos`` defined in ``synnefo.conf``: |
341 | 22a8a5e1 | Dimitris Aragiorgis | |
342 | 22a8a5e1 | Dimitris Aragiorgis | .. code-block:: console |
343 | 22a8a5e1 | Dimitris Aragiorgis | |
344 | 22a8a5e1 | Dimitris Aragiorgis | snf-deploy prepare nfs |
345 | 0c29b5ba | Dimitris Aragiorgis | |
346 | bb535968 | Constantinos Venetsanopoulos | If everything is setup correctly and all prerequisites are met, we can start |
347 | bb535968 | Constantinos Venetsanopoulos | the Synnefo deployment. |
348 | 77180645 | Dimitris Aragiorgis | |
349 | bb535968 | Constantinos Venetsanopoulos | Synnefo deployment |
350 | bb535968 | Constantinos Venetsanopoulos | ------------------ |
351 | bb535968 | Constantinos Venetsanopoulos | |
352 | bb535968 | Constantinos Venetsanopoulos | To install the Synnefo stack on the existing cluster run: |
353 | 77180645 | Dimitris Aragiorgis | |
354 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
355 | 77180645 | Dimitris Aragiorgis | |
356 | 77180645 | Dimitris Aragiorgis | snf-deploy synnefo -vvv |
357 | 77180645 | Dimitris Aragiorgis | |
358 | bb535968 | Constantinos Venetsanopoulos | This might take a while. |
359 | 77180645 | Dimitris Aragiorgis | |
360 | bb535968 | Constantinos Venetsanopoulos | If this finishes without errors, check for successful installation by visiting |
361 | bb535968 | Constantinos Venetsanopoulos | from your local machine (make sure you have already setup your local |
362 | bb535968 | Constantinos Venetsanopoulos | ``resolv.conf`` to point at the cluster's DNS): |
363 | 77180645 | Dimitris Aragiorgis | |
364 | 77180645 | Dimitris Aragiorgis | | https://accounts.synnefo.deploy.local/im/ |
365 | 77180645 | Dimitris Aragiorgis | |
366 | 77180645 | Dimitris Aragiorgis | and login with: |
367 | 77180645 | Dimitris Aragiorgis | |
368 | 77180645 | Dimitris Aragiorgis | | username: dimara@grnet.gr password: lala |
369 | 77180645 | Dimitris Aragiorgis | |
370 | bb535968 | Constantinos Venetsanopoulos | or the ``user_name`` and ``user_passwd`` defined in your ``synnefo.conf``. |
371 | bb535968 | Constantinos Venetsanopoulos | Take a small tour checking out Pithos and the rest of the Web UI. You can |
372 | bb535968 | Constantinos Venetsanopoulos | upload a sample file on Pithos to see that Pithos is working. Do not try to |
373 | bb535968 | Constantinos Venetsanopoulos | create a VM yet, since we have not yet added a Ganeti backend. |
374 | bb535968 | Constantinos Venetsanopoulos | |
375 | bb535968 | Constantinos Venetsanopoulos | If everything seems to work, we go ahead to the last step which is adding a |
376 | bb535968 | Constantinos Venetsanopoulos | Ganeti backend. |
377 | 77180645 | Dimitris Aragiorgis | |
378 | 77180645 | Dimitris Aragiorgis | Adding a Ganeti Backend |
379 | 21d3d487 | Constantinos Venetsanopoulos | ----------------------- |
380 | 77180645 | Dimitris Aragiorgis | |
381 | bb535968 | Constantinos Venetsanopoulos | Assuming that everything works as expected, you must have Astakos, Pithos, CMS, |
382 | bb535968 | Constantinos Venetsanopoulos | DB and RabbitMQ up and running. Cyclades should work too, but partially. That's |
383 | bb535968 | Constantinos Venetsanopoulos | because no backend is registered yet. Let's setup one. Currently, Synnefo |
384 | bb535968 | Constantinos Venetsanopoulos | supports only Ganeti clusters as valid backends. They have to be created |
385 | bb535968 | Constantinos Venetsanopoulos | independently with `snf-deploy` and once they are up and running, we register |
386 | bb535968 | Constantinos Venetsanopoulos | them to Cyclades. From version 0.12, Synnefo supports multiple Ganeti backends. |
387 | bb535968 | Constantinos Venetsanopoulos | `snf-deploy` defines them in ``ganeti.conf``. |
388 | 77180645 | Dimitris Aragiorgis | |
389 | bb535968 | Constantinos Venetsanopoulos | After setting up ``ganeti.conf``, run: |
390 | 77180645 | Dimitris Aragiorgis | |
391 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
392 | 77180645 | Dimitris Aragiorgis | |
393 | bb535968 | Constantinos Venetsanopoulos | snf-deploy backend create --backend-name ganeti1 -vvv |
394 | bb535968 | Constantinos Venetsanopoulos | |
395 | bb535968 | Constantinos Venetsanopoulos | where ``ganeti1`` should have previously been defined as a section in |
396 | bb535968 | Constantinos Venetsanopoulos | ``ganeti.conf``. This will create the ``ganeti1`` backend on the corresponding |
397 | bb535968 | Constantinos Venetsanopoulos | nodes (``cluster_nodes``, ``master_node``) defined in the ``ganeti1`` section |
398 | bb535968 | Constantinos Venetsanopoulos | of the ``ganeti.conf`` file. If you are an experienced user and want to deploy |
399 | bb535968 | Constantinos Venetsanopoulos | more than one Ganeti backend you should create multiple sections in |
400 | bb535968 | Constantinos Venetsanopoulos | ``ganeti.conf`` and re-run the above command with the corresponding backend |
401 | bb535968 | Constantinos Venetsanopoulos | names. |
402 | 77180645 | Dimitris Aragiorgis | |
403 | bb535968 | Constantinos Venetsanopoulos | After creating and adding the Ganeti backend, we need to setup the backend |
404 | bb535968 | Constantinos Venetsanopoulos | networking. To do so, we run: |
405 | 77180645 | Dimitris Aragiorgis | |
406 | bb535968 | Constantinos Venetsanopoulos | .. code-block:: console |
407 | bb535968 | Constantinos Venetsanopoulos | |
408 | 9bd540f5 | Dimitris Aragiorgis | snf-deploy backend network --backend-name ganeti1 |
409 | bb535968 | Constantinos Venetsanopoulos | |
410 | bb535968 | Constantinos Venetsanopoulos | And finally, we need to setup the backend storage: |
411 | 77180645 | Dimitris Aragiorgis | |
412 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
413 | 77180645 | Dimitris Aragiorgis | |
414 | 9bd540f5 | Dimitris Aragiorgis | snf-deploy backend storage --backend-name ganeti1 |
415 | 77180645 | Dimitris Aragiorgis | |
416 | bb535968 | Constantinos Venetsanopoulos | This command will first check the ``extra_disk`` in ``nodes.conf`` and try to |
417 | bb535968 | Constantinos Venetsanopoulos | find it on the nodes of the cluster. If the nodes indeed have that disk, |
418 | bb535968 | Constantinos Venetsanopoulos | `snf-deploy` will create a PV and the corresponding VG and will enable LVM and |
419 | bb535968 | Constantinos Venetsanopoulos | DRBD storage in the Ganeti cluster. |
420 | bb535968 | Constantinos Venetsanopoulos | |
421 | bb535968 | Constantinos Venetsanopoulos | If the option is blank or `snf-deploy` can't find the disk on the nodes, LVM |
422 | bb535968 | Constantinos Venetsanopoulos | and DRBD will be disabled and only Ganeti's ``file`` disk template will be |
423 | bb535968 | Constantinos Venetsanopoulos | enabled. |
424 | bb535968 | Constantinos Venetsanopoulos | |
425 | bb535968 | Constantinos Venetsanopoulos | To test everything went as expected, visit from your local machine: |
426 | 77180645 | Dimitris Aragiorgis | |
427 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
428 | 77180645 | Dimitris Aragiorgis | |
429 | 77180645 | Dimitris Aragiorgis | https://cyclades.synnefo.deploy.local/ui/ |
430 | 77180645 | Dimitris Aragiorgis | |
431 | bb535968 | Constantinos Venetsanopoulos | and try to create a VM. Also create a Private Network and try to connect it. If |
432 | bb535968 | Constantinos Venetsanopoulos | everything works, you have setup Synnefo successfully. Enjoy! |
433 | 77180645 | Dimitris Aragiorgis | |
434 | 77180645 | Dimitris Aragiorgis | |
435 | 21d3d487 | Constantinos Venetsanopoulos | snf-deploy as a DevTool |
436 | 21d3d487 | Constantinos Venetsanopoulos | ======================= |
437 | 77180645 | Dimitris Aragiorgis | |
438 | bb535968 | Constantinos Venetsanopoulos | For developers, a single node setup is highly recommended and `snf-deploy` is a |
439 | bb535968 | Constantinos Venetsanopoulos | very helpful tool. `snf-deploy` also supports updating packages that are |
440 | bb535968 | Constantinos Venetsanopoulos | locally generated. For this to work please add all \*.deb files in packages |
441 | bb535968 | Constantinos Venetsanopoulos | directory (see ``deploy.conf``) and set the ``use_local_packages`` option to |
442 | bb535968 | Constantinos Venetsanopoulos | ``True``. Then run: |
443 | 77180645 | Dimitris Aragiorgis | |
444 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
445 | 77180645 | Dimitris Aragiorgis | |
446 | 22a8a5e1 | Dimitris Aragiorgis | snf-deploy synnefo update --use-local-packages |
447 | 22a8a5e1 | Dimitris Aragiorgis | snf-deploy backend update --backend-name ganeti2 --use-local-packages |
448 | 77180645 | Dimitris Aragiorgis | |
449 | bb535968 | Constantinos Venetsanopoulos | For advanced users, `snf-deploy` gives the ability to run one or more times |
450 | bb535968 | Constantinos Venetsanopoulos | independently some of the supported actions. To find out which are those, run: |
451 | 77180645 | Dimitris Aragiorgis | |
452 | 77180645 | Dimitris Aragiorgis | .. code-block:: console |
453 | 77180645 | Dimitris Aragiorgis | |
454 | bb535968 | Constantinos Venetsanopoulos | snf-deploy run --help |