root / docs / networks.rst @ 30fe9049
History | View | Annotate | Download (16.9 kB)
1 |
.. _networks: |
---|---|
2 |
|
3 |
Network Service (part of Cyclades) |
4 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
5 |
|
6 |
Network setup overview |
7 |
====================== |
8 |
|
9 |
Currently the Network Service is part of Cyclades and comes along with the |
10 |
Cyclades software components. |
11 |
|
12 |
Networking is deployment-specific and must be customized based on the specific |
13 |
needs of the system administrator. However, to do so, the administrator needs |
14 |
to understand how each level handles Virtual Networks, to be able to setup the |
15 |
backend appropriately. |
16 |
|
17 |
Since v0.11 Synnefo supports multiple Ganeti clusters (backends). Having in |
18 |
mind that every backend has its locality, there is a high possibility each |
19 |
cluster to have different infrastracture (wires, routers, subnets, gateways, |
20 |
etc.). |
21 |
|
22 |
In the following sections we investigate in a top-down approach, the way |
23 |
networks are defined from the Cyclades, Ganeti, and Backend persperctive. |
24 |
|
25 |
Network @ Cyclades level |
26 |
------------------------ |
27 |
|
28 |
Cyclades understands two types of Virtual Networks: |
29 |
|
30 |
a) Public Networks |
31 |
b) Private Networks |
32 |
|
33 |
Public Networks are created by the administrator via `snf-manage` commands |
34 |
and can be used by all end-users. Each public network is assigned to a |
35 |
single backend but one backend can have multiple public networks. |
36 |
|
37 |
Private Networks are created by the end-user from the Web UI or the kamaki |
38 |
client and provide isolated Layer 2 connectivity to the end-user. With regard |
39 |
to the fact that a user's VMs may be allocated across different Ganeti clusters |
40 |
(backends), private networks are created in all backends to ensure VMs |
41 |
connectivity. |
42 |
|
43 |
Both types of networks are created dynamically. |
44 |
|
45 |
From the VM perspective, each NIC is attached to a specific Network. |
46 |
|
47 |
When a new VM is created the backend allocator (in Cyclades) decides in which |
48 |
backend to spawn it. Depending on the chosen backend, Synnefo finds the first |
49 |
non-full public Network that exists in the backend. Then attaches the VM's |
50 |
first NIC to this network. |
51 |
|
52 |
Once the VM is created, the user is able to connect the VM to multiple |
53 |
private networks, that himself has already created. |
54 |
|
55 |
A Network can have the following attributes: |
56 |
|
57 |
- IPv4 subnet (mandatory) |
58 |
- IPv4 gateway |
59 |
- IPv6 subnet |
60 |
- IPv6 gateway |
61 |
- public/private flag |
62 |
- flavor |
63 |
|
64 |
Flavor is a way to abstact infrastructure specific options, that are used to |
65 |
ensure connectivity and isolation to the VMs connected to the network. It is a |
66 |
set of options that eventually will guide scripts to set up rules, while |
67 |
creating virtual interfaces in the node level. The available flavors and their |
68 |
options can be found in the Synnefo settings and are configurable. |
69 |
|
70 |
To ensure L2 isolation, Synnefo supports two different mechanisms (see also Node |
71 |
Level section): |
72 |
|
73 |
- assigning one physical VLAN per network |
74 |
- assigning one MAC prefix per network, so that every NIC attached to this |
75 |
network will have this prefix. Isolation is then achieved by filtering |
76 |
rules (via `ebtables`) based on a specific mask (ff:ff:f0:00:00:00, see Node |
77 |
Level section for more details). |
78 |
|
79 |
Having this in mind and in order to prevent assignment of duplicate VLAN/MAC |
80 |
prefix to different networks, Synnefo supports two types of Pools: |
81 |
|
82 |
- Bridge Pool (corresponding to a number of VLANs bridged to those bridges) |
83 |
- MAC prefix Pool |
84 |
|
85 |
For Pool handling refer to the corresponding doc section. |
86 |
|
87 |
Finally, each supported flavor must declare the following options (see also |
88 |
Ganeti Level section): |
89 |
|
90 |
- ``mode`` ('bridged' or 'routed'), |
91 |
- ``link`` ('br100', 'rt200', 'pool') |
92 |
- ``mac_prefix`` ('aa:00:05', 'pool', None) |
93 |
- ``tags`` (['ip-less-routed' or 'mac-filtered' or 'physical-vlan' or None]) |
94 |
|
95 |
Existing network flavors are the following: |
96 |
|
97 |
============== ======= =============================== ====================== ================== |
98 |
Flavor Name Mode Link MAC prefix Tags |
99 |
============== ======= =============================== ====================== ================== |
100 |
IP_LESS_ROUTED routed ``DEFAULT_ROUTING_TABLE`` ``DEFAULT_MAC_PREFIX`` 'ip-less-routed' |
101 |
MAC_FILTERED bridged ``DEFAULT_MAC_FILTERED_BRIDGE`` 'pool' 'private'filtered' |
102 |
PHYSICAL_VLAN bridged 'pool' ``DEFAULT_MAC_PREFIX`` 'physical-vlan' |
103 |
CUSTOM bridged ``DEFAULT_BRIDGE`` ``DEFAULT_MAC_PREFIX`` |
104 |
============== ======= =============================== ====================== ================== |
105 |
|
106 |
``DEFAULT_ROUTING_TABLE``, ``DEFAULT_MAC_PREFIX``, ``DEFAULT_BRIDGE``, ``DEFAULT_MAC_FILTERED_BRIDGE`` |
107 |
are all configurable settings in ``/etc/synnefo/20-snf-cyclades-app-api.conf``. 'pool' is used |
108 |
to denote that a link or MAC prefix will be allocated from the corresponging Pool. |
109 |
|
110 |
The administrator is able to create any of the above flavors |
111 |
and override their default values by explicitly passing mode, link, etc. using |
112 |
the `snf-manage network-create` command. |
113 |
|
114 |
The end-user is allowed to create only networks of flavor ``MAC_FILTERED`` and |
115 |
``PHYSICAL_VLAN``. Currently, only ``MAC_FILTERED`` and ``PHYSICAL_VLAN`` can |
116 |
use existing pools and cannot be overriden. |
117 |
|
118 |
Network @ Ganeti level |
119 |
---------------------- |
120 |
|
121 |
Currently, stable Ganeti does not support IP Pool management. However, the |
122 |
functionality has been merged in the official Ganeti master branch and will |
123 |
appear on Ganeti 2.7.0. So, you can either checkout the Ganeti master branch |
124 |
and build your packages, or clone our local repo |
125 |
https://code.grnet.gr/git/ganeti-local and checkout the |
126 |
`stable-2.6-ippool-hotplug-esi` branch. This is the Ganeti stable branch with |
127 |
IP pool management, Hotplugging and ExtStorage Interface features merged on top |
128 |
of it. The last two features are not a hard Synnefo requirement, but will |
129 |
enable you to do neat things when you get experienced with Synnefo. They are |
130 |
going to be pushed for review upstream sometime soon. |
131 |
|
132 |
Any network created in Synnefo is also created in one (for public networks) or |
133 |
all (for private networks) Ganeti backends. In Ganeti a network can have the |
134 |
following options: |
135 |
|
136 |
- network (192.168.0.0/24, mandatory) |
137 |
- gateway (192.168.0.1) |
138 |
- network6 (2001:648:2ffc:1201::/64) |
139 |
- gateway6 (2001:648:2ffc:1201::1) |
140 |
- mac_prefix (aa:00:01) |
141 |
- type (private, public) |
142 |
- tags |
143 |
|
144 |
Networks in Ganeti cannot be used unless they are connected to a nodegroup in |
145 |
order to define the connectivity mode and link. Synnefo, after creating a |
146 |
network, connects it to all nodegroups of the Ganeti cluster(s) with the given |
147 |
mode and link (defined in the network flavor). |
148 |
|
149 |
Ganeti makes use of environment variables to inform scripts about each NIC's |
150 |
setup. `kvm-vif-script` that comes with `snf-network` sets up the nfdhcpd lease and |
151 |
applies any rules needed depending on the network's mode, link, mac_prefix and |
152 |
tags. |
153 |
|
154 |
Network @ Physical host level |
155 |
----------------------------- |
156 |
|
157 |
Currently, networking infrastructure must be pre-provisioned before creating |
158 |
networks in Synnefo. According to which flavors you want to support, you should |
159 |
have already setup all your physical hosts correspondingly. This means you |
160 |
need: |
161 |
|
162 |
- one bridge for the ``CUSTOM`` flavor (br0, see Fig. 1) |
163 |
- one bridge for the ``MAC_FILTERED`` flavor (prv0, see Fig. 2) |
164 |
- a number of bridges and their corresponding VLANs (bridged to them) for |
165 |
the ``PHYSICAL_VLAN`` flavor (prv1..prv100, see Fig. 3) |
166 |
- a routing table for the ``IP_LESS_ROUTED`` flavor (snf_public, see Fig. 4) |
167 |
|
168 |
Please refer to the following figures, which clarify each infrastructure setup |
169 |
and how connectivity and isolation is achieved in every case for every type of |
170 |
network. |
171 |
|
172 |
|
173 |
FLAVORS |
174 |
======= |
175 |
|
176 |
As mentioned earlier supported flavors are: |
177 |
|
178 |
- CUSTOM |
179 |
- IP_LESS_ROUTED |
180 |
- MAC_FILTERED |
181 |
- PHYSICAL_VLAN |
182 |
|
183 |
In the following sections we mention what configuration imposes each flavor from |
184 |
Synnefo, Ganeti and Physical host perspective. |
185 |
|
186 |
|
187 |
|
188 |
DEFAULT SCENARIO |
189 |
---------------- |
190 |
|
191 |
In this case we will bridge all primary interfaces of the VMs on one bridge that must |
192 |
be the same collition domain with the router. The router sould then forward packets |
193 |
(if a public IPv4 Subnet is available) or do NAT in order to provide internet access to |
194 |
the VMs. |
195 |
|
196 |
To this end we will use the CUSTOM flavor and pre-provision in each Ganeti |
197 |
node one bridge (e.g. ``br100``). If we assume that ``eth1`` is the physical interface |
198 |
connected to the router, run: |
199 |
|
200 |
.. image:: images/network-bridged.png |
201 |
:align: right |
202 |
:height: 550px |
203 |
:width: 500px |
204 |
|
205 |
.. code-block:: console |
206 |
|
207 |
# brctl addbr br100 |
208 |
# brctl addif br100 eth1 |
209 |
# ip link set br100 up |
210 |
|
211 |
# brctl show |
212 |
bridge name bridge id STP enabled interfaces |
213 |
br100 8000.8a3c3ede3583 no eth1 |
214 |
|
215 |
|
216 |
|
217 |
Then in Cyclades run: |
218 |
|
219 |
.. code-block:: console |
220 |
|
221 |
# snf-manage network-create --subnet=5.6.7.0/27 --gateway=5.6.7.1 --subnet6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --public --dhcp --flavor=CUSTOM --link=br100 ----name=default --backend-id=1 |
222 |
|
223 |
# snf-manage network-list |
224 |
id name flavor owner mac_prefix dhcp state link vms public IPv4 Subnet IPv4 Gateway |
225 |
1 default CUSTOM True ACTIVE br100 True 5.6.7.0/27 5.6.7.1 |
226 |
|
227 |
This will add a network in Synnefo DB and create a network in Ganeti backend by |
228 |
issuing: |
229 |
|
230 |
.. code-block:: console |
231 |
|
232 |
# gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --network-type=public --tags=nfdhcpd snf-net-1 |
233 |
|
234 |
# gnt-network connect snf-net-1 default bridged br100 |
235 |
# gnt-network list snf-net-1 |
236 |
Network Subnet Gateway NetworkType MacPrefix GroupList Tags |
237 |
snf-net-1 5.6.7.0/27 5.6.7.1 public None default(bridged, br100) nfdhcpd |
238 |
|
239 |
|
240 |
To enable NAT in a Internal Router if you do not have a public IP range available |
241 |
but only a public routable IP (e.g 1.2.3.4): |
242 |
|
243 |
.. code-block:: console |
244 |
|
245 |
# ip addr add 5.6.7.1/27 dev eth1 |
246 |
# iptables -t nat -A POSTROUTING -o eth1 --to-source 1.2.3.4 -j SNAT |
247 |
|
248 |
IP_LESS_ROUTED |
249 |
-------------- |
250 |
|
251 |
.. image:: images/network-routed.png |
252 |
:align: right |
253 |
:height: 580px |
254 |
:width: 500px |
255 |
|
256 |
To create a network with IP_LESS_ROUTED flavor run you have to pre-provision in |
257 |
each Ganeti node one routing table (e.g. ``snf_public``) that will do all the |
258 |
routing from/to the VMs' taps. Additionally you must enable ``Proxy-ARP`` |
259 |
support. All traffic will be on a single iterface (e.g. ``eth1``). |
260 |
|
261 |
.. code-block:: console |
262 |
|
263 |
# echo 1 > /proc/sys/net/ipv4/conf/ip_fowarding |
264 |
# echo 10 snf_public >> /etc/iproute2/rt_tables |
265 |
# ip route add 5.6.7.0/27 dev eth1 |
266 |
# ip route add 5.6.7.0/27 dev eth1 table snf_public |
267 |
# ip route add default via 5.6.7.1 dev eth1 table snf_public |
268 |
# ip rule add iif eth1 lookup snf_public |
269 |
# arptables -A OUTPUT -o eth1 --opcode 1 --mangle-ip-s 5.6.7.30 # last ip in Subnet |
270 |
|
271 |
Then in Cyclades run: |
272 |
|
273 |
.. code-block:: console |
274 |
|
275 |
# snf-manage network-create --subnet=5.6.7.0/27 --gateway=5.6.7.1 --subnet6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --public --dhcp --flavor=IP_LESS_ROUTED --name=routed --backend-id=1 |
276 |
|
277 |
# snf-manage network-list |
278 |
id name flavor owner mac_prefix dhcp state link vms public IPv4 Subnet IPv4 Gateway |
279 |
2 routed IP_LESS_ROUTED True ACTIVE snf_public True 5.6.7.0/27 5.6.7.1 |
280 |
|
281 |
|
282 |
This will add a network in Synnefo DB and create a network in Ganeti backend by |
283 |
issuing: |
284 |
|
285 |
.. code-block:: console |
286 |
|
287 |
# gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --network-type=public --tags=nfdhcpd,ip-less-routed snf-net-2 |
288 |
|
289 |
# gnt-network connect snf-net-2 default bridged br100 |
290 |
# gnt-network list snf-net-2 |
291 |
Network Subnet Gateway NetworkType MacPrefix GroupList Tags |
292 |
dimara-net-1 62.217.123.128/27 62.217.123.129 public None default(routed, snf_public) nfdhcpd,ip-less-routed |
293 |
|
294 |
|
295 |
|
296 |
|
297 |
MAC_FILTERED |
298 |
------------ |
299 |
|
300 |
|
301 |
To create a network with MAC_FILTERED flavor you have to pre-provision in each Ganeti |
302 |
node one bridge (e.g. ``prv0``) that will be bridged with one interface (e.g. ``eth2``) |
303 |
across the whole cluster. |
304 |
|
305 |
.. image:: images/network-mac.png |
306 |
:align: right |
307 |
:height: 500px |
308 |
:width: 500px |
309 |
|
310 |
.. code-block:: console |
311 |
|
312 |
# brctl addbr prv0 |
313 |
# brctl addif prv0 eth2 |
314 |
# ip link set prv0 up |
315 |
|
316 |
# brctl show |
317 |
bridge name bridge id STP enabled interfaces |
318 |
prv0 8000.8a3c3ede3583 no eth2 |
319 |
|
320 |
|
321 |
|
322 |
Then in Cyclades first create a pool for MAC prefixes by running: |
323 |
|
324 |
.. code-block:: console |
325 |
|
326 |
# snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536 |
327 |
|
328 |
and the create the network: |
329 |
|
330 |
.. code-block:: console |
331 |
|
332 |
# snf-manage network-create --subnet=192.168.1.0/24 --gateway=192.168.1.0/24 --dhcp --flavor=MAC_FILTERED --link=prv0 --name=mac --backend-id=1 |
333 |
# snf-manage network-list |
334 |
id name flavor owner mac_prefix dhcp state link vms public IPv4 Subnet IPv4 Gateway |
335 |
3 mac MAC_FILTERED aa:00:01 True ACTIVE prv0 False 192.168.1.0/24 192.168.1.1 |
336 |
|
337 |
Edit the synnefo setting `DEFAULT_MAC_FILTERED_BRIDGE` to `prv0`. |
338 |
|
339 |
This will add a network in Synnefo DB and create a network in Ganeti backend by |
340 |
issuing: |
341 |
|
342 |
.. code-block:: console |
343 |
|
344 |
# gnt-network add --network=192.168.1.0/24 --gateway=192.168.1.1 --network-type=private --tags=nfdhcpd,private-filtered snf-net-3 |
345 |
|
346 |
# gnt-network connect snf-net-3 default bridged prv0 |
347 |
# gnt-network list snf-net-3 |
348 |
Network Subnet Gateway NetworkType MacPrefix GroupList Tags |
349 |
snf-net-3 192.168.1.0/24 192.168.1.1 private aa:00:01 default(bridged, prv0) nfdhcpd,private-filtered |
350 |
|
351 |
|
352 |
|
353 |
|
354 |
|
355 |
|
356 |
PHYSICAL_VLAN |
357 |
------------- |
358 |
|
359 |
|
360 |
To create a network with PHYSICAL_VALN flavor you have to pre-provision in each Ganeti |
361 |
node a range of bridges (e.g. ``prv1..20``) that will be bridged with the corresponding VLANs (e.g. ``401..420``) |
362 |
across the whole cluster. To this end if we assume that ``eth3`` is the interface to use, run: |
363 |
|
364 |
.. image:: images/network-vlan.png |
365 |
:align: right |
366 |
:height: 480px |
367 |
:width: 500px |
368 |
|
369 |
|
370 |
.. code-block:: console |
371 |
|
372 |
# for i in {1..20}; do |
373 |
br=prv$i ; vlanid=$((400+i)) ; vlan=eth3.$vlanid |
374 |
brctl addbr $br ; ip link set $br up |
375 |
vconfig add eth0 vlanid ; ip link set vlan up |
376 |
brctl addif $br $vlan |
377 |
done |
378 |
# brctl show |
379 |
bridge name bridge id STP enabled interfaces |
380 |
prv1 8000.8a3c3ede3583 no eth3.401 |
381 |
prv2 8000.8a3c3ede3583 no eth3.402 |
382 |
... |
383 |
|
384 |
|
385 |
Then in Cyclades first create a pool for bridges by running: |
386 |
|
387 |
.. code-block:: console |
388 |
|
389 |
# snf-manage pool-create --type=bridge --base=prv --size=20 |
390 |
|
391 |
and the create the network: |
392 |
|
393 |
.. code-block:: console |
394 |
|
395 |
# snf-manage network-create --subnet=192.168.1.0/24 --gateway=192.168.1.0/24 --dhcp --flavor=PHYSICAL_VLAN --name=vlan --backend-id=1 |
396 |
|
397 |
# snf-manage network-list |
398 |
id name flavor owner mac_prefix dhcp state link vms public IPv4 Subnet IPv4 Gateway |
399 |
4 vlan PHYSICAL_VLAN True ACTIVE prv1 False 192.168.1.0/24 192.168.1.1 |
400 |
|
401 |
This will add a network in Synnefo DB and create a network in Ganeti backend by |
402 |
issuing: |
403 |
|
404 |
.. code-block:: console |
405 |
|
406 |
# gnt-network add --network=192.168.1.0/24 --gateway=192.168.1.1 --network-type=private --tags=nfdhcpd,physica-vlan snf-net-4 |
407 |
|
408 |
# gnt-network connect snf-net-4 default bridged prv1 |
409 |
# gnt-network list snf-net-4 |
410 |
Network Subnet Gateway NetworkType MacPrefix GroupList Tags |
411 |
snf-net-4 192.168.1.0/24 192.168.1.1 private None default(bridged, prv1) nfdhcpd,physical-vlan |
412 |
|
413 |
|
414 |
|
415 |
ADVANCED SCENARIO |
416 |
----------------- |
417 |
|
418 |
To create a network with CUSTOM flavor you have to pass your self mode, link, |
419 |
mac prefix, tags for the network. You are not allowed to use the existing pools |
420 |
(only MAC_FILTERED, PHYSICAL_VLAN use them) so link and mac prefix uniqueness |
421 |
cannot be guaranteed. |
422 |
|
423 |
Lets assume a bridge ``br200`` that serves a VPN network to GRNET already exists |
424 |
on Ganeti nodes and we want to create for a certain user a private network so |
425 |
that he can access the VPN. Then we run in Cyclades: |
426 |
|
427 |
.. code-block:: console |
428 |
|
429 |
# snf-manage network-create --subnet=192.168.1.0/24 --gateway=192.168.1.0/24 --dhcp --flavor=CUSTOM --mode=bridged --link=br200 --mac-prefix=bb:00:44 --owner=user@grnet.gr --tags=nfdhcpd,vpn --name=vpn --backend-id=1 |
430 |
|
431 |
# snf-manage network-list |
432 |
id name flavor owner mac_prefix dhcp state link vms public IPv4 Subnet IPv4 Gateway |
433 |
5 vpn CUSTOM user@grnet.gr bb:00:44 True ACTIVE br200 False 192.168.1.0/24 192.168.1.1 |
434 |
|
435 |
This will add a network in Synnefo DB and create a network in Ganeti backend by |
436 |
issuing: |
437 |
|
438 |
.. code-block:: console |
439 |
|
440 |
# gnt-network add --network=192.168.1.0/24 --gateway=192.168.1.1 --network-type=private --tags=nfdhcpd snf-net-5 |
441 |
|
442 |
# gnt-network connect snf-net-5 default bridged br200 |
443 |
# gnt-network list snf-net-5 |
444 |
Network Subnet Gateway NetworkType MacPrefix GroupList Tags |
445 |
snf-net-5 192.168.1.0/24 192.168.1.1 private bb:00:55 default(bridged, br200) nfdhcpd,private-filtered |
446 |
|
447 |
|