Statistics
| Branch: | Tag: | Revision:

root / docs / networks.rst @ be8c3784

History | View | Annotate | Download (16.9 kB)

1
.. _networks:
2

    
3
Network Service (part of Cyclades)
4
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5

    
6
Network setup overview
7
======================
8

    
9
Currently the Network Service is part of Cyclades and comes along with the
10
Cyclades software components.
11

    
12
Networking is deployment-specific and must be customized based on the specific
13
needs of the system administrator. However, to do so, the administrator needs
14
to understand how each level handles Virtual Networks, to be able to setup the
15
backend appropriately.
16

    
17
Since v0.11 Synnefo supports multiple Ganeti clusters (backends). Having in
18
mind that every backend has its locality, there is a high possibility each
19
cluster to have different infrastracture (wires, routers, subnets, gateways,
20
etc.).
21

    
22
In the following sections we investigate in a top-down approach, the way
23
networks are defined from the Cyclades, Ganeti, and Backend persperctive.
24

    
25
Network @ Cyclades level
26
------------------------
27

    
28
Cyclades understands two types of Virtual Networks:
29

    
30
a) Public Networks
31
b) Private Networks
32

    
33
Public Networks are created by the administrator via `snf-manage` commands
34
and can be used by all end-users. Each public network is assigned to a
35
single backend but one backend can have multiple public networks.
36

    
37
Private Networks are created by the end-user from the Web UI or the kamaki
38
client and provide isolated Layer 2 connectivity to the end-user. With regard
39
to the fact that a user's VMs may be allocated across different Ganeti clusters
40
(backends), private networks are created in all backends to ensure VMs
41
connectivity.
42

    
43
Both types of networks are created dynamically.
44

    
45
From the VM perspective, each NIC is attached to a specific Network.
46

    
47
When a new VM is created the backend allocator (in Cyclades) decides in which
48
backend  to spawn it. Depending on the chosen backend, Synnefo finds the first
49
non-full public Network that exists in the backend. Then attaches the VM's
50
first NIC to this network.
51

    
52
Once the VM is created, the user is able to connect the VM to multiple
53
private networks, that himself has already created.
54

    
55
A Network can have the following attributes:
56

    
57
 - IPv4 subnet (mandatory)
58
 - IPv4 gateway
59
 - IPv6 subnet
60
 - IPv6 gateway
61
 - public/private flag
62
 - flavor
63

    
64
Flavor is a way to abstact infrastructure specific options, that are used to
65
ensure connectivity and isolation to the VMs connected to the network. It is a
66
set of options that eventually will guide scripts to set up rules, while
67
creating virtual interfaces in the node level. The available flavors and their
68
options can be found in the Synnefo settings and are configurable.
69

    
70
To ensure L2 isolation, Synnefo supports two different mechanisms (see also Node
71
Level section):
72

    
73
 - assigning one physical VLAN per network
74
 - assigning one MAC prefix per network, so that every NIC attached to this
75
   network will have this prefix. Isolation is then achieved by filtering
76
   rules (via `ebtables`) based on a specific mask (ff:ff:f0:00:00:00, see Node
77
   Level section for more details).
78

    
79
Having this in mind and in order to prevent assignment of duplicate VLAN/MAC
80
prefix to different networks, Synnefo supports two types of Pools:
81

    
82
 - Bridge Pool (corresponding to a number of VLANs bridged to those bridges)
83
 - MAC prefix Pool
84

    
85
For Pool handling refer to the corresponding doc section.
86

    
87
Finally, each supported flavor must declare the following options (see also
88
Ganeti Level section):
89

    
90
 - ``mode`` ('bridged' or 'routed'),
91
 - ``link`` ('br100', 'rt200', 'pool')
92
 - ``mac_prefix`` ('aa:00:05', 'pool', None)
93
 - ``tags`` (['ip-less-routed' or 'mac-filtered' or 'physical-vlan' or None])
94

    
95
Existing network flavors are the following:
96

    
97
==============   =======   ===============================   ======================  ==================
98
Flavor Name      Mode      Link                              MAC prefix              Tags
99
==============   =======   ===============================   ======================  ==================
100
IP_LESS_ROUTED   routed    ``DEFAULT_ROUTING_TABLE``         ``DEFAULT_MAC_PREFIX``  'ip-less-routed'
101
MAC_FILTERED     bridged   ``DEFAULT_MAC_FILTERED_BRIDGE``   'pool'                  'private'filtered'
102
PHYSICAL_VLAN    bridged   'pool'                            ``DEFAULT_MAC_PREFIX``  'physical-vlan'
103
CUSTOM           bridged   ``DEFAULT_BRIDGE``                ``DEFAULT_MAC_PREFIX``
104
==============   =======   ===============================   ======================  ==================
105

    
106
``DEFAULT_ROUTING_TABLE``, ``DEFAULT_MAC_PREFIX``, ``DEFAULT_BRIDGE``, ``DEFAULT_MAC_FILTERED_BRIDGE``
107
are all configurable settings in ``/etc/synnefo/20-snf-cyclades-app-api.conf``. 'pool' is used
108
to denote that a link or MAC prefix will be allocated from the corresponging Pool.
109

    
110
The administrator is able to create any of the above flavors
111
and override their default values by explicitly passing mode, link, etc. using
112
the `snf-manage network-create` command. 
113

    
114
The end-user is allowed to create only networks of flavor ``MAC_FILTERED`` and
115
``PHYSICAL_VLAN``. Currently, only ``MAC_FILTERED`` and ``PHYSICAL_VLAN`` can
116
use existing pools and cannot be overriden.
117

    
118
Network @ Ganeti level
119
----------------------
120

    
121
Currently, stable Ganeti does not support IP Pool management. However, the
122
functionality has been merged in the official Ganeti master branch and will
123
appear on Ganeti 2.7.0. So, you can either checkout the Ganeti master branch
124
and build your packages, or clone our local repo
125
https://code.grnet.gr/git/ganeti-local and checkout the
126
`stable-2.6-ippool-hotplug-esi` branch. This is the Ganeti stable branch with
127
IP pool management, Hotplugging and ExtStorage Interface features merged on top
128
of it. The last two features are not a hard Synnefo requirement, but will
129
enable you to do neat things when you get experienced with Synnefo. They are
130
going to be pushed for review upstream sometime soon.
131

    
132
Any network created in Synnefo is also created in one (for public networks) or
133
all (for private networks) Ganeti backends. In Ganeti a network can have the
134
following options:
135

    
136
 - network (192.168.0.0/24, mandatory)
137
 - gateway (192.168.0.1)
138
 - network6 (2001:648:2ffc:1201::/64)
139
 - gateway6 (2001:648:2ffc:1201::1)
140
 - mac_prefix (aa:00:01)
141
 - type (private, public)
142
 - tags
143

    
144
Networks in Ganeti cannot be used unless they are connected to a nodegroup in
145
order to define the connectivity mode and link. Synnefo, after creating a
146
network, connects it to all nodegroups of the Ganeti cluster(s) with the given
147
mode and link (defined in the network flavor).
148

    
149
Ganeti makes use of environment variables to inform scripts about each NIC's
150
setup. `kvm-vif-script` that comes with `snf-network` sets up the nfdhcpd lease and
151
applies any rules needed depending on the network's mode, link, mac_prefix and
152
tags.
153

    
154
Network @ Physical host level
155
-----------------------------
156

    
157
Currently, networking infrastructure must be pre-provisioned before creating
158
networks in Synnefo. According to which flavors you want to support, you should
159
have already setup all your physical hosts correspondingly. This means you
160
need:
161

    
162
 - one bridge for the ``CUSTOM`` flavor (br0, see Fig. 1)
163
 - one bridge for the ``MAC_FILTERED`` flavor (prv0, see Fig. 2)
164
 - a number of bridges and their corresponding VLANs (bridged to them) for
165
   the ``PHYSICAL_VLAN`` flavor (prv1..prv100, see Fig. 3)
166
 - a routing table for the ``IP_LESS_ROUTED`` flavor (snf_public, see Fig. 4)
167

    
168
Please refer to the following figures, which clarify each infrastructure setup
169
and how connectivity and isolation is achieved in every case for every type of
170
network.
171

    
172

    
173
FLAVORS
174
=======
175

    
176
As mentioned earlier supported flavors are:
177

    
178
 - CUSTOM
179
 - IP_LESS_ROUTED
180
 - MAC_FILTERED
181
 - PHYSICAL_VLAN
182

    
183
In the following sections we mention what configuration imposes each flavor from
184
Synnefo, Ganeti and Physical host perspective.
185

    
186

    
187

    
188
DEFAULT SCENARIO
189
----------------
190

    
191
In this case we will bridge all primary interfaces of the VMs on one bridge that must
192
be the same collition domain with the router. The router sould then forward packets
193
(if a public IPv4 Subnet is available) or do NAT in order to provide internet access to
194
the VMs.
195

    
196
To this end we will use the CUSTOM flavor and pre-provision in each Ganeti
197
node one bridge (e.g. ``br100``). If we assume that ``eth1`` is the physical interface
198
connected to the router, run:
199

    
200
.. image:: images/network-bridged.png
201
   :align: right
202
   :height: 550px
203
   :width: 500px
204

    
205
.. code-block:: console
206

    
207
   # brctl addbr br100
208
   # brctl addif br100 eth1
209
   # ip link set br100 up
210

    
211
   # brctl show
212
   bridge name bridge id         STP enabled interfaces
213
   br100       8000.8a3c3ede3583 no          eth1
214

    
215

    
216

    
217
Then in Cyclades run:
218

    
219
.. code-block:: console
220

    
221
   # snf-manage network-create --subnet=5.6.7.0/27 --gateway=5.6.7.1 --subnet6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --public --dhcp=True --flavor=CUSTOM --link=br100 ----name=default --backend-id=1
222

    
223
   # snf-manage network-list
224
   id    name     flavor   owner mac_prefix   dhcp    state         link  vms public IPv4 Subnet   IPv4 Gateway
225
   1     default  CUSTOM                      True    ACTIVE        br100     True   5.6.7.0/27    5.6.7.1
226

    
227
This will add a network in Synnefo DB and create a network in Ganeti backend by
228
issuing:
229

    
230
.. code-block:: console
231

    
232
   # gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --network-type=public --tags=nfdhcpd snf-net-1
233

    
234
   # gnt-network connect snf-net-1 default bridged br100
235
   # gnt-network list snf-net-1
236
   Network   Subnet     Gateway NetworkType MacPrefix GroupList               Tags
237
   snf-net-1 5.6.7.0/27 5.6.7.1 public      None      default(bridged, br100) nfdhcpd
238

    
239

    
240
To enable NAT in a Internal Router if you do not have a public IP range available
241
but only a public routable IP (e.g 1.2.3.4):
242

    
243
.. code-block:: console
244

    
245
   # ip addr add 5.6.7.1/27 dev eth1
246
   # iptables -t nat -A POSTROUTING -o eth1 --to-source 1.2.3.4 -j SNAT
247

    
248
IP_LESS_ROUTED
249
--------------
250

    
251
.. image:: images/network-routed.png
252
   :align: right
253
   :height: 580px
254
   :width: 500px
255

    
256
To create a network with IP_LESS_ROUTED flavor run you have to pre-provision in
257
each Ganeti node one routing table (e.g. ``snf_public``) that will do all the
258
routing from/to the VMs' taps. Additionally you must enable ``Proxy-ARP``
259
support. All traffic will be on a single iterface (e.g. ``eth1``).
260

    
261
.. code-block:: console
262

    
263
   # echo 1 > /proc/sys/net/ipv4/conf/ip_fowarding
264
   # echo 10 snf_public >> /etc/iproute2/rt_tables
265
   # ip route add 5.6.7.0/27 dev eth1
266
   # ip route add 5.6.7.0/27 dev eth1 table snf_public
267
   # ip route add default via 5.6.7.1 dev eth1 table snf_public
268
   # ip rule add iif eth1 lookup snf_public
269
   # arptables -A OUTPUT -o eth1 --opcode 1 --mangle-ip-s 5.6.7.30  # last ip in Subnet
270

    
271
Then in Cyclades run:
272

    
273
.. code-block:: console
274

    
275
   # snf-manage network-create --subnet=5.6.7.0/27 --gateway=5.6.7.1 --subnet6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --public --dhcp=True --flavor=IP_LESS_ROUTED --name=routed --backend-id=1
276

    
277
   # snf-manage network-list
278
   id    name     flavor         owner mac_prefix   dhcp    state   link      vms  public IPv4 Subnet   IPv4 Gateway
279
   2     routed   IP_LESS_ROUTED                    True    ACTIVE  snf_public     True   5.6.7.0/27    5.6.7.1
280

    
281

    
282
This will add a network in Synnefo DB and create a network in Ganeti backend by
283
issuing:
284

    
285
.. code-block:: console
286

    
287
   # gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1  --network-type=public  --tags=nfdhcpd,ip-less-routed  snf-net-2
288

    
289
   # gnt-network connect snf-net-2 default bridged br100
290
   # gnt-network list snf-net-2
291
   Network      Subnet            Gateway        NetworkType MacPrefix GroupList                   Tags
292
   dimara-net-1 62.217.123.128/27 62.217.123.129 public      None      default(routed, snf_public) nfdhcpd,ip-less-routed
293

    
294

    
295

    
296

    
297
MAC_FILTERED
298
------------
299

    
300

    
301
To create a network with MAC_FILTERED flavor you have to pre-provision in each Ganeti
302
node one bridge (e.g. ``prv0``) that will be bridged with one interface (e.g. ``eth2``)
303
across the whole cluster.
304

    
305
.. image:: images/network-mac.png
306
   :align: right
307
   :height: 500px
308
   :width: 500px
309

    
310
.. code-block:: console
311

    
312
   # brctl addbr prv0
313
   # brctl addif prv0 eth2
314
   # ip link set prv0 up
315

    
316
   # brctl show
317
   bridge name bridge id         STP enabled interfaces
318
   prv0        8000.8a3c3ede3583 no          eth2
319

    
320

    
321

    
322
Then in Cyclades first create a pool for MAC prefixes by running:
323

    
324
.. code-block:: console
325

    
326
   # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
327

    
328
and the create the network:
329

    
330
.. code-block:: console
331

    
332
   # snf-manage network-create --subnet=192.168.1.0/24 --gateway=192.168.1.0/24 --dhcp=True --flavor=MAC_FILTERED --link=prv0 --name=mac --backend-id=1
333
   # snf-manage network-list
334
   id    name     flavor       owner mac_prefix   dhcp    state         link  vms public IPv4 Subnet    IPv4 Gateway
335
   3     mac      MAC_FILTERED       aa:00:01     True    ACTIVE        prv0      False  192.168.1.0/24 192.168.1.1
336

    
337
Edit the synnefo setting `DEFAULT_MAC_FILTERED_BRIDGE` to `prv0`.
338

    
339
This will add a network in Synnefo DB and create a network in Ganeti backend by
340
issuing:
341

    
342
.. code-block:: console
343

    
344
   # gnt-network add --network=192.168.1.0/24  --gateway=192.168.1.1  --network-type=private  --tags=nfdhcpd,private-filtered snf-net-3
345

    
346
   # gnt-network connect snf-net-3 default bridged prv0
347
   # gnt-network list snf-net-3
348
   Network   Subnet         Gateway     NetworkType MacPrefix GroupList               Tags
349
   snf-net-3 192.168.1.0/24 192.168.1.1 private     aa:00:01  default(bridged, prv0) nfdhcpd,private-filtered
350

    
351

    
352

    
353

    
354

    
355

    
356
PHYSICAL_VLAN
357
-------------
358

    
359

    
360
To create a network with PHYSICAL_VALN flavor you have to pre-provision in each Ganeti
361
node a range of bridges (e.g. ``prv1..20``) that will be bridged with the corresponding VLANs (e.g. ``401..420``)
362
across the whole cluster. To this end if we assume that ``eth3`` is the interface to use, run:
363

    
364
.. image:: images/network-vlan.png
365
   :align: right
366
   :height: 480px
367
   :width: 500px
368

    
369

    
370
.. code-block:: console
371

    
372
   # for i in {1..20}; do
373
      br=prv$i ; vlanid=$((400+i)) ; vlan=eth3.$vlanid
374
      brctl addbr $br ; ip link set $br up
375
      vconfig add eth0 vlanid ; ip link set vlan up
376
      brctl addif $br $vlan
377
   done
378
   # brctl show
379
   bridge name     bridge id               STP enabled     interfaces
380
   prv1            8000.8a3c3ede3583       no              eth3.401
381
   prv2            8000.8a3c3ede3583       no              eth3.402
382
   ...
383

    
384

    
385
Then in Cyclades first create a pool for bridges by running:
386

    
387
.. code-block:: console
388

    
389
   # snf-manage pool-create --type=bridge --base=prv --size=20
390

    
391
and the create the network:
392

    
393
.. code-block:: console
394

    
395
   # snf-manage network-create --subnet=192.168.1.0/24  --gateway=192.168.1.0/24  --dhcp=True --flavor=PHYSICAL_VLAN  --name=vlan  --backend-id=1
396

    
397
   # snf-manage network-list
398
   id    name     flavor       owner mac_prefix   dhcp    state         link  vms public IPv4 Subnet    IPv4 Gateway
399
   4     vlan     PHYSICAL_VLAN                   True    ACTIVE        prv1      False  192.168.1.0/24 192.168.1.1
400

    
401
This will add a network in Synnefo DB and create a network in Ganeti backend by
402
issuing:
403

    
404
.. code-block:: console
405

    
406
   # gnt-network add --network=192.168.1.0/24 --gateway=192.168.1.1 --network-type=private --tags=nfdhcpd,physica-vlan snf-net-4
407

    
408
   # gnt-network connect snf-net-4 default bridged prv1
409
   # gnt-network list snf-net-4
410
   Network   Subnet         Gateway     NetworkType MacPrefix GroupList               Tags
411
   snf-net-4 192.168.1.0/24 192.168.1.1 private     None      default(bridged, prv1)  nfdhcpd,physical-vlan
412

    
413

    
414

    
415
ADVANCED SCENARIO
416
-----------------
417

    
418
To create a network with CUSTOM flavor you have to pass your self mode, link,
419
mac prefix, tags for the network. You are not allowed to use the existing pools
420
(only MAC_FILTERED, PHYSICAL_VLAN use them) so link and mac prefix uniqueness
421
cannot be guaranteed.
422

    
423
Lets assume a bridge ``br200`` that serves a VPN network to GRNET already exists
424
on Ganeti nodes and we want to create for a certain user a private network so
425
that he can access the VPN. Then we run in Cyclades:
426

    
427
.. code-block:: console
428

    
429
   # snf-manage network-create --subnet=192.168.1.0/24 --gateway=192.168.1.0/24 --dhcp=True --flavor=CUSTOM --mode=bridged --link=br200 --mac-prefix=bb:00:44 --owner=user@grnet.gr --tags=nfdhcpd,vpn --name=vpn --backend-id=1
430

    
431
   # snf-manage network-list
432
   id    name     flavor       owner              mac_prefix   dhcp    state         link  vms public IPv4 Subnet    IPv4 Gateway
433
   5     vpn      CUSTOM       user@grnet.gr      bb:00:44     True    ACTIVE        br200     False  192.168.1.0/24 192.168.1.1
434

    
435
This will add a network in Synnefo DB and create a network in Ganeti backend by
436
issuing:
437

    
438
.. code-block:: console
439

    
440
   # gnt-network add --network=192.168.1.0/24 --gateway=192.168.1.1 --network-type=private --tags=nfdhcpd snf-net-5
441

    
442
   # gnt-network connect snf-net-5 default bridged br200
443
   # gnt-network list snf-net-5
444
   Network   Subnet         Gateway     NetworkType MacPrefix GroupList               Tags
445
   snf-net-5 192.168.1.0/24 192.168.1.1 private     bb:00:55  default(bridged, br200) nfdhcpd,private-filtered
446

    
447