Statistics
| Branch: | Tag: | Revision:

root / docs / networks.rst @ 5031beee

History | View | Annotate | Download (18.2 kB)

1
.. _networks:
2

    
3
Network Service (part of Cyclades)
4
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5

    
6
Network setup overview
7
======================
8

    
9
Currently the Network Service is part of Cyclades and comes along with the
10
Cyclades software components.
11

    
12
Networking is deployment-specific and must be customized based on the specific
13
needs of the system administrator. However, to do so, the administrator needs
14
to understand how each level handles Virtual Networks, to be able to setup the
15
backend appropriately.
16

    
17
Since v0.11 Synnefo supports multiple Ganeti clusters (backends). Having in
18
mind that every backend has its locality, there is a high possibility each
19
cluster to have different infrastracture (wires, routers, subnets, gateways,
20
etc.).
21

    
22
In the following sections we investigate in a top-down approach, the way
23
networks are defined from the Cyclades, Ganeti, and Backend persperctive.
24

    
25
Network @ Cyclades level
26
------------------------
27

    
28
Cyclades networks support a range of different options to cover the specific
29
needs of each deployment.
30

    
31
First of all, as far as visibility and accessibility is concerned, a network
32
can be either `public` or `private`. Public networks are created by the
33
administrator via the command line interface (`snf-manage`) and are visible to
34
all end-users. On the other hand, private networks are created by the end-user
35
from the Web UI or the kamaki client and provide isolated Layer 2 connectivity
36
to the end-user.
37

    
38
Both networks can have an IPv4 subnet or/and an IPv6 subnet along with the
39
corresponding gateway. For IPv4 networks, if the `--dhcp` option is set,
40
Cyclades will treat the IPv4 subnet as an IP pool, and will assign to each VM
41
that is connected to this network an IPv4 address from this pool.
42

    
43
A public network can also be marked as a floating IP pool with the
44
`--floating-ip-pool` option. Floating IPs, are IPv4 addresses that can be
45
dynamically by added and removed from running VMs. A user can reserve and
46
release a floating IP address that he can later add and remove it from running
47
VMs. Also the user can release a floating IP if it not used by any of his
48
VMs.
49

    
50
Private networks and floating IPs must be accessible from all instances across
51
all Ganeti backends. So, such networks must exist in all backends, and
52
are dynamically created when new Ganeti backends are added. Specially for
53
private networks, to avoid the overhead of creating the network to all
54
backends, Cyclades create these networks on demand, when an instance that
55
lives in a backend tries to connect to this network.
56

    
57
The administrator may also want to connect instances to some network, without
58
supporting floating IPs (e.g. to enforce each VM to be connected to a specific
59
network). This can be achieved by setting the `DEFAULT_INSTANCE_NETWORKS`
60
setting to the list of the selected networks. The special keyword
61
`SNF:ANY_PUBLIC` may be used as a network identifier, to indicate to the system
62
to peak any of the public networks that has a free IP address. Public networks
63
that are not floating IP pools, do not need to exist to all Ganeti backends,
64
since the Cyclades backend allocator, will route spawned vms to a Ganeti
65
backend that the selected networks exist. The administrator can choose in
66
which backends to create the network via the `--backends` command line option.
67

    
68
Another distinction between networks is their flavor. Flavor is a way to
69
abstract infrastructure specific options, that are used to ensure connectivity
70
and isolation to the VMs connected to the network. It is a set of options that
71
eventually will guide scripts to set up rules, while creating virtual
72
interfaces in the node level. Each of these flavors define attributes that will
73
be used at Ganeti level to create the physical network. These attributes are:
74

    
75
* ``mode``: Whether the network is in 'bridged' or 'routed' mode.
76
* ``link``: Bridge for 'bridged' networks and routing table for 'routed'
77
  networks. e.g. 'br100', 'rt200'
78
* ``mac_prefix``: A MAC prefix for the network. e.g. 'aa:00:05'
79
* ``tags``: A list of tags to be used at the Ganeti level.
80

    
81
To ensure L2 isolation, Synnefo supports two different mechanisms (see also Node
82
Level section):
83

    
84
* assigning one physical VLAN per network
85
* assigning one MAC prefix per network, so that every NIC attached to this
86
  network will have this prefix. Isolation is then achieved by filtering
87
  rules (via `ebtables`) based on a specific mask (ff:ff:f0:00:00:00, see Node
88
  Level section for more details).
89

    
90
Having this in mind and in order to prevent assignment of duplicate VLAN/MAC
91
prefix to different networks, Synnefo supports two types of Pools:
92

    
93
- Bridge Pool (corresponding to a number of VLANs bridged to those bridges)
94
- MAC prefix Pool
95

    
96
For Pool handling refer to the corresponding doc section. To use this pools,
97
set either `--link` or `--mac-prefix` to the reserved keyword `pool`.
98

    
99
Existing network flavors are the following:
100

    
101
==============   =======   ===============================   ======================  ==================
102
Flavor Name      Mode      Link                              MAC prefix              Tags
103
==============   =======   ===============================   ======================  ==================
104
IP_LESS_ROUTED   routed    ``DEFAULT_ROUTING_TABLE``         ``DEFAULT_MAC_PREFIX``  'ip-less-routed'
105
MAC_FILTERED     bridged   ``DEFAULT_MAC_FILTERED_BRIDGE``   'pool'                  'private'filtered'
106
PHYSICAL_VLAN    bridged   'pool'                            ``DEFAULT_MAC_PREFIX``  'physical-vlan'
107
CUSTOM           bridged   ``DEFAULT_BRIDGE``                ``DEFAULT_MAC_PREFIX``
108
==============   =======   ===============================   ======================  ==================
109

    
110
``DEFAULT_ROUTING_TABLE``, ``DEFAULT_MAC_PREFIX``, ``DEFAULT_BRIDGE``,
111
``DEFAULT_MAC_FILTERED_BRIDGE`` are all configurable settings in
112
``/etc/synnefo/20-snf-cyclades-app-api.conf``. 'pool' is used to denote that a
113
link or MAC prefix will be allocated from the corresponding Pool. Finally,
114
most of these attributes, may be overridden when creating networks with
115
`snf-manage network-create command`.
116

    
117
The administrator is able to create any of the above flavors
118
and override their default values by explicitly passing mode, link, etc. using
119
the `snf-manage network-create` command. 
120

    
121
The administrator can create networks of any flavor, but end-users is allowed
122
to create via API only networks with flavors that are set in the
123
`API_ENABLED_NETWORK_FLAVORS` setting.
124

    
125
Network @ Ganeti level
126
----------------------
127

    
128
Currently, stable Ganeti does not support IP Pool management. However, the
129
functionality has been merged in the official Ganeti master branch and will
130
appear on Ganeti 2.7.0. So, you can either checkout the Ganeti master branch
131
and build your packages, or clone our local repo
132
https://code.grnet.gr/git/ganeti-local and checkout the
133
`stable-2.6-ippool-hotplug-esi` branch. This is the Ganeti stable branch with
134
IP pool management, Hotplugging and ExtStorage Interface features merged on top
135
of it. The last two features are not a hard Synnefo requirement, but will
136
enable you to do neat things when you get experienced with Synnefo. They are
137
going to be pushed for review upstream sometime soon.
138

    
139
Any network created in Synnefo is also created in one (for public networks) or
140
all (for private networks) Ganeti backends. In Ganeti a network can have the
141
following options:
142

    
143
 - network (192.168.0.0/24, mandatory)
144
 - gateway (192.168.0.1)
145
 - network6 (2001:648:2ffc:1201::/64)
146
 - gateway6 (2001:648:2ffc:1201::1)
147
 - mac_prefix (aa:00:01)
148
 - type (private, public)
149
 - tags
150

    
151
Networks in Ganeti cannot be used unless they are connected to a nodegroup in
152
order to define the connectivity mode and link. Synnefo, after creating a
153
network, connects it to all nodegroups of the Ganeti cluster(s) with the given
154
mode and link (defined in the network flavor).
155

    
156
Ganeti makes use of environment variables to inform scripts about each NIC's
157
setup. `kvm-vif-script` that comes with `snf-network` sets up the nfdhcpd lease and
158
applies any rules needed depending on the network's mode, link, mac_prefix and
159
tags.
160

    
161
Network @ Physical host level
162
-----------------------------
163

    
164
Currently, networking infrastructure must be pre-provisioned before creating
165
networks in Synnefo. According to which flavors you want to support, you should
166
have already setup all your physical hosts correspondingly. This means you
167
need:
168

    
169
 - one bridge for the ``CUSTOM`` flavor (br0, see Fig. 1)
170
 - one bridge for the ``MAC_FILTERED`` flavor (prv0, see Fig. 2)
171
 - a number of bridges and their corresponding VLANs (bridged to them) for
172
   the ``PHYSICAL_VLAN`` flavor (prv1..prv100, see Fig. 3)
173
 - a routing table for the ``IP_LESS_ROUTED`` flavor (snf_public, see Fig. 4)
174

    
175
Please refer to the following figures, which clarify each infrastructure setup
176
and how connectivity and isolation is achieved in every case for every type of
177
network.
178

    
179

    
180
FLAVORS
181
=======
182

    
183
As mentioned earlier supported flavors are:
184

    
185
 - CUSTOM
186
 - IP_LESS_ROUTED
187
 - MAC_FILTERED
188
 - PHYSICAL_VLAN
189

    
190
In the following sections we mention what configuration imposes each flavor from
191
Synnefo, Ganeti and Physical host perspective.
192

    
193

    
194

    
195
DEFAULT SCENARIO
196
----------------
197

    
198
In this case we will bridge all primary interfaces of the VMs on one bridge that must
199
be the same collition domain with the router. The router sould then forward packets
200
(if a public IPv4 Subnet is available) or do NAT in order to provide internet access to
201
the VMs.
202

    
203
To this end we will use the CUSTOM flavor and pre-provision in each Ganeti
204
node one bridge (e.g. ``br100``). If we assume that ``eth1`` is the physical interface
205
connected to the router, run:
206

    
207
.. image:: images/network-bridged.png
208
   :align: right
209
   :height: 550px
210
   :width: 500px
211

    
212
.. code-block:: console
213

    
214
   # brctl addbr br100
215
   # brctl addif br100 eth1
216
   # ip link set br100 up
217

    
218
   # brctl show
219
   bridge name bridge id         STP enabled interfaces
220
   br100       8000.8a3c3ede3583 no          eth1
221

    
222

    
223

    
224
Then in Cyclades run:
225

    
226
.. code-block:: console
227

    
228
   # snf-manage network-create --subnet=5.6.7.0/27 --gateway=5.6.7.1 --subnet6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --public --dhcp=True --flavor=CUSTOM --link=br100 ----name=default --backend-id=1
229

    
230
   # snf-manage network-list
231
   id    name     flavor   owner mac_prefix   dhcp    state         link  vms public IPv4 Subnet   IPv4 Gateway
232
   1     default  CUSTOM                      True    ACTIVE        br100     True   5.6.7.0/27    5.6.7.1
233

    
234
This will add a network in Synnefo DB and create a network in Ganeti backend by
235
issuing:
236

    
237
.. code-block:: console
238

    
239
   # gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --network-type=public --tags=nfdhcpd snf-net-1
240

    
241
   # gnt-network connect snf-net-1 default bridged br100
242
   # gnt-network list snf-net-1
243
   Network   Subnet     Gateway NetworkType MacPrefix GroupList               Tags
244
   snf-net-1 5.6.7.0/27 5.6.7.1 public      None      default(bridged, br100) nfdhcpd
245

    
246

    
247
To enable NAT in a Internal Router if you do not have a public IP range available
248
but only a public routable IP (e.g 1.2.3.4):
249

    
250
.. code-block:: console
251

    
252
   # ip addr add 5.6.7.1/27 dev eth1
253
   # iptables -t nat -A POSTROUTING -o eth1 --to-source 1.2.3.4 -j SNAT
254

    
255
IP_LESS_ROUTED
256
--------------
257

    
258
.. image:: images/network-routed.png
259
   :align: right
260
   :height: 580px
261
   :width: 500px
262

    
263
To create a network with IP_LESS_ROUTED flavor run you have to pre-provision in
264
each Ganeti node one routing table (e.g. ``snf_public``) that will do all the
265
routing from/to the VMs' taps. Additionally you must enable ``Proxy-ARP``
266
support. All traffic will be on a single iterface (e.g. ``eth1``).
267

    
268
.. code-block:: console
269

    
270
   # echo 1 > /proc/sys/net/ipv4/conf/ip_fowarding
271
   # echo 10 snf_public >> /etc/iproute2/rt_tables
272
   # ip route add 5.6.7.0/27 dev eth1
273
   # ip route add 5.6.7.0/27 dev eth1 table snf_public
274
   # ip route add default via 5.6.7.1 dev eth1 table snf_public
275
   # ip rule add iif eth1 lookup snf_public
276
   # arptables -A OUTPUT -o eth1 --opcode 1 --mangle-ip-s 5.6.7.30  # last ip in Subnet
277

    
278
Then in Cyclades run:
279

    
280
.. code-block:: console
281

    
282
   # snf-manage network-create --subnet=5.6.7.0/27 --gateway=5.6.7.1 --subnet6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1 --public --dhcp=True --flavor=IP_LESS_ROUTED --name=routed --backend-id=1
283

    
284
   # snf-manage network-list
285
   id    name     flavor         owner mac_prefix   dhcp    state   link      vms  public IPv4 Subnet   IPv4 Gateway
286
   2     routed   IP_LESS_ROUTED                    True    ACTIVE  snf_public     True   5.6.7.0/27    5.6.7.1
287

    
288

    
289
This will add a network in Synnefo DB and create a network in Ganeti backend by
290
issuing:
291

    
292
.. code-block:: console
293

    
294
   # gnt-network add --network=5.6.7.0/27 --gateway=5.6.7.1 --network6=2001:648:2FFC:1322::/64 --gateway6=2001:648:2FFC:1322::1  --network-type=public  --tags=nfdhcpd,ip-less-routed  snf-net-2
295

    
296
   # gnt-network connect snf-net-2 default bridged br100
297
   # gnt-network list snf-net-2
298
   Network      Subnet            Gateway        NetworkType MacPrefix GroupList                   Tags
299
   dimara-net-1 62.217.123.128/27 62.217.123.129 public      None      default(routed, snf_public) nfdhcpd,ip-less-routed
300

    
301

    
302

    
303

    
304
MAC_FILTERED
305
------------
306

    
307

    
308
To create a network with MAC_FILTERED flavor you have to pre-provision in each Ganeti
309
node one bridge (e.g. ``prv0``) that will be bridged with one interface (e.g. ``eth2``)
310
across the whole cluster.
311

    
312
.. image:: images/network-mac.png
313
   :align: right
314
   :height: 500px
315
   :width: 500px
316

    
317
.. code-block:: console
318

    
319
   # brctl addbr prv0
320
   # brctl addif prv0 eth2
321
   # ip link set prv0 up
322

    
323
   # brctl show
324
   bridge name bridge id         STP enabled interfaces
325
   prv0        8000.8a3c3ede3583 no          eth2
326

    
327

    
328

    
329
Then in Cyclades first create a pool for MAC prefixes by running:
330

    
331
.. code-block:: console
332

    
333
   # snf-manage pool-create --type=mac-prefix --base=aa:00:0 --size=65536
334

    
335
and the create the network:
336

    
337
.. code-block:: console
338

    
339
   # snf-manage network-create --subnet=192.168.1.0/24 --gateway=192.168.1.0/24 --dhcp=True --flavor=MAC_FILTERED --link=prv0 --name=mac --backend-id=1
340
   # snf-manage network-list
341
   id    name     flavor       owner mac_prefix   dhcp    state         link  vms public IPv4 Subnet    IPv4 Gateway
342
   3     mac      MAC_FILTERED       aa:00:01     True    ACTIVE        prv0      False  192.168.1.0/24 192.168.1.1
343

    
344
Edit the synnefo setting `DEFAULT_MAC_FILTERED_BRIDGE` to `prv0`.
345

    
346
This will add a network in Synnefo DB and create a network in Ganeti backend by
347
issuing:
348

    
349
.. code-block:: console
350

    
351
   # gnt-network add --network=192.168.1.0/24  --gateway=192.168.1.1  --network-type=private  --tags=nfdhcpd,private-filtered snf-net-3
352

    
353
   # gnt-network connect snf-net-3 default bridged prv0
354
   # gnt-network list snf-net-3
355
   Network   Subnet         Gateway     NetworkType MacPrefix GroupList               Tags
356
   snf-net-3 192.168.1.0/24 192.168.1.1 private     aa:00:01  default(bridged, prv0) nfdhcpd,private-filtered
357

    
358

    
359

    
360

    
361

    
362

    
363
PHYSICAL_VLAN
364
-------------
365

    
366

    
367
To create a network with PHYSICAL_VALN flavor you have to pre-provision in each Ganeti
368
node a range of bridges (e.g. ``prv1..20``) that will be bridged with the corresponding VLANs (e.g. ``401..420``)
369
across the whole cluster. To this end if we assume that ``eth3`` is the interface to use, run:
370

    
371
.. image:: images/network-vlan.png
372
   :align: right
373
   :height: 480px
374
   :width: 500px
375

    
376

    
377
.. code-block:: console
378

    
379
   # for i in {1..20}; do
380
      br=prv$i ; vlanid=$((400+i)) ; vlan=eth3.$vlanid
381
      brctl addbr $br ; ip link set $br up
382
      vconfig add eth0 vlanid ; ip link set vlan up
383
      brctl addif $br $vlan
384
   done
385
   # brctl show
386
   bridge name     bridge id               STP enabled     interfaces
387
   prv1            8000.8a3c3ede3583       no              eth3.401
388
   prv2            8000.8a3c3ede3583       no              eth3.402
389
   ...
390

    
391

    
392
Then in Cyclades first create a pool for bridges by running:
393

    
394
.. code-block:: console
395

    
396
   # snf-manage pool-create --type=bridge --base=prv --size=20
397

    
398
and the create the network:
399

    
400
.. code-block:: console
401

    
402
   # snf-manage network-create --subnet=192.168.1.0/24  --gateway=192.168.1.0/24  --dhcp=True --flavor=PHYSICAL_VLAN  --name=vlan  --backend-id=1
403

    
404
   # snf-manage network-list
405
   id    name     flavor       owner mac_prefix   dhcp    state         link  vms public IPv4 Subnet    IPv4 Gateway
406
   4     vlan     PHYSICAL_VLAN                   True    ACTIVE        prv1      False  192.168.1.0/24 192.168.1.1
407

    
408
This will add a network in Synnefo DB and create a network in Ganeti backend by
409
issuing:
410

    
411
.. code-block:: console
412

    
413
   # gnt-network add --network=192.168.1.0/24 --gateway=192.168.1.1 --network-type=private --tags=nfdhcpd,physica-vlan snf-net-4
414

    
415
   # gnt-network connect snf-net-4 default bridged prv1
416
   # gnt-network list snf-net-4
417
   Network   Subnet         Gateway     NetworkType MacPrefix GroupList               Tags
418
   snf-net-4 192.168.1.0/24 192.168.1.1 private     None      default(bridged, prv1)  nfdhcpd,physical-vlan
419

    
420

    
421

    
422
ADVANCED SCENARIO
423
-----------------
424

    
425
To create a network with CUSTOM flavor you have to pass your self mode, link,
426
mac prefix, tags for the network. You are not allowed to use the existing pools
427
(only MAC_FILTERED, PHYSICAL_VLAN use them) so link and mac prefix uniqueness
428
cannot be guaranteed.
429

    
430
Lets assume a bridge ``br200`` that serves a VPN network to GRNET already exists
431
on Ganeti nodes and we want to create for a certain user a private network so
432
that he can access the VPN. Then we run in Cyclades:
433

    
434
.. code-block:: console
435

    
436
   # snf-manage network-create --subnet=192.168.1.0/24 --gateway=192.168.1.0/24 --dhcp=True --flavor=CUSTOM --mode=bridged --link=br200 --mac-prefix=bb:00:44 --owner=user@grnet.gr --tags=nfdhcpd,vpn --name=vpn --backend-id=1
437

    
438
   # snf-manage network-list
439
   id    name     flavor       owner              mac_prefix   dhcp    state         link  vms public IPv4 Subnet    IPv4 Gateway
440
   5     vpn      CUSTOM       user@grnet.gr      bb:00:44     True    ACTIVE        br200     False  192.168.1.0/24 192.168.1.1
441

    
442
This will add a network in Synnefo DB and create a network in Ganeti backend by
443
issuing:
444

    
445
.. code-block:: console
446

    
447
   # gnt-network add --network=192.168.1.0/24 --gateway=192.168.1.1 --network-type=private --tags=nfdhcpd snf-net-5
448

    
449
   # gnt-network connect snf-net-5 default bridged br200
450
   # gnt-network list snf-net-5
451
   Network   Subnet         Gateway     NetworkType MacPrefix GroupList               Tags
452
   snf-net-5 192.168.1.0/24 192.168.1.1 private     bb:00:55  default(bridged, br200) nfdhcpd,private-filtered
453

    
454