+Network Management
+------------------
+
+Ganeti used to describe NICs of an Instance with an IP, a MAC, a connectivity
+link and mode. This had three major shortcomings:
+
+ * there was no easy way to assign a unique IP to an instance
+ * network info (subnet, gateway, domain, etc.) was not available on target
+ node (kvm-ifup, hooks, etc)
+ * one should explicitly pass L2 info (mode, and link) to every NIC
+
+Plus there was no easy way to get the current networking overview (which
+instances are on the same L2 or L3 network, which IPs are reserved, etc).
+
+All the above required an external management tool that has an overall view
+and provides the corresponding info to Ganeti.
+
+gnt-network aims to support a big part of this functionality inside Ganeti and
+abstract the network as a separate entity. Currently, a Ganeti network
+provides the following:
+
+ * A single IPv4 pool, subnet and gateway
+ * Connectivity info per nodegroup (mode, link)
+ * MAC prefix for each NIC inside the network
+ * IPv6 prefix/Gateway related to this network
+ * Tags
+
+IP pool management ensures IP uniqueness inside this network. The user can
+pass `ip=pool,network=test` and will:
+
+1. Get the first available IP in the pool
+2. Inherit the connectivity mode and link of the network's netparams
+3. NIC will obtain the MAC prefix of the network
+4. All network related info will be available as environment variables in
+ kvm-ifup scripts and hooks, so that they can dynamically manage all
+ networking-related setup on the host.
+
+Hands on with gnt-network
++++++++++++++++++++++++++
+
+To create a network do::
+
+ # gnt-network add --network=192.0.2.0/24 --gateway=192.0.2.1 test
+
+Please see all other available options (--add-reserved-ips, --mac-prefix,
+--network6, --subnet6, --tags).
+
+To make this network available on a nodegroup you should specify the
+connectivity mode and link during connection::
+
+ # gnt-network connect test bridged br100 default nodegroup1
+
+To add a NIC inside this network::
+
+ # gnt-instance modify --net -1:add,ip=pool,network=test inst1
+
+This will let a NIC obtain a unique IP inside this network, and inherit the
+nodegroup's netparams (bridged, br100). IP here is optional. If missing the
+NIC will just get the L2 info.
+
+To move an existing NIC from a network to another and remove its IP::
+
+ # gnt-instance modify --net -1:ip=none,network=test1 inst1
+
+This will release the old IP from the old IP pool and the NIC will inherit the
+new nicparams.
+
+On the above actions there is a extra option `--no-conflicts-ckeck`. This
+does not check for conflicting setups. Specifically:
+
+1. When a network is added, IPs of nodes and master are not being checked.
+2. When connecting a network on a nodegroup, IPs of instances inside this
+ nodegroup are not checked whether they reside inside the subnet or not.
+3. When specifying explicitly a IP without passing a network, Ganeti will not
+ check if this IP is included inside any available network on the nodegroup.
+
+External components
++++++++++++++++++++
+
+All the aforementioned steps assure NIC configuration from the Ganeti
+perspective. Of course this has nothing to do, how the instance eventually will
+get the desired connectivity (IPv4, IPv6, default routes, DNS info, etc) and
+where will the IP resolve. This functionality is managed by the external
+components.
+
+Let's assume that the VM will need to obtain a dynamic IP via DHCP, get a SLAAC
+address, and use DHCPv6 for other configuration information (in case RFC-6106
+is not supported by the client, e.g. Windows). This means that the following
+external services are needed:
+
+1. A DHCP server
+2. An IPv6 router sending Router Advertisements
+3. A DHCPv6 server exporting DNS info
+4. A dynamic DNS server
+
+These components must be configured dynamically and on a per NIC basis.
+The way to do this is by using custom kvm-ifup scripts and hooks.
+
+snf-network
+~~~~~~~~~~~
+
+The snf-network package [1,3] includes custom scripts that will provide the
+aforementioned functionality. `kvm-vif-bridge` and `vif-custom` is an
+alternative to `kvm-ifup` and `vif-ganeti` that take into account all network
+info being exported. Their actions depend on network tags. Specifically:
+
+`dns`: will update an external DDNS server (nsupdate on a bind server)
+
+`ip-less-routed`: will setup routes, rules and proxy ARP
+This setup assumes a pre-existing routing table along with some local
+configuration and provides connectivity to instances via an external
+gateway/router without requiring nodes to have an IP inside this network.
+
+`private-filtered`: will setup ebtables rules to ensure L2 isolation on a
+common bridge. Only packets with the same MAC prefix will be forwarded to the
+corresponding virtual interface.
+
+`nfdhcpd`: will update an external DHCP server
+
+nfdhcpd
+~~~~~~~
+
+snf-network works with nfdhcpd [2,3]: a custom user space DHCP
+server based on NFQUEUE. Currently, nfdhcpd replies on BOOTP/DHCP requests
+originating from a tap or a bridge. Additionally in case of a routed setup it
+provides a ra-stateless configuration by responding to router and neighbour
+solicitations along with DHCPv6 requests for DNS options. Its db is
+dynamically updated using text files inside a local dir with inotify
+(snf-network just adds a per NIC binding file with all relevant info if the
+corresponding network tag is found). Still we need to mangle all these
+packets and send them to the corresponding NFQUEUE.
+
+Known shortcomings
+++++++++++++++++++
+
+Currently the following things are some know weak points of the gnt-network
+design and implementation:
+
+ * Cannot define a network without an IP pool
+ * The pool defines the size of the network
+ * Reserved IPs must be defined explicitly (inconvenient for a big range)
+ * Cannot define an IPv6 only network
+
+Future work
++++++++++++
+
+Any upcoming patches should target:
+
+ * Separate L2, L3, IPv6, IP pool info
+ * Support a set of IP pools per network
+ * Make IP/network in NIC object take a list of entries
+ * Introduce external scripts for node configuration
+ (dynamically create/destroy bridges/routes upon network connect/disconnect)
+
+[1] https://code.grnet.gr/git/snf-network
+[2] https://code.grnet.gr/git/snf-nfdhcpd
+[3] deb http:/apt.dev.grnet.gr/ wheezy/
+
projects / ganeti-local / blobdiff