Revision 213ffbb1 doc/admin.rst
b/doc/admin.rst | ||
---|---|---|
816 | 816 |
disks), because we needed to get rid of both the original nodes (A and |
817 | 817 |
B). |
818 | 818 |
|
819 |
Network Management |
|
820 |
------------------ |
|
821 |
|
|
822 |
Ganeti used to describe NICs of an Instance with an IP, a MAC, a connectivity |
|
823 |
link and mode. This had three major shortcomings: |
|
824 |
|
|
825 |
* there was no easy way to assign a unique IP to an instance |
|
826 |
* network info (subnet, gateway, domain, etc.) was not available on target |
|
827 |
node (kvm-ifup, hooks, etc) |
|
828 |
* one should explicitly pass L2 info (mode, and link) to every NIC |
|
829 |
|
|
830 |
Plus there was no easy way to get the current networking overview (which |
|
831 |
instances are on the same L2 or L3 network, which IPs are reserved, etc). |
|
832 |
|
|
833 |
All the above required an external management tool that has an overall view |
|
834 |
and provides the corresponding info to Ganeti. |
|
835 |
|
|
836 |
gnt-network aims to support a big part of this functionality inside Ganeti and |
|
837 |
abstract the network as a separate entity. Currently, a Ganeti network |
|
838 |
provides the following: |
|
839 |
|
|
840 |
* A single IPv4 pool, subnet and gateway |
|
841 |
* Connectivity info per nodegroup (mode, link) |
|
842 |
* MAC prefix for each NIC inside the network |
|
843 |
* IPv6 prefix/Gateway related to this network |
|
844 |
* Tags |
|
845 |
|
|
846 |
IP pool management ensures IP uniqueness inside this network. The user can |
|
847 |
pass `ip=pool,network=test` and will: |
|
848 |
|
|
849 |
1. Get the first available IP in the pool |
|
850 |
2. Inherit the connectivity mode and link of the network's netparams |
|
851 |
3. NIC will obtain the MAC prefix of the network |
|
852 |
4. All network related info will be available as environment variables in |
|
853 |
kvm-ifup scripts and hooks, so that they can dynamically manage all |
|
854 |
networking-related setup on the host. |
|
855 |
|
|
856 |
Hands on with gnt-network |
|
857 |
+++++++++++++++++++++++++ |
|
858 |
|
|
859 |
To create a network do:: |
|
860 |
|
|
861 |
# gnt-network add --network=192.0.2.0/24 --gateway=192.0.2.1 test |
|
862 |
|
|
863 |
Please see all other available options (--add-reserved-ips, --mac-prefix, |
|
864 |
--network6, --subnet6, --tags). |
|
865 |
|
|
866 |
To make this network available on a nodegroup you should specify the |
|
867 |
connectivity mode and link during connection:: |
|
868 |
|
|
869 |
# gnt-network connect test bridged br100 default nodegroup1 |
|
870 |
|
|
871 |
To add a NIC inside this network:: |
|
872 |
|
|
873 |
# gnt-instance modify --net -1:add,ip=pool,network=test inst1 |
|
874 |
|
|
875 |
This will let a NIC obtain a unique IP inside this network, and inherit the |
|
876 |
nodegroup's netparams (bridged, br100). IP here is optional. If missing the |
|
877 |
NIC will just get the L2 info. |
|
878 |
|
|
879 |
To move an existing NIC from a network to another and remove its IP:: |
|
880 |
|
|
881 |
# gnt-instance modify --net -1:ip=none,network=test1 inst1 |
|
882 |
|
|
883 |
This will release the old IP from the old IP pool and the NIC will inherit the |
|
884 |
new nicparams. |
|
885 |
|
|
886 |
On the above actions there is a extra option `--no-conflicts-ckeck`. This |
|
887 |
does not check for conflicting setups. Specifically: |
|
888 |
|
|
889 |
1. When a network is added, IPs of nodes and master are not being checked. |
|
890 |
2. When connecting a network on a nodegroup, IPs of instances inside this |
|
891 |
nodegroup are not checked whether they reside inside the subnet or not. |
|
892 |
3. When specifying explicitly a IP without passing a network, Ganeti will not |
|
893 |
check if this IP is included inside any available network on the nodegroup. |
|
894 |
|
|
895 |
External components |
|
896 |
+++++++++++++++++++ |
|
897 |
|
|
898 |
All the aforementioned steps assure NIC configuration from the Ganeti |
|
899 |
perspective. Of course this has nothing to do, how the instance eventually will |
|
900 |
get the desired connectivity (IPv4, IPv6, default routes, DNS info, etc) and |
|
901 |
where will the IP resolve. This functionality is managed by the external |
|
902 |
components. |
|
903 |
|
|
904 |
Let's assume that the VM will need to obtain a dynamic IP via DHCP, get a SLAAC |
|
905 |
address, and use DHCPv6 for other configuration information (in case RFC-6106 |
|
906 |
is not supported by the client, e.g. Windows). This means that the following |
|
907 |
external services are needed: |
|
908 |
|
|
909 |
1. A DHCP server |
|
910 |
2. An IPv6 router sending Router Advertisements |
|
911 |
3. A DHCPv6 server exporting DNS info |
|
912 |
4. A dynamic DNS server |
|
913 |
|
|
914 |
These components must be configured dynamically and on a per NIC basis. |
|
915 |
The way to do this is by using custom kvm-ifup scripts and hooks. |
|
916 |
|
|
917 |
snf-network |
|
918 |
~~~~~~~~~~~ |
|
919 |
|
|
920 |
The snf-network package [1,3] includes custom scripts that will provide the |
|
921 |
aforementioned functionality. `kvm-vif-bridge` and `vif-custom` is an |
|
922 |
alternative to `kvm-ifup` and `vif-ganeti` that take into account all network |
|
923 |
info being exported. Their actions depend on network tags. Specifically: |
|
924 |
|
|
925 |
`dns`: will update an external DDNS server (nsupdate on a bind server) |
|
926 |
|
|
927 |
`ip-less-routed`: will setup routes, rules and proxy ARP |
|
928 |
This setup assumes a pre-existing routing table along with some local |
|
929 |
configuration and provides connectivity to instances via an external |
|
930 |
gateway/router without requiring nodes to have an IP inside this network. |
|
931 |
|
|
932 |
`private-filtered`: will setup ebtables rules to ensure L2 isolation on a |
|
933 |
common bridge. Only packets with the same MAC prefix will be forwarded to the |
|
934 |
corresponding virtual interface. |
|
935 |
|
|
936 |
`nfdhcpd`: will update an external DHCP server |
|
937 |
|
|
938 |
nfdhcpd |
|
939 |
~~~~~~~ |
|
940 |
|
|
941 |
snf-network works with nfdhcpd [2,3]: a custom user space DHCP |
|
942 |
server based on NFQUEUE. Currently, nfdhcpd replies on BOOTP/DHCP requests |
|
943 |
originating from a tap or a bridge. Additionally in case of a routed setup it |
|
944 |
provides a ra-stateless configuration by responding to router and neighbour |
|
945 |
solicitations along with DHCPv6 requests for DNS options. Its db is |
|
946 |
dynamically updated using text files inside a local dir with inotify |
|
947 |
(snf-network just adds a per NIC binding file with all relevant info if the |
|
948 |
corresponding network tag is found). Still we need to mangle all these |
|
949 |
packets and send them to the corresponding NFQUEUE. |
|
950 |
|
|
951 |
Known shortcomings |
|
952 |
++++++++++++++++++ |
|
953 |
|
|
954 |
Currently the following things are some know weak points of the gnt-network |
|
955 |
design and implementation: |
|
956 |
|
|
957 |
* Cannot define a network without an IP pool |
|
958 |
* The pool defines the size of the network |
|
959 |
* Reserved IPs must be defined explicitly (inconvenient for a big range) |
|
960 |
* Cannot define an IPv6 only network |
|
961 |
|
|
962 |
Future work |
|
963 |
+++++++++++ |
|
964 |
|
|
965 |
Any upcoming patches should target: |
|
966 |
|
|
967 |
* Separate L2, L3, IPv6, IP pool info |
|
968 |
* Support a set of IP pools per network |
|
969 |
* Make IP/network in NIC object take a list of entries |
|
970 |
* Introduce external scripts for node configuration |
|
971 |
(dynamically create/destroy bridges/routes upon network connect/disconnect) |
|
972 |
|
|
973 |
[1] https://code.grnet.gr/git/snf-network |
|
974 |
[2] https://code.grnet.gr/git/snf-nfdhcpd |
|
975 |
[3] deb http:/apt.dev.grnet.gr/ wheezy/ |
|
976 |
|
|
819 | 977 |
Node operations |
820 | 978 |
--------------- |
821 | 979 |
|
Also available in: Unified diff