4 Documents Ganeti version 2.10
6 Ganeti was developed to run on internal, trusted systems. As such, the
7 security model is all-or-nothing.
9 Up to version 2.3 all Ganeti code ran as root. Since version 2.4 it is
10 possible to run all daemons except the node daemon and the monitoring daemon
11 as non-root users by specifying user names and groups at build time.
12 The node daemon continues to require root privileges to create logical volumes,
13 DRBD devices, start instances, etc. Cluster commands can be run as root or by
14 users in a group specified at build time. The monitoring daemon requires root
15 privileges in order to be able to access and present information that are only
16 avilable to root (such as the output of the ``xm`` command of Xen).
21 For a host on which the Ganeti software has been installed, but not
22 joined to a cluster, there are no changes to the system.
24 For a host that has been joined to the cluster, there are very important
27 - The host will have its SSH host key replaced with the one of the
28 cluster (which is the one the initial node had at the cluster
30 - A new public key will be added to root's ``authorized_keys`` file,
31 granting root access to all nodes of the cluster. The private part of
32 the key is also distributed to all nodes. Old files are renamed.
33 - Communication between nodes is encrypted using SSL/TLS. A common key
34 and certificate combo is shared between all nodes of the cluster. At
35 this time, no CA is used.
36 - The Ganeti node daemon will accept RPC requests from any host within
37 the cluster with the correct certificate, and the operations it will
38 do as a result of these requests are:
40 - running commands under the ``/etc/ganeti/hooks`` directory
41 - creating DRBD disks between it and the IP it has been told
42 - overwrite a defined list of files on the host
44 As you can see, as soon as a node is joined, it becomes equal to all
45 other nodes in the cluster, and the security of the cluster is
46 determined by the weakest node.
48 Note that only the SSH key will allow other machines to run any command
49 on this node; the RPC method will run only:
51 - well defined commands to create, remove, activate logical volumes,
52 drbd devices, start/stop instances, etc;
53 - run well-defined SSH commands on other nodes in the cluster
54 - scripts under the ``/etc/ganeti/hooks`` directory
55 - scripts under the ``/etc/ganeti/restricted-commands`` directory, if
56 this feature has been enabled at build time (see below)
58 It is therefore important to make sure that the contents of the
59 ``/etc/ganeti/hooks`` and ``/etc/ganeti/restricted-commands``
60 directories are supervised and only trusted sources can populate them.
65 The restricted commands feature is new in Ganeti 2.7. It enables the
66 administrator to run any commands in the
67 ``/etc/ganeti/restricted-commands`` directory, if the feature has been
68 enabled at build time, subject to the following restrictions:
70 - No parameters may be passed
71 - No absolute or relative path may be passed, only a filename
72 - The ``/etc/ganeti/restricted-commands`` directory must
73 be owned by root:root and have mode 0755 or stricter
74 - Executables must be regular files or symlinks, and must be executable
77 Note that it's not possible to list the contents of the directory, and
78 there is an intentional delay when trying to execute a non-existing
79 command (to slow-down dictionary attacks).
81 Since for Ganeti itself this functionality is not needed, and is only
82 provided as a way to help administrate or recover nodes, it is a local
83 site decision whether to enable or not the restricted commands feature.
85 By default, this feature is disabled.
91 As mentioned above, there are multiple ways of communication between
94 - SSH-based, for high-volume traffic like image dumps or for low-level
95 command, e.g. restarting the Ganeti node daemon
96 - RPC communication between master and nodes
97 - DRBD real-time disk replication traffic
99 The SSH traffic is protected (after the initial login to a new node) by
100 the cluster-wide shared SSH key.
102 RPC communication between the master and nodes is protected using
103 SSL/TLS encryption. Both the client and the server must have the
104 cluster-wide shared SSL/TLS certificate and verify it when establishing
105 the connection by comparing fingerprints. We decided not to use a CA to
106 simplify the key handling.
108 The DRBD traffic is not protected by encryption, as DRBD does not
109 support this. It's therefore recommended to implement host-level
110 firewalling or to use a separate range of IP addresses for the DRBD
111 traffic (this is supported in Ganeti through the use of a secondary
112 interface) which is not routed outside the cluster. DRBD connections are
113 protected from erroneous connections to other machines (as may happen
114 due to software issues), and from accepting connections from other
115 machines, by using a shared secret, exchanged via RPC requests from the
116 master to the nodes when configuring the device.
121 The command-line tools to master daemon communication is done via a
122 UNIX socket, whose permissions are reset to ``0660`` after listening but
123 before serving requests. This permission-based protection is documented
124 and works on Linux, but is not-portable; however, Ganeti doesn't work on
125 non-Linux system at the moment.
130 The ``luxid`` daemon (automatically enabled if ``confd`` is enabled at
131 build time) serves local (UNIX socket) queries about the run-time
132 configuration. Answering these means talking to other cluster nodes,
133 exactly as ``masterd`` does. See the notes for ``masterd`` regarding
134 permission-based protection.
139 In Ganeti 2.8, the ``confd`` daemon (if enabled at build time), serves
140 network-originated queries about parts of the static cluster
143 If Ganeti is not configured (at build time) to use separate users,
144 ``confd`` has access to all Ganeti related files (including internal RPC
145 SSL certificates). This makes it a bit more sensitive to bugs (a remote
146 attacker could get direct access to the intra-cluster RPC), so to harden
147 security it's recommended to:
149 - disable confd at build time if it (and ``luxid``) is not needed in
151 - configure Ganeti (at build time) to use separate users, so that the
152 confd daemon doesn't also have access to the server SSL/TLS
154 - add firewall rules to protect the ``confd`` port or bind it to a
155 trusted address. Make sure that all nodes can access the daemon, as
156 the monitoring daemon requires it.
161 The monitoring daemon provides information about the status and the
162 performance of the cluster over HTTP.
163 It is currently unencrypted and non-authenticated, therefore it is strongly
164 advised to set proper firewalling rules to prevent unwanted access.
166 The monitoring daemon runs as root, because it needs to be able to access
167 privileged information (such as the state of the instances as provided by
168 the Xen hypervisor). Nevertheless, the security implications are mitigated
169 by the fact that the agent only provides reporting functionalities,
170 without the ability to actually modify the state of the cluster.
175 Starting with Ganeti 2.0, Remote API traffic is encrypted using SSL/TLS
176 by default. It supports Basic authentication as per :rfc:`2617`. Users
177 can be granted different capabilities. Details can be found in the
178 :ref:`RAPI documentation <rapi-users>`.
180 Paths for certificate, private key and CA files required for SSL/TLS
181 will be set at source configure time. Symlinks or command line
182 parameters may be used to use different files.
184 Inter-cluster instance moves
185 ----------------------------
187 To move instances between clusters, different clusters must be able to
188 communicate with each other over a secure channel. Up to and including
189 Ganeti 2.1, clusters were self-contained entities and had no knowledge
190 of other clusters. With Ganeti 2.2, clusters can exchange data if tokens
191 (an encryption certificate) was exchanged by a trusted third party
197 When running KVM instances under Ganeti three security models ara
198 available: "none", "user" and "pool".
200 Under security model "none" instances run by default as root. This means
201 that, if an instance gets jail broken, it will be able to own the host
202 node, and thus the ganeti cluster. This is the default model, and the
203 only one available before Ganeti 2.1.2.
205 Under security model "user" an instance is run as the user specified by
206 the hypervisor parameter "security_domain". This makes it easy to run
207 all instances as non privileged users, and allows one to manually
208 allocate specific users to specific instances or sets of instances. If
209 the specified user doesn't have permissions a jail broken instance will
210 need some local privilege escalation before being able to take over the
211 node and the cluster. It's possible though for a jail broken instance to
212 affect other ones running under the same user.
214 Under security model "pool" a global cluster-level uid pool is used to
215 start each instance on the same node under a different user. The uids in
216 the cluster pool can be set with ``gnt-cluster init`` and ``gnt-cluster
217 modify``, and must correspond to existing users on all nodes. Ganeti
218 will then allocate one to each instance, as needed. This way a jail
219 broken instance won't be able to affect any other. Since the users are
220 handed out by ganeti in a per-node randomized way, in this mode there is
221 no way to make sure a particular instance is always run as a certain
222 user. Use mode "user" for that.
224 In addition to these precautions, if you want to avoid instances sending
225 traffic on your node network, you can use an iptables rule such as::
227 iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j LOG \
228 --log-prefix "ganeti uid pool user network traffic"
229 iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j DROP
231 This won't affect regular instance traffic (that comes out of the tapX
232 allocated to the instance, and can be filtered or subject to appropriate
233 policy routes) but will stop any user generated traffic that might come
234 from a jailbroken instance.
236 .. vim: set textwidth=72 :
projects / ganeti-local / blob