root / doc / security.rst @ 992fd37d
History | View | Annotate | Download (10.4 kB)
1 | 28cc354f | Iustin Pop | Security in Ganeti |
---|---|---|---|
2 | 6884c0ca | Iustin Pop | ================== |
3 | 28cc354f | Iustin Pop | |
4 | c7a02959 | Thomas Thrainer | Documents Ganeti version 2.11 |
5 | 3397d13e | Michael Hanselmann | |
6 | 28cc354f | Iustin Pop | Ganeti was developed to run on internal, trusted systems. As such, the |
7 | 28cc354f | Iustin Pop | security model is all-or-nothing. |
8 | 28cc354f | Iustin Pop | |
9 | 3397d13e | Michael Hanselmann | Up to version 2.3 all Ganeti code ran as root. Since version 2.4 it is |
10 | 0a689f79 | Michele Tartara | possible to run all daemons except the node daemon and the monitoring daemon |
11 | 0a689f79 | Michele Tartara | as non-root users by specifying user names and groups at build time. |
12 | 0a689f79 | Michele Tartara | The node daemon continues to require root privileges to create logical volumes, |
13 | 0a689f79 | Michele Tartara | DRBD devices, start instances, etc. Cluster commands can be run as root or by |
14 | 0a689f79 | Michele Tartara | users in a group specified at build time. The monitoring daemon requires root |
15 | 0a689f79 | Michele Tartara | privileges in order to be able to access and present information that are only |
16 | 0a689f79 | Michele Tartara | avilable to root (such as the output of the ``xm`` command of Xen). |
17 | 28cc354f | Iustin Pop | |
18 | 28cc354f | Iustin Pop | Host issues |
19 | 28cc354f | Iustin Pop | ----------- |
20 | 28cc354f | Iustin Pop | |
21 | 7faf5110 | Michael Hanselmann | For a host on which the Ganeti software has been installed, but not |
22 | 7faf5110 | Michael Hanselmann | joined to a cluster, there are no changes to the system. |
23 | 28cc354f | Iustin Pop | |
24 | 28cc354f | Iustin Pop | For a host that has been joined to the cluster, there are very important |
25 | 28cc354f | Iustin Pop | changes: |
26 | 6884c0ca | Iustin Pop | |
27 | 6884c0ca | Iustin Pop | - The host will have its SSH host key replaced with the one of the |
28 | 6884c0ca | Iustin Pop | cluster (which is the one the initial node had at the cluster |
29 | 6884c0ca | Iustin Pop | creation) |
30 | 3397d13e | Michael Hanselmann | - A new public key will be added to root's ``authorized_keys`` file, |
31 | 6884c0ca | Iustin Pop | granting root access to all nodes of the cluster. The private part of |
32 | 6884c0ca | Iustin Pop | the key is also distributed to all nodes. Old files are renamed. |
33 | 6884c0ca | Iustin Pop | - Communication between nodes is encrypted using SSL/TLS. A common key |
34 | 6884c0ca | Iustin Pop | and certificate combo is shared between all nodes of the cluster. At |
35 | 6884c0ca | Iustin Pop | this time, no CA is used. |
36 | 992fd37d | Helga Velroyen | - The Ganeti node daemon will accept RPC requests from any host that is |
37 | 992fd37d | Helga Velroyen | master candidate within the cluster, and the operations it will |
38 | 6884c0ca | Iustin Pop | do as a result of these requests are: |
39 | 6884c0ca | Iustin Pop | |
40 | 3397d13e | Michael Hanselmann | - running commands under the ``/etc/ganeti/hooks`` directory |
41 | 6884c0ca | Iustin Pop | - creating DRBD disks between it and the IP it has been told |
42 | 6884c0ca | Iustin Pop | - overwrite a defined list of files on the host |
43 | 28cc354f | Iustin Pop | |
44 | 28cc354f | Iustin Pop | As you can see, as soon as a node is joined, it becomes equal to all |
45 | 992fd37d | Helga Velroyen | other nodes in the cluster wrt to SSH and equal to all non-master |
46 | 992fd37d | Helga Velroyen | candidate nodes wrt to RPC, and the security of the cluster is |
47 | 28cc354f | Iustin Pop | determined by the weakest node. |
48 | 28cc354f | Iustin Pop | |
49 | 3397d13e | Michael Hanselmann | Note that only the SSH key will allow other machines to run any command |
50 | 3397d13e | Michael Hanselmann | on this node; the RPC method will run only: |
51 | 6884c0ca | Iustin Pop | |
52 | 6884c0ca | Iustin Pop | - well defined commands to create, remove, activate logical volumes, |
53 | 6884c0ca | Iustin Pop | drbd devices, start/stop instances, etc; |
54 | 3397d13e | Michael Hanselmann | - run well-defined SSH commands on other nodes in the cluster |
55 | 3397d13e | Michael Hanselmann | - scripts under the ``/etc/ganeti/hooks`` directory |
56 | d70571bb | Iustin Pop | - scripts under the ``/etc/ganeti/restricted-commands`` directory, if |
57 | d70571bb | Iustin Pop | this feature has been enabled at build time (see below) |
58 | 28cc354f | Iustin Pop | |
59 | 28cc354f | Iustin Pop | It is therefore important to make sure that the contents of the |
60 | d70571bb | Iustin Pop | ``/etc/ganeti/hooks`` and ``/etc/ganeti/restricted-commands`` |
61 | d70571bb | Iustin Pop | directories are supervised and only trusted sources can populate them. |
62 | d70571bb | Iustin Pop | |
63 | d70571bb | Iustin Pop | Restricted commands |
64 | d70571bb | Iustin Pop | ~~~~~~~~~~~~~~~~~~~ |
65 | d70571bb | Iustin Pop | |
66 | d70571bb | Iustin Pop | The restricted commands feature is new in Ganeti 2.7. It enables the |
67 | d70571bb | Iustin Pop | administrator to run any commands in the |
68 | d70571bb | Iustin Pop | ``/etc/ganeti/restricted-commands`` directory, if the feature has been |
69 | d70571bb | Iustin Pop | enabled at build time, subject to the following restrictions: |
70 | d70571bb | Iustin Pop | |
71 | d70571bb | Iustin Pop | - No parameters may be passed |
72 | d70571bb | Iustin Pop | - No absolute or relative path may be passed, only a filename |
73 | d70571bb | Iustin Pop | - The ``/etc/ganeti/restricted-commands`` directory must |
74 | d70571bb | Iustin Pop | be owned by root:root and have mode 0755 or stricter |
75 | d70571bb | Iustin Pop | - Executables must be regular files or symlinks, and must be executable |
76 | d70571bb | Iustin Pop | by root:root |
77 | d70571bb | Iustin Pop | |
78 | d70571bb | Iustin Pop | Note that it's not possible to list the contents of the directory, and |
79 | d70571bb | Iustin Pop | there is an intentional delay when trying to execute a non-existing |
80 | d70571bb | Iustin Pop | command (to slow-down dictionary attacks). |
81 | d70571bb | Iustin Pop | |
82 | d70571bb | Iustin Pop | Since for Ganeti itself this functionality is not needed, and is only |
83 | d70571bb | Iustin Pop | provided as a way to help administrate or recover nodes, it is a local |
84 | d70571bb | Iustin Pop | site decision whether to enable or not the restricted commands feature. |
85 | d70571bb | Iustin Pop | |
86 | d70571bb | Iustin Pop | By default, this feature is disabled. |
87 | d70571bb | Iustin Pop | |
88 | 28cc354f | Iustin Pop | |
89 | 28cc354f | Iustin Pop | Cluster issues |
90 | 28cc354f | Iustin Pop | -------------- |
91 | 28cc354f | Iustin Pop | |
92 | 3397d13e | Michael Hanselmann | As mentioned above, there are multiple ways of communication between |
93 | 3397d13e | Michael Hanselmann | cluster nodes: |
94 | 6884c0ca | Iustin Pop | |
95 | 6884c0ca | Iustin Pop | - SSH-based, for high-volume traffic like image dumps or for low-level |
96 | 6884c0ca | Iustin Pop | command, e.g. restarting the Ganeti node daemon |
97 | 6884c0ca | Iustin Pop | - RPC communication between master and nodes |
98 | 6884c0ca | Iustin Pop | - DRBD real-time disk replication traffic |
99 | 28cc354f | Iustin Pop | |
100 | 4fbe765c | Michael Hanselmann | The SSH traffic is protected (after the initial login to a new node) by |
101 | 4fbe765c | Michael Hanselmann | the cluster-wide shared SSH key. |
102 | 28cc354f | Iustin Pop | |
103 | 7faf5110 | Michael Hanselmann | RPC communication between the master and nodes is protected using |
104 | 992fd37d | Helga Velroyen | SSL/TLS encryption. The server must have must have the cluster-wide |
105 | 992fd37d | Helga Velroyen | shared SSL/TLS certificate. When acting as a client, the nodes use an |
106 | 992fd37d | Helga Velroyen | individual SSL/TLS certificate. On incoming requests, the server checks |
107 | 992fd37d | Helga Velroyen | whether the client's certificate is that of a master candidate by |
108 | 992fd37d | Helga Velroyen | verifying its finterprint to a list of known master candidate |
109 | 992fd37d | Helga Velroyen | certificates. We decided not to use a CA (yet) to simplify the key |
110 | 992fd37d | Helga Velroyen | handling. |
111 | 28cc354f | Iustin Pop | |
112 | 6884c0ca | Iustin Pop | The DRBD traffic is not protected by encryption, as DRBD does not |
113 | 6884c0ca | Iustin Pop | support this. It's therefore recommended to implement host-level |
114 | 888891c7 | Michael Hanselmann | firewalling or to use a separate range of IP addresses for the DRBD |
115 | 3397d13e | Michael Hanselmann | traffic (this is supported in Ganeti through the use of a secondary |
116 | 3397d13e | Michael Hanselmann | interface) which is not routed outside the cluster. DRBD connections are |
117 | 3397d13e | Michael Hanselmann | protected from erroneous connections to other machines (as may happen |
118 | 3397d13e | Michael Hanselmann | due to software issues), and from accepting connections from other |
119 | 3397d13e | Michael Hanselmann | machines, by using a shared secret, exchanged via RPC requests from the |
120 | 3397d13e | Michael Hanselmann | master to the nodes when configuring the device. |
121 | 6884c0ca | Iustin Pop | |
122 | 6884c0ca | Iustin Pop | Master daemon |
123 | 6884c0ca | Iustin Pop | ------------- |
124 | 6884c0ca | Iustin Pop | |
125 | 3397d13e | Michael Hanselmann | The command-line tools to master daemon communication is done via a |
126 | 3397d13e | Michael Hanselmann | UNIX socket, whose permissions are reset to ``0660`` after listening but |
127 | 7faf5110 | Michael Hanselmann | before serving requests. This permission-based protection is documented |
128 | 7faf5110 | Michael Hanselmann | and works on Linux, but is not-portable; however, Ganeti doesn't work on |
129 | 7faf5110 | Michael Hanselmann | non-Linux system at the moment. |
130 | 4fbe765c | Michael Hanselmann | |
131 | b51e14c0 | Thomas Thrainer | Luxi daemon |
132 | b51e14c0 | Thomas Thrainer | ----------- |
133 | b51e14c0 | Thomas Thrainer | |
134 | b51e14c0 | Thomas Thrainer | The ``luxid`` daemon (automatically enabled if ``confd`` is enabled at |
135 | b51e14c0 | Thomas Thrainer | build time) serves local (UNIX socket) queries about the run-time |
136 | b51e14c0 | Thomas Thrainer | configuration. Answering these means talking to other cluster nodes, |
137 | b51e14c0 | Thomas Thrainer | exactly as ``masterd`` does. See the notes for ``masterd`` regarding |
138 | b51e14c0 | Thomas Thrainer | permission-based protection. |
139 | b51e14c0 | Thomas Thrainer | |
140 | d70571bb | Iustin Pop | Conf daemon |
141 | d70571bb | Iustin Pop | ----------- |
142 | d70571bb | Iustin Pop | |
143 | cd30f3e6 | Michele Tartara | In Ganeti 2.8, the ``confd`` daemon (if enabled at build time), serves |
144 | b51e14c0 | Thomas Thrainer | network-originated queries about parts of the static cluster |
145 | b51e14c0 | Thomas Thrainer | configuration. |
146 | b51e14c0 | Thomas Thrainer | |
147 | b51e14c0 | Thomas Thrainer | If Ganeti is not configured (at build time) to use separate users, |
148 | b51e14c0 | Thomas Thrainer | ``confd`` has access to all Ganeti related files (including internal RPC |
149 | b51e14c0 | Thomas Thrainer | SSL certificates). This makes it a bit more sensitive to bugs (a remote |
150 | b51e14c0 | Thomas Thrainer | attacker could get direct access to the intra-cluster RPC), so to harden |
151 | b51e14c0 | Thomas Thrainer | security it's recommended to: |
152 | b51e14c0 | Thomas Thrainer | |
153 | b51e14c0 | Thomas Thrainer | - disable confd at build time if it (and ``luxid``) is not needed in |
154 | b51e14c0 | Thomas Thrainer | your setup. |
155 | b51e14c0 | Thomas Thrainer | - configure Ganeti (at build time) to use separate users, so that the |
156 | b51e14c0 | Thomas Thrainer | confd daemon doesn't also have access to the server SSL/TLS |
157 | cd30f3e6 | Michele Tartara | certificates. |
158 | b51e14c0 | Thomas Thrainer | - add firewall rules to protect the ``confd`` port or bind it to a |
159 | b51e14c0 | Thomas Thrainer | trusted address. Make sure that all nodes can access the daemon, as |
160 | b51e14c0 | Thomas Thrainer | the monitoring daemon requires it. |
161 | d70571bb | Iustin Pop | |
162 | 0a689f79 | Michele Tartara | Monitoring daemon |
163 | 0a689f79 | Michele Tartara | ----------------- |
164 | 0a689f79 | Michele Tartara | |
165 | 0a689f79 | Michele Tartara | The monitoring daemon provides information about the status and the |
166 | 0a689f79 | Michele Tartara | performance of the cluster over HTTP. |
167 | 0a689f79 | Michele Tartara | It is currently unencrypted and non-authenticated, therefore it is strongly |
168 | 0a689f79 | Michele Tartara | advised to set proper firewalling rules to prevent unwanted access. |
169 | 0a689f79 | Michele Tartara | |
170 | 0a689f79 | Michele Tartara | The monitoring daemon runs as root, because it needs to be able to access |
171 | 0a689f79 | Michele Tartara | privileged information (such as the state of the instances as provided by |
172 | 0a689f79 | Michele Tartara | the Xen hypervisor). Nevertheless, the security implications are mitigated |
173 | 0a689f79 | Michele Tartara | by the fact that the agent only provides reporting functionalities, |
174 | 0a689f79 | Michele Tartara | without the ability to actually modify the state of the cluster. |
175 | 0a689f79 | Michele Tartara | |
176 | 4fbe765c | Michael Hanselmann | Remote API |
177 | 4fbe765c | Michael Hanselmann | ---------- |
178 | 4fbe765c | Michael Hanselmann | |
179 | 7faf5110 | Michael Hanselmann | Starting with Ganeti 2.0, Remote API traffic is encrypted using SSL/TLS |
180 | f1d243bd | Michael Hanselmann | by default. It supports Basic authentication as per :rfc:`2617`. Users |
181 | f1d243bd | Michael Hanselmann | can be granted different capabilities. Details can be found in the |
182 | f1d243bd | Michael Hanselmann | :ref:`RAPI documentation <rapi-users>`. |
183 | 4fbe765c | Michael Hanselmann | |
184 | 7faf5110 | Michael Hanselmann | Paths for certificate, private key and CA files required for SSL/TLS |
185 | 7faf5110 | Michael Hanselmann | will be set at source configure time. Symlinks or command line |
186 | 7faf5110 | Michael Hanselmann | parameters may be used to use different files. |
187 | 558fd122 | Michael Hanselmann | |
188 | 5b2069a9 | Michael Hanselmann | Inter-cluster instance moves |
189 | 5b2069a9 | Michael Hanselmann | ---------------------------- |
190 | 5b2069a9 | Michael Hanselmann | |
191 | 5b2069a9 | Michael Hanselmann | To move instances between clusters, different clusters must be able to |
192 | 5b2069a9 | Michael Hanselmann | communicate with each other over a secure channel. Up to and including |
193 | 5b2069a9 | Michael Hanselmann | Ganeti 2.1, clusters were self-contained entities and had no knowledge |
194 | 5b2069a9 | Michael Hanselmann | of other clusters. With Ganeti 2.2, clusters can exchange data if tokens |
195 | 5b2069a9 | Michael Hanselmann | (an encryption certificate) was exchanged by a trusted third party |
196 | 5b2069a9 | Michael Hanselmann | before. |
197 | 5b2069a9 | Michael Hanselmann | |
198 | 4e9dcb8a | Guido Trotter | KVM Security |
199 | 4e9dcb8a | Guido Trotter | ------------ |
200 | 4e9dcb8a | Guido Trotter | |
201 | 4e9dcb8a | Guido Trotter | When running KVM instances under Ganeti three security models ara |
202 | 3397d13e | Michael Hanselmann | available: "none", "user" and "pool". |
203 | 4e9dcb8a | Guido Trotter | |
204 | 3397d13e | Michael Hanselmann | Under security model "none" instances run by default as root. This means |
205 | 4e9dcb8a | Guido Trotter | that, if an instance gets jail broken, it will be able to own the host |
206 | 4e9dcb8a | Guido Trotter | node, and thus the ganeti cluster. This is the default model, and the |
207 | 4e9dcb8a | Guido Trotter | only one available before Ganeti 2.1.2. |
208 | 4e9dcb8a | Guido Trotter | |
209 | 3397d13e | Michael Hanselmann | Under security model "user" an instance is run as the user specified by |
210 | 3397d13e | Michael Hanselmann | the hypervisor parameter "security_domain". This makes it easy to run |
211 | f7b769b1 | Iustin Pop | all instances as non privileged users, and allows one to manually |
212 | f7b769b1 | Iustin Pop | allocate specific users to specific instances or sets of instances. If |
213 | f7b769b1 | Iustin Pop | the specified user doesn't have permissions a jail broken instance will |
214 | f7b769b1 | Iustin Pop | need some local privilege escalation before being able to take over the |
215 | f7b769b1 | Iustin Pop | node and the cluster. It's possible though for a jail broken instance to |
216 | 4e9dcb8a | Guido Trotter | affect other ones running under the same user. |
217 | 4e9dcb8a | Guido Trotter | |
218 | 3397d13e | Michael Hanselmann | Under security model "pool" a global cluster-level uid pool is used to |
219 | 4e9dcb8a | Guido Trotter | start each instance on the same node under a different user. The uids in |
220 | 4e9dcb8a | Guido Trotter | the cluster pool can be set with ``gnt-cluster init`` and ``gnt-cluster |
221 | 4e9dcb8a | Guido Trotter | modify``, and must correspond to existing users on all nodes. Ganeti |
222 | 4e9dcb8a | Guido Trotter | will then allocate one to each instance, as needed. This way a jail |
223 | 4e9dcb8a | Guido Trotter | broken instance won't be able to affect any other. Since the users are |
224 | 4e9dcb8a | Guido Trotter | handed out by ganeti in a per-node randomized way, in this mode there is |
225 | 4e9dcb8a | Guido Trotter | no way to make sure a particular instance is always run as a certain |
226 | 3397d13e | Michael Hanselmann | user. Use mode "user" for that. |
227 | 4e9dcb8a | Guido Trotter | |
228 | 4e9dcb8a | Guido Trotter | In addition to these precautions, if you want to avoid instances sending |
229 | 4e9dcb8a | Guido Trotter | traffic on your node network, you can use an iptables rule such as:: |
230 | 4e9dcb8a | Guido Trotter | |
231 | 4e9dcb8a | Guido Trotter | iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j LOG \ |
232 | 4e9dcb8a | Guido Trotter | --log-prefix "ganeti uid pool user network traffic" |
233 | 4e9dcb8a | Guido Trotter | iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j DROP |
234 | 4e9dcb8a | Guido Trotter | |
235 | 4e9dcb8a | Guido Trotter | This won't affect regular instance traffic (that comes out of the tapX |
236 | 4e9dcb8a | Guido Trotter | allocated to the instance, and can be filtered or subject to appropriate |
237 | 4e9dcb8a | Guido Trotter | policy routes) but will stop any user generated traffic that might come |
238 | 4e9dcb8a | Guido Trotter | from a jailbroken instance. |
239 | 4e9dcb8a | Guido Trotter | |
240 | 558fd122 | Michael Hanselmann | .. vim: set textwidth=72 : |
241 | c71a1a3d | Iustin Pop | .. Local Variables: |
242 | c71a1a3d | Iustin Pop | .. mode: rst |
243 | c71a1a3d | Iustin Pop | .. fill-column: 72 |
244 | c71a1a3d | Iustin Pop | .. End: |