root / doc / security.rst @ 992fd37d
History | View | Annotate | Download (10.4 kB)
1 |
Security in Ganeti |
---|---|
2 |
================== |
3 |
|
4 |
Documents Ganeti version 2.11 |
5 |
|
6 |
Ganeti was developed to run on internal, trusted systems. As such, the |
7 |
security model is all-or-nothing. |
8 |
|
9 |
Up to version 2.3 all Ganeti code ran as root. Since version 2.4 it is |
10 |
possible to run all daemons except the node daemon and the monitoring daemon |
11 |
as non-root users by specifying user names and groups at build time. |
12 |
The node daemon continues to require root privileges to create logical volumes, |
13 |
DRBD devices, start instances, etc. Cluster commands can be run as root or by |
14 |
users in a group specified at build time. The monitoring daemon requires root |
15 |
privileges in order to be able to access and present information that are only |
16 |
avilable to root (such as the output of the ``xm`` command of Xen). |
17 |
|
18 |
Host issues |
19 |
----------- |
20 |
|
21 |
For a host on which the Ganeti software has been installed, but not |
22 |
joined to a cluster, there are no changes to the system. |
23 |
|
24 |
For a host that has been joined to the cluster, there are very important |
25 |
changes: |
26 |
|
27 |
- The host will have its SSH host key replaced with the one of the |
28 |
cluster (which is the one the initial node had at the cluster |
29 |
creation) |
30 |
- A new public key will be added to root's ``authorized_keys`` file, |
31 |
granting root access to all nodes of the cluster. The private part of |
32 |
the key is also distributed to all nodes. Old files are renamed. |
33 |
- Communication between nodes is encrypted using SSL/TLS. A common key |
34 |
and certificate combo is shared between all nodes of the cluster. At |
35 |
this time, no CA is used. |
36 |
- The Ganeti node daemon will accept RPC requests from any host that is |
37 |
master candidate within the cluster, and the operations it will |
38 |
do as a result of these requests are: |
39 |
|
40 |
- running commands under the ``/etc/ganeti/hooks`` directory |
41 |
- creating DRBD disks between it and the IP it has been told |
42 |
- overwrite a defined list of files on the host |
43 |
|
44 |
As you can see, as soon as a node is joined, it becomes equal to all |
45 |
other nodes in the cluster wrt to SSH and equal to all non-master |
46 |
candidate nodes wrt to RPC, and the security of the cluster is |
47 |
determined by the weakest node. |
48 |
|
49 |
Note that only the SSH key will allow other machines to run any command |
50 |
on this node; the RPC method will run only: |
51 |
|
52 |
- well defined commands to create, remove, activate logical volumes, |
53 |
drbd devices, start/stop instances, etc; |
54 |
- run well-defined SSH commands on other nodes in the cluster |
55 |
- scripts under the ``/etc/ganeti/hooks`` directory |
56 |
- scripts under the ``/etc/ganeti/restricted-commands`` directory, if |
57 |
this feature has been enabled at build time (see below) |
58 |
|
59 |
It is therefore important to make sure that the contents of the |
60 |
``/etc/ganeti/hooks`` and ``/etc/ganeti/restricted-commands`` |
61 |
directories are supervised and only trusted sources can populate them. |
62 |
|
63 |
Restricted commands |
64 |
~~~~~~~~~~~~~~~~~~~ |
65 |
|
66 |
The restricted commands feature is new in Ganeti 2.7. It enables the |
67 |
administrator to run any commands in the |
68 |
``/etc/ganeti/restricted-commands`` directory, if the feature has been |
69 |
enabled at build time, subject to the following restrictions: |
70 |
|
71 |
- No parameters may be passed |
72 |
- No absolute or relative path may be passed, only a filename |
73 |
- The ``/etc/ganeti/restricted-commands`` directory must |
74 |
be owned by root:root and have mode 0755 or stricter |
75 |
- Executables must be regular files or symlinks, and must be executable |
76 |
by root:root |
77 |
|
78 |
Note that it's not possible to list the contents of the directory, and |
79 |
there is an intentional delay when trying to execute a non-existing |
80 |
command (to slow-down dictionary attacks). |
81 |
|
82 |
Since for Ganeti itself this functionality is not needed, and is only |
83 |
provided as a way to help administrate or recover nodes, it is a local |
84 |
site decision whether to enable or not the restricted commands feature. |
85 |
|
86 |
By default, this feature is disabled. |
87 |
|
88 |
|
89 |
Cluster issues |
90 |
-------------- |
91 |
|
92 |
As mentioned above, there are multiple ways of communication between |
93 |
cluster nodes: |
94 |
|
95 |
- SSH-based, for high-volume traffic like image dumps or for low-level |
96 |
command, e.g. restarting the Ganeti node daemon |
97 |
- RPC communication between master and nodes |
98 |
- DRBD real-time disk replication traffic |
99 |
|
100 |
The SSH traffic is protected (after the initial login to a new node) by |
101 |
the cluster-wide shared SSH key. |
102 |
|
103 |
RPC communication between the master and nodes is protected using |
104 |
SSL/TLS encryption. The server must have must have the cluster-wide |
105 |
shared SSL/TLS certificate. When acting as a client, the nodes use an |
106 |
individual SSL/TLS certificate. On incoming requests, the server checks |
107 |
whether the client's certificate is that of a master candidate by |
108 |
verifying its finterprint to a list of known master candidate |
109 |
certificates. We decided not to use a CA (yet) to simplify the key |
110 |
handling. |
111 |
|
112 |
The DRBD traffic is not protected by encryption, as DRBD does not |
113 |
support this. It's therefore recommended to implement host-level |
114 |
firewalling or to use a separate range of IP addresses for the DRBD |
115 |
traffic (this is supported in Ganeti through the use of a secondary |
116 |
interface) which is not routed outside the cluster. DRBD connections are |
117 |
protected from erroneous connections to other machines (as may happen |
118 |
due to software issues), and from accepting connections from other |
119 |
machines, by using a shared secret, exchanged via RPC requests from the |
120 |
master to the nodes when configuring the device. |
121 |
|
122 |
Master daemon |
123 |
------------- |
124 |
|
125 |
The command-line tools to master daemon communication is done via a |
126 |
UNIX socket, whose permissions are reset to ``0660`` after listening but |
127 |
before serving requests. This permission-based protection is documented |
128 |
and works on Linux, but is not-portable; however, Ganeti doesn't work on |
129 |
non-Linux system at the moment. |
130 |
|
131 |
Luxi daemon |
132 |
----------- |
133 |
|
134 |
The ``luxid`` daemon (automatically enabled if ``confd`` is enabled at |
135 |
build time) serves local (UNIX socket) queries about the run-time |
136 |
configuration. Answering these means talking to other cluster nodes, |
137 |
exactly as ``masterd`` does. See the notes for ``masterd`` regarding |
138 |
permission-based protection. |
139 |
|
140 |
Conf daemon |
141 |
----------- |
142 |
|
143 |
In Ganeti 2.8, the ``confd`` daemon (if enabled at build time), serves |
144 |
network-originated queries about parts of the static cluster |
145 |
configuration. |
146 |
|
147 |
If Ganeti is not configured (at build time) to use separate users, |
148 |
``confd`` has access to all Ganeti related files (including internal RPC |
149 |
SSL certificates). This makes it a bit more sensitive to bugs (a remote |
150 |
attacker could get direct access to the intra-cluster RPC), so to harden |
151 |
security it's recommended to: |
152 |
|
153 |
- disable confd at build time if it (and ``luxid``) is not needed in |
154 |
your setup. |
155 |
- configure Ganeti (at build time) to use separate users, so that the |
156 |
confd daemon doesn't also have access to the server SSL/TLS |
157 |
certificates. |
158 |
- add firewall rules to protect the ``confd`` port or bind it to a |
159 |
trusted address. Make sure that all nodes can access the daemon, as |
160 |
the monitoring daemon requires it. |
161 |
|
162 |
Monitoring daemon |
163 |
----------------- |
164 |
|
165 |
The monitoring daemon provides information about the status and the |
166 |
performance of the cluster over HTTP. |
167 |
It is currently unencrypted and non-authenticated, therefore it is strongly |
168 |
advised to set proper firewalling rules to prevent unwanted access. |
169 |
|
170 |
The monitoring daemon runs as root, because it needs to be able to access |
171 |
privileged information (such as the state of the instances as provided by |
172 |
the Xen hypervisor). Nevertheless, the security implications are mitigated |
173 |
by the fact that the agent only provides reporting functionalities, |
174 |
without the ability to actually modify the state of the cluster. |
175 |
|
176 |
Remote API |
177 |
---------- |
178 |
|
179 |
Starting with Ganeti 2.0, Remote API traffic is encrypted using SSL/TLS |
180 |
by default. It supports Basic authentication as per :rfc:`2617`. Users |
181 |
can be granted different capabilities. Details can be found in the |
182 |
:ref:`RAPI documentation <rapi-users>`. |
183 |
|
184 |
Paths for certificate, private key and CA files required for SSL/TLS |
185 |
will be set at source configure time. Symlinks or command line |
186 |
parameters may be used to use different files. |
187 |
|
188 |
Inter-cluster instance moves |
189 |
---------------------------- |
190 |
|
191 |
To move instances between clusters, different clusters must be able to |
192 |
communicate with each other over a secure channel. Up to and including |
193 |
Ganeti 2.1, clusters were self-contained entities and had no knowledge |
194 |
of other clusters. With Ganeti 2.2, clusters can exchange data if tokens |
195 |
(an encryption certificate) was exchanged by a trusted third party |
196 |
before. |
197 |
|
198 |
KVM Security |
199 |
------------ |
200 |
|
201 |
When running KVM instances under Ganeti three security models ara |
202 |
available: "none", "user" and "pool". |
203 |
|
204 |
Under security model "none" instances run by default as root. This means |
205 |
that, if an instance gets jail broken, it will be able to own the host |
206 |
node, and thus the ganeti cluster. This is the default model, and the |
207 |
only one available before Ganeti 2.1.2. |
208 |
|
209 |
Under security model "user" an instance is run as the user specified by |
210 |
the hypervisor parameter "security_domain". This makes it easy to run |
211 |
all instances as non privileged users, and allows one to manually |
212 |
allocate specific users to specific instances or sets of instances. If |
213 |
the specified user doesn't have permissions a jail broken instance will |
214 |
need some local privilege escalation before being able to take over the |
215 |
node and the cluster. It's possible though for a jail broken instance to |
216 |
affect other ones running under the same user. |
217 |
|
218 |
Under security model "pool" a global cluster-level uid pool is used to |
219 |
start each instance on the same node under a different user. The uids in |
220 |
the cluster pool can be set with ``gnt-cluster init`` and ``gnt-cluster |
221 |
modify``, and must correspond to existing users on all nodes. Ganeti |
222 |
will then allocate one to each instance, as needed. This way a jail |
223 |
broken instance won't be able to affect any other. Since the users are |
224 |
handed out by ganeti in a per-node randomized way, in this mode there is |
225 |
no way to make sure a particular instance is always run as a certain |
226 |
user. Use mode "user" for that. |
227 |
|
228 |
In addition to these precautions, if you want to avoid instances sending |
229 |
traffic on your node network, you can use an iptables rule such as:: |
230 |
|
231 |
iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j LOG \ |
232 |
--log-prefix "ganeti uid pool user network traffic" |
233 |
iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j DROP |
234 |
|
235 |
This won't affect regular instance traffic (that comes out of the tapX |
236 |
allocated to the instance, and can be filtered or subject to appropriate |
237 |
policy routes) but will stop any user generated traffic that might come |
238 |
from a jailbroken instance. |
239 |
|
240 |
.. vim: set textwidth=72 : |
241 |
.. Local Variables: |
242 |
.. mode: rst |
243 |
.. fill-column: 72 |
244 |
.. End: |