root / doc / security.rst @ ab6536ba
History | View | Annotate | Download (10.1 kB)
1 |
Security in Ganeti |
---|---|
2 |
================== |
3 |
|
4 |
Documents Ganeti version 2.9 |
5 |
|
6 |
Ganeti was developed to run on internal, trusted systems. As such, the |
7 |
security model is all-or-nothing. |
8 |
|
9 |
Up to version 2.3 all Ganeti code ran as root. Since version 2.4 it is |
10 |
possible to run all daemons except the node daemon and the monitoring daemon |
11 |
as non-root users by specifying user names and groups at build time. |
12 |
The node daemon continues to require root privileges to create logical volumes, |
13 |
DRBD devices, start instances, etc. Cluster commands can be run as root or by |
14 |
users in a group specified at build time. The monitoring daemon requires root |
15 |
privileges in order to be able to access and present information that are only |
16 |
avilable to root (such as the output of the ``xm`` command of Xen). |
17 |
|
18 |
Host issues |
19 |
----------- |
20 |
|
21 |
For a host on which the Ganeti software has been installed, but not |
22 |
joined to a cluster, there are no changes to the system. |
23 |
|
24 |
For a host that has been joined to the cluster, there are very important |
25 |
changes: |
26 |
|
27 |
- The host will have its SSH host key replaced with the one of the |
28 |
cluster (which is the one the initial node had at the cluster |
29 |
creation) |
30 |
- A new public key will be added to root's ``authorized_keys`` file, |
31 |
granting root access to all nodes of the cluster. The private part of |
32 |
the key is also distributed to all nodes. Old files are renamed. |
33 |
- Communication between nodes is encrypted using SSL/TLS. A common key |
34 |
and certificate combo is shared between all nodes of the cluster. At |
35 |
this time, no CA is used. |
36 |
- The Ganeti node daemon will accept RPC requests from any host within |
37 |
the cluster with the correct certificate, and the operations it will |
38 |
do as a result of these requests are: |
39 |
|
40 |
- running commands under the ``/etc/ganeti/hooks`` directory |
41 |
- creating DRBD disks between it and the IP it has been told |
42 |
- overwrite a defined list of files on the host |
43 |
|
44 |
As you can see, as soon as a node is joined, it becomes equal to all |
45 |
other nodes in the cluster, and the security of the cluster is |
46 |
determined by the weakest node. |
47 |
|
48 |
Note that only the SSH key will allow other machines to run any command |
49 |
on this node; the RPC method will run only: |
50 |
|
51 |
- well defined commands to create, remove, activate logical volumes, |
52 |
drbd devices, start/stop instances, etc; |
53 |
- run well-defined SSH commands on other nodes in the cluster |
54 |
- scripts under the ``/etc/ganeti/hooks`` directory |
55 |
- scripts under the ``/etc/ganeti/restricted-commands`` directory, if |
56 |
this feature has been enabled at build time (see below) |
57 |
|
58 |
It is therefore important to make sure that the contents of the |
59 |
``/etc/ganeti/hooks`` and ``/etc/ganeti/restricted-commands`` |
60 |
directories are supervised and only trusted sources can populate them. |
61 |
|
62 |
Restricted commands |
63 |
~~~~~~~~~~~~~~~~~~~ |
64 |
|
65 |
The restricted commands feature is new in Ganeti 2.7. It enables the |
66 |
administrator to run any commands in the |
67 |
``/etc/ganeti/restricted-commands`` directory, if the feature has been |
68 |
enabled at build time, subject to the following restrictions: |
69 |
|
70 |
- No parameters may be passed |
71 |
- No absolute or relative path may be passed, only a filename |
72 |
- The ``/etc/ganeti/restricted-commands`` directory must |
73 |
be owned by root:root and have mode 0755 or stricter |
74 |
- Executables must be regular files or symlinks, and must be executable |
75 |
by root:root |
76 |
|
77 |
Note that it's not possible to list the contents of the directory, and |
78 |
there is an intentional delay when trying to execute a non-existing |
79 |
command (to slow-down dictionary attacks). |
80 |
|
81 |
Since for Ganeti itself this functionality is not needed, and is only |
82 |
provided as a way to help administrate or recover nodes, it is a local |
83 |
site decision whether to enable or not the restricted commands feature. |
84 |
|
85 |
By default, this feature is disabled. |
86 |
|
87 |
|
88 |
Cluster issues |
89 |
-------------- |
90 |
|
91 |
As mentioned above, there are multiple ways of communication between |
92 |
cluster nodes: |
93 |
|
94 |
- SSH-based, for high-volume traffic like image dumps or for low-level |
95 |
command, e.g. restarting the Ganeti node daemon |
96 |
- RPC communication between master and nodes |
97 |
- DRBD real-time disk replication traffic |
98 |
|
99 |
The SSH traffic is protected (after the initial login to a new node) by |
100 |
the cluster-wide shared SSH key. |
101 |
|
102 |
RPC communication between the master and nodes is protected using |
103 |
SSL/TLS encryption. Both the client and the server must have the |
104 |
cluster-wide shared SSL/TLS certificate and verify it when establishing |
105 |
the connection by comparing fingerprints. We decided not to use a CA to |
106 |
simplify the key handling. |
107 |
|
108 |
The DRBD traffic is not protected by encryption, as DRBD does not |
109 |
support this. It's therefore recommended to implement host-level |
110 |
firewalling or to use a separate range of IP addresses for the DRBD |
111 |
traffic (this is supported in Ganeti through the use of a secondary |
112 |
interface) which is not routed outside the cluster. DRBD connections are |
113 |
protected from erroneous connections to other machines (as may happen |
114 |
due to software issues), and from accepting connections from other |
115 |
machines, by using a shared secret, exchanged via RPC requests from the |
116 |
master to the nodes when configuring the device. |
117 |
|
118 |
Master daemon |
119 |
------------- |
120 |
|
121 |
The command-line tools to master daemon communication is done via a |
122 |
UNIX socket, whose permissions are reset to ``0660`` after listening but |
123 |
before serving requests. This permission-based protection is documented |
124 |
and works on Linux, but is not-portable; however, Ganeti doesn't work on |
125 |
non-Linux system at the moment. |
126 |
|
127 |
Luxi daemon |
128 |
----------- |
129 |
|
130 |
The ``luxid`` daemon (automatically enabled if ``confd`` is enabled at |
131 |
build time) serves local (UNIX socket) queries about the run-time |
132 |
configuration. Answering these means talking to other cluster nodes, |
133 |
exactly as ``masterd`` does. See the notes for ``masterd`` regarding |
134 |
permission-based protection. |
135 |
|
136 |
Conf daemon |
137 |
----------- |
138 |
|
139 |
In Ganeti 2.8, the ``confd`` daemon (if enabled at build time), serves |
140 |
network-originated queries about parts of the static cluster |
141 |
configuration. |
142 |
|
143 |
If Ganeti is not configured (at build time) to use separate users, |
144 |
``confd`` has access to all Ganeti related files (including internal RPC |
145 |
SSL certificates). This makes it a bit more sensitive to bugs (a remote |
146 |
attacker could get direct access to the intra-cluster RPC), so to harden |
147 |
security it's recommended to: |
148 |
|
149 |
- disable confd at build time if it (and ``luxid``) is not needed in |
150 |
your setup. |
151 |
- configure Ganeti (at build time) to use separate users, so that the |
152 |
confd daemon doesn't also have access to the server SSL/TLS |
153 |
certificates. |
154 |
- add firewall rules to protect the ``confd`` port or bind it to a |
155 |
trusted address. Make sure that all nodes can access the daemon, as |
156 |
the monitoring daemon requires it. |
157 |
|
158 |
Monitoring daemon |
159 |
----------------- |
160 |
|
161 |
The monitoring daemon provides information about the status and the |
162 |
performance of the cluster over HTTP. |
163 |
It is currently unencrypted and non-authenticated, therefore it is strongly |
164 |
advised to set proper firewalling rules to prevent unwanted access. |
165 |
|
166 |
The monitoring daemon runs as root, because it needs to be able to access |
167 |
privileged information (such as the state of the instances as provided by |
168 |
the Xen hypervisor). Nevertheless, the security implications are mitigated |
169 |
by the fact that the agent only provides reporting functionalities, |
170 |
without the ability to actually modify the state of the cluster. |
171 |
|
172 |
Remote API |
173 |
---------- |
174 |
|
175 |
Starting with Ganeti 2.0, Remote API traffic is encrypted using SSL/TLS |
176 |
by default. It supports Basic authentication as per :rfc:`2617`. Users |
177 |
can be granted different capabilities. Details can be found in the |
178 |
:ref:`RAPI documentation <rapi-users>`. |
179 |
|
180 |
Paths for certificate, private key and CA files required for SSL/TLS |
181 |
will be set at source configure time. Symlinks or command line |
182 |
parameters may be used to use different files. |
183 |
|
184 |
Inter-cluster instance moves |
185 |
---------------------------- |
186 |
|
187 |
To move instances between clusters, different clusters must be able to |
188 |
communicate with each other over a secure channel. Up to and including |
189 |
Ganeti 2.1, clusters were self-contained entities and had no knowledge |
190 |
of other clusters. With Ganeti 2.2, clusters can exchange data if tokens |
191 |
(an encryption certificate) was exchanged by a trusted third party |
192 |
before. |
193 |
|
194 |
KVM Security |
195 |
------------ |
196 |
|
197 |
When running KVM instances under Ganeti three security models ara |
198 |
available: "none", "user" and "pool". |
199 |
|
200 |
Under security model "none" instances run by default as root. This means |
201 |
that, if an instance gets jail broken, it will be able to own the host |
202 |
node, and thus the ganeti cluster. This is the default model, and the |
203 |
only one available before Ganeti 2.1.2. |
204 |
|
205 |
Under security model "user" an instance is run as the user specified by |
206 |
the hypervisor parameter "security_domain". This makes it easy to run |
207 |
all instances as non privileged users, and allows one to manually |
208 |
allocate specific users to specific instances or sets of instances. If |
209 |
the specified user doesn't have permissions a jail broken instance will |
210 |
need some local privilege escalation before being able to take over the |
211 |
node and the cluster. It's possible though for a jail broken instance to |
212 |
affect other ones running under the same user. |
213 |
|
214 |
Under security model "pool" a global cluster-level uid pool is used to |
215 |
start each instance on the same node under a different user. The uids in |
216 |
the cluster pool can be set with ``gnt-cluster init`` and ``gnt-cluster |
217 |
modify``, and must correspond to existing users on all nodes. Ganeti |
218 |
will then allocate one to each instance, as needed. This way a jail |
219 |
broken instance won't be able to affect any other. Since the users are |
220 |
handed out by ganeti in a per-node randomized way, in this mode there is |
221 |
no way to make sure a particular instance is always run as a certain |
222 |
user. Use mode "user" for that. |
223 |
|
224 |
In addition to these precautions, if you want to avoid instances sending |
225 |
traffic on your node network, you can use an iptables rule such as:: |
226 |
|
227 |
iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j LOG \ |
228 |
--log-prefix "ganeti uid pool user network traffic" |
229 |
iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j DROP |
230 |
|
231 |
This won't affect regular instance traffic (that comes out of the tapX |
232 |
allocated to the instance, and can be filtered or subject to appropriate |
233 |
policy routes) but will stop any user generated traffic that might come |
234 |
from a jailbroken instance. |
235 |
|
236 |
.. vim: set textwidth=72 : |
237 |
.. Local Variables: |
238 |
.. mode: rst |
239 |
.. fill-column: 72 |
240 |
.. End: |