root / doc / security.rst @ 0a689f79
History | View | Annotate | Download (9.7 kB)
1 |
Security in Ganeti |
---|---|
2 |
================== |
3 |
|
4 |
Documents Ganeti version 2.7 |
5 |
|
6 |
Ganeti was developed to run on internal, trusted systems. As such, the |
7 |
security model is all-or-nothing. |
8 |
|
9 |
Up to version 2.3 all Ganeti code ran as root. Since version 2.4 it is |
10 |
possible to run all daemons except the node daemon and the monitoring daemon |
11 |
as non-root users by specifying user names and groups at build time. |
12 |
The node daemon continues to require root privileges to create logical volumes, |
13 |
DRBD devices, start instances, etc. Cluster commands can be run as root or by |
14 |
users in a group specified at build time. The monitoring daemon requires root |
15 |
privileges in order to be able to access and present information that are only |
16 |
avilable to root (such as the output of the ``xm`` command of Xen). |
17 |
|
18 |
Host issues |
19 |
----------- |
20 |
|
21 |
For a host on which the Ganeti software has been installed, but not |
22 |
joined to a cluster, there are no changes to the system. |
23 |
|
24 |
For a host that has been joined to the cluster, there are very important |
25 |
changes: |
26 |
|
27 |
- The host will have its SSH host key replaced with the one of the |
28 |
cluster (which is the one the initial node had at the cluster |
29 |
creation) |
30 |
- A new public key will be added to root's ``authorized_keys`` file, |
31 |
granting root access to all nodes of the cluster. The private part of |
32 |
the key is also distributed to all nodes. Old files are renamed. |
33 |
- Communication between nodes is encrypted using SSL/TLS. A common key |
34 |
and certificate combo is shared between all nodes of the cluster. At |
35 |
this time, no CA is used. |
36 |
- The Ganeti node daemon will accept RPC requests from any host within |
37 |
the cluster with the correct certificate, and the operations it will |
38 |
do as a result of these requests are: |
39 |
|
40 |
- running commands under the ``/etc/ganeti/hooks`` directory |
41 |
- creating DRBD disks between it and the IP it has been told |
42 |
- overwrite a defined list of files on the host |
43 |
|
44 |
As you can see, as soon as a node is joined, it becomes equal to all |
45 |
other nodes in the cluster, and the security of the cluster is |
46 |
determined by the weakest node. |
47 |
|
48 |
Note that only the SSH key will allow other machines to run any command |
49 |
on this node; the RPC method will run only: |
50 |
|
51 |
- well defined commands to create, remove, activate logical volumes, |
52 |
drbd devices, start/stop instances, etc; |
53 |
- run well-defined SSH commands on other nodes in the cluster |
54 |
- scripts under the ``/etc/ganeti/hooks`` directory |
55 |
- scripts under the ``/etc/ganeti/restricted-commands`` directory, if |
56 |
this feature has been enabled at build time (see below) |
57 |
|
58 |
It is therefore important to make sure that the contents of the |
59 |
``/etc/ganeti/hooks`` and ``/etc/ganeti/restricted-commands`` |
60 |
directories are supervised and only trusted sources can populate them. |
61 |
|
62 |
Restricted commands |
63 |
~~~~~~~~~~~~~~~~~~~ |
64 |
|
65 |
The restricted commands feature is new in Ganeti 2.7. It enables the |
66 |
administrator to run any commands in the |
67 |
``/etc/ganeti/restricted-commands`` directory, if the feature has been |
68 |
enabled at build time, subject to the following restrictions: |
69 |
|
70 |
- No parameters may be passed |
71 |
- No absolute or relative path may be passed, only a filename |
72 |
- The ``/etc/ganeti/restricted-commands`` directory must |
73 |
be owned by root:root and have mode 0755 or stricter |
74 |
- Executables must be regular files or symlinks, and must be executable |
75 |
by root:root |
76 |
|
77 |
Note that it's not possible to list the contents of the directory, and |
78 |
there is an intentional delay when trying to execute a non-existing |
79 |
command (to slow-down dictionary attacks). |
80 |
|
81 |
Since for Ganeti itself this functionality is not needed, and is only |
82 |
provided as a way to help administrate or recover nodes, it is a local |
83 |
site decision whether to enable or not the restricted commands feature. |
84 |
|
85 |
By default, this feature is disabled. |
86 |
|
87 |
|
88 |
Cluster issues |
89 |
-------------- |
90 |
|
91 |
As mentioned above, there are multiple ways of communication between |
92 |
cluster nodes: |
93 |
|
94 |
- SSH-based, for high-volume traffic like image dumps or for low-level |
95 |
command, e.g. restarting the Ganeti node daemon |
96 |
- RPC communication between master and nodes |
97 |
- DRBD real-time disk replication traffic |
98 |
|
99 |
The SSH traffic is protected (after the initial login to a new node) by |
100 |
the cluster-wide shared SSH key. |
101 |
|
102 |
RPC communication between the master and nodes is protected using |
103 |
SSL/TLS encryption. Both the client and the server must have the |
104 |
cluster-wide shared SSL/TLS certificate and verify it when establishing |
105 |
the connection by comparing fingerprints. We decided not to use a CA to |
106 |
simplify the key handling. |
107 |
|
108 |
The DRBD traffic is not protected by encryption, as DRBD does not |
109 |
support this. It's therefore recommended to implement host-level |
110 |
firewalling or to use a separate range of IP addresses for the DRBD |
111 |
traffic (this is supported in Ganeti through the use of a secondary |
112 |
interface) which is not routed outside the cluster. DRBD connections are |
113 |
protected from erroneous connections to other machines (as may happen |
114 |
due to software issues), and from accepting connections from other |
115 |
machines, by using a shared secret, exchanged via RPC requests from the |
116 |
master to the nodes when configuring the device. |
117 |
|
118 |
Master daemon |
119 |
------------- |
120 |
|
121 |
The command-line tools to master daemon communication is done via a |
122 |
UNIX socket, whose permissions are reset to ``0660`` after listening but |
123 |
before serving requests. This permission-based protection is documented |
124 |
and works on Linux, but is not-portable; however, Ganeti doesn't work on |
125 |
non-Linux system at the moment. |
126 |
|
127 |
Conf daemon |
128 |
----------- |
129 |
|
130 |
In Ganeti 2.7, the ``confd`` daemon (if enabled at build time), serves |
131 |
both network-originated queries (about the static configuration) and |
132 |
local (UNIX socket) queries (about the run-time configuration; answering |
133 |
these means talking to other cluster nodes, which makes use of the |
134 |
internal RPC SSL certificate). This makes it a bit more sensitive to |
135 |
bugs (a remote attacker could get direct access to the intra-cluster |
136 |
RPC), so to harden security it's recommended to: |
137 |
|
138 |
- disable confd at build time if it's not needed in your setup |
139 |
- otherwise, configure Ganeti (at build time) to use separate users, so |
140 |
that the confd daemon doesn't also have access to the server SSL/TLS |
141 |
certificates |
142 |
|
143 |
It is planned to split the two functionalities (local/remote querying) |
144 |
of confd into two separate daemons in a future Ganeti version. |
145 |
|
146 |
Monitoring daemon |
147 |
----------------- |
148 |
|
149 |
The monitoring daemon provides information about the status and the |
150 |
performance of the cluster over HTTP. |
151 |
It is currently unencrypted and non-authenticated, therefore it is strongly |
152 |
advised to set proper firewalling rules to prevent unwanted access. |
153 |
|
154 |
The monitoring daemon runs as root, because it needs to be able to access |
155 |
privileged information (such as the state of the instances as provided by |
156 |
the Xen hypervisor). Nevertheless, the security implications are mitigated |
157 |
by the fact that the agent only provides reporting functionalities, |
158 |
without the ability to actually modify the state of the cluster. |
159 |
|
160 |
Remote API |
161 |
---------- |
162 |
|
163 |
Starting with Ganeti 2.0, Remote API traffic is encrypted using SSL/TLS |
164 |
by default. It supports Basic authentication as per :rfc:`2617`. Users |
165 |
can be granted different capabilities. Details can be found in the |
166 |
:ref:`RAPI documentation <rapi-users>`. |
167 |
|
168 |
Paths for certificate, private key and CA files required for SSL/TLS |
169 |
will be set at source configure time. Symlinks or command line |
170 |
parameters may be used to use different files. |
171 |
|
172 |
Inter-cluster instance moves |
173 |
---------------------------- |
174 |
|
175 |
To move instances between clusters, different clusters must be able to |
176 |
communicate with each other over a secure channel. Up to and including |
177 |
Ganeti 2.1, clusters were self-contained entities and had no knowledge |
178 |
of other clusters. With Ganeti 2.2, clusters can exchange data if tokens |
179 |
(an encryption certificate) was exchanged by a trusted third party |
180 |
before. |
181 |
|
182 |
KVM Security |
183 |
------------ |
184 |
|
185 |
When running KVM instances under Ganeti three security models ara |
186 |
available: "none", "user" and "pool". |
187 |
|
188 |
Under security model "none" instances run by default as root. This means |
189 |
that, if an instance gets jail broken, it will be able to own the host |
190 |
node, and thus the ganeti cluster. This is the default model, and the |
191 |
only one available before Ganeti 2.1.2. |
192 |
|
193 |
Under security model "user" an instance is run as the user specified by |
194 |
the hypervisor parameter "security_domain". This makes it easy to run |
195 |
all instances as non privileged users, and allows one to manually |
196 |
allocate specific users to specific instances or sets of instances. If |
197 |
the specified user doesn't have permissions a jail broken instance will |
198 |
need some local privilege escalation before being able to take over the |
199 |
node and the cluster. It's possible though for a jail broken instance to |
200 |
affect other ones running under the same user. |
201 |
|
202 |
Under security model "pool" a global cluster-level uid pool is used to |
203 |
start each instance on the same node under a different user. The uids in |
204 |
the cluster pool can be set with ``gnt-cluster init`` and ``gnt-cluster |
205 |
modify``, and must correspond to existing users on all nodes. Ganeti |
206 |
will then allocate one to each instance, as needed. This way a jail |
207 |
broken instance won't be able to affect any other. Since the users are |
208 |
handed out by ganeti in a per-node randomized way, in this mode there is |
209 |
no way to make sure a particular instance is always run as a certain |
210 |
user. Use mode "user" for that. |
211 |
|
212 |
In addition to these precautions, if you want to avoid instances sending |
213 |
traffic on your node network, you can use an iptables rule such as:: |
214 |
|
215 |
iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j LOG \ |
216 |
--log-prefix "ganeti uid pool user network traffic" |
217 |
iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j DROP |
218 |
|
219 |
This won't affect regular instance traffic (that comes out of the tapX |
220 |
allocated to the instance, and can be filtered or subject to appropriate |
221 |
policy routes) but will stop any user generated traffic that might come |
222 |
from a jailbroken instance. |
223 |
|
224 |
.. vim: set textwidth=72 : |
225 |
.. Local Variables: |
226 |
.. mode: rst |
227 |
.. fill-column: 72 |
228 |
.. End: |