Security in Ganeti
==================
+Documents Ganeti version 2.7
+
Ganeti was developed to run on internal, trusted systems. As such, the
security model is all-or-nothing.
-All the Ganeti code runs as root, because all the operations that Ganeti
-is doing require privileges: creating logical volumes, drbd devices,
-starting instances, etc. Running as root does not mean setuid, but that
-you need to be root to run the cluster commands.
+Up to version 2.3 all Ganeti code ran as root. Since version 2.4 it is
+possible to run all daemons except the node daemon as non-root users by
+specifying user names and groups at build time. The node daemon
+continues to require root privileges to create logical volumes, DRBD
+devices, start instances, etc. Cluster commands can be run as root or by
+users in a group specified at build time.
Host issues
-----------
-For a host on which the Ganeti software has been installed, but not joined to a
-cluster, there are no changes to the system.
+For a host on which the Ganeti software has been installed, but not
+joined to a cluster, there are no changes to the system.
For a host that has been joined to the cluster, there are very important
changes:
- The host will have its SSH host key replaced with the one of the
cluster (which is the one the initial node had at the cluster
creation)
-- A new public key will be added to root's authorized_keys file,
+- A new public key will be added to root's ``authorized_keys`` file,
granting root access to all nodes of the cluster. The private part of
the key is also distributed to all nodes. Old files are renamed.
- Communication between nodes is encrypted using SSL/TLS. A common key
the cluster with the correct certificate, and the operations it will
do as a result of these requests are:
- - running commands under the /etc/ganeti/hooks directory
+ - running commands under the ``/etc/ganeti/hooks`` directory
- creating DRBD disks between it and the IP it has been told
- overwrite a defined list of files on the host
other nodes in the cluster, and the security of the cluster is
determined by the weakest node.
-Note that only the SSH key will allow other machines to run random
-commands on this node; the RPC method will run only:
+Note that only the SSH key will allow other machines to run any command
+on this node; the RPC method will run only:
- well defined commands to create, remove, activate logical volumes,
drbd devices, start/stop instances, etc;
-- run SSH commands on other nodes in the cluster, again well-defined
-- scripts under the /etc/ganeti/hooks directory
+- run well-defined SSH commands on other nodes in the cluster
+- scripts under the ``/etc/ganeti/hooks`` directory
+- scripts under the ``/etc/ganeti/restricted-commands`` directory, if
+ this feature has been enabled at build time (see below)
It is therefore important to make sure that the contents of the
-/etc/ganeti/hooks directory is supervised and only trusted sources can
-populate it.
+``/etc/ganeti/hooks`` and ``/etc/ganeti/restricted-commands``
+directories are supervised and only trusted sources can populate them.
+
+Restricted commands
+~~~~~~~~~~~~~~~~~~~
+
+The restricted commands feature is new in Ganeti 2.7. It enables the
+administrator to run any commands in the
+``/etc/ganeti/restricted-commands`` directory, if the feature has been
+enabled at build time, subject to the following restrictions:
+
+- No parameters may be passed
+- No absolute or relative path may be passed, only a filename
+- The ``/etc/ganeti/restricted-commands`` directory must
+ be owned by root:root and have mode 0755 or stricter
+- Executables must be regular files or symlinks, and must be executable
+ by root:root
+
+Note that it's not possible to list the contents of the directory, and
+there is an intentional delay when trying to execute a non-existing
+command (to slow-down dictionary attacks).
+
+Since for Ganeti itself this functionality is not needed, and is only
+provided as a way to help administrate or recover nodes, it is a local
+site decision whether to enable or not the restricted commands feature.
+
+By default, this feature is disabled.
+
Cluster issues
--------------
-As told above, there are multiple ways of communication between cluster
-nodes:
+As mentioned above, there are multiple ways of communication between
+cluster nodes:
- SSH-based, for high-volume traffic like image dumps or for low-level
command, e.g. restarting the Ganeti node daemon
The SSH traffic is protected (after the initial login to a new node) by
the cluster-wide shared SSH key.
-RPC communication between the master and nodes is protected using SSL/TLS
-encryption. Both the client and the server must have the cluster-wide
-shared SSL/TLS certificate and verify it when establishing the connection
-by comparing fingerprints. We decided not to use a CA to simplify the
-key handling.
+RPC communication between the master and nodes is protected using
+SSL/TLS encryption. Both the client and the server must have the
+cluster-wide shared SSL/TLS certificate and verify it when establishing
+the connection by comparing fingerprints. We decided not to use a CA to
+simplify the key handling.
The DRBD traffic is not protected by encryption, as DRBD does not
support this. It's therefore recommended to implement host-level
firewalling or to use a separate range of IP addresses for the DRBD
-traffic (this is supported in Ganeti) which is not routed outside the
-cluster. DRBD connections are protected from connecting due to bugs to
-other machines, and from accepting connections from other machines, by
-using a shared secret, exchanged via RPC requests from the master to the
-nodes when configuring the device.
+traffic (this is supported in Ganeti through the use of a secondary
+interface) which is not routed outside the cluster. DRBD connections are
+protected from erroneous connections to other machines (as may happen
+due to software issues), and from accepting connections from other
+machines, by using a shared secret, exchanged via RPC requests from the
+master to the nodes when configuring the device.
Master daemon
-------------
-The command-line tools to master daemon communication is done via an UNIX
-socket, whose permissions are reset to ``0600`` after listening but before
-serving requests. This permission-based protection is documented and works on
-Linux, but is not-portable; however, Ganeti doesn't work on non-Linux system at
-the moment.
+The command-line tools to master daemon communication is done via a
+UNIX socket, whose permissions are reset to ``0660`` after listening but
+before serving requests. This permission-based protection is documented
+and works on Linux, but is not-portable; however, Ganeti doesn't work on
+non-Linux system at the moment.
+
+Conf daemon
+-----------
+
+In Ganeti 2.7, the ``confd`` daemon (if enabled at build time), serves
+both network-originated queries (about the static configuration) and
+local (UNIX socket) queries (about the run-time configuration; answering
+these means talking to other cluster nodes, which makes use of the
+internal RPC SSL certificate). This makes it a bit more sensitive to
+bugs (a remote attacker could get direct access to the intra-cluster
+RPC), so to harden security it's recommended to:
+
+- disable confd at build time if it's not needed in your setup
+- otherwise, configure Ganeti (at build time) to use separate users, so
+ that the confd daemon doesn't also have access to the server SSL/TLS
+ certificates
+
+It is planned to split the two functionalities (local/remote querying)
+of confd into two separate daemons in a future Ganeti version.
Remote API
----------
-Starting with Ganeti 2.0, Remote API traffic is encrypted using SSL/TLS by
-default. It supports Basic authentication as per RFC2617.
-
-Paths for certificate, private key and CA files required for SSL/TLS will
-be set at source configure time. Symlinks or command line parameters may
-be used to use different files.
+Starting with Ganeti 2.0, Remote API traffic is encrypted using SSL/TLS
+by default. It supports Basic authentication as per :rfc:`2617`. Users
+can be granted different capabilities. Details can be found in the
+:ref:`RAPI documentation <rapi-users>`.
+
+Paths for certificate, private key and CA files required for SSL/TLS
+will be set at source configure time. Symlinks or command line
+parameters may be used to use different files.
+
+Inter-cluster instance moves
+----------------------------
+
+To move instances between clusters, different clusters must be able to
+communicate with each other over a secure channel. Up to and including
+Ganeti 2.1, clusters were self-contained entities and had no knowledge
+of other clusters. With Ganeti 2.2, clusters can exchange data if tokens
+(an encryption certificate) was exchanged by a trusted third party
+before.
+
+KVM Security
+------------
+
+When running KVM instances under Ganeti three security models ara
+available: "none", "user" and "pool".
+
+Under security model "none" instances run by default as root. This means
+that, if an instance gets jail broken, it will be able to own the host
+node, and thus the ganeti cluster. This is the default model, and the
+only one available before Ganeti 2.1.2.
+
+Under security model "user" an instance is run as the user specified by
+the hypervisor parameter "security_domain". This makes it easy to run
+all instances as non privileged users, and allows one to manually
+allocate specific users to specific instances or sets of instances. If
+the specified user doesn't have permissions a jail broken instance will
+need some local privilege escalation before being able to take over the
+node and the cluster. It's possible though for a jail broken instance to
+affect other ones running under the same user.
+
+Under security model "pool" a global cluster-level uid pool is used to
+start each instance on the same node under a different user. The uids in
+the cluster pool can be set with ``gnt-cluster init`` and ``gnt-cluster
+modify``, and must correspond to existing users on all nodes. Ganeti
+will then allocate one to each instance, as needed. This way a jail
+broken instance won't be able to affect any other. Since the users are
+handed out by ganeti in a per-node randomized way, in this mode there is
+no way to make sure a particular instance is always run as a certain
+user. Use mode "user" for that.
+
+In addition to these precautions, if you want to avoid instances sending
+traffic on your node network, you can use an iptables rule such as::
+
+ iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j LOG \
+ --log-prefix "ganeti uid pool user network traffic"
+ iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j DROP
+
+This won't affect regular instance traffic (that comes out of the tapX
+allocated to the instance, and can be filtered or subject to appropriate
+policy routes) but will stop any user generated traffic that might come
+from a jailbroken instance.
+
+.. vim: set textwidth=72 :
+.. Local Variables:
+.. mode: rst
+.. fill-column: 72
+.. End:
projects / ganeti-local / blobdiff