snf-occi's documentation!
====================================
-**snf-occi** snf-occi implements the OCCI specification on top of synnefo’s API in order to achieve greater interoperability in common tasks referring cyclades management. This module is a translation bridge between OCCI and the Openstack API and is designed to be as independent as possible from the rest IaaS, providing an OCCI compatibility layer to other services using Openstack API.
+**snf-occi** implements the OCCI specification on top of synnefo’s API in order to achieve greater interoperability. This module is a translation bridge between OCCI and the synnefo endpoints. It is designed to be as independent as possible from the rest IaaS, providing an OCCI compatibility layer.
+
+**snf-occi** utilizes the API library for the components of the synnefo ecosystem provided by `kamaki <http://www.synnefo.org/docs/kamaki/latest>`_ . It is built on top of kamaki.clients lib in order to communicate with the required synnefo APIs.
-**snf-occi** is based in modules provided by kamaki cli-tool when dealing with REST API calls to Openstack.
.. toctree::
:maxdepth: 2
The master document for the OCCI specification is at `OCCI Specification <http://occi-wg.org/about/specification/>`_
-OCCI and Cyclades
+OCCI and Synnefo
-----------------
-The OCCI implementation in Cyclades is going to be based in the OCCI Infrastructure specification, in which common Cloud IaaS components are described. The correspondence between OCCI and Cyclades is as follows:
+The OCCI implementation for Synnefo is in accordance with the OCCI Infrastructure specification, that describes common Cloud IaaS components. The correspondence between OCCI and Cyclades is as follows:
+-------------------------+-------------------------+
|OCCI |Cyclades |
+-------------------------+-------------------------+
|Resource Template |Synnefo flavors |
+-------------------------+-------------------------+
-|Networking |NA |
+|Network |Synnefo networks |
+-------------------------+-------------------------+
|Storage |NA |
+-------------------------+-------------------------+
**Note:** Metadata info in synnefo's servers cannot be represented (clearly) using OCCI's components.
-OCCI requirements
-------------------
-Due to OCCI's structure there cannot be straightforward mapping to Cyclades/OpenStack API. The missing elements are networking and storage capabilities using current Cyclades API.
-
OCCI operations
-****************
+------------------
Below you can see the required procedures/operations for OCCI compatibility.
* Handling the query interface
* Query interface must be found under path /-/
* Retrieve all registered Kinds, Actions and Mixins
- * Add a mixin definition
- * Remove a mixin definition
+ * Add a Mixin definition
+ * Remove a Mixin definition
* Operation on paths in the name-space
* Retrieving the state of the name-space hierarchy
OCCI client/server library
-==========================
+---------------------------
-pyssf is a collection of OCCI python modules. It aims to provide a high-level interface for the integration of OCCI to other new or existing applications.
+**pyssf** is a collection of OCCI python modules. It aims at providing a high-level interface for the integration of OCCI to other new or existing applications.
-Features:
----------
+**Features:**
-* It includes a REST API service with the OCCI specifications already implemented
-* It only requires a custom backend and registry to interact with Cyclades
-Current progress
-=================
-By now we have considered implementing only the **Compute** backend of the OCCI to Cyclades/Openstack API bridge and we are planning to extend it for **networking** and **storage** capabilities.
+* It includes the implementation of a REST API service with the OCCI specifications already implemented
+* It only requires a custom backend and registry to interact with synnefo main components (e.g., Cyclades, Astakos. etc.)
-Installation
+snf-occi Installation
+======================
+
+Requirements
-------------
First, you need to install the required dependencies which can be found here:
-* `pyssf <https://pypi.python.org/pypi/pyssf>`_
-* `kamaki <https://code.grnet.gr/projects/kamaki>`_
+* `pyssf at pypi <https://pypi.python.org/pypi/pyssf>`_
+* `Kamaki repository <https://code.grnet.gr/projects/kamaki>`_ (Detailed description for installation of kamaki can be found in this `Guide <http://www.synnefo.org/docs/kamaki/latest/installation.html>`_).
+
+Moreover, some additional packages need to be installed::
+
+$ apt-get install python-pip
+$ pip install webob
+$ apt-get install python-dev
+$ pip install eventlet
+$ apt-get install python-m2crypto
+$ apt-get install python-pastedeploy
+$ apt-get install libvomsapi1
+
+
+Installation
+-------------
+
+Upon the completion of the previous steps, you can install **snf-occi** API translation server by cloning our latest source code:
+
+* `snf-occi Repository <https://code.grnet.gr/git/snf-occi>`_
+
+**NOTE**: Before running setup.py, you have to edit the config.py in order to setup the following information:
+
+* Server related settings, e.g., API server port, hostname and core architecture
+* Endpoints for the ComputeClient and AstakosClient
+* Enable / Disable VOMS authentication
+* Paths to directories containing certificates and configuration files that enable the process for VOMS authentication
+
+
+
+Finally, the installation of snf-occi is done with the following steps::
+
+$ python setup.py build
+$ python setup.py install
+
+
+In case that VOMS authentication mode is disabled, then the snf-occi server can be started by typing **snf-occi**.
+
+
+Enabling VOMS authentication
+============================
+
+
+VOMS Requirements
+------------------
+
+* Installation of **EUgridPMA** certificates, which must be located on the standard location **/etc/grid-security/certificates**:
+
+::
+
+$ wget -q -O - https://dist.eugridpma.info/distribution/igtf/current/GPG-KEY-EUGridPMA-RPM-3 | sudo apt-key add -
+$ echo "deb http://repository.egi.eu/sw/production/cas/1/current egi-igtf core" > /etc/apt/sources.list.d/egi-cas.list (as root)
+$ sudo apt-get update
+$ sudo apt-get install ca-policy-egi-core
+
+* Installation of the **fetch-crl** package:
+
+::
+
+$ sudo apt-get install fetch-crl
+$ sudo fetch-crl
+
+Moreover, a valid certificate issued by a valid CA is required for the server hosting snf-occi. The certificates of valid CAs are located in **/etc/grid-security/certificates/**. The server certificate and key file need to be located in **/etc/ssl** (if the directories with the certificate and key files differ, then the paths to these directories must be appropriately set in the **snfOCCI/config.py**).
+
+::
+
+$ ls /etc/ssl/certs/server.crt
+$ ls /e tc/ssl/private/server.key
+
+
+Apache Installation and Configuration
+------------------------------------------------
+To enable VOMS authentication in snf-occi, the existence of a working Apache installation is a prerequisite. The installation and configuration process is as follows:
+
+* Installation of Apache WSGI with mod_ssl enabled::
+
+ $ sudo apt-get install apache2 libapache2-mod-wsgi
+ $ sudo a2enmod ssl
+ $ sudo a2enmod headers
+
+* Create a user synnefo and configure Apache::
+
+ $ sudo adduser synnefo
+
+Assuming that the snf-occi server has the FQDM nodeX.example.com, then the following configurations are required:
+
+
+* In **/etc/apache2/sites-enabled/snf_VOMS** add::
+
+ WSGIDaemonProcess snf_voms user=synnefo group=nogroup processes=3 threads=10
+ Listen 8888
+ <VirtualHost _default_:8888>
+ LogLevel warn
+ ErrorLog ${APACHE_LOG_DIR}/error.log
+ CustomLog ${APACHE_LOG_DIR}/ssl_access.log combined
+
+ RequestHeader set X-Forwarded-Protocol "https"
+
+ SSLEngine on
+ SSLCertificateFile /etc/ssl/certs/server.crt
+ SSLCertificateKeyFile /etc/ssl/private/server.key
+
+ SSLCACertificatePath /etc/grid-security/certificates
+ SSLCARevocationPath /etc/grid-security/certificates
+ SSLVerifyClient optional
+ SSLVerifyDepth 20
+ SSLProtocol all -SSLv2
+ SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
+ SSLOptions +StdEnvVars +ExportCertData
+ WSGIScriptAlias /main /usr/lib/cgi-bin/snf_voms/main
+ WSGIProcessGroup snf_voms
+ WSGIPassAuthorization On
+ WSGIScriptAlias / /usr/lib/cgi-bin/snf_voms/main
+ </VirtualHost>
+
+ Listen 5000
+ <VirtualHost _default_:5000>
+ LogLevel warn
+ ErrorLog ${APACHE_LOG_DIR}/error.log
+ CustomLog ${APACHE_LOG_DIR}/ssl_access.log combined
+
+ SSLEngine on
+ SSLCertificateFile /etc/ssl/certs/server.crt
+ SSLCertificateKeyFile /etc/ssl/private/server.key
+
+ SSLCACertificatePath /etc/grid-security/certificates
+ SSLCARevocationPath /etc/grid-security/certificates
+ SSLVerifyClient optional
+ SSLVerifyDepth 20
+ SSLProtocol all -SSLv2
+ SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
+ SSLOptions +StdEnvVars +ExportCertData
+
+ WSGIScriptAlias /main /usr/lib/cgi-bin/snf_voms/main_auth
+ WSGIScriptAlias /main /usr/lib/cgi-bin/snf_voms/main_auth
+ WSGIProcessGroup snf_voms
+ </VirtualHost>
+
+
+* In **/etc/apache2/httpd.conf** add::
+ ServerName nodeX.example.com
+
+
+* In **/etc/apache2/conf.d/wsgi-snf_voms.conf** add::
+
+ WSGIScriptAlias /main /var/www/cgi-bin/snf_voms/main
+ WSGIScriptAlias /main /usr/lib/cgi-bin/snf_voms/main_auth
+
+ <Location "/main">
+ SSLRequireSSL
+ Authtype none
+ </Location>
+
+* To support VOMS authentication, snf-occi must be executed as a WSGI application. To achive this goal, the respective scripts included in the snf-occi repository need to be placed in the appropriate directories. In more detail, you have to copy the following scripts located in the **snfOCCI/httpd** directory in the appropriate directory and create the following links ::
+
+ $ mkdir /usr/lib/cgi-bin/snf_voms
+ $ cp snf-occi/snfOCCI/httpd/snf_voms.py /usr/lib/cgi-bin/snf_voms/snf_voms.py
+ $ ln /usr/lib/cgi-bin/snf_voms/snf_voms.py /usr/lib/cgi-bin/snf_voms/main
+ $ cp snf-occi/snfOCCI/httpd/snf_voms_auth.py /usr/lib/cgi-bin/snf_voms/snf_voms_auth.py
+ $ ln /usr/lib/cgi-bin/snf_voms/snf_voms_auth.py /usr/lib/cgi-bin/snf_voms/main_auth
+ $ cp snf-occi/snfOCCI/snf_voms_auth-paste.ini /home/synneo/snf_voms_auth-paste.ini
+ $ cp snf-occi/snfOCCI/snf_voms-paste.ini /home/synnefo/snf_voms-paste.ini
+
+
+
+* In **/etc/apache2/envvars** add::
+
+ export OPENSSL_ALLOW_PROXY_CERTS=1
+
+
+Configure VO Membership Information
+------------------------------------
+
+In order to support VOMS authentication, some configuration information is required. This information is provided in the following files. For example, to allow the access for members of the fedcloud.egi.eu VO, the following configuration files need to be present in the directory **/etc/snf** :
+
+* In **/etc/snf/voms.json** add::
+
+ {
+ "fedcloud.egi.eu":
+ {
+ "tenant": "EGI_FCTF"
+ }
+ }
+
+Moreover, the vomsdir information and the vomses file need to be created and altered respectively for each allowed VO (see also `Fedcloud-tf:CLI Environment <https://wiki.egi.eu/wiki/Fedcloud-tf:CLI_Environment>`_). For example, the respective folders and files for the **fedcloud.egi.eu** VO are created in the following two steps:
+
+* Creation of the vomsdir information::
-Then you can install **snf-occi** API translation server by cloning our latest source code:
+ $ mkdir -p /etc/grid-security/vomsdir/fedcloud.egi.eu
+ $ cat > /etc/grid-security/vomsdir/fedcloud.egi.eu/voms1.egee.cesnet.cz.lsc <<EOF
+ /DC=org/DC=terena/DC=tcs/OU=Domain Control Validated/CN=voms1.egee.cesnet.cz
+ /C=NL/O=TERENA/CN=TERENA eScience SSL CA
+ EOF
+ $ cat > /etc/grid-security/vomsdir/fedcloud.egi.eu/voms2.egee.cesnet.cz.lsc <<EOF
+ /DC=org/DC=terena/DC=tcs/OU=Domain Control Validated /CN=voms2.grid.cesnet.cz
+ /C=NL/O=TERENA/CN=TERENA eScience SSL CA
+ EOF
-* `snf-occi <https://code.grnet.gr/projects/snf-occi>`_
-**NOTE**: Before running setup.py you have to edit the **config.py** setting up:
+* Creation / Extension of the vomses file::
-* API Server port
-* VM hostname naming pattern (FQDN providing the id of each compute resource)
-* VM core architecture
+ $ cat >> /etc/vomses <<EOF
+ "fedcloud.egi.eu" "voms1.egee.cesnet.cz" "15002" "/DC=org/DC=terena/DC=tcs/OU=Domain Control Validated/CN=voms1.egee.cesnet.cz" "fedcloud.egi.eu" "24"
+ "fedcloud.egi.eu" "voms2.grid.cesnet.cz" "15002" "/DC=org/DC=terena/DC=tcs/OU=Domain Control Validated /CN=voms2.grid.cesnet.cz" "fedcloud.egi.eu" "24"
+ EOF
-Finally you can start the API translation server by running **snf-occi**
+Upon the completion of all configuration actions, start (or restart) the Apache server:
-Examples:
----------
-For the examples below we assume server is running on localhost (port 8888) and authentication token is $AUTH. For the HTTP requests we are using **curl**.
+$ sudo service apache2 start
+
+
+Usage Scenarios
+================
+
+For the examples below, we assume that the snf-occi server is running on <snf-occi_host> (port 8888) and authentication token is $AUTH. For the HTTPS requests we are using **curl**.
+
+The user must have a valid authentication token in order to interact with the snf-occi endpoint. In case that the VOMS authentication is enabled, then the user can provide his/her proxy certificate in order to obtain a valid authentication token.
+
+* A user participating in a specific VO must have a valid VOMS proxy. To obtain a proxy certificate type the command (More information about the setup of a FedCloud command-line environment in order to obtain a proxy certificate can be found at `Fedcloud-tf:CLI Environment <https://wiki.egi.eu/wiki/Fedcloud-tf:CLI_Environment>`_.)::
+
+ $ voms-proxy-init -voms fedcloud.egi.eu -rfc
+
+
+* Retrieve an authentication token (hence referring as $AUTH) from the snf-occi server, assuming that the proxy certificate is the file $X509_USER_PROXY (e.g., /tmp/x509up_u1000) and <snf-occi_host> is the hostname of server hosting snf-occi::
+
+ $ curl --capath /etc/grid-security/certificates --cert $X509_USER_PROXY -d '{"auth":{"voms": true, "tenantName": "fedcloud.egi.eu"}}' -v -X GET https://<snf-occi_host>:5000/main/v2.0/tokens
* Retrieve all registered Kinds, Actions and Mixins:
::
- curl -v -X GET localhost:8888/-/ -H 'Auth-Token: $AUTH'
+ $ curl --capath /etc/grid-security/certificates -X GET https://<snf-occi_host>:8888/-/ -H 'X-Auth-Token:$AUTH'
+
-* Create a new VM described by the flavor 'C2R2048D20' and using the image 'Debian'
+* List all VMs::
+
+ $ curl --capath /etc/grid-security/certificates -X GET https://<snf-occi_host>:8888/compute/ -H 'X-Auth-Token:$AUTH'
+
+
+* Create a new VM using the the flavor 'c1r1024d5rdb' and the image 'ubuntu_server'
::
- curl -v -X POST localhost:8888/compute/
- -H 'Category: compute; scheme=http://schemas.ogf.org/occi/infrastructure#; class="kind";'
- -H 'X-OCCI-Attribute: occi.core.title = newVM' -H 'Category: C2R2048D20; scheme=http://schemas.ogf.org/occi/infrastructure#; '
- -H 'Category: Debian; scheme=http://schemas.ogf.org/occi/infrastructure#;' -H 'Auth-Token: $AUTH'
- -H 'Content-type: text/occi'
+ $ curl --capath /etc/grid-security/certificates -X POST https://<snf-occi_host>:8888/compute/ -H 'Category: compute; scheme=http://schemas.ogf.org/occi/infrastructure#; class="kind";' -H 'X-OCCI-Attribute: occi.core.title = newVM' -H 'Category: c1r1024d5drbd; scheme=http://schemas.ogf.org/occi/resource_tpl#; ' -H 'Category: ubuntu_server; scheme=http://schemas.ogf.org/occi/os_tpl#;' -H 'X-Auth-Token:$AUTH' -H 'Content-type: text/occi'
-* Retrieve all the details of th VM with identifier $ID
- ::
+* Retrieve all the details of th VM with identifier $ID::
- curl -v -X GET localhost:8888/compute/$ID -H 'Auth-Token: $AUTH'
+ $ curl --capath /etc/grid-security/certificates -X GET https://<snf-occi_host>:8888/compute/$ID -H 'X-Auth-Token: $AUTH'
-* Delete the VM with identifier $ID
- ::
+
+* Delete the VM with identifier $ID::
- curl -v -X DELETE localhost:8888/compute/$ID -H 'Auth-Token: $AUTH'
+ $ curl --capath /etc/grid-security/certificates -X DELETE https://<snf-occi_host>:8888/compute/$ID -H 'X-Auth-Token: $AUTH'
+
+Moreover, the `rOCCI cli <https://github.com/gwdg/rOCCI-cli>`_ can be used directly from shell defining the following parameters:
+ * -- endpoint https://<snf-occi_host>:8888
+ * --auth x509
+ * -- voms
+ * --user-cred $X509_USER_PROXY
+
+
+Future Directions
+-----------------
+
+The snf-occi server is constantly evolving and being enhanced with more new features in order to support more advanced functionalities. For instance, some upcoming features are:
+
+* VM Contextualization
+* Network Management
+* Data Management
+* VM Image Management
+
+
Indices and tables
==================
* :ref:`genindex`
* :ref:`search`
-
projects / snf-occi / commitdiff