Synnefo » Pithos

Pithos is a storage service implemented by GRNET (http://www.grnet.gr). Data is stored as objects, organized in containers, belonging to an account. This hierarchy of storage layers has been inspired by the OpenStack Object Storage (OOS) API and similar CloudFiles API by Rackspace. The Pithos API follows the OOS API as closely as possible. One of the design requirements has been to be able to use Pithos with clients built for the OOS, without changes.

This document's goals are:

=========================  ================================

Revision                   Description

=========================  ================================

0.2 (May 31, 2011)         Add object meta listing and filtering in containers.

\                          Include underlying storage characteristics in container meta.

\                          Support for partial object updates through POST.

The Pithos API

--------------

* Top level: ``https://hostname/v1/``

* Account level: ``https://hostname/v1/<account>``

* Container level: ``https://hostname/v1/<account>/<container>``

* Object level: ``https://hostname/v1/<account>/<container>/<object>``

The allowable request operations and respective return codes per level are presented in the remainder of this chapter. Common to all requests are the following return codes.

HEAD

""""

==========================  =====================

Reply Header Name           Value

X-Account-Bytes-Used        The total number of bytes stored

X-Account-Bytes-Remaining   The total number of bytes remaining (**TBD**)

X-Account-Last-Login        The last login (**TBD**)

X-Account-Meta-*            Optional user defined metadata

==========================  =====================

|

limit                   The amount of results requested (default is 10000)

marker                  Return containers with name lexicographically after marker

format                  Optional extended reply type (can be ``json`` or ``xml``)

======================  =========================

The reply is a list of container names. Account headers (as in a ``HEAD`` request) will also be included.

If a ``format=xml`` or ``format=json`` argument is given, extended information on the containers will be returned, serialized in the chosen format.

For each container, the information will include all container metadata (names will be in lower case and with hyphens replaced with underscores):

For examples of container details returned in JSON/XML formats refer to the OOS API documentation.

HEAD

""""

The keys returned in ``X-Container-Object-Meta`` are all the unique strings after the ``X-Object-Meta-`` prefix.

path                    Assume ``prefix=path`` and ``delimiter=/``

format                  Optional extended reply type (can be ``json`` or ``xml``)

meta                    Return objects having the specified meta keys (can be a comma separated list)

======================  ===================================

The ``path`` parameter overrides ``prefix`` and ``delimiter``. When using ``path``, results will include objects ending in ``delimiter``.

If a ``format=xml`` or ``format=json`` argument is given, extended information on the objects will be returned, serialized in the chosen format.

For each object, the information will include all object metadata (names will be in lower case and with hyphens replaced with underscores):

Extended replies may also include virtual directory markers in separate sections of the ``json`` or ``xml`` results.

Virtual directory markers are only included when ``delimiter`` is explicitly set. They correspond to the substrings up to and including the first occurrence of the delimiter.

In JSON results they appear as dictionaries with only a ``"subdir"`` key. In XML results they appear interleaved with ``<object>`` tags as ``<subdir name="..." />``.

In case there is an object with the same name as a virtual directory marker, the object will be returned.

For examples of object details returned in JSON/XML formats refer to the OOS API documentation.

===========================  ===============================

HEAD

""""

==========================  ===============================

Reply Header Name           Value

ETag                        The ETag of the object

Content-Length              The size of the object

Content-Type                The MIME content type of the object

Content-Encoding            The encoding of the object (optional)

Content-Disposition         The presentation style of the object (optional)

X-Object-Manifest           Large object support (optional)

X-Object-Meta-*             Optional user defined metadata

==========================  ===============================

Request Parameter Name  Value

======================  ===================================

format                  Optional extended reply type (can be ``json`` or ``xml``)

======================  ===================================

Hashmaps expose the underlying storage format of the object. Note that each hash is computed after trimming trailing null bytes of the corresponding block.

::

Example ``format=xml`` reply:

    <hash>...</hash>

  </object>

The ``Range`` header may include multiple ranges, as outlined in RFC2616. Then the ``Content-Type`` of the reply will be ``multipart/byteranges`` and each part will include a ``Content-Range`` header.

==========================  ===============================

Content-Length              The size of the data returned

Content-Type                The MIME content type of the object

Content-Range               The range of data included (only on a single range request)

Content-Encoding            The encoding of the object (optional)

Content-Disposition         The presentation style of the object (optional)

X-Object-Manifest           Large object support (optional)

X-Object-Meta-*             Optional user defined metadata

==========================  ===============================

Transfer-Encoding     Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored)

X-Copy-From           The source path in the form ``/<container>/<object>``

X-Move-From           The source path in the form ``/<container>/<object>``

Content-Encoding      The encoding of the object (optional)

Content-Disposition   The presentation style of the object (optional)

X-Object-Manifest     Large object support (optional)

X-Object-Meta-*       Optional user defined metadata

====================  ================================

Content-Type          The MIME content type of the object (optional)

Content-Encoding      The encoding of the object (optional)

Content-Disposition   The presentation style of the object (optional)

X-Object-Manifest     Large object support (optional)

X-Object-Meta-*       Optional user defined metadata

====================  ================================

Content-Encoding      The encoding of the object (optional)

Content-Disposition   The presentation style of the object (optional)

X-Object-Manifest     Large object support (optional)

X-Object-Meta-*       Optional user defined metadata

====================  ================================

To update an object:

204 (No Content)             The request succeeded

===========================  ==============================

Summary

^^^^^^^

* All metadata replies, at all levels, include latest modification information.

* At all levels, a ``GET`` request may use ``If-Modified-Since`` and ``If-Unmodified-Since`` headers.

* Container/object lists include all associated metadata if the reply is of type json/xml. Some names are kept to their OOS API equivalents for compatibility. 

* Multi-range object GET support as outlined in RFC2616.

* Object hashmap retrieval through GET and the ``format`` parameter.

* Partial object updates through POST, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers.

* Object ``MOVE`` support.

Clarifications/suggestions:

* The ``Accept`` header may be used in requests instead of the ``format`` parameter to specify the desired reply format. The parameter overrides the header.

* Container/object lists use a ``200`` return code if the reply is of type json/xml. The reply will include an empty json/xml.

* In headers, dates are formatted according to RFC 1123. In extended information listings, dates are formatted according to ISO 8601.

* While ``X-Object-Manifest`` can be set and unset, large object support is not yet implemented (**TBD**).

The Pithos Client

* Moved to trash and then deleted.

* Shared with specific permissions.

* Made public (shared with non-Pithos users).

Some of these functions are performed by the client software and some by the Pithos server. Client-driven functionality is based on specific metadata that should be handled equally across implementations. These metadata names are discussed in the next chapter.