========================= ================================
Revision Description
========================= ================================
-0.5 (July 21, 2011) Object update from another object's data.
+0.6 (Sept 13, 2011) Reply with Merkle hash as the ETag when updating objects.
+\ Include version id in object replace/change replies.
+\ Change conflict (409) replies format to text.
+\ Tags should be migrated to a meta value.
+\ Container ``PUT`` updates metadata/policy.
+\ Report allowed actions in shared object replies.
+0.5 (July 22, 2011) Object update from another object's data.
\ Support object truncate.
\ Create object using a standard HTML form.
\ Purge container/object history.
\ List other accounts that share objects with a user.
\ List shared containers/objects.
\ Update implementation guidelines.
+\ Check preconditions when creating/updating objects.
0.4 (July 01, 2011) Object permissions and account groups.
\ Control versioning behavior and container quotas with container policy directives.
\ Support updating/deleting individual metadata with ``POST``.
====================== =========================
The reply is a list of account names.
-If a ``format=xml`` or ``format=json`` argument is given, extended information on the containers will be returned, serialized in the chosen format.
+If a ``format=xml`` or ``format=json`` argument is given, extended information on the accounts will be returned, serialized in the chosen format.
For each account, the information will include the following (names will be in lower case and with hyphens replaced with underscores):
=========================== ============================
Name Description
=========================== ============================
name The name of the account
-last_modified The last container modification date (regardless of ``until``)
+last_modified The last account modification date (regardless of ``until``)
=========================== ============================
Example ``format=json`` reply:
Return Code Description
=========================== =====================
200 (OK) The request succeeded
-204 (No Content) The account has no containers (only for non-extended replies)
+204 (No Content) The user has no access to other accounts (only for non-extended replies)
=========================== =====================
Will use a ``200`` return code if the reply is of type json/xml.
HEAD
""""
+==================== ===========================
+Request Header Name Value
+==================== ===========================
+If-Modified-Since Retrieve if account has changed since provided timestamp
+If-Unmodified-Since Retrieve if account has not changed since provided timestamp
+==================== ===========================
+
+|
+
====================== ===================================
Request Parameter Name Value
====================== ===================================
POST
""""
-====================== ============================================
-Request Parameter Name Value
-====================== ============================================
-update Do not replace metadata/groups (no value parameter)
-====================== ============================================
-
-|
-
==================== ===========================
Request Header Name Value
==================== ===========================
X-Account-Meta-* Optional user defined metadata
==================== ===========================
+|
+
+====================== ============================================
+Request Parameter Name Value
+====================== ============================================
+update Do not replace metadata/groups (no value parameter)
+====================== ============================================
+
No reply content/headers.
The operation will overwrite all user defined metadata, except if ``update`` is defined.
HEAD
""""
+==================== ===========================
+Request Header Name Value
+==================== ===========================
+If-Modified-Since Retrieve if container has changed since provided timestamp
+If-Unmodified-Since Retrieve if container has not changed since provided timestamp
+==================== ===========================
+
+|
+
====================== ===================================
Request Parameter Name Value
====================== ===================================
X-Container-Block-Size The block size used by the storage backend
X-Container-Block-Hash The hash algorithm used for block identifiers in object hashmaps
X-Container-Until-Timestamp The last container modification date until the timestamp provided
-X-Container-Object-Meta A list with all meta keys used by objects
+X-Container-Object-Meta A list with all meta keys used by objects (**TBD**)
X-Container-Policy-* Container behavior and limits
X-Container-Meta-* Optional user defined metadata
Last-Modified The last container modification date (regardless of ``until``)
=========================== ===============================
-The keys returned in ``X-Container-Object-Meta`` are all the unique strings after the ``X-Object-Meta-`` prefix, formatted as a comma-separated list. See container ``PUT`` for a reference of policy directives.
+The keys returned in ``X-Container-Object-Meta`` are all the unique strings after the ``X-Object-Meta-`` prefix, formatted as a comma-separated list. See container ``PUT`` for a reference of policy directives. (**TBD**)
================ ===============================
Return Code Description
* ``versioning``: Set to ``auto``, ``manual`` or ``none`` (default is ``manual``)
* ``quota``: Size limit in KB (default is ``0`` - unlimited)
-
+
+If the container already exists, the operation is equal to a ``POST`` with ``update`` defined.
+
================ ===============================
Return Code Description
================ ===============================
POST
""""
-====================== ============================================
-Request Parameter Name Value
-====================== ============================================
-update Do not replace metadata/policy (no value parameter)
-====================== ============================================
-
-|
-
==================== ================================
Request Header Name Value
==================== ================================
X-Container-Meta-* Optional user defined metadata
==================== ================================
+|
+
+====================== ============================================
+Request Parameter Name Value
+====================== ============================================
+update Do not replace metadata/policy (no value parameter)
+====================== ============================================
+
No reply content/headers.
The operation will overwrite all user defined metadata, except if ``update`` is defined.
HEAD
""""
+==================== ================================
+Request Header Name Value
+==================== ================================
+If-Match Retrieve if ETags match
+If-None-Match Retrieve if ETags don't match
+If-Modified-Since Retrieve if object has changed since provided timestamp
+If-Unmodified-Since Retrieve if object has not changed since provided timestamp
+==================== ================================
+
+|
+
====================== ===================================
Request Parameter Name Value
====================== ===================================
X-Object-Manifest Object parts prefix in ``<container>/<object>`` form (optional)
X-Object-Sharing Object permissions (optional)
X-Object-Shared-By Object inheriting permissions (optional)
+X-Object-Allowed-To Allowed actions on object (optional)
X-Object-Public Object's publicly accessible URI (optional)
X-Object-Meta-* Optional user defined metadata
========================== ===============================
X-Object-Manifest Object parts prefix in ``<container>/<object>`` form (optional)
X-Object-Sharing Object permissions (optional)
X-Object-Shared-By Object inheriting permissions (optional)
+X-Object-Allowed-To Allowed actions on object (optional)
X-Object-Public Object's publicly accessible URI (optional)
X-Object-Meta-* Optional user defined metadata
========================== ===============================
==================== ================================
Request Header Name Value
==================== ================================
+If-Match Put if ETags match with current object
+If-None-Match Put if ETags don't match with current object
ETag The MD5 hash of the object (optional to check written data)
Content-Length The size of the data written
Content-Type The MIME content type of the object
format Optional extended request type (can be ``json``) to create the object by suppling its hashmap instead
====================== ===================================
-The request is the object's data (or part of it), except if a hashmap is provided with the ``format`` parameter. If format is used and all different parts are stored in the server, the object is created, otherwise the server returns Conflict (409) with the list of the missing parts.
+The request is the object's data (or part of it), except if a hashmap is provided with the ``format`` parameter. If format is used and all different parts are stored in the server, the object is created, otherwise the server returns Conflict (409) with the list of the missing parts (in a simple text format, with one hash per line).
Hashmaps expose the underlying storage format of the object.
Reply Header Name Value
========================== ===============================
ETag The MD5 hash of the object (on create)
+X-Object-Version The object's new version
========================== ===============================
The ``X-Object-Sharing`` header may include either a ``read=...`` comma-separated user/group list, or a ``write=...`` comma-separated user/group list, or both separated by a semicolon (``;``). Groups are specified as ``<account>:<group>``. To publish the object, set ``X-Object-Public`` to ``true``. To unpublish, set to ``false``, or use an empty header value.
Return Code Description
=========================== ==============================
201 (Created) The object has been created
-409 (Conflict) The object can not be created from the provided hashmap, or there are conflicting permissions (a list of missing hashes, or a conflicting sharing path will be included in the reply - in JSON format)
+409 (Conflict) The object can not be created from the provided hashmap, or there are conflicting permissions (a list of missing hashes, or a list of conflicting sharing paths will be included in the reply - in simple text format)
411 (Length Required) Missing ``Content-Length`` or ``Content-Type`` in the request
422 (Unprocessable Entity) The MD5 checksum of the data written to the storage system does not match the (optionally) supplied ETag value
=========================== ==============================
==================== ================================
Request Header Name Value
==================== ================================
+If-Match Proceed if ETags match with object
+If-None-Match Proceed if ETags don't match with object
Destination The destination path in the form ``/<container>/<object>``
Content-Type The MIME content type of the object (optional)
Content-Encoding The encoding of the object (optional)
Refer to ``PUT``/``POST`` for a description of request headers. Metadata is also copied, updated with any values defined. Sharing/publishing options are not copied.
-No reply content/headers.
+========================== ===============================
+Reply Header Name Value
+========================== ===============================
+X-Object-Version The object's new version
+========================== ===============================
+
+|
=========================== ==============================
Return Code Description
=========================== ==============================
201 (Created) The object has been created
-409 (Conflict) There are conflicting permissions (a conflicting sharing path will be included in the reply - in JSON format)
+409 (Conflict) There are conflicting permissions (a list of conflicting sharing paths will be included in the reply - in simple text format)
=========================== ==============================
POST
""""
-====================== ============================================
-Request Parameter Name Value
-====================== ============================================
-update Do not replace metadata (no value parameter)
-====================== ============================================
-
-|
-
==================== ================================
Request Header Name Value
==================== ================================
+If-Match Proceed if ETags match with object
+If-None-Match Proceed if ETags don't match with object
Content-Length The size of the data written (optional, to update)
Content-Type The MIME content type of the object (optional, to update)
Content-Range The range of data supplied (optional, to update)
X-Object-Meta-* Optional user defined metadata
==================== ================================
+|
+
+====================== ============================================
+Request Parameter Name Value
+====================== ============================================
+update Do not replace metadata (no value parameter)
+====================== ============================================
+
The ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest`` and ``X-Object-Meta-*`` headers are considered to be user defined metadata. An operation without the ``update`` parameter will overwrite all previous values and remove any keys not supplied. When using ``update`` any metadata with an empty value will be deleted.
To change permissions, include an ``X-Object-Sharing`` header (as defined in ``PUT``). To publish, include an ``X-Object-Public`` header, with a value of ``true``. If no such headers are defined, no changes will be applied to sharing/public. Use empty values to remove permissions/unpublish (unpublishing also works with ``false`` as a header value). Sharing options are applied to the object - not its versions.
Optionally, truncate the updated object to the desired length with the ``X-Object-Bytes`` header.
-A data update will trigger an ETag change. The new ETag will not correspond to the object's MD5 sum (**TBD**) and will be included in reply headers.
+A data update will trigger an ETag change. Updated ETags correspond to the single Merkle hash of the object's hashmap (refer to http://bittorrent.org/beps/bep_0030.html for more information).
No reply content. No reply headers if only metadata is updated.
Reply Header Name Value
========================== ===============================
ETag The new ETag of the object (data updated)
+X-Object-Version The object's new version
========================== ===============================
|
=========================== ==============================
202 (Accepted) The request has been accepted (not a data update)
204 (No Content) The request succeeded (data updated)
-409 (Conflict) There are conflicting permissions (a conflicting sharing path will be included in the reply - in JSON format)
+409 (Conflict) There are conflicting permissions (a list of conflicting sharing paths will be included in the reply - in simple text format)
411 (Length Required) Missing ``Content-Length`` in the request
416 (Range Not Satisfiable) The supplied range is invalid
=========================== ==============================
Reply Header Name Value
========================== ===============================
ETag The MD5 hash of the object
+X-Object-Version The object's new version
========================== ===============================
|
Read and write control in Pithos is managed by setting appropriate permissions with the ``X-Object-Sharing`` header. The permissions are applied using prefix-based inheritance. Thus, each set of authorization directives is applied to all objects sharing the same prefix with the object where the corresponding ``X-Object-Sharing`` header is defined. For simplicity, nested/overlapping permissions are not allowed. Setting ``X-Object-Sharing`` will fail, if the object is already "covered", or another object with a longer common-prefix name already has permissions. When retrieving an object, the ``X-Object-Shared-By`` header reports where it gets its permissions from. If not present, the object is the actual source of authorization directives.
-A user may ``GET`` another account or container. The result will include a limited reply, containing only the allowed containers or objects respectively. A top-level request with an authentication token, will return a list of allowed accounts, so the user can easily find out which other users share objects.
+A user may ``GET`` another account or container. The result will include a limited reply, containing only the allowed containers or objects respectively. A top-level request with an authentication token, will return a list of allowed accounts, so the user can easily find out which other users share objects. The ``X-Object-Allowed-To`` header lists the actions allowed on an object, if it does not belong to the requesting user.
Objects that are marked as public, via the ``X-Object-Public`` meta, are also available at the corresponding URI returned for ``HEAD`` or ``GET``. Requests for public objects do not need to include an ``X-Auth-Token``. Pithos will ignore request parameters and only include the following headers in the reply (all ``X-Object-*`` meta is hidden):
* Support for ``X-Account-Meta-*`` style headers at the account level. Use ``POST`` to update.
* Support for ``X-Container-Meta-*`` style headers at the container level. Can be set when creating via ``PUT``. Use ``POST`` to update.
-* Header ``X-Container-Object-Meta`` at the container level and parameter ``meta`` in container listings.
+* Header ``X-Container-Object-Meta`` at the container level and parameter ``meta`` in container listings. (**TBD**)
* Container policies to manage behavior and limits.
* Headers ``X-Container-Block-*`` at the container level, exposing the underlying storage characteristics.
* All metadata replies, at all levels, include latest modification information.
-* At all levels, a ``GET`` request may use ``If-Modified-Since`` and ``If-Unmodified-Since`` headers.
+* At all levels, a ``HEAD`` or ``GET`` request may use ``If-Modified-Since`` and ``If-Unmodified-Since`` headers.
* Container/object lists include all associated metadata if the reply is of type json/xml. Some names are kept to their OOS API equivalents for compatibility.
* Option to include only shared containers/objects in listings.
* Object metadata allowed, in addition to ``X-Object-Meta-*``: ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``. These are all replaced with every update operation, except if using the ``update`` parameter (in which case individual keys can also be deleted). Deleting meta by providing empty values also works when copying/moving an object.
* Object hashmap retrieval through ``GET`` and the ``format`` parameter.
* Object create via hashmap through ``PUT`` and the ``format`` parameter.
* Object create using ``POST`` to support standard HTML forms.
-* Partial object updates through ``POST``, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers. Use another object's data to update with ``X-Source-Object`` and ``X-Source-Version``. Truncate with ``X-Object-Bytes``.
+* Partial object updates through ``POST``, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers. Use another object's data to update with ``X-Source-Object`` and ``X-Source-Version``. Truncate with ``X-Object-Bytes``. New ETag corresponds to the Merkle hash of the object's hashmap.
+* Include new version identifier in replies for object replace/change requests.
* Object ``MOVE`` support.
+* Conditional object create/update operations, using ``If-Match`` and ``If-None-Match`` headers.
* Time-variant account/container listings via the ``until`` parameter.
* Object versions - parameter ``version`` in ``HEAD``/``GET`` (list versions with ``GET``), ``X-Object-Version-*`` meta in replies, ``X-Source-Version`` in ``PUT``/``COPY``.
-* Sharing/publishing with ``X-Object-Sharing``, ``X-Object-Public`` at the object level. Cross-user operations are allowed - controlled by sharing directives. Permissions may include groups defined with ``X-Account-Group-*`` at the account level. These apply to the object - not its versions.
+* Sharing/publishing with ``X-Object-Sharing``, ``X-Object-Public`` at the object level. Cross-user operations are allowed - controlled by sharing directives. Available actions in cross-user requests are reported with ``X-Object-Allowed-To``. Permissions may include groups defined with ``X-Account-Group-*`` at the account level. These apply to the object - not its versions.
* Support for prefix-based inheritance when enforcing permissions. Parent object carrying the authorization directives is reported in ``X-Object-Shared-By``.
* Large object support with ``X-Object-Manifest``.
* Trace the user that created/modified an object with ``X-Object-Modified-By``.
* The ``trash`` element, which contains files that have been marked for deletion, but can still be recovered.
* The ``shared`` element, which contains all objects shared by the user to other users of the system.
* The ``others`` element, which contains all objects that other users share with the user.
-* The ``tags`` element, which lists the names of tags the user has defined. This can be an entry point to list all files that have been assigned a specific tag or manage tags in general (remove a tag completely, rename a tag etc.).
* The ``groups`` element, which contains the names of groups the user has defined. Each group consists of a user list. Group creation, deletion, and manipulation is carried out by actions originating here.
* The ``history`` element, which allows browsing past instances of ``home`` and - optionally - ``trash``.
Objects in Pithos can be:
-* Assigned custom tags.
* Moved to trash and then deleted.
* Shared with specific permissions.
* Made public (shared with non-Pithos users).
Some of these functions are performed by the client software and some by the Pithos server.
+In the first version of Pithos, objects could also be assigned custom tags. This is no longer supported. Existing deployments can migrate tags into a specific metadata value, i.e. ``X-Object-Meta-Tags``.
+
Implementation Guidelines
^^^^^^^^^^^^^^^^^^^^^^^^^
-Pithos clients should use the ``pithos`` and ``trash`` containers for active and inactive objects respectively. If any of these containers is not found, the client software should create it, without interrupting the user's workflow. The ``home`` element corresponds to ``pithos`` and the ``trash`` element to ``trash``. Use ``PUT`` with the ``X-Move-From`` header, or ``MOVE`` to transfer objects from one container to the other. Use ``DELETE`` to remove from ``pithos`` without trashing, or to remove from ``trash``. When moving objects, detect naming conflicts with the ``If-Match`` or ``If-None-Match`` headers (**TBD**). Such conflicts should be resolved by the user.
+Pithos clients should use the ``pithos`` and ``trash`` containers for active and inactive objects respectively. If any of these containers is not found, the client software should create it, without interrupting the user's workflow. The ``home`` element corresponds to ``pithos`` and the ``trash`` element to ``trash``. Use ``PUT`` with the ``X-Move-From`` header, or ``MOVE`` to transfer objects from one container to the other. Use ``DELETE`` to remove from ``pithos`` without trashing, or to remove from ``trash``. When moving objects, detect naming conflicts with the ``If-Match`` or ``If-None-Match`` headers. Such conflicts should be resolved by the user.
Object names should use the ``/`` delimiter to impose a hierarchy of folders and files.
Public objects are not included in ``shared`` and ``others`` listings. It is suggested that they are marked in a visually distinctive way in ``pithos`` listings (for example using an icon overlay).
-At the object level, tags are implemented by managing metadata keys. The client software should allow the user to use any string as a tag and set the corresponding ``X-Object-Meta-<tag>`` key at the server. The API extensions provided, allow for listing all tags in a container and filtering object listings based on one or more tags. The tag list is sufficient for implementing the ``tags`` element, either as a special, virtual folder (as done in the first version of Pithos), or as an application menu.
-
A special application menu, or a section in application preferences, should be devoted to managing groups (the ``groups`` element). All group-related actions are implemented at the account level.
-Browsing past versions of objects should be available both at the object and the container level. At the object level, a list of past versions can be included in the screen showing details or more information on the object (metadata, tags, permissions, etc.). At the container level, it is suggested that clients use a ``history`` element, which presents to the user a read-only, time-variable view of ``pithos`` contents. This can be accomplished via the ``until`` parameter in listings. Optionally, ``history`` may include ``trash``.
+Browsing past versions of objects should be available both at the object and the container level. At the object level, a list of past versions can be included in the screen showing details or more information on the object (metadata, permissions, etc.). At the container level, it is suggested that clients use a ``history`` element, which presents to the user a read-only, time-variable view of ``pithos`` contents. This can be accomplished via the ``until`` parameter in listings. Optionally, ``history`` may include ``trash``.
Recommended Practices and Examples
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* List metadata keys used by objects in a container
- Will be in the ``X-Container-Object-Meta`` reply header, included in container information or object list (``HEAD`` or ``GET``).
+ Will be in the ``X-Container-Object-Meta`` reply header, included in container information or object list (``HEAD`` or ``GET``). (**TBD**)
* List objects in a container having a specific meta defined ::