========================= ================================
Revision Description
========================= ================================
-0.8 (Jan 19, 2012) Update allowed versioning values.
+0.9 (Feb 17, 2012) Change permissions model.
+\ Do not include user-defined metadata in account/container/object listings.
+0.8 (Jan 24, 2012) Update allowed versioning values.
\ Change policy/meta formatting in JSON/XML replies.
\ Document that all non-ASCII characters in headers should be URL-encoded.
\ Support metadata-based queries when listing objects at the container level.
\ Add object UUID field.
\ Always reply with the MD5 in the ETag.
\ Note that ``/login`` will only work if an external authentication system is defined.
+\ Include option to ignore Content-Type on ``COPY``/``MOVE``.
+\ Use format parameter for conflict (409) and uploaded hash list (container level) replies.
0.7 (Nov 21, 2011) Suggest upload/download methods using hashmaps.
\ Propose syncing algorithm.
\ Support cross-account object copy and move.
====================== =========================
next The URI to redirect to when the process is finished
renew Force token renewal (no value parameter)
+force Force logout current user (no value parameter)
====================== =========================
When done with logging in, the service's login URI should redirect to the URI provided with ``next``, adding ``user`` and ``token`` parameters, which contain the account and token fields respectively.
Cross-user requests are not allowed to use ``until`` and only include the account/container modification dates in the reply.
If a ``format=xml`` or ``format=json`` argument is given, extended information on the containers will be returned, serialized in the chosen format.
-For each container, the information will include all container metadata (names will be in lower case and with hyphens replaced with underscores):
+For each container, the information will include all container metadata, except user-defined (names will be in lower case and with hyphens replaced with underscores):
=========================== ============================
Name Description
bytes The total size of the objects inside the container
last_modified The last container modification date (regardless of ``until``)
x_container_until_timestamp The last container modification date until the timestamp provided
-x_container_policy_* Container behavior and limits
-x_container_meta_* Optional user defined metadata
+x_container_policy Container behavior and limits
=========================== ============================
Example ``format=json`` reply:
"bytes": 62452,
"count": 8374,
"last_modified": "2011-12-02T08:10:41.565891+00:00",
- "x_container_policy": {"quota": "53687091200", "versioning": "auto"},
- "x_container_meta": {"a": "b", "1": "2"}}, ...]
+ "x_container_policy": {"quota": "53687091200", "versioning": "auto"}}, ...]
Example ``format=xml`` reply:
<key>quota</key><value>53687091200</value>
<key>versioning</key><value>auto</value>
</x_container_policy>
- <x_container_meta>
- <key>a</key><value>b</value>
- <key>1</key><value>2</value>
- </x_container_meta>
</container>
<container>...</container>
</account>
-For more examples of container details returned in JSON/XML formats refer to the OOS API documentation. In addition to the OOS API, Pithos returns all fields. Policy and metadata values are grouped and returned as key-value pairs.
+For more examples of container details returned in JSON/XML formats refer to the OOS API documentation. In addition to the OOS API, Pithos returns policy fields, grouped as key-value pairs.
=========================== =====================
Return Code Description
=========================== ===============================
If a ``format=xml`` or ``format=json`` argument is given, extended information on the objects will be returned, serialized in the chosen format.
-For each object, the information will include all object metadata (names will be in lower case and with hyphens replaced with underscores):
+For each object, the information will include all object metadata, except user-defined (names will be in lower case and with hyphens replaced with underscores). User-defined metadata includes ``X-Object-Meta-*``, ``X-Object-Manifest``, ``Content-Disposition`` and ``Content-Encoding`` keys. Also, sharing directives will only be included with the actual shared objects (inherited permissions are not calculated):
========================== ======================================
Name Description
hash The ETag of the object
bytes The size of the object
content_type The MIME content type of the object
-content_encoding The encoding of the object (optional)
-content-disposition The presentation style of the object (optional)
last_modified The last object modification date (regardless of version)
x_object_hash The Merkle hash
x_object_uuid The object's UUID
x_object_version The object's version identifier
x_object_version_timestamp The object's version timestamp
x_object_modified_by The user that committed the object's version
-x_object_manifest Object parts prefix in ``<container>/<object>`` form (optional)
x_object_sharing Object permissions (optional)
-x_object_shared_by Object inheriting permissions (optional)
x_object_allowed_to Allowed actions on object (optional)
x_object_public Object's publicly accessible URI (optional)
-x_object_meta_* Optional user defined metadata
========================== ======================================
+Sharing metadata and last modification timestamp will only be returned if there is no ``until`` parameter defined.
+
Extended replies may also include virtual directory markers in separate sections of the ``json`` or ``xml`` results.
Virtual directory markers are only included when ``delimiter`` is explicitly set. They correspond to the substrings up to and including the first occurrence of the delimiter.
In JSON results they appear as dictionaries with only a ``subdir`` key. In XML results they appear interleaved with ``<object>`` tags as ``<subdir name="..." />``.
"hash": "d41d8cd98f00b204e9800998ecf8427e",
"content_type": "application/octet-stream",
"last_modified": "2011-12-02T08:10:41.565891+00:00",
- "x_object_meta": {"asdf": "qwerty"},
"x_object_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"x_object_uuid": "8ed9af1b-c948-4bb6-82b0-48344f5c822c",
"x_object_version": 98,
<hash>d41d8cd98f00b204e9800998ecf8427e</hash>
<content_type>application/octet-stream</content_type>
<last_modified>2011-12-02T08:10:41.565891+00:00</last_modified>
- <x_object_meta>
- <key>asdf</key><value>qwerty</value>
- </x_object_meta>
<x_object_hash>e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</x_object_hash>
<x_object_uuid>8ed9af1b-c948-4bb6-82b0-48344f5c822c</x_object_uuid>
<x_object_version>98</x_object_version>
<object>...</object>
</container>
-For more examples of container details returned in JSON/XML formats refer to the OOS API documentation. In addition to the OOS API, Pithos returns all fields. Metadata values are grouped and returned as key-value pairs.
+For more examples of container details returned in JSON/XML formats refer to the OOS API documentation. In addition to the OOS API, Pithos returns more fields that should help with synchronization.
=========================== ===============================
Return Code Description
====================== ============================================
Request Parameter Name Value
====================== ============================================
+format Optional hash list reply type (can be ``json`` or ``xml``)
update Do not replace metadata/policy (no value parameter)
====================== ============================================
-No reply content/headers, except when uploading data, where the reply consists of a list of hashes for the blocks received (in a simple text format, with one hash per line).
+No reply content/headers, except when uploading data, where the reply consists of a list of hashes for the blocks received (in the format specified).
The operation will overwrite all user defined metadata, except if ``update`` is defined.
To change policy, include an ``X-Container-Policy-*`` header with the name in the key. If no ``X-Container-Policy-*`` header is present, no changes will be applied to policy. The ``update`` parameter also applies to policy - deleted values will revert to defaults. To delete/revert a specific policy directive, use ``update`` and an empty header value. See container ``PUT`` for a reference of policy directives.
X-Object-Meta-* Optional user defined metadata
========================== ===============================
-|
+Sharing headers (``X-Object-Sharing``, ``X-Object-Shared-By`` and ``X-Object-Allowed-To``) are only included if the request is for the object's latest version (no specific ``version`` parameter is set).
=========================== ==============================
Return Code Description
====================== ===================================
Request Parameter Name Value
====================== ===================================
-format Optional extended request type (can be ``json`` or ``xml``)
+format Optional extended request/conflict response type (can be ``json`` or ``xml``)
hashmap Optional hashmap provided instead of data (no value parameter)
====================== ===================================
-The request is the object's data (or part of it), except if a hashmap is provided (using ``hashmap`` and ``format`` parameters). If using a hashmap and all different parts are stored in the server, the object is created, otherwise the server returns Conflict (409) with the list of the missing parts (in a simple text format, with one hash per line).
+The request is the object's data (or part of it), except if a hashmap is provided (using ``hashmap`` and ``format`` parameters). If using a hashmap and all different parts are stored in the server, the object is created. Otherwise the server returns Conflict (409) with the list of the missing parts (in simple text format, with one hash per line, or in JSON/XML - depending on the ``format`` parameter).
Hashmaps should be formatted as outlined in ``GET``.
Return Code Description
============================== ==============================
201 (Created) The object has been created
-409 (Conflict) The object can not be created from the provided hashmap, or there are conflicting permissions (a list of missing hashes, or a list of conflicting sharing paths will be included in the reply - in simple text format)
+409 (Conflict) The object can not be created from the provided hashmap (a list of missing hashes will be included in the reply)
411 (Length Required) Missing ``Content-Length`` or ``Content-Type`` in the request
413 (Request Entity Too Large) Insufficient quota to complete the request
422 (Unprocessable Entity) The MD5 checksum of the data written to the storage system does not match the (optionally) supplied ETag value
X-Object-Meta-* Optional user defined metadata
==================== ================================
-:sup:`*` *When using django locally with the supplied web server, do provide a valid Content-Type, as a type of text/plain is applied by default to all requests.*
+:sup:`*` *When using django locally with the supplied web server, use the ignore_content_type parameter, or do provide a valid Content-Type, as a type of text/plain is applied by default to all requests. Client software should always state ignore_content_type, except when a Content-Type is explicitly defined by the user.*
+
+====================== ===================================
+Request Parameter Name Value
+====================== ===================================
+format Optional conflict response type (can be ``json`` or ``xml``)
+ignore_content_type Ignore the supplied Content-Type
+====================== ===================================
Refer to ``PUT``/``POST`` for a description of request headers. Metadata is also copied, updated with any values defined. Sharing/publishing options are not copied.
Return Code Description
============================== ==============================
201 (Created) The object has been created
-409 (Conflict) There are conflicting permissions (a list of conflicting sharing paths will be included in the reply - in simple text format)
413 (Request Entity Too Large) Insufficient quota to complete the request
============================== ==============================
====================== ============================================
Request Parameter Name Value
====================== ============================================
+format Optional conflict response type (can be ``json`` or ``xml``)
update Do not replace metadata (no value parameter)
====================== ============================================
============================== ==============================
202 (Accepted) The request has been accepted (not a data update)
204 (No Content) The request succeeded (data updated)
-409 (Conflict) There are conflicting permissions (a list of conflicting sharing paths will be included in the reply - in simple text format)
411 (Length Required) Missing ``Content-Length`` in the request
413 (Request Entity Too Large) Insufficient quota to complete the request
416 (Range Not Satisfiable) The supplied range is invalid
<input type="submit">
</form>
-This will create/override the object with the given name, as if using ``PUT``. The ``Content-Type`` of the object will be set to the value of the corresponding header sent in the part of the request containing the data (usually, automatically handled by the browser). Metadata, sharing and other object attributes can not be set this way.
+This will create/override the object with the given name, as if using ``PUT``. The ``Content-Type`` of the object will be set to the value of the corresponding header sent in the part of the request containing the data (usually, automatically handled by the browser). Metadata, sharing and other object attributes can not be set this way. The response will contain the object's ETag.
========================== ===============================
Reply Header Name Value
Sharing and Public Objects
^^^^^^^^^^^^^^^^^^^^^^^^^^
-Read and write control in Pithos is managed by setting appropriate permissions with the ``X-Object-Sharing`` header. The permissions are applied using prefix-based inheritance. Thus, each set of authorization directives is applied to all objects sharing the same prefix with the object where the corresponding ``X-Object-Sharing`` header is defined. For simplicity, nested/overlapping permissions are not allowed. Setting ``X-Object-Sharing`` will fail, if the object is already "covered", or another object with a longer common-prefix name already has permissions. When retrieving an object, the ``X-Object-Shared-By`` header reports where it gets its permissions from. If not present, the object is the actual source of authorization directives.
+Read and write control in Pithos is managed by setting appropriate permissions with the ``X-Object-Sharing`` header. The permissions are applied using directory-based inheritance. A directory is an object with the corresponding content type. The default delimiter is ``/``. Thus, each set of authorization directives is applied to all objects in the directory object where the corresponding ``X-Object-Sharing`` header is defined. If there are nested/overlapping permissions, the closest to the object is applied. When retrieving an object, the ``X-Object-Shared-By`` header reports where it gets its permissions from. If not present, the object is the actual source of authorization directives.
A user may ``GET`` another account or container. The result will include a limited reply, containing only the allowed containers or objects respectively. A top-level request with an authentication token, will return a list of allowed accounts, so the user can easily find out which other users share objects. The ``X-Object-Allowed-To`` header lists the actions allowed on an object, if it does not belong to the requesting user.
* Headers ``X-Container-Block-*`` at the container level, exposing the underlying storage characteristics.
* All metadata replies, at all levels, include latest modification information.
* At all levels, a ``HEAD`` or ``GET`` request may use ``If-Modified-Since`` and ``If-Unmodified-Since`` headers.
-* Container/object lists include all associated metadata if the reply is of type JSON/XML. Some names are kept to their OOS API equivalents for compatibility.
+* Container/object lists include more fields if the reply is of type JSON/XML. Some names are kept to their OOS API equivalents for compatibility.
* Option to include only shared containers/objects in listings.
* Object metadata allowed, in addition to ``X-Object-Meta-*``: ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``. These are all replaced with every update operation, except if using the ``update`` parameter (in which case individual keys can also be deleted). Deleting meta by providing empty values also works when copying/moving an object.
* Multi-range object ``GET`` support as outlined in RFC2616.
* Object create using ``POST`` to support standard HTML forms.
* Partial object updates through ``POST``, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers. Use another object's data to update with ``X-Source-Object`` and ``X-Source-Version``. Truncate with ``X-Object-Bytes``.
* Include new version identifier in replies for object replace/change requests.
-* Object ``MOVE`` support.
+* Object ``MOVE`` support and ``ignore_content_type`` parameter in both ``COPY`` and ``MOVE``.
* Conditional object create/update operations, using ``If-Match`` and ``If-None-Match`` headers.
* Time-variant account/container listings via the ``until`` parameter.
* Object versions - parameter ``version`` in ``HEAD``/``GET`` (list versions with ``GET``), ``X-Object-Version-*`` meta in replies, ``X-Source-Version`` in ``PUT``/``COPY``.
* Sharing/publishing with ``X-Object-Sharing``, ``X-Object-Public`` at the object level. Cross-user operations are allowed - controlled by sharing directives. Available actions in cross-user requests are reported with ``X-Object-Allowed-To``. Permissions may include groups defined with ``X-Account-Group-*`` at the account level. These apply to the object - not its versions.
-* Support for prefix-based inheritance when enforcing permissions. Parent object carrying the authorization directives is reported in ``X-Object-Shared-By``.
+* Support for directory-based inheritance when enforcing permissions. Parent object carrying the authorization directives is reported in ``X-Object-Shared-By``.
* Copy and move between accounts with ``X-Source-Account`` and ``Destination-Account`` headers.
* Large object support with ``X-Object-Manifest``.
* Trace the user that created/modified an object with ``X-Object-Modified-By``.
projects / pithos / blobdiff