X-Git-Url: https://code.grnet.gr/git/pithos/blobdiff_plain/6d897a63a8c4d5e2a36d4b0df38685025c53f946..3be0f6c1467057cfeff98ba41640531fbd285c6d:/docs/source/devguide.rst
diff --git a/docs/source/devguide.rst b/docs/source/devguide.rst
index c69387f..5f4fd4c 100644
--- a/docs/source/devguide.rst
+++ b/docs/source/devguide.rst
@@ -19,18 +19,34 @@ The present document is meant to be read alongside the OOS API documentation. Th
Whatever marked as to be determined (**TBD**), should not be considered by implementors.
+More info about Pithos can be found here: https://code.grnet.gr/projects/pithos
+
Document Revisions
^^^^^^^^^^^^^^^^^^
========================= ================================
Revision Description
========================= ================================
-0.7 (Oct 4, 2011) Suggest upload/download methods using hashmaps.
+0.8 (Feb 9, 2012) Update allowed versioning values.
+\ Change policy/meta formatting in JSON/XML replies.
+\ Document that all non-ASCII characters in headers should be URL-encoded.
+\ Support metadata-based queries when listing objects at the container level.
+\ Note Content-Type issue when using the internal django web server.
+\ Add object UUID field.
+\ Always reply with the MD5 in the ETag.
+\ Note that ``/login`` will only work if an external authentication system is defined.
+\ Include option to ignore Content-Type on ``COPY``/``MOVE``.
+\ Use format parameter for conflict (409) and uploaded hash list (container level) replies.
+\ Change permissions model.
+0.7 (Nov 21, 2011) Suggest upload/download methods using hashmaps.
\ Propose syncing algorithm.
\ Support cross-account object copy and move.
\ Pass token as a request parameter when using ``POST`` via an HTML form.
\ Optionally use source account to update object from another object.
\ Use container ``POST`` to upload missing blocks of data.
+\ Report policy in account headers.
+\ Add insufficient quota reply.
+\ Use special meta to always report Merkle hash.
0.6 (Sept 13, 2011) Reply with Merkle hash as the ETag when updating objects.
\ Include version id in object replace/change replies.
\ Change conflict (409) replies format to text.
@@ -67,13 +83,11 @@ Revision Description
Pithos Users and Authentication
-------------------------------
-Pithos keeps separate databases for users and objects.
-
-Each user is uniquely identified by the ``Uniq`` field. This should be used as the user's account in the API. The API uses the ``Token`` field to authenticate a user, thus allowing cross-account requests. All API requests require a token.
+In Pithos, each user is uniquely identified by a token. All API requests require a token and each token is internally resolved to an account string. The API uses the account string to identify the user's own files, thus whether a request is local or cross-account.
-User entries can be modified/added via the management interface available at ``https://hostname/admin``.
+Pithos does not keep a user database. For development and testing purposes, user identifiers and their corresponding tokens can be defined in the settings file. However, Pithos is designed with an external authentication service in mind. This service must handle the details of validating user credentials and communicate with Pithos via a middleware software component that, given a token, fills in the internal request account variable.
-Pithos is also compatible with Shibboleth (http://shibboleth.internet2.edu/). The connection between Shibboleth and Pithos is done by ``https://hostname/login``. An application that wishes to connect to Pithos, but does not have a token, should redirect the user to the login URI.
+Client software using Pithos, if not already knowing a user's identifier and token, should forward to the ``/login`` URI. The Pithos server, depending on its configuration will redirect to the appropriate login page.
The login URI accepts the following parameters:
@@ -84,9 +98,9 @@ next The URI to redirect to when the process is finished
renew Force token renewal (no value parameter)
====================== =========================
-The login process starts by redirecting the user to an external URI (controlled by Shibboleth), where the actual authentication credentials are entered. Then, the user is redirected back to the login URI from Shibboleth, with various identification information in the request headers.
+When done with logging in, the service's login URI should redirect to the URI provided with ``next``, adding ``user`` and ``token`` parameters, which contain the account and token fields respectively.
-If the user does not exist in the database, Pithos adds the user and creates a random token. If the user exists, the token has not expired and ``renew`` is not set, the existing token is reused. Finally, the login URI redirects to the URI provided with ``next``, adding the ``user`` and ``token`` parameters, which contain the ``Uniq`` and ``Token`` fields respectively.
+A user management service that implements a login URI according to these conventions is Astakos (https://code.grnet.gr/projects/astakos), by GRNET.
The Pithos API
--------------
@@ -106,7 +120,8 @@ The allowable request operations and respective return codes per level are prese
Return Code Description
========================= ================================
400 (Bad Request) The request is invalid
-401 (Unauthorized) Request not allowed
+401 (Unauthorized) Missing or invalid token
+403 (Forbidden) Request not allowed
404 (Not Found) The requested resource was not found
503 (Service Unavailable) The request cannot be completed because of an internal error
========================= ================================
@@ -158,7 +173,7 @@ Example ``format=json`` reply:
::
- [{"name": "user", "last_modified": "2011-07-19T10:48:16"}, ...]
+ [{"name": "user", "last_modified": "2011-12-02T08:10:41.565891+00:00"}, ...]
Example ``format=xml`` reply:
@@ -168,7 +183,7 @@ Example ``format=xml`` reply:
user
- 2011-07-19T10:48:16
+ 2011-12-02T08:10:41.565891+00:00...
@@ -180,7 +195,7 @@ Return Code Description
204 (No Content) The user has no access to other accounts (only for non-extended replies)
=========================== =====================
-Will use a ``200`` return code if the reply is of type json/xml.
+Will use a ``200`` return code if the reply is of type JSON/XML.
Account Level
^^^^^^^^^^^^^
@@ -219,12 +234,10 @@ Cross-user requests are not allowed to use ``until`` and only include the accoun
Reply Header Name Value
========================== =====================
X-Account-Container-Count The total number of containers
-X-Account-Object-Count The total number of objects (**TBD**)
X-Account-Bytes-Used The total number of bytes stored
-X-Account-Bytes-Remaining The total number of bytes remaining (**TBD**)
-X-Account-Last-Login The last login (**TBD**)
X-Account-Until-Timestamp The last account modification date until the timestamp provided
X-Account-Group-* Optional user defined groups
+X-Account-Policy-* Account behavior and limits
X-Account-Meta-* Optional user defined metadata
Last-Modified The last account modification date (regardless of ``until``)
========================== =====================
@@ -278,7 +291,41 @@ x_container_policy_* Container behavior and limits
x_container_meta_* Optional user defined metadata
=========================== ============================
-For examples of container details returned in JSON/XML formats refer to the OOS API documentation.
+Example ``format=json`` reply:
+
+::
+
+ [{"name": "pithos",
+ "bytes": 62452,
+ "count": 8374,
+ "last_modified": "2011-12-02T08:10:41.565891+00:00",
+ "x_container_policy": {"quota": "53687091200", "versioning": "auto"},
+ "x_container_meta": {"a": "b", "1": "2"}}, ...]
+
+Example ``format=xml`` reply:
+
+::
+
+
+
+
+ pithos
+ 62452
+ 8374
+ 2011-12-02T08:10:41.565891+00:00
+
+ quota53687091200
+ versioningauto
+
+
+ ab
+ 12
+
+
+ ...
+
+
+For more examples of container details returned in JSON/XML formats refer to the OOS API documentation. In addition to the OOS API, Pithos returns all fields. Policy and metadata values are grouped and returned as key-value pairs.
=========================== =====================
Return Code Description
@@ -289,7 +336,7 @@ Return Code Description
412 (Precondition Failed) The condition set can not be satisfied
=========================== =====================
-Will use a ``200`` return code if the reply is of type json/xml.
+Will use a ``200`` return code if the reply is of type JSON/XML.
POST
@@ -402,7 +449,7 @@ prefix Return objects starting with prefix
delimiter Return objects up to the delimiter (discussion follows)
path Assume ``prefix=path`` and ``delimiter=/``
format Optional extended reply type (can be ``json`` or ``xml``)
-meta Return objects having the specified meta keys (can be a comma separated list)
+meta Return objects that satisfy the key queries in the specified comma separated list (use ````, ``!`` for existence queries, ```` for value queries, where ```` can be one of ``=``, ``!=``, ``<=``, ``>=``, ``<``, ``>``)
shared Show only shared objects (no value parameter)
until Optional timestamp
====================== ===================================
@@ -436,6 +483,8 @@ content_type The MIME content type of the object
content_encoding The encoding of the object (optional)
content-disposition The presentation style of the object (optional)
last_modified The last object modification date (regardless of version)
+x_object_hash The Merkle hash
+x_object_uuid The object's UUID
x_object_version The object's version identifier
x_object_version_timestamp The object's version timestamp
x_object_modified_by The user that committed the object's version
@@ -447,12 +496,54 @@ x_object_public Object's publicly accessible URI (optional)
x_object_meta_* Optional user defined metadata
========================== ======================================
+Sharing metadata will only be returned if there is no ``until`` parameter defined.
+
Extended replies may also include virtual directory markers in separate sections of the ``json`` or ``xml`` results.
Virtual directory markers are only included when ``delimiter`` is explicitly set. They correspond to the substrings up to and including the first occurrence of the delimiter.
-In JSON results they appear as dictionaries with only a ``"subdir"`` key. In XML results they appear interleaved with ``