X-Git-Url: https://code.grnet.gr/git/pithos/blobdiff_plain/c9aba6cc1f4183787497b3efa73620359ce3ee12..0a8bf8fb8356f5b17592195cd9eb8b56571659f5:/docs/source/devguide.rst diff --git a/docs/source/devguide.rst b/docs/source/devguide.rst index adab85f..b172413 100644 --- a/docs/source/devguide.rst +++ b/docs/source/devguide.rst @@ -19,14 +19,31 @@ The present document is meant to be read alongside the OOS API documentation. Th Whatever marked as to be determined (**TBD**), should not be considered by implementors. +More info about Pithos can be found here: https://code.grnet.gr/projects/pithos + Document Revisions ^^^^^^^^^^^^^^^^^^ ========================= ================================ Revision Description ========================= ================================ -0.7 (Sept 28, 2011) Suggest upload/download methods using hashmaps. +0.8 (Jan 19, 2012) Update allowed versioning values. +\ Change policy/meta formatting in JSON/XML replies. +\ Document that all non-ASCII characters in headers should be URL-encoded. +\ Support metadata-based queries when listing objects at the container level. +\ Note Content-Type issue when using the internal django web server. +\ Add object UUID field. +\ Always reply with the MD5 in the ETag. +\ Note that ``/login`` will only work if an external authentication system is defined. +0.7 (Nov 21, 2011) Suggest upload/download methods using hashmaps. \ Propose syncing algorithm. +\ Support cross-account object copy and move. +\ Pass token as a request parameter when using ``POST`` via an HTML form. +\ Optionally use source account to update object from another object. +\ Use container ``POST`` to upload missing blocks of data. +\ Report policy in account headers. +\ Add insufficient quota reply. +\ Use special meta to always report Merkle hash. 0.6 (Sept 13, 2011) Reply with Merkle hash as the ETag when updating objects. \ Include version id in object replace/change replies. \ Change conflict (409) replies format to text. @@ -63,13 +80,11 @@ Revision Description Pithos Users and Authentication ------------------------------- -Pithos keeps separate databases for users and objects. - -Each user is uniquely identified by the ``Uniq`` field. This should be used as the user's account in the API. The API uses the ``Token`` field to authenticate a user, thus allowing cross-account requests. All API requests require a token. +In Pithos, each user is uniquely identified by a token. All API requests require a token and each token is internally resolved to an account string. The API uses the account string to identify the user's own files, thus whether a request is local or cross-account. -User entries can be modified/added via the management interface available at ``https://hostname/admin``. +Pithos does not keep a user database. For development and testing purposes, user identifiers and their corresponding tokens can be defined in the settings file. However, Pithos is designed with an external authentication service in mind. This service must handle the details of validating user credentials and communicate with Pithos via a middleware software component that, given a token, fills in the internal request account variable. -Pithos is also compatible with Shibboleth (http://shibboleth.internet2.edu/). The connection between Shibboleth and Pithos is done by ``https://hostname/login``. An application that wishes to connect to Pithos, but does not have a token, should redirect the user to the login URI. +Client software using Pithos, if not already knowing a user's identifier and token, should forward to the ``/login`` URI. The Pithos server, depending on its configuration will redirect to the appropriate login page. The login URI accepts the following parameters: @@ -80,9 +95,9 @@ next The URI to redirect to when the process is finished renew Force token renewal (no value parameter) ====================== ========================= -The login process starts by redirecting the user to an external URI (controlled by Shibboleth), where the actual authentication credentials are entered. Then, the user is redirected back to the login URI from Shibboleth, with various identification information in the request headers. +When done with logging in, the service's login URI should redirect to the URI provided with ``next``, adding ``user`` and ``token`` parameters, which contain the account and token fields respectively. -If the user does not exist in the database, Pithos adds the user and creates a random token. If the user exists, the token has not expired and ``renew`` is not set, the existing token is reused. Finally, the login URI redirects to the URI provided with ``next``, adding the ``user`` and ``token`` parameters, which contain the ``Uniq`` and ``Token`` fields respectively. +A user management service that implements a login URI according to these conventions is Astakos (https://code.grnet.gr/projects/astakos), by GRNET. The Pithos API -------------- @@ -102,7 +117,8 @@ The allowable request operations and respective return codes per level are prese Return Code Description ========================= ================================ 400 (Bad Request) The request is invalid -401 (Unauthorized) Request not allowed +401 (Unauthorized) Missing or invalid token +403 (Forbidden) Request not allowed 404 (Not Found) The requested resource was not found 503 (Service Unavailable) The request cannot be completed because of an internal error ========================= ================================ @@ -154,7 +170,7 @@ Example ``format=json`` reply: :: - [{"name": "user", "last_modified": "2011-07-19T10:48:16"}, ...] + [{"name": "user", "last_modified": "2011-12-02T08:10:41.565891+00:00"}, ...] Example ``format=xml`` reply: @@ -164,7 +180,7 @@ Example ``format=xml`` reply: user - 2011-07-19T10:48:16 + 2011-12-02T08:10:41.565891+00:00 ... @@ -176,7 +192,7 @@ Return Code Description 204 (No Content) The user has no access to other accounts (only for non-extended replies) =========================== ===================== -Will use a ``200`` return code if the reply is of type json/xml. +Will use a ``200`` return code if the reply is of type JSON/XML. Account Level ^^^^^^^^^^^^^ @@ -215,12 +231,10 @@ Cross-user requests are not allowed to use ``until`` and only include the accoun Reply Header Name Value ========================== ===================== X-Account-Container-Count The total number of containers -X-Account-Object-Count The total number of objects (**TBD**) X-Account-Bytes-Used The total number of bytes stored -X-Account-Bytes-Remaining The total number of bytes remaining (**TBD**) -X-Account-Last-Login The last login (**TBD**) X-Account-Until-Timestamp The last account modification date until the timestamp provided X-Account-Group-* Optional user defined groups +X-Account-Policy-* Account behavior and limits X-Account-Meta-* Optional user defined metadata Last-Modified The last account modification date (regardless of ``until``) ========================== ===================== @@ -274,7 +288,41 @@ x_container_policy_* Container behavior and limits x_container_meta_* Optional user defined metadata =========================== ============================ -For examples of container details returned in JSON/XML formats refer to the OOS API documentation. +Example ``format=json`` reply: + +:: + + [{"name": "pithos", + "bytes": 62452, + "count": 8374, + "last_modified": "2011-12-02T08:10:41.565891+00:00", + "x_container_policy": {"quota": "53687091200", "versioning": "auto"}, + "x_container_meta": {"a": "b", "1": "2"}}, ...] + +Example ``format=xml`` reply: + +:: + + + + + pithos + 62452 + 8374 + 2011-12-02T08:10:41.565891+00:00 + + quota53687091200 + versioningauto + + + ab + 12 + + + ... + + +For more examples of container details returned in JSON/XML formats refer to the OOS API documentation. In addition to the OOS API, Pithos returns all fields. Policy and metadata values are grouped and returned as key-value pairs. =========================== ===================== Return Code Description @@ -285,7 +333,7 @@ Return Code Description 412 (Precondition Failed) The condition set can not be satisfied =========================== ===================== -Will use a ``200`` return code if the reply is of type json/xml. +Will use a ``200`` return code if the reply is of type JSON/XML. POST @@ -398,7 +446,7 @@ prefix Return objects starting with prefix delimiter Return objects up to the delimiter (discussion follows) path Assume ``prefix=path`` and ``delimiter=/`` format Optional extended reply type (can be ``json`` or ``xml``) -meta Return objects having the specified meta keys (can be a comma separated list) +meta Return objects that satisfy the key queries in the specified comma separated list (use ````, ``!`` for existence queries, ```` for value queries, where ```` can be one of ``=``, ``!=``, ``<=``, ``>=``, ``<``, ``>``) shared Show only shared objects (no value parameter) until Optional timestamp ====================== =================================== @@ -432,6 +480,8 @@ content_type The MIME content type of the object content_encoding The encoding of the object (optional) content-disposition The presentation style of the object (optional) last_modified The last object modification date (regardless of version) +x_object_hash The Merkle hash +x_object_uuid The object's UUID x_object_version The object's version identifier x_object_version_timestamp The object's version timestamp x_object_modified_by The user that committed the object's version @@ -445,10 +495,50 @@ x_object_meta_* Optional user defined metadata Extended replies may also include virtual directory markers in separate sections of the ``json`` or ``xml`` results. Virtual directory markers are only included when ``delimiter`` is explicitly set. They correspond to the substrings up to and including the first occurrence of the delimiter. -In JSON results they appear as dictionaries with only a ``"subdir"`` key. In XML results they appear interleaved with ```` tags as ````. +In JSON results they appear as dictionaries with only a ``subdir`` key. In XML results they appear interleaved with ```` tags as ````. In case there is an object with the same name as a virtual directory marker, the object will be returned. -For examples of object details returned in JSON/XML formats refer to the OOS API documentation. +Example ``format=json`` reply: + +:: + + [{"name": "object", + "bytes": 0, + "hash": "d41d8cd98f00b204e9800998ecf8427e", + "content_type": "application/octet-stream", + "last_modified": "2011-12-02T08:10:41.565891+00:00", + "x_object_meta": {"asdf": "qwerty"}, + "x_object_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "x_object_uuid": "8ed9af1b-c948-4bb6-82b0-48344f5c822c", + "x_object_version": 98, + "x_object_version_timestamp": "1322813441.565891", + "x_object_modified_by": "user"}, ...] + +Example ``format=xml`` reply: + +:: + + + + + object + 0 + d41d8cd98f00b204e9800998ecf8427e + application/octet-stream + 2011-12-02T08:10:41.565891+00:00 + + asdfqwerty + + e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 + 8ed9af1b-c948-4bb6-82b0-48344f5c822c + 98 + 1322813441.565891 + chazapis + + ... + + +For more examples of container details returned in JSON/XML formats refer to the OOS API documentation. In addition to the OOS API, Pithos returns all fields. Metadata values are grouped and returned as key-value pairs. =========================== =============================== Return Code Description @@ -459,7 +549,7 @@ Return Code Description 412 (Precondition Failed) The condition set can not be satisfied =========================== =============================== -Will use a ``200`` return code if the reply is of type json/xml. +Will use a ``200`` return code if the reply is of type JSON/XML. PUT @@ -477,7 +567,7 @@ No reply content/headers. If no policy is defined, the container will be created with the default values. Available policy directives: -* ``versioning``: Set to ``auto``, ``manual`` or ``none`` (default is ``manual``) +* ``versioning``: Set to ``auto`` or ``none`` (default is ``auto``) * ``quota``: Size limit in KB (default is ``0`` - unlimited) If the container already exists, the operation is equal to a ``POST`` with ``update`` defined. @@ -496,6 +586,9 @@ POST ==================== ================================ Request Header Name Value ==================== ================================ +Content-Length The size of the supplied data (optional, to upload) +Content-Type The MIME content type of the supplied data (optional, to upload) +Transfer-Encoding Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored) X-Container-Policy-* Container behavior and limits X-Container-Meta-* Optional user defined metadata ==================== ================================ @@ -508,11 +601,13 @@ Request Parameter Name Value update Do not replace metadata/policy (no value parameter) ====================== ============================================ -No reply content/headers. +No reply content/headers, except when uploading data, where the reply consists of a list of hashes for the blocks received (in a simple text format, with one hash per line). The operation will overwrite all user defined metadata, except if ``update`` is defined. To change policy, include an ``X-Container-Policy-*`` header with the name in the key. If no ``X-Container-Policy-*`` header is present, no changes will be applied to policy. The ``update`` parameter also applies to policy - deleted values will revert to defaults. To delete/revert a specific policy directive, use ``update`` and an empty header value. See container ``PUT`` for a reference of policy directives. +To upload blocks of data to the container, set ``Content-Type`` to ``application/octet-stream`` and ``Content-Length`` to a valid value (except if using ``chunked`` as the ``Transfer-Encoding``). + ================ =============================== Return Code Description ================ =============================== @@ -590,6 +685,8 @@ Content-Type The MIME content type of the object Last-Modified The last object modification date (regardless of version) Content-Encoding The encoding of the object (optional) Content-Disposition The presentation style of the object (optional) +X-Object-Hash The Merkle hash +X-Object-UUID The object's UUID X-Object-Version The object's version identifier X-Object-Version-Timestamp The object's version timestamp X-Object-Modified-By The user that comitted the object's version @@ -636,7 +733,7 @@ version Optional version identifier or ``list`` (specify a forma The reply is the object's data (or part of it), except if a hashmap is requested with ``hashmap``, or a version list with ``version=list`` (in both cases an extended reply format must be specified). Object headers (as in a ``HEAD`` request) are always included. -Hashmaps expose the underlying storage format of the object. Note that each hash is computed after trimming trailing null bytes of the corresponding block. +Hashmaps expose the underlying storage format of the object. Note that each hash is computed after trimming trailing null bytes of the corresponding block. The ``X-Object-Hash`` header reports the single Merkle hash of the object's hashmap (refer to http://bittorrent.org/beps/bep_0030.html for more information). Example ``format=json`` reply: @@ -660,7 +757,7 @@ Example ``format=json`` reply: :: - {"versions": [[23, 1307700892], [28, 1307700898], ...]} + {"versions": [[85, "1322734861.248469"], [86, "1322734905.009272"], ...]} Example ``format=xml`` reply: @@ -668,8 +765,8 @@ Example ``format=xml`` reply: - 23 - 28 + 85 + 86 ... @@ -685,6 +782,8 @@ Content-Range The range of data included (only on a single range r Last-Modified The last object modification date (regardless of version) Content-Encoding The encoding of the object (optional) Content-Disposition The presentation style of the object (optional) +X-Object-Hash The Merkle hash +X-Object-UUID The object's UUID X-Object-Version The object's version identifier X-Object-Version-Timestamp The object's version timestamp X-Object-Modified-By The user that comitted the object's version @@ -723,6 +822,7 @@ Content-Type The MIME content type of the object Transfer-Encoding Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored) X-Copy-From The source path in the form ``//`` X-Move-From The source path in the form ``//`` +X-Source-Account The source account to copy/move from X-Source-Version The source version to copy from Content-Encoding The encoding of the object (optional) Content-Disposition The presentation style of the object (optional) @@ -748,20 +848,21 @@ Hashmaps should be formatted as outlined in ``GET``. ========================== =============================== Reply Header Name Value ========================== =============================== -ETag The MD5 hash of the object (on create) +ETag The MD5 hash of the object X-Object-Version The object's new version ========================== =============================== The ``X-Object-Sharing`` header may include either a ``read=...`` comma-separated user/group list, or a ``write=...`` comma-separated user/group list, or both separated by a semicolon (``;``). Groups are specified as ``:``. To publish the object, set ``X-Object-Public`` to ``true``. To unpublish, set to ``false``, or use an empty header value. -=========================== ============================== -Return Code Description -=========================== ============================== -201 (Created) The object has been created -409 (Conflict) The object can not be created from the provided hashmap, or there are conflicting permissions (a list of missing hashes, or a list of conflicting sharing paths will be included in the reply - in simple text format) -411 (Length Required) Missing ``Content-Length`` or ``Content-Type`` in the request -422 (Unprocessable Entity) The MD5 checksum of the data written to the storage system does not match the (optionally) supplied ETag value -=========================== ============================== +============================== ============================== +Return Code Description +============================== ============================== +201 (Created) The object has been created +409 (Conflict) The object can not be created from the provided hashmap, or there are conflicting permissions (a list of missing hashes, or a list of conflicting sharing paths will be included in the reply - in simple text format) +411 (Length Required) Missing ``Content-Length`` or ``Content-Type`` in the request +413 (Request Entity Too Large) Insufficient quota to complete the request +422 (Unprocessable Entity) The MD5 checksum of the data written to the storage system does not match the (optionally) supplied ETag value +============================== ============================== COPY @@ -773,7 +874,8 @@ Request Header Name Value If-Match Proceed if ETags match with object If-None-Match Proceed if ETags don't match with object Destination The destination path in the form ``//`` -Content-Type The MIME content type of the object (optional) +Destination-Account The destination account to copy to +Content-Type The MIME content type of the object (optional :sup:`*`) Content-Encoding The encoding of the object (optional) Content-Disposition The presentation style of the object (optional) X-Source-Version The source version to copy from @@ -783,6 +885,8 @@ X-Object-Public Object is publicly accessible (optional) X-Object-Meta-* Optional user defined metadata ==================== ================================ +:sup:`*` *When using django locally with the supplied web server, do provide a valid Content-Type, as a type of text/plain is applied by default to all requests.* + Refer to ``PUT``/``POST`` for a description of request headers. Metadata is also copied, updated with any values defined. Sharing/publishing options are not copied. ========================== =============================== @@ -793,12 +897,13 @@ X-Object-Version The object's new version | -=========================== ============================== -Return Code Description -=========================== ============================== -201 (Created) The object has been created -409 (Conflict) There are conflicting permissions (a list of conflicting sharing paths will be included in the reply - in simple text format) -=========================== ============================== +============================== ============================== +Return Code Description +============================== ============================== +201 (Created) The object has been created +409 (Conflict) There are conflicting permissions (a list of conflicting sharing paths will be included in the reply - in simple text format) +413 (Request Entity Too Large) Insufficient quota to complete the request +============================== ============================== MOVE @@ -822,6 +927,7 @@ Transfer-Encoding Set to ``chunked`` to specify incremental uploading (if us Content-Encoding The encoding of the object (optional) Content-Disposition The presentation style of the object (optional) X-Source-Object Update with data from the object at path ``//`` (optional, to update) +X-Source-Account The source account to update from X-Source-Version The source version to update from (optional, to update) X-Object-Bytes The updated object's final size (optional, when updating) X-Object-Manifest Object parts prefix in ``/`` form (optional) @@ -854,7 +960,7 @@ To update an object's data: Optionally, truncate the updated object to the desired length with the ``X-Object-Bytes`` header. -A data update will trigger an ETag change. Updated ETags correspond to the single Merkle hash of the object's hashmap (refer to http://bittorrent.org/beps/bep_0030.html for more information). +A data update will trigger an ETag change. Updated ETags may happen asynchronously and appear at the server with a delay. No reply content. No reply headers if only metadata is updated. @@ -867,25 +973,25 @@ X-Object-Version The object's new version | -=========================== ============================== -Return Code Description -=========================== ============================== -202 (Accepted) The request has been accepted (not a data update) -204 (No Content) The request succeeded (data updated) -409 (Conflict) There are conflicting permissions (a list of conflicting sharing paths will be included in the reply - in simple text format) -411 (Length Required) Missing ``Content-Length`` in the request -416 (Range Not Satisfiable) The supplied range is invalid -=========================== ============================== +============================== ============================== +Return Code Description +============================== ============================== +202 (Accepted) The request has been accepted (not a data update) +204 (No Content) The request succeeded (data updated) +409 (Conflict) There are conflicting permissions (a list of conflicting sharing paths will be included in the reply - in simple text format) +411 (Length Required) Missing ``Content-Length`` in the request +413 (Request Entity Too Large) Insufficient quota to complete the request +416 (Range Not Satisfiable) The supplied range is invalid +============================== ============================== -The ``POST`` method can also be used for creating an object via a standard HTML form. If the request ``Content-Type`` is ``multipart/form-data``, none of the above headers will be processed. The form should have exactly two fields, as in the following example. :: +The ``POST`` method can also be used for creating an object via a standard HTML form. If the request ``Content-Type`` is ``multipart/form-data``, none of the above headers will be processed. The form should have an ``X-Object-Data`` field, as in the following example. The token is passed as a request parameter. :: -
- +
-This will create/override the object with the given name, as if using ``PUT``. The ``Content-Type`` of the object will be set to the value of the corresponding header sent in the part of the request containing the data. Metadata, sharing and other object attributes can not be set this way. +This will create/override the object with the given name, as if using ``PUT``. The ``Content-Type`` of the object will be set to the value of the corresponding header sent in the part of the request containing the data (usually, automatically handled by the browser). Metadata, sharing and other object attributes can not be set this way. ========================== =============================== Reply Header Name Value @@ -896,11 +1002,12 @@ X-Object-Version The object's new version | -=========================== ============================== -Return Code Description -=========================== ============================== -201 (Created) The object has been created -=========================== ============================== +============================== ============================== +Return Code Description +============================== ============================== +201 (Created) The object has been created +413 (Request Entity Too Large) Insufficient quota to complete the request +============================== ============================== DELETE @@ -953,18 +1060,20 @@ List of differences from the OOS API: * Support for ``X-Account-Meta-*`` style headers at the account level. Use ``POST`` to update. * Support for ``X-Container-Meta-*`` style headers at the container level. Can be set when creating via ``PUT``. Use ``POST`` to update. * Header ``X-Container-Object-Meta`` at the container level and parameter ``meta`` in container listings. (**TBD**) -* Container policies to manage behavior and limits. +* Account and container policies to manage behavior and limits. Container behavior overrides account settings. Account quota sets the maximum bytes limit, regardless of container values. * Headers ``X-Container-Block-*`` at the container level, exposing the underlying storage characteristics. * All metadata replies, at all levels, include latest modification information. * At all levels, a ``HEAD`` or ``GET`` request may use ``If-Modified-Since`` and ``If-Unmodified-Since`` headers. -* Container/object lists include all associated metadata if the reply is of type json/xml. Some names are kept to their OOS API equivalents for compatibility. +* Container/object lists include all associated metadata if the reply is of type JSON/XML. Some names are kept to their OOS API equivalents for compatibility. * Option to include only shared containers/objects in listings. * Object metadata allowed, in addition to ``X-Object-Meta-*``: ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``. These are all replaced with every update operation, except if using the ``update`` parameter (in which case individual keys can also be deleted). Deleting meta by providing empty values also works when copying/moving an object. * Multi-range object ``GET`` support as outlined in RFC2616. * Object hashmap retrieval through ``GET`` and the ``format`` parameter. * Object create via hashmap through ``PUT`` and the ``format`` parameter. +* The object's Merkle hash is always returned in the ``X-Object-Hash`` header. +* The object's UUID is always returned in the ``X-Object-UUID`` header. The UUID remains unchanged, even when the object's data or metadata changes, or the object is moved to another path (is renamed). A new UUID is assigned when creating or copying an object. * Object create using ``POST`` to support standard HTML forms. -* Partial object updates through ``POST``, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers. Use another object's data to update with ``X-Source-Object`` and ``X-Source-Version``. Truncate with ``X-Object-Bytes``. New ETag corresponds to the Merkle hash of the object's hashmap. +* Partial object updates through ``POST``, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers. Use another object's data to update with ``X-Source-Object`` and ``X-Source-Version``. Truncate with ``X-Object-Bytes``. * Include new version identifier in replies for object replace/change requests. * Object ``MOVE`` support. * Conditional object create/update operations, using ``If-Match`` and ``If-None-Match`` headers. @@ -972,18 +1081,20 @@ List of differences from the OOS API: * Object versions - parameter ``version`` in ``HEAD``/``GET`` (list versions with ``GET``), ``X-Object-Version-*`` meta in replies, ``X-Source-Version`` in ``PUT``/``COPY``. * Sharing/publishing with ``X-Object-Sharing``, ``X-Object-Public`` at the object level. Cross-user operations are allowed - controlled by sharing directives. Available actions in cross-user requests are reported with ``X-Object-Allowed-To``. Permissions may include groups defined with ``X-Account-Group-*`` at the account level. These apply to the object - not its versions. * Support for prefix-based inheritance when enforcing permissions. Parent object carrying the authorization directives is reported in ``X-Object-Shared-By``. +* Copy and move between accounts with ``X-Source-Account`` and ``Destination-Account`` headers. * Large object support with ``X-Object-Manifest``. * Trace the user that created/modified an object with ``X-Object-Modified-By``. * Purge container/object history with the ``until`` parameter in ``DELETE``. Clarifications/suggestions: +* All non-ASCII characters in headers should be URL-encoded. * Authentication is done by another system. The token is used in the same way, but it is obtained differently. The top level ``GET`` request is kept compatible with the OOS API and allows for guest/testing operations. * Some processing is done in the variable part of all ``X-*-Meta-*`` headers. If it includes underscores, they will be converted to dashes and the first letter of all intra-dash strings will be capitalized. * A ``GET`` reply for a level will include all headers of the corresponding ``HEAD`` request. * To avoid conflicts between objects and virtual directory markers in container listings, it is recommended that object names do not end with the delimiter used. * The ``Accept`` header may be used in requests instead of the ``format`` parameter to specify the desired request/reply format. The parameter overrides the header. -* Container/object lists use a ``200`` return code if the reply is of type json/xml. The reply will include an empty json/xml. +* Container/object lists use a ``200`` return code if the reply is of type JSON/XML. The reply will include an empty JSON/XML. * In headers, dates are formatted according to RFC 1123. In extended information listings, the ``last_modified`` field is formatted according to ISO 8601 (for OOS API compatibility). All other fields (Pithos extensions) use integer tiemstamps. * The ``Last-Modified`` header value always reflects the actual latest change timestamp, regardless of time control parameters and version requests. Time precondition checks with ``If-Modified-Since`` and ``If-Unmodified-Since`` headers are applied to this value. * A copy/move using ``PUT``/``COPY``/``MOVE`` will always update metadata, keeping all old values except the ones redefined in the request headers. @@ -1049,9 +1160,9 @@ In the case of an upload, only the missing blocks will be submitted to the serve * Server responds with status ``409`` (Conflict): * Server's response body contains the hashes of the blocks that do not exist on the server. - * For each one of the hash values in the server's response: + * For each hash value in the server's response (or all hashes together): - * Send a ``PUT`` request to the server with the corresponding data block. Individual blocks are uploaded to a file named ``.upload``. + * Send a ``POST`` request to the destination container with the corresponding data. * Repeat hashmap ``PUT``. Fail if the server's response is not ``201``. @@ -1092,7 +1203,7 @@ Consider the following algorithm for synchronizing a local folder with the serve Notes: -* States represent file hashes (either MD5 or Merkle). Deleted or non-existing files are assumed to have a magic hash (e.g. empty string). +* States represent file hashes (it is suggested to use Merkle). Deleted or non-existing files are assumed to have a magic hash (e.g. empty string). * Updating a state (either local or remote) implies downloading, uploading or deleting the appropriate file. Recommended Practices and Examples @@ -1173,7 +1284,7 @@ Assuming an authentication token is obtained, the following high-level operation curl -X PUT -D - \ -H "X-Auth-Token: 0000" \ - -H "Content-Type: application/folder" \ + -H "Content-Type: application/directory" \ https://pithos.dev.grnet.gr/v1/user/pithos/folder * Add a new object ::