1 Pithos v2 Developer Guide
2 =========================
7 Pithos is a storage service implemented by GRNET (http://www.grnet.gr). Data is stored as objects, organized in containers, belonging to an account. This hierarchy of storage layers has been inspired by the OpenStack Object Storage (OOS) API and similar CloudFiles API by Rackspace. The Pithos API follows the OOS API as closely as possible. One of the design requirements has been to be able to use Pithos with clients built for the OOS, without changes.
9 However, to be able to take full advantage of the Pithos infrastructure, client software should be aware of the extensions that differentiate Pithos from OOS. Pithos objects can be updated, or appended to. Automatic version management, allows taking account and container listings back in time, as well as reading previous instances of objects.
11 The storage backend of Pithos is block oriented, permitting efficient, deduplicated data placement. The block structure of objects is exposed at the API layer, in order to encourage external software to implement advanced data management operations.
13 This document's goals are:
15 * Define the Pithos ReST API that allows the storage and retrieval of data and metadata via HTTP calls
16 * Specify metadata semantics and user interface guidelines for a common experience across client software implementations
18 The present document is meant to be read alongside the OOS API documentation. Thus, it is suggested that the reader is familiar with associated technologies, the OOS API as well as the first version of the Pithos API. This document refers to the second version of Pithos. Information on the first version of the storage API can be found at http://code.google.com/p/gss.
20 Whatever marked as to be determined (**TBD**), should not be considered by implementors.
25 ========================= ================================
27 ========================= ================================
28 0.3 (June 14, 2011) Large object support with ``X-Object-Manifest``.
29 \ Allow for publicly available objects via ``https://hostname/public``.
30 \ Support time-variant account/container listings.
31 \ Add source version when duplicating with PUT/COPY/MOVE.
32 \ Request version in object HEAD/GET requests (list versions with GET).
33 0.2 (May 31, 2011) Add object meta listing and filtering in containers.
34 \ Include underlying storage characteristics in container meta.
35 \ Support for partial object updates through POST.
36 \ Expose object hashmaps through GET.
37 \ Support for multi-range object GET requests.
38 0.1 (May 17, 2011) Initial release. Based on OpenStack Object Storage Developer Guide API v1 (Apr. 15, 2011).
39 ========================= ================================
44 The URI requests supported by the Pithos API follow one of the following forms:
46 * Top level: ``https://hostname/v1/``
47 * Account level: ``https://hostname/v1/<account>``
48 * Container level: ``https://hostname/v1/<account>/<container>``
49 * Object level: ``https://hostname/v1/<account>/<container>/<object>``
51 All requests must include an ``X-Auth-Token``. The process of obtaining the token is still to be determined (**TBD**).
53 The allowable request operations and respective return codes per level are presented in the remainder of this chapter. Common to all requests are the following return codes.
55 ========================= ================================
56 Return Code Description
57 ========================= ================================
58 400 (Bad Request) The request is invalid
59 401 (Unauthorized) Request not allowed
60 404 (Not Found) The requested resource was not found
61 503 (Service Unavailable) The request cannot be completed because of an internal error
62 ========================= ================================
69 ========= ==================
71 ========= ==================
72 GET Authentication. This is kept for compatibility with the OOS API
73 ========= ==================
78 If the ``X-Auth-User`` and ``X-Auth-Key`` headers are given, a dummy ``X-Auth-Token`` and ``X-Storage-Url`` will be replied, which can be used as a guest token/namespace for testing Pithos.
80 ================ =====================
81 Return Code Description
82 ================ =====================
83 204 (No Content) The request succeeded
84 ================ =====================
92 ========= ==================
94 ========= ==================
95 HEAD Retrieve account metadata
97 POST Update account metadata
98 ========= ==================
103 ====================== ===================================
104 Request Parameter Name Value
105 ====================== ===================================
106 until Optional timestamp
107 ====================== ===================================
111 ========================== =====================
112 Reply Header Name Value
113 ========================== =====================
114 X-Account-Container-Count The total number of containers
115 X-Account-Object-Count The total number of objects (**TBD**)
116 X-Account-Bytes-Used The total number of bytes stored
117 X-Account-Bytes-Remaining The total number of bytes remaining (**TBD**)
118 X-Account-Last-Login The last login (**TBD**)
119 X-Account-Until-Timestamp The last account modification date until the timestamp provided
120 X-Account-Meta-* Optional user defined metadata
121 Last-Modified The last account modification date (regardless of ``until``)
122 ========================== =====================
126 ================ =====================
127 Return Code Description
128 ================ =====================
129 204 (No Content) The request succeeded
130 ================ =====================
136 ==================== ===========================
137 Request Header Name Value
138 ==================== ===========================
139 If-Modified-Since Retrieve if account has changed since provided timestamp
140 If-Unmodified-Since Retrieve if account has not changed since provided timestamp
141 ==================== ===========================
145 ====================== =========================
146 Request Parameter Name Value
147 ====================== =========================
148 limit The amount of results requested (default is 10000)
149 marker Return containers with name lexicographically after marker
150 format Optional extended reply type (can be ``json`` or ``xml``)
151 until Optional timestamp
152 ====================== =========================
154 The reply is a list of container names. Account headers (as in a ``HEAD`` request) will also be included.
155 If a ``format=xml`` or ``format=json`` argument is given, extended information on the containers will be returned, serialized in the chosen format.
156 For each container, the information will include all container metadata (names will be in lower case and with hyphens replaced with underscores):
158 =========================== ============================
160 =========================== ============================
161 name The name of the container
162 count The number of objects inside the container
163 bytes The total size of the objects inside the container
164 last_modified The last container modification date (regardless of ``until``)
165 x_container_until_timestamp The last container modification date until the timestamp provided
166 x_container_meta_* Optional user defined metadata
167 =========================== ============================
169 For examples of container details returned in JSON/XML formats refer to the OOS API documentation.
171 =========================== =====================
172 Return Code Description
173 =========================== =====================
174 200 (OK) The request succeeded
175 204 (No Content) The account has no containers (only for non-extended replies)
176 304 (Not Modified) The account has not been modified
177 412 (Precondition Failed) The condition set can not be satisfied
178 =========================== =====================
180 Will use a ``200`` return code if the reply is of type json/xml.
186 ==================== ===========================
187 Request Header Name Value
188 ==================== ===========================
189 X-Account-Meta-* Optional user defined metadata
190 ==================== ===========================
192 No reply content/headers.
194 The update operation will overwrite all user defined metadata.
196 ================ ===============================
197 Return Code Description
198 ================ ===============================
199 202 (Accepted) The request has been accepted
200 ================ ===============================
208 ========= ============================
209 Operation Description
210 ========= ============================
211 HEAD Retrieve container metadata
213 PUT Create/update container
214 POST Update container metadata
215 DELETE Delete container
216 ========= ============================
222 ====================== ===================================
223 Request Parameter Name Value
224 ====================== ===================================
225 until Optional timestamp
226 ====================== ===================================
230 =========================== ===============================
231 Reply Header Name Value
232 =========================== ===============================
233 X-Container-Object-Count The total number of objects in the container
234 X-Container-Bytes-Used The total number of bytes of all objects stored
235 X-Container-Block-Size The block size used by the storage backend
236 X-Container-Block-Hash The hash algorithm used for block identifiers in object hashmaps
237 X-Container-Until-Timestamp The last container modification date until the timestamp provided
238 X-Container-Object-Meta A list with all meta keys used by objects
239 X-Container-Meta-* Optional user defined metadata
240 Last-Modified The last container modification date (regardless of ``until``)
241 =========================== ===============================
243 The keys returned in ``X-Container-Object-Meta`` are all the unique strings after the ``X-Object-Meta-`` prefix.
245 ================ ===============================
246 Return Code Description
247 ================ ===============================
248 204 (No Content) The request succeeded
249 ================ ===============================
255 ==================== ===========================
256 Request Header Name Value
257 ==================== ===========================
258 If-Modified-Since Retrieve if container has changed since provided timestamp
259 If-Unmodified-Since Retrieve if container has not changed since provided timestamp
260 ==================== ===========================
264 ====================== ===================================
265 Request Parameter Name Value
266 ====================== ===================================
267 limit The amount of results requested (default is 10000)
268 marker Return containers with name lexicographically after marker
269 prefix Return objects starting with prefix
270 delimiter Return objects up to the delimiter (discussion follows)
271 path Assume ``prefix=path`` and ``delimiter=/``
272 format Optional extended reply type (can be ``json`` or ``xml``)
273 meta Return objects having the specified meta keys (can be a comma separated list)
274 until Optional timestamp
275 ====================== ===================================
277 The ``path`` parameter overrides ``prefix`` and ``delimiter``. When using ``path``, results will include objects ending in ``delimiter``.
279 The keys given with ``meta`` will be matched with the strings after the ``X-Object-Meta-`` prefix.
281 The reply is a list of object names. Container headers (as in a ``HEAD`` request) will also be included.
282 If a ``format=xml`` or ``format=json`` argument is given, extended information on the objects will be returned, serialized in the chosen format.
283 For each object, the information will include all object metadata (names will be in lower case and with hyphens replaced with underscores):
285 ========================== ======================================
287 ========================== ======================================
288 name The name of the object
289 hash The ETag of the object
290 bytes The size of the object
291 content_type The MIME content type of the object
292 content_encoding The encoding of the object (optional)
293 content-disposition The presentation style of the object (optional)
294 last_modified The last object modification date (regardless of version)
295 x_object_version The object's version identifier
296 x_object_version_timestamp The object's version timestamp
297 x_object_manifest Object parts prefix in ``<container>/<object>`` form (optional)
298 x_object_public Object is publicly accessible (optional) (**TBD**)
299 x_object_meta_* Optional user defined metadata
300 ========================== ======================================
302 Extended replies may also include virtual directory markers in separate sections of the ``json`` or ``xml`` results.
303 Virtual directory markers are only included when ``delimiter`` is explicitly set. They correspond to the substrings up to and including the first occurrence of the delimiter.
304 In JSON results they appear as dictionaries with only a ``"subdir"`` key. In XML results they appear interleaved with ``<object>`` tags as ``<subdir name="..." />``.
305 In case there is an object with the same name as a virtual directory marker, the object will be returned.
307 For examples of object details returned in JSON/XML formats refer to the OOS API documentation.
309 =========================== ===============================
310 Return Code Description
311 =========================== ===============================
312 200 (OK) The request succeeded
313 204 (No Content) The account has no containers (only for non-extended replies)
314 304 (Not Modified) The container has not been modified
315 412 (Precondition Failed) The condition set can not be satisfied
316 =========================== ===============================
318 Will use a ``200`` return code if the reply is of type json/xml.
324 ==================== ================================
325 Request Header Name Value
326 ==================== ================================
327 X-Container-Meta-* Optional user defined metadata
328 ==================== ================================
330 No reply content/headers.
332 ================ ===============================
333 Return Code Description
334 ================ ===============================
335 201 (Created) The container has been created
336 202 (Accepted) The request has been accepted
337 ================ ===============================
343 ==================== ================================
344 Request Header Name Value
345 ==================== ================================
346 X-Container-Meta-* Optional user defined metadata
347 ==================== ================================
349 No reply content/headers.
351 The update operation will overwrite all user defined metadata.
353 ================ ===============================
354 Return Code Description
355 ================ ===============================
356 202 (Accepted) The request has been accepted
357 ================ ===============================
363 No request parameters/headers.
365 No reply content/headers.
367 ================ ===============================
368 Return Code Description
369 ================ ===============================
370 204 (No Content) The request succeeded
371 409 (Conflict) The container is not empty
372 ================ ===============================
380 ========= =================================
381 Operation Description
382 ========= =================================
383 HEAD Retrieve object metadata
385 PUT Write object data or copy/move object
388 POST Update object metadata/data
390 ========= =================================
396 ====================== ===================================
397 Request Parameter Name Value
398 ====================== ===================================
399 version Optional version identifier
400 ====================== ===================================
404 ========================== ===============================
405 Reply Header Name Value
406 ========================== ===============================
407 ETag The ETag of the object
408 Content-Length The size of the object
409 Content-Type The MIME content type of the object
410 Last-Modified The last object modification date (regardless of version)
411 Content-Encoding The encoding of the object (optional)
412 Content-Disposition The presentation style of the object (optional)
413 X-Object-Version The object's version identifier
414 X-Object-Version-Timestamp The object's version timestamp
415 X-Object-Manifest Object parts prefix in ``<container>/<object>`` form (optional)
416 X-Object-Public Object is publicly accessible (optional) (**TBD**)
417 X-Object-Meta-* Optional user defined metadata
418 ========================== ===============================
422 ================ ===============================
423 Return Code Description
424 ================ ===============================
425 200 (No Content) The request succeeded
426 ================ ===============================
432 ==================== ================================
433 Request Header Name Value
434 ==================== ================================
435 Range Optional range of data to retrieve
436 If-Match Retrieve if ETags match
437 If-None-Match Retrieve if ETags don't match
438 If-Modified-Since Retrieve if object has changed since provided timestamp
439 If-Unmodified-Since Retrieve if object has not changed since provided timestamp
440 ==================== ================================
444 ====================== ===================================
445 Request Parameter Name Value
446 ====================== ===================================
447 format Optional extended reply type (can be ``json`` or ``xml``)
448 version Optional version identifier or ``list`` (specify a format if requesting a list)
449 ====================== ===================================
451 The reply is the object's data (or part of it), except if a hashmap is requested with the ``format`` parameter, or a version list with ``version=list`` (in which case an extended reply format must be specified). Object headers (as in a ``HEAD`` request) are always included.
453 Hashmaps expose the underlying storage format of the object. Note that each hash is computed after trimming trailing null bytes of the corresponding block.
455 Example ``format=json`` reply:
459 {"block_hash": "sha1", "hashes": ["7295c41da03d7f916440b98e32c4a2a39351546c", ...], "block_size": 131072, "bytes": 242}
461 Example ``format=xml`` reply:
465 <?xml version="1.0" encoding="UTF-8"?>
466 <object name="file" bytes="24223726" block_size="131072" block_hash="sha1">
467 <hash>7295c41da03d7f916440b98e32c4a2a39351546c</hash>
471 Version lists include the version identifier and timestamp for each available object version. Version identifiers are integers, with the only requirement that newer versions have a larger identifier than previous ones.
473 Example ``format=json`` reply:
477 {"versions": [[23, 1307700892], [28, 1307700898], ...]}
479 Example ``format=xml`` reply:
483 <?xml version="1.0" encoding="UTF-8"?>
485 <version timestamp="1307700892">23</version>
486 <version timestamp="1307700898">28</version>
487 <version timestamp="...">...</version>
490 The ``Range`` header may include multiple ranges, as outlined in RFC2616. Then the ``Content-Type`` of the reply will be ``multipart/byteranges`` and each part will include a ``Content-Range`` header.
492 ========================== ===============================
493 Reply Header Name Value
494 ========================== ===============================
495 ETag The ETag of the object
496 Content-Length The size of the data returned
497 Content-Type The MIME content type of the object
498 Content-Range The range of data included (only on a single range request)
499 Last-Modified The last object modification date (regardless of version)
500 Content-Encoding The encoding of the object (optional)
501 Content-Disposition The presentation style of the object (optional)
502 X-Object-Version The object's version identifier
503 X-Object-Version-Timestamp The object's version timestamp
504 X-Object-Manifest Object parts prefix in ``<container>/<object>`` form (optional)
505 X-Object-Public Object is publicly accessible (optional) (**TBD**)
506 X-Object-Meta-* Optional user defined metadata
507 ========================== ===============================
511 =========================== ==============================
512 Return Code Description
513 =========================== ==============================
514 200 (OK) The request succeeded
515 206 (Partial Content) The range request succeeded
516 304 (Not Modified) The object has not been modified
517 412 (Precondition Failed) The condition set can not be satisfied
518 416 (Range Not Satisfiable) The requested range is out of limits
519 =========================== ==============================
525 ==================== ================================
526 Request Header Name Value
527 ==================== ================================
528 ETag The MD5 hash of the object (optional to check written data)
529 Content-Length The size of the data written
530 Content-Type The MIME content type of the object
531 Transfer-Encoding Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored)
532 X-Copy-From The source path in the form ``/<container>/<object>``
533 X-Move-From The source path in the form ``/<container>/<object>``
534 X-Source-Version The source version to copy from
535 Content-Encoding The encoding of the object (optional)
536 Content-Disposition The presentation style of the object (optional)
537 X-Object-Manifest Object parts prefix in ``<container>/<object>`` form (optional)
538 X-Object-Public Object is publicly accessible (optional) (**TBD**)
539 X-Object-Meta-* Optional user defined metadata
540 ==================== ================================
544 ========================== ===============================
545 Reply Header Name Value
546 ========================== ===============================
547 ETag The MD5 hash of the object (on create)
548 ========================== ===============================
552 =========================== ==============================
553 Return Code Description
554 =========================== ==============================
555 201 (Created) The object has been created
556 411 (Length Required) Missing ``Content-Length`` or ``Content-Type`` in the request
557 422 (Unprocessable Entity) The MD5 checksum of the data written to the storage system does not match the (optionally) supplied ETag value
558 =========================== ==============================
564 ==================== ================================
565 Request Header Name Value
566 ==================== ================================
567 Destination The destination path in the form ``/<container>/<object>``
568 Content-Type The MIME content type of the object (optional)
569 Content-Encoding The encoding of the object (optional)
570 Content-Disposition The presentation style of the object (optional)
571 X-Source-Version The source version to copy from
572 X-Object-Manifest Object parts prefix in ``<container>/<object>`` form (optional)
573 X-Object-Public Object is publicly accessible (optional) (**TBD**)
574 X-Object-Meta-* Optional user defined metadata
575 ==================== ================================
577 No reply content/headers.
579 =========================== ==============================
580 Return Code Description
581 =========================== ==============================
582 201 (Created) The object has been created
583 =========================== ==============================
589 Same as ``COPY``, without the ``X-Source-Version`` request header. The ``MOVE`` operation is always applied on the latest version.
595 ==================== ================================
596 Request Header Name Value
597 ==================== ================================
598 Content-Length The size of the data written (optional, to update)
599 Content-Type The MIME content type of the object (optional, to update)
600 Content-Range The range of data supplied (optional, to update)
601 Transfer-Encoding Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored)
602 Content-Encoding The encoding of the object (optional)
603 Content-Disposition The presentation style of the object (optional)
604 X-Object-Manifest Object parts prefix in ``<container>/<object>`` form (optional)
605 X-Object-Public Object is publicly accessible (optional) (**TBD**)
606 X-Object-Meta-* Optional user defined metadata
607 ==================== ================================
609 The ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``, ``X-Object-Public`` (**TBD**) and ``X-Object-Meta-*`` headers are considered to be user defined metadata. The update operation will overwrite all previous values and remove any keys not supplied.
613 * Supply ``Content-Length`` (except if using chunked transfers), ``Content-Type`` and ``Content-Range`` headers.
614 * Set ``Content-Type`` to ``application/octet-stream``.
615 * Set ``Content-Range`` as specified in RFC2616, with the following differences:
617 * Client software MAY omit ``last-byte-pos`` of if the length of the range being transferred is unknown or difficult to determine.
618 * Client software SHOULD not specify the ``instance-length`` (use a ``*``), unless there is a reason for performing a size check at the server.
619 * If ``Content-Range`` used has a ``byte-range-resp-spec = *``, data supplied will be appended to the object.
621 A data update will trigger an ETag change. The new ETag will not correspond to the object's MD5 sum (**TBD**) and will be included in reply headers.
623 No reply content. No reply headers if only metadata is updated.
625 ========================== ===============================
626 Reply Header Name Value
627 ========================== ===============================
628 ETag The new ETag of the object (data updated)
629 ========================== ===============================
633 =========================== ==============================
634 Return Code Description
635 =========================== ==============================
636 202 (Accepted) The request has been accepted (not a data update)
637 204 (No Content) The request succeeded (data updated)
638 411 (Length Required) Missing ``Content-Length`` in the request
639 416 (Range Not Satisfiable) The supplied range is out of limits or invalid size
640 =========================== ==============================
646 No request parameters/headers.
648 No reply content/headers.
650 =========================== ==============================
651 Return Code Description
652 =========================== ==============================
653 204 (No Content) The request succeeded
654 =========================== ==============================
659 Objects that are marked as public, via the ``X-Object-Public`` meta (**TBD**), are also available at the corresponding URI ``https://hostname/public/<account>/<container>/<object>`` for ``HEAD`` or ``GET``. Requests for public objects do not need to include an ``X-Auth-Token``. Pithos will ignore request parameters and only include the following headers in the reply (all ``X-Object-*`` meta is hidden).
661 ========================== ===============================
662 Reply Header Name Value
663 ========================== ===============================
664 ETag The ETag of the object
665 Content-Length The size of the data returned
666 Content-Type The MIME content type of the object
667 Content-Range The range of data included (only on a single range request)
668 Last-Modified The last object modification date (regardless of version)
669 Content-Encoding The encoding of the object (optional)
670 Content-Disposition The presentation style of the object (optional)
671 ========================== ===============================
676 List of differences from the OOS API:
678 * Support for ``X-Account-Meta-*`` style headers at the account level. Use ``POST`` to update.
679 * Support for ``X-Container-Meta-*`` style headers at the account level. Can be set when creating via ``PUT``. Use ``POST`` to update.
680 * Header ``X-Container-Object-Meta`` at the container level and parameter ``meta`` in container listings.
681 * Headers ``X-Container-Block-*`` at the container level, exposing the underlying storage characteristics.
682 * All metadata replies, at all levels, include latest modification information.
683 * At all levels, a ``GET`` request may use ``If-Modified-Since`` and ``If-Unmodified-Since`` headers.
684 * Container/object lists include all associated metadata if the reply is of type json/xml. Some names are kept to their OOS API equivalents for compatibility.
685 * Object metadata allowed, in addition to ``X-Object-Meta-*``: ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``, ``X-Object-Public`` (**TBD**). These are all replaced with every update operation.
686 * Multi-range object GET support as outlined in RFC2616.
687 * Object hashmap retrieval through GET and the ``format`` parameter.
688 * Partial object updates through POST, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers.
689 * Object ``MOVE`` support.
690 * Time-variant account/container listings via the ``until`` parameter.
691 * Object versions - parameter ``version`` in HEAD/GET (list versions with GET), ``X-Object-Version-*`` meta in replies, ``X-Source-Version`` in PUT/COPY.
692 * Publicly accessible objects via ``https://hostname/public``. Control with ``X-Object-Public`` (**TBD**).
693 * Large object support with ``X-Object-Manifest``.
695 Clarifications/suggestions:
697 * Authentication is done by another system. The token is used in the same way, but it is obtained differently. The top level ``GET`` request is kept compatible with the OOS API and allows for guest/testing operations.
698 * Some processing is done in the variable part of all ``X-*-Meta-*`` headers. If it includes underscores, they will be converted to dashes and the first letter of all intra-dash strings will be capitalized.
699 * A ``GET`` reply for a level will include all headers of the corresponding ``HEAD`` request.
700 * To avoid conflicts between objects and virtual directory markers in container listings, it is recommended that object names do not end with the delimiter used.
701 * The ``Accept`` header may be used in requests instead of the ``format`` parameter to specify the desired reply format. The parameter overrides the header.
702 * Container/object lists use a ``200`` return code if the reply is of type json/xml. The reply will include an empty json/xml.
703 * In headers, dates are formatted according to RFC 1123. In extended information listings, dates are formatted according to ISO 8601.
704 * The ``Last-Modified`` header value always reflects the actual latest change timestamp, regardless of time control parameters and version requests. Time precondition checks with ``If-Modified-Since`` and ``If-Unmodified-Since`` headers are applied to this value.
705 * A ``HEAD`` or ``GET`` for an ``X-Object-Manifest`` object, will include modified ``Content-Length`` and ``ETag`` headers, according to the characteristics of the objects under the specified prefix. The ``Etag`` will be the MD5 hash of the corresponding ETags concatenated. In extended container listings there is no metadata processing.
713 Hopefully this API will allow for a multitude of client implementations, each supporting a different device or operating system. All clients will be able to manipulate containers and objects - even software only designed for OOS API compatibility. But a Pithos interface should not be only about showing containers and folders. There are some extra user interface elements and functionalities that should be common to all implementations.
715 Upon entrance to the service, a user is presented with the following elements - which can be represented as folders or with other related icons:
717 * The ``home`` element, which is used as the default entry point to the user's "files". Objects under ``home`` are represented in the usual hierarchical organization of folders and files.
718 * The ``trash`` element, which contains files that have been marked for deletion, but can still be recovered.
719 * The ``shared`` element, which contains all objects shared by the user to other users of the system.
720 * The ``others`` element, which contains all objects that other users share with the user.
721 * The ``tags`` element, which lists the names of tags the user has defined. This can be an entry point to list all files that have been assigned a specific tag or manage tags in general (remove a tag completely, rename a tag etc.).
722 * The ``groups`` element, which contains the names of groups the user has defined. Each group consists of a user list. Group creation, deletion, and manipulation is carried out by actions originating here.
724 Objects in Pithos can be:
726 * Assigned custom tags.
727 * Moved to trash and then deleted.
728 * Shared with specific permissions.
729 * Made public (shared with non-Pithos users).
730 * Restored from previous versions.
732 Some of these functions are performed by the client software and some by the Pithos server. Client-driven functionality is based on specific metadata that should be handled equally across implementations. These metadata names are discussed in the next chapter.
734 Conventions and Metadata Specification
735 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
737 Pithos clients should use the ``pithos`` container for all Pithos objects. Object names use the ``/`` delimiter to impose a hierarchy of folders and files.
739 At the object level, tags are implemented by managing metadata keys. The client software should allow the user to use any string as a tag (except ``trash``) and then set the corresponding ``X-Object-Meta-<tag>`` key at the server. The API extensions provided, allow for listing all tags in a container and filtering object listings based on one or more tags. The tag list is sufficient for implementing the ``tags`` element, either as a special, virtual folder (as done in the first version of Pithos), or as an application menu.
741 To manage the deletion of files use the same API and the ``X-Object-Meta-Trash`` key. The string ``trash`` can not be used as a tag. The ``trash`` element should be presented as a folder, although with no hierarchy.
743 The metadata specification is summarized in the following table.
745 =========================== ==============================
747 =========================== ==============================
748 X-Object-Meta-Trash Set to ``true`` if the object has been moved to the trash
749 X-Object-Meta-* Use for other tags that apply to the object
750 =========================== ==============================
752 Recommended Practices and Examples
753 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
755 Assuming an authentication token is obtained (**TBD**), the following high-level operations are available - shown with ``curl``:
757 * Get account information ::
760 -H "X-Auth-Token: 0000" \
761 https://pithos.dev.grnet.gr/v1/user
763 * List available containers ::
766 -H "X-Auth-Token: 0000" \
767 https://pithos.dev.grnet.gr/v1/user
769 * Get container information ::
772 -H "X-Auth-Token: 0000" \
773 https://pithos.dev.grnet.gr/v1/user/pithos
775 * Add a new container ::
778 -H "X-Auth-Token: 0000" \
779 https://pithos.dev.grnet.gr/v1/user/test
781 * Delete a container ::
783 curl -X DELETE -D - \
784 -H "X-Auth-Token: 0000" \
785 https://pithos.dev.grnet.gr/v1/user/test
787 * List objects in a container ::
790 -H "X-Auth-Token: 0000" \
791 https://pithos.dev.grnet.gr/v1/user/pithos
793 * List objects in a container (extended reply) ::
796 -H "X-Auth-Token: 0000" \
797 https://pithos.dev.grnet.gr/v1/user/pithos?format=json
799 * List metadata keys used by objects in a container
801 Will be in the ``X-Container-Object-Meta`` reply header, included in container information or object list (``HEAD`` or ``GET``).
803 * List objects in a container having a specific meta defined ::
806 -H "X-Auth-Token: 0000" \
807 https://pithos.dev.grnet.gr/v1/user/pithos?meta=trash
809 This is the recommended way of tagging/retrieving objects in trash.
811 * Retrieve an object ::
814 -H "X-Auth-Token: 0000" \
815 https://pithos.dev.grnet.gr/v1/user/pithos/README.txt
817 * Retrieve an object (specific ranges of data) ::
820 -H "X-Auth-Token: 0000" \
821 -H "Range: bytes=0-9" \
822 https://pithos.dev.grnet.gr/v1/user/pithos/README.txt
824 This will return the first 10 bytes. To get the first 10, bytes 30-39 and the last 100 use ``Range: bytes=0-9,30-39,-100``.
826 * Add a new object (folder type) (**TBD**) ::
829 -H "X-Auth-Token: 0000" \
830 -H "Content-Type: application/folder" \
831 https://pithos.dev.grnet.gr/v1/user/pithos/folder
833 * Add a new object ::
836 -H "X-Auth-Token: 0000" \
837 -H "Content-Type: text/plain" \
839 https://pithos.dev.grnet.gr/v1/user/pithos/folder/EXAMPLE.txt
841 * Update an object ::
844 -H "X-Auth-Token: 0000" \
845 -H "Content-Length: 10" \
846 -H "Content-Type: application/octet-stream" \
847 -H "Content-Range: bytes 10-19/*" \
849 https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
851 This will update bytes 10-19 with the data specified.
853 * Update an object (append) ::
856 -H "X-Auth-Token: 0000" \
857 -H "Content-Length: 10" \
858 -H "Content-Type: application/octet-stream" \
859 -H "Content-Range: bytes */*" \
861 https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
863 * Add object metadata ::
866 -H "X-Auth-Token: 0000" \
867 -H "X-Object-Meta-First: first_meta_value" \
868 -H "X-Object-Meta-Second: second_meta_value" \
869 https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
871 * Delete object metadata ::
874 -H "X-Auth-Token: 0000" \
875 -H "X-Object-Meta-First: first_meta_value" \
876 https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
878 Metadata can only be "set". To delete ``X-Object-Meta-Second``, reset all metadata.
880 * Delete an object ::
882 curl -X DELETE -D - \
883 -H "X-Auth-Token: 0000" \
884 https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
projects / pithos / blob