Statistics
| Branch: | Tag: | Revision:

root / docs / pithos-api-guide.rst @ dc068945

History | View | Annotate | Download (67.9 kB)

1
Pithos+ API
2
===========
3

    
4
Introduction
5
------------
6

    
7
Pithos is a storage service implemented by GRNET (http://www.grnet.gr). Data is stored as objects, organized in containers, belonging to an account. This hierarchy of storage layers has been inspired by the OpenStack Object Storage (OOS) API and similar CloudFiles API by Rackspace. The Pithos API follows the OOS API as closely as possible. One of the design requirements has been to be able to use Pithos with clients built for the OOS, without changes.
8

    
9
However, to be able to take full advantage of the Pithos infrastructure, client software should be aware of the extensions that differentiate Pithos from OOS. Pithos objects can be updated, or appended to. Pithos will store sharing permissions per object and enforce corresponding authorization policies. Automatic version management, allows taking account and container listings back in time, as well as reading previous instances of objects.
10

    
11
The storage backend of Pithos is block oriented, permitting efficient, deduplicated data placement. The block structure of objects is exposed at the API layer, in order to encourage external software to implement advanced data management operations.
12

    
13
This document's goals are:
14

    
15
* Define the Pithos ReST API that allows the storage and retrieval of data and metadata via HTTP calls
16
* Specify metadata semantics and user interface guidelines for a common experience across client software implementations
17

    
18
The present document is meant to be read alongside the OOS API documentation. Thus, it is suggested that the reader is familiar with associated technologies, the OOS API as well as the first version of the Pithos API. This document refers to the second version of Pithos. Information on the first version of the storage API can be found at http://code.google.com/p/gss.
19

    
20
Whatever marked as to be determined (**TBD**), should not be considered by implementors.
21

    
22
More info about Pithos can be found here: https://code.grnet.gr/projects/pithos
23

    
24
Document Revisions
25
^^^^^^^^^^^^^^^^^^
26

    
27
=========================  ================================
28
Revision                   Description
29
=========================  ================================
30
0.13 (Jan 21, 2013)        Proxy identity management services
31
\                          Uuid to displayname translation
32
0.9 (Feb 17, 2012)         Change permissions model.
33
0.10 (Jul 18, 2012)        Support for bulk COPY/MOVE/DELETE
34
\                          Optionally include public objects in listings.
35
0.9 (Feb 17, 2012)         Change permissions model.
36
\                          Do not include user-defined metadata in account/container/object listings.
37
0.8 (Jan 24, 2012)         Update allowed versioning values.
38
\                          Change policy/meta formatting in JSON/XML replies.
39
\                          Document that all non-ASCII characters in headers should be URL-encoded.
40
\                          Support metadata-based queries when listing objects at the container level.
41
\                          Note Content-Type issue when using the internal django web server.
42
\                          Add object UUID field.
43
\                          Always reply with the MD5 in the ETag.
44
\                          Note that ``/login`` will only work if an external authentication system is defined.
45
\                          Include option to ignore Content-Type on ``COPY``/``MOVE``.
46
\                          Use format parameter for conflict (409) and uploaded hash list (container level) replies.
47
0.7 (Nov 21, 2011)         Suggest upload/download methods using hashmaps.
48
\                          Propose syncing algorithm.
49
\                          Support cross-account object copy and move.
50
\                          Pass token as a request parameter when using ``POST`` via an HTML form.
51
\                          Optionally use source account to update object from another object.
52
\                          Use container ``POST`` to upload missing blocks of data.
53
\                          Report policy in account headers.
54
\                          Add insufficient quota reply.
55
\                          Use special meta to always report Merkle hash.
56
0.6 (Sept 13, 2011)        Reply with Merkle hash as the ETag when updating objects.
57
\                          Include version id in object replace/change replies.
58
\                          Change conflict (409) replies format to text.
59
\                          Tags should be migrated to a meta value.
60
\                          Container ``PUT`` updates metadata/policy.
61
\                          Report allowed actions in shared object replies.
62
\                          Provide ``https://hostname/login`` for Shibboleth authentication.
63
\                          Use ``hashmap`` parameter in object ``GET``/``PUT`` to use hashmaps.
64
0.5 (July 22, 2011)        Object update from another object's data.
65
\                          Support object truncate.
66
\                          Create object using a standard HTML form.
67
\                          Purge container/object history.
68
\                          List other accounts that share objects with a user.
69
\                          List shared containers/objects.
70
\                          Update implementation guidelines.
71
\                          Check preconditions when creating/updating objects.
72
0.4 (July 01, 2011)        Object permissions and account groups.
73
\                          Control versioning behavior and container quotas with container policy directives.
74
\                          Support updating/deleting individual metadata with ``POST``.
75
\                          Create object using hashmap.
76
0.3 (June 14, 2011)        Large object support with ``X-Object-Manifest``.
77
\                          Allow for publicly available objects via ``https://hostname/public``.
78
\                          Support time-variant account/container listings.
79
\                          Add source version when duplicating with ``PUT``/``COPY``.
80
\                          Request version in object ``HEAD``/``GET`` requests (list versions with ``GET``).
81
0.2 (May 31, 2011)         Add object meta listing and filtering in containers.
82
\                          Include underlying storage characteristics in container meta.
83
\                          Support for partial object updates through ``POST``.
84
\                          Expose object hashmaps through ``GET``.
85
\                          Support for multi-range object ``GET`` requests.
86
0.1 (May 17, 2011)         Initial release. Based on OpenStack Object Storage Developer Guide API v1 (Apr. 15, 2011).
87
=========================  ================================
88

    
89
Pithos Users and Authentication
90
-------------------------------
91

    
92
In Pithos, each user is uniquely identified by a token. All API requests require a token and each token is internally resolved to an account string. The API uses the account string to identify the user's own files, thus whether a request is local or cross-account.
93

    
94
Pithos does not keep a user database. For development and testing purposes, user identifiers and their corresponding tokens can be defined in the settings file. However, Pithos is designed with an external authentication service in mind. This service must handle the details of validating user credentials and communicate with Pithos via a middleware software component that, given a token, fills in the internal request account variable.
95

    
96
Client software using Pithos, if not already knowing a user's identifier and token, should forward to the ``/login`` URI. The Pithos server, depending on its configuration will redirect to the appropriate login page.
97

    
98
The login URI accepts the following parameters:
99

    
100
======================  =========================
101
Request Parameter Name  Value
102
======================  =========================
103
next                    The URI to redirect to when the process is finished
104
renew                   Force token renewal (no value parameter)
105
force                   Force logout current user (no value parameter)
106
======================  =========================
107

    
108
When done with logging in, the service's login URI should redirect to the URI provided with ``next``, adding ``user`` and ``token`` parameters, which contain the account and token fields respectively.
109

    
110
A user management service that implements a login URI according to these conventions is Astakos (https://code.grnet.gr/projects/astakos), by GRNET.
111

    
112
User feedback
113
-------------
114

    
115
Client software using Pithos, should forward to the ``/feedback`` URI. The Pithos service, depending on its configuration will delegate the request to the appropriate identity management URI.
116

    
117
========================= =========  ==================
118
Uri                       Method     Description
119
========================= =========  ==================
120
``/feedback``             POST       Send feedback
121
========================= =========  ==================
122

    
123
|
124

    
125
======================  =========================
126
Request Parameter Name  Value
127
======================  =========================
128
feedback_msg            Feedback message
129
feedback_data           Additional information about service client status
130
======================  =========================
131

    
132
|
133

    
134
====================  ===========================
135
Request Header Name   Value
136
====================  ===========================
137
X-Auth-Token          User authentication token
138
====================  ===========================
139

    
140
|
141

    
142
=========================== =====================
143
Return Code                 Description
144
=========================== =====================
145
200 (OK)                    The request succeeded
146
502 (Bad Gateway)           Send feedback failure
147
400 (Bad Request)           Method not allowed or invalid message data
148
401 (Unauthorized)          Missing or expired user token
149
500 (Internal Server Error) The request cannot be completed because of an internal error
150
=========================== =====================
151

    
152
User translation catalogs
153
-------------------------
154

    
155
Client software using Pithos, should forward to the ``/user_catalogs`` URI to get uuid to displayname translations and vice versa. The Pithos service, depending on its configuration will delegate the request to the appropriate identity management URI.
156

    
157
================================ =========  ==================
158
Uri                              Method     Description
159
================================ =========  ==================
160
``/user_catalogs``               POST       Get 2 catalogs containing uuid to displayname mapping and the opposite
161
================================ =========  ==================
162

    
163
|
164

    
165
====================  ===========================
166
Request Header Name   Value
167
====================  ===========================
168
X-Auth-Token          User authentication token
169
====================  ===========================
170

    
171
The request body is a json formatted dictionary containing a list with uuids and another list of displaynames to translate.
172

    
173
Example request content:
174

    
175
::
176

    
177
  {"displaynames": ["user1@example.com", "user2@example.com"],
178
   "uuids":["ff53baa9-c025-4d56-a6e3-963db0438830", "a9dc21d2-bcb2-4104-9a9e-402b7c70d6d8"]}
179

    
180
Example reply:
181

    
182
::
183

    
184
  {"displayname_catalog": {"user1@example.com": "a9dc21d2-bcb2-4104-9a9e-402b7c70d6d8",
185
                        "user2@example.com": "816351c7-7405-4f26-a968-6380cf47ba1f"},
186
  'uuid_catalog': {"a9dc21d2-bcb2-4104-9a9e-402b7c70d6d8": "user1@example.com",
187
                   "ff53baa9-c025-4d56-a6e3-963db0438830": "user2@example.com"}}
188

    
189

    
190
|
191

    
192
=========================== =====================
193
Return Code                 Description
194
=========================== =====================
195
200 (OK)                    The request succeeded
196
400 (Bad Request)           Method not allowed or request body is not json formatted
197
401 (Unauthorized)          Missing or expired or invalid user token
198
500 (Internal Server Error) The request cannot be completed because of an internal error
199
=========================== =====================
200

    
201
The Pithos API
202
--------------
203

    
204
The URI requests supported by the Pithos API follow one of the following forms:
205

    
206
* Top level: ``https://hostname/v1/``
207
* Account level: ``https://hostname/v1/<account>``
208
* Container level: ``https://hostname/v1/<account>/<container>``
209
* Object level: ``https://hostname/v1/<account>/<container>/<object>``
210

    
211
All requests must include an ``X-Auth-Token`` - as a header, or a parameter.
212

    
213
The allowable request operations and respective return codes per level are presented in the remainder of this chapter. Common to all requests are the following return codes.
214

    
215
==============================  ================================
216
Return Code                     Description
217
==============================  ================================
218
400 (Bad Request)               The request is invalid
219
401 (Unauthorized)              Missing or invalid token
220
403 (Forbidden)                 Request not allowed
221
404 (Not Found)                 The requested resource was not found
222
413 (Request Entity Too Large)  Insufficient quota to complete the request
223
503 (Service Unavailable)       The request cannot be completed because of an internal error
224
==============================  ================================
225

    
226
Top Level
227
^^^^^^^^^
228

    
229
List of operations:
230

    
231
=========  ==================
232
Operation  Description
233
=========  ==================
234
GET        Authentication (for compatibility with the OOS API) or list allowed accounts
235
=========  ==================
236

    
237
GET
238
"""
239

    
240
If the ``X-Auth-User`` and ``X-Auth-Key`` headers are given, a dummy ``X-Auth-Token`` and ``X-Storage-Url`` will be replied, which can be used as a guest token/namespace for testing Pithos.
241

    
242
================  =====================
243
Return Code       Description
244
================  =====================
245
204 (No Content)  The request succeeded
246
================  =====================
247

    
248
If an ``X-Auth-Token`` is already present, the operation will be interpreted as a request to list other accounts that share objects to the user.
249

    
250
======================  =========================
251
Request Parameter Name  Value
252
======================  =========================
253
limit                   The amount of results requested (default is 10000)
254
marker                  Return containers with name lexicographically after marker
255
format                  Optional extended reply type (can be ``json`` or ``xml``)
256
======================  =========================
257

    
258
The reply is a list of account names.
259
If a ``format=xml`` or ``format=json`` argument is given, extended information on the accounts will be returned, serialized in the chosen format.
260
For each account, the information will include the following (names will be in lower case and with hyphens replaced with underscores):
261

    
262
===========================  ============================
263
Name                         Description
264
===========================  ============================
265
name                         The name of the account
266
last_modified                The last account modification date (regardless of ``until``)
267
===========================  ============================
268

    
269
Example ``format=json`` reply:
270

    
271
::
272

    
273
  [{"name": "user", "last_modified": "2011-12-02T08:10:41.565891+00:00"}, ...]
274

    
275
Example ``format=xml`` reply:
276

    
277
::
278

    
279
  <?xml version="1.0" encoding="UTF-8"?>
280
  <accounts>
281
    <account>
282
      <name>user</name>
283
      <last_modified>2011-12-02T08:10:41.565891+00:00</last_modified>
284
    </account>
285
    <account>...</account>
286
  </accounts>
287

    
288
===========================  =====================
289
Return Code                  Description
290
===========================  =====================
291
200 (OK)                     The request succeeded
292
204 (No Content)             The user has no access to other accounts (only for non-extended replies)
293
===========================  =====================
294

    
295
Will use a ``200`` return code if the reply is of type JSON/XML.
296

    
297
Account Level
298
^^^^^^^^^^^^^
299

    
300
List of operations:
301

    
302
=========  ==================
303
Operation  Description
304
=========  ==================
305
HEAD       Retrieve account metadata
306
GET        List containers
307
POST       Update account metadata
308
=========  ==================
309

    
310
HEAD
311
""""
312

    
313
====================  ===========================
314
Request Header Name   Value
315
====================  ===========================
316
If-Modified-Since     Retrieve if account has changed since provided timestamp
317
If-Unmodified-Since   Retrieve if account has not changed since provided timestamp
318
====================  ===========================
319

    
320
|
321

    
322
======================  ===================================
323
Request Parameter Name  Value
324
======================  ===================================
325
until                   Optional timestamp
326
======================  ===================================
327

    
328
Cross-user requests are not allowed to use ``until`` and only include the account modification date in the reply.
329

    
330
==========================  =====================
331
Reply Header Name           Value
332
==========================  =====================
333
X-Account-Container-Count   The total number of containers
334
X-Account-Bytes-Used        The total number of bytes stored
335
X-Account-Until-Timestamp   The last account modification date until the timestamp provided
336
X-Account-Group-*           Optional user defined groups
337
X-Account-Policy-*          Account behavior and limits
338
X-Account-Meta-*            Optional user defined metadata
339
Last-Modified               The last account modification date (regardless of ``until``)
340
==========================  =====================
341

    
342
|
343

    
344
================  =====================
345
Return Code       Description
346
================  =====================
347
204 (No Content)  The request succeeded
348
================  =====================
349

    
350

    
351
GET
352
"""
353

    
354
====================  ===========================
355
Request Header Name   Value
356
====================  ===========================
357
If-Modified-Since     Retrieve if account has changed since provided timestamp
358
If-Unmodified-Since   Retrieve if account has not changed since provided timestamp
359
====================  ===========================
360

    
361
|
362

    
363
======================  =========================
364
Request Parameter Name  Value
365
======================  =========================
366
limit                   The amount of results requested (default is 10000)
367
marker                  Return containers with name lexicographically after marker
368
format                  Optional extended reply type (can be ``json`` or ``xml``)
369
shared                  Show only shared containers (no value parameter)
370
public                  Show only public containers (no value parameter)
371
until                   Optional timestamp
372
======================  =========================
373

    
374
The reply is a list of container names. Account headers (as in a ``HEAD`` request) will also be included.
375
Cross-user requests are not allowed to use ``until`` and only include the account/container modification dates in the reply.
376

    
377
If a ``format=xml`` or ``format=json`` argument is given, extended information on the containers will be returned, serialized in the chosen format.
378
For each container, the information will include all container metadata, except user-defined (names will be in lower case and with hyphens replaced with underscores):
379

    
380
===========================  ============================
381
Name                         Description
382
===========================  ============================
383
name                         The name of the container
384
count                        The number of objects inside the container
385
bytes                        The total size of the objects inside the container
386
last_modified                The last container modification date (regardless of ``until``)
387
x_container_until_timestamp  The last container modification date until the timestamp provided
388
x_container_policy           Container behavior and limits
389
===========================  ============================
390

    
391
Example ``format=json`` reply:
392

    
393
::
394

    
395
  [{"name": "pithos",
396
    "bytes": 62452,
397
    "count": 8374,
398
    "last_modified": "2011-12-02T08:10:41.565891+00:00",
399
    "x_container_policy": {"quota": "53687091200", "versioning": "auto"}}, ...]
400

    
401
Example ``format=xml`` reply:
402

    
403
::
404

    
405
  <?xml version="1.0" encoding="UTF-8"?>
406
  <account name="user">
407
    <container>
408
      <name>pithos</name>
409
      <bytes>62452</bytes>
410
      <count>8374</count>
411
      <last_modified>2011-12-02T08:10:41.565891+00:00</last_modified>
412
      <x_container_policy>
413
        <key>quota</key><value>53687091200</value>
414
        <key>versioning</key><value>auto</value>
415
      </x_container_policy>
416
    </container>
417
    <container>...</container>
418
  </account>
419

    
420
For more examples of container details returned in JSON/XML formats refer to the OOS API documentation. In addition to the OOS API, Pithos returns policy fields, grouped as key-value pairs.
421

    
422
===========================  =====================
423
Return Code                  Description
424
===========================  =====================
425
200 (OK)                     The request succeeded
426
204 (No Content)             The account has no containers (only for non-extended replies)
427
304 (Not Modified)           The account has not been modified
428
412 (Precondition Failed)    The condition set can not be satisfied
429
===========================  =====================
430

    
431
Will use a ``200`` return code if the reply is of type JSON/XML.
432

    
433

    
434
POST
435
""""
436

    
437
====================  ===========================
438
Request Header Name   Value
439
====================  ===========================
440
X-Account-Group-*     Optional user defined groups
441
X-Account-Meta-*      Optional user defined metadata
442
====================  ===========================
443

    
444
|
445

    
446
======================  ============================================
447
Request Parameter Name  Value
448
======================  ============================================
449
update                  Do not replace metadata/groups (no value parameter)
450
======================  ============================================
451

    
452
No reply content/headers.
453

    
454
The operation will overwrite all user defined metadata, except if ``update`` is defined.
455
To create a group, include an ``X-Account-Group-*`` header with the name in the key and a comma separated list of user names in the value. If no ``X-Account-Group-*`` header is present, no changes will be applied to groups. The ``update`` parameter also applies to groups. To delete a specific group, use ``update`` and an empty header value.
456

    
457
================  ===============================
458
Return Code       Description
459
================  ===============================
460
202 (Accepted)    The request has been accepted
461
================  ===============================
462

    
463

    
464
Container Level
465
^^^^^^^^^^^^^^^
466

    
467
List of operations:
468

    
469
=========  ============================
470
Operation  Description
471
=========  ============================
472
HEAD       Retrieve container metadata
473
GET        List objects
474
PUT        Create/update container
475
POST       Update container metadata
476
DELETE     Delete container
477
=========  ============================
478

    
479

    
480
HEAD
481
""""
482

    
483
====================  ===========================
484
Request Header Name   Value
485
====================  ===========================
486
If-Modified-Since     Retrieve if container has changed since provided timestamp
487
If-Unmodified-Since   Retrieve if container has not changed since provided timestamp
488
====================  ===========================
489

    
490
|
491

    
492
======================  ===================================
493
Request Parameter Name  Value
494
======================  ===================================
495
until                   Optional timestamp
496
======================  ===================================
497

    
498
Cross-user requests are not allowed to use ``until`` and only include the container modification date in the reply.
499

    
500
===========================  ===============================
501
Reply Header Name            Value
502
===========================  ===============================
503
X-Container-Object-Count     The total number of objects in the container
504
X-Container-Bytes-Used       The total number of bytes of all objects stored
505
X-Container-Block-Size       The block size used by the storage backend
506
X-Container-Block-Hash       The hash algorithm used for block identifiers in object hashmaps
507
X-Container-Until-Timestamp  The last container modification date until the timestamp provided
508
X-Container-Object-Meta      A list with all meta keys used by objects (**TBD**)
509
X-Container-Policy-*         Container behavior and limits
510
X-Container-Meta-*           Optional user defined metadata
511
Last-Modified                The last container modification date (regardless of ``until``)
512
===========================  ===============================
513

    
514
The keys returned in ``X-Container-Object-Meta`` are all the unique strings after the ``X-Object-Meta-`` prefix, formatted as a comma-separated list. See container ``PUT`` for a reference of policy directives. (**TBD**)
515

    
516
================  ===============================
517
Return Code       Description
518
================  ===============================
519
204 (No Content)  The request succeeded
520
================  ===============================
521

    
522

    
523
GET
524
"""
525

    
526
====================  ===========================
527
Request Header Name   Value
528
====================  ===========================
529
If-Modified-Since     Retrieve if container has changed since provided timestamp
530
If-Unmodified-Since   Retrieve if container has not changed since provided timestamp
531
====================  ===========================
532

    
533
|
534

    
535
======================  ===================================
536
Request Parameter Name  Value
537
======================  ===================================
538
limit                   The amount of results requested (default is 10000)
539
marker                  Return containers with name lexicographically after marker
540
prefix                  Return objects starting with prefix
541
delimiter               Return objects up to the delimiter (discussion follows)
542
path                    Assume ``prefix=path`` and ``delimiter=/``
543
format                  Optional extended reply type (can be ``json`` or ``xml``)
544
meta                    Return objects that satisfy the key queries in the specified comma separated list (use ``<key>``, ``!<key>`` for existence queries, ``<key><op><value>`` for value queries, where ``<op>`` can be one of ``=``, ``!=``, ``<=``, ``>=``, ``<``, ``>``)
545
shared                  Show only shared objects (no value parameter)
546
public                  Show only public containers (no value parameter)
547
until                   Optional timestamp
548
======================  ===================================
549

    
550
The ``path`` parameter overrides ``prefix`` and ``delimiter``. When using ``path``, results will include objects ending in ``delimiter``.
551

    
552
The keys given with ``meta`` will be matched with the strings after the ``X-Object-Meta-`` prefix.
553

    
554
The reply is a list of object names. Container headers (as in a ``HEAD`` request) will also be included.
555
Cross-user requests are not allowed to use ``until`` and include the following limited set of headers in the reply:
556

    
557
===========================  ===============================
558
Reply Header Name            Value
559
===========================  ===============================
560
X-Container-Block-Size       The block size used by the storage backend
561
X-Container-Block-Hash       The hash algorithm used for block identifiers in object hashmaps
562
X-Container-Object-Meta      A list with all meta keys used by allowed objects (**TBD**)
563
Last-Modified                The last container modification date
564
===========================  ===============================
565

    
566
If a ``format=xml`` or ``format=json`` argument is given, extended information on the objects will be returned, serialized in the chosen format.
567
For each object, the information will include all object metadata, except user-defined (names will be in lower case and with hyphens replaced with underscores). User-defined metadata includes ``X-Object-Meta-*``, ``X-Object-Manifest``, ``Content-Disposition`` and ``Content-Encoding`` keys. Also, sharing directives will only be included with the actual shared objects (inherited permissions are not calculated):
568

    
569
==========================  ======================================
570
Name                        Description
571
==========================  ======================================
572
name                        The name of the object
573
hash                        The ETag of the object
574
bytes                       The size of the object
575
content_type                The MIME content type of the object
576
last_modified               The last object modification date (regardless of version)
577
x_object_hash               The Merkle hash
578
x_object_uuid               The object's UUID
579
x_object_version            The object's version identifier
580
x_object_version_timestamp  The object's version timestamp
581
x_object_modified_by        The user that committed the object's version
582
x_object_sharing            Object permissions (optional)
583
x_object_allowed_to         Allowed actions on object (optional)
584
x_object_public             Object's publicly accessible URI (optional)
585
==========================  ======================================
586

    
587
Sharing metadata and last modification timestamp will only be returned if there is no ``until`` parameter defined.
588

    
589
Extended replies may also include virtual directory markers in separate sections of the ``json`` or ``xml`` results.
590
Virtual directory markers are only included when ``delimiter`` is explicitly set. They correspond to the substrings up to and including the first occurrence of the delimiter.
591
In JSON results they appear as dictionaries with only a ``subdir`` key. In XML results they appear interleaved with ``<object>`` tags as ``<subdir name="..." />``.
592
In case there is an object with the same name as a virtual directory marker, the object will be returned.
593

    
594
Example ``format=json`` reply:
595

    
596
::
597

    
598
  [{"name": "object",
599
    "bytes": 0,
600
    "hash": "d41d8cd98f00b204e9800998ecf8427e",
601
    "content_type": "application/octet-stream",
602
    "last_modified": "2011-12-02T08:10:41.565891+00:00",
603
    "x_object_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
604
    "x_object_uuid": "8ed9af1b-c948-4bb6-82b0-48344f5c822c",
605
    "x_object_version": 98,
606
    "x_object_version_timestamp": "1322813441.565891",
607
    "x_object_modified_by": "user"}, ...]
608

    
609
Example ``format=xml`` reply:
610

    
611
::
612

    
613
  <?xml version="1.0" encoding="UTF-8"?>
614
  <container name="pithos">
615
    <object>
616
      <name>object</name>
617
      <bytes>0</bytes>
618
      <hash>d41d8cd98f00b204e9800998ecf8427e</hash>
619
      <content_type>application/octet-stream</content_type>
620
      <last_modified>2011-12-02T08:10:41.565891+00:00</last_modified>
621
      <x_object_hash>e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</x_object_hash>
622
      <x_object_uuid>8ed9af1b-c948-4bb6-82b0-48344f5c822c</x_object_uuid>
623
      <x_object_version>98</x_object_version>
624
      <x_object_version_timestamp>1322813441.565891</x_object_version_timestamp>
625
      <x_object_modified_by>chazapis</x_object_modified_by>
626
    </object>
627
    <object>...</object>
628
  </container>
629

    
630
For more examples of container details returned in JSON/XML formats refer to the OOS API documentation. In addition to the OOS API, Pithos returns more fields that should help with synchronization.
631

    
632
===========================  ===============================
633
Return Code                  Description
634
===========================  ===============================
635
200 (OK)                     The request succeeded
636
204 (No Content)             The account has no containers (only for non-extended replies)
637
304 (Not Modified)           The container has not been modified
638
412 (Precondition Failed)    The condition set can not be satisfied
639
===========================  ===============================
640

    
641
Will use a ``200`` return code if the reply is of type JSON/XML.
642

    
643

    
644
PUT
645
"""
646

    
647
====================  ================================
648
Request Header Name   Value
649
====================  ================================
650
X-Container-Policy-*  Container behavior and limits
651
X-Container-Meta-*    Optional user defined metadata
652
====================  ================================
653
 
654
No reply content/headers.
655

    
656
If no policy is defined, the container will be created with the default values.
657
Available policy directives:
658

    
659
* ``versioning``: Set to ``auto`` or ``none`` (default is ``auto``)
660
* ``quota``: Size limit in KB (default is ``0`` - unlimited)
661

    
662
If the container already exists, the operation is equal to a ``POST`` with ``update`` defined.
663

    
664
================  ===============================
665
Return Code       Description
666
================  ===============================
667
201 (Created)     The container has been created
668
202 (Accepted)    The request has been accepted
669
================  ===============================
670

    
671

    
672
POST
673
""""
674

    
675
====================  ================================
676
Request Header Name   Value
677
====================  ================================
678
Content-Length        The size of the supplied data (optional, to upload)
679
Content-Type          The MIME content type of the supplied data (optional, to upload)
680
Transfer-Encoding     Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored)
681
X-Container-Policy-*  Container behavior and limits
682
X-Container-Meta-*    Optional user defined metadata
683
====================  ================================
684

    
685
|
686

    
687
======================  ============================================
688
Request Parameter Name  Value
689
======================  ============================================
690
format                  Optional hash list reply type (can be ``json`` or ``xml``)
691
update                  Do not replace metadata/policy (no value parameter)
692
======================  ============================================
693

    
694
No reply content/headers, except when uploading data, where the reply consists of a list of hashes for the blocks received (in the format specified).
695

    
696
The operation will overwrite all user defined metadata, except if ``update`` is defined.
697
To change policy, include an ``X-Container-Policy-*`` header with the name in the key. If no ``X-Container-Policy-*`` header is present, no changes will be applied to policy. The ``update`` parameter also applies to policy - deleted values will revert to defaults. To delete/revert a specific policy directive, use ``update`` and an empty header value. See container ``PUT`` for a reference of policy directives.
698

    
699
To upload blocks of data to the container, set ``Content-Type`` to ``application/octet-stream`` and ``Content-Length`` to a valid value (except if using ``chunked`` as the ``Transfer-Encoding``).
700

    
701
================  ===============================
702
Return Code       Description
703
================  ===============================
704
202 (Accepted)    The request has been accepted
705
================  ===============================
706

    
707

    
708
DELETE
709
""""""
710

    
711
======================  ===================================
712
Request Parameter Name  Value
713
======================  ===================================
714
until                   Optional timestamp
715
delimiter               Optional delete objects starting with container name and delimiter
716
======================  ===================================
717

    
718
If ``until`` is defined, the container is "purged" up to that time (the history of all objects up to then is deleted). If also ``delimiter`` is defined, purge is applied only on the container.
719

    
720
No reply content/headers.
721

    
722
================  ===============================
723
Return Code       Description
724
================  ===============================
725
204 (No Content)  The request succeeded
726
409 (Conflict)    The container is not empty
727
================  ===============================
728

    
729

    
730
Object Level
731
^^^^^^^^^^^^
732

    
733
List of operations:
734

    
735
=========  =================================
736
Operation  Description
737
=========  =================================
738
HEAD       Retrieve object metadata
739
GET        Read object data
740
PUT        Write object data or copy/move object
741
COPY       Copy object
742
MOVE       Move object
743
POST       Update object metadata/data
744
DELETE     Delete object
745
=========  =================================
746

    
747

    
748
HEAD
749
""""
750

    
751
====================  ================================
752
Request Header Name   Value
753
====================  ================================
754
If-Match              Retrieve if ETags match
755
If-None-Match         Retrieve if ETags don't match
756
If-Modified-Since     Retrieve if object has changed since provided timestamp
757
If-Unmodified-Since   Retrieve if object has not changed since provided timestamp
758
====================  ================================
759

    
760
|
761

    
762
======================  ===================================
763
Request Parameter Name  Value
764
======================  ===================================
765
version                 Optional version identifier
766
======================  ===================================
767

    
768
|
769

    
770
==========================  ===============================
771
Reply Header Name           Value
772
==========================  ===============================
773
ETag                        The ETag of the object
774
Content-Length              The size of the object
775
Content-Type                The MIME content type of the object
776
Last-Modified               The last object modification date (regardless of version)
777
Content-Encoding            The encoding of the object (optional)
778
Content-Disposition         The presentation style of the object (optional)
779
X-Object-Hash               The Merkle hash
780
X-Object-UUID               The object's UUID
781
X-Object-Version            The object's version identifier
782
X-Object-Version-Timestamp  The object's version timestamp
783
X-Object-Modified-By        The user that comitted the object's version
784
X-Object-Manifest           Object parts prefix in ``<container>/<object>`` form (optional)
785
X-Object-Sharing            Object permissions (optional)
786
X-Object-Shared-By          Object inheriting permissions (optional)
787
X-Object-Allowed-To         Allowed actions on object (optional)
788
X-Object-Public             Object's publicly accessible URI (optional)
789
X-Object-Meta-*             Optional user defined metadata
790
==========================  ===============================
791

    
792
|
793

    
794
================  ===============================
795
Return Code       Description
796
================  ===============================
797
200 (No Content)  The request succeeded
798
================  ===============================
799

    
800

    
801
GET
802
"""
803

    
804
====================  ================================
805
Request Header Name   Value
806
====================  ================================
807
Range                 Optional range of data to retrieve
808
If-Range              Retrieve the missing part if entity is unchanged; otherwise, retrieve the entire new entity (used together with Range header)
809
If-Match              Retrieve if ETags match
810
If-None-Match         Retrieve if ETags don't match
811
If-Modified-Since     Retrieve if object has changed since provided timestamp
812
If-Unmodified-Since   Retrieve if object has not changed since provided timestamp
813
====================  ================================
814

    
815
|
816

    
817
======================  ===================================
818
Request Parameter Name  Value
819
======================  ===================================
820
format                  Optional extended reply type (can be ``json`` or ``xml``)
821
hashmap                 Optional request for hashmap (no value parameter)
822
version                 Optional version identifier or ``list`` (specify a format if requesting a list)
823
======================  ===================================
824

    
825
The reply is the object's data (or part of it), except if a hashmap is requested with ``hashmap``, or a version list with ``version=list`` (in both cases an extended reply format must be specified). Object headers (as in a ``HEAD`` request) are always included.
826

    
827
Hashmaps expose the underlying storage format of the object. Note that each hash is computed after trimming trailing null bytes of the corresponding block. The ``X-Object-Hash`` header reports the single Merkle hash of the object's hashmap (refer to http://bittorrent.org/beps/bep_0030.html for more information).
828

    
829
Example ``format=json`` reply:
830

    
831
::
832

    
833
  {"block_hash": "sha1", "hashes": ["7295c41da03d7f916440b98e32c4a2a39351546c", ...], "block_size": 131072, "bytes": 242}
834

    
835
Example ``format=xml`` reply:
836

    
837
::
838

    
839
  <?xml version="1.0" encoding="UTF-8"?>
840
  <object name="file" bytes="24223726" block_size="131072" block_hash="sha1">
841
    <hash>7295c41da03d7f916440b98e32c4a2a39351546c</hash>
842
    <hash>...</hash>
843
  </object>
844

    
845
Version lists include the version identifier and timestamp for each available object version. Version identifiers can be arbitrary strings, so use the timestamp to find newer versions.
846

    
847
Example ``format=json`` reply:
848

    
849
::
850

    
851
  {"versions": [[85, "1322734861.248469"], [86, "1322734905.009272"], ...]}
852

    
853
Example ``format=xml`` reply:
854

    
855
::
856

    
857
  <?xml version="1.0" encoding="UTF-8"?>
858
  <object name="file">
859
    <version timestamp="1322734861.248469">85</version>
860
    <version timestamp="1322734905.009272">86</version>
861
    <version timestamp="...">...</version>
862
  </object>
863

    
864
The ``Range`` header may include multiple ranges, as outlined in RFC2616. Then the ``Content-Type`` of the reply will be ``multipart/byteranges`` and each part will include a ``Content-Range`` header.
865

    
866
==========================  ===============================
867
Reply Header Name           Value
868
==========================  ===============================
869
ETag                        The ETag of the object
870
Content-Length              The size of the data returned
871
Content-Type                The MIME content type of the object
872
Content-Range               The range of data included (only on a single range request)
873
Last-Modified               The last object modification date (regardless of version)
874
Content-Encoding            The encoding of the object (optional)
875
Content-Disposition         The presentation style of the object (optional)
876
X-Object-Hash               The Merkle hash
877
X-Object-UUID               The object's UUID
878
X-Object-Version            The object's version identifier
879
X-Object-Version-Timestamp  The object's version timestamp
880
X-Object-Modified-By        The user that comitted the object's version
881
X-Object-Manifest           Object parts prefix in ``<container>/<object>`` form (optional)
882
X-Object-Sharing            Object permissions (optional)
883
X-Object-Shared-By          Object inheriting permissions (optional)
884
X-Object-Allowed-To         Allowed actions on object (optional)
885
X-Object-Public             Object's publicly accessible URI (optional)
886
X-Object-Meta-*             Optional user defined metadata
887
==========================  ===============================
888

    
889
Sharing headers (``X-Object-Sharing``, ``X-Object-Shared-By`` and ``X-Object-Allowed-To``) are only included if the request is for the object's latest version (no specific ``version`` parameter is set).
890

    
891
===========================  ==============================
892
Return Code                  Description
893
===========================  ==============================
894
200 (OK)                     The request succeeded
895
206 (Partial Content)        The range request succeeded
896
304 (Not Modified)           The object has not been modified
897
412 (Precondition Failed)    The condition set can not be satisfied
898
416 (Range Not Satisfiable)  The requested range is out of limits
899
===========================  ==============================
900

    
901

    
902
PUT
903
"""
904

    
905
====================  ================================
906
Request Header Name   Value
907
====================  ================================
908
If-Match              Put if ETags match with current object
909
If-None-Match         Put if ETags don't match with current object
910
ETag                  The MD5 hash of the object (optional to check written data)
911
Content-Length        The size of the data written
912
Content-Type          The MIME content type of the object
913
Transfer-Encoding     Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored)
914
X-Copy-From           The source path in the form ``/<container>/<object>``
915
X-Move-From           The source path in the form ``/<container>/<object>``
916
X-Source-Account      The source account to copy/move from
917
X-Source-Version      The source version to copy from
918
Content-Encoding      The encoding of the object (optional)
919
Content-Disposition   The presentation style of the object (optional)
920
X-Object-Manifest     Object parts prefix in ``<container>/<object>`` form (optional)
921
X-Object-Sharing      Object permissions (optional)
922
X-Object-Public       Object is publicly accessible (optional)
923
X-Object-Meta-*       Optional user defined metadata
924
====================  ================================
925

    
926
|
927

    
928
======================  ===================================
929
Request Parameter Name  Value
930
======================  ===================================
931
format                  Optional extended request/conflict response type (can be ``json`` or ``xml``)
932
hashmap                 Optional hashmap provided instead of data (no value parameter)
933
delimiter               Optional copy/move objects starting with object's path and delimiter (to be used with X-Copy-From/X-Move-From)
934
======================  ===================================
935

    
936
The request is the object's data (or part of it), except if a hashmap is provided (using ``hashmap`` and ``format`` parameters). If using a hashmap and all different parts are stored in the server, the object is created. Otherwise the server returns Conflict (409) with the list of the missing parts (in simple text format, with one hash per line, or in JSON/XML - depending on the ``format`` parameter).
937

    
938
Hashmaps should be formatted as outlined in ``GET``.
939

    
940
==========================  ===============================
941
Reply Header Name           Value
942
==========================  ===============================
943
ETag                        The MD5 hash of the object
944
X-Object-Version            The object's new version
945
==========================  ===============================
946

    
947
The ``X-Object-Sharing`` header may include either a ``read=...`` comma-separated user/group list, or a ``write=...`` comma-separated user/group list, or both separated by a semicolon (``;``). Groups are specified as ``<account>:<group>``. To publish the object, set ``X-Object-Public`` to ``true``. To unpublish, set to ``false``, or use an empty header value.
948

    
949
==============================  ==============================
950
Return Code                     Description
951
==============================  ==============================
952
201 (Created)                   The object has been created
953
409 (Conflict)                  The object can not be created from the provided hashmap (a list of missing hashes will be included in the reply)
954
411 (Length Required)           Missing ``Content-Length`` or ``Content-Type`` in the request
955
413 (Request Entity Too Large)  Insufficient quota to complete the request
956
422 (Unprocessable Entity)      The MD5 checksum of the data written to the storage system does not match the (optionally) supplied ETag value
957
==============================  ==============================
958

    
959

    
960
COPY
961
""""
962

    
963
====================  ================================
964
Request Header Name   Value
965
====================  ================================
966
If-Match              Proceed if ETags match with object
967
If-None-Match         Proceed if ETags don't match with object
968
Destination           The destination path in the form ``/<container>/<object>``
969
Destination-Account   The destination account to copy to
970
Content-Type          The MIME content type of the object (optional :sup:`*`)
971
Content-Encoding      The encoding of the object (optional)
972
Content-Disposition   The presentation style of the object (optional)
973
X-Source-Version      The source version to copy from
974
X-Object-Manifest     Object parts prefix in ``<container>/<object>`` form (optional)
975
X-Object-Sharing      Object permissions (optional)
976
X-Object-Public       Object is publicly accessible (optional)
977
X-Object-Meta-*       Optional user defined metadata
978
====================  ================================
979

    
980
:sup:`*` *When using django locally with the supplied web server, use the ignore_content_type parameter, or do provide a valid Content-Type, as a type of text/plain is applied by default to all requests. Client software should always state ignore_content_type, except when a Content-Type is explicitly defined by the user.*
981

    
982
======================  ===================================
983
Request Parameter Name  Value
984
======================  ===================================
985
format                  Optional conflict response type (can be ``json`` or ``xml``)
986
ignore_content_type     Ignore the supplied Content-Type
987
delimiter               Optional copy objects starting with object's path and delimiter
988
======================  ===================================
989

    
990
Refer to ``PUT``/``POST`` for a description of request headers. Metadata is also copied, updated with any values defined. Sharing/publishing options are not copied.
991

    
992
==========================  ===============================
993
Reply Header Name           Value
994
==========================  ===============================
995
X-Object-Version            The object's new version
996
==========================  ===============================
997

    
998
|
999

    
1000
==============================  ==============================
1001
Return Code                     Description
1002
==============================  ==============================
1003
201 (Created)                   The object has been created
1004
413 (Request Entity Too Large)  Insufficient quota to complete the request
1005
==============================  ==============================
1006

    
1007

    
1008
MOVE
1009
""""
1010

    
1011
Same as ``COPY``, without the ``X-Source-Version`` request header. The ``MOVE`` operation is always applied on the latest version.
1012

    
1013

    
1014
POST
1015
""""
1016

    
1017
====================  ================================
1018
Request Header Name   Value
1019
====================  ================================
1020
If-Match              Proceed if ETags match with object
1021
If-None-Match         Proceed if ETags don't match with object
1022
Content-Length        The size of the data written (optional, to update)
1023
Content-Type          The MIME content type of the object (optional, to update)
1024
Content-Range         The range of data supplied (optional, to update)
1025
Transfer-Encoding     Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored)
1026
Content-Encoding      The encoding of the object (optional)
1027
Content-Disposition   The presentation style of the object (optional)
1028
X-Source-Object       Update with data from the object at path ``/<container>/<object>`` (optional, to update)
1029
X-Source-Account      The source account to update from
1030
X-Source-Version      The source version to update from (optional, to update)
1031
X-Object-Bytes        The updated object's final size (optional, when updating)
1032
X-Object-Manifest     Object parts prefix in ``<container>/<object>`` form (optional)
1033
X-Object-Sharing      Object permissions (optional)
1034
X-Object-Public       Object is publicly accessible (optional)
1035
X-Object-Meta-*       Optional user defined metadata
1036
====================  ================================
1037

    
1038
|
1039

    
1040
======================  ============================================
1041
Request Parameter Name  Value
1042
======================  ============================================
1043
format                  Optional conflict response type (can be ``json`` or ``xml``)
1044
update                  Do not replace metadata (no value parameter)
1045
======================  ============================================
1046

    
1047
The ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest`` and ``X-Object-Meta-*`` headers are considered to be user defined metadata. An operation without the ``update`` parameter will overwrite all previous values and remove any keys not supplied. When using ``update`` any metadata with an empty value will be deleted.
1048

    
1049
To change permissions, include an ``X-Object-Sharing`` header (as defined in ``PUT``). To publish, include an ``X-Object-Public`` header, with a value of ``true``. If no such headers are defined, no changes will be applied to sharing/public. Use empty values to remove permissions/unpublish (unpublishing also works with ``false`` as a header value). Sharing options are applied to the object - not its versions.
1050

    
1051
To update an object's data:
1052

    
1053
* Either set ``Content-Type`` to ``application/octet-stream``, or provide an object with ``X-Source-Object``. If ``Content-Type`` has some other value, it will be ignored and only the metadata will be updated.
1054
* If the data is supplied in the request (using ``Content-Type`` instead of ``X-Source-Object``), a valid ``Content-Length`` header is required - except if using chunked transfers (set ``Transfer-Encoding`` to ``chunked``).
1055
* Set ``Content-Range`` as specified in RFC2616, with the following differences:
1056

    
1057
  * Client software MAY omit ``last-byte-pos`` of if the length of the range being transferred is unknown or difficult to determine.
1058
  * Client software SHOULD not specify the ``instance-length`` (use a ``*``), unless there is a reason for performing a size check at the server.
1059
* If ``Content-Range`` used has a ``byte-range-resp-spec = *``, data will be appended to the object.
1060

    
1061
Optionally, truncate the updated object to the desired length with the ``X-Object-Bytes`` header.
1062

    
1063
A data update will trigger an ETag change. Updated ETags may happen asynchronously and appear at the server with a delay.
1064

    
1065
No reply content. No reply headers if only metadata is updated.
1066

    
1067
==========================  ===============================
1068
Reply Header Name           Value
1069
==========================  ===============================
1070
ETag                        The new ETag of the object (data updated)
1071
X-Object-Version            The object's new version
1072
==========================  ===============================
1073

    
1074
|
1075

    
1076
==============================  ==============================
1077
Return Code                     Description
1078
==============================  ==============================
1079
202 (Accepted)                  The request has been accepted (not a data update)
1080
204 (No Content)                The request succeeded (data updated)
1081
411 (Length Required)           Missing ``Content-Length`` in the request
1082
413 (Request Entity Too Large)  Insufficient quota to complete the request
1083
416 (Range Not Satisfiable)     The supplied range is invalid
1084
==============================  ==============================
1085

    
1086
The ``POST`` method can also be used for creating an object via a standard HTML form. If the request ``Content-Type`` is ``multipart/form-data``, none of the above headers will be processed. The form should have an ``X-Object-Data`` field, as in the following example. The token is passed as a request parameter. ::
1087

    
1088
  <form method="post" action="https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt?X-Auth-Token=0000" enctype="multipart/form-data">
1089
    <input type="file" name="X-Object-Data">
1090
    <input type="submit">
1091
  </form>
1092

    
1093
This will create/override the object with the given name, as if using ``PUT``. The ``Content-Type`` of the object will be set to the value of the corresponding header sent in the part of the request containing the data (usually, automatically handled by the browser). Metadata, sharing and other object attributes can not be set this way. The response will contain the object's ETag.
1094

    
1095
==========================  ===============================
1096
Reply Header Name           Value
1097
==========================  ===============================
1098
ETag                        The MD5 hash of the object
1099
X-Object-Version            The object's new version
1100
==========================  ===============================
1101

    
1102
|
1103

    
1104
==============================  ==============================
1105
Return Code                     Description
1106
==============================  ==============================
1107
201 (Created)                   The object has been created
1108
413 (Request Entity Too Large)  Insufficient quota to complete the request
1109
==============================  ==============================
1110

    
1111

    
1112
DELETE
1113
""""""
1114

    
1115
======================  ===================================
1116
Request Parameter Name  Value
1117
======================  ===================================
1118
until                   Optional timestamp
1119
delimiter               Optional delete also objects starting with object's path and delimiter
1120
======================  ===================================
1121

    
1122
If ``until`` is defined, the object is "purged" up to that time (the history up to then is deleted). If also ``delimiter`` is defined, purge is applied only on the object.
1123

    
1124
No reply content/headers.
1125

    
1126
===========================  ==============================
1127
Return Code                  Description
1128
===========================  ==============================
1129
204 (No Content)             The request succeeded
1130
===========================  ==============================
1131

    
1132
Sharing and Public Objects
1133
^^^^^^^^^^^^^^^^^^^^^^^^^^
1134

    
1135
Read and write control in Pithos is managed by setting appropriate permissions with the ``X-Object-Sharing`` header. The permissions are applied using directory-based inheritance. A directory is an object with the corresponding content type. The default delimiter is ``/``. Thus, each set of authorization directives is applied to all objects in the directory object where the corresponding ``X-Object-Sharing`` header is defined. If there are nested/overlapping permissions, the closest to the object is applied. When retrieving an object, the ``X-Object-Shared-By`` header reports where it gets its permissions from. If not present, the object is the actual source of authorization directives.
1136

    
1137
A user may ``GET`` another account or container. The result will include a limited reply, containing only the allowed containers or objects respectively. A top-level request with an authentication token, will return a list of allowed accounts, so the user can easily find out which other users share objects. The ``X-Object-Allowed-To`` header lists the actions allowed on an object, if it does not belong to the requesting user.
1138

    
1139
Objects that are marked as public, via the ``X-Object-Public`` meta, are also available at the corresponding URI returned for ``HEAD`` or ``GET``. Requests for public objects do not need to include an ``X-Auth-Token``. Pithos will ignore request parameters and only include the following headers in the reply (all ``X-Object-*`` meta is hidden):
1140

    
1141
==========================  ===============================
1142
Reply Header Name           Value
1143
==========================  ===============================
1144
ETag                        The ETag of the object
1145
Content-Length              The size of the data returned
1146
Content-Type                The MIME content type of the object
1147
Content-Range               The range of data included (only on a single range request)
1148
Last-Modified               The last object modification date (regardless of version)
1149
Content-Encoding            The encoding of the object (optional)
1150
Content-Disposition         The presentation style of the object (optional)
1151
==========================  ===============================
1152

    
1153
Public objects are not included and do not influence cross-user listings. They are, however, readable by all users.
1154

    
1155
Summary
1156
^^^^^^^
1157

    
1158
List of differences from the OOS API:
1159

    
1160
* Support for ``X-Account-Meta-*`` style headers at the account level. Use ``POST`` to update.
1161
* Support for ``X-Container-Meta-*`` style headers at the container level. Can be set when creating via ``PUT``. Use ``POST`` to update.
1162
* Header ``X-Container-Object-Meta`` at the container level and parameter ``meta`` in container listings. (**TBD**)
1163
* Account and container policies to manage behavior and limits. Container behavior overrides account settings. Account quota sets the maximum bytes limit, regardless of container values.
1164
* Headers ``X-Container-Block-*`` at the container level, exposing the underlying storage characteristics.
1165
* All metadata replies, at all levels, include latest modification information.
1166
* At all levels, a ``HEAD`` or ``GET`` request may use ``If-Modified-Since`` and ``If-Unmodified-Since`` headers.
1167
* Container/object lists include more fields if the reply is of type JSON/XML. Some names are kept to their OOS API equivalents for compatibility.
1168
* Option to include only shared containers/objects in listings.
1169
* Object metadata allowed, in addition to ``X-Object-Meta-*``: ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``. These are all replaced with every update operation, except if using the ``update`` parameter (in which case individual keys can also be deleted). Deleting meta by providing empty values also works when copying/moving an object.
1170
* Multi-range object ``GET`` support as outlined in RFC2616.
1171
* Object hashmap retrieval through ``GET`` and the ``format`` parameter.
1172
* Object create via hashmap through ``PUT`` and the ``format`` parameter.
1173
* The object's Merkle hash is always returned in the ``X-Object-Hash`` header.
1174
* The object's UUID is always returned in the ``X-Object-UUID`` header. The UUID remains unchanged, even when the object's data or metadata changes, or the object is moved to another path (is renamed). A new UUID is assigned when creating or copying an object.
1175
* Object create using ``POST`` to support standard HTML forms.
1176
* Partial object updates through ``POST``, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers. Use another object's data to update with ``X-Source-Object`` and ``X-Source-Version``. Truncate with ``X-Object-Bytes``.
1177
* Include new version identifier in replies for object replace/change requests.
1178
* Object ``MOVE`` support and ``ignore_content_type`` parameter in both ``COPY`` and ``MOVE``.
1179
* Conditional object create/update operations, using ``If-Match`` and ``If-None-Match`` headers.
1180
* Time-variant account/container listings via the ``until`` parameter.
1181
* Object versions - parameter ``version`` in ``HEAD``/``GET`` (list versions with ``GET``), ``X-Object-Version-*`` meta in replies, ``X-Source-Version`` in ``PUT``/``COPY``.
1182
* Sharing/publishing with ``X-Object-Sharing``, ``X-Object-Public`` at the object level. Cross-user operations are allowed - controlled by sharing directives. Available actions in cross-user requests are reported with ``X-Object-Allowed-To``. Permissions may include groups defined with ``X-Account-Group-*`` at the account level. These apply to the object - not its versions.
1183
* Support for directory-based inheritance when enforcing permissions. Parent object carrying the authorization directives is reported in ``X-Object-Shared-By``.
1184
* Copy and move between accounts with ``X-Source-Account`` and ``Destination-Account`` headers.
1185
* Large object support with ``X-Object-Manifest``.
1186
* Trace the user that created/modified an object with ``X-Object-Modified-By``.
1187
* Purge container/object history with the ``until`` parameter in ``DELETE``.
1188

    
1189
Clarifications/suggestions:
1190

    
1191
* All non-ASCII characters in headers should be URL-encoded.
1192
* Authentication is done by another system. The token is used in the same way, but it is obtained differently. The top level ``GET`` request is kept compatible with the OOS API and allows for guest/testing operations.
1193
* Some processing is done in the variable part of all ``X-*-Meta-*`` headers. If it includes underscores, they will be converted to dashes and the first letter of all intra-dash strings will be capitalized.
1194
* A ``GET`` reply for a level will include all headers of the corresponding ``HEAD`` request.
1195
* To avoid conflicts between objects and virtual directory markers in container listings, it is recommended that object names do not end with the delimiter used.
1196
* The ``Accept`` header may be used in requests instead of the ``format`` parameter to specify the desired request/reply format. The parameter overrides the header.
1197
* Container/object lists use a ``200`` return code if the reply is of type JSON/XML. The reply will include an empty JSON/XML.
1198
* In headers, dates are formatted according to RFC 1123. In extended information listings, the ``last_modified`` field is formatted according to ISO 8601 (for OOS API compatibility). All other fields (Pithos extensions) use integer tiemstamps.
1199
* The ``Last-Modified`` header value always reflects the actual latest change timestamp, regardless of time control parameters and version requests. Time precondition checks with ``If-Modified-Since`` and ``If-Unmodified-Since`` headers are applied to this value.
1200
* A copy/move using ``PUT``/``COPY``/``MOVE`` will always update metadata, keeping all old values except the ones redefined in the request headers.
1201
* A ``HEAD`` or ``GET`` for an ``X-Object-Manifest`` object, will include modified ``Content-Length`` and ``ETag`` headers, according to the characteristics of the objects under the specified prefix. The ``Etag`` will be the MD5 hash of the corresponding ETags concatenated. In extended container listings there is no metadata processing.
1202

    
1203
Recommended Practices and Examples
1204
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1205

    
1206
Assuming an authentication token is obtained, the following high-level operations are available - shown with ``curl``:
1207

    
1208
* Get account information ::
1209

    
1210
    curl -X HEAD -D - \
1211
         -H "X-Auth-Token: 0000" \
1212
         https://pithos.dev.grnet.gr/v1/user
1213

    
1214
* List available containers ::
1215

    
1216
    curl -X GET -D - \
1217
         -H "X-Auth-Token: 0000" \
1218
         https://pithos.dev.grnet.gr/v1/user
1219

    
1220
* Get container information ::
1221

    
1222
    curl -X HEAD -D - \
1223
         -H "X-Auth-Token: 0000" \
1224
         https://pithos.dev.grnet.gr/v1/user/pithos
1225

    
1226
* Add a new container ::
1227

    
1228
    curl -X PUT -D - \
1229
         -H "X-Auth-Token: 0000" \
1230
         https://pithos.dev.grnet.gr/v1/user/test
1231

    
1232
* Delete a container ::
1233

    
1234
    curl -X DELETE -D - \
1235
         -H "X-Auth-Token: 0000" \
1236
         https://pithos.dev.grnet.gr/v1/user/test
1237

    
1238
* List objects in a container ::
1239

    
1240
    curl -X GET -D - \
1241
         -H "X-Auth-Token: 0000" \
1242
         https://pithos.dev.grnet.gr/v1/user/pithos
1243

    
1244
* List objects in a container (extended reply) ::
1245

    
1246
    curl -X GET -D - \
1247
         -H "X-Auth-Token: 0000" \
1248
         https://pithos.dev.grnet.gr/v1/user/pithos?format=json
1249

    
1250
  It is recommended that extended replies are cached and subsequent requests utilize the ``If-Modified-Since`` header.
1251

    
1252
* List metadata keys used by objects in a container
1253

    
1254
  Will be in the ``X-Container-Object-Meta`` reply header, included in container information or object list (``HEAD`` or ``GET``). (**TBD**)
1255

    
1256
* List objects in a container having a specific meta defined ::
1257

    
1258
    curl -X GET -D - \
1259
         -H "X-Auth-Token: 0000" \
1260
         https://pithos.dev.grnet.gr/v1/user/pithos?meta=favorites
1261

    
1262
* Retrieve an object ::
1263

    
1264
    curl -X GET -D - \
1265
         -H "X-Auth-Token: 0000" \
1266
         https://pithos.dev.grnet.gr/v1/user/pithos/README.txt
1267

    
1268
* Retrieve an object (specific ranges of data) ::
1269

    
1270
    curl -X GET -D - \
1271
         -H "X-Auth-Token: 0000" \
1272
         -H "Range: bytes=0-9" \
1273
         https://pithos.dev.grnet.gr/v1/user/pithos/README.txt
1274

    
1275
  This will return the first 10 bytes. To get the first 10, bytes 30-39 and the last 100 use ``Range: bytes=0-9,30-39,-100``.
1276

    
1277
* Add a new object (folder type) (**TBD**) ::
1278

    
1279
    curl -X PUT -D - \
1280
         -H "X-Auth-Token: 0000" \
1281
         -H "Content-Type: application/directory" \
1282
         https://pithos.dev.grnet.gr/v1/user/pithos/folder
1283

    
1284
* Add a new object ::
1285

    
1286
    curl -X PUT -D - \
1287
         -H "X-Auth-Token: 0000" \
1288
         -H "Content-Type: text/plain" \
1289
         -T EXAMPLE.txt
1290
         https://pithos.dev.grnet.gr/v1/user/pithos/folder/EXAMPLE.txt
1291

    
1292
* Update an object ::
1293

    
1294
    curl -X POST -D - \
1295
         -H "X-Auth-Token: 0000" \
1296
         -H "Content-Length: 10" \
1297
         -H "Content-Type: application/octet-stream" \
1298
         -H "Content-Range: bytes 10-19/*" \
1299
         -d "0123456789" \
1300
         https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
1301

    
1302
  This will update bytes 10-19 with the data specified.
1303

    
1304
* Update an object (append) ::
1305

    
1306
    curl -X POST -D - \
1307
         -H "X-Auth-Token: 0000" \
1308
         -H "Content-Length: 10" \
1309
         -H "Content-Type: application/octet-stream" \
1310
         -H "Content-Range: bytes */*" \
1311
         -d "0123456789" \
1312
         https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
1313

    
1314
* Update an object (truncate) ::
1315

    
1316
    curl -X POST -D - \
1317
         -H "X-Auth-Token: 0000" \
1318
         -H "X-Source-Object: /folder/EXAMPLE.txt" \
1319
         -H "Content-Range: bytes 0-0/*" \
1320
         -H "X-Object-Bytes: 0" \
1321
         https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
1322

    
1323
  This will truncate the object to 0 bytes.
1324

    
1325
* Add object metadata ::
1326

    
1327
    curl -X POST -D - \
1328
         -H "X-Auth-Token: 0000" \
1329
         -H "X-Object-Meta-First: first_meta_value" \
1330
         -H "X-Object-Meta-Second: second_meta_value" \
1331
         https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
1332

    
1333
* Delete object metadata ::
1334

    
1335
    curl -X POST -D - \
1336
         -H "X-Auth-Token: 0000" \
1337
         -H "X-Object-Meta-First: first_meta_value" \
1338
         https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
1339

    
1340
  Metadata can only be "set". To delete ``X-Object-Meta-Second``, reset all metadata.
1341

    
1342
* Delete an object ::
1343

    
1344
    curl -X DELETE -D - \
1345
         -H "X-Auth-Token: 0000" \
1346
         https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt