========================= ================================
Revision Description
========================= ================================
+0.5 (July 11, 2011) Object update from another object's data.
0.4 (July 01, 2011) Object permissions and account groups.
\ Control versioning behavior and container quotas with container policy directives.
\ Support updating/deleting individual metadata with ``POST``.
Transfer-Encoding Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored)
Content-Encoding The encoding of the object (optional)
Content-Disposition The presentation style of the object (optional)
+X-Source-Object Update with data from the object at path ``/<container>/<object>`` (optional, to update)
+X-Source-Version The source version to update from (optional, to update)
X-Object-Manifest Object parts prefix in ``<container>/<object>`` form (optional)
X-Object-Sharing Object permissions (optional)
X-Object-Public Object is publicly accessible (optional)
To update an object's data:
-* Set ``Content-Type`` to ``application/octet-stream``. If ``Content-Type`` has some other value, it will be ignored and only the metadata will be updated.
-* Supply ``Content-Length`` (except if using chunked transfers), ``Content-Type`` and ``Content-Range`` headers.
+* Either set ``Content-Type`` to ``application/octet-stream``, or provide an object with ``X-Source-Object``. If ``Content-Type`` has some other value, it will be ignored and only the metadata will be updated.
+* If the data is supplied in the request (using ``Content-Type`` instead of ``X-Source-Object``), a valid ``Content-Length`` header is required - except if using chunked transfers (set ``Transfer-Encoding`` to ``chunked``).
* Set ``Content-Range`` as specified in RFC2616, with the following differences:
* Client software MAY omit ``last-byte-pos`` of if the length of the range being transferred is unknown or difficult to determine.
* Client software SHOULD not specify the ``instance-length`` (use a ``*``), unless there is a reason for performing a size check at the server.
-* If ``Content-Range`` used has a ``byte-range-resp-spec = *``, data supplied will be appended to the object.
+* If ``Content-Range`` used has a ``byte-range-resp-spec = *``, data will be appended to the object.
A data update will trigger an ETag change. The new ETag will not correspond to the object's MD5 sum (**TBD**) and will be included in reply headers.
* Object metadata allowed, in addition to ``X-Object-Meta-*``: ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``. These are all replaced with every update operation, except if using the ``update`` parameter (in which case individual keys can also be deleted). Deleting meta by providing empty values also works when copying/moving an object.
* Multi-range object GET support as outlined in RFC2616.
* Object hashmap retrieval through GET and the ``format`` parameter.
-* Partial object updates through POST, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers.
+* Partial object updates through POST, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers. Use another object's data to update with ``X-Source-Object`` and ``X-Source-Version``.
* Object ``MOVE`` support.
* Time-variant account/container listings via the ``until`` parameter.
* Object versions - parameter ``version`` in HEAD/GET (list versions with GET), ``X-Object-Version-*`` meta in replies, ``X-Source-Version`` in PUT/COPY.
Pithos clients should use the ``pithos`` container for all Pithos objects. Object names use the ``/`` delimiter to impose a hierarchy of folders and files.
-At the object level, tags are implemented by managing metadata keys. The client software should allow the user to use any string as a tag (except ``trash``) and then set the corresponding ``X-Object-Meta-<tag>`` key at the server. The API extensions provided, allow for listing all tags in a container and filtering object listings based on one or more tags. The tag list is sufficient for implementing the ``tags`` element, either as a special, virtual folder (as done in the first version of Pithos), or as an application menu.
-
-To manage the deletion of files use the same API and the ``X-Object-Meta-Trash`` key. The string ``trash`` can not be used as a tag. The ``trash`` element should be presented as a folder, although with no hierarchy.
+At the object level, tags are implemented by managing metadata keys. The client software should allow the user to use any string as a tag and then set the corresponding ``X-Object-Meta-<tag>`` key at the server. The API extensions provided, allow for listing all tags in a container and filtering object listings based on one or more tags. The tag list is sufficient for implementing the ``tags`` element, either as a special, virtual folder (as done in the first version of Pithos), or as an application menu.
The metadata specification is summarized in the following table.
=========================== ==============================
Metadata Name Value
=========================== ==============================
-X-Object-Meta-Trash Set to ``true`` if the object has been moved to the trash
X-Object-Meta-* Use for other tags that apply to the object
=========================== ==============================
curl -X GET -D - \
-H "X-Auth-Token: 0000" \
- https://pithos.dev.grnet.gr/v1/user/pithos?meta=trash
-
- This is the recommended way of tagging/retrieving objects in trash.
+ https://pithos.dev.grnet.gr/v1/user/pithos?meta=favorites
* Retrieve an object ::
curl -X DELETE -D - \
-H "X-Auth-Token: 0000" \
https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
-
copy_from = request.META.get('HTTP_X_COPY_FROM')
move_from = request.META.get('HTTP_X_MOVE_FROM')
if copy_from or move_from:
- # TODO: Why is this required? Copy this ammount?
- content_length = get_content_length(request)
+ content_length = get_content_length(request) # Required by the API.
if move_from:
try:
if k in prev_meta:
meta[k] = prev_meta[k]
- # A Content-Type header indicates data updates.
- if not content_type or content_type != 'application/octet-stream':
+ # A Content-Type or X-Source-Object header indicates data updates.
+ src_object = request.META.get('HTTP_X_SOURCE_OBJECT')
+ if (not content_type or content_type != 'application/octet-stream') and not src_object:
# Do permissions first, as it may fail easier.
if permissions is not None:
try:
ranges = get_content_range(request)
if not ranges:
raise RangeNotSatisfiable('Invalid Content-Range header')
- # Require either a Content-Length, or 'chunked' Transfer-Encoding.
- content_length = -1
- if request.META.get('HTTP_TRANSFER_ENCODING') != 'chunked':
- content_length = get_content_length(request)
try:
size, hashmap = backend.get_object_hashmap(request.user, v_account, v_container, v_object)
offset = size
elif offset > size:
raise RangeNotSatisfiable('Supplied offset is beyond object limits')
- if length is None or content_length == -1:
- length = content_length # Nevermind the error.
- elif length != content_length:
- raise BadRequest('Content length does not match range length')
+ if src_object:
+ src_container, src_name = split_container_object_string(src_object)
+ src_version = request.META.get('HTTP_X_SOURCE_VERSION')
+ try:
+ src_size, src_hashmap = backend.get_object_hashmap(request.user, v_account, src_container, src_name, src_version)
+ except NotAllowedError:
+ raise Unauthorized('Access denied')
+ except NameError:
+ raise ItemNotFound('Source object does not exist')
+
+ if length is None:
+ length = src_size
+ elif length > src_size:
+ raise BadRequest('Object length is smaller than range length')
+ else:
+ # Require either a Content-Length, or 'chunked' Transfer-Encoding.
+ content_length = -1
+ if request.META.get('HTTP_TRANSFER_ENCODING') != 'chunked':
+ content_length = get_content_length(request)
+
+ if length is None:
+ length = content_length
+ else:
+ if content_length == -1:
+ # TODO: Get up to length bytes in chunks.
+ length = content_length
+ elif length != content_length:
+ raise BadRequest('Content length does not match range length')
if total is not None and (total != size or offset >= size or (length > 0 and offset + length >= size)):
raise RangeNotSatisfiable('Supplied range will change provided object limits')
- sock = raw_input_socket(request)
- data = ''
- for d in socket_read_iterator(sock, length, backend.block_size):
- # TODO: Raise 408 (Request Timeout) if this takes too long.
- # TODO: Raise 499 (Client Disconnect) if a length is defined and we stop before getting this much data.
- data += d
- bytes = put_object_block(hashmap, data, offset)
- offset += bytes
- data = data[bytes:]
- if len(data) > 0:
- put_object_block(hashmap, data, offset)
+ if src_object:
+ if offset % backend.block_size == 0:
+ # Update the hashes only.
+ sbi = 0
+ while length > 0:
+ bi = int(offset / backend.block_size)
+ bl = min(length, backend.block_size)
+ if bi < len(hashmap):
+ if bl == backend.block_size:
+ hashmap[bi] = src_hashmap[sbi]
+ else:
+ data = backend.get_block(src_hashmap[sbi])
+ hashmap[bi] = backend.update_block(hashmap[bi], data[:bl], 0)
+ else:
+ hashmap.append(src_hashmap[sbi])
+ offset += bl
+ length -= bl
+ sbi += 1
+ else:
+ data = ''
+ sbi = 0
+ while length > 0:
+ data += backend.get_block(src_hashmap[sbi])
+ if length < backend.block_size:
+ data = data[:length]
+ bytes = put_object_block(hashmap, data, offset)
+ offset += bytes
+ data = data[bytes:]
+ length -= bytes
+ sbi += 1
+ else:
+ sock = raw_input_socket(request)
+ data = ''
+ for d in socket_read_iterator(sock, length, backend.block_size):
+ # TODO: Raise 408 (Request Timeout) if this takes too long.
+ # TODO: Raise 499 (Client Disconnect) if a length is defined and we stop before getting this much data.
+ data += d
+ bytes = put_object_block(hashmap, data, offset)
+ offset += bytes
+ data = data[bytes:]
+ if len(data) > 0:
+ put_object_block(hashmap, data, offset)
if offset > size:
size = offset
raise Conflict(json.dumps(e.data))
if public is not None:
try:
- backend.update_object_public(request.user, v_account, v_container, v_object, public)
+ backend.update_object_public(request.user, v_account, dest_container, dest_name, public)
except NotAllowedError:
raise Unauthorized('Access denied')
except NameError:
projects / pithos / commitdiff