Revision ab2e317e

b/docs/source/devguide.rst
25 25
=========================  ================================
26 26
Revision                   Description
27 27
=========================  ================================
28
0.5 (July 11, 2011)        Object update from another object's data.
28 29
0.4 (July 01, 2011)        Object permissions and account groups.
29 30
\                          Control versioning behavior and container quotas with container policy directives.
30 31
\                          Support updating/deleting individual metadata with ``POST``.
......
676 677
Transfer-Encoding     Set to ``chunked`` to specify incremental uploading (if used, ``Content-Length`` is ignored)
677 678
Content-Encoding      The encoding of the object (optional)
678 679
Content-Disposition   The presentation style of the object (optional)
680
X-Source-Object       Update with data from the object at path ``/<container>/<object>`` (optional, to update)
681
X-Source-Version      The source version to update from (optional, to update)
679 682
X-Object-Manifest     Object parts prefix in ``<container>/<object>`` form (optional)
680 683
X-Object-Sharing      Object permissions (optional)
681 684
X-Object-Public       Object is publicly accessible (optional)
......
688 691

  
689 692
To update an object's data:
690 693

  
691
* Set ``Content-Type`` to ``application/octet-stream``. If ``Content-Type`` has some other value, it will be ignored and only the metadata will be updated.
692
* Supply ``Content-Length`` (except if using chunked transfers), ``Content-Type`` and ``Content-Range`` headers.
694
* Either set ``Content-Type`` to ``application/octet-stream``, or provide an object with ``X-Source-Object``. If ``Content-Type`` has some other value, it will be ignored and only the metadata will be updated.
695
* If the data is supplied in the request (using ``Content-Type`` instead of ``X-Source-Object``), a valid ``Content-Length`` header is required - except if using chunked transfers (set ``Transfer-Encoding`` to ``chunked``).
693 696
* Set ``Content-Range`` as specified in RFC2616, with the following differences:
694 697

  
695 698
  * Client software MAY omit ``last-byte-pos`` of if the length of the range being transferred is unknown or difficult to determine.
696 699
  * Client software SHOULD not specify the ``instance-length`` (use a ``*``), unless there is a reason for performing a size check at the server.
697
* If ``Content-Range`` used has a ``byte-range-resp-spec = *``, data supplied will be appended to the object.
700
* If ``Content-Range`` used has a ``byte-range-resp-spec = *``, data will be appended to the object.
698 701

  
699 702
A data update will trigger an ETag change. The new ETag will not correspond to the object's MD5 sum (**TBD**) and will be included in reply headers.
700 703

  
......
767 770
* Object metadata allowed, in addition to ``X-Object-Meta-*``: ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``. These are all replaced with every update operation, except if using the ``update`` parameter (in which case individual keys can also be deleted). Deleting meta by providing empty values also works when copying/moving an object.
768 771
* Multi-range object GET support as outlined in RFC2616.
769 772
* Object hashmap retrieval through GET and the ``format`` parameter.
770
* Partial object updates through POST, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers.
773
* Partial object updates through POST, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers. Use another object's data to update with ``X-Source-Object`` and ``X-Source-Version``.
771 774
* Object ``MOVE`` support.
772 775
* Time-variant account/container listings via the ``until`` parameter.
773 776
* Object versions - parameter ``version`` in HEAD/GET (list versions with GET), ``X-Object-Version-*`` meta in replies, ``X-Source-Version`` in PUT/COPY.
......
821 824

  
822 825
Pithos clients should use the ``pithos`` container for all Pithos objects. Object names use the ``/`` delimiter to impose a hierarchy of folders and files.
823 826

  
824
At the object level, tags are implemented by managing metadata keys. The client software should allow the user to use any string as a tag (except ``trash``) and then set the corresponding ``X-Object-Meta-<tag>`` key at the server. The API extensions provided, allow for listing all tags in a container and filtering object listings based on one or more tags. The tag list is sufficient for implementing the ``tags`` element, either as a special, virtual folder (as done in the first version of Pithos), or as an application menu.
825

  
826
To manage the deletion of files use the same API and the ``X-Object-Meta-Trash`` key. The string ``trash`` can not be used as a tag. The ``trash`` element should be presented as a folder, although with no hierarchy.
827
At the object level, tags are implemented by managing metadata keys. The client software should allow the user to use any string as a tag and then set the corresponding ``X-Object-Meta-<tag>`` key at the server. The API extensions provided, allow for listing all tags in a container and filtering object listings based on one or more tags. The tag list is sufficient for implementing the ``tags`` element, either as a special, virtual folder (as done in the first version of Pithos), or as an application menu.
827 828

  
828 829
The metadata specification is summarized in the following table.
829 830

  
830 831
===========================  ==============================
831 832
Metadata Name                Value
832 833
===========================  ==============================
833
X-Object-Meta-Trash          Set to ``true`` if the object has been moved to the trash
834 834
X-Object-Meta-*              Use for other tags that apply to the object
835 835
===========================  ==============================
836 836

  
......
891 891

  
892 892
    curl -X GET -D - \
893 893
         -H "X-Auth-Token: 0000" \
894
         https://pithos.dev.grnet.gr/v1/user/pithos?meta=trash
895

  
896
  This is the recommended way of tagging/retrieving objects in trash.
894
         https://pithos.dev.grnet.gr/v1/user/pithos?meta=favorites
897 895

  
898 896
* Retrieve an object ::
899 897

  
......
969 967
    curl -X DELETE -D - \
970 968
         -H "X-Auth-Token: 0000" \
971 969
         https://pithos.dev.grnet.gr/v1/user/folder/EXAMPLE.txt
972

  
b/pithos/api/functions.py
597 597
    copy_from = request.META.get('HTTP_X_COPY_FROM')
598 598
    move_from = request.META.get('HTTP_X_MOVE_FROM')
599 599
    if copy_from or move_from:
600
        # TODO: Why is this required? Copy this ammount?
601
        content_length = get_content_length(request)
600
        content_length = get_content_length(request) # Required by the API.
602 601
        
603 602
        if move_from:
604 603
            try:
......
745 744
            if k in prev_meta:
746 745
                meta[k] = prev_meta[k]
747 746
    
748
    # A Content-Type header indicates data updates.
749
    if not content_type or content_type != 'application/octet-stream':
747
    # A Content-Type or X-Source-Object header indicates data updates.
748
    src_object = request.META.get('HTTP_X_SOURCE_OBJECT')
749
    if (not content_type or content_type != 'application/octet-stream') and not src_object:
750 750
        # Do permissions first, as it may fail easier.
751 751
        if permissions is not None:
752 752
            try:
......
783 783
    ranges = get_content_range(request)
784 784
    if not ranges:
785 785
        raise RangeNotSatisfiable('Invalid Content-Range header')
786
    # Require either a Content-Length, or 'chunked' Transfer-Encoding.
787
    content_length = -1
788
    if request.META.get('HTTP_TRANSFER_ENCODING') != 'chunked':
789
        content_length = get_content_length(request)
790 786
    
791 787
    try:
792 788
        size, hashmap = backend.get_object_hashmap(request.user, v_account, v_container, v_object)
......
800 796
        offset = size
801 797
    elif offset > size:
802 798
        raise RangeNotSatisfiable('Supplied offset is beyond object limits')
803
    if length is None or content_length == -1:
804
        length = content_length # Nevermind the error.
805
    elif length != content_length:
806
        raise BadRequest('Content length does not match range length')
799
    if src_object:
800
        src_container, src_name = split_container_object_string(src_object)
801
        src_version = request.META.get('HTTP_X_SOURCE_VERSION')
802
        try:
803
            src_size, src_hashmap = backend.get_object_hashmap(request.user, v_account, src_container, src_name, src_version)
804
        except NotAllowedError:
805
            raise Unauthorized('Access denied')
806
        except NameError:
807
            raise ItemNotFound('Source object does not exist')
808
        
809
        if length is None:
810
            length = src_size
811
        elif length > src_size:
812
            raise BadRequest('Object length is smaller than range length')
813
    else:
814
        # Require either a Content-Length, or 'chunked' Transfer-Encoding.
815
        content_length = -1
816
        if request.META.get('HTTP_TRANSFER_ENCODING') != 'chunked':
817
            content_length = get_content_length(request)
818
        
819
        if length is None:
820
            length = content_length
821
        else:
822
            if content_length == -1:
823
                # TODO: Get up to length bytes in chunks.
824
                length = content_length
825
            elif length != content_length:
826
                raise BadRequest('Content length does not match range length')
807 827
    if total is not None and (total != size or offset >= size or (length > 0 and offset + length >= size)):
808 828
        raise RangeNotSatisfiable('Supplied range will change provided object limits')
809 829
    
810
    sock = raw_input_socket(request)
811
    data = ''
812
    for d in socket_read_iterator(sock, length, backend.block_size):
813
        # TODO: Raise 408 (Request Timeout) if this takes too long.
814
        # TODO: Raise 499 (Client Disconnect) if a length is defined and we stop before getting this much data.
815
        data += d
816
        bytes = put_object_block(hashmap, data, offset)
817
        offset += bytes
818
        data = data[bytes:]
819
    if len(data) > 0:
820
        put_object_block(hashmap, data, offset)
830
    if src_object:
831
        if offset % backend.block_size == 0:
832
            # Update the hashes only.
833
            sbi = 0
834
            while length > 0:
835
                bi = int(offset / backend.block_size)
836
                bl = min(length, backend.block_size)
837
                if bi < len(hashmap):
838
                    if bl == backend.block_size:
839
                        hashmap[bi] = src_hashmap[sbi]
840
                    else:
841
                        data = backend.get_block(src_hashmap[sbi])
842
                        hashmap[bi] = backend.update_block(hashmap[bi], data[:bl], 0)
843
                else:
844
                    hashmap.append(src_hashmap[sbi])
845
                offset += bl
846
                length -= bl
847
                sbi += 1
848
        else:
849
            data = ''
850
            sbi = 0
851
            while length > 0:
852
                data += backend.get_block(src_hashmap[sbi])
853
                if length < backend.block_size:
854
                    data = data[:length]
855
                bytes = put_object_block(hashmap, data, offset)
856
                offset += bytes
857
                data = data[bytes:]
858
                length -= bytes
859
                sbi += 1
860
    else:
861
        sock = raw_input_socket(request)
862
        data = ''
863
        for d in socket_read_iterator(sock, length, backend.block_size):
864
            # TODO: Raise 408 (Request Timeout) if this takes too long.
865
            # TODO: Raise 499 (Client Disconnect) if a length is defined and we stop before getting this much data.
866
            data += d
867
            bytes = put_object_block(hashmap, data, offset)
868
            offset += bytes
869
            data = data[bytes:]
870
        if len(data) > 0:
871
            put_object_block(hashmap, data, offset)
821 872
    
822 873
    if offset > size:
823 874
        size = offset
b/pithos/api/util.py
263 263
        raise Conflict(json.dumps(e.data))
264 264
    if public is not None:
265 265
        try:
266
            backend.update_object_public(request.user, v_account, v_container, v_object, public)
266
            backend.update_object_public(request.user, v_account, dest_container, dest_name, public)
267 267
        except NotAllowedError:
268 268
            raise Unauthorized('Access denied')
269 269
        except NameError:

Also available in: Unified diff