/ - Diff - Pithos - Greek Research and Technology Network's projects

Revision a6eb13e9

     =========================  ================================
     Revision                   Description
     =========================  ================================
 .4 (June 22, 2011)        Support updating/deleting individual metadata with ``POST``.
 .3 (June 14, 2011)        Large object support with ``X-Object-Manifest``.
     \                          Allow for publicly available objects via ``https://hostname/public``.
     \                          Support time-variant account/container listings.
     \                          Add source version when duplicating with PUT/COPY/MOVE.
     \                          Add source version when duplicating with PUT/COPY.
     \                          Request version in object HEAD/GET requests (list versions with GET).
 .2 (May 31, 2011)         Add object meta listing and filtering in containers.
     \                          Include underlying storage characteristics in container meta.
-...
     POST
     """"
     ======================  ============================================
     Request Parameter Name  Value
     ======================  ============================================
     update                  Do not replace metadata (no value parameter)
     ======================  ============================================
+    |
     ====================  ===========================
     Request Header Name   Value
     ====================  ===========================
-...
     No reply content/headers.
     The update operation will overwrite all user defined metadata.
     The operation will overwrite all user defined metadata, except if ``update`` is defined.
     ================  ===============================
     Return Code       Description
-...
     POST
     """"
     ======================  ============================================
     Request Parameter Name  Value
     ======================  ============================================
     update                  Do not replace metadata (no value parameter)
     ======================  ============================================
+    |
     ====================  ================================
     Request Header Name   Value
     ====================  ================================
-...
     No reply content/headers.
     The update operation will overwrite all user defined metadata.
     The operation will overwrite all user defined metadata, except if ``update`` is defined.
     ================  ===============================
     Return Code       Description
-...
     X-Object-Meta-*       Optional user defined metadata
     ====================  ================================
     Refer to ``POST`` for a description of request headers. Metadata is also copied, updated with any values defined.
     No reply content/headers.
     ===========================  ==============================
-...
     POST
     """"
     ======================  ============================================
     Request Parameter Name  Value
     ======================  ============================================
     update                  Do not replace metadata (no value parameter)
     ======================  ============================================
+    |
     ====================  ================================
     Request Header Name   Value
     ====================  ================================
-...
     X-Object-Meta-*       Optional user defined metadata
     ====================  ================================
     The ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``, ``X-Object-Public`` (**TBD**) and ``X-Object-Meta-*`` headers are considered to be user defined metadata. The update operation will overwrite all previous values and remove any keys not supplied.
     The ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``, ``X-Object-Public`` (**TBD**) and ``X-Object-Meta-*`` headers are considered to be user defined metadata. An operation without the ``update`` parameter will overwrite all previous values and remove any keys not supplied. When using ``update`` any metadata with an empty value will be deleted.
     To update an object:
     To update an object's data:
     * Set ``Content-Type`` to ``application/octet-stream``. If ``Content-Type`` has some other value, it will be ignored and only the metadata will be updated.
     * Supply ``Content-Length`` (except if using chunked transfers), ``Content-Type`` and ``Content-Range`` headers.
     * Set ``Content-Type`` to ``application/octet-stream``.
     * Set ``Content-Range`` as specified in RFC2616, with the following differences:
       * Client software MAY omit ``last-byte-pos`` of if the length of the range being transferred is unknown or difficult to determine.
-...
 (Accepted)               The request has been accepted (not a data update)
 (No Content)             The request succeeded (data updated)
 (Length Required)        Missing ``Content-Length`` in the request
 (Range Not Satisfiable)  The supplied range is out of limits or invalid size
 (Range Not Satisfiable)  The supplied range is invalid
     ===========================  ==============================
-...
     * All metadata replies, at all levels, include latest modification information.
     * At all levels, a ``GET`` request may use ``If-Modified-Since`` and ``If-Unmodified-Since`` headers.
     * Container/object lists include all associated metadata if the reply is of type json/xml. Some names are kept to their OOS API equivalents for compatibility.
     * Object metadata allowed, in addition to ``X-Object-Meta-*``: ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``, ``X-Object-Public`` (**TBD**). These are all replaced with every update operation.
     * Object metadata allowed, in addition to ``X-Object-Meta-*``: ``Content-Encoding``, ``Content-Disposition``, ``X-Object-Manifest``, ``X-Object-Public`` (**TBD**). These are all replaced with every update operation, except if using the ``update`` parameter (in which case individual keys can also be deleted). Deleting meta by providing empty values also works when copying/moving an object.
     * Multi-range object GET support as outlined in RFC2616.
     * Object hashmap retrieval through GET and the ``format`` parameter.
     * Partial object updates through POST, using the ``Content-Length``, ``Content-Type``, ``Content-Range`` and ``Transfer-Encoding`` headers.
-...
     * Container/object lists use a ``200`` return code if the reply is of type json/xml. The reply will include an empty json/xml.
     * In headers, dates are formatted according to RFC 1123. In extended information listings, dates are formatted according to ISO 8601.
     * The ``Last-Modified`` header value always reflects the actual latest change timestamp, regardless of time control parameters and version requests. Time precondition checks with ``If-Modified-Since`` and ``If-Unmodified-Since`` headers are applied to this value.
     * A copy/move using ``PUT``/``COPY``/``MOVE`` will always update metadata, keeping all old values except the ones redefined in the request headers.
     * A ``HEAD`` or ``GET`` for an ``X-Object-Manifest`` object, will include modified ``Content-Length`` and ``ETag`` headers, according to the characteristics of the objects under the specified prefix. The ``Etag`` will be the MD5 hash of the corresponding ETags concatenated. In extended container listings there is no metadata processing.
     The Pithos Client
-...
              -H "X-Auth-Token: 0000" \
              https://pithos.dev.grnet.gr/v1/user/pithos?format=json
       It is recommended that extended replies are cached and subsequent requests utilize the ``If-Modified-Since`` header.
     * List metadata keys used by objects in a container
       Will be in the ``X-Container-Object-Meta`` reply header, included in container information or object list (``HEAD`` or ``GET``).

         #                       unauthorized (401),
         #                       badRequest (400)
         meta = get_account_meta(request)
         backend.update_account_meta(request.user, v_account, meta, replace=True)
         meta = get_account_meta(request)
         replace = True
         if 'update' in request.GET:
             replace = False
         backend.update_account_meta(request.user, v_account, meta, replace)
         return HttpResponse(status=202)
     @api_method('GET', format_allowed=True)
-...
         #                       badRequest (400)
         meta = get_container_meta(request)
         replace = True
         if 'update' in request.GET:
             replace = False
         try:
             backend.update_container_meta(request.user, v_account, v_container, meta, replace=True)
             backend.update_container_meta(request.user, v_account, v_container, meta, replace)
         except NameError:
             raise ItemNotFound('Container does not exist')
         return HttpResponse(status=202)

         """Get all prefix-* request headers in a dict. Reformat keys with format_meta_key()."""
         prefix = 'HTTP_' + prefix.upper().replace('-', '_')
         return dict([(format_meta_key(k[5:]), v) for k, v in request.META.iteritems() if k.startswith(prefix)])
         return dict([(format_meta_key(k[5:]), v) for k, v in request.META.iteritems() if k.startswith(prefix) and len(k) > len(prefix)])
     def get_account_meta(request):
         """Get metadata from an account request."""
-...
         meta = get_object_meta(request)
         permissions = get_sharing(request)
         src_version = request.META.get('HTTP_X_SOURCE_VERSION')
         try:
             if move:
                 src_meta = backend.get_object_meta(request.user, v_account, src_container, src_name)
             else:
                 src_meta = backend.get_object_meta(request.user, v_account, src_container, src_name, src_version)
         except NameError, IndexError:
             raise ItemNotFound('Container or object does not exist')
         # Keep previous values of 'Content-Type' (if a new one is absent) and 'hash'.
         if 'Content-Type' in meta and 'Content-Type' in src_meta:
             del(src_meta['Content-Type'])
         for k in ('Content-Type', 'hash'):
             if k in src_meta:
                 meta[k] = src_meta[k]
         src_version = request.META.get('HTTP_X_SOURCE_VERSION')
         try:
             if move:
                 backend.move_object(request.user, v_account, src_container, src_name, dest_container, dest_name, meta, True, permissions)
                 backend.move_object(request.user, v_account, src_container, src_name, dest_container, dest_name, meta, False, permissions)
             else:
                 backend.copy_object(request.user, v_account, src_container, src_name, dest_container, dest_name, meta, True, permissions, src_version)
                 backend.copy_object(request.user, v_account, src_container, src_name, dest_container, dest_name, meta, False, permissions, src_version)
         except NameError, IndexError:
             raise ItemNotFound('Container or object does not exist')
         except ValueError:

         Note that the account level is always valid as it is checked from another subsystem.
         When not replacing metadata, keys with empty values should be deleted.
         The following variables should be available:
             'hash_algorithm': Suggested is 'sha256'
             'block_size': Suggested is 4MB
-...
             """Update the metadata associated with the object.
             Parameters:
                 'meta': Dictionary with metadata to update.
                 'meta': Dictionary with metadata to update
                 'replace': Replace instead of update
             Raises:

             logger.debug("get_object_permissions: %s %s %s", account, container, name)
             path = self._get_objectinfo(account, container, name)[0]
             return self._get_permissions(path)
             perm_path, perms = self._get_permissions(path)
             if path == perm_path:
                 return perms
             return {}
         def update_object_permissions(self, user, account, container, name, permissions):
             """Update the permissions associated with the object."""
-...
             src_version_id, dest_version_id = self._copy_version(path, path, not replace, True)
             for k, v in meta.iteritems():
                 sql = 'insert or replace into metadata (version_id, key, value) values (?, ?, ?)'
                 self.con.execute(sql, (dest_version_id, k, v))
                 if not replace and v == '':
                     sql = 'delete from metadata where version_id = ? and key = ?'
                     self.con.execute(sql, (dest_version_id, k))
                 else:
                     sql = 'insert or replace into metadata (version_id, key, value) values (?, ?, ?)'
                     self.con.execute(sql, (dest_version_id, k, v))
             self.con.commit()
         def _can_read(self, user, path):
-...
             return r, w
         def _get_permissions(self, path):
             sql = 'select read, write from permissions where name = ?'
             c = self.con.execute(sql, (path,))
             # Check for permissions at path or above.
             sql = 'select name, read, write from permissions where ? like name || ?'
             c = self.con.execute(sql, (path, '%'))
             row = c.fetchone()
             if not row:
                 return {}
                 return path, {}
             r, w = row
             name, r, w = row
             if r == '' and w == '':
                 return {'private': True}
             ret = {}
-...
                 ret['write'] = w.split(',')
             if r != '':
                 ret['read'] = r.split(',')
             return ret
             return name, ret
         def _put_permissions(self, path, r, w):
             sql = 'insert or replace into permissions (name, read, write) values (?, ?, ?)'

Also available in: Unified diff

Synnefo » Pithos

Revision a6eb13e9