-Copyright 2011 GRNET S.A. All rights reserved.
+Copyright 2011-2012 GRNET S.A. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, are permitted provided that the following
Whatever marked as to be determined (**TBD**), should not be considered by implementors.
+More info about Pithos can be found here: https://code.grnet.gr/projects/pithos
+
Document Revisions
^^^^^^^^^^^^^^^^^^
========================= ================================
Revision Description
========================= ================================
-0.8 (Dec 22, 2011) Update allowed versioning values.
+0.8 (Jan 19, 2012) Update allowed versioning values.
\ Change policy/meta formatting in JSON/XML replies.
\ Document that all non-ASCII characters in headers should be URL-encoded.
\ Support metadata-based queries when listing objects at the container level.
\ Note Content-Type issue when using the internal django web server.
\ Add object UUID field.
\ Always reply with the MD5 in the ETag.
+\ Note that ``/login`` will only work if an external authentication system is defined.
0.7 (Nov 21, 2011) Suggest upload/download methods using hashmaps.
\ Propose syncing algorithm.
\ Support cross-account object copy and move.
Pithos Users and Authentication
-------------------------------
-Pithos keeps separate databases for users and objects.
-
-Each user is uniquely identified by the ``Uniq`` field. This should be used as the user's account in the API. The API uses the ``Token`` field to authenticate a user, thus allowing cross-account requests. All API requests require a token.
+In Pithos, each user is uniquely identified by a token. All API requests require a token and each token is internally resolved to an account string. The API uses the account string to identify the user's own files, thus whether a request is local or cross-account.
-User entries can be modified/added via the management interface available at ``https://hostname/admin``.
+Pithos does not keep a user database. For development and testing purposes, user identifiers and their corresponding tokens can be defined in the settings file. However, Pithos is designed with an external authentication service in mind. This service must handle the details of validating user credentials and communicate with Pithos via a middleware software component that, given a token, fills in the internal request account variable.
-Pithos is also compatible with Shibboleth (http://shibboleth.internet2.edu/). The connection between Shibboleth and Pithos is done by ``https://hostname/login``. An application that wishes to connect to Pithos, but does not have a token, should redirect the user to the login URI.
+Client software using Pithos, if not already knowing a user's identifier and token, should forward to the ``/login`` URI. The Pithos server, depending on its configuration will redirect to the appropriate login page.
The login URI accepts the following parameters:
renew Force token renewal (no value parameter)
====================== =========================
-The login process starts by redirecting the user to an external URI (controlled by Shibboleth), where the actual authentication credentials are entered. Then, the user is redirected back to the login URI from Shibboleth, with various identification information in the request headers.
+When done with logging in, the service's login URI should redirect to the URI provided with ``next``, adding ``user`` and ``token`` parameters, which contain the account and token fields respectively.
-If the user does not exist in the database, Pithos adds the user and creates a random token. If the user exists, the token has not expired and ``renew`` is not set, the existing token is reused. Finally, the login URI redirects to the URI provided with ``next``, adding the ``user`` and ``token`` parameters, which contain the ``Uniq`` and ``Token`` fields respectively.
+A user management service that implements a login URI according to these conventions is Astakos (https://code.grnet.gr/projects/astakos), by GRNET.
The Pithos API
--------------
#!/usr/bin/env python
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
#!/usr/bin/env python
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
#!/usr/bin/env python
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
#!/usr/bin/env python
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
r.close()
if not l:
return l
- return l[0]
\ No newline at end of file
+ return l[0]
--- /dev/null
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or
+# without modification, are permitted provided that the following
+# conditions are met:
+#
+# 1. Redistributions of source code must retain the above
+# copyright notice, this list of conditions and the following
+# disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above
+# copyright notice, this list of conditions and the following
+# disclaimer in the documentation and/or other materials
+# provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY GRNET S.A. ``AS IS'' AND ANY EXPRESS
+# OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL GRNET S.A OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
+# AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
+# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+#
+# The views and conclusions contained in the software and
+# documentation are those of the authors and should not be
+# interpreted as representing official policies, either expressed
+# or implied, of GRNET S.A.
+
+import logging
+
+from django.http import HttpResponseNotFound, HttpResponseRedirect
+from django.utils.http import urlencode
+from django.conf import settings
+
+
+logger = logging.getLogger(__name__)
+
+
+def redirect_to_login_service(request):
+ users = settings.AUTHENTICATION_USERS
+ host = settings.AUTHENTICATION_HOST
+ if users is not None or host is None:
+ return HttpResponseNotFound()
+
+ if request.is_secure():
+ proto = 'https://'
+ else:
+ proto = 'http://'
+ next = request.GET.get('next', '')
+ params = {'next': next}
+ renew = 'renew' in request.GET
+ if renew:
+ params['renew'] = True
+ uri = proto + host + '/login?' + urlencode(params)
+ return HttpResponseRedirect(uri)
\ No newline at end of file
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
-#
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
+#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# conditions are met:
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
-#
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
+#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# conditions are met:
urlpatterns = patterns('',
- (r'v1(?:$|/)', include('pithos.api.urls')),
- (r'v1\.0(?:$|/)', include('pithos.api.urls')),
- (r'public/(?P<v_public>.+?)/?$', 'pithos.api.public.public_demux'),
- (r'ui', include('pithos.ui.urls'))
+ (r'^v1(?:$|/)', include('pithos.api.urls')),
+ (r'^v1\.0(?:$|/)', include('pithos.api.urls')),
+ (r'^public/(?P<v_public>.+?)/?$', 'pithos.api.public.public_demux'),
+ (r'^login/?$', 'pithos.api.login.redirect_to_login_service'),
+ (r'^ui', include('pithos.ui.urls'))
)
-# Copyright 2011 GRNET S.A. All rights reserved.
-#
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
+#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# conditions are met:
warnings.filters = original_filters
return backend
-
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# or implied, of GRNET S.A.
from dbworker import DBWorker
-from sqlalchemy import Table, Column, String, Integer, MetaData
-from sqlalchemy.sql import select
+from sqlalchemy import Table, Column, String, Integer, Boolean, MetaData
+from sqlalchemy.sql import and_, select
from sqlalchemy.schema import Index
metadata = MetaData()
columns=[]
columns.append(Column('public_id', Integer, primary_key=True))
- columns.append(Column('path', String(2048)))
+ columns.append(Column('path', String(2048), nullable=False))
+ columns.append(Column('active', Boolean, nullable=False, default=True))
self.public = Table('public', metadata, *columns, mysql_engine='InnoDB', sqlite_autoincrement=True)
# place an index on path
Index('idx_public_path', self.public.c.path, unique=True)
metadata.create_all(self.engine)
def public_set(self, path):
- s = self.public.select()
+ s = select([self.public.c.public_id])
s = s.where(self.public.c.path == path)
r = self.conn.execute(s)
- public = r.fetchall()
+ row = r.fetchone()
r.close()
- if len(public) == 0:
+ if row:
+ s = self.public.update().where(self.public.c.public_id == row[0])
+ s = s.values(active=True)
+ else:
s = self.public.insert()
- r = self.conn.execute(s, path = path)
- r.close()
+ s = s.values(path=path, active=True)
+ r = self.conn.execute(s)
+ r.close()
def public_unset(self, path):
- s = self.public.delete().where(self.public.c.path == path)
+ s = self.public.update()
+ s = s.where(self.public.c.path == path)
+ s = s.values(active=False)
r = self.conn.execute(s)
r.close()
def public_get(self, path):
- s = select([self.public.c.public_id], self.public.c.path == path)
+ s = select([self.public.c.public_id])
+ s = s.where(and_(self.public.c.path == path,
+ self.public.c.active == True))
r = self.conn.execute(s)
row = r.fetchone()
r.close()
return None
def public_path(self, public):
- s = select([self.public.c.path], self.public.c.public_id == public)
+ s = select([self.public.c.path])
+ s = s.where(and_(self.public.c.public_id == public,
+ self.public.c.active == True))
r = self.conn.execute(s)
row = r.fetchone()
r.close()
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
execute(""" create table if not exists public
( public_id integer primary key autoincrement,
- path text ) """)
+ path text not null,
+ active boolean not null default 1 ) """)
execute(""" create unique index if not exists idx_public_path
on public(path) """)
def public_set(self, path):
q = "insert or ignore into public (path) values (?)"
self.execute(q, (path,))
+ q = "update public set active = 1 where path = ?"
+ self.execute(q, (path,))
def public_unset(self, path):
- q = "delete from public where path = ?"
+ q = "update public set active = 0 where path = ?"
self.execute(q, (path,))
def public_get(self, path):
- q = "select public_id from public where path = ?"
+ q = "select public_id from public where path = ? and active = 1"
self.execute(q, (path,))
row = self.fetchone()
if row:
return None
def public_path(self, public):
- q = "select path from public where public_id = ?"
+ q = "select path from public where public_id = ? and active = 1"
self.execute(q, (public,))
row = self.fetchone()
if row:
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
@backend_method
def get_uuid(self, user, uuid):
"""Return the (account, container, name) for the UUID given."""
+
logger.debug("get_uuid: %s", uuid)
info = self.node.latest_uuid(uuid)
if info is None:
@backend_method
def get_public(self, user, public):
"""Return the (account, container, name) for the public id given."""
+
logger.debug("get_public: %s", public)
if public is None or public < ULTIMATE_ANSWER:
raise NameError
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
#!/usr/bin/env python
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
#!/usr/bin/env python
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
#!/usr/bin/env python
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
#!/usr/bin/env python
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
#!/usr/bin/env python
#coding=utf8
-# Copyright 2011 GRNET S.A. All rights reserved.
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
-# Copyright 2011 GRNET S.A. All rights reserved.
+#!/usr/bin/env python
+
+# Copyright 2011-2012 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
],
},
)
-
projects / pithos / commitdiff