Revision dfb1fafa
| /dev/null | ||
|---|---|---|
| 1 |
Copyright 2011-2012 GRNET S.A. All rights reserved. |
|
| 2 |
|
|
| 3 |
Redistribution and use in source and binary forms, with or |
|
| 4 |
without modification, are permitted provided that the following |
|
| 5 |
conditions are met: |
|
| 6 |
|
|
| 7 |
1. Redistributions of source code must retain the above |
|
| 8 |
copyright notice, this list of conditions and the following |
|
| 9 |
disclaimer. |
|
| 10 |
|
|
| 11 |
2. Redistributions in binary form must reproduce the above |
|
| 12 |
copyright notice, this list of conditions and the following |
|
| 13 |
disclaimer in the documentation and/or other materials |
|
| 14 |
provided with the distribution. |
|
| 15 |
|
|
| 16 |
THIS SOFTWARE IS PROVIDED BY GRNET S.A. ``AS IS'' AND ANY EXPRESS |
|
| 17 |
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED |
|
| 18 |
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR |
|
| 19 |
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL GRNET S.A OR |
|
| 20 |
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
|
| 21 |
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
|
| 22 |
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF |
|
| 23 |
USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED |
|
| 24 |
AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT |
|
| 25 |
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN |
|
| 26 |
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE |
|
| 27 |
POSSIBILITY OF SUCH DAMAGE. |
|
| 28 |
|
|
| 29 |
The views and conclusions contained in the software and |
|
| 30 |
documentation are those of the authors and should not be |
|
| 31 |
interpreted as representing official policies, either expressed |
|
| 32 |
or implied, of GRNET S.A. |
|
| /dev/null | ||
|---|---|---|
| 1 |
README |
|
| 2 |
====== |
|
| 3 |
|
|
| 4 |
Pithos is a file storage service, built by GRNET using Django (https://www.djangoproject.com/). |
|
| 5 |
Learn more about Pithos at: http://code.grnet.gr/projects/pithos |
|
| 6 |
Consult LICENSE for licensing information. |
|
| 7 |
|
|
| 8 |
Documentation |
|
| 9 |
------------- |
|
| 10 |
|
|
| 11 |
All docs are in the docs/source directory. The .rst files are perfectly readable in source form. |
|
| 12 |
|
|
| 13 |
To build the documentation you need to have Sphinx (http://sphinx.pocoo.org/) installed. |
|
| 14 |
|
|
| 15 |
On a typical debian-based Linux system run: |
|
| 16 |
apt-get install python-django python-setuptools python-sphinx |
|
| 17 |
apt-get install python-sqlalchemy python-mysqldb python-psycopg2 |
|
| 18 |
|
|
| 19 |
Then run: |
|
| 20 |
python setup.py build_sphinx |
|
| 21 |
|
|
| 22 |
The documentation will be built in the docs/build/html directory. |
|
| 23 |
|
|
| 24 |
Also run: |
|
| 25 |
python setup.py build_sphinx -b text |
|
| 26 |
|
|
| 27 |
Then find the plain text version of the docs in docs/build/text. |
|
| 28 |
|
|
| 29 |
Running the server |
|
| 30 |
------------------ |
|
| 31 |
|
|
| 32 |
Make sure you have all required packages installed: |
|
| 33 |
apt-get install python-django python-setuptools python-sphinx |
|
| 34 |
apt-get install python-sqlalchemy python-mysqldb python-psycopg2 |
|
| 35 |
|
|
| 36 |
Then run: |
|
| 37 |
python manage.py syncdb |
|
| 38 |
python manage.py runserver |
|
| 39 |
|
|
| 40 |
This server is useful during development, but should not be used for deployment. |
|
| 41 |
To deploy Pithos using Apache, take a look at the Administrator Guide in docs. |
|
| 42 |
|
|
| 43 |
Using the tools |
|
| 44 |
--------------- |
|
| 45 |
|
|
| 46 |
In the pithos/tools directory you will find the following utilities: |
|
| 47 |
|
|
| 48 |
pithos-sh Pithos shell |
|
| 49 |
pithos-sync Pithos synchronization client |
|
| 50 |
pithos-fs Pithos FUSE implementation |
|
| 51 |
pithos-test Pithos server tests |
|
| 52 |
|
|
| 53 |
Also, the pithos/lib directory contains a python library that can be used |
|
| 54 |
to access Pithos and manage stored objects. All tools use the included lib. |
|
| 55 |
|
|
| 56 |
Connection options can be set via environmental variables: |
|
| 57 |
|
|
| 58 |
PITHOS_USER Login user |
|
| 59 |
PITHOS_AUTH Login token |
|
| 60 |
PITHOS_SERVER Pithos server (default: plus.pithos.grnet.gr) |
|
| 61 |
PITHOS_API Pithos server path (default: v1) |
|
| 62 |
PITHOS_SYNC_CONTAINER Container to sync with (default: pithos) |
|
| /dev/null | ||
|---|---|---|
| 1 |
UPGRADE |
|
| 2 |
======= |
|
| 3 |
|
|
| 4 |
0.7.9 -> 0.7.10 |
|
| 5 |
--------------- |
|
| 6 |
* Update settings.py (BACKEND_*, SERVICE_NAME, *_EMAIL, *_TARGET, IM_*) |
|
| 7 |
* Update 'attributes' table in mysql (backend): |
|
| 8 |
|
|
| 9 |
mysql> update attributes set `key`='ETag' where `key`='hash'; |
|
| 10 |
|
|
| 11 |
* Upgrade 'im_user' table (im app): |
|
| 12 |
|
|
| 13 |
ALTER TABLE im_user ADD COLUMN 'password' VARCHAR(255); |
|
| 14 |
|
|
| 15 |
0.7.10 -> 0.8.0 |
|
| 16 |
--------------- |
|
| 17 |
* Upgrade 'public' table in mysql (backend): |
|
| 18 |
* Run: mysqldump pithosdb public > public-table.sql |
|
| 19 |
* mysql> drop table public; |
|
| 20 |
* Update the codebase and run the server so the new public table is created |
|
| 21 |
* From the sql dump above, take the row: |
|
| 22 |
|
|
| 23 |
INSERT INTO `public` VALUES (...); |
|
| 24 |
|
|
| 25 |
Rewrite as: |
|
| 26 |
|
|
| 27 |
INSERT INTO `public`(`path`) VALUES (...); |
|
| 28 |
|
|
| 29 |
And execute in the database |
|
| 30 |
* Create settings.local with local setting overrides |
|
| 31 |
* Install python-django-south |
|
| 32 |
* Setup south: |
|
| 33 |
python manage.py syncdb |
|
| 34 |
python manage.py migrate im 0001 --fake |
|
| 35 |
python manage.py migrate im |
|
| 36 |
|
|
| 37 |
0.8.0 -> 0.8.1 |
|
| 38 |
-------------- |
|
| 39 |
* Reset 'policy' table in mysql (backend): |
|
| 40 |
|
|
| 41 |
mysql> update policy set `value`='auto' where `key`='versioning'; |
|
| 42 |
|
|
| 43 |
0.8.1 -> 0.8.2 |
|
| 44 |
-------------- |
|
| 45 |
* Add the 'X-Forwarded-Protocol' header directive in the apache configuration, as described in the admin guide |
|
| 46 |
* Update 'attributes' table in mysql (backend): |
|
| 47 |
|
|
| 48 |
mysql> CREATE TABLE `attributes_new` ( |
|
| 49 |
`serial` int(11) NOT NULL, |
|
| 50 |
`domain` varchar(255) COLLATE utf8_bin NOT NULL, |
|
| 51 |
`key` varchar(255) COLLATE utf8_bin NOT NULL, |
|
| 52 |
`value` varchar(255) COLLATE utf8_bin DEFAULT NULL, |
|
| 53 |
PRIMARY KEY (`serial`,`domain`,`key`), |
|
| 54 |
CONSTRAINT FOREIGN KEY (`serial`) REFERENCES `versions` (`serial`) ON DELETE CASCADE ON UPDATE CASCADE |
|
| 55 |
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; |
|
| 56 |
mysql> insert into attributes_new select `serial`, 'pithos', `key`, `value` from attributes; |
|
| 57 |
mysql> drop table attributes; |
|
| 58 |
mysql> alter table attributes_new rename to attributes; |
|
| 59 |
|
|
| 60 |
* Update 'versions' table in mysql (backend): |
|
| 61 |
|
|
| 62 |
mysql> create temporary table tmp_uuids as select distinct node, uuid() as `uuid` from versions; |
|
| 63 |
mysql> alter table versions add column `uuid` varchar(64) DEFAULT '' NOT NULL after `muser`; |
|
| 64 |
mysql> update versions v, tmp_uuids u set v.`uuid` = u.`uuid` where v.`node` = u.`node`; |
|
| 65 |
mysql> create index idx_versions_node_uuid on versions(uuid); |
|
| 66 |
|
|
| 67 |
0.8.2 -> 0.9.0 |
|
| 68 |
-------------- |
|
| 69 |
* No upgrade path provided. Please reinstall and reconfigure. |
|
| b/snf-pithos-app/COPYRIGHT | ||
|---|---|---|
| 1 |
Copyright 2011 GRNET S.A. All rights reserved. |
|
| 1 |
Copyright 2011-2012 GRNET S.A. All rights reserved.
|
|
| 2 | 2 |
|
| 3 | 3 |
Redistribution and use in source and binary forms, with or |
| 4 | 4 |
without modification, are permitted provided that the following |
| b/snf-pithos-app/README | ||
|---|---|---|
| 1 |
README |
|
| 2 |
====== |
|
| 3 |
|
|
| 4 |
Pithos is a file storage service, built by GRNET using Django (https://www.djangoproject.com/). |
|
| 5 |
Learn more about Pithos at: http://code.grnet.gr/projects/pithos |
|
| 6 |
|
|
| 7 |
Consult COPYRIGHT for licensing information. |
|
| 8 |
|
|
| 9 |
About Pithos application |
|
| 10 |
------------------------ |
|
| 11 |
|
|
| 12 |
This package contains the Django application that implements the Pithos v2 API. It depends on the Pithos backend, which does the actual data and metadata operations. |
|
| 13 |
|
|
| 14 |
How to run |
|
| 15 |
---------- |
|
| 16 |
|
|
| 17 |
Use snf-webproject to run Pithos automatically. |
|
| 18 |
|
|
| 19 |
To use Pithos in a custom Django project, just add ``pithos.api`` to ``INSTALLED_APPS``. |
|
| 20 |
|
|
| 21 |
Settings |
|
| 22 |
-------- |
|
| 23 |
|
|
| 24 |
Configure in ``settings.py`` or a ``.conf`` file in ``/etc/synnefo`` if using snf-webproject. |
|
| 25 |
|
|
| 26 |
=============================== ================================================ ============================================================ |
|
| 27 |
Name Default value Description |
|
| 28 |
=============================== ================================================ ============================================================ |
|
| 29 |
PITHOS_AUTHENTICATION_URL \http://127.0.0.1:8000/im/authenticate Astakos API URL |
|
| 30 |
PITHOS_AUTHENTICATION_USERS A dictionary of sample users (token to username) Set to empty or None to disable |
|
| 31 |
PITHOS_BACKEND_DB_MODULE pithos.backends.lib.sqlalchemy |
|
| 32 |
PITHOS_BACKEND_DB_CONNECTION sqlite:////tmp/pithos-backend.db SQLAlchemy database connection string |
|
| 33 |
PITHOS_BACKEND_BLOCK_MODULE pithos.backends.lib.hashfiler |
|
| 34 |
PITHOS_BACKEND_BLOCK_PATH /tmp/pithos-data/ Map and block storage path |
|
| 35 |
PITHOS_BACKEND_QUEUE_MODULE None Use ``pithos.backends.lib.rabbitmq`` to enable |
|
| 36 |
PITHOS_BACKEND_QUEUE_CONNECTION None Format like ``rabbitmq://guest:guest@localhost:5672/pithos`` |
|
| 37 |
PITHOS_BACKEND_QUOTA 50 GB (50 * 1024 ** 3) Default user quota |
|
| 38 |
PITHOS_BACKEND_VERSIONING auto Default versioning policy for containers |
|
| 39 |
=============================== ================================================ ============================================================ |
|
| 40 |
|
|
| 41 |
Administrator functions |
|
| 42 |
----------------------- |
|
| 43 |
|
|
| 44 |
Available as extensions to Django's command-line management utility: |
|
| 45 |
|
|
| 46 |
============ ====================== |
|
| 47 |
Name Description |
|
| 48 |
============ ====================== |
|
| 49 |
storagequota Get/set a user's quota |
|
| 50 |
============ ====================== |
|
| 51 |
|
|
| 1 | 52 |
|
| b/snf-pithos-app/README.upgrade | ||
|---|---|---|
| 1 |
UPGRADE |
|
| 2 |
======= |
|
| 3 |
|
|
| 4 |
0.7.9 -> 0.7.10 |
|
| 5 |
--------------- |
|
| 6 |
* Update settings.py (BACKEND_*, SERVICE_NAME, \*_EMAIL, \*_TARGET, IM_*) |
|
| 7 |
* Update ``attributes`` table in mysql (backend):: |
|
| 8 |
|
|
| 9 |
mysql> update attributes set `key`='ETag' where `key`='hash'; |
|
| 10 |
|
|
| 11 |
* Upgrade ``im_user`` table (im app):: |
|
| 12 |
|
|
| 13 |
ALTER TABLE im_user ADD COLUMN 'password' VARCHAR(255); |
|
| 14 |
|
|
| 15 |
0.7.10 -> 0.8.0 |
|
| 16 |
--------------- |
|
| 17 |
* Upgrade ``public`` table in mysql (backend): |
|
| 18 |
|
|
| 19 |
* Run: mysqldump pithosdb public > public-table.sql |
|
| 20 |
* mysql> drop table public; |
|
| 21 |
* Update the codebase and run the server so the new public table is created |
|
| 22 |
* From the sql dump above, take the row:: |
|
| 23 |
|
|
| 24 |
INSERT INTO `public` VALUES (...); |
|
| 25 |
|
|
| 26 |
Rewrite as:: |
|
| 27 |
|
|
| 28 |
INSERT INTO `public`(`path`) VALUES (...); |
|
| 29 |
|
|
| 30 |
And execute in the database |
|
| 31 |
|
|
| 32 |
* Create settings.local with local setting overrides |
|
| 33 |
* Install python-django-south |
|
| 34 |
* Setup south:: |
|
| 35 |
|
|
| 36 |
python manage.py syncdb |
|
| 37 |
python manage.py migrate im 0001 --fake |
|
| 38 |
python manage.py migrate im |
|
| 39 |
|
|
| 40 |
0.8.0 -> 0.8.1 |
|
| 41 |
-------------- |
|
| 42 |
* Reset ``policy`` table in mysql (backend):: |
|
| 43 |
|
|
| 44 |
mysql> update policy set `value`='auto' where `key`='versioning'; |
|
| 45 |
|
|
| 46 |
0.8.1 -> 0.8.2 |
|
| 47 |
-------------- |
|
| 48 |
* Add the ``X-Forwarded-Protocol`` header directive in the apache configuration, as described in the admin guide |
|
| 49 |
* Update ``attributes`` table in mysql (backend):: |
|
| 50 |
|
|
| 51 |
mysql> CREATE TABLE `attributes_new` ( |
|
| 52 |
`serial` int(11) NOT NULL, |
|
| 53 |
`domain` varchar(255) COLLATE utf8_bin NOT NULL, |
|
| 54 |
`key` varchar(255) COLLATE utf8_bin NOT NULL, |
|
| 55 |
`value` varchar(255) COLLATE utf8_bin DEFAULT NULL, |
|
| 56 |
PRIMARY KEY (`serial`,`domain`,`key`), |
|
| 57 |
CONSTRAINT FOREIGN KEY (`serial`) REFERENCES `versions` (`serial`) ON DELETE CASCADE ON UPDATE CASCADE |
|
| 58 |
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; |
|
| 59 |
mysql> insert into attributes_new select `serial`, 'pithos', `key`, `value` from attributes; |
|
| 60 |
mysql> drop table attributes; |
|
| 61 |
mysql> alter table attributes_new rename to attributes; |
|
| 62 |
|
|
| 63 |
* Update ``versions`` table in mysql (backend):: |
|
| 64 |
|
|
| 65 |
mysql> create temporary table tmp_uuids as select distinct node, uuid() as `uuid` from versions; |
|
| 66 |
mysql> alter table versions add column `uuid` varchar(64) DEFAULT '' NOT NULL after `muser`; |
|
| 67 |
mysql> update versions v, tmp_uuids u set v.`uuid` = u.`uuid` where v.`node` = u.`node`; |
|
| 68 |
mysql> create index idx_versions_node_uuid on versions(uuid); |
|
| 69 |
|
|
| 70 |
0.8.2 -> 0.9.0 |
|
| 71 |
-------------- |
|
| 72 |
* No upgrade path provided. Please reinstall and reconfigure. |
|
| b/snf-pithos-backend/COPYRIGHT | ||
|---|---|---|
| 1 |
Copyright 2011 GRNET S.A. All rights reserved. |
|
| 1 |
Copyright 2011-2012 GRNET S.A. All rights reserved.
|
|
| 2 | 2 |
|
| 3 | 3 |
Redistribution and use in source and binary forms, with or |
| 4 | 4 |
without modification, are permitted provided that the following |
| b/snf-pithos-backend/README | ||
|---|---|---|
| 1 |
README |
|
| 2 |
====== |
|
| 1 | 3 |
|
| 4 |
Pithos is a file storage service, built by GRNET using Django (https://www.djangoproject.com/). |
|
| 5 |
Learn more about Pithos at: http://code.grnet.gr/projects/pithos |
|
| 6 |
|
|
| 7 |
Consult COPYRIGHT for licensing information. |
|
| 8 |
|
|
| 9 |
About Pithos backend |
|
| 10 |
-------------------- |
|
| 11 |
|
|
| 12 |
The backend is the library that handles the actual data storage and metadata management for Pithos. It is required by the Pithos application and can be used by any python script in need of direct access to Pithos storage services. |
|
| b/snf-pithos-tools/COPYRIGHT | ||
|---|---|---|
| 1 |
Copyright 2011 GRNET S.A. All rights reserved. |
|
| 1 |
Copyright 2011-2012 GRNET S.A. All rights reserved.
|
|
| 2 | 2 |
|
| 3 | 3 |
Redistribution and use in source and binary forms, with or |
| 4 | 4 |
without modification, are permitted provided that the following |
| b/snf-pithos-tools/README | ||
|---|---|---|
| 1 |
README |
|
| 2 |
====== |
|
| 1 | 3 |
|
| 4 |
Pithos is a file storage service, built by GRNET using Django (https://www.djangoproject.com/). |
|
| 5 |
Learn more about Pithos at: http://code.grnet.gr/projects/pithos |
|
| 6 |
|
|
| 7 |
Consult COPYRIGHT for licensing information. |
|
| 8 |
|
|
| 9 |
About Pithos tools |
|
| 10 |
------------------ |
|
| 11 |
|
|
| 12 |
This packange contains the following utilities: |
|
| 13 |
|
|
| 14 |
============== ============================= |
|
| 15 |
pithos-sh Pithos shell |
|
| 16 |
pithos-sync Pithos synchronization client |
|
| 17 |
pithos-fs Pithos FUSE implementation |
|
| 18 |
pithos-test Pithos server tests |
|
| 19 |
============== ============================= |
|
| 20 |
|
|
| 21 |
Also, installs the pithos.tools.lib, which contains a python library that can be used to access Pithos and manage stored objects. All tools use the included lib. |
|
| 22 |
|
|
| 23 |
Connection options can be set via environmental variables: |
|
| 24 |
|
|
| 25 |
====================== ========================================================= |
|
| 26 |
PITHOS_USER Login user |
|
| 27 |
PITHOS_TOKEN Login token |
|
| 28 |
PITHOS_URL Pithos server (default: \https://plus.pithos.grnet.gr/v1) |
|
| 29 |
PITHOS_SYNC_CONTAINER Container to sync with (default: pithos) |
|
| 30 |
====================== ========================================================= |
|
Also available in: Unified diff