Statistics
| Branch: | Tag: | Revision:

root / snf-pithos-backend / pithos / backends / lib @ 329b8349

# Date Author Comment
329b8349 01/29/2014 05:35 pm Sofia Papagiannaki

pithos: Resolve db migration conflict

e0525d86 01/29/2014 04:12 pm Sofia Papagiannaki

pithos: Evaluate "available" during object metadata access

7be22e8d 01/29/2014 04:11 pm Chrysostomos Nanakos

pithos: Use new Archipelago conffile for mapper

876d7486 01/29/2014 04:11 pm Sofia Papagiannaki

pithos: Lazy evaluation of "available" attribute

Whenever the backend tries to access the object's content
checks its database records whether the snapshot map is available or not.
If it is not, it pings Archipelago to check the status of the map.
If the map exists, then it updates the snapshot record in the Pithos...

eaaec098 01/29/2014 04:11 pm Sofia Papagiannaki

pithos: Database migration for "available" attribute.

863d25a5 01/29/2014 04:11 pm Filippos Giannakos

pithos: Close archipelago resource after reading it.

Archipelago implicitly opens a resource after reading the map file. So we should
close it, after successfully reading it.

ba9df843 01/29/2014 04:11 pm Chrysostomos Nanakos

pithos: Use new Archipelago conffile for blocker

f62290bb 01/29/2014 03:53 pm Chrysostomos Nanakos

pithos: pep8 fixes

be433794 01/29/2014 03:53 pm Chrysostomos Nanakos

pithos: Move Xseg Pool initialization

Move Xseg Object Pool initialization to hashfiler module

9b3a95f1 01/29/2014 03:53 pm Chrysostomos Nanakos

pithos: Use new ArchipelagoConfFile constant

Remove BACKEND_ARCHIPELAGO_CONF and use the new ArchipelagoConfFile
introduced in WorkerGlue class.

b20f5e4a 01/29/2014 03:53 pm Chrysostomos Nanakos

pithos: New constants in ModularBackend class

Introduce new constants archipelago_conf_file and xseg_pool_size
in ModularBackend class. The new constant DEFAULT_ARCHIPELAGO_CONF_FILE
is set in the default path '/etc/archipelago/archipelago.conf'.

Create the XSEG Object Pool while initializing ModularBackend class.

cbe81673 01/29/2014 03:52 pm Chrysostomos Nanakos

pithos: pep8 fixes

e7682feb 01/29/2014 03:52 pm Filippos Giannakos

pithos: Do not put xseg request before map retrieval

During map_retr_archipelago, we should not put the xseg request which fetched
the map data, before actually retrieving them.

The ctypes "from_address" method, does not copy the data where the pointer
points. It simply creates a ctypes instance using the given memory. Ctypes...

32293ec0 01/29/2014 03:52 pm Chrysostomos Nanakos

pithos: Add Archipelago Blocker and Mapper

a1557c9c 01/29/2014 03:52 pm Chrysostomos Nanakos

pithos: Remove File - RADOS Blocker and Mapper

Add new constant for Archipelago configuration file in pithos API,
PITHOS_BACKEND_ARCHIPELAGO_CONF

f75f40cb 01/29/2014 03:52 pm Chrysostomos Nanakos

pithos: pep8 fixes

caf6cb1f 01/29/2014 03:46 pm Christos Stavrakakis

Merge branch 'release-0.15' into develop

Conflicts:
snf-deploy/snfdeploy/__init__.py
version

73c37386 01/29/2014 01:41 pm Sofia Papagiannaki

pithos: Database data migration for the plankton properties.

b045ca90 12/16/2013 05:44 pm Sofia Papagiannaki

pithos: Reduce interaction with the database

2c2513fc 12/11/2013 12:20 pm Sofia Papagiannaki

pithos: Fix ordering in detailed listing of public objects

Refs: #4727

ba402621 12/07/2013 11:59 am Sofia Papagiannaki

pithos: Fix ordering in detailed listing of public objects

Refs: #4727

80a548e3 11/11/2013 12:59 pm Christos Stavrakakis

Merge branch 'master' into develop

Conflicts:
Changelog
astakosclient/astakosclient/keypath.py
ci/utils.py
docs/admin-guide.rst
snf-astakos-app/astakos/im/cookie.py
snf-astakos-app/astakos/im/functions.py
snf-astakos-app/astakos/im/quotas.py
snf-astakos-app/astakos/im/views/projects.py...

f759041f 10/29/2013 04:15 pm Ilias Tsitsimpis

snf-pithos-app: Use the new AstakosClient

d3c34119 10/22/2013 07:44 pm Sofia Papagiannaki

pithos: Update/fix sqlite backend module

3d5906fe 10/16/2013 06:21 pm Giorgos Korfiatis

Resolve some undefined names

Add missing imports; remove call to sqlite3_changes; set key and val
in render_meta.

ae0a4858 10/03/2013 02:52 pm Nanakos Chrysostomos

pithos: Fix latest version list with specific timestamp

Return the sorted list of the latest attributes keys

87835e94 10/02/2013 11:27 pm Nanakos Chrysostomos

pithos: Fix timestamps

4a7b190f 09/30/2013 03:25 pm Nanakos Chrysostomos

pithos: Optimize latest_attribute_keys

pep8 fixes

b83df225 09/23/2013 09:49 pm Nanakos Chrysostomos

pithos: Optimize xfeature_get_bulk query

e98c5561 09/19/2013 05:46 pm Nanakos Chrysostomos

pithos: Further optimize latest_version_list query

76da16c3 09/19/2013 04:05 pm Nanakos Chrysostomos

pithos: Optimize latest_version_list query

7fef13f0 09/19/2013 12:30 pm Nanakos Chrysostomos

pithos: Optimize statistics_latest subquery

dc88754b 09/16/2013 04:19 pm Nanakos Chrysostomos

pithos: Performance Optimizations in Object Listing.

de8fb87d 09/02/2013 03:40 pm Sofia Papagiannaki

pithos: Destroy test database after test execution
:
Pithos uses a pool for the backend objects.
Pooled backend objects override the default close behavior
and return the object to the pool instead.
Therefore, django was unable to destroy the database
since there were active connections....

6e0f3e65 08/08/2013 05:40 pm Sofia Papagiannaki

pithos: Fix shared listing for implicitly shared objects

Provide also the appropriate tests

Refs: #4131

057c0977 08/07/2013 11:33 am Christos Stavrakakis

Merge branch 'hotfix-0.14.5' into develop

Conflicts:
snf-pithos-backend/pithos/backends/modular.py
version

d2fc71c9 08/05/2013 05:02 pm Sofia Papagiannaki

pithos: Fix access object version data/metadata

adce84cd 08/02/2013 08:02 pm Sofia Papagiannaki

pithos: Fix TypeError when trying to retrieve invalid object version

d4e4e501 07/25/2013 12:46 pm Christos Stavrakakis

Merge branch 'hotfix-0.14.3' into develop

Conflicts:
Changelog
snf-astakos-app/astakos/im/messages.py
snf-pithos-app/pithos/api/manage_accounts/__init__.py
snf-pithos-app/pithos/api/management/commands/reconcile-resources-pithos.py
snf-pithos-app/pithos/api/settings.py...

96090728 07/22/2013 06:40 pm Sofia Papagiannaki

pithos: Fix line continuation format

2ff02341 07/22/2013 06:19 pm Sofia Papagiannaki

pithos: Fix reconcile-resource-pithos command

  • Improved performance (reduce interaction with the database)
  • Fix command when `userid` option is provided

Conflicts:
snf-pithos-app/pithos/api/management/commands/reconcile-resources-pithos.py
snf-pithos-backend/pithos/backends/lib/sqlite/node.py

fbafd482 07/17/2013 07:06 pm Sofia Papagiannaki

pithos: Revert pep8 changes from sqlalchemy ClauseElements

pep8 complains for comparison with boolean in sqlalchemy.sql.expression.ClauseElement
with "E712 comparison to True should be 'if cond is True:' or 'if cond:'"
but sqlalchemy does not understand it otherwise.

72e191e5 07/17/2013 11:41 am Sofia Papagiannaki

pithos: Fix migrations

Some data migrations require to import pithos app settings.
These migrations used to fail in nodes where snf-pithos-app
is not installed.
Disable these migrations if they fail to import the necessary settings.

29148653 07/17/2013 11:41 am Sofia Papagiannaki

pithos: pep8 fixes

57ccbdbf 07/15/2013 04:01 pm Filippos Giannakos

pithos: Add missing blocksize attribute to blocker.

Blocksize was undefined for the blocker that combines files and rados objecsts.
Define it, according to the block parameters.

890f3d4c 07/12/2013 01:22 pm Christos Stavrakakis

Merge branch 'hotfix-0.14.2' into develop

Conflicts:
Changelog
snf-astakos-app/astakos/im/messages.py
snf-astakos-app/astakos/im/views/target/__init__.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/logic/management/commands/reconcile-servers.py...

9f5c8386 07/11/2013 06:22 pm Sofia Papagiannaki

pithos: Fix attribute copy

The newly created attributed used to point in the source node
instead of the destination

Refs: 3510

762baa68 07/11/2013 02:56 pm Christos Stavrakakis

pithos: fix typo

18d46d23 07/11/2013 01:55 pm Sofia Papagiannaki

pithos backend: Extend list domain objects method

This method optionally accepts a user paramter and used to return
the objects shared to the user and having a specific domain.
This commit extends this functionality and returns also the objects
owned by the user and having the specific domain...

4100e0ee 07/11/2013 01:55 pm Sofia Papagiannaki

pithos: Update get objects in a specific domain

Improve performance

Refs: #3510

8b365874 07/11/2013 01:55 pm Sofia Papagiannaki

pithos: improve performance of get_domain_list

Before it used to query the database
for all the objects having the specific domain
and finally filtered out the objects to which the user had no access.
Now gets first the allowed_paths for the user and then...

a63f36a2 07/09/2013 03:54 pm Sofia Papagiannaki

pithos backend: Extend list domain objects method

This method optionally accepts a user paramter and used to return
the objects shared to the user and having a specific domain.
This commit extends this functionality and returns also the objects
owned by the user and having the specific domain...

0534576c 07/05/2013 06:52 pm Sofia Papagiannaki

pithos: Restore sqlite backend module

ad9ada51 07/05/2013 01:24 pm Sofia Papagiannaki

pithos: Update get objects in a specific domain

Improve performance

Refs: #3510

a5b0519c 07/04/2013 12:51 pm Georgios D. Tsoukalas

pithos: fix syntax in sqlite backend

ce264c16 06/28/2013 09:27 pm Christos Stavrakakis

Merge branch 'hotfix-0.14.2' into develop

Conflicts:
snf-astakos-app/astakos/im/views/projects.py
snf-cyclades-app/synnefo/logic/backend.py
snf-django-lib/snf_django/lib/astakos.py
version

83a3723e 06/28/2013 12:28 pm Sofia Papagiannaki

pithos: fix empty account usage computation

Usage computation for empty accounts used to fail with:
TypeError: 'NoneType' object is unsubscriptable

7736e11a 06/28/2013 12:20 pm Sofia Papagiannaki

pithos: improve performance of get_domain_list

Before it used to query the database
for all the objects having the specific domain
and finally filtered out the objects to which the user had no access.
Now gets first the allowed_paths for the user and then...

2e8edd42 06/17/2013 06:02 pm Sofia Papagiannaki

pithos: update pithos statistics computation

Compute account statistics instead of storing them in the database
Fix container statistics computation
Related commit: 3a9f1f113c1e2c853b7ae223fbcbb843d19f7f95

985b9b09 06/17/2013 06:02 pm Sofia Papagiannaki

pithos: enforce transaction isolation level

Refs: #3582 #3585

0f510652 06/17/2013 06:02 pm Sofia Papagiannaki

pithos: update recursively statistics up to the container level

To achieve atomicity we lock the container path.
However, the write operations update the statistics for the ancestor
nodes (container/account).
Therefore, the backend restricts the recursion up to the container...

f554bdb3 06/07/2013 01:36 pm Giorgos Korfiatis

pithos: Fix ASTAKOS_BASE_URL import in migration

89362181 05/13/2013 05:40 pm Sofia Papagiannaki

pithos: remove dependency from synnefo.lib.queue.Receipt

47b8d4b1 05/13/2013 12:15 pm Christos Stavrakakis

Merge branch 'feature-quotas' into develop

Conflicts:
docs/dev-guide.rst

91114c83 05/09/2013 04:04 pm Sofia Papagiannaki

Merge branch 'tmp-pithos-db-escape-warning' into develop

7a65e284 05/01/2013 03:43 pm Sofia Papagiannaki

pithos: Remove unnecessary logging in public url unset

860c5d9c 05/01/2013 03:28 pm Sofia Papagiannaki

pithos: Remove obsolete settings from database migrations

Use astakosclient to retrieve user information

ae6199c5 05/01/2013 12:00 pm Sofia Papagiannaki

pithos: Update pithos management commands

Refs: #3625

d1e7d2b4 05/01/2013 12:00 pm Sofia Papagiannaki

pithos: Utilize astakosclient for quota handling

Refs: #3625, #3650

8570576b 05/01/2013 11:58 am Sofia Papagiannaki

pithos: extend sqlite library for commission reconciliation

Refs: #3594

235a4227 04/22/2013 01:31 pm Sofia Papagiannaki

pithos: use other db escape string literal character

Refs: #3645

595b2b0d 04/22/2013 10:10 am Sofia Papagiannaki

Add database migration

Provide an alembic migration to add an index
on the attributes.domain field

5576e6dd 04/22/2013 10:10 am Sofia Papagiannaki

Pithos backend: Return objects in a specific domain

Refs: #3510

cf53943b 04/18/2013 05:12 pm Sofia Papagiannaki

Fix trasactionless query in pithos sqlite backend init

Refs: #3580

26837206 04/18/2013 05:12 pm Georgios D. Tsoukalas

fix trasactionless query in pithos backend init

The query was submitted for execution without any prior transaction
setup, which caused:

1. psycopg2 to implicitly start a transaction
2. the new transaction to remain unknown to the sqlalchemy layer
since it was implicit...

e1e37cf9 04/16/2013 11:59 am Sofia Papagiannaki

Fix pithos-reconcile-commissions command

Get pending commissions specifically for pithos

Refs: #3594

613659fc 04/15/2013 07:14 pm Sofia Papagiannaki

pithos: Register commission serials to allow for recoery

Refs: #3594

04a1b675 04/08/2013 06:17 pm Christos Stavrakakis

Move astakos lib from snf-common to snf-django-lib

4a105ce2 03/28/2013 05:21 pm Sofia Papagiannaki

Change pithos public URL implementation

Refs: commit: 56f3c759

Do not include encoded serial in the public URL.

Change PITHOS_PUBLIC_URL_MIN_LENGTH setting to
PITHOS_PUBLIC_URL_SECURITY.

Log public set/unset functions.

56f3c759 03/27/2013 06:51 pm Sofia Papagiannaki

Change pithos public implementation

Public URLs should consist of a random part.
A public URL should be unique and should not be
equal to any deleted public URL.
Finally publishing a public object has no effect.
Therefore, to generate a new public URL for an...

78348987 03/19/2013 01:24 pm Sofia Papagiannaki

Eliminate mutable default arguments

0a92ff85 03/19/2013 11:22 am Sofia Papagiannaki

Fix pithos purging accounting

Refs: #3442
Refs: #1733

Pithos has two different version accounting
policies:
  • free: do not include object history version size
    to the total account usage
  • debit: include object history version size to
    the total account usage...
2bbf1544 03/06/2013 05:44 pm Georgios D. Tsoukalas

Fix UUID bug breaking image list and VM creation

latest_uuid(uuid) returned the latest version with the given uuid.
This is called from pithos backend method get_uuid(),
which is called from plankton backend method get_image().

For the use in get_image() the uuid alone is not enough,...

97341ccb 03/05/2013 05:02 pm Georgios D. Tsoukalas

Fix serious inefficiency in pithos 0.13 migration

As was run on production during migration.

The migration looped over all rows (node, muser) from versions,
and update muser in each one.

However, the tuples (node, muser) are not unique in the table,
and more important, the muser values are much fewer than the nodes...

e4d84803 02/18/2013 10:23 pm Sofia Papagiannaki

Update pithos migration

Display progress

0e1fce89 02/18/2013 06:03 pm Sofia Papagiannaki

Improve pithos migrate

Cache also non existing accounts.

442bc80c 02/14/2013 04:27 pm Sofia Papagiannaki

Fix pithos purging

Error if there were no objects to purge

94bff756 02/14/2013 04:27 pm Sofia Papagiannaki

Provide command for managing pithos duplicate accounts

42ea7e66 02/14/2013 04:26 pm Sofia Papagiannaki

Migrate pithos object modification user

4860a34d 01/29/2013 05:25 pm Filippos Giannakos

Merge branch 'feature-rados' into develop

Conflicts:
snf-pithos-app/README

e3a9d32a 01/29/2013 04:44 pm Filippos Giannakos

Remove automatic RADOS pool creation.

If a specified RADOS pool does not exist, do not try and create it. Throw an
exception instead.

e31e4274 01/28/2013 03:43 pm Sofia Papagiannaki

Change error handling in uuid to displayname pithos path migration.

Stop migraton if it fails to retrieve a uuid/displayname.

2a7276e7 01/24/2013 02:47 pm Sofia Papagiannaki

Expose /user_catalogs service api call

Change migration scripts to use this instead of the user api call

890c2065 01/23/2013 06:51 pm Sofia Papagiannaki

Change feedback & user_catalogs astakos && pithos api calls

  • Substitute ``username`` with ``displayname``
11c079c8 01/23/2013 06:12 pm Filippos Giannakos

Fix exception handling in RADOS blocker/mapper

Handle only KeyError exception in blocker/mapper initialization, and propagate
all other, as they indicate errors.

43411855 01/15/2013 05:06 pm Sofia Papagiannaki

Optionally import rados

7dc78140 01/15/2013 04:01 pm Sofia Papagiannaki

Merge branch 'radosblocker' of https://code.grnet.gr/git/pithos into latest-quota

Conflicts:
snf-pithos-app/pithos/api/util.py
snf-pithos-backend/pithos/backends/modular.py
snf-pithos-backend/pithos/backends/util.py

32454501 01/07/2013 04:13 am Sofia Papagiannaki

Change pithos sharing to work with account uuids

50149cc8 01/03/2013 06:18 pm Sofia Papagiannaki

Migrate username to uuid in pithos permissions

096a7c3b 01/02/2013 11:45 am Sofia Papagiannaki

Fix negative population in statistics table