Statistics
| Branch: | Tag: | Revision:

root / snf-pithos-backend / pithos / backends / modular.py @ 4a7b190f

History | View | Annotate | Download (72.6 kB)

# Date Author Comment
e751cad7 09/20/2013 01:07 pm Nanakos Chrysostomos

Merge branch 'feature-pithos-performance-optimizations-develop' into feature-pithos-performance-optimizations-perf

7fef13f0 09/19/2013 12:30 pm Nanakos Chrysostomos

pithos: Optimize statistics_latest subquery

ed2064f8 09/18/2013 05:17 pm Christos Stavrakakis

Merge branch 'hotfix-0.14.6' into develop

Conflicts:
Changelog
docs/admin-guide.rst
snf-cyclades-app/synnefo/api/management/commands/network-remove.py
snf-cyclades-app/synnefo/api/servers.py
snf-cyclades-app/synnefo/api/tests/servers.py
snf-cyclades-app/synnefo/logic/backend.py...

16b0ed4a 09/17/2013 11:27 am Nanakos Chrysostomos

pithos: Fixed missing _get_permissions_path function.

683d4324 09/16/2013 05:39 pm Sofia Papagiannaki

pithos: Add missing locks

_lookup_container() locks the container path
but it was not called from some write backend methods
that expect the container path to be locked.

In such cases optionally _lookup_object() calls
the _lookup_container() in order to lock the container path.

dc88754b 09/16/2013 04:19 pm Nanakos Chrysostomos

pithos: Performance Optimizations in Object Listing.

a06c276e 09/12/2013 02:34 pm Sofia Papagiannaki

pithos: Acquire locks in alphabetical path order in copy object

d47565d8 09/12/2013 01:09 pm Build Bot

Revert "pithos: Use pithos backend as context manager"

This reverts commit 23b41f6fefa1a066748810c4642e58e18a4b83f9.

23b41f6f 09/12/2013 11:06 am Sofia Papagiannaki

pithos: Use pithos backend as context manager

Substitue pre_exec() and post_exec()

48bb9c89 09/12/2013 10:22 am Sofia Papagiannaki

pithos: Add missing locks

_lookup_container() locks the container path
but it was not called by some write backend methods
which expected the container path to be locked.

In those cases optionally _lookup_object() calls
the _lookup_container() in order to lock the container path....

af395b9c 09/12/2013 10:15 am Sofia Papagiannaki

Revert "pithos: conditional select for update in lookup account"

This reverts commit 8a7ddfb0545348849d99fb686fec51659cbead58.

Select for update is not useful in case of account creation.
If there is no such account, 0 rows are returned
so there are no rows locked.

3a5994a8 09/09/2013 01:59 pm Sofia Papagiannaki

Pithos: Catch invalid hashmap input

Refs: #3745

76ef2478 09/09/2013 01:59 pm Sofia Papagiannaki

pithos: Remove obsolete backend method

f6d499b0 09/04/2013 07:49 pm Sofia Papagiannaki

pithos: Set attr lock_container_path during backend initialization

33af031c 08/08/2013 12:42 pm Sofia Papagiannaki

pithos: Fix delete deleted object

Delete object used to create a new object version flagged as deleted
regarless of the status of the previous version.
However, delete requests of a deleted object
should return 404 (NOT FOUND) response.

b1aca3e6 08/08/2013 12:12 pm Sofia Papagiannaki

pithos: Fix backend logging.

  • Log unicode arguments
  • Do not log get_block return value (too long)
057c0977 08/07/2013 11:33 am Christos Stavrakakis

Merge branch 'hotfix-0.14.5' into develop

Conflicts:
snf-pithos-backend/pithos/backends/modular.py
version

d2fc71c9 08/05/2013 05:02 pm Sofia Papagiannaki

pithos: Fix access object version data/metadata

adce84cd 08/02/2013 08:02 pm Sofia Papagiannaki

pithos: Fix TypeError when trying to retrieve invalid object version

9e3a38bb 07/23/2013 03:33 pm Sofia Papagiannaki

pithos: Do not report size change in move object operation

Refs: #4154

1f96b68d 07/17/2013 11:41 am Sofia Papagiannaki

pithos backend: Introduce debug_method decorator

Substitute debug statements with a backend method decorator
which registers the method execution arguments, the return values
and the exceptions occured.

29148653 07/17/2013 11:41 am Sofia Papagiannaki

pithos: pep8 fixes

18d46d23 07/11/2013 01:55 pm Sofia Papagiannaki

pithos backend: Extend list domain objects method

This method optionally accepts a user paramter and used to return
the objects shared to the user and having a specific domain.
This commit extends this functionality and returns also the objects
owned by the user and having the specific domain...

4100e0ee 07/11/2013 01:55 pm Sofia Papagiannaki

pithos: Update get objects in a specific domain

Improve performance

Refs: #3510

8b365874 07/11/2013 01:55 pm Sofia Papagiannaki

pithos: improve performance of get_domain_list

Before it used to query the database
for all the objects having the specific domain
and finally filtered out the objects to which the user had no access.
Now gets first the allowed_paths for the user and then...

3248f20a 07/10/2013 04:37 pm Sofia Papagiannaki

pithos: Handle backend transaction management

In commit 9f135224edf6e50386dc29e0a35806a92c56a50c
the transaction management has been moved
from the pithos backend to pithos frontend.
This commit updates all modules use pithos backend
to handle transactions manually.

a63f36a2 07/09/2013 03:54 pm Sofia Papagiannaki

pithos backend: Extend list domain objects method

This method optionally accepts a user paramter and used to return
the objects shared to the user and having a specific domain.
This commit extends this functionality and returns also the objects
owned by the user and having the specific domain...

cc62d2ab 07/05/2013 06:53 pm Sofia Papagiannaki

pithos: Cleanup api_method code

ad9ada51 07/05/2013 01:24 pm Sofia Papagiannaki

pithos: Update get objects in a specific domain

Improve performance

Refs: #3510

0df5250a 07/04/2013 10:48 am Sofia Papagiannaki

Merge branch 'tmp-pithos-tests' into feature-pithos-atomic-api

Conflicts:
snf-pithos-app/pithos/api/util.py

133e3fcf 07/03/2013 06:37 pm Sofia Papagiannaki

pithos: return always etag

Refs: #3554

If update md5 is deactivated Pithos does not return an etag header.
Pithos in that case should return the merkle hash as etag

369a7b41 07/02/2013 04:53 pm Sofia Papagiannaki

pithos: Change block size by configuration

This is useful for testing corner cases efficiently.

ff0beb64 07/02/2013 04:23 pm Sofia Papagiannaki

pithos: move transaction managment and quotaholder sync in frontend

Remove backend_method decorator and move logic in the pithos api method

Conflicts:
snf-pithos-app/pithos/api/util.py

7736e11a 06/28/2013 12:20 pm Sofia Papagiannaki

pithos: improve performance of get_domain_list

Before it used to query the database
for all the objects having the specific domain
and finally filtered out the objects to which the user had no access.
Now gets first the allowed_paths for the user and then...

9f135224 06/19/2013 04:24 pm Sofia Papagiannaki

pithos: move transaction managment and quotaholder sync in frontend

Remove backend_method decorator and move logic in the pithos api method

b90584d0 06/19/2013 04:19 pm Sofia Papagiannaki

pithos: introduce lock_container_path api_method argument

8a7ddfb0 06/19/2013 04:19 pm Sofia Papagiannaki

pithos: conditional select for update in lookup account

263d27d9 06/19/2013 04:13 pm Sofia Papagiannaki

pithos: enable api calls to lock explicitly the path

2e8edd42 06/17/2013 06:02 pm Sofia Papagiannaki

pithos: update pithos statistics computation

Compute account statistics instead of storing them in the database
Fix container statistics computation
Related commit: 3a9f1f113c1e2c853b7ae223fbcbb843d19f7f95

985b9b09 06/17/2013 06:02 pm Sofia Papagiannaki

pithos: enforce transaction isolation level

Refs: #3582 #3585

0f510652 06/17/2013 06:02 pm Sofia Papagiannaki

pithos: update recursively statistics up to the container level

To achieve atomicity we lock the container path.
However, the write operations update the statistics for the ancestor
nodes (container/account).
Therefore, the backend restricts the recursion up to the container...

7ee27246 06/04/2013 02:15 pm Georgios D. Tsoukalas

pithos-backend: disable AstakosClient if no URL given

If the backend is initialized with astakos_url=None,
or AstakosClient cannot be imported,
a fake AstakosClient is instantiated which will raise
an AssertionError on any attempt to access it.

Read-only uses of pithos backend, as in pithcat,...

02de6286 05/15/2013 06:18 pm Sofia Papagiannaki

pithos: change resource usage retrieval

Replace ``astakosclient.get_info()`` with ``astakosclient.get_quotas()``

71585c27 05/01/2013 12:00 pm Sofia Papagiannaki

pithos: Update merged code

ae6199c5 05/01/2013 12:00 pm Sofia Papagiannaki

pithos: Update pithos management commands

Refs: #3625

16f2673e 05/01/2013 12:00 pm Sofia Papagiannaki

pithos: Rename backend quotaholder properties

Refer to astakosclient instead of quotaholder

b17e5550 05/01/2013 12:00 pm Giorgos Korfiatis

WIP astakosclient for quotas in pithos

9b264076 05/01/2013 11:58 am Sofia Papagiannaki

pithos:Ensure serial registration before commission

Commit transaction before accepting the commission
and delete registered serials after successful commission.

Refs: #3594

e77b7a99 04/22/2013 10:10 am Sofia Papagiannaki

pithos backend: update get_domain_objects method

Rename and change it to backend method
(add the backend_method decorator)

5576e6dd 04/22/2013 10:10 am Sofia Papagiannaki

Pithos backend: Return objects in a specific domain

Refs: #3510

fed9c5c7 04/17/2013 06:20 pm Sofia Papagiannaki

pithos: Fix container quota violation control

Pithos backend before performing the quota violation checks
creates a new version of object and updates the
usage of the ancestor nodes (the container).

In Postgresql SELECT does see the effects
of previous updates executed within its own transaction...

19ddd41b 04/17/2013 06:17 pm Sofia Papagiannaki

Change pithos default container policy quota to unlimited

Refs: #3543

  • Substitute the PITHOS_BACKEND_QUOTA setting with two
    distinct settings: PITHOS_BACKEND_ACCOUNT_QUOTA &
    PITHOS_BACKEND_CONTAINER_QUOTA
  • Set PITHOS_BACKEND_CONTAINER_QUOTA default value...
613659fc 04/15/2013 07:14 pm Sofia Papagiannaki

pithos: Register commission serials to allow for recoery

Refs: #3594

c77af544 03/28/2013 10:57 pm Sofia Papagiannaki

Change pithos backend DEFAULT_PUBLIC_URL_SECURITY

Change value from 8 to 16, to conform with the default
PITHOS_PUBLIC_URL_SECURITY setting.

4a105ce2 03/28/2013 05:21 pm Sofia Papagiannaki

Change pithos public URL implementation

Refs: commit: 56f3c759

Do not include encoded serial in the public URL.

Change PITHOS_PUBLIC_URL_MIN_LENGTH setting to
PITHOS_PUBLIC_URL_SECURITY.

Log public set/unset functions.

56f3c759 03/27/2013 06:51 pm Sofia Papagiannaki

Change pithos public implementation

Public URLs should consist of a random part.
A public URL should be unique and should not be
equal to any deleted public URL.
Finally publishing a public object has no effect.
Therefore, to generate a new public URL for an...

78348987 03/19/2013 01:24 pm Sofia Papagiannaki

Eliminate mutable default arguments

0a92ff85 03/19/2013 11:22 am Sofia Papagiannaki

Fix pithos purging accounting

Refs: #3442
Refs: #1733

Pithos has two different version accounting
policies:
  • free: do not include object history version size
    to the total account usage
  • debit: include object history version size to
    the total account usage...
b336e6fa 03/11/2013 06:18 pm Georgios D. Tsoukalas

Fix+move HTTP quotaholder client in synnefo.lib

Allow per-service configuration of the (http) quotaholder client.
Kamaki is no longer needed in service (or ganeti) nodes,
because the client has been moved to snf-common.

Also fix the default quotaholder settings for pithos backend to be disabled...

2bbf1544 03/06/2013 05:44 pm Georgios D. Tsoukalas

Fix UUID bug breaking image list and VM creation

latest_uuid(uuid) returned the latest version with the given uuid.
This is called from pithos backend method get_uuid(),
which is called from plankton backend method get_image().

For the use in get_image() the uuid alone is not enough,...

1f3f907f 02/19/2013 12:00 pm Sofia Papagiannaki

Explicitly enabled/disable pithos quotaholder component

Use PITHOS_USE_QUOTAHOLDER setting

8ed3f04c 02/15/2013 01:53 pm Sofia Papagiannaki

Pithos stop unnecessary contacts with quotaholder

Do not issue commission if there is not size change

ccfd4e44 01/29/2013 01:31 pm Sofia Papagiannaki

Pithos: handle quotaholder exception errors

6eb0de11 01/25/2013 04:04 pm Sofia Papagiannaki

Remove trailing space

e20a751d 01/16/2013 02:47 pm Sofia Papagiannaki

Restore container policy check

b8098f77 01/16/2013 02:47 pm Sofia Papagiannaki

Debit pithos object owner not the request user

7f1f0464 01/15/2013 04:46 pm Georgios D. Tsoukalas

fix radosblocker merge, fix some tabs/whitespace

478f763e 01/15/2013 03:34 pm Sofia Papagiannaki

Fix pithos backend external_quota argument initialization

07fcdb96 01/14/2013 02:18 pm Sofia Papagiannaki

Return quotaholder holding in pithos backend

0ed09c7c 01/11/2013 04:07 pm Sofia Papagiannaki

Fix pithos get_account_meta.

Do now show usage if request user is other than the owner

c846fad1 01/11/2013 12:00 am Sofia Papagiannaki

Merge branch 'latest-quota' of https://code.grnet.gr/git/synnefo into latest-quota

Conflicts:
snf-astakos-app/astakos/im/forms.py
snf-astakos-app/astakos/im/functions.py

1dd34bdd 01/07/2013 04:13 am Sofia Papagiannaki

Fix pithos versioning

Do not remove previous object version in case of free versioning policy

fff37615 01/07/2013 04:13 am Sofia Papagiannaki

Fix pithos versioning (better implementation)

56a42b0b 01/02/2013 03:50 pm Sofia Papagiannaki

fix indentation in pithos.backend.modular

3173699c 12/20/2012 10:41 am Sofia Papagiannaki

Pithos backend should send the token to the quotaholder

019d9e19 12/20/2012 02:54 am Sofia Papagiannaki

Pithos issue_commission using user uuid instead of id

bf45cb4a 12/14/2012 04:22 pm Sofia Papagiannaki

Merge branch 'dev' of https://code.grnet.gr/git/pithos into latest-quota

Conflicts:
.gitignore
devtools/fabfile.py
snf-pithos-app/pithos/api/util.py

d62295ba 12/14/2012 03:26 pm Sofia Papagiannaki

Merge branch 'projects' of https://code.grnet.gr/git/astakos into latest-quota

Conflicts:
.gitignore
snf-astakos-app/conf/20-snf-astakos-app-settings.conf

b3119c3e 11/27/2012 05:47 pm Georgios D. Tsoukalas

fix quotaholder imports

b6c16e87 11/20/2012 12:13 pm Sofia Papagiannaki

Intergation with updated quotaholder module

73fbe301 11/12/2012 03:50 pm Sofia Papagiannaki

If PITHOS_QUOTAHOLDER_URL is None do not try to connect to the quotaholder

d3655326 11/12/2012 03:48 pm Sofia Papagiannaki

Fix indent and add logging in case of QuotaError

b2691c21 11/12/2012 01:04 pm Sofia Papagiannaki

If PITHOS_QUOTAHOLDER_URL is None, perform internal pithos quota checking

8313c681 11/12/2012 12:44 pm root

Merge branch 'dev' of https://code.grnet.gr/git/pithos into dev

Conflicts:
snf-pithos-app/pithos/api/util.py
snf-pithos-backend/pithos/backends/modular.py

21a85c0b 11/09/2012 08:15 pm Sofia Papagiannaki

Merge branch 'dev' of https://code.grnet.gr/git/pithos into dev

Conflicts:
snf-pithos-app/pithos/api/dispatch.py
snf-pithos-app/pithos/api/management/commands/storagequota.py
snf-pithos-app/pithos/api/util.py
snf-pithos-backend/pithos/backends/modular.py

b1dadd0e 11/09/2012 08:04 pm Sofia Papagiannaki

By default report size change in object delete/update (do not include older version size)

2d8e6f94 11/09/2012 07:34 pm Sofia Papagiannaki

Send versions as string in resource events queue

9d4502a8 11/09/2012 07:34 pm Sofia Papagiannaki

Include version information in aquarium messages

f75a34ee 11/09/2012 07:34 pm Sofia Papagiannaki

Change QuotaholderSync to QuotaholderSerial

4cfccdd2 11/09/2012 07:34 pm Sofia Papagiannaki

incorporate AMQPClient instead of synnefo.lib.queue methods for sending queue messages

49f44d54 11/09/2012 07:33 pm Sofia Papagiannaki

fix backend variable name

31aa43c8 11/09/2012 07:33 pm Sofia Papagiannaki

fix mispelling

1af183f2 11/09/2012 07:33 pm Sofia Papagiannaki

New database tables to keep track of quota holder commisions

d50ed8d4 11/09/2012 07:33 pm Sofia Papagiannaki

Fix code formatting to conform to the PEP 8 style guide

90311d98 11/09/2012 11:02 am root

rename resource to pithos+.diskspace

a7f7699d 11/09/2012 11:02 am root

make sure backend hasattr 'messages'

ecd01afd 11/09/2012 11:02 am root

fix: initialize per-request quotaholder seriallist

470984e0 11/09/2012 11:02 am root

use astakos resource name in issue_commission

85352a77 11/09/2012 11:02 am root

quotaholder serial handling fix

5578064f 11/09/2012 11:02 am root

accept/reject commission on backend_method deco