pithos: Move Xseg Pool initialization
Move Xseg Object Pool initialization to hashfiler module
pithos: pep8 fixes
pithos: Do not put xseg request before map retrieval
During map_retr_archipelago, we should not put the xseg request which fetchedthe map data, before actually retrieving them.
The ctypes "from_address" method, does not copy the data where the pointerpoints. It simply creates a ctypes instance using the given memory. Ctypes...
pithos: Add Archipelago Blocker and Mapper
pithos: Remove File - RADOS Blocker and Mapper
Add new constant for Archipelago configuration file in pithos API,PITHOS_BACKEND_ARCHIPELAGO_CONF
Merge branch 'release-0.15' into develop
Conflicts: snf-deploy/snfdeploy/__init__.py version
pithos: Database data migration for the plankton properties.
pithos: Reduce interaction with the database
pithos: Fix ordering in detailed listing of public objects
Refs: #4727
Merge branch 'master' into develop
Conflicts: Changelog astakosclient/astakosclient/keypath.py ci/utils.py docs/admin-guide.rst snf-astakos-app/astakos/im/cookie.py snf-astakos-app/astakos/im/functions.py snf-astakos-app/astakos/im/quotas.py snf-astakos-app/astakos/im/views/projects.py...
snf-pithos-app: Use the new AstakosClient
pithos: Update/fix sqlite backend module
Resolve some undefined names
Add missing imports; remove call to sqlite3_changes; set key and valin render_meta.
pithos: Fix latest version list with specific timestamp
Return the sorted list of the latest attributes keys
pithos: Fix timestamps
pithos: Optimize latest_attribute_keys
pep8 fixes
pithos: Optimize xfeature_get_bulk query
pithos: Further optimize latest_version_list query
pithos: Optimize latest_version_list query
pithos: Optimize statistics_latest subquery
pithos: Performance Optimizations in Object Listing.
pithos: Destroy test database after test execution:Pithos uses a pool for the backend objects.Pooled backend objects override the default close behaviorand return the object to the pool instead.Therefore, django was unable to destroy the databasesince there were active connections....
pithos: Fix shared listing for implicitly shared objects
Provide also the appropriate tests
Refs: #4131
Merge branch 'hotfix-0.14.5' into develop
Conflicts: snf-pithos-backend/pithos/backends/modular.py version
pithos: Fix access object version data/metadata
pithos: Fix TypeError when trying to retrieve invalid object version
Merge branch 'hotfix-0.14.3' into develop
Conflicts: Changelog snf-astakos-app/astakos/im/messages.py snf-pithos-app/pithos/api/manage_accounts/__init__.py snf-pithos-app/pithos/api/management/commands/reconcile-resources-pithos.py snf-pithos-app/pithos/api/settings.py...
pithos: Fix line continuation format
pithos: Fix reconcile-resource-pithos command
Conflicts: snf-pithos-app/pithos/api/management/commands/reconcile-resources-pithos.py snf-pithos-backend/pithos/backends/lib/sqlite/node.py
pithos: Revert pep8 changes from sqlalchemy ClauseElements
pep8 complains for comparison with boolean in sqlalchemy.sql.expression.ClauseElementwith "E712 comparison to True should be 'if cond is True:' or 'if cond:'" but sqlalchemy does not understand it otherwise.
pithos: Fix migrations
Some data migrations require to import pithos app settings.These migrations used to fail in nodes where snf-pithos-appis not installed.Disable these migrations if they fail to import the necessary settings.
pithos: Add missing blocksize attribute to blocker.
Blocksize was undefined for the blocker that combines files and rados objecsts.Define it, according to the block parameters.
Merge branch 'hotfix-0.14.2' into develop
Conflicts: Changelog snf-astakos-app/astakos/im/messages.py snf-astakos-app/astakos/im/views/target/__init__.py snf-cyclades-app/synnefo/logic/backend.py snf-cyclades-app/synnefo/logic/management/commands/reconcile-servers.py...
pithos: Fix attribute copy
The newly created attributed used to point in the source nodeinstead of the destination
Refs: 3510
pithos: fix typo
pithos backend: Extend list domain objects method
This method optionally accepts a user paramter and used to returnthe objects shared to the user and having a specific domain.This commit extends this functionality and returns also the objectsowned by the user and having the specific domain...
pithos: Update get objects in a specific domain
Improve performance
Refs: #3510
pithos: improve performance of get_domain_list
Before it used to query the databasefor all the objects having the specific domainand finally filtered out the objects to which the user had no access.Now gets first the allowed_paths for the user and then...
pithos: Restore sqlite backend module
pithos: fix syntax in sqlite backend
Conflicts: snf-astakos-app/astakos/im/views/projects.py snf-cyclades-app/synnefo/logic/backend.py snf-django-lib/snf_django/lib/astakos.py version
pithos: fix empty account usage computation
Usage computation for empty accounts used to fail with:TypeError: 'NoneType' object is unsubscriptable
pithos: update pithos statistics computation
Compute account statistics instead of storing them in the databaseFix container statistics computationRelated commit: 3a9f1f113c1e2c853b7ae223fbcbb843d19f7f95
pithos: enforce transaction isolation level
Refs: #3582 #3585
pithos: update recursively statistics up to the container level
To achieve atomicity we lock the container path.However, the write operations update the statistics for the ancestornodes (container/account).Therefore, the backend restricts the recursion up to the container...
pithos: Fix ASTAKOS_BASE_URL import in migration
pithos: remove dependency from synnefo.lib.queue.Receipt
Merge branch 'feature-quotas' into develop
Conflicts: docs/dev-guide.rst
Merge branch 'tmp-pithos-db-escape-warning' into develop
pithos: Remove unnecessary logging in public url unset
pithos: Remove obsolete settings from database migrations
Use astakosclient to retrieve user information
pithos: Update pithos management commands
Refs: #3625
pithos: Utilize astakosclient for quota handling
Refs: #3625, #3650
pithos: extend sqlite library for commission reconciliation
Refs: #3594
pithos: use other db escape string literal character
Refs: #3645
Add database migration
Provide an alembic migration to add an indexon the attributes.domain field
Pithos backend: Return objects in a specific domain
Fix trasactionless query in pithos sqlite backend init
Refs: #3580
fix trasactionless query in pithos backend init
The query was submitted for execution without any prior transactionsetup, which caused:
1. psycopg2 to implicitly start a transaction2. the new transaction to remain unknown to the sqlalchemy layer since it was implicit...
Fix pithos-reconcile-commissions command
Get pending commissions specifically for pithos
pithos: Register commission serials to allow for recoery
Move astakos lib from snf-common to snf-django-lib
Change pithos public URL implementation
Refs: commit: 56f3c759
Do not include encoded serial in the public URL.
Change PITHOS_PUBLIC_URL_MIN_LENGTH setting toPITHOS_PUBLIC_URL_SECURITY.
Log public set/unset functions.
Change pithos public implementation
Public URLs should consist of a random part.A public URL should be unique and should not beequal to any deleted public URL.Finally publishing a public object has no effect.Therefore, to generate a new public URL for an...
Eliminate mutable default arguments
Fix pithos purging accounting
Refs: #3442Refs: #1733
Fix UUID bug breaking image list and VM creation
latest_uuid(uuid) returned the latest version with the given uuid.This is called from pithos backend method get_uuid(),which is called from plankton backend method get_image().
For the use in get_image() the uuid alone is not enough,...
Fix serious inefficiency in pithos 0.13 migration
As was run on production during migration.
The migration looped over all rows (node, muser) from versions,and update muser in each one.
However, the tuples (node, muser) are not unique in the table,and more important, the muser values are much fewer than the nodes...
Update pithos migration
Display progress
Improve pithos migrate
Cache also non existing accounts.
Fix pithos purging
Error if there were no objects to purge
Provide command for managing pithos duplicate accounts
Migrate pithos object modification user
Merge branch 'feature-rados' into develop
Conflicts: snf-pithos-app/README
Remove automatic RADOS pool creation.
If a specified RADOS pool does not exist, do not try and create it. Throw anexception instead.
Change error handling in uuid to displayname pithos path migration.
Stop migraton if it fails to retrieve a uuid/displayname.
Expose /user_catalogs service api call
Change migration scripts to use this instead of the user api call
Change feedback & user_catalogs astakos && pithos api calls
Fix exception handling in RADOS blocker/mapper
Handle only KeyError exception in blocker/mapper initialization, and propagateall other, as they indicate errors.
Optionally import rados
Merge branch 'radosblocker' of https://code.grnet.gr/git/pithos into latest-quota
Conflicts: snf-pithos-app/pithos/api/util.py snf-pithos-backend/pithos/backends/modular.py snf-pithos-backend/pithos/backends/util.py
Change pithos sharing to work with account uuids
Migrate username to uuid in pithos permissions
Fix negative population in statistics table
Provide error feedback during pithos database path migration to container uuid instead of email
Update Astakos API to provider calls for retrieving uuid from the username and vice versa, extend astakos client library (snf-common) and update pithos to use uuids instead of email for account identification
fix pool creation
Merge remote-tracking branch 'origin/dev' into radosblocker
Conflicts: snf-pithos-app/pithos/api/util.py snf-pithos-backend/pithos/backends/lib/hashfiler/blocker.py snf-pithos-backend/pithos/backends/lib/hashfiler/mapper.py snf-pithos-backend/pithos/backends/modular.py...
make rados storage configurable
add support to store blocks on RADOS
wip issue_commision(). no accept or reject yet
Fix undefined variable
Fix sqlalchemy warning: IN-predicate invoked with an empty sequence.
Include version information in aquarium messages