Statistics
| Branch: | Tag: | Revision:

root / kamaki / clients / pithos.py @ 43ee6ae1

History | View | Annotate | Download (32.9 kB)

# Date Author Comment
6069b53b 12/07/2012 04:05 pm Stavros Sachtouris

Trace errors in cli.commands mechanism-not prpgtd

1d329d27 12/05/2012 07:24 pm Stavros Sachtouris

Better pithos upload progressbar

4c1882ab 12/04/2012 07:35 pm Stavros Sachtouris

add unittests requirment (progress)

d88ba587 12/04/2012 01:17 pm Stavros Sachtouris

Update docs and doc-parsed comments in storage

docs: replace Glance with Plankton
storage: standard sphincs-parsable comments

5fb18128 11/30/2012 06:29 pm Stavros Sachtouris

There are always some typos...

bc223d91 11/30/2012 06:24 pm Stavros Sachtouris

Update comment docs at pithos client

4375e020 11/30/2012 06:06 pm Stavros Sachtouris

Change order of params for get_object_meta

961e6040 11/28/2012 04:54 pm Stavros Sachtouris

Rearrange sections in docs, add developer corner

4067cdaf 11/26/2012 02:12 pm Stavros Sachtouris

Fix sphinx auto-doc syntax warning

2b74ab4a 11/19/2012 06:23 pm Stavros Sachtouris

Unittest parallel server creation

cccff590 11/16/2012 06:17 pm Stavros Sachtouris

Thread limit can be set by user

cad39033 11/16/2012 05:31 pm Stavros Sachtouris

Gradual thread inc/decrease in up/download

Thread inc/decrease is now generic for all clients that might need it
with the method _watch_thread_limit

Algorithm: set a small thread limit and when reached wait for active threads to
finish. Measure avg time. If avg time decreases, increase thread limit...

e9abe82b 11/16/2012 02:10 pm Stavros Sachtouris

Avoid to many threads in upload

22beceb3 11/05/2012 04:35 pm Stavros Sachtouris

Minor improvements in pithos unitesting

f27ed9a0 11/05/2012 03:55 pm Stavros Sachtouris

Sync progressbar to threads, adjust ctrl-c handler

In pithos download/upload ctrl-c is caught when multithreading,
and wait for active threads to finish before exiting

e02728f9 11/02/2012 07:28 pm Stavros Sachtouris

Use standard threading instead of gevent/greenlets

gevent depedency is not needed after that

3dabe5d2 11/02/2012 01:16 pm Stavros Sachtouris

Make all source pep8 compliant

68da89ac 10/31/2012 11:50 am Stavros Sachtouris

Merge branch 'master' of https://code.grnet.gr/git/kamaki

Conflicts:
kamaki/__init__.py
kamaki/cli.py
kamaki/clients/__init__.py
kamaki/clients/astakos.py
kamaki/clients/compute.py
kamaki/clients/cyclades.py
kamaki/clients/image.py
kamaki/clients/pithos.py...

c270fe96 10/26/2012 04:18 pm Stavros Sachtouris

Remove relative imports from all pkgs, fix astakos

astakos imports an anused (and soon deprecated) module

6ce9fc72 10/25/2012 07:37 pm Stavros Sachtouris

Handle connection.close in repeated requests

a62c75d1 10/23/2012 05:54 pm Stavros Sachtouris

Push error raising upload for testing

19f4195f 10/23/2012 05:21 pm Stavros Sachtouris

Raise block upload greenlet failure error

33e3437a 10/15/2012 12:44 pm Stavros Sachtouris

Quota Holder init (no functionality)

e3f5a1f6 09/20/2012 05:31 pm Stavros Sachtouris

Minor fixes

ebdecdc7 09/20/2012 12:26 pm Stavros Sachtouris

Unitest purge, create_by_manifestation, versionlst

22fc09fb 09/19/2012 05:17 pm Stavros Sachtouris

Support range at file (threaded) download

642f1bbd 09/19/2012 04:43 pm Stavros Sachtouris

Add range in stream download

38dc5d2f 09/19/2012 02:06 pm Stavros Sachtouris

Get rid of pithos_sh

The old profecy "there will be only one [Pithos Client]" is finally
fullfilled

a23f6ffe 09/19/2012 01:42 pm Stavros Sachtouris

Kamaki CLI/clients sharers implementation

mimics pithos_sh account sharers

48c3782c 09/18/2012 06:31 pm Stavros Sachtouris

Get rid of pithos_sh hash methods in download

973fbcad 09/18/2012 06:10 pm Stavros Sachtouris

Much faster download resume

Don't preload local file blocks, but load each one dynamically
only before it is downloaded, and compare

624ee36f 09/18/2012 05:29 pm Stavros Sachtouris

Minor CLI download behavior fixes

20c13fcc 09/18/2012 04:58 pm Stavros Sachtouris

Pithos download supports resume (again)

1785ad41 09/18/2012 03:55 pm Stavros Sachtouris

Voodo-fix mysterious header-destroying request bug

0004301f 09/18/2012 02:07 pm Stavros Sachtouris

Fix safe headers/params bug, pithos download bug

9a7efb0d 09/18/2012 12:11 pm Stavros Sachtouris

Greenlet/thread safe header/param requests

Requests can now distinguish between "global" headers/params
and thread-specific.

fbfee225 09/14/2012 03:50 pm Stavros Sachtouris

Refactor pithos+ download

3-modes:
a sequential
b sequential with resume
c asynchronous/parallel
bug: Still can't multi-download correctly

fb0cd49a 09/13/2012 01:28 pm Stavros Sachtouris

Workarround an eventlist random bug in downloading

699d3bb1 09/13/2012 12:47 pm Stavros Sachtouris

Make object-download code readable

64ab4c13 09/12/2012 06:47 pm Stavros Sachtouris

Mild refactoring to download and upload @pithos.py

f0598cb2 09/12/2012 01:21 pm Stavros Sachtouris

Fix gevent bug, move release action to connection

74275b1a 09/12/2012 12:16 pm Stavros Sachtouris

Minor bugfixes

5b263ba2 09/11/2012 06:10 pm Stavros Sachtouris

Implement kamakicon, use it

kamakicon: an httplib/ObjectPool based connection class

b1713259 09/10/2012 04:30 pm Stavros Sachtouris

Minor bugfixes

6ad245d5 09/07/2012 12:51 pm Stavros Sachtouris

Use connection pool after redesigning http connect

HTTPRequestsXxx objects now use connection pooling
Atually, connections are not pooled, but responses are.

Implementation of connection pooling with httplib not ready yet

Pooling tested in uploads

2f749e6e 08/29/2012 05:26 pm Stavros Sachtouris

Major changes in client, fixed setup

Seperate Client to Client and HTTPConnection
The later is put in a separate package kamaki.client.connection
It manages headers, params, method, url and data of a connection
but the actuall http connection is not implemented at this...

8af4cc0b 08/28/2012 01:27 pm Stavros Sachtouris

Imporove meta methods

e92440bd 08/28/2012 12:17 pm Stavros Sachtouris

Minor bug fixes

d804de82 08/24/2012 05:51 pm Stavros Sachtouris

Add more functionalities to dnl/hashmap/versions

download: progressbar control, overide file, resume (default)
all: extra options for conditional retrieval

d726b3d0 08/24/2012 01:46 pm Stavros Sachtouris

Fixed bug with reset_headers

Now headers are reset after each request by default
but there is an option for not reseting headers, in low
level

65c08a69 08/23/2012 07:35 pm Stavros Sachtouris

Versions in download

56f0908a 08/23/2012 07:15 pm Stavros Sachtouris

Improved incremental download, hashmap, versions

a5e0629d 08/23/2012 11:33 am Stavros Sachtouris

Add full pithos+ move in client.pithos

7d420701 08/22/2012 06:25 pm Stavros Sachtouris

Full pithos+ copy in clients.pithos

2dfcddd8 08/21/2012 04:25 pm Stavros Sachtouris

Correct some typos

65a45524 08/09/2012 06:23 pm Stavros Sachtouris

Add arguments/options to upload

manifest is now a seperated method for uploading
old, naive upload is preserved with --unchunked option

a298f2ab 08/09/2012 03:32 pm Stavros Sachtouris

Add options to store_delete

7234a287 08/09/2012 01:28 pm Stavros Sachtouris

Add options to meta

c3659429 08/08/2012 09:15 pm Stavros Sachtouris

Complete store_list is superset of pithos_sh List

50dff4ea 08/08/2012 07:06 pm Stavros Sachtouris

Add options to store_list

435008b6 08/01/2012 05:48 pm Stavros Sachtouris

Asynchronous uploads with gevent-greenlet

Merge with koukis-fix

0aaad5c7 07/30/2012 06:49 pm Stavros Sachtouris

(Almost) complete pithos client unittesting

transfer_encoding still unchecked

f70616fc 07/27/2012 06:09 pm Stavros Sachtouris

Adjust unit-test object_head/get/put/copy +bugs

Missing:
- account_quota/versioning
- container_post with transfer-encoding
- object_put with transfer-encoding
- object_copy with manifest
- adjust object_move/post/delete

16ce7b91 07/26/2012 03:36 pm Stavros Sachtouris

Adjusting account/container opps + bug fixes

c2236544 07/26/2012 12:04 pm Stavros Sachtouris

Fixed bug with large file uploads

6ff87724 07/25/2012 05:00 pm Stavros Sachtouris

Added test_object_delete

89c2e77b 07/25/2012 04:37 pm Stavros Sachtouris

Added objectcopy/move/post _ minor bugfixes

b2796c01 07/24/2012 07:06 pm Stavros Sachtouris

Started testing object_copy +minor fixes

9f74ca46 07/24/2012 05:37 pm Stavros Sachtouris

Added tests for object_put + minor bug fixes

3e544e5b 07/19/2012 06:27 pm Stavros Sachtouris

More testing in storage

Completed testing at container level, started at object level

379cd4bb 07/18/2012 06:58 pm Stavros Sachtouris

Implemented tests (Pithos+, partial, for now)

Used unittest, implemented tests for all account level Pithos+
operations and some container level operations
TBC

edbee9c8 07/18/2012 03:06 pm Stavros Sachtouris

Pretyfication adjustments

Fixed colors in flavor and image error messages
Pretyfied image client code

4adfa919 07/17/2012 02:59 pm Stavros Sachtouris

Full Pithos+ support at object level

Rest calls HEAD GET PUT COPY MOVE POST DELETE at object level fully
support Pithos+ capabilities by adding generic methods. Adjusted
existing methods to take advantage of these
Not thoroughly tested...

17edd3f4 07/16/2012 04:06 pm Stavros Sachtouris

Full pithos+ support at container level

Full pithos+ support with ne container_<REST> methods
Old methods adjusted to use this functionality
No changes on CLI

4fd88feb 07/16/2012 01:49 pm Stavros Sachtouris

Modified pithos method signatures

Introduced generic account_head/post/get methods that the other
more specific methods call

3a9e54b0 07/16/2012 01:31 pm Stavros Sachtouris

Completed full pithos+ support at account level

Added full support for pithos+ headers and params for account
POST based operations, modified existing methods to take
advantage of this change

b758e547 07/16/2012 12:30 pm Stavros Sachtouris

Added full pithos support to list_containers

In pithos, added full support for optional parameters and headers supported
in Pithos API in list_containers() (account get)

6657ec8c 07/16/2012 12:00 pm Stavros Sachtouris

Pithos level extentions added

get_account_info now supports until parameter, and if-(un)modified-since headers
in request

e742badc 07/10/2012 05:09 pm Stavros Sachtouris

Simplified Client code

Added set_header method then replaced code that pushes headers to the query with calls to this much simpler method

d1856abf 07/10/2012 05:09 pm Stavros Sachtouris

Added policy handling

In CLI setversioning, versioning, setquota, quota for account and container
In storage(pithos) set_container_versioning, set_container_quota, set_account_versioning, set_account_quota
also get_*_policy methods are split to get_*_versioning and get_*_quota

eb903ba7 07/10/2012 05:09 pm Stavros Sachtouris

Added set_account_group

Adds a custom group of users

c2867610 07/10/2012 05:09 pm Stavros Sachtouris

Implemented group, delgroup and copy

In CLI:
- added store group to retrieve user grouos of account
- added store delgroup to delete a user group by groupname
- added store copy to copy an object from container/path to destination container/path...

28470086 07/10/2012 05:09 pm Stavros Sachtouris

Implemented setpermitions and delpermitions

In CLI added setpermitions and delpermitions

In pithos added set_object_sharing and shortcut method erase_object_sharing

f49084df 07/10/2012 05:09 pm Stavros Sachtouris

Not optimal append and truncate

In CLI added append and truncate for objects

In storage added update_object which acts as a multifunction. It currently loads the whole file on memory, so it needs optimization

ab474306 07/10/2012 05:09 pm Stavros Sachtouris

Optimised append

In pithos a new append_object is introduced, that appends a file in blocks

561116a6 07/10/2012 05:09 pm Stavros Sachtouris

Optimised truncate

In pithos added truncate_object that truncates and object using pithos-enabled headers

ee62607e 07/10/2012 05:09 pm Stavros Sachtouris

Implemented overwrite

In CLI added store_overwrite

In pithos added overwrite_object which, in case of large updates, overwrites block by block

bcabbc35 07/10/2012 05:09 pm Stavros Sachtouris

Added progress bar

In CLI store_overwrite with prgress bar, added progress bar at store_append
In pithos added generator watcher method support

6de1f262 07/10/2012 05:09 pm Stavros Sachtouris

Fixed and optiomized delmeta for objects

In storage, fixed delmeta which was disfunctional
In pithos added an optimized implementation using the pithos update parameter

44b8928d 07/10/2012 05:09 pm Giorgos Verigakis

Make source PEP8 compliant

b66dfee7 07/10/2012 05:09 pm Stavros Sachtouris

Added delete_container (remove an empty container)

In CLI:
- Added store_delete_container

In storage:
- Added delete_container

0e4bec91 07/10/2012 05:09 pm Stavros Sachtouris

Added set_account_meta

In CLI:
- added store_set_account_meta
- added auxiliary dict_from_args in utils.py

In storage:
- added set_account_meta
- created client/util.py to store auxiliary methods semantically indepented Client
- added auxiliary prefix_keys to client/utils.py...

af3b2b36 07/10/2012 05:09 pm Stavros Sachtouris

Added store_setmeta (pithos API implementation)

-In CLI:
- store_meta for account, container and/or object
- minor fixes to meta

In storage,pithos:
- methods replace_account/container/object_meta in storage (OpenStack API)
- methods set_account/container/object_meta in pithos (Pithos API)...

e6b39366 07/10/2012 05:09 pm Stavros Sachtouris

Added store_delmenta (delete specific meta)

In CLI:
- store_delmeta for account, container, object
In storage,pithos:
- store_account_delmeta
- store_container_delemeta
- store_object_delmeta (is buggy, hope to fix it soon)...
87688514 07/10/2012 05:09 pm Stavros Sachtouris

Added un/publish object

8614814c 05/17/2012 03:36 pm Giorgos Verigakis

Make source PEP8 compliant

c4bff6ac 05/09/2012 01:18 pm Giorgos Verigakis

Do not read more than the given size

6e796ef8 05/09/2012 01:18 pm Nikos Skalkotos

Fix `var not defined' bug introduced in 31231dee4f

36cf04b8 05/09/2012 01:18 pm Giorgos Verigakis

Add missing headers in PUT

b982462f 05/09/2012 01:18 pm Giorgos Verigakis

Send application/octet-stream Content-Type on create_object