History | View | Annotate | Download (26.6 kB)
Unittest parallel server creation
Thread limit can be set by user
Gradual thread inc/decrease in up/download
Thread inc/decrease is now generic for all clients that might need itwith the method _watch_thread_limit
Algorithm: set a small thread limit and when reached wait for active threads tofinish. Measure avg time. If avg time decreases, increase thread limit...
Avoid to many threads in upload
Minor improvements in pithos unitesting
Sync progressbar to threads, adjust ctrl-c handler
In pithos download/upload ctrl-c is caught when multithreading,and wait for active threads to finish before exiting
Use standard threading instead of gevent/greenlets
gevent depedency is not needed after that
Make all source pep8 compliant
Merge branch 'master' of https://code.grnet.gr/git/kamaki
Conflicts: kamaki/__init__.py kamaki/cli.py kamaki/clients/__init__.py kamaki/clients/astakos.py kamaki/clients/compute.py kamaki/clients/cyclades.py kamaki/clients/image.py kamaki/clients/pithos.py...
Remove relative imports from all pkgs, fix astakos
astakos imports an anused (and soon deprecated) module
Handle connection.close in repeated requests
Push error raising upload for testing
Raise block upload greenlet failure error
Quota Holder init (no functionality)
Minor fixes
Unitest purge, create_by_manifestation, versionlst
Support range at file (threaded) download
Add range in stream download
Get rid of pithos_sh
The old profecy "there will be only one [Pithos Client]" is finallyfullfilled
Kamaki CLI/clients sharers implementation
mimics pithos_sh account sharers
Get rid of pithos_sh hash methods in download
Much faster download resume
Don't preload local file blocks, but load each one dynamicallyonly before it is downloaded, and compare
Minor CLI download behavior fixes
Pithos download supports resume (again)
Voodo-fix mysterious header-destroying request bug
Fix safe headers/params bug, pithos download bug
Greenlet/thread safe header/param requests
Requests can now distinguish between "global" headers/paramsand thread-specific.
Refactor pithos+ download
3-modes:a sequentialb sequential with resumec asynchronous/parallelbug: Still can't multi-download correctly
Workarround an eventlist random bug in downloading
Make object-download code readable
Mild refactoring to download and upload @pithos.py
Fix gevent bug, move release action to connection
Minor bugfixes
Implement kamakicon, use it
kamakicon: an httplib/ObjectPool based connection class
Use connection pool after redesigning http connect
HTTPRequestsXxx objects now use connection poolingAtually, connections are not pooled, but responses are.
Implementation of connection pooling with httplib not ready yet
Pooling tested in uploads
Major changes in client, fixed setup
Seperate Client to Client and HTTPConnectionThe later is put in a separate package kamaki.client.connectionIt manages headers, params, method, url and data of a connectionbut the actuall http connection is not implemented at this...
Imporove meta methods
Minor bug fixes
Add more functionalities to dnl/hashmap/versions
download: progressbar control, overide file, resume (default)all: extra options for conditional retrieval
Fixed bug with reset_headers
Now headers are reset after each request by defaultbut there is an option for not reseting headers, in lowlevel
Versions in download
Improved incremental download, hashmap, versions
Add full pithos+ move in client.pithos
Full pithos+ copy in clients.pithos
Correct some typos
Add arguments/options to upload
manifest is now a seperated method for uploadingold, naive upload is preserved with --unchunked option
Add options to store_delete
Add options to meta
Complete store_list is superset of pithos_sh List
Add options to store_list
Asynchronous uploads with gevent-greenlet
Merge with koukis-fix
(Almost) complete pithos client unittesting
transfer_encoding still unchecked
Adjust unit-test object_head/get/put/copy +bugs
Missing:- account_quota/versioning- container_post with transfer-encoding- object_put with transfer-encoding- object_copy with manifest- adjust object_move/post/delete
Adjusting account/container opps + bug fixes
Fixed bug with large file uploads
Added test_object_delete
Added objectcopy/move/post _ minor bugfixes
Started testing object_copy +minor fixes
Added tests for object_put + minor bug fixes
More testing in storage
Completed testing at container level, started at object level
Implemented tests (Pithos+, partial, for now)
Used unittest, implemented tests for all account level Pithos+operations and some container level operationsTBC
Pretyfication adjustments
Fixed colors in flavor and image error messagesPretyfied image client code
Full Pithos+ support at object level
Rest calls HEAD GET PUT COPY MOVE POST DELETE at object level fullysupport Pithos+ capabilities by adding generic methods. Adjustedexisting methods to take advantage of theseNot thoroughly tested...
Full pithos+ support at container level
Full pithos+ support with ne container_<REST> methodsOld methods adjusted to use this functionalityNo changes on CLI
Modified pithos method signatures
Introduced generic account_head/post/get methods that the othermore specific methods call
Completed full pithos+ support at account level
Added full support for pithos+ headers and params for accountPOST based operations, modified existing methods to takeadvantage of this change
Added full pithos support to list_containers
In pithos, added full support for optional parameters and headers supportedin Pithos API in list_containers() (account get)
Pithos level extentions added
get_account_info now supports until parameter, and if-(un)modified-since headersin request
Simplified Client code
Added set_header method then replaced code that pushes headers to the query with calls to this much simpler method
Added policy handling
In CLI setversioning, versioning, setquota, quota for account and containerIn storage(pithos) set_container_versioning, set_container_quota, set_account_versioning, set_account_quota also get_*_policy methods are split to get_*_versioning and get_*_quota
Added set_account_group
Adds a custom group of users
Implemented group, delgroup and copy
In CLI: - added store group to retrieve user grouos of account - added store delgroup to delete a user group by groupname - added store copy to copy an object from container/path to destination container/path...
Implemented setpermitions and delpermitions
In CLI added setpermitions and delpermitions
In pithos added set_object_sharing and shortcut method erase_object_sharing
Not optimal append and truncate
In CLI added append and truncate for objects
In storage added update_object which acts as a multifunction. It currently loads the whole file on memory, so it needs optimization
Optimised append
In pithos a new append_object is introduced, that appends a file in blocks
Optimised truncate
In pithos added truncate_object that truncates and object using pithos-enabled headers
Implemented overwrite
In CLI added store_overwrite
In pithos added overwrite_object which, in case of large updates, overwrites block by block
Added progress bar
In CLI store_overwrite with prgress bar, added progress bar at store_appendIn pithos added generator watcher method support
Fixed and optiomized delmeta for objects
In storage, fixed delmeta which was disfunctionalIn pithos added an optimized implementation using the pithos update parameter
Make source PEP8 compliant
Added delete_container (remove an empty container)
In CLI: - Added store_delete_container
In storage: - Added delete_container
Added set_account_meta
In CLI: - added store_set_account_meta - added auxiliary dict_from_args in utils.py
In storage: - added set_account_meta - created client/util.py to store auxiliary methods semantically indepented Client - added auxiliary prefix_keys to client/utils.py...
Added store_setmeta (pithos API implementation)
-In CLI: - store_meta for account, container and/or object - minor fixes to meta
In storage,pithos: - methods replace_account/container/object_meta in storage (OpenStack API) - methods set_account/container/object_meta in pithos (Pithos API)...
Added store_delmenta (delete specific meta)
In CLI: - store_delmeta for account, container, object
In storage,pithos: - store_account_delmeta - store_container_delemeta - store_object_delmeta (is buggy, hope to fix it soon)...
Added un/publish object
Do not read more than the given size
Fix `var not defined' bug introduced in 31231dee4f
Add missing headers in PUT
Send application/octet-stream Content-Type on create_object
Add a purge command
Data corruption fix
Duplicate blocks were added once to the hash map.
Add an optional size argument in create_object
Add a store list command
Refactored networking
Adds requests dependency.