Filippos Giannakos [Wed, 17 Jul 2013 12:39:04 +0000 (15:39 +0300)]
sosd: Add X_HASH support with precomputed values
Filippos Giannakos [Wed, 17 Jul 2013 12:37:43 +0000 (15:37 +0300)]
xseg: Add X_HASH to xseg protocol.
Add X_HASH operator to xseg, and the appopriate X_HASH reply structure. Remove
obsolete snapshot reply structure.
Filippos Giannakos [Wed, 17 Jul 2013 12:37:09 +0000 (15:37 +0300)]
vlmc: Remove obsolete snapshot handling functionality
Filippos Giannakos [Wed, 17 Jul 2013 12:35:47 +0000 (15:35 +0300)]
mapper: Add hash support for snapshots.
Add the ability to hash snapshots and create pithos compatible images.
Also, as a minor improvement add object_to_map representation to map functions.
Filippos Giannakos [Wed, 17 Jul 2013 12:31:07 +0000 (15:31 +0300)]
filed: Support X_HASH with precomputed hashes.
Support X_HASH with precomputed hash values, if the same block was hashed
before.
Filippos Giannakos [Tue, 16 Jul 2013 08:14:39 +0000 (11:14 +0300)]
tests: Add testing for cloning archipelago snapshots
Filippos Giannakos [Tue, 16 Jul 2013 08:13:49 +0000 (11:13 +0300)]
mapper: Add XF_CONTADDR flag and improve cloning
Support cloning from both pithos images and archipelago snapshots.
Filippos Giannakos [Tue, 16 Jul 2013 08:12:49 +0000 (11:12 +0300)]
mapper: Fix get_mapnode boundary.
Return NULL, when requesting object index == nr_obs
Filippos Giannakos [Tue, 16 Jul 2013 08:11:35 +0000 (11:11 +0300)]
mapper: Fail when issuing a map object load fails
Filippos Giannakos [Tue, 16 Jul 2013 08:10:44 +0000 (11:10 +0300)]
mapper: Always use datalen buffers when loading v2
Filippos Giannakos [Tue, 16 Jul 2013 08:09:57 +0000 (11:09 +0300)]
mapper: Set correct map object when copying up
Filippos Giannakos [Mon, 15 Jul 2013 13:39:13 +0000 (16:39 +0300)]
Merge branch 'feature-functionaltest' into feature-snapshot
Filippos Giannakos [Mon, 15 Jul 2013 13:28:43 +0000 (16:28 +0300)]
testing: Add tests for concurrent operations
Filippos Giannakos [Mon, 15 Jul 2013 13:26:22 +0000 (16:26 +0300)]
python xseg: Enrich request/xseg python api.
Add more constructors for the Request class.
Add xseg_wait_requests to wait on multiple requests.
Modify vlmc script to use the new constructors.
Filippos Giannakos [Mon, 15 Jul 2013 13:25:29 +0000 (16:25 +0300)]
filed: Add debug logging while reading/writing
Filippos Giannakos [Fri, 12 Jul 2013 07:47:24 +0000 (10:47 +0300)]
tests: Add basic functionality testing for peers
Filippos Giannakos [Thu, 11 Jul 2013 15:12:20 +0000 (18:12 +0300)]
Improve python archipelago common infrastructure.
The changes include:
* Convert pfiled to filed and add the extra arguments
* Check if arguments are not None instead of simply checking for arguments
* Move request to seperate method and remove __del__ method
* Add get_serviced and get_datalen to request
* Add support to set string add data request
* Make all peer executables relative to a BIN_DIR
* Create/destroy segment takes spec as arguments
* Parsing spec on Xsegctx __init__ does not modify spec parameter
Filippos Giannakos [Thu, 11 Jul 2013 15:11:23 +0000 (18:11 +0300)]
peer: Create pidfile with read/write permissions
Filippos Giannakos [Thu, 11 Jul 2013 14:38:18 +0000 (17:38 +0300)]
mapper: Fail loading of deleted volumes
Filippos Giannakos [Thu, 11 Jul 2013 14:37:43 +0000 (17:37 +0300)]
mapper: Fail request when range is not valid
Filippos Giannakos [Thu, 11 Jul 2013 14:35:15 +0000 (17:35 +0300)]
Set zero sized clone to inherit the size of the parent
Uniformly set that when the requested size is zero, then either inherit the size
of the parent map, or fail if there is no parent.
Filippos Giannakos [Thu, 11 Jul 2013 14:33:08 +0000 (17:33 +0300)]
vlmcd: resize request on X_INFO
Filippos Giannakos [Thu, 11 Jul 2013 14:32:46 +0000 (17:32 +0300)]
mapper: Add resize request on X_INF
Filippos Giannakos [Thu, 11 Jul 2013 14:30:49 +0000 (17:30 +0300)]
filed: Add resize request on X_INFO
Filippos Giannakos [Thu, 11 Jul 2013 14:30:04 +0000 (17:30 +0300)]
mapper: add debug messages while handling objects in v2
Filippos Giannakos [Thu, 11 Jul 2013 14:23:34 +0000 (17:23 +0300)]
xseg: remove some unused logging
Filippos Giannakos [Mon, 8 Jul 2013 08:59:51 +0000 (11:59 +0300)]
mapperd: Modify version2 map IO functions.
Modify version2 map IO functions to support load/write map data for a specified
object range.
Filippos Giannakos [Mon, 8 Jul 2013 08:57:45 +0000 (11:57 +0300)]
mapperd: Add counter on map to count used objects
Add map counter to count used objects, so we can quickly calculate whether all
objects are in READY state, and there are no pending actions on them.
Filippos Giannakos [Mon, 8 Jul 2013 08:52:43 +0000 (11:52 +0300)]
mapperd: Fail load if map upgrade fails
Filippos Giannakos [Mon, 8 Jul 2013 08:51:44 +0000 (11:51 +0300)]
filed: Change req->datalen with req->size.
Do not use req->datalen to fullfill requests, but req->size, which was the
requested size.
Filippos Giannakos [Mon, 8 Jul 2013 08:50:33 +0000 (11:50 +0300)]
Sosd: Change req->datalen with req->size.
Do not use req->datalen to fullfill requests, but req->size, which was the
requested size.
Filippos Giannakos [Wed, 3 Jul 2013 14:37:39 +0000 (17:37 +0300)]
mt-sosd: Use common hash.c for hashing
Filippos Giannakos [Wed, 3 Jul 2013 10:24:47 +0000 (13:24 +0300)]
xhash: Add XHASHS_ENOENT return value on remove
Filippos Giannakos [Wed, 3 Jul 2013 10:21:37 +0000 (13:21 +0300)]
mapper: Split loading/writing map metadata from map data
Filippos Giannakos [Wed, 3 Jul 2013 10:21:19 +0000 (13:21 +0300)]
mapper: Make callbacks state threads
Filippos Giannakos [Tue, 2 Jul 2013 15:54:06 +0000 (18:54 +0300)]
mapper: Fix v2 map loading/writing.
When changing chunks, make sure that each chunks ends at object border, so we
can safely load/store multiple chunks of map data.
Filippos Giannakos [Tue, 2 Jul 2013 15:18:43 +0000 (18:18 +0300)]
mapper: Add support to wait for a request when applicable
Filippos Giannakos [Tue, 2 Jul 2013 14:01:09 +0000 (17:01 +0300)]
Rename archip-pfiled to archip-filed
Filippos Giannakos [Tue, 2 Jul 2013 14:00:37 +0000 (17:00 +0300)]
xseg-tool: Add size options to clone command
Filippos Giannakos [Mon, 1 Jul 2013 12:55:06 +0000 (15:55 +0300)]
mapper: Increase epoch when creating volume.
Increase epoch during volume creation, to avoid name conflicting when deleting
and creating a new volume with the same name.
Also enforce limits on max epoch.
Filippos Giannakos [Mon, 1 Jul 2013 12:54:22 +0000 (15:54 +0300)]
mapper: Set the right map node on copyup_cb_copy
Filippos Giannakos [Mon, 1 Jul 2013 10:58:23 +0000 (13:58 +0300)]
mapper: Do not write in read only volumes
Filippos Giannakos [Mon, 1 Jul 2013 10:03:50 +0000 (13:03 +0300)]
Rename dummy to archip-dummy. Add it to peers.
Filippos Giannakos [Mon, 1 Jul 2013 10:02:43 +0000 (13:02 +0300)]
Clean up userspace peers makefile
Filippos Giannakos [Mon, 1 Jul 2013 09:57:34 +0000 (12:57 +0300)]
Remove mt-vlmcd target from peers Makefile
Filippos Giannakos [Mon, 1 Jul 2013 09:53:08 +0000 (12:53 +0300)]
filed: use common hexlify function
Filippos Giannakos [Mon, 1 Jul 2013 09:48:47 +0000 (12:48 +0300)]
Rename mt-pfiled to filed
Filippos Giannakos [Mon, 1 Jul 2013 09:48:12 +0000 (12:48 +0300)]
Remove old filed code
Filippos Giannakos [Mon, 1 Jul 2013 09:05:12 +0000 (12:05 +0300)]
mapper: Set copyup node, when writing new object to map
Filippos Giannakos [Fri, 28 Jun 2013 08:41:26 +0000 (11:41 +0300)]
vlmcd: Temp hack to pass snapshot target to mapper
Filippos Giannakos [Fri, 28 Jun 2013 08:40:47 +0000 (11:40 +0300)]
xseg-tool: Add message when snapshotting
Filippos Giannakos [Fri, 28 Jun 2013 08:39:31 +0000 (11:39 +0300)]
peer.c: Fix compilation warning message.
Filippos Giannakos [Fri, 28 Jun 2013 08:38:51 +0000 (11:38 +0300)]
Add include guards to peer.h
Filippos Giannakos [Fri, 28 Jun 2013 08:38:09 +0000 (11:38 +0300)]
Split hash helper functions to a submodule.
Filippos Giannakos [Fri, 28 Jun 2013 08:34:23 +0000 (11:34 +0300)]
mapperd: Refactor and add several improvements.
Completely refactor mapperd code, split logic, map handling and raw map
interpretation into different submodules.
Also add several functionality improvements such as:
* Split snapshot from hashing
* Practically unlimited size for pithos maps
* Practically unlimited archipelago volume size
* New maps now store targets without hashing
* Auto update old map to the newest versions.
Filippos Giannakos [Fri, 28 Jun 2013 08:33:35 +0000 (11:33 +0300)]
Remove old mapperd code.
Filippos Giannakos [Fri, 28 Jun 2013 08:32:27 +0000 (11:32 +0300)]
Add exceptions to check_licence tool
Add exceptions to check_licence to ignore several files created by the build
process.
Filippos Giannakos [Mon, 8 Jul 2013 09:03:13 +0000 (12:03 +0300)]
Merge branch 'feature-deployment' into develop
Filippos Giannakos [Tue, 18 Jun 2013 10:01:59 +0000 (13:01 +0300)]
archipelago: Modify launch scripts.
Modify archipelago launch scripts to support more descriptive configuration.
Configuration now describes the peers which will be lauched along with their
roles and specific options for each one of them.
Filippos Giannakos [Mon, 1 Jul 2013 08:45:53 +0000 (11:45 +0300)]
Merge branch 'feature-filedrevised' into develop
Filippos Giannakos [Mon, 1 Jul 2013 08:45:20 +0000 (11:45 +0300)]
Merge branch 'feature-xcachetype-rebased2' into develop
Filippos Giannakos [Mon, 1 Jul 2013 08:41:13 +0000 (11:41 +0300)]
pfiled; Replace directrory placement based on target.
Do not rely on target name to generate random directories, when the target
starts with the given prefix. Instead, hash the target to get the directory
structure.
Filippos Giannakos [Wed, 29 May 2013 14:58:17 +0000 (17:58 +0300)]
mt-pfiled: remove pithos dir remains
Filippos Giannakos [Wed, 29 May 2013 14:49:51 +0000 (17:49 +0300)]
mt-pfiled: create path, ignores given prefix
Filippos Giannakos [Wed, 29 May 2013 13:35:58 +0000 (16:35 +0300)]
Modify default archipelago config for mt-pfiled
Adjust default filed config to the new mt-pfiled peer.
Filippos Giannakos [Wed, 29 May 2013 13:34:51 +0000 (16:34 +0300)]
mt-pfiled: Remove pithos dir.
filed now uses only one directory for storing blocks. There should be no confict
between archipelago and pithos blocks, since they are located in diffrent
"namespaces"
Filippos Giannakos [Wed, 29 May 2013 12:32:28 +0000 (15:32 +0300)]
Fix bugs in mt-pfiled
Filippos Giannakos [Fri, 17 May 2013 08:57:20 +0000 (11:57 +0300)]
Temp mt-pfiled commit
Filippos Giannakos [Tue, 14 May 2013 14:05:47 +0000 (17:05 +0300)]
xseg-tool: fix acquire/release functions
Filippos Giannakos [Tue, 14 May 2013 14:03:47 +0000 (17:03 +0300)]
Reimplement mt-pfiled.
Reimplement mt-pfiled using the new xcache type, to cache open file descriptors.
Also implement snapshot support and proper idempotent locking system.
Filippos Giannakos [Mon, 10 Jun 2013 09:58:05 +0000 (12:58 +0300)]
xcache: Add missing licence headers
Filippos Giannakos [Fri, 7 Jun 2013 08:21:32 +0000 (11:21 +0300)]
xcache: Unique names in xcache test2
During xcache second test, each thread must be able to generate its unique names
for the cache entries. "%tid%uniqueidperstring" format is not sufficient,
because another thread with diffrent tid can produce the same string. In this
commit we use the "tid:%tid_i:%uniqueidperstring" format.
Filippos Giannakos [Wed, 5 Jun 2013 13:56:25 +0000 (16:56 +0300)]
xcache: Fix tab indendation in comments
Alex Pyrgiotis [Wed, 5 Jun 2013 13:16:22 +0000 (16:16 +0300)]
xcache: Make invalidate check the rm_table flag
Alex Pyrgiotis [Wed, 5 Jun 2013 10:54:34 +0000 (13:54 +0300)]
xcache: Explicitly set flag to use rm tables
Introduce a new flag (XCACHE_USE_RMTABLE) that will be set only by the
peers that want to use the rm_table. The rest of the peers don't need to
set anything. This should boost speed for the latter peers, since they
won't unnecessarily try to acquire the rm_lock on inserts and puts
Changes:
* The "post_evict" hook is no longer necessary and is superseded by this
flag.
* This flag is passed during xcache_init in the "flags" parameter.
Alex Pyrgiotis [Sun, 19 May 2013 21:55:42 +0000 (00:55 +0300)]
xcache: Refactor put, add more assertions
Refactor xcache_entry_put to bypass the ABA problem with the refcount of
cache entries. The proposed solution is not the most optimal, but will
suffice until a better one is implemented.
Also, add more assertions in various places in the code to ease the
debugging process.
Alex Pyrgiotis [Tue, 14 May 2013 13:52:57 +0000 (16:52 +0300)]
xcache_test: Fix compilation warnings
Alex Pyrgiotis [Tue, 14 May 2013 13:51:24 +0000 (16:51 +0300)]
xcache: Code re-arrangements
Filippos Giannakos [Mon, 29 Apr 2013 15:52:02 +0000 (18:52 +0300)]
Add xcache xtype implementation
Filippos Giannakos [Mon, 29 Apr 2013 15:48:00 +0000 (18:48 +0300)]
Add xbinheap xtype implementation
Filippos Giannakos [Mon, 10 Jun 2013 09:37:51 +0000 (12:37 +0300)]
Merge branch 'feature-licence' into develop
Filippos Giannakos [Fri, 7 Jun 2013 08:10:17 +0000 (11:10 +0300)]
Do not remove config.mk from top Makefile
Do not remove config.mk from top Makefile during clean action
Filippos Giannakos [Fri, 7 Jun 2013 07:38:41 +0000 (10:38 +0300)]
Remove arch-script files
Filippos Giannakos [Fri, 7 Jun 2013 07:33:01 +0000 (10:33 +0300)]
Add xseg/tags to gitignore
Filippos Giannakos [Fri, 7 Jun 2013 07:29:56 +0000 (10:29 +0300)]
Fix licence headers in multiple files
Add licence headers to all code files and adjust interpreter in script files.
Filippos Giannakos [Thu, 6 Jun 2013 15:14:29 +0000 (18:14 +0300)]
Add tool to check licence in files
Filippos Giannakos [Thu, 6 Jun 2013 15:08:05 +0000 (18:08 +0300)]
Fix typos in licence headers
Filippos Giannakos [Wed, 5 Jun 2013 14:36:47 +0000 (17:36 +0300)]
sosd: Fix uninitialized variable use.
In break lock, instead of cheking the return value of the rados call, we
erroneously checked an uninitialized local variable.
(cherry picked from commit
ba7ffa08f1b68282ef0cffb49388fc1b705d6cd3)
Filippos Giannakos [Wed, 5 Jun 2013 12:17:10 +0000 (15:17 +0300)]
Merge branch 'feature-queues' into develop
Filippos Giannakos [Mon, 29 Apr 2013 15:43:44 +0000 (18:43 +0300)]
Add xwaitq xtype implementation
Filippos Giannakos [Mon, 29 Apr 2013 15:40:25 +0000 (18:40 +0300)]
Add workq xtype implementation
Filippos Giannakos [Wed, 5 Jun 2013 12:16:49 +0000 (15:16 +0300)]
Merge branch 'feature-xhashlimit' into develop
Filippos Giannakos [Thu, 14 Feb 2013 17:28:09 +0000 (19:28 +0200)]
Add limit to xhash
Filippos Giannakos [Wed, 5 Jun 2013 12:16:26 +0000 (15:16 +0300)]
Merge branch 'feature-variousimprovements' into develop
Filippos Giannakos [Fri, 22 Feb 2013 12:00:02 +0000 (14:00 +0200)]
Remove unused linker file from xobj.o
Alex Pyrgiotis [Mon, 15 Apr 2013 13:12:25 +0000 (16:12 +0300)]
Add LIKELY/UNLIKELY branch prediction macros
The macros are purposefully in capital letters in order not to clash
with the likely/unlikely macros in kernel.
Filippos Giannakos [Fri, 22 Feb 2013 11:21:23 +0000 (13:21 +0200)]
Improve xlock try lock function
Filippos Giannakos [Wed, 5 Jun 2013 12:15:56 +0000 (15:15 +0300)]
Merge branch 'feature-cleanerbuilddeps' into develop
Filippos Giannakos [Mon, 29 Apr 2013 15:05:33 +0000 (18:05 +0300)]
Make xtypes build system cleaner
xtypes should provide
for userspace:
a "_exports.h" file
a xtype dir in the sys/user directory
.o and .pic.o targets in the Makefile.
for kernel:
xtype .k.c target in the Makefile
Filippos Giannakos [Wed, 5 Jun 2013 12:12:13 +0000 (15:12 +0300)]
Bump version to 0.3next.