Filippos Giannakos [Mon, 8 Jul 2013 08:50:33 +0000 (11:50 +0300)]
Sosd: Change req->datalen with req->size.
Do not use req->datalen to fullfill requests, but req->size, which was the
requested size.
Filippos Giannakos [Wed, 3 Jul 2013 14:37:39 +0000 (17:37 +0300)]
mt-sosd: Use common hash.c for hashing
Filippos Giannakos [Wed, 3 Jul 2013 10:24:47 +0000 (13:24 +0300)]
xhash: Add XHASHS_ENOENT return value on remove
Filippos Giannakos [Wed, 3 Jul 2013 10:21:37 +0000 (13:21 +0300)]
mapper: Split loading/writing map metadata from map data
Filippos Giannakos [Wed, 3 Jul 2013 10:21:19 +0000 (13:21 +0300)]
mapper: Make callbacks state threads
Filippos Giannakos [Tue, 2 Jul 2013 15:54:06 +0000 (18:54 +0300)]
mapper: Fix v2 map loading/writing.
When changing chunks, make sure that each chunks ends at object border, so we
can safely load/store multiple chunks of map data.
Filippos Giannakos [Tue, 2 Jul 2013 15:18:43 +0000 (18:18 +0300)]
mapper: Add support to wait for a request when applicable
Filippos Giannakos [Tue, 2 Jul 2013 14:01:09 +0000 (17:01 +0300)]
Rename archip-pfiled to archip-filed
Filippos Giannakos [Tue, 2 Jul 2013 14:00:37 +0000 (17:00 +0300)]
xseg-tool: Add size options to clone command
Filippos Giannakos [Mon, 1 Jul 2013 12:55:06 +0000 (15:55 +0300)]
mapper: Increase epoch when creating volume.
Increase epoch during volume creation, to avoid name conflicting when deleting
and creating a new volume with the same name.
Also enforce limits on max epoch.
Filippos Giannakos [Mon, 1 Jul 2013 12:54:22 +0000 (15:54 +0300)]
mapper: Set the right map node on copyup_cb_copy
Filippos Giannakos [Mon, 1 Jul 2013 10:58:23 +0000 (13:58 +0300)]
mapper: Do not write in read only volumes
Filippos Giannakos [Mon, 1 Jul 2013 10:03:50 +0000 (13:03 +0300)]
Rename dummy to archip-dummy. Add it to peers.
Filippos Giannakos [Mon, 1 Jul 2013 10:02:43 +0000 (13:02 +0300)]
Clean up userspace peers makefile
Filippos Giannakos [Mon, 1 Jul 2013 09:57:34 +0000 (12:57 +0300)]
Remove mt-vlmcd target from peers Makefile
Filippos Giannakos [Mon, 1 Jul 2013 09:53:08 +0000 (12:53 +0300)]
filed: use common hexlify function
Filippos Giannakos [Mon, 1 Jul 2013 09:48:47 +0000 (12:48 +0300)]
Rename mt-pfiled to filed
Filippos Giannakos [Mon, 1 Jul 2013 09:48:12 +0000 (12:48 +0300)]
Remove old filed code
Filippos Giannakos [Mon, 1 Jul 2013 09:05:12 +0000 (12:05 +0300)]
mapper: Set copyup node, when writing new object to map
Filippos Giannakos [Fri, 28 Jun 2013 08:41:26 +0000 (11:41 +0300)]
vlmcd: Temp hack to pass snapshot target to mapper
Filippos Giannakos [Fri, 28 Jun 2013 08:40:47 +0000 (11:40 +0300)]
xseg-tool: Add message when snapshotting
Filippos Giannakos [Fri, 28 Jun 2013 08:39:31 +0000 (11:39 +0300)]
peer.c: Fix compilation warning message.
Filippos Giannakos [Fri, 28 Jun 2013 08:38:51 +0000 (11:38 +0300)]
Add include guards to peer.h
Filippos Giannakos [Fri, 28 Jun 2013 08:38:09 +0000 (11:38 +0300)]
Split hash helper functions to a submodule.
Filippos Giannakos [Fri, 28 Jun 2013 08:34:23 +0000 (11:34 +0300)]
mapperd: Refactor and add several improvements.
Completely refactor mapperd code, split logic, map handling and raw map
interpretation into different submodules.
Also add several functionality improvements such as:
* Split snapshot from hashing
* Practically unlimited size for pithos maps
* Practically unlimited archipelago volume size
* New maps now store targets without hashing
* Auto update old map to the newest versions.
Filippos Giannakos [Fri, 28 Jun 2013 08:33:35 +0000 (11:33 +0300)]
Remove old mapperd code.
Filippos Giannakos [Fri, 28 Jun 2013 08:32:27 +0000 (11:32 +0300)]
Add exceptions to check_licence tool
Add exceptions to check_licence to ignore several files created by the build
process.
Filippos Giannakos [Mon, 8 Jul 2013 09:03:13 +0000 (12:03 +0300)]
Merge branch 'feature-deployment' into develop
Filippos Giannakos [Tue, 18 Jun 2013 10:01:59 +0000 (13:01 +0300)]
archipelago: Modify launch scripts.
Modify archipelago launch scripts to support more descriptive configuration.
Configuration now describes the peers which will be lauched along with their
roles and specific options for each one of them.
Filippos Giannakos [Mon, 1 Jul 2013 08:45:53 +0000 (11:45 +0300)]
Merge branch 'feature-filedrevised' into develop
Filippos Giannakos [Mon, 1 Jul 2013 08:45:20 +0000 (11:45 +0300)]
Merge branch 'feature-xcachetype-rebased2' into develop
Filippos Giannakos [Mon, 1 Jul 2013 08:41:13 +0000 (11:41 +0300)]
pfiled; Replace directrory placement based on target.
Do not rely on target name to generate random directories, when the target
starts with the given prefix. Instead, hash the target to get the directory
structure.
Filippos Giannakos [Wed, 29 May 2013 14:58:17 +0000 (17:58 +0300)]
mt-pfiled: remove pithos dir remains
Filippos Giannakos [Wed, 29 May 2013 14:49:51 +0000 (17:49 +0300)]
mt-pfiled: create path, ignores given prefix
Filippos Giannakos [Wed, 29 May 2013 13:35:58 +0000 (16:35 +0300)]
Modify default archipelago config for mt-pfiled
Adjust default filed config to the new mt-pfiled peer.
Filippos Giannakos [Wed, 29 May 2013 13:34:51 +0000 (16:34 +0300)]
mt-pfiled: Remove pithos dir.
filed now uses only one directory for storing blocks. There should be no confict
between archipelago and pithos blocks, since they are located in diffrent
"namespaces"
Filippos Giannakos [Wed, 29 May 2013 12:32:28 +0000 (15:32 +0300)]
Fix bugs in mt-pfiled
Filippos Giannakos [Fri, 17 May 2013 08:57:20 +0000 (11:57 +0300)]
Temp mt-pfiled commit
Filippos Giannakos [Tue, 14 May 2013 14:05:47 +0000 (17:05 +0300)]
xseg-tool: fix acquire/release functions
Filippos Giannakos [Tue, 14 May 2013 14:03:47 +0000 (17:03 +0300)]
Reimplement mt-pfiled.
Reimplement mt-pfiled using the new xcache type, to cache open file descriptors.
Also implement snapshot support and proper idempotent locking system.
Filippos Giannakos [Mon, 10 Jun 2013 09:58:05 +0000 (12:58 +0300)]
xcache: Add missing licence headers
Filippos Giannakos [Fri, 7 Jun 2013 08:21:32 +0000 (11:21 +0300)]
xcache: Unique names in xcache test2
During xcache second test, each thread must be able to generate its unique names
for the cache entries. "%tid%uniqueidperstring" format is not sufficient,
because another thread with diffrent tid can produce the same string. In this
commit we use the "tid:%tid_i:%uniqueidperstring" format.
Filippos Giannakos [Wed, 5 Jun 2013 13:56:25 +0000 (16:56 +0300)]
xcache: Fix tab indendation in comments
Alex Pyrgiotis [Wed, 5 Jun 2013 13:16:22 +0000 (16:16 +0300)]
xcache: Make invalidate check the rm_table flag
Alex Pyrgiotis [Wed, 5 Jun 2013 10:54:34 +0000 (13:54 +0300)]
xcache: Explicitly set flag to use rm tables
Introduce a new flag (XCACHE_USE_RMTABLE) that will be set only by the
peers that want to use the rm_table. The rest of the peers don't need to
set anything. This should boost speed for the latter peers, since they
won't unnecessarily try to acquire the rm_lock on inserts and puts
Changes:
* The "post_evict" hook is no longer necessary and is superseded by this
flag.
* This flag is passed during xcache_init in the "flags" parameter.
Alex Pyrgiotis [Sun, 19 May 2013 21:55:42 +0000 (00:55 +0300)]
xcache: Refactor put, add more assertions
Refactor xcache_entry_put to bypass the ABA problem with the refcount of
cache entries. The proposed solution is not the most optimal, but will
suffice until a better one is implemented.
Also, add more assertions in various places in the code to ease the
debugging process.
Alex Pyrgiotis [Tue, 14 May 2013 13:52:57 +0000 (16:52 +0300)]
xcache_test: Fix compilation warnings
Alex Pyrgiotis [Tue, 14 May 2013 13:51:24 +0000 (16:51 +0300)]
xcache: Code re-arrangements
Filippos Giannakos [Mon, 29 Apr 2013 15:52:02 +0000 (18:52 +0300)]
Add xcache xtype implementation
Filippos Giannakos [Mon, 29 Apr 2013 15:48:00 +0000 (18:48 +0300)]
Add xbinheap xtype implementation
Filippos Giannakos [Mon, 10 Jun 2013 09:37:51 +0000 (12:37 +0300)]
Merge branch 'feature-licence' into develop
Filippos Giannakos [Fri, 7 Jun 2013 08:10:17 +0000 (11:10 +0300)]
Do not remove config.mk from top Makefile
Do not remove config.mk from top Makefile during clean action
Filippos Giannakos [Fri, 7 Jun 2013 07:38:41 +0000 (10:38 +0300)]
Remove arch-script files
Filippos Giannakos [Fri, 7 Jun 2013 07:33:01 +0000 (10:33 +0300)]
Add xseg/tags to gitignore
Filippos Giannakos [Fri, 7 Jun 2013 07:29:56 +0000 (10:29 +0300)]
Fix licence headers in multiple files
Add licence headers to all code files and adjust interpreter in script files.
Filippos Giannakos [Thu, 6 Jun 2013 15:14:29 +0000 (18:14 +0300)]
Add tool to check licence in files
Filippos Giannakos [Thu, 6 Jun 2013 15:08:05 +0000 (18:08 +0300)]
Fix typos in licence headers
Filippos Giannakos [Wed, 5 Jun 2013 14:36:47 +0000 (17:36 +0300)]
sosd: Fix uninitialized variable use.
In break lock, instead of cheking the return value of the rados call, we
erroneously checked an uninitialized local variable.
(cherry picked from commit
ba7ffa08f1b68282ef0cffb49388fc1b705d6cd3)
Filippos Giannakos [Wed, 5 Jun 2013 12:17:10 +0000 (15:17 +0300)]
Merge branch 'feature-queues' into develop
Filippos Giannakos [Mon, 29 Apr 2013 15:43:44 +0000 (18:43 +0300)]
Add xwaitq xtype implementation
Filippos Giannakos [Mon, 29 Apr 2013 15:40:25 +0000 (18:40 +0300)]
Add workq xtype implementation
Filippos Giannakos [Wed, 5 Jun 2013 12:16:49 +0000 (15:16 +0300)]
Merge branch 'feature-xhashlimit' into develop
Filippos Giannakos [Thu, 14 Feb 2013 17:28:09 +0000 (19:28 +0200)]
Add limit to xhash
Filippos Giannakos [Wed, 5 Jun 2013 12:16:26 +0000 (15:16 +0300)]
Merge branch 'feature-variousimprovements' into develop
Filippos Giannakos [Fri, 22 Feb 2013 12:00:02 +0000 (14:00 +0200)]
Remove unused linker file from xobj.o
Alex Pyrgiotis [Mon, 15 Apr 2013 13:12:25 +0000 (16:12 +0300)]
Add LIKELY/UNLIKELY branch prediction macros
The macros are purposefully in capital letters in order not to clash
with the likely/unlikely macros in kernel.
Filippos Giannakos [Fri, 22 Feb 2013 11:21:23 +0000 (13:21 +0200)]
Improve xlock try lock function
Filippos Giannakos [Wed, 5 Jun 2013 12:15:56 +0000 (15:15 +0300)]
Merge branch 'feature-cleanerbuilddeps' into develop
Filippos Giannakos [Mon, 29 Apr 2013 15:05:33 +0000 (18:05 +0300)]
Make xtypes build system cleaner
xtypes should provide
for userspace:
a "_exports.h" file
a xtype dir in the sys/user directory
.o and .pic.o targets in the Makefile.
for kernel:
xtype .k.c target in the Makefile
Filippos Giannakos [Wed, 5 Jun 2013 12:12:13 +0000 (15:12 +0300)]
Bump version to 0.3next.
Filippos Giannakos [Wed, 5 Jun 2013 09:42:58 +0000 (12:42 +0300)]
Merge branch 'release-0.3' into develop
Filippos Giannakos [Wed, 5 Jun 2013 09:42:45 +0000 (12:42 +0300)]
Merge branch 'feature-sosd' into develop
Filippos Giannakos [Wed, 5 Jun 2013 09:21:30 +0000 (12:21 +0300)]
Bump version to 0.3
Filippos Giannakos [Mon, 3 Jun 2013 14:43:06 +0000 (17:43 +0300)]
Bump version.
Bumped version to 0.3rc2
Filippos Giannakos [Fri, 31 May 2013 09:32:28 +0000 (12:32 +0300)]
peer.c: Call custom peer finalize with st_threads
Filippos Giannakos [Fri, 31 May 2013 08:46:43 +0000 (11:46 +0300)]
bench: Fix target to not include null termination.
Request of the target must not be null terminated. Remove null termination from
the requests constructed by peers.
Filippos Giannakos [Fri, 31 May 2013 08:19:10 +0000 (11:19 +0300)]
mapperd: Add debug messages for copyup target
Filippos Giannakos [Fri, 31 May 2013 07:09:28 +0000 (10:09 +0300)]
mapperd: Fix signaling bug when load/open fails.
When a map load/open fails, any other threads that tried to get the map during
these operations wait on the map. If any of these operations fail, the cache was
dropped, without signaling any of the waiters. This commit fixes this bug, by
signaling the map when open or load fail.
Filippos Giannakos [Thu, 30 May 2013 15:23:09 +0000 (18:23 +0300)]
mapperd: Add debug message to signal map and put map
Filippos Giannakos [Thu, 30 May 2013 13:45:50 +0000 (16:45 +0300)]
Rename archipelago peers with archip- prefix
Filippos Giannakos [Thu, 30 May 2013 13:38:49 +0000 (16:38 +0300)]
Change ext_script dir to archipelago.
Change ext_script ganeti directory from vlmc to archipelago.
Filippos Giannakos [Thu, 30 May 2013 09:40:52 +0000 (12:40 +0300)]
Bump version.
Bump versio to 0.3rc1
Filippos Giannakos [Fri, 17 May 2013 14:36:00 +0000 (17:36 +0300)]
Adjust mt-sosd to the new RADOS lock API.
Filippos Giannakos [Wed, 29 May 2013 12:28:44 +0000 (15:28 +0300)]
xseg: Set serviced = 0 on {get, put} request
Filippos Giannakos [Fri, 24 May 2013 14:08:03 +0000 (17:08 +0300)]
xseg.h: Move end of include guard to the header's end.
Filippos Giannakos [Fri, 17 May 2013 15:08:52 +0000 (18:08 +0300)]
Merge branch 'develop-cherrypick' into develop
Merged commits cherry-picked from release-0.2 branch
Filippos Giannakos [Fri, 19 Apr 2013 08:48:58 +0000 (11:48 +0300)]
Fix bug in mapperd.
When signaling map, use signal_broadcast to wake up all waiters instead of just
one, to avoid missing signals when cached is dropped for the map.
(cherry picked from commit
2b91a67a55cdd9b6a16fd54e3e87389aa181a7ee)
Filippos Giannakos [Wed, 17 Apr 2013 13:36:31 +0000 (16:36 +0300)]
Add wrapper function for ST peerd loop.
(cherry picked from commit
d932df7a6a3ae8b17d9e0981fcb3ee2af059324c)
Conflicts:
xseg/peers/user/peer.c
Filippos Giannakos [Wed, 17 Apr 2013 12:38:02 +0000 (15:38 +0300)]
mapperd: Always sleep before checking request state.
(cherry picked from commit
e0f306aa00f5aefa72dc4f607876ee87ddc2ea37)
Filippos Giannakos [Mon, 15 Apr 2013 12:45:42 +0000 (15:45 +0300)]
Make xseg request state volatile
(cherry picked from commit
8904f78564a1d1f96eb0453e7773779dc931d861)
Filippos Giannakos [Mon, 15 Apr 2013 09:35:47 +0000 (12:35 +0300)]
Make peers close stdin when entering daemon mode.
(cherry picked from commit
04ea45339fd0e5a92c3290807805766acc1e7223)
Filippos Giannakos [Mon, 15 Apr 2013 08:31:53 +0000 (11:31 +0300)]
Close orig stdout, stderr, when logging to file
(cherry picked from commit
1eeb0111a9d7ac5700d7aedc9f442b748b9cb8e7)
Filippos Giannakos [Mon, 15 Apr 2013 08:30:03 +0000 (11:30 +0300)]
Add wait time while starting a peer.
While starting a peer, wait a few seconds, before checking if the peer is up n
running. In high load conditions, this check was too soon and could fail,
resulting in an overall start failure.
(cherry picked from commit
1b4a1ad9cd39b620b35bd30970edda304fcb92a6)
Filippos Giannakos [Mon, 8 Apr 2013 15:37:23 +0000 (18:37 +0300)]
Remove async unsafe functions from signal handlers
(cherry picked from commit
2d2a24a26ff9f3a7fd651b1c2386fbd31166a5e1)
Filippos Giannakos [Mon, 8 Apr 2013 15:37:00 +0000 (18:37 +0300)]
Fix bug in user xseglog2
(cherry picked from commit
c30767e3f311fd97bb1f5f546b8654911d712072)
Filippos Giannakos [Mon, 8 Apr 2013 10:52:28 +0000 (13:52 +0300)]
Fix user space logrotation bug.
Replace file stream functions in user space logging with their file descriptor
counter parts, which are safe to be called from a signal handler.
(cherry picked from commit
342ac1dc1a5ebc9aef55808b2d42436be3efb88d)
Filippos Giannakos [Tue, 26 Mar 2013 11:44:53 +0000 (13:44 +0200)]
Remove ctypeslib dependency from python-xseg
(cherry picked from commit
93f23c27d9567647a27a035732edd2a4ef66d83c)
Filippos Giannakos [Fri, 22 Mar 2013 11:17:23 +0000 (13:17 +0200)]
Move src dir to archipelago-modules-dkms
(cherry picked from commit
9c39a188e8ff0ae49ecb7f6587b693c5a546289a)
Filippos Giannakos [Thu, 21 Mar 2013 14:14:45 +0000 (16:14 +0200)]
Export KVER if none existing.
Also modulesdir now points to the KVER we are building for.
(cherry picked from commit
201e6354bbe2f6f4a4d921992a425e32444514fa)
Filippos Giannakos [Thu, 21 Mar 2013 13:40:25 +0000 (15:40 +0200)]
Add xseg to src dir.
(cherry picked from commit
0e4ae29f8d36524c61720a7f1dcafcdc9cd6bb03)