Alex Pyrgiotis [Thu, 21 Feb 2013 10:34:35 +0000 (12:34 +0200)]
Print more useful results
Alex Pyrgiotis [Wed, 20 Feb 2013 22:16:55 +0000 (00:16 +0200)]
Fix signal handling of peers
Alex Pyrgiotis [Wed, 20 Feb 2013 15:28:23 +0000 (17:28 +0200)]
Tweak performance of bench
Alex Pyrgiotis [Wed, 20 Feb 2013 07:58:43 +0000 (09:58 +0200)]
Fix LFSR behavior
Alex Pyrgiotis [Tue, 19 Feb 2013 17:15:05 +0000 (19:15 +0200)]
Fix flag handling of struct bench
Alex Pyrgiotis [Tue, 19 Feb 2013 16:59:37 +0000 (18:59 +0200)]
Fix target names and chunk offsets
Alex Pyrgiotis [Tue, 19 Feb 2013 15:24:06 +0000 (17:24 +0200)]
Increment completed requests of insane timers
Even if a timer is not used for a benchmark, we need to increment its
completed requests since these are needed to calculate how many requests
are left etc.
Alex Pyrgiotis [Tue, 19 Feb 2013 14:28:43 +0000 (16:28 +0200)]
Implement most semi-finished functionalities
Summary of new functionalities:
* Add support for 4 different xseg operations (read, write, info,
delete).
* Add support for random I/O patterns with a fast Galois LFSR.
* Add insanity levels in timers, which allow the user to exclude obscure
timers from the benchmark (or explicitly allow them)
* Generate target names based on seeds given by the user or produced by
the benchmark (useful for reproducibility of benchmark)
Alex Pyrgiotis [Mon, 18 Feb 2013 19:56:20 +0000 (21:56 +0200)]
Add an LFSR PRNG for bench
The PRNG introduced by this commit is a Galois-XNOR LFSR, which can
generate unique pseudorandom numbers. Also, seeded with the same
seed, it will produce the same sequence.
This implementation is driven by the needs of XSEG for a blazing fast
random generator. Initial tests show that it's capable of producing 2^38
unique pseudorandom numbers in less than 10 minutes (which equals to 2
nanoseconds per number).
Alex Pyrgiotis [Sat, 16 Feb 2013 19:24:24 +0000 (21:24 +0200)]
Add utility functions to header file
Alex Pyrgiotis [Sat, 16 Feb 2013 18:51:10 +0000 (20:51 +0200)]
Design the program's skeleton for new features
Bench peer must be able to test all xseg operations in a synchronous or
asynchronous way. Also, it must have a validation mode to sanity-check
if xseg requests are issued properly. Most of the above are present in
this commit, either in code or TODO form.
WARNING: code is incomplete and won't compile
Alex Pyrgiotis [Fri, 15 Feb 2013 09:40:11 +0000 (11:40 +0200)]
Fix segfault, parameters validation
Alex Pyrgiotis [Thu, 14 Feb 2013 18:00:03 +0000 (20:00 +0200)]
Initial support for xseg ops
Alex Pyrgiotis [Thu, 14 Feb 2013 13:38:51 +0000 (15:38 +0200)]
Alpha release of bench peer
On this release, the bench peer can measure the total elapsed time
of the benchmark. TODO:
* Allow bench peer to be multi-threaded.
* Adjust benchmarking precision, by eliminating or including more
trivial timers.
* Measure mean time, standard deviation of requests.
Alex Pyrgiotis [Tue, 12 Feb 2013 17:39:26 +0000 (19:39 +0200)]
Make bench send requests faster
Alex Pyrgiotis [Tue, 12 Feb 2013 12:02:58 +0000 (14:02 +0200)]
Fix missing function declaration
Alex Pyrgiotis [Tue, 12 Feb 2013 11:28:37 +0000 (13:28 +0200)]
Simplify port binding code
Alex Pyrgiotis [Tue, 12 Feb 2013 07:02:09 +0000 (09:02 +0200)]
Fix segfault
Alex Pyrgiotis [Tue, 12 Feb 2013 06:57:14 +0000 (08:57 +0200)]
Keep naming in peerd_loop consistent
Alex Pyrgiotis [Mon, 11 Feb 2013 16:38:06 +0000 (18:38 +0200)]
Fix timers and add them to critical paths
Fixed incorrect handling of elapsed time and used appropriate timers to
benchmark crtical paths.
Alex Pyrgiotis [Mon, 11 Feb 2013 16:34:50 +0000 (18:34 +0200)]
Complete unification of thread_loop/peerd_loop
Alex Pyrgiotis [Mon, 11 Feb 2013 07:00:08 +0000 (09:00 +0200)]
Fix segfault
Alex Pyrgiotis [Mon, 11 Feb 2013 01:10:12 +0000 (03:10 +0200)]
Unify thread loop with peerd_loop
Since peerd_loop with thread_loop don't have many differences between
them, they can be unified in a single loop. TODO is to choose a unified
name for both threads and peers, as well as fault check the
implementation.
Alex Pyrgiotis [Sat, 9 Feb 2013 11:32:49 +0000 (13:32 +0200)]
Add timer implementation
This an initial timer implementation. It is not very polished at the
momment and some implementation choices will have to be reviewed later
on.
Alex Pyrgiotis [Fri, 8 Feb 2013 09:28:04 +0000 (11:28 +0200)]
Add support for custom peer loop
There is a new field in struct peerd called "custom_peerd_loop". If any
peer wants to use its own loop instead of the generic loop, it can plug
it in that field.
Alex Pyrgiotis [Thu, 7 Feb 2013 11:58:58 +0000 (13:58 +0200)]
Parse and error-check arguments
Alex Pyrgiotis [Wed, 6 Feb 2013 15:54:15 +0000 (17:54 +0200)]
Add initial support for benchmarking xseg
Alex Pyrgiotis [Mon, 4 Feb 2013 10:29:45 +0000 (12:29 +0200)]
Fix grep_xseg to work both with include/exclude
Alex Pyrgiotis [Fri, 1 Feb 2013 18:47:30 +0000 (20:47 +0200)]
Fix case where user is root
In this case, /home/root/... path doesn't exist, so we use a hacky way
to get the scripts' path.
Alex Pyrgiotis [Fri, 1 Feb 2013 11:12:30 +0000 (13:12 +0200)]
Improve arch-scripts
Alex Pyrgiotis [Fri, 1 Feb 2013 10:52:53 +0000 (12:52 +0200)]
Add O_EXCL flag when opening segment for creation
Security fix. One could potentially pass a wrong argument in xseg_create
and destroy an already initialized segment. Ensuring creation
exclusiveness makes xseg less error-prone.
Alex Pyrgiotis [Fri, 1 Feb 2013 10:41:14 +0000 (12:41 +0200)]
Add S_IWUSR permission bit for O_CREAT
In Ubuntu -haven't tested it elsewhere yet- gcc has a global cflag
called "D_FORTIFY_SOURCE=2" that errs when files are not created
with explicit user permissions. Since files are O_WRONLY, I added
the S_IWUSR permission bit.
Alex Pyrgiotis [Fri, 1 Feb 2013 09:28:32 +0000 (11:28 +0200)]
Add simple bash scripts to make and grep xseg
Filippos Giannakos [Wed, 6 Mar 2013 11:27:48 +0000 (13:27 +0200)]
Merge branch 'feature-impovexseg' into develop
Filippos Giannakos [Wed, 27 Feb 2013 16:12:43 +0000 (18:12 +0200)]
Add defer support to peers
Add support to peer skeleton to defer requests to another peer, by forwarding
requests to another port.
Filippos Giannakos [Fri, 22 Feb 2013 16:50:13 +0000 (18:50 +0200)]
Reimplement xseg paths and add xseg_forward
Reimplement xseg paths. Paths are a tool for the administrator to dynamically
alter a running xseg setup, by forcing a request to be submitted to a different
port than the original destination.
xseg_forward, on the other hand, allows a peer to alter the normal path of a
request by changing the effective destination of the request, and thus allows
the dynamic creation of temporal paths based on various criteria.
Also, add CAN_RECEIVE and CAN_ACCEPT port flags, allowing each peer to notify
the segment whether it is able to accept or receive on the specified port.
Filippos Giannakos [Fri, 22 Feb 2013 15:55:59 +0000 (17:55 +0200)]
Change libxseg minor version to 2
Filippos Giannakos [Wed, 27 Feb 2013 10:27:13 +0000 (12:27 +0200)]
Fix permissions in ext storage log directory.
Filippos Giannakos [Tue, 26 Feb 2013 10:47:14 +0000 (12:47 +0200)]
Revert "Add ceph dependency to python archipelago setup.py"
This reverts commit
9c609c9b409d2be8c6559f965770773edf6e22ee.
Filippos Giannakos [Mon, 25 Feb 2013 16:06:29 +0000 (18:06 +0200)]
Add ceph dependency to python archipelago setup.py
Filippos Giannakos [Mon, 25 Feb 2013 16:00:41 +0000 (18:00 +0200)]
Remove dummy, vlmc-xseg, pfiled from peers build.
Filippos Giannakos [Mon, 25 Feb 2013 15:23:48 +0000 (17:23 +0200)]
Bump version.
Bumped version to 0.2next.
Filippos Giannakos [Mon, 25 Feb 2013 15:23:01 +0000 (17:23 +0200)]
Add python xseg dependencies to setup.py
Filippos Giannakos [Mon, 25 Feb 2013 15:22:05 +0000 (17:22 +0200)]
Add python archipelago dependencies to setup.py
Filippos Giannakos [Fri, 22 Feb 2013 15:33:38 +0000 (17:33 +0200)]
Change libxseg minor from 0.1 to 1
Filippos Giannakos [Fri, 22 Feb 2013 13:33:23 +0000 (15:33 +0200)]
Fix xseg, archipelago python cleanup
Filippos Giannakos [Mon, 18 Feb 2013 17:26:21 +0000 (19:26 +0200)]
Fix python package names
Filippos Giannakos [Mon, 18 Feb 2013 16:19:19 +0000 (18:19 +0200)]
Fix xseg dependency for archipelago tool
Filippos Giannakos [Tue, 12 Feb 2013 17:53:45 +0000 (19:53 +0200)]
Make archipelago tool handle updates
Filippos Giannakos [Tue, 12 Feb 2013 17:35:31 +0000 (19:35 +0200)]
Makefile now supports userspace-only build
Filippos Giannakos [Tue, 12 Feb 2013 15:22:42 +0000 (17:22 +0200)]
Merge branch 'feature-segdev' into develop
Filippos Giannakos [Mon, 11 Feb 2013 14:41:09 +0000 (16:41 +0200)]
Remove segdev usercount logging
Filippos Giannakos [Mon, 4 Feb 2013 17:22:56 +0000 (19:22 +0200)]
xseg_segdev user driver: close fd on deallocate
Filippos Giannakos [Mon, 4 Feb 2013 16:57:14 +0000 (18:57 +0200)]
xseg_segdev kernel driver: put segdev on initialization
Filippos Giannakos [Mon, 4 Feb 2013 16:30:17 +0000 (18:30 +0200)]
print usercount on segdev destroy
Filippos Giannakos [Mon, 4 Feb 2013 16:28:19 +0000 (18:28 +0200)]
xsegbd: use xseg leave instead of unmapping the segment
Filippos Giannakos [Mon, 4 Feb 2013 15:52:41 +0000 (17:52 +0200)]
xseg_segdev user driver: do not try to destroy old segment on allocation
Filippos Giannakos [Mon, 4 Feb 2013 15:39:25 +0000 (17:39 +0200)]
segdev: put segdev on write error
Filippos Giannakos [Mon, 4 Feb 2013 15:35:27 +0000 (17:35 +0200)]
xseg_segdev driver: segdev_put if allocate fails
Filippos Giannakos [Mon, 4 Feb 2013 15:33:51 +0000 (17:33 +0200)]
xseg_segdev driver: do not destroy old segment when allocating a new one
Filippos Giannakos [Mon, 4 Feb 2013 15:32:26 +0000 (17:32 +0200)]
Fix possible race in segdev_get and segdev_destroy segment
Filippos Giannakos [Tue, 12 Feb 2013 11:32:29 +0000 (13:32 +0200)]
Fix autopkg_debian to work with buildbot
Filippos Giannakos [Tue, 12 Feb 2013 11:01:14 +0000 (13:01 +0200)]
Merge branch 'feature-merklehash' into develop
Filippos Giannakos [Mon, 11 Feb 2013 10:57:04 +0000 (12:57 +0200)]
Mapper: use merkle hash to calculate name of snapshot
Filippos Giannakos [Mon, 11 Feb 2013 18:05:59 +0000 (20:05 +0200)]
Makefiles do not remove version files
Filippos Giannakos [Mon, 11 Feb 2013 18:04:40 +0000 (20:04 +0200)]
Add xseg version file to gitignore
Filippos Giannakos [Mon, 11 Feb 2013 17:26:22 +0000 (19:26 +0200)]
Fix wrong entry points in python-xseg
Filippos Giannakos [Mon, 11 Feb 2013 13:38:15 +0000 (15:38 +0200)]
vlmc: Fix config STORAGE option in mapinfo
Filippos Giannakos [Fri, 8 Feb 2013 14:07:35 +0000 (16:07 +0200)]
Add ctypes import to vlmc.py
Filippos Giannakos [Fri, 8 Feb 2013 13:44:33 +0000 (15:44 +0200)]
Merge branch 'feature-fixwarnings' into develop
Filippos Giannakos [Mon, 4 Feb 2013 14:29:32 +0000 (16:29 +0200)]
Fix compilation warnings
Filippos Giannakos [Fri, 8 Feb 2013 12:32:56 +0000 (14:32 +0200)]
PEP8 fixes for python xseg
Filippos Giannakos [Fri, 8 Feb 2013 12:28:53 +0000 (14:28 +0200)]
PEP8 fixes in vlmc ext_scripts
Filippos Giannakos [Fri, 8 Feb 2013 12:22:45 +0000 (14:22 +0200)]
More PEP8 fixes
Filippos Giannakos [Fri, 8 Feb 2013 12:16:59 +0000 (14:16 +0200)]
PEP8 fixes
Filippos Giannakos [Fri, 8 Feb 2013 10:02:48 +0000 (12:02 +0200)]
Adjust vlmc ext scripts to the new archipelago module
Filippos Giannakos [Fri, 8 Feb 2013 09:22:35 +0000 (11:22 +0200)]
vlmc tool: split showmapped in get mapped and show mapped
also add is_mapped functionality
Filippos Giannakos [Thu, 7 Feb 2013 16:17:31 +0000 (18:17 +0200)]
Fix Xseg_ctx spec parsing
Filippos Giannakos [Thu, 7 Feb 2013 15:06:06 +0000 (17:06 +0200)]
Make vlmc snapshot return snapshot name on success
Filippos Giannakos [Thu, 7 Feb 2013 14:29:34 +0000 (16:29 +0200)]
Fix xsegbd races when releasing/opening device
Filippos Giannakos [Thu, 7 Feb 2013 14:29:09 +0000 (16:29 +0200)]
Make vlmc map return id of xsegbd_device
Filippos Giannakos [Wed, 6 Feb 2013 16:40:17 +0000 (18:40 +0200)]
Make vlmc module function take kwargs
Filippos Giannakos [Wed, 6 Feb 2013 16:25:38 +0000 (18:25 +0200)]
Make archipelago functions, take kwargs
Filippos Giannakos [Wed, 6 Feb 2013 15:08:34 +0000 (17:08 +0200)]
Fix archipelago / vlmc modules
Filippos Giannakos [Wed, 6 Feb 2013 10:35:55 +0000 (12:35 +0200)]
Make archipelago/vlmc tool modular
Filippos Giannakos [Wed, 6 Feb 2013 09:10:14 +0000 (11:10 +0200)]
Add devflow version for xseg
Filippos Giannakos [Tue, 5 Feb 2013 18:19:56 +0000 (20:19 +0200)]
Fix sys.exit in cli entry point
Filippos Giannakos [Tue, 5 Feb 2013 16:41:26 +0000 (18:41 +0200)]
Add archipelago devflow version
Filippos Giannakos [Tue, 5 Feb 2013 15:39:33 +0000 (17:39 +0200)]
Make archipelago tool, python module
Filippos Giannakos [Mon, 4 Feb 2013 11:21:12 +0000 (13:21 +0200)]
Make install use version supplied by version file
Christos Stavrakakis [Wed, 30 Jan 2013 15:53:45 +0000 (17:53 +0200)]
Add ci files
Filippos Giannakos [Tue, 5 Feb 2013 18:22:12 +0000 (20:22 +0200)]
Fix xsegbd_args in archipelago tool
Filippos Giannakos [Mon, 4 Feb 2013 14:10:19 +0000 (16:10 +0200)]
Fix TOPDIR export in base.mk
Filippos Giannakos [Fri, 1 Feb 2013 11:33:49 +0000 (13:33 +0200)]
Make {create, destroy}_segment use xseg bindings
Also fix SPEC encoding bug
Filippos Giannakos [Fri, 1 Feb 2013 09:15:25 +0000 (11:15 +0200)]
Fix imports in vlmc tool
Filippos Giannakos [Thu, 31 Jan 2013 14:23:02 +0000 (16:23 +0200)]
Add experimental map info command
Filippos Giannakos [Thu, 31 Jan 2013 14:00:29 +0000 (16:00 +0200)]
Add vlmc info command
Filippos Giannakos [Thu, 31 Jan 2013 13:14:44 +0000 (15:14 +0200)]
Make vlmc list volumes using python rados module
Filippos Giannakos [Wed, 30 Jan 2013 17:35:53 +0000 (19:35 +0200)]
Pythonize archipelago tool. Also add independent peer manipulation
Filippos Giannakos [Wed, 30 Jan 2013 12:09:10 +0000 (14:09 +0200)]
Make vlmc use configurable lock file dir.
Set as /var/lock as default. Also increased spin lock sleep time to 0.2s