archipelago
10 years agoModify debian dir to new layout
Filippos Giannakos [Mon, 17 Feb 2014 10:39:52 +0000 (12:39 +0200)]
Modify debian dir to new layout

10 years agoFix wrong dependency version
Filippos Giannakos [Thu, 30 Jan 2014 13:49:40 +0000 (15:49 +0200)]
Fix wrong dependency version

10 years agoChange modules depend version to binary version
Filippos Giannakos [Thu, 30 Jan 2014 13:39:37 +0000 (15:39 +0200)]
Change modules depend version to binary version

10 years agoMerge branch 'debian-feature-new-configfile-rebased' into debian-develop
Filippos Giannakos [Tue, 28 Jan 2014 15:11:31 +0000 (17:11 +0200)]
Merge branch 'debian-feature-new-configfile-rebased' into debian-develop

10 years agoModify changelog for 0.3.5 debian-feature-new-configfile-rebased
Filippos Giannakos [Mon, 23 Dec 2013 15:57:24 +0000 (17:57 +0200)]
Modify changelog for 0.3.5

10 years agoModify changelog for 0.3.4
Filippos Giannakos [Tue, 17 Dec 2013 12:08:27 +0000 (14:08 +0200)]
Modify changelog for 0.3.4

10 years agoModify changelog for 0.3.3
Filippos Giannakos [Wed, 4 Dec 2013 15:32:22 +0000 (17:32 +0200)]
Modify changelog for 0.3.3

Conflicts:
debian/changelog

10 years agoAdd ganeti hooks to archipelago ganeti
Filippos Giannakos [Wed, 4 Dec 2013 15:29:18 +0000 (17:29 +0200)]
Add ganeti hooks to archipelago ganeti

10 years agoMerge branch 'debian-feature-changesettingslocation' into debian-develop
Filippos Giannakos [Wed, 31 Jul 2013 07:13:38 +0000 (10:13 +0300)]
Merge branch 'debian-feature-changesettingslocation' into debian-develop

10 years agoInstall archipelago conf file
Filippos Giannakos [Fri, 26 Jul 2013 07:29:19 +0000 (10:29 +0300)]
Install archipelago conf file

10 years agoAdd archipelago-rados-dbg package
Filippos Giannakos [Thu, 25 Jul 2013 12:36:49 +0000 (15:36 +0300)]
Add archipelago-rados-dbg package

10 years agoSplit rados driver to a different package.
Filippos Giannakos [Thu, 25 Jul 2013 12:18:55 +0000 (15:18 +0300)]
Split rados driver to a different package.

Split rados driver to a different package, to avoid depending archipelago to
librados by default.
Also now we depend also to librados >= 0.65 which has all the necessary patches
merged.

10 years agoAdd python-pkg-resources as a dependency
Filippos Giannakos [Thu, 25 Jul 2013 08:16:45 +0000 (11:16 +0300)]
Add python-pkg-resources as a dependency

10 years agoModify kernel source rules version regex
Filippos Giannakos [Thu, 18 Jul 2013 13:15:00 +0000 (16:15 +0300)]
Modify kernel source rules version regex

10 years agoModify version regex to comply with the new devflow
Filippos Giannakos [Thu, 18 Jul 2013 13:12:44 +0000 (16:12 +0300)]
Modify version regex to comply with the new devflow

10 years agoMerge branch 'develop' into debian-develop
Filippos Giannakos [Mon, 22 Jul 2013 11:16:49 +0000 (14:16 +0300)]
Merge branch 'develop' into debian-develop

Conflicts:
xseg/tools/archipelago/archipelago/common.py
xseg/tools/archipelago/archipelago/vlmc.py

10 years agoext_storage: Check for 'pithos:' on create.
Filippos Giannakos [Mon, 22 Jul 2013 10:36:05 +0000 (13:36 +0300)]
ext_storage: Check for 'pithos:' on create.

10 years agoAdapt archipelago defaults to the new config
Filippos Giannakos [Mon, 22 Jul 2013 10:22:01 +0000 (13:22 +0300)]
Adapt archipelago defaults to the new config

10 years agovlmc: Add X_HASH operation
Filippos Giannakos [Mon, 22 Jul 2013 10:15:47 +0000 (13:15 +0300)]
vlmc: Add X_HASH operation

10 years agoarchipelago: Declare segment as Class and modify settings
Filippos Giannakos [Mon, 22 Jul 2013 10:14:25 +0000 (13:14 +0300)]
archipelago: Declare segment as Class and modify settings

Declare xseg segment as class, and make use of its methods. Also modify config
file check to expect segment parameters instead of one segment spec.

10 years agovlmc: Fix get_vlmc_port for all operations
Filippos Giannakos [Fri, 19 Jul 2013 15:24:46 +0000 (18:24 +0300)]
vlmc: Fix get_vlmc_port for all operations

10 years agoarchipelago: Add spec config when creating segment
Filippos Giannakos [Fri, 19 Jul 2013 14:48:11 +0000 (17:48 +0300)]
archipelago: Add spec config when creating segment

10 years agoarchipelago: Fix tabs in python modules
Filippos Giannakos [Fri, 19 Jul 2013 14:42:33 +0000 (17:42 +0300)]
archipelago: Fix tabs in python modules

10 years agoMerge branch 'feature-bench-improvements-rebased2' into develop
Filippos Giannakos [Fri, 19 Jul 2013 14:14:33 +0000 (17:14 +0300)]
Merge branch 'feature-bench-improvements-rebased2' into develop

10 years agobench: Create separate option for progress type
Alex Pyrgiotis [Fri, 19 Jul 2013 08:57:42 +0000 (11:57 +0300)]
bench: Create separate option for progress type

Progress type option is now decoupled from the progress option. This
paves the way for silent progress reports.

10 years agobench: Break usage message in sections
Alex Pyrgiotis [Fri, 19 Jul 2013 07:49:10 +0000 (10:49 +0300)]
bench: Break usage message in sections

10 years agobench: Turn ping off by default
Alex Pyrgiotis [Fri, 19 Jul 2013 07:28:03 +0000 (10:28 +0300)]
bench: Turn ping off by default

10 years agobench: Break large source file into smaller files
Alex Pyrgiotis [Fri, 19 Jul 2013 07:12:47 +0000 (10:12 +0300)]
bench: Break large source file into smaller files

The bench-utils.c file started to have utilities that could stand
semantically under a separate category. Namely, verification is now at
bench-verify.c and progress report under bench-report.c.

10 years agoMerge branch 'feature-vlmcmap' into develop
Filippos Giannakos [Fri, 19 Jul 2013 14:03:38 +0000 (17:03 +0300)]
Merge branch 'feature-vlmcmap' into develop

10 years agovlmc: Do not map an already mounted volume
Filippos Giannakos [Thu, 18 Jul 2013 16:16:22 +0000 (19:16 +0300)]
vlmc: Do not map an already mounted volume

10 years agoMerge branch 'feature-vlmcmultipleports' into develop
Filippos Giannakos [Fri, 19 Jul 2013 14:02:12 +0000 (17:02 +0300)]
Merge branch 'feature-vlmcmultipleports' into develop

10 years agoarchipelago: Fix concurrent vlmc ops
Filippos Giannakos [Fri, 19 Jul 2013 09:30:22 +0000 (12:30 +0300)]
archipelago: Fix concurrent vlmc ops

10 years agoarchipelago: Add support for nested locking
Filippos Giannakos [Fri, 19 Jul 2013 06:28:46 +0000 (09:28 +0300)]
archipelago: Add support for nested locking

10 years agoarchipelago: Add support for concurrent vlmc ops
Filippos Giannakos [Fri, 19 Jul 2013 06:12:53 +0000 (09:12 +0300)]
archipelago: Add support for concurrent vlmc ops

10 years agoMerge branch 'feature-bench-improvements-rebased2' into develop
Filippos Giannakos [Thu, 18 Jul 2013 12:20:02 +0000 (15:20 +0300)]
Merge branch 'feature-bench-improvements-rebased2' into develop

10 years agobench: Introduce new progress report and intervals
Alex Pyrgiotis [Wed, 17 Jul 2013 19:19:54 +0000 (22:19 +0300)]
bench: Introduce new progress report and intervals

This commit renews the progress report that bench has by allowing to
report bandwidth results during benchmark.

Also, an additional functionality is that of report intervals. Report
intervals allow the user to choose exactly when progress report will be
printed.

Both of these features are documented extensively in the usage section.

10 years agobench: Track elapsed time for each timer
Alex Pyrgiotis [Wed, 17 Jul 2013 19:12:23 +0000 (22:12 +0300)]
bench: Track elapsed time for each timer

Also, create two functions timespecsub() and timespecadd() that can
subtract and add two timespec structs.

10 years agobench: Handle progress report better
Alex Pyrgiotis [Tue, 16 Jul 2013 10:39:13 +0000 (13:39 +0300)]
bench: Handle progress report better

Pave the way to include bandwidth/IOPS results during progress report

10 years agobench: Add --objname option
Alex Pyrgiotis [Mon, 8 Jul 2013 11:45:10 +0000 (14:45 +0300)]
bench: Add --objname option

The --objname option allows the user to define the name of the target.
This implies that for the duration of the benchmark, a sole object will
be used. This also comes along with a set of restrictions, that are
throughly explained in the help message

10 years agobench: Fix segfault while freeing resources
Alex Pyrgiotis [Mon, 8 Jul 2013 10:59:33 +0000 (13:59 +0300)]
bench: Fix segfault while freeing resources

10 years agobench: Add --prefix option
Alex Pyrgiotis [Mon, 8 Jul 2013 10:46:03 +0000 (13:46 +0300)]
bench: Add --prefix option

The --prefix option allows the user to choose whatever prefix he/she
wants for the object names that will be generated.

This commit also cleans code that has to do with seeding and creating
object names. It also paves the way for the --objname option.

10 years agobench: Fix minor validation issue
Alex Pyrgiotis [Mon, 10 Jun 2013 15:44:25 +0000 (18:44 +0300)]
bench: Fix minor validation issue

10 years agobench: Ping target peer before starting timers
Alex Pyrgiotis [Sat, 22 Jun 2013 20:00:28 +0000 (23:00 +0300)]
bench: Ping target peer before starting timers

In some cases, the target peer may take some time before it finishes
with its initializations. Since we don't want to measure this idle time,
we send a ping request first and can start start the benchmark after we
receive an ACK.

NOTE: This behaviour can be switched off with the "--ping no" option.

10 years agobench: Remove dead code
Alex Pyrgiotis [Mon, 3 Jun 2013 11:05:07 +0000 (14:05 +0300)]
bench: Remove dead code

10 years agobench: Improve benchmark results
Alex Pyrgiotis [Fri, 31 May 2013 10:35:35 +0000 (13:35 +0300)]
bench: Improve benchmark results

Changes include:

* Show the average request latency when run with insanity "eccentric"
* Don't show bandwidth results for info/delete requests

10 years agobench: Remove dead code
Alex Pyrgiotis [Mon, 3 Jun 2013 11:05:07 +0000 (14:05 +0300)]
bench: Remove dead code

10 years agobench: Fix an LFSR init issue and minor comments
Alex Pyrgiotis [Mon, 3 Jun 2013 10:35:48 +0000 (13:35 +0300)]
bench: Fix an LFSR init issue and minor comments

If bench is requested to issue only one request, the LFSR initialization
fails. Since LFSR is not needed in this case, we simply switch to
sequential pattern for this request.

10 years agobench: Remove unnecessary malloc
Alex Pyrgiotis [Fri, 31 May 2013 15:25:17 +0000 (18:25 +0300)]
bench: Remove unnecessary malloc

10 years agobench: Improve benchmark results
Alex Pyrgiotis [Fri, 31 May 2013 10:35:35 +0000 (13:35 +0300)]
bench: Improve benchmark results

Changes include:

* Show the average request latency when run with insanity "eccentric"
* Don't show bandwidth results for info/delete requests

10 years agobench: Minor correction on seed validation
Alex Pyrgiotis [Mon, 20 May 2013 13:02:24 +0000 (16:02 +0300)]
bench: Minor correction on seed validation

10 years agobench: Fix data verification
Alex Pyrgiotis [Thu, 16 May 2013 08:14:17 +0000 (11:14 +0300)]
bench: Fix data verification

Current situation: In full verification mode, each number written across
a data chunk has 64 bits length, yet, due to a wrong LFSR
initialization, only 31 bits are utilized.

Implement a 63/64 bits utilization.

10 years agoMerge branch 'feature-snapshot' into develop
Filippos Giannakos [Thu, 18 Jul 2013 09:58:59 +0000 (12:58 +0300)]
Merge branch 'feature-snapshot' into develop

10 years agovlmc: add cont_addr option in snapshot command
Filippos Giannakos [Thu, 18 Jul 2013 09:29:53 +0000 (12:29 +0300)]
vlmc: add cont_addr option in snapshot command

10 years agopython Makefile: replace zero-length array with pointer
Filippos Giannakos [Thu, 18 Jul 2013 09:26:13 +0000 (12:26 +0300)]
python Makefile: replace zero-length array with pointer

Replace zero-length array with pointer for the xseg_reply_map flexible array
member.

10 years agoPeers Makefile: Reenable sosd
Filippos Giannakos [Thu, 18 Jul 2013 07:15:25 +0000 (10:15 +0300)]
Peers Makefile: Reenable sosd

10 years agotests: Add SosdTest
Filippos Giannakos [Thu, 18 Jul 2013 07:15:04 +0000 (10:15 +0300)]
tests: Add SosdTest

10 years agososd: Fix compilation error and hashing bug.
Filippos Giannakos [Thu, 18 Jul 2013 07:14:01 +0000 (10:14 +0300)]
sosd: Fix compilation error and hashing bug.

* Fix compilation error
* Correctly return the hased value in hash reply
* Resize X_INFO request when necessary

10 years agomapper: Use correct size for epoch when loading v2 metadata
Filippos Giannakos [Thu, 18 Jul 2013 07:13:23 +0000 (10:13 +0300)]
mapper: Use correct size for epoch when loading v2 metadata

10 years agomapper: Fix calculation of copy up target.
Filippos Giannakos [Thu, 18 Jul 2013 07:12:03 +0000 (10:12 +0300)]
mapper: Fix calculation of copy up target.

When hexlifing epoch and index, use the correct size of the source.

10 years agopeers: Fix bug in hexlify.
Filippos Giannakos [Thu, 18 Jul 2013 07:10:42 +0000 (10:10 +0300)]
peers: Fix bug in hexlify.

Ditch strncpy which alwayes NULL terminates when hexlifing and use a custom
hexlify function.

10 years agotests: Add extra tests for hashing.
Filippos Giannakos [Wed, 17 Jul 2013 12:40:13 +0000 (15:40 +0300)]
tests: Add extra tests for hashing.

10 years agopython: Extend request interface with X_HASH
Filippos Giannakos [Wed, 17 Jul 2013 12:39:49 +0000 (15:39 +0300)]
python: Extend request interface with X_HASH

10 years agososd: Add X_HASH support with precomputed values
Filippos Giannakos [Wed, 17 Jul 2013 12:39:04 +0000 (15:39 +0300)]
sosd: Add X_HASH support with precomputed values

10 years agoxseg: Add X_HASH to xseg protocol.
Filippos Giannakos [Wed, 17 Jul 2013 12:37:43 +0000 (15:37 +0300)]
xseg: Add X_HASH to xseg protocol.

Add X_HASH operator to xseg, and the appopriate X_HASH reply structure. Remove
obsolete snapshot reply structure.

10 years agovlmc: Remove obsolete snapshot handling functionality
Filippos Giannakos [Wed, 17 Jul 2013 12:37:09 +0000 (15:37 +0300)]
vlmc: Remove obsolete snapshot handling functionality

10 years agomapper: Add hash support for snapshots.
Filippos Giannakos [Wed, 17 Jul 2013 12:35:47 +0000 (15:35 +0300)]
mapper: Add hash support for snapshots.

Add the ability to hash snapshots and create pithos compatible images.

Also, as a minor improvement add object_to_map representation to map functions.

10 years agofiled: Support X_HASH with precomputed hashes.
Filippos Giannakos [Wed, 17 Jul 2013 12:31:07 +0000 (15:31 +0300)]
filed: Support X_HASH with precomputed hashes.

Support X_HASH with precomputed hash values, if the same block was hashed
before.

10 years agotests: Add testing for cloning archipelago snapshots
Filippos Giannakos [Tue, 16 Jul 2013 08:14:39 +0000 (11:14 +0300)]
tests: Add testing for cloning archipelago snapshots

10 years agomapper: Add XF_CONTADDR flag and improve cloning
Filippos Giannakos [Tue, 16 Jul 2013 08:13:49 +0000 (11:13 +0300)]
mapper: Add XF_CONTADDR flag and improve cloning

Support cloning from both pithos images and archipelago snapshots.

10 years agomapper: Fix get_mapnode boundary.
Filippos Giannakos [Tue, 16 Jul 2013 08:12:49 +0000 (11:12 +0300)]
mapper: Fix get_mapnode boundary.

Return NULL, when requesting object index == nr_obs

10 years agomapper: Fail when issuing a map object load fails
Filippos Giannakos [Tue, 16 Jul 2013 08:11:35 +0000 (11:11 +0300)]
mapper: Fail when issuing a map object load fails

10 years agomapper: Always use datalen buffers when loading v2
Filippos Giannakos [Tue, 16 Jul 2013 08:10:44 +0000 (11:10 +0300)]
mapper: Always use datalen buffers when loading v2

10 years agomapper: Set correct map object when copying up
Filippos Giannakos [Tue, 16 Jul 2013 08:09:57 +0000 (11:09 +0300)]
mapper: Set correct map object when copying up

10 years agoMerge branch 'feature-functionaltest' into feature-snapshot
Filippos Giannakos [Mon, 15 Jul 2013 13:39:13 +0000 (16:39 +0300)]
Merge branch 'feature-functionaltest' into feature-snapshot

10 years agotesting: Add tests for concurrent operations
Filippos Giannakos [Mon, 15 Jul 2013 13:28:43 +0000 (16:28 +0300)]
testing: Add tests for concurrent operations

10 years agopython xseg: Enrich request/xseg python api.
Filippos Giannakos [Mon, 15 Jul 2013 13:26:22 +0000 (16:26 +0300)]
python xseg: Enrich request/xseg python api.

Add more constructors for the Request class.
Add xseg_wait_requests to wait on multiple requests.
Modify vlmc script to use the new constructors.

10 years agofiled: Add debug logging while reading/writing
Filippos Giannakos [Mon, 15 Jul 2013 13:25:29 +0000 (16:25 +0300)]
filed: Add debug logging while reading/writing

10 years agotests: Add basic functionality testing for peers
Filippos Giannakos [Fri, 12 Jul 2013 07:47:24 +0000 (10:47 +0300)]
tests: Add basic functionality testing for peers

10 years agoImprove python archipelago common infrastructure.
Filippos Giannakos [Thu, 11 Jul 2013 15:12:20 +0000 (18:12 +0300)]
Improve python archipelago common infrastructure.

The changes include:
 * Convert pfiled to filed and add the extra arguments
 * Check if arguments are not None instead of simply checking for arguments
 * Move request to seperate method and remove __del__ method
 * Add get_serviced and get_datalen to request
 * Add support to set string add data request
 * Make all peer executables relative to a BIN_DIR
 * Create/destroy segment takes spec as arguments
 * Parsing spec on Xsegctx __init__ does not modify spec parameter

10 years agopeer: Create pidfile with read/write permissions
Filippos Giannakos [Thu, 11 Jul 2013 15:11:23 +0000 (18:11 +0300)]
peer: Create pidfile with read/write permissions

10 years agomapper: Fail loading of deleted volumes
Filippos Giannakos [Thu, 11 Jul 2013 14:38:18 +0000 (17:38 +0300)]
mapper: Fail loading of deleted volumes

10 years agomapper: Fail request when range is not valid
Filippos Giannakos [Thu, 11 Jul 2013 14:37:43 +0000 (17:37 +0300)]
mapper: Fail request when range is not valid

10 years agoSet zero sized clone to inherit the size of the parent
Filippos Giannakos [Thu, 11 Jul 2013 14:35:15 +0000 (17:35 +0300)]
Set zero sized clone to inherit the size of the parent

Uniformly set that when the requested size is zero, then either inherit the size
of the parent map, or fail if there is no parent.

10 years agovlmcd: resize request on X_INFO
Filippos Giannakos [Thu, 11 Jul 2013 14:33:08 +0000 (17:33 +0300)]
vlmcd: resize request on X_INFO

10 years agomapper: Add resize request on X_INF
Filippos Giannakos [Thu, 11 Jul 2013 14:32:46 +0000 (17:32 +0300)]
mapper: Add resize request on X_INF

10 years agofiled: Add resize request on X_INFO
Filippos Giannakos [Thu, 11 Jul 2013 14:30:49 +0000 (17:30 +0300)]
filed: Add resize request on X_INFO

10 years agomapper: add debug messages while handling objects in v2
Filippos Giannakos [Thu, 11 Jul 2013 14:30:04 +0000 (17:30 +0300)]
mapper: add debug messages while handling objects in v2

10 years agoxseg: remove some unused logging
Filippos Giannakos [Thu, 11 Jul 2013 14:23:34 +0000 (17:23 +0300)]
xseg: remove some unused logging

10 years agomapperd: Modify version2 map IO functions.
Filippos Giannakos [Mon, 8 Jul 2013 08:59:51 +0000 (11:59 +0300)]
mapperd: Modify version2 map IO functions.

Modify version2 map IO functions to support load/write map data for a specified
object range.

10 years agomapperd: Add counter on map to count used objects
Filippos Giannakos [Mon, 8 Jul 2013 08:57:45 +0000 (11:57 +0300)]
mapperd: Add counter on map to count used objects

Add map counter to count used objects, so we can quickly calculate whether all
objects are in READY state, and there are no pending actions on them.

10 years agomapperd: Fail load if map upgrade fails
Filippos Giannakos [Mon, 8 Jul 2013 08:52:43 +0000 (11:52 +0300)]
mapperd: Fail load if map upgrade fails

10 years agofiled: Change req->datalen with req->size.
Filippos Giannakos [Mon, 8 Jul 2013 08:51:44 +0000 (11:51 +0300)]
filed: Change req->datalen with req->size.

Do not use req->datalen to fullfill requests, but req->size, which was the
requested size.

10 years agoSosd: Change req->datalen with req->size.
Filippos Giannakos [Mon, 8 Jul 2013 08:50:33 +0000 (11:50 +0300)]
Sosd: Change req->datalen with req->size.

Do not use req->datalen to fullfill requests, but req->size, which was the
requested size.

10 years agomt-sosd: Use common hash.c for hashing
Filippos Giannakos [Wed, 3 Jul 2013 14:37:39 +0000 (17:37 +0300)]
mt-sosd: Use common hash.c for hashing

10 years agoxhash: Add XHASHS_ENOENT return value on remove
Filippos Giannakos [Wed, 3 Jul 2013 10:24:47 +0000 (13:24 +0300)]
xhash: Add XHASHS_ENOENT return value on remove

10 years agomapper: Split loading/writing map metadata from map data
Filippos Giannakos [Wed, 3 Jul 2013 10:21:37 +0000 (13:21 +0300)]
mapper: Split loading/writing map metadata from map data

10 years agomapper: Make callbacks state threads
Filippos Giannakos [Wed, 3 Jul 2013 10:21:19 +0000 (13:21 +0300)]
mapper: Make callbacks state threads

10 years agomapper: Fix v2 map loading/writing.
Filippos Giannakos [Tue, 2 Jul 2013 15:54:06 +0000 (18:54 +0300)]
mapper: Fix v2 map loading/writing.

When changing chunks, make sure that each chunks ends at object border, so we
can safely load/store multiple chunks of map data.

10 years agomapper: Add support to wait for a request when applicable
Filippos Giannakos [Tue, 2 Jul 2013 15:18:43 +0000 (18:18 +0300)]
mapper: Add support to wait for a request when applicable

10 years agoRename archip-pfiled to archip-filed
Filippos Giannakos [Tue, 2 Jul 2013 14:01:09 +0000 (17:01 +0300)]
Rename archip-pfiled to archip-filed