Statistics
| Branch: | Tag: | Revision:

root @ adc46059

# Date Author Comment
adc46059 09/28/2012 12:19 pm Christos Stavrakakis

Perform allocations in separate transactions

Add ForUpdateManager to the Backend model, in order to guarantee
exclusive order when allocating a VM.

Use manual transactions when allocating Backend and public IP for a new
VM, since these allocations need exclusive access (SELECT..FOR UPDATE)...

5706f527 09/28/2012 12:19 pm Christos Stavrakakis

Fix typo

b123630d 09/28/2012 11:10 am Christos Stavrakakis

Merge branch 'public_networks' into devel-0.11

Conflicts:
snf-cyclades-app/synnefo/logic/dispatcher.py

5da13d77 09/27/2012 04:42 pm Vangelis Koukis

Close DB conn in dispatcher before every message

Have the dispatcher close the Django DB connection before processing
an incoming message. This allows the actual, underlying DBAPI connection
to return to the DB connection pool, if used, and enables the dispatcher...

5f6ad491 09/27/2012 03:59 pm Christos Stavrakakis

Fix cleanup of DB pooled connections

Fix bug in cleanup of DB pooled connections before putting them back to
the pool. The way the connection was cleaned, resulted in mishandling of
the transactions.

Accoring to psycopg2 docs, in case of a READ COMMITED isolation level:...

3f018af1 09/27/2012 03:59 pm Christos Stavrakakis

Make snf-dispatcher work with gevent

Remove workers from snf-dispatcher in order to make it compatible with gevent
and specifically the green, monkey patched version of psycopg2
(synnefo.lib.db.psyco_gevent). Also after daemonizing, gevent must be
reinitialized because the deamon context forks.

7f9f3018 09/27/2012 03:59 pm Christos Stavrakakis

Add pool-create and pool-remove commands

6c9c95d8 09/27/2012 03:59 pm Christos Stavrakakis

Change process title to simplify 'ps' display

Use python-setproctitle to change the process title of the running
daemon. Make it appear as a native executable to simplify
administration. Only set executable name, because cannot set individual
values for arguments.

26659d3a 09/27/2012 03:59 pm Christos Stavrakakis

Add setproctitle to dependencies

5e1a5f9b 09/27/2012 03:59 pm Christos Stavrakakis

Fix presentation of pools

1ed2557d 09/27/2012 03:59 pm Christos Stavrakakis

Remove unnecessary locking

Selecting FOR_UPDATE on network object is not any more required, since
the IP Pool have been moved to the IPPoolTable. Selecting FOR_UPDATE is
needed there.

ec22dfc4 09/27/2012 03:59 pm Christos Stavrakakis

Fix two bugs in network reconciliation

d2e73c0c 09/27/2012 03:59 pm Christos Stavrakakis

Fix race conditions when updating network objects

Use for_update() object manager in order to exclude race conditions
when updating network objects.

8ec69269 09/27/2012 03:59 pm Christos Stavrakakis

Fix double logging of dispatcher

Dispatcher had as default handlers the console and the file handler.
However, when daemonizing, we redirect the stdout and stderr to the file
handler, which resulted in the log messages to be printed twice. So we
remove the console handler from the dict configuration, and we only add...

24683dda 09/27/2012 03:59 pm Christos Stavrakakis

Fix typos

e64e7ade 09/27/2012 03:59 pm Christos Stavrakakis

Catch corner case in reconciliation

Catch error produced when the DB entry for an orphan VM in Ganeti does
not exist. This may happen because either an entry from DB was removed
or because a VM in Ganeti with the synnefo prefix was created.

d2f3853b 09/27/2012 03:59 pm Christos Stavrakakis

Fix broken logging

Rename LOGGING setup to LOGGING_SETUP, as this name is required by
the LoggingConfigMiddleware. Django's default LOGGING setting can
be used with Django>=1.3

Also change default handler to 'console' and add a more verbose format.

e66c7993 09/27/2012 03:59 pm Georgios D. Tsoukalas

don't take mutex for cleanup

a8935947 09/27/2012 03:59 pm Georgios D. Tsoukalas

Enhance pooling API, implementation, and tests

- Verify objects (new and old) with a hook _pool_verify.
Discard objects from the pool that fail verification.
A newly created object failing verification is fatal.

- Remove all hooks from the pool set access critical section....

fb02390b 09/27/2012 03:59 pm Georgios D. Tsoukalas

complete some functionality

d8fe0948 09/27/2012 03:59 pm Georgios D. Tsoukalas

port pooled psycopg2 to new pool api

c156d23a 09/27/2012 03:59 pm Georgios D. Tsoukalas

remove obsolete kwarg verify

b9c83400 09/27/2012 03:59 pm Christos Stavrakakis

Fix bug in http connection pool

Do not reset connetion _pool attribute, since it is not (yet?)
initialized when getting the object from the pool.

68453d22 09/27/2012 03:59 pm Christos Stavrakakis

Enhange logging in db and http pools

a4148a73 09/27/2012 03:59 pm Christos Stavrakakis

Fix no-time in eventd messages

Ganeti opcodes may not have an end-timestamp when they go to
an error state. In that case send the job end-timestamp.

afc9e0ef 09/27/2012 03:35 pm Christos Stavrakakis

Add setproctitle to dependencies

e32d0966 09/26/2012 12:29 pm Vangelis Koukis

Rename Cyclades management commands for uniformity

Rename Cyclades management commands for uniformity.

1dc821c9 09/21/2012 04:28 pm Christos Stavrakakis

Change process title to simplify 'ps' display

Use python-setproctitle to change the process title of the running
daemon. Make it appear as a native executable to simplify
administration. Only set executable name, because cannot set individual
values for arguments.

47d1d754 09/21/2012 11:23 am Christos Stavrakakis

Add some logging to backend allocator

4936f2e2 09/21/2012 10:42 am Christos Stavrakakis

Make snf-dispatcher work with gevent

Remove workers from snf-dispatcher in order to make it compatible with gevent
and specifically the green, monkey patched version of psycopg2
(synnefo.lib.db.psyco_gevent). Also after daemonizing, gevent must be
reinitialized because the deamon context forks.

fdc94944 09/20/2012 12:14 pm Christos Stavrakakis

Refactor IP pools

Treat IP pool the same way as bridge and mac-prefix pools. The pool is
removed from the Network table and moved to an IPPoolTable containing the
IP pools for all networks. Also added support for external reservations.

Migration files for transfering the pool to the new tables are included....

2dbe7563 09/20/2012 12:14 pm Christos Stavrakakis

Add tests for PoolManager(s)

121c7507 09/20/2012 12:14 pm Christos Stavrakakis

Fix bugs in IPPool

- Do not always reserve gateway
- Initial reservations must be in reserved map

9727ac2f 09/20/2012 12:14 pm Christos Stavrakakis

Handle case of no pools in reconcile-pools

99a74636 09/19/2012 01:16 pm Christos Stavrakakis

Catch error in removing nic

0eed6cf8 09/19/2012 01:16 pm Christos Stavrakakis

Management commands for pools

Create three new management commands for listing, showing and modifying
a pool.

af6a3bc5 09/19/2012 01:16 pm Christos Stavrakakis

Unique mac-prefix only for mac-filtered networks

b533e9b2 09/19/2012 01:16 pm Christos Stavrakakis

Add logging to synefo.db.models

03992c72 09/19/2012 01:16 pm Christos Stavrakakis

Refactor bridge and mac-prefix pool tables

Redesign the implementation of pools and map mac-prefix and bridge pools
to that design.

Each pool type is mapped to a table in the database, and each row of
that table is a pool of that type. All pool tables must contain the...

4a37a0be 09/19/2012 11:34 am Christos Stavrakakis

Fix bug in server-inspect

56543400 09/19/2012 11:21 am Christos Stavrakakis

Fix bug in snf-manage network-remove

b7e74f05 09/19/2012 11:21 am Christos Stavrakakis

Wrap network-create command in transaction

00698e35 09/18/2012 10:31 am Christos Stavrakakis

Fix cleanup of DB pooled connections

Fix bug in cleanup of DB pooled connections before putting them back to
the pool. The way the connection was cleaned, resulted in mishandling of
the transactions.

Accoring to psycopg2 docs, in case of a READ COMMITED isolation level:...

f2ccb23e 09/18/2012 10:23 am Christos Stavrakakis

Fix no-time in eventd messages

Ganeti opcodes may not have an end-timestamp when they go to
an error state. In that case send the job end-timestamp.

702850c4 09/12/2012 01:17 pm Christos Stavrakakis

Enhange logging in db and http pools

ac7e2fd7 09/12/2012 01:16 pm Christos Stavrakakis

Fix bug in http connection pool

Do not reset connetion _pool attribute, since it is not (yet?)
initialized when getting the object from the pool.

3930527c 09/11/2012 01:43 pm Christos Stavrakakis

Merge branch 'pooling' into devel-0.11

a0b5eea5 09/10/2012 12:03 pm Christos Stavrakakis

Fix broken logging

Rename LOGGING setup to LOGGING_SETUP, as this name is required by
the LoggingConfigMiddleware. Django's default LOGGING setting can
be used with Django>=1.3

Also change default handler to 'console' and add a more verbose format.

a1284c0c 09/07/2012 06:24 pm Georgios D. Tsoukalas

remove obsolete kwarg verify

341c818e 09/07/2012 05:41 pm Dimitris Aragiorgis

Enable hotplug during create_instance

This passes hotplug = True to ganeti and enables hotplug per instance.
It checks GANETI_USE_HOTPLUG setting.

Signed-off-by: Dimitris Aragiorgis <>

47c66641 09/07/2012 05:41 pm Dimitris Aragiorgis

Support multiple public networks in UI

Encapsulate in nic dict a public flag (network.public) and
remove twisted oldstyle identification way (network_id == 'public')
for retrieving public interfaces.
All network references are done via their network_id.
...

db2eb7c8 09/07/2012 05:36 pm Dimitris Aragiorgis

Remove add_reserved_ips from rapi client

In order to be compatible with grnet and upstream ganeti

Signed-off-by: Dimitris Aragiorgis <>

8d03316d 09/07/2012 05:36 pm Dimitris Aragiorgis

Allow connecting a VM to a network only if active

Signed-off-by: Dimitris Aragiorgis <>

7cfbbf32 09/07/2012 05:36 pm Christos Stavrakakis

New method for creating BackendNetwork entries

Create a new method to the Network model, in order to create the
BackendNetwork entries tha correspond to this network. Also remove
the custom save method, since when multiple public network will be
supported, we want them to be correlated with specific backends.

370f69ec 09/07/2012 05:36 pm Christos Stavrakakis

Remove dead code

7fede91e 09/07/2012 05:36 pm Christos Stavrakakis

Implement multiple public networks

Implement multiple public networks in order to support:
- different public networks per backend
- non-consecutive IP ranges in each backend

To support this, public networks are treated different from private
ones. Namely, they are correlated with one backend (and so have...

b14725eb 09/07/2012 05:36 pm Christos Stavrakakis

Do not allow creation of two big/small networks

- Networks smaller than /29 are not supported in Ganeti.
- Big networks will result in degraded performance in Ganeti
and DB due to the big reservation map.

30418cbe 09/07/2012 05:36 pm Christos Stavrakakis

Reserve gateway's IP when creating an IP pool

9ef880a4 09/07/2012 05:35 pm Dimitris Aragiorgis

Update rapi to reflect to the upstream ganeti

reserved_ips->add_reserved_ips
Remove Network*All()

Signed-off-by: Dimitris Aragiorgis <>

c4dcfcf6 09/07/2012 03:36 pm Georgios D. Tsoukalas

port pooled psycopg2 to new pool api

90644e50 09/07/2012 03:15 pm Georgios D. Tsoukalas

don't take mutex for cleanup

57607cbd 09/07/2012 03:15 pm Georgios D. Tsoukalas

Enhance pooling API, implementation, and tests

- Verify objects (new and old) with a hook _pool_verify.
Discard objects from the pool that fail verification.
A newly created object failing verification is fatal.

- Remove all hooks from the pool set access critical section....

afe19961 09/07/2012 03:15 pm Georgios D. Tsoukalas

complete some functionality

ab133e68 09/07/2012 01:25 pm Vangelis Koukis

Move pooling psycopg2 wrapper to synnefo.lib.db

Move pooling psycopg2 wrapper to synnefo.lib.db,
please see commit e83ed1fb8fb4d0d19c2d3f86152fde60b3cadfd9 for all the
details on how to use the pooling wrapper for psycopg2.

27cfa807 09/06/2012 05:19 pm Vangelis Koukis

Pool HTTP connections to the astakos service

Have the astakos client use pooled HTTP connections to the astakos
service, uses facilities from synnefo/lib/pool/http.py.

818bb9bf 09/04/2012 04:45 pm Georgios D. Tsoukalas

simple http_request objects over connection pools

2adcd773 08/31/2012 04:47 pm Kostas Papadimitriou

Do not send reserved_ips argument

In rapi CreateNetwork method

e49565e1 08/27/2012 05:26 pm Kostas Papadimitriou

Propagate NIC change event to the associated vm

so that nic changed attributes get propagated to the vm views (e.g. nic ip
change).

053d0dfc 08/27/2012 03:42 pm Constantinos Venetsanopoulos

Update admin guide to reflect Cyclades v0.11

In Cyclades v0.11 we introduce support for multiple backends
and support for Public/Private Network management. Plus,
management of IP and MAC address pools.

The quick install admin guide is updated to reflect those changes....

97d3fca1 08/27/2012 12:42 pm Kostas Papadimitriou

Catch error caused by duplicate NIC removal in ui

NICs get removed once their associated vm or network trigger their
corresponding remove signal. Synchronous removal of vm/network will
cause duplicate NIC removes to be called.

e83ed1fb 08/25/2012 03:53 pm Vangelis Koukis

Add pooling synnefo.db.pooled_psycopg2 DBAPI drv

Add DBAPI driver synnefo.db.pooled_psycopg2, which is a
thin, pooling wrapper around psycopg2.

To use, first monkey-patch psycopg2:

from synnefo.db.pooled_psycopg2 import monkey_patch_psycopg2
monkey_patch_psycopg2()...

7138c600 08/25/2012 03:03 pm Vangelis Koukis

Make pool more robust on obj creation failure

Make ObjectPool handle failures during object creation more robustly,
and release the pool semaphore.

3447b13d 08/24/2012 07:17 pm Vangelis Koukis

Extend ObjectPool functionality, tests

09cdd926 08/24/2012 02:32 pm Vangelis Koukis

Rename ObjectPool methods to pool_{put(), get()}

222b0a40 08/09/2012 04:35 pm Kostas Papadimitriou

Revert plankton backend to its original state

trash development code was added by mistake in commit `c6dda6c`

c6dda6cd 08/06/2012 05:56 pm Kostas Papadimitriou

Update GANETI_DISK_TEMPLATES setting description

Describe how to set ext disk template provider.

76cc5746 08/06/2012 05:49 pm Kostas Papadimitriou

Include common date utc formatting methods

296682fe 08/06/2012 11:45 am Kostas Papadimitriou

Support for ext disk providers

Extract provider from flavor disk template name and set the additional
`provider` parameter in disk options. Flavor disk_template should be
formated as `ext_<provider_name>`.

3878553d 08/06/2012 11:30 am Kostas Papadimitriou

Networks list api response fix

include deleted networks in reponse if changes-since parameter is set. This
enables ui application to identify deleted networks and remove them from
user interface.

5c029c29 08/06/2012 11:28 am Kostas Papadimitriou

Remove spare div from html

8aed3cc3 08/06/2012 11:28 am Kostas Papadimitriou

Fix html typo

0b682c29 08/06/2012 11:28 am Kostas Papadimitriou

Firefox ui fix

add overflow to `hidden` to properly clear upper div in single vm view.

de67123e 08/04/2012 01:34 am Vangelis Koukis

Support timeout in ObjectPool.get(), fix tests

Support optional timeout argument in ObjectPool.get(),
if supported by the underlying Semaphore implementation.

Implement command-line argument for ObjectPool unit tests to run
inside gevent monkey-patched environment.

749b8b8e 08/03/2012 06:21 pm Vangelis Koukis

Make minor fixes to Singleton class unit tests

45e32a00 08/03/2012 06:16 pm Vangelis Koukis

Add ObjectPool class in synnefo.lib

Add ObjectPool class in synnefo.lib.
The ObjectPool class is an abstract pool of objects,
based on Semaphore locks, as provided by the threading
package.

Tested to work with a gevent-monkey-patched threading package.
Uncomment the relevant lines in pool/tests.py to test with greenlet...

1d7c75e4 08/01/2012 06:08 pm Kostas Papadimitriou

Fix servers api list response

filter out deleted vms !!!!!!!

e524ed5f 08/01/2012 05:38 pm Kostas Papadimitriou

Fix servers api response

do not filter out deleted vms if valid changes-since parameter is set on
the request. This allows ui application to be able to identify and remove
deleted vms.

0224049a 07/27/2012 09:48 pm Christos Stavrakakis

Add check for maximum secret encryption key

SECRET_ENCRYPTION_KEY can be up to 32 bytes long, since bigger keys are
not supported by AES encryption. Add comment to the setting and better
message to exception.

601ce073 07/27/2012 09:26 pm Christos Stavrakakis

Fix bug in migration file

Do not allocate a bridge from the pool for deleted networks.

ca8b1c21 07/27/2012 07:28 pm Christos Stavrakakis

Rename setting

Rename ENCRYPTION_KEY to SECRET_ENCRYPTION_KEY to prevent from being
displayed at django debug traces.

16ddd1fe 07/27/2012 07:26 pm Christos Stavrakakis

Add setting indicating whether to use hotplug

8731a246 07/27/2012 07:20 pm Christos Stavrakakis

Revert "Start pool indexing from 0"

This reverts commit 2f998a3756c6fc7ddce9e0f83dea738e473320b8.

06e47471 07/27/2012 05:14 pm Christos Stavrakakis

Update Changelog

f91dee69 07/27/2012 04:20 pm Christos Stavrakakis

Fix bug in creating network

Exclude deleted networks.

f15a0994 07/27/2012 03:50 pm Christos Stavrakakis

New dependencies of snf-cyclades-app

2f998a37 07/27/2012 03:41 pm Christos Stavrakakis

Start pool indexing from 0

f4a66897 07/27/2012 03:40 pm Christos Stavrakakis

Limit allowed private networks per user

51fc0054 07/27/2012 03:13 pm Christos Stavrakakis

Get cluster name from Ganeti

Query Ganeti for getting the cluster name, instead of reading the
ssconf files.

b606853b 07/27/2012 02:59 pm Christos Stavrakakis

Merge branch 'master' into devel-0.11

Conflicts:
snf-cyclades-gtools/synnefo/ganeti/eventd.py

3f52dd82 07/27/2012 02:57 pm Christos Stavrakakis

Management command for removing backend