Statistics
| Branch: | Tag: | Revision:

root / snf-cyclades-app / synnefo / logic @ b578d9e7

# Date Author Comment
b578d9e7 11/22/2012 01:26 pm Christos Stavrakakis

snf-dispatcher: Update NICs only when needed

Do not remove and recreate the NICs of a VM when NICs have not changed.

a8858945 11/22/2012 01:24 pm Christos Stavrakakis

Make amqp_puka get logger as argument

2333a2c4 11/16/2012 02:54 pm Christos Stavrakakis

Check clustername when modifying backend

f7759b2d 11/16/2012 12:55 pm Christos Stavrakakis

Add check for clustername when adding backend

d4be7102 11/16/2012 12:24 pm Christos Stavrakakis

Fix lectical in pool-modify

f0fe50c2 11/12/2012 05:14 pm Christos Stavrakakis

Fix typo

b84ed662 11/09/2012 05:32 pm Christos Stavrakakis

Refactor code in management commands

39a6388d 11/09/2012 01:08 pm Christos Stavrakakis

Improve management commands

6639a243 11/08/2012 06:19 pm Christos Stavrakakis

Merge branch 'master' into devel-0.12

Conflicts:
snf-cyclades-app/synnefo/api/management/commands/flavor-list.py
snf-cyclades-app/synnefo/api/management/commands/server-list.py
snf-cyclades-app/synnefo/api/networks.py
snf-cyclades-app/synnefo/logic/rapi.py

fdfd8c6d 11/08/2012 01:21 pm Christos Stavrakakis

Fix dispatcher logging

7f2dbcad 11/07/2012 12:52 pm Christos Stavrakakis

Use select_for_update for VirtualMachines

Use select_for_update method when accessing VM objects to prevent
race conditions. Also move start_action from logic to api.

fd2bdbb2 11/07/2012 11:04 am Christos Stavrakakis

Process OP_NETWORK_SET_PARAMS Ganeti jobs

Modify snf-ganeti-eventd and snf-dispatcher to handle jobs with
OP_NETWORK_SET_PARAMS opcode. Handling of these jobs is needed
to update the IP Pool of network, when reserving and releasing
IPs directly from the Ganeti backend.

d1b1e683 11/06/2012 01:30 pm Dimitris Aragiorgis

PUBLIC_ROUTED minor changes

Do not allow api to create public networks and type other
than PRIVATE_MAC_FILTERED/PRIVATE_PHYSICAL_VLAN.

Rename PUBLIC_ROUTED_USE_POOL setting to PUBLIC_USE_POOL.

Let synnefo do IP pool management only if network is private or...

6de25206 11/06/2012 12:57 pm Dimitris Aragiorgis

Add conflicts_check option in network rapi calls

Ganeti by default checks for conflicting IPs when creating/connecting
networks. This feature can be bypassed for private networks. We do
not care for IP uniqueness in case of private networks as long as they...

a406663f 11/06/2012 12:57 pm Dimitris Aragiorgis

Update rapi client

Get rapi client from snf-ganeti. Add depends argument (for chain jobs)
where needed.

Argument tags must be a list of strings and arguments add_reserved_ips and
remove_reserved_ips must be None or a list of strings.

Signed-off-by: Dimitris Aragiorgis <>

2d762302 11/06/2012 12:57 pm Dimitris Aragiorgis

Minor backend fixes in networks

Add conflicts_check option when creating and add network. Check only
if a network is public.

Let tags argument in rapi calls be a list.

Signed-off-by: Dimitris Aragiorgis <>

f66d8b04 11/05/2012 09:45 am Christos Stavrakakis

Display VM build percentage in management commands

b74d3b64 11/05/2012 09:14 am Christos Stavrakakis

Allow custom link, mac_prefix in network-create

Add extra command line arguments, to allow the user specify the
link and mac_prefix of the new network if needed.

5dd9d123 11/02/2012 11:29 pm Christos Stavrakakis

Introduce per-user allocation of VMs to backends

Add BACKEND_PER_USER setting, a mapping from users to backends. This setting
can by used in order to bypass the default backend allocation process, and
allocate all the new VMs of a specific user to a specific Backend.

e221ade2 11/01/2012 01:24 pm Christos Stavrakakis

Do not allow any API action on suspended VMs

Do not allow any modification of any administratively suspended VM through the
API.

038d6772 10/30/2012 01:00 pm Christos Stavrakakis

Merge branch 'ui-0.11' into devel-0.12

71ec054d 10/30/2012 12:49 pm Christos Stavrakakis

Allow custom link, mac_prefix in network-create

Add extra command line arguments, to allow the user specify the
link and mac_prefix of the new network if needed.

1e2cc762 10/29/2012 02:28 pm Kostas Papadimitriou

Merge remote-tracking branch 'origin/devel-0.12' into ui-0.11

ed275428 10/24/2012 01:27 pm Christos Stavrakakis

Merge branch 'master' into devel-0.12

Conflicts:
snf-cyclades-app/setup.py

eeffcbea 10/23/2012 04:28 pm Kostas Papadimitriou

Merge branch 'devel-0.12' into ui-0.11

67619fd7 10/23/2012 04:19 pm Christos Stavrakakis

Implement queue-retry management command

Implement a new command which consumes all the messages from the
dead-letter queues and re-sends the messages back to the snf-dispatcher
in order to handle them again.

Messages are annotated with "from-dead-letter" field in order to detect...

40ef487d 10/23/2012 03:50 pm Christos Stavrakakis

Fix bug, UI not updating NICs

Dummy-save the Network when processing messages from Ganeti hooks
about a VM NICs. This is needed, because the UI uses the HTTP changes-sinces
when requesting VMs and Networks, and combines the result in order to show
the new NICs. Not updating the Network object, resulted in VMs IPv4...

659de616 10/23/2012 03:07 pm Christos Stavrakakis

Declare dead-letter exchanges and queues

Modify snf-dispatcher in order to declare the dead-letter exchanges and
queues

b9d91e62 10/23/2012 03:07 pm Christos Stavrakakis

Use basic_reject in snf-dispatcher callbacks

Modify snf-dispatcher callbacks in order to reject messages that caused
an unknown exception. Because a dead-letter-exchange is associated
with the queues that snf-dispatcher is using, these messages will be
transfered to the dead-letter-exchange.

37db18f3 10/19/2012 12:06 pm Christos Stavrakakis

Add mac2eui64 to snf-cyclades-app

Copy mac2eui64 from snf-network to snf-cyclades-app, since it is
now used in the reconciliation process to caclculate the IPv6
address of the VMs.

b1bb9251 10/18/2012 06:24 pm Christos Stavrakakis

Reconnect to different Broker if idle for too long

Modify snf-dispatcher and add a timeout option to basic_wait method. If
timeout is reached, the connection to the current RabbitMQ broker is
closed and dispatcher tries to connect to a different one.

This commit is supplementary of 6d27ead, which detects connection...

6d27eadd 10/18/2012 05:16 pm Christos Stavrakakis

Detect and handle connection problems in amqp_puka

Use the TCP keepalive feature in amqp_puka in order to detect connection
errors to the RabbitMQ broker, and retry to a different one. Also setup
the keepalive parameters to relative small values, to detect failures...

3308a83f 10/17/2012 11:58 am Christos Stavrakakis

Fix bug in reconcile-networks

When only detecting, reconciliation can not continue, if the
BackendNetwork object does not exist.

87920bc3 10/16/2012 08:50 pm Christos Stavrakakis

Do not give IPs to networks without dhcp

dab038a2 10/16/2012 01:38 pm Christos Stavrakakis

Fix typos

c7cbb6f3 10/16/2012 12:47 pm Christos Stavrakakis

Use - instead of _ in backend-update-status

122c4019 10/15/2012 12:56 pm Christos Stavrakakis

Fix bug in reconcile-networks

Fix bug when connecting a network to a nodegroup in network
reconciliation. Use connect_network instead of obsolete
connect_network_group method. Also fix job dependency handling in this
method.

e97288bc 10/15/2012 12:55 pm Christos Stavrakakis

Refactor process op and network status callbacks

d01cd522 10/10/2012 03:19 pm Christos Stavrakakis

Various fixes about logging

Fix logging from snf-manage commands. Modify snf-manage implementation
in order to setup the loggers for all snf-manage commands. Configuration
is based on SNF_MANAGE_LOGGING_SETUP setting from snf-common or on
basicConfig if that setting is missing....

e60b4630 10/10/2012 01:36 pm Christos Stavrakakis

Fix bug when synchronously creating networks

fa454545 10/10/2012 01:05 pm Christos Stavrakakis

Release address only if a NIC has an IP

0cd83e6c 10/10/2012 01:01 pm Christos Stavrakakis

Give more weight to cpu_ratio

Change how scores are computed in default_allocator so that vcpu
ratio has more importance the free memory and disk ratio

b16db49b 10/09/2012 04:27 pm Kostas Papadimitriou

Merge branch 'devel-0.11' into ui-0.11

Conflicts:
snf-cyclades-app/synnefo/db/models.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/ui/static/snf/js/models.js

2ef10562 10/08/2012 02:30 pm Christos Stavrakakis

Change eventd and dispatcher AMQP retries

Set unlimited connection retries to AMQP hosts for snf-ganeti-eventd,
since death of snf-ganeti-eventd results in loss of events.
Also increase retries of snf-dispatcher.

4216cd83 10/04/2012 06:18 pm Christos Stavrakakis

Fix bug about RAPI client pooling

Modify the pooled_rapi_client context manager to handle exceptions
and release the client in any case. Also modify uses of reconciliation
to handle exceptions inside the scope of the context manager

6cc3a31c 10/04/2012 03:23 pm Christos Stavrakakis

Add missing ipv6 arguments to create_network

6de6677b 10/04/2012 01:36 pm Christos Stavrakakis

Fix bug when hidding image password

Use deepcopy instead of copy, since the dictionary
is nested

c4ce868e 10/04/2012 01:17 pm Christos Stavrakakis

Fix bug in snf-dispatcher about ignored messages

Update backendtime only for jobs that have been successfully completed,
since only those jobs update the state of the VM. Else a "race
condition" may occur when a successful job (e.g. OP_INSTANCE_REMOVE)
completes before an error job for the same VM and messages arrive in...

6afeb85d 10/03/2012 03:58 pm Christos Stavrakakis

Hide image password from log file

ffa7f6f6 10/03/2012 01:09 pm Christos Stavrakakis

Garbage collect GanetiRapiClient pools

Modify the rapi_pool to delete unused GanetiRapiClient pools. This
clients become unused when the credentials of a backend change.

bf5c82dc 10/03/2012 01:09 pm Christos Stavrakakis

Add extra logging

9cf4a3c0 10/03/2012 01:08 pm Christos Stavrakakis

Refresh backends in locked mode

3524241a 10/03/2012 12:22 pm Christos Stavrakakis

Pool Ganeti RAPI clients

- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.
- Create GanetiRapiClientPool for pooling GanetiRapiClient objects
- Create context manager in db.models for pool handling
- Update code using the RAPI client

fbb74fff 10/01/2012 11:15 am Christos Stavrakakis

Do not print public networks when adding backend

56359f67 10/01/2012 11:15 am Christos Stavrakakis

Fix white space

27d6d48d 10/01/2012 11:13 am Stratos Psomadakis

Add support for Archipelagos/extstorage COW images

f96f60fd 09/28/2012 03:32 pm Christos Stavrakakis

Move check for free IP to backend_allocator

Filter out the backends that do no have a free IP in BackendAllocator
instead of the default_allocator, in order to remove imports from
synnefo in the allocation strategy.

6883dcf3 09/28/2012 01:21 pm Christos Stavrakakis

Fix bugs in reconcile-networks

Perform logical and instead of or, when merging backend ip maps, since
now the maps are inverted. Also do not report as stale backend networks
that are already deleted.

bb80a8d7 09/28/2012 01:21 pm Christos Stavrakakis

Take care for messages without timestamp

Catch exception produced in snf-dispatcher when receiving messages without
proper timestamp.

adc46059 09/28/2012 12:19 pm Christos Stavrakakis

Perform allocations in separate transactions

Add ForUpdateManager to the Backend model, in order to guarantee
exclusive order when allocating a VM.

Use manual transactions when allocating Backend and public IP for a new
VM, since these allocations need exclusive access (SELECT..FOR UPDATE)...

5706f527 09/28/2012 12:19 pm Christos Stavrakakis

Fix typo

b123630d 09/28/2012 11:10 am Christos Stavrakakis

Merge branch 'public_networks' into devel-0.11

Conflicts:
snf-cyclades-app/synnefo/logic/dispatcher.py

5da13d77 09/27/2012 04:42 pm Vangelis Koukis

Close DB conn in dispatcher before every message

Have the dispatcher close the Django DB connection before processing
an incoming message. This allows the actual, underlying DBAPI connection
to return to the DB connection pool, if used, and enables the dispatcher...

3f018af1 09/27/2012 03:59 pm Christos Stavrakakis

Make snf-dispatcher work with gevent

Remove workers from snf-dispatcher in order to make it compatible with gevent
and specifically the green, monkey patched version of psycopg2
(synnefo.lib.db.psyco_gevent). Also after daemonizing, gevent must be
reinitialized because the deamon context forks.

7f9f3018 09/27/2012 03:59 pm Christos Stavrakakis

Add pool-create and pool-remove commands

6c9c95d8 09/27/2012 03:59 pm Christos Stavrakakis

Change process title to simplify 'ps' display

Use python-setproctitle to change the process title of the running
daemon. Make it appear as a native executable to simplify
administration. Only set executable name, because cannot set individual
values for arguments.

5e1a5f9b 09/27/2012 03:59 pm Christos Stavrakakis

Fix presentation of pools

ec22dfc4 09/27/2012 03:59 pm Christos Stavrakakis

Fix two bugs in network reconciliation

d2e73c0c 09/27/2012 03:59 pm Christos Stavrakakis

Fix race conditions when updating network objects

Use for_update() object manager in order to exclude race conditions
when updating network objects.

8ec69269 09/27/2012 03:59 pm Christos Stavrakakis

Fix double logging of dispatcher

Dispatcher had as default handlers the console and the file handler.
However, when daemonizing, we redirect the stdout and stderr to the file
handler, which resulted in the log messages to be printed twice. So we
remove the console handler from the dict configuration, and we only add...

24683dda 09/27/2012 03:59 pm Christos Stavrakakis

Fix typos

e64e7ade 09/27/2012 03:59 pm Christos Stavrakakis

Catch corner case in reconciliation

Catch error produced when the DB entry for an orphan VM in Ganeti does
not exist. This may happen because either an entry from DB was removed
or because a VM in Ganeti with the synnefo prefix was created.

1dc821c9 09/21/2012 04:28 pm Christos Stavrakakis

Change process title to simplify 'ps' display

Use python-setproctitle to change the process title of the running
daemon. Make it appear as a native executable to simplify
administration. Only set executable name, because cannot set individual
values for arguments.

47d1d754 09/21/2012 11:23 am Christos Stavrakakis

Add some logging to backend allocator

4936f2e2 09/21/2012 10:42 am Christos Stavrakakis

Make snf-dispatcher work with gevent

Remove workers from snf-dispatcher in order to make it compatible with gevent
and specifically the green, monkey patched version of psycopg2
(synnefo.lib.db.psyco_gevent). Also after daemonizing, gevent must be
reinitialized because the deamon context forks.

fdc94944 09/20/2012 12:14 pm Christos Stavrakakis

Refactor IP pools

Treat IP pool the same way as bridge and mac-prefix pools. The pool is
removed from the Network table and moved to an IPPoolTable containing the
IP pools for all networks. Also added support for external reservations.

Migration files for transfering the pool to the new tables are included....

9727ac2f 09/20/2012 12:14 pm Christos Stavrakakis

Handle case of no pools in reconcile-pools

0eed6cf8 09/19/2012 01:16 pm Christos Stavrakakis

Management commands for pools

Create three new management commands for listing, showing and modifying
a pool.

af6a3bc5 09/19/2012 01:16 pm Christos Stavrakakis

Unique mac-prefix only for mac-filtered networks

03992c72 09/19/2012 01:16 pm Christos Stavrakakis

Refactor bridge and mac-prefix pool tables

Redesign the implementation of pools and map mac-prefix and bridge pools
to that design.

Each pool type is mapped to a table in the database, and each row of
that table is a pool of that type. All pool tables must contain the...

4a37a0be 09/19/2012 11:34 am Christos Stavrakakis

Fix bug in server-inspect

341c818e 09/07/2012 05:41 pm Dimitris Aragiorgis

Enable hotplug during create_instance

This passes hotplug = True to ganeti and enables hotplug per instance.
It checks GANETI_USE_HOTPLUG setting.

Signed-off-by: Dimitris Aragiorgis <>

7cfbbf32 09/07/2012 05:36 pm Christos Stavrakakis

New method for creating BackendNetwork entries

Create a new method to the Network model, in order to create the
BackendNetwork entries tha correspond to this network. Also remove
the custom save method, since when multiple public network will be
supported, we want them to be correlated with specific backends.

370f69ec 09/07/2012 05:36 pm Christos Stavrakakis

Remove dead code

7fede91e 09/07/2012 05:36 pm Christos Stavrakakis

Implement multiple public networks

Implement multiple public networks in order to support:
- different public networks per backend
- non-consecutive IP ranges in each backend

To support this, public networks are treated different from private
ones. Namely, they are correlated with one backend (and so have...

30418cbe 09/07/2012 05:36 pm Christos Stavrakakis

Reserve gateway's IP when creating an IP pool

f8d1eaf4 08/09/2012 05:00 pm Kostas Papadimitriou

Revert commented line in logic backend

the line was uncommented by mistake in `7d43565`

6138f0ef 08/08/2012 08:28 pm Kostas Papadimitriou

Additional build progress callback handlers

Handle image-{error, info, warning, helper} messages and create the
corresponding virtual machine diagnostics.

7d43565f 08/08/2012 08:27 pm Kostas Papadimitriou

Logic backend api helper to create vm diagnostics

a3b1aee2 08/06/2012 06:17 pm Nikos Skalkotos

Make dispatcher aware of the new monitoring msgs

Make dispatcher aware of image-warning and image-info messages

0827883e 08/06/2012 06:17 pm Nikos Skalkotos

Comply with the new monitoring system of snf-image

For now only the image-copy-progress messages are processed

296682fe 08/06/2012 11:45 am Kostas Papadimitriou

Support for ext disk providers

Extract provider from flavor disk template name and set the additional
`provider` parameter in disk options. Flavor disk_template should be
formated as `ext_<provider_name>`.

16ddd1fe 07/27/2012 07:26 pm Christos Stavrakakis

Add setting indicating whether to use hotplug

3f52dd82 07/27/2012 02:57 pm Christos Stavrakakis

Management command for removing backend

14fd6983 07/27/2012 02:57 pm Christos Stavrakakis

Add command for checking pool consistency

Add new management command 'reconcile-pools' which checks the consistency of the
pools and the uniqueness of the associated resources. Currently this
commands detects the inconcistencies, without trying to resolve them.

c807398f 07/27/2012 02:57 pm Kostas Papadimitriou

Fix serious bug in reconciliation

Do not perform reconciliation for VMs that are hosted in Backends
that are currently offline, since it is impossible to get the
state of these VMs from the backend. Before this patch, the
reconciliation mechanism would report these VMs as stale (and with...

3165f027 07/27/2012 02:57 pm Christos Stavrakakis

Implement MAC Prefix Management

In order to guarantee uniqueness of MAC address accross all backends,
synnefo assignes a unique mac prefix to each network in each backend.

A mac-prefix of 20 bits, which is picked from the MacPrefixPool, is
assigned to each Network. The mac-prefix of the network in each...

a9ba398e 07/26/2012 06:29 pm Christos Stavrakakis

Fix bug when adding a new backend

Fix wrong reporting messages when adding a new backend.

aeb523a4 07/26/2012 06:29 pm Christos Stavrakakis

Get public network by public attribute

Do not find the public network by the primary key, as the initial_data.json
is obsolete. Instead, ise the public attribute of the Network Model.
Currently only one public Network is supported, and the used can not
create or delete the public Network. But this can easily change in the...

90b29b33 07/24/2012 02:27 pm Christos Stavrakakis

Fix creation of networks in new backend