Statistics
| Branch: | Tag: | Revision:

root / snf-cyclades-app / synnefo / logic @ dab038a2

# Date Author Comment
dab038a2 10/16/2012 01:38 pm Christos Stavrakakis

Fix typos

c7cbb6f3 10/16/2012 12:47 pm Christos Stavrakakis

Use - instead of _ in backend-update-status

122c4019 10/15/2012 12:56 pm Christos Stavrakakis

Fix bug in reconcile-networks

Fix bug when connecting a network to a nodegroup in network
reconciliation. Use connect_network instead of obsolete
connect_network_group method. Also fix job dependency handling in this
method.

e97288bc 10/15/2012 12:55 pm Christos Stavrakakis

Refactor process op and network status callbacks

d01cd522 10/10/2012 03:19 pm Christos Stavrakakis

Various fixes about logging

Fix logging from snf-manage commands. Modify snf-manage implementation
in order to setup the loggers for all snf-manage commands. Configuration
is based on SNF_MANAGE_LOGGING_SETUP setting from snf-common or on
basicConfig if that setting is missing....

e60b4630 10/10/2012 01:36 pm Christos Stavrakakis

Fix bug when synchronously creating networks

fa454545 10/10/2012 01:05 pm Christos Stavrakakis

Release address only if a NIC has an IP

0cd83e6c 10/10/2012 01:01 pm Christos Stavrakakis

Give more weight to cpu_ratio

Change how scores are computed in default_allocator so that vcpu
ratio has more importance the free memory and disk ratio

2ef10562 10/08/2012 02:30 pm Christos Stavrakakis

Change eventd and dispatcher AMQP retries

Set unlimited connection retries to AMQP hosts for snf-ganeti-eventd,
since death of snf-ganeti-eventd results in loss of events.
Also increase retries of snf-dispatcher.

4216cd83 10/04/2012 06:18 pm Christos Stavrakakis

Fix bug about RAPI client pooling

Modify the pooled_rapi_client context manager to handle exceptions
and release the client in any case. Also modify uses of reconciliation
to handle exceptions inside the scope of the context manager

6cc3a31c 10/04/2012 03:23 pm Christos Stavrakakis

Add missing ipv6 arguments to create_network

6de6677b 10/04/2012 01:36 pm Christos Stavrakakis

Fix bug when hidding image password

Use deepcopy instead of copy, since the dictionary
is nested

c4ce868e 10/04/2012 01:17 pm Christos Stavrakakis

Fix bug in snf-dispatcher about ignored messages

Update backendtime only for jobs that have been successfully completed,
since only those jobs update the state of the VM. Else a "race
condition" may occur when a successful job (e.g. OP_INSTANCE_REMOVE)
completes before an error job for the same VM and messages arrive in...

6afeb85d 10/03/2012 03:58 pm Christos Stavrakakis

Hide image password from log file

ffa7f6f6 10/03/2012 01:09 pm Christos Stavrakakis

Garbage collect GanetiRapiClient pools

Modify the rapi_pool to delete unused GanetiRapiClient pools. This
clients become unused when the credentials of a backend change.

bf5c82dc 10/03/2012 01:09 pm Christos Stavrakakis

Add extra logging

9cf4a3c0 10/03/2012 01:08 pm Christos Stavrakakis

Refresh backends in locked mode

3524241a 10/03/2012 12:22 pm Christos Stavrakakis

Pool Ganeti RAPI clients

- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.
- Create GanetiRapiClientPool for pooling GanetiRapiClient objects
- Create context manager in db.models for pool handling
- Update code using the RAPI client

fbb74fff 10/01/2012 11:15 am Christos Stavrakakis

Do not print public networks when adding backend

56359f67 10/01/2012 11:15 am Christos Stavrakakis

Fix white space

27d6d48d 10/01/2012 11:13 am Stratos Psomadakis

Add support for Archipelagos/extstorage COW images

f96f60fd 09/28/2012 03:32 pm Christos Stavrakakis

Move check for free IP to backend_allocator

Filter out the backends that do no have a free IP in BackendAllocator
instead of the default_allocator, in order to remove imports from
synnefo in the allocation strategy.

6883dcf3 09/28/2012 01:21 pm Christos Stavrakakis

Fix bugs in reconcile-networks

Perform logical and instead of or, when merging backend ip maps, since
now the maps are inverted. Also do not report as stale backend networks
that are already deleted.

bb80a8d7 09/28/2012 01:21 pm Christos Stavrakakis

Take care for messages without timestamp

Catch exception produced in snf-dispatcher when receiving messages without
proper timestamp.

adc46059 09/28/2012 12:19 pm Christos Stavrakakis

Perform allocations in separate transactions

Add ForUpdateManager to the Backend model, in order to guarantee
exclusive order when allocating a VM.

Use manual transactions when allocating Backend and public IP for a new
VM, since these allocations need exclusive access (SELECT..FOR UPDATE)...

5706f527 09/28/2012 12:19 pm Christos Stavrakakis

Fix typo

b123630d 09/28/2012 11:10 am Christos Stavrakakis

Merge branch 'public_networks' into devel-0.11

Conflicts:
snf-cyclades-app/synnefo/logic/dispatcher.py

5da13d77 09/27/2012 04:42 pm Vangelis Koukis

Close DB conn in dispatcher before every message

Have the dispatcher close the Django DB connection before processing
an incoming message. This allows the actual, underlying DBAPI connection
to return to the DB connection pool, if used, and enables the dispatcher...

3f018af1 09/27/2012 03:59 pm Christos Stavrakakis

Make snf-dispatcher work with gevent

Remove workers from snf-dispatcher in order to make it compatible with gevent
and specifically the green, monkey patched version of psycopg2
(synnefo.lib.db.psyco_gevent). Also after daemonizing, gevent must be
reinitialized because the deamon context forks.

7f9f3018 09/27/2012 03:59 pm Christos Stavrakakis

Add pool-create and pool-remove commands

6c9c95d8 09/27/2012 03:59 pm Christos Stavrakakis

Change process title to simplify 'ps' display

Use python-setproctitle to change the process title of the running
daemon. Make it appear as a native executable to simplify
administration. Only set executable name, because cannot set individual
values for arguments.

5e1a5f9b 09/27/2012 03:59 pm Christos Stavrakakis

Fix presentation of pools

ec22dfc4 09/27/2012 03:59 pm Christos Stavrakakis

Fix two bugs in network reconciliation

d2e73c0c 09/27/2012 03:59 pm Christos Stavrakakis

Fix race conditions when updating network objects

Use for_update() object manager in order to exclude race conditions
when updating network objects.

8ec69269 09/27/2012 03:59 pm Christos Stavrakakis

Fix double logging of dispatcher

Dispatcher had as default handlers the console and the file handler.
However, when daemonizing, we redirect the stdout and stderr to the file
handler, which resulted in the log messages to be printed twice. So we
remove the console handler from the dict configuration, and we only add...

24683dda 09/27/2012 03:59 pm Christos Stavrakakis

Fix typos

e64e7ade 09/27/2012 03:59 pm Christos Stavrakakis

Catch corner case in reconciliation

Catch error produced when the DB entry for an orphan VM in Ganeti does
not exist. This may happen because either an entry from DB was removed
or because a VM in Ganeti with the synnefo prefix was created.

1dc821c9 09/21/2012 04:28 pm Christos Stavrakakis

Change process title to simplify 'ps' display

Use python-setproctitle to change the process title of the running
daemon. Make it appear as a native executable to simplify
administration. Only set executable name, because cannot set individual
values for arguments.

47d1d754 09/21/2012 11:23 am Christos Stavrakakis

Add some logging to backend allocator

4936f2e2 09/21/2012 10:42 am Christos Stavrakakis

Make snf-dispatcher work with gevent

Remove workers from snf-dispatcher in order to make it compatible with gevent
and specifically the green, monkey patched version of psycopg2
(synnefo.lib.db.psyco_gevent). Also after daemonizing, gevent must be
reinitialized because the deamon context forks.

fdc94944 09/20/2012 12:14 pm Christos Stavrakakis

Refactor IP pools

Treat IP pool the same way as bridge and mac-prefix pools. The pool is
removed from the Network table and moved to an IPPoolTable containing the
IP pools for all networks. Also added support for external reservations.

Migration files for transfering the pool to the new tables are included....

9727ac2f 09/20/2012 12:14 pm Christos Stavrakakis

Handle case of no pools in reconcile-pools

0eed6cf8 09/19/2012 01:16 pm Christos Stavrakakis

Management commands for pools

Create three new management commands for listing, showing and modifying
a pool.

af6a3bc5 09/19/2012 01:16 pm Christos Stavrakakis

Unique mac-prefix only for mac-filtered networks

03992c72 09/19/2012 01:16 pm Christos Stavrakakis

Refactor bridge and mac-prefix pool tables

Redesign the implementation of pools and map mac-prefix and bridge pools
to that design.

Each pool type is mapped to a table in the database, and each row of
that table is a pool of that type. All pool tables must contain the...

4a37a0be 09/19/2012 11:34 am Christos Stavrakakis

Fix bug in server-inspect

341c818e 09/07/2012 05:41 pm Dimitris Aragiorgis

Enable hotplug during create_instance

This passes hotplug = True to ganeti and enables hotplug per instance.
It checks GANETI_USE_HOTPLUG setting.

Signed-off-by: Dimitris Aragiorgis <>

7cfbbf32 09/07/2012 05:36 pm Christos Stavrakakis

New method for creating BackendNetwork entries

Create a new method to the Network model, in order to create the
BackendNetwork entries tha correspond to this network. Also remove
the custom save method, since when multiple public network will be
supported, we want them to be correlated with specific backends.

370f69ec 09/07/2012 05:36 pm Christos Stavrakakis

Remove dead code

7fede91e 09/07/2012 05:36 pm Christos Stavrakakis

Implement multiple public networks

Implement multiple public networks in order to support:
- different public networks per backend
- non-consecutive IP ranges in each backend

To support this, public networks are treated different from private
ones. Namely, they are correlated with one backend (and so have...

30418cbe 09/07/2012 05:36 pm Christos Stavrakakis

Reserve gateway's IP when creating an IP pool

296682fe 08/06/2012 11:45 am Kostas Papadimitriou

Support for ext disk providers

Extract provider from flavor disk template name and set the additional
`provider` parameter in disk options. Flavor disk_template should be
formated as `ext_<provider_name>`.

16ddd1fe 07/27/2012 07:26 pm Christos Stavrakakis

Add setting indicating whether to use hotplug

3f52dd82 07/27/2012 02:57 pm Christos Stavrakakis

Management command for removing backend

14fd6983 07/27/2012 02:57 pm Christos Stavrakakis

Add command for checking pool consistency

Add new management command 'reconcile-pools' which checks the consistency of the
pools and the uniqueness of the associated resources. Currently this
commands detects the inconcistencies, without trying to resolve them.

c807398f 07/27/2012 02:57 pm Kostas Papadimitriou

Fix serious bug in reconciliation

Do not perform reconciliation for VMs that are hosted in Backends
that are currently offline, since it is impossible to get the
state of these VMs from the backend. Before this patch, the
reconciliation mechanism would report these VMs as stale (and with...

3165f027 07/27/2012 02:57 pm Christos Stavrakakis

Implement MAC Prefix Management

In order to guarantee uniqueness of MAC address accross all backends,
synnefo assignes a unique mac prefix to each network in each backend.

A mac-prefix of 20 bits, which is picked from the MacPrefixPool, is
assigned to each Network. The mac-prefix of the network in each...

a9ba398e 07/26/2012 06:29 pm Christos Stavrakakis

Fix bug when adding a new backend

Fix wrong reporting messages when adding a new backend.

aeb523a4 07/26/2012 06:29 pm Christos Stavrakakis

Get public network by public attribute

Do not find the public network by the primary key, as the initial_data.json
is obsolete. Instead, ise the public attribute of the Network Model.
Currently only one public Network is supported, and the used can not
create or delete the public Network. But this can easily change in the...

90b29b33 07/24/2012 02:27 pm Christos Stavrakakis

Fix creation of networks in new backend

2b36eda5 07/24/2012 02:26 pm Christos Stavrakakis

Fix bug in process_net_status

Check that nic has an ipv4 before trying to release it.

ad297723 07/23/2012 03:31 pm Christos Stavrakakis

Setting for using IP Pool to public network

Introduce new setting 'PUBLIC_ROUTED_USE_POOL', to indicate whether
synnefo will be responsible for the IP management of the public network,
or this responsibility will be delegated to Ganeti.

e8afa9ff 07/23/2012 01:40 pm Christos Stavrakakis

Consider cpu ratio when allocating vm's

d5b7fd7f 07/23/2012 12:33 pm Christos Stavrakakis

Fix case of empty address pool

967d3f83 07/23/2012 12:16 pm Christos Stavrakakis

Transaction on create_instance

c55b0cdc 07/23/2012 11:49 am Christos Stavrakakis

Reconciliation for network IP pool management

Reconcile the contents of each networks IP pool by comparing it
with combination of IP pools of all ganeti backends.

Add extra option for detecting conflicting IPs.

864bed43 07/23/2012 11:49 am Christos Stavrakakis

Implement model manager using row-level db locking

Implement ForUpdateManager that uses SELECT .. FOR UPDATE statement in
order to acquire row-level exclusive locks to DB. Essentially this
manager backports the select_for_update() method that is implemented in...

8a4a931d 07/23/2012 11:49 am Christos Stavrakakis

Fix IP pool exceptions

439ff929 07/23/2012 11:49 am Christos Stavrakakis

Display IP pool map in network-inspect command

bd392934 07/23/2012 11:49 am Christos Stavrakakis

Release IP addresses when nics are destroyed

cc3f266e 07/23/2012 11:49 am Christos Stavrakakis

Add support for IPv6

- Extend the Network model with fields for holding the IPv6 subnet and
gateway.
- Update network API methods.
- Modify ganeti hooks to return IPv6 address for all network interfaces.
IPv6 is now derived from the IPv6 subnet of the network in Ganeti,...

9c65f166 07/23/2012 11:49 am Christos Stavrakakis

Fix error state of BackendNetwork

77f0fa63 07/23/2012 11:49 am Christos Stavrakakis

Implement IP management

Delegate IP management to synnefo, in order to be able to allocate IP
addresses accross multiple Ganeti backends. This is achieved by creating a
Pool of IPs for each Network, in the form of a bitarray, which is stored
in the DB as a base64 string....

839e2bd0 07/23/2012 11:48 am Christos Stavrakakis

Update network settings

56a1134d 07/23/2012 11:48 am Christos Stavrakakis

Rename management commands

0196d9a3 07/23/2012 11:43 am Christos Stavrakakis

Add dirty bit to NetworkInterface

Extend NetworkInterface model with a dirty bit, indicating whether
the index of the NIC is consistent with the one in Ganeti. This flag is
set when deleting a NIC, and is switched off after processing the
notification from Ganeti. Removing a NIC with the dirty bit on is not...

91954b45 07/23/2012 11:42 am Christos Stavrakakis

Revert "Fix bug in dispatcher callback"

This reverts commit 5d40a128fb593ed641400c462f0e00a1b457f76e.

788a54be 07/23/2012 11:42 am Christos Stavrakakis

Remove machine if ganeti job fails

Set the VM deleted flag when receiving a failure notification for a
Ganeti OP_INSTANCE_REMOVE operation, even if the VM operstate is not
ERROR. The server may not exist in the backend not only because the
creation failed, but also because a remove notification was lost.

0e9a423f 07/23/2012 11:39 am Christos Stavrakakis

Implement network reconciliation

00051a6a 07/23/2012 11:39 am Christos Stavrakakis

New management commands to inspect network/server

d30f29aa 07/23/2012 11:39 am Christos Stavrakakis

Common functions for instance/network backend name

44e2c577 07/23/2012 11:39 am Christos Stavrakakis

Fix use of network name prefix

15cb13b5 07/23/2012 11:39 am Christos Stavrakakis

Fix bug about unknown network in instance's nics

e41977e2 07/23/2012 11:39 am Christos Stavrakakis

Fix management commands

- Fix bug in backend_add
- Display backend_name in network_inspect command

0b827aa4 07/23/2012 11:39 am Christos Stavrakakis

Bug fixes concerning networks

910f80db 07/23/2012 11:39 am Christos Stavrakakis

Fix bug in dispatcher callback

During handling of ganeti-op-status messages, state was set to error
only if the the opcode was OP_INSTANCE_CREATE. This resulted to
vm's never being deleted, if they did not exist in Ganeti.

36f4cb29 07/23/2012 11:39 am Christos Stavrakakis

Add backend method for removing a vm's nic

22ee6892 07/23/2012 11:39 am Christos Stavrakakis

Modification in logic for new-style networks

b3d28af2 07/23/2012 11:39 am Christos Stavrakakis

Synchronize networks when adding a new backend

b4eaff05 07/23/2012 11:39 am Christos Stavrakakis

Rename backend_create to backend_add

fd65ab41 07/23/2012 11:36 am Christos Stavrakakis

Implement BackendNetwork model

Create a new model, to represent the state of each network in each
backend.

a17a8e98 07/23/2012 11:36 am Christos Stavrakakis

Create network queues and refactor dispatcher

cc7c0f44 07/23/2012 11:33 am Christos Stavrakakis

Implement backend allocator and management command

New module implementing instance allocation to one of the available
backends. Allocation is performed based on memory and disk utilization.
Statistics for each backend are stored in DB and updated periodically....

4161cb41 07/23/2012 11:33 am Christos Stavrakakis

Fix issues concerning reconciliation

f5b4f2a3 07/23/2012 11:33 am Christos Stavrakakis

Use the Backend client instead of global RAPI

Replace use of global RAPI client, with the cached client from the
VirtualMachine or Backend model.

Dispatch network actions to all backends.

1a894bfe 07/23/2012 11:33 am Christos Stavrakakis

New management commands concerning backends

- Create command to create a backend (snf-manage backend_create)
- Create command to modify a backend (snf-manage backend_list)
- Create command to list backends (snf-manage backend_modify)
- Modify listservers command to display the backend that hosts the VM

924d8085 07/23/2012 11:32 am Christos Stavrakakis

Rename backend_id to backend_vm_id

Rename backend_id field of VirtualMachine model to avoid conflict between the
backend that hosts the VM and the id that the VM will have on the backend.

cf2a3529 07/16/2012 11:19 am Christos Stavrakakis

Remove stale pid lockfiles on daemon invocation

8c6de8ca 07/10/2012 12:05 pm Christos Stavrakakis

Rename management command

28f83276 07/09/2012 03:38 pm Kostas Papadimitriou

Merge branch 'simplequotas' into queues+minor_net