Statistics
| Branch: | Tag: | Revision:

root / snf-cyclades-app / synnefo / logic @ 039e3e61

# Date Author Comment
039e3e61 01/07/2013 11:15 am Christos Stavrakakis

Refactor tests

b66e4833 12/20/2012 05:29 pm Christos Stavrakakis

Add more tests for db, logic and api

5480daec 12/20/2012 05:22 pm Christos Stavrakakis

Remove dummy function

c8908e51 12/19/2012 05:32 pm Christos Stavrakakis

Add more tests for db and logic

52194c77 12/19/2012 05:32 pm Christos Stavrakakis

Refactor some test code

e6f6627c 12/14/2012 06:22 pm Christos Stavrakakis

Misc minor bug fixes

47eeffa9 12/14/2012 06:22 pm Christos Stavrakakis

Fix current db, logic and api tests

f45a7ac4 12/12/2012 02:05 pm Christos Stavrakakis

Fix variable scope bug in process_net_status

21c9042c 12/12/2012 11:12 am Christos Stavrakakis

Do not allow creation of more than one pools

Do not allow creation of more than one Bridge or MacPrefix pool, since
we can not handle them yet.

c414bc87 12/10/2012 04:14 pm Christos Stavrakakis

Add backend-id option to reconcile-servers

Make reconcile-servers perform reconciliation only for the specified
backend.

c6ad2f2d 12/06/2012 02:02 pm Christos Stavrakakis

reconciliation:avoid double asking Ganeti

Modify reconcile-servers to not communicate 2 times with Ganeti
to get the operstate and the NICs of the VMs. Merge this two one
call.

f83ebc98 12/05/2012 02:33 pm Christos Stavrakakis

Modify backend-modify --drained and --offline args

Make backend-modify drained argument as --drained=True|False instead
of --drained and --no-drained. The same for offline.

88b8f176 12/05/2012 02:08 pm Christos Stavrakakis

Mark as drained newly added backends

cc3c59b3 12/05/2012 01:33 pm Christos Stavrakakis

Rename reconcile command to reconcile-servers

Rename `snf-manage reconcile` management command to `snf-manage
reconcile-servers`, to be consistent with `snf-manage
reconcile-networks`. Also, treat '--detect-all' option as default
when no other '--detect-*' is specified.

b7d38981 12/04/2012 04:51 pm Dimitris Aragiorgis

Introduce network flavors to replace network types

Add tags and mode to network db model. Replace existing type with
flavor. Introduce 4 different flavors. Each flavor has mode, link,
mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,
MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....

600a3d4d 12/03/2012 02:59 pm Christos Stavrakakis

Implement quotas-init and quotas-verify

Implement quotas-init and quotas-verify management commands.
- quotas-init sets quotas of Quotaholder to the state of quotas in
Cyclades DB
- quotas-verify detects incosistencies between quotas in Quotaholder and...

2e1e6844 11/30/2012 02:47 pm Christos Stavrakakis

Temporary fix for allocating archipelagos VMs

006c6249 11/30/2012 12:49 pm Christos Stavrakakis

Fix typos

2a599282 11/30/2012 12:20 pm Christos Stavrakakis

Merge handling of ext_<provider> disk templates

33050118 11/29/2012 03:55 pm Georgios D. Tsoukalas

Merge branch 'origin/devel-0.13' into quota

Conflicts:
snf-cyclades-app/synnefo/api/servers.py

cb4eee84 11/26/2012 12:40 pm Christos Stavrakakis

Issue commission when deleting VM/Network

Issue commission to the Quotaholder when being notified from the Ganeti
backend that a VM/Network has been deleted. Although the resource has
already been deleting, we are interacting with the quotaholder in a
two step phase: issue commission and accept commission. It would be...

11a54cb9 11/26/2012 12:40 pm Christos Stavrakakis

Implement reconciliation for pending quotas

10de1102 11/26/2012 12:40 pm Christos Stavrakakis

Remove transaction from update_network_state

Remove transaction decorator from update_network_state function, since
this function is only called from process_network_status, which already runs
inside a transaction.

4d5d0b9c 11/26/2012 12:31 pm Christos Stavrakakis

Do not automatically update network state

Each modification in a BackendNetwork object, triggers an update
to the corresponding Network object. Until now this was done by
the BackendNetwork save() method. Now it is explicitly done
from the backend pross_network_status.

d7841399 11/23/2012 03:13 pm Christos Stavrakakis

Remove IGNORE_FLAVOR_DISK_SIZES setting

79b7dbb7 11/23/2012 01:50 pm Christos Stavrakakis

Use vmapi when creating servers

Use the vmapi when creating servers. Creete instance RAPI call takes the
configuration url instead all the image parametere. The id and format
of the image are still passed, just for readability and debugging.

Also renamed params_url to config-url.

b578d9e7 11/22/2012 01:26 pm Christos Stavrakakis

snf-dispatcher: Update NICs only when needed

Do not remove and recreate the NICs of a VM when NICs have not changed.

3e7c63f8 11/22/2012 01:26 pm Christos Stavrakakis

Implement resizing of pools

Extend PoolManager to support extending and shrinking of Pools. Also
add extra option '--size' to pool-modify command.

7a0aa449 11/22/2012 01:26 pm Christos Stavrakakis

Add function for pretty printing tables

a8858945 11/22/2012 01:24 pm Christos Stavrakakis

Make amqp_puka get logger as argument

2333a2c4 11/16/2012 02:54 pm Christos Stavrakakis

Check clustername when modifying backend

f7759b2d 11/16/2012 12:55 pm Christos Stavrakakis

Add check for clustername when adding backend

d4be7102 11/16/2012 12:24 pm Christos Stavrakakis

Fix lectical in pool-modify

f0fe50c2 11/12/2012 05:14 pm Christos Stavrakakis

Fix typo

b84ed662 11/09/2012 05:32 pm Christos Stavrakakis

Refactor code in management commands

39a6388d 11/09/2012 01:08 pm Christos Stavrakakis

Improve management commands

6639a243 11/08/2012 06:19 pm Christos Stavrakakis

Merge branch 'master' into devel-0.12

Conflicts:
snf-cyclades-app/synnefo/api/management/commands/flavor-list.py
snf-cyclades-app/synnefo/api/management/commands/server-list.py
snf-cyclades-app/synnefo/api/networks.py
snf-cyclades-app/synnefo/logic/rapi.py

fdfd8c6d 11/08/2012 01:21 pm Christos Stavrakakis

Fix dispatcher logging

7f2dbcad 11/07/2012 12:52 pm Christos Stavrakakis

Use select_for_update for VirtualMachines

Use select_for_update method when accessing VM objects to prevent
race conditions. Also move start_action from logic to api.

fd2bdbb2 11/07/2012 11:04 am Christos Stavrakakis

Process OP_NETWORK_SET_PARAMS Ganeti jobs

Modify snf-ganeti-eventd and snf-dispatcher to handle jobs with
OP_NETWORK_SET_PARAMS opcode. Handling of these jobs is needed
to update the IP Pool of network, when reserving and releasing
IPs directly from the Ganeti backend.

d1b1e683 11/06/2012 01:30 pm Dimitris Aragiorgis

PUBLIC_ROUTED minor changes

Do not allow api to create public networks and type other
than PRIVATE_MAC_FILTERED/PRIVATE_PHYSICAL_VLAN.

Rename PUBLIC_ROUTED_USE_POOL setting to PUBLIC_USE_POOL.

Let synnefo do IP pool management only if network is private or...

6de25206 11/06/2012 12:57 pm Dimitris Aragiorgis

Add conflicts_check option in network rapi calls

Ganeti by default checks for conflicting IPs when creating/connecting
networks. This feature can be bypassed for private networks. We do
not care for IP uniqueness in case of private networks as long as they...

a406663f 11/06/2012 12:57 pm Dimitris Aragiorgis

Update rapi client

Get rapi client from snf-ganeti. Add depends argument (for chain jobs)
where needed.

Argument tags must be a list of strings and arguments add_reserved_ips and
remove_reserved_ips must be None or a list of strings.

Signed-off-by: Dimitris Aragiorgis <>

2d762302 11/06/2012 12:57 pm Dimitris Aragiorgis

Minor backend fixes in networks

Add conflicts_check option when creating and add network. Check only
if a network is public.

Let tags argument in rapi calls be a list.

Signed-off-by: Dimitris Aragiorgis <>

f66d8b04 11/05/2012 09:45 am Christos Stavrakakis

Display VM build percentage in management commands

b74d3b64 11/05/2012 09:14 am Christos Stavrakakis

Allow custom link, mac_prefix in network-create

Add extra command line arguments, to allow the user specify the
link and mac_prefix of the new network if needed.

5dd9d123 11/02/2012 11:29 pm Christos Stavrakakis

Introduce per-user allocation of VMs to backends

Add BACKEND_PER_USER setting, a mapping from users to backends. This setting
can by used in order to bypass the default backend allocation process, and
allocate all the new VMs of a specific user to a specific Backend.

e221ade2 11/01/2012 01:24 pm Christos Stavrakakis

Do not allow any API action on suspended VMs

Do not allow any modification of any administratively suspended VM through the
API.

038d6772 10/30/2012 01:00 pm Christos Stavrakakis

Merge branch 'ui-0.11' into devel-0.12

71ec054d 10/30/2012 12:49 pm Christos Stavrakakis

Allow custom link, mac_prefix in network-create

Add extra command line arguments, to allow the user specify the
link and mac_prefix of the new network if needed.

1e2cc762 10/29/2012 02:28 pm Kostas Papadimitriou

Merge remote-tracking branch 'origin/devel-0.12' into ui-0.11

ed275428 10/24/2012 01:27 pm Christos Stavrakakis

Merge branch 'master' into devel-0.12

Conflicts:
snf-cyclades-app/setup.py

eeffcbea 10/23/2012 04:28 pm Kostas Papadimitriou

Merge branch 'devel-0.12' into ui-0.11

67619fd7 10/23/2012 04:19 pm Christos Stavrakakis

Implement queue-retry management command

Implement a new command which consumes all the messages from the
dead-letter queues and re-sends the messages back to the snf-dispatcher
in order to handle them again.

Messages are annotated with "from-dead-letter" field in order to detect...

40ef487d 10/23/2012 03:50 pm Christos Stavrakakis

Fix bug, UI not updating NICs

Dummy-save the Network when processing messages from Ganeti hooks
about a VM NICs. This is needed, because the UI uses the HTTP changes-sinces
when requesting VMs and Networks, and combines the result in order to show
the new NICs. Not updating the Network object, resulted in VMs IPv4...

659de616 10/23/2012 03:07 pm Christos Stavrakakis

Declare dead-letter exchanges and queues

Modify snf-dispatcher in order to declare the dead-letter exchanges and
queues

b9d91e62 10/23/2012 03:07 pm Christos Stavrakakis

Use basic_reject in snf-dispatcher callbacks

Modify snf-dispatcher callbacks in order to reject messages that caused
an unknown exception. Because a dead-letter-exchange is associated
with the queues that snf-dispatcher is using, these messages will be
transfered to the dead-letter-exchange.

37db18f3 10/19/2012 12:06 pm Christos Stavrakakis

Add mac2eui64 to snf-cyclades-app

Copy mac2eui64 from snf-network to snf-cyclades-app, since it is
now used in the reconciliation process to caclculate the IPv6
address of the VMs.

b1bb9251 10/18/2012 06:24 pm Christos Stavrakakis

Reconnect to different Broker if idle for too long

Modify snf-dispatcher and add a timeout option to basic_wait method. If
timeout is reached, the connection to the current RabbitMQ broker is
closed and dispatcher tries to connect to a different one.

This commit is supplementary of 6d27ead, which detects connection...

6d27eadd 10/18/2012 05:16 pm Christos Stavrakakis

Detect and handle connection problems in amqp_puka

Use the TCP keepalive feature in amqp_puka in order to detect connection
errors to the RabbitMQ broker, and retry to a different one. Also setup
the keepalive parameters to relative small values, to detect failures...

3308a83f 10/17/2012 11:58 am Christos Stavrakakis

Fix bug in reconcile-networks

When only detecting, reconciliation can not continue, if the
BackendNetwork object does not exist.

87920bc3 10/16/2012 08:50 pm Christos Stavrakakis

Do not give IPs to networks without dhcp

dab038a2 10/16/2012 01:38 pm Christos Stavrakakis

Fix typos

c7cbb6f3 10/16/2012 12:47 pm Christos Stavrakakis

Use - instead of _ in backend-update-status

122c4019 10/15/2012 12:56 pm Christos Stavrakakis

Fix bug in reconcile-networks

Fix bug when connecting a network to a nodegroup in network
reconciliation. Use connect_network instead of obsolete
connect_network_group method. Also fix job dependency handling in this
method.

e97288bc 10/15/2012 12:55 pm Christos Stavrakakis

Refactor process op and network status callbacks

d01cd522 10/10/2012 03:19 pm Christos Stavrakakis

Various fixes about logging

Fix logging from snf-manage commands. Modify snf-manage implementation
in order to setup the loggers for all snf-manage commands. Configuration
is based on SNF_MANAGE_LOGGING_SETUP setting from snf-common or on
basicConfig if that setting is missing....

e60b4630 10/10/2012 01:36 pm Christos Stavrakakis

Fix bug when synchronously creating networks

fa454545 10/10/2012 01:05 pm Christos Stavrakakis

Release address only if a NIC has an IP

0cd83e6c 10/10/2012 01:01 pm Christos Stavrakakis

Give more weight to cpu_ratio

Change how scores are computed in default_allocator so that vcpu
ratio has more importance the free memory and disk ratio

b16db49b 10/09/2012 04:27 pm Kostas Papadimitriou

Merge branch 'devel-0.11' into ui-0.11

Conflicts:
snf-cyclades-app/synnefo/db/models.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/ui/static/snf/js/models.js

2ef10562 10/08/2012 02:30 pm Christos Stavrakakis

Change eventd and dispatcher AMQP retries

Set unlimited connection retries to AMQP hosts for snf-ganeti-eventd,
since death of snf-ganeti-eventd results in loss of events.
Also increase retries of snf-dispatcher.

4216cd83 10/04/2012 06:18 pm Christos Stavrakakis

Fix bug about RAPI client pooling

Modify the pooled_rapi_client context manager to handle exceptions
and release the client in any case. Also modify uses of reconciliation
to handle exceptions inside the scope of the context manager

6cc3a31c 10/04/2012 03:23 pm Christos Stavrakakis

Add missing ipv6 arguments to create_network

6de6677b 10/04/2012 01:36 pm Christos Stavrakakis

Fix bug when hidding image password

Use deepcopy instead of copy, since the dictionary
is nested

c4ce868e 10/04/2012 01:17 pm Christos Stavrakakis

Fix bug in snf-dispatcher about ignored messages

Update backendtime only for jobs that have been successfully completed,
since only those jobs update the state of the VM. Else a "race
condition" may occur when a successful job (e.g. OP_INSTANCE_REMOVE)
completes before an error job for the same VM and messages arrive in...

6afeb85d 10/03/2012 03:58 pm Christos Stavrakakis

Hide image password from log file

ffa7f6f6 10/03/2012 01:09 pm Christos Stavrakakis

Garbage collect GanetiRapiClient pools

Modify the rapi_pool to delete unused GanetiRapiClient pools. This
clients become unused when the credentials of a backend change.

bf5c82dc 10/03/2012 01:09 pm Christos Stavrakakis

Add extra logging

9cf4a3c0 10/03/2012 01:08 pm Christos Stavrakakis

Refresh backends in locked mode

3524241a 10/03/2012 12:22 pm Christos Stavrakakis

Pool Ganeti RAPI clients

- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.
- Create GanetiRapiClientPool for pooling GanetiRapiClient objects
- Create context manager in db.models for pool handling
- Update code using the RAPI client

fbb74fff 10/01/2012 11:15 am Christos Stavrakakis

Do not print public networks when adding backend

56359f67 10/01/2012 11:15 am Christos Stavrakakis

Fix white space

27d6d48d 10/01/2012 11:13 am Stratos Psomadakis

Add support for Archipelagos/extstorage COW images

f96f60fd 09/28/2012 03:32 pm Christos Stavrakakis

Move check for free IP to backend_allocator

Filter out the backends that do no have a free IP in BackendAllocator
instead of the default_allocator, in order to remove imports from
synnefo in the allocation strategy.

6883dcf3 09/28/2012 01:21 pm Christos Stavrakakis

Fix bugs in reconcile-networks

Perform logical and instead of or, when merging backend ip maps, since
now the maps are inverted. Also do not report as stale backend networks
that are already deleted.

bb80a8d7 09/28/2012 01:21 pm Christos Stavrakakis

Take care for messages without timestamp

Catch exception produced in snf-dispatcher when receiving messages without
proper timestamp.

adc46059 09/28/2012 12:19 pm Christos Stavrakakis

Perform allocations in separate transactions

Add ForUpdateManager to the Backend model, in order to guarantee
exclusive order when allocating a VM.

Use manual transactions when allocating Backend and public IP for a new
VM, since these allocations need exclusive access (SELECT..FOR UPDATE)...

5706f527 09/28/2012 12:19 pm Christos Stavrakakis

Fix typo

b123630d 09/28/2012 11:10 am Christos Stavrakakis

Merge branch 'public_networks' into devel-0.11

Conflicts:
snf-cyclades-app/synnefo/logic/dispatcher.py

5da13d77 09/27/2012 04:42 pm Vangelis Koukis

Close DB conn in dispatcher before every message

Have the dispatcher close the Django DB connection before processing
an incoming message. This allows the actual, underlying DBAPI connection
to return to the DB connection pool, if used, and enables the dispatcher...

3f018af1 09/27/2012 03:59 pm Christos Stavrakakis

Make snf-dispatcher work with gevent

Remove workers from snf-dispatcher in order to make it compatible with gevent
and specifically the green, monkey patched version of psycopg2
(synnefo.lib.db.psyco_gevent). Also after daemonizing, gevent must be
reinitialized because the deamon context forks.

7f9f3018 09/27/2012 03:59 pm Christos Stavrakakis

Add pool-create and pool-remove commands

6c9c95d8 09/27/2012 03:59 pm Christos Stavrakakis

Change process title to simplify 'ps' display

Use python-setproctitle to change the process title of the running
daemon. Make it appear as a native executable to simplify
administration. Only set executable name, because cannot set individual
values for arguments.

5e1a5f9b 09/27/2012 03:59 pm Christos Stavrakakis

Fix presentation of pools

ec22dfc4 09/27/2012 03:59 pm Christos Stavrakakis

Fix two bugs in network reconciliation

d2e73c0c 09/27/2012 03:59 pm Christos Stavrakakis

Fix race conditions when updating network objects

Use for_update() object manager in order to exclude race conditions
when updating network objects.

8ec69269 09/27/2012 03:59 pm Christos Stavrakakis

Fix double logging of dispatcher

Dispatcher had as default handlers the console and the file handler.
However, when daemonizing, we redirect the stdout and stderr to the file
handler, which resulted in the log messages to be printed twice. So we
remove the console handler from the dict configuration, and we only add...

24683dda 09/27/2012 03:59 pm Christos Stavrakakis

Fix typos

e64e7ade 09/27/2012 03:59 pm Christos Stavrakakis

Catch corner case in reconciliation

Catch error produced when the DB entry for an orphan VM in Ganeti does
not exist. This may happen because either an entry from DB was removed
or because a VM in Ganeti with the synnefo prefix was created.