Statistics
| Branch: | Tag: | Revision:

root / snf-cyclades-app / synnefo / logic / backend.py @ 9dcfad23

History | View | Annotate | Download (34.8 kB)

# Date Author Comment
5c8076b6 09/20/2013 01:45 pm Christos Stavrakakis

cyclades: Helper function for commissioning

Create 'quotas.handle_resource_commission' function for handling
commissions for VMs and Networks.

562bf712 09/19/2013 07:50 pm Christos Stavrakakis

cyclades: refactor server creation command

Split the server creation command in more steps in order to be
decorated with the 'server_command' function and follow the logic of all
other server commands. The main change is that the commission job is not
accepted when the VM is stored in DB, but when the OP_INSTANCE_CREATE...

ed2064f8 09/18/2013 05:17 pm Christos Stavrakakis

Merge branch 'hotfix-0.14.6' into develop

Conflicts:
Changelog
docs/admin-guide.rst
snf-cyclades-app/synnefo/api/management/commands/network-remove.py
snf-cyclades-app/synnefo/api/servers.py
snf-cyclades-app/synnefo/api/tests/servers.py
snf-cyclades-app/synnefo/logic/backend.py...

25891abc 09/13/2013 04:37 pm Christos Stavrakakis

cyclades: Simplify code

Simplify code by removing unnecessary temp variable.

db3037f1 09/12/2013 06:29 pm Giorgos Korfiatis

cyclades: Do not release resources if already deleted

Ensure that a commission to release vm resources is issued only once.
If a vm is marked `deleted', then resources must have already been
released.

32a0b855 09/12/2013 06:29 pm Giorgos Korfiatis

cyclades: Do not save stale objects after a commit

Call to issue_and_accept_commission saves the resource object (eg a vm)
and commits the transaction. This has the side-effect that the resource
object becomes stale: its lock is released and another thread may acquire...

90858bda 09/10/2013 03:29 pm Christos Stavrakakis

cyclades: Update NICs even if job failed

Make snf-ganeti-eventd send the NICs of the instance not only on
success, but also if the job has failed or has been canceled. Also,
update snf-dispatcher to handle those messages. This step is required to
clear BUILDING NICs that fail.

2a2b01e5 09/10/2013 03:29 pm Christos Stavrakakis

cyclades: Create NIC before send job to Ganeti

Create NICs in building state before enqueuing corresponding jobs to
Ganeti. This has already been done when creating the instance, but not
when connecting it to other networks.

198d91c3 09/06/2013 12:22 pm Christos Stavrakakis

cyclades: Fix bug in snf-dispatcher logic

This commit reverts commits #07602322 and #30514662 which introduced a
bug in dispatcher's logic. These commit fixed the issue that, in a case
the Cyclades DB is unsynced with Ganeti, a resource that does not exist...

bbae3e45 08/28/2013 01:55 pm Christos Stavrakakis

cyclades: Fix bug in handling of reboot type.

'type' attribute of reboot OS API call was directly mapped to the
'reboot_type' option of Ganeti reboot job. However these two are
semantically different. The 'SOFT' and 'HARD' reboot types of OS API are
mapped to 'shutdown_timeout' option of Ganeti jobs. However, setting...

6488097c 08/27/2013 11:25 am Christos Stavrakakis

cyclades: Do not pass 'hotplug' if disabled

Fix passing of hotplug argument to GanetiRAPIClient, in case the
GANETI_USE_HOTPLUG is unset. If the Ganeti version does not support
hotplug, passing hotplug=False, will make the jobs fail at Ganeti opcode
validation. This commit fixes this by passing the argument, only when...

231b0fb6 08/20/2013 08:40 am Christos Stavrakakis

cyclades: Fix check for firewall profile

27435d42 08/19/2013 11:25 am Christos Stavrakakis

cyclades: Remove firewall tag when removing NIC

Remove firewall tag from ganeti instance when disconnecting them from
networks. Until now, this was not needed because a vm could not be
disconnected from public networks. Since floating IPs, a vm may be
dynamically be connected and disconnected to public networks, and so a...

9122ffab 07/25/2013 04:42 pm Christos Stavrakakis

cyclades: Add 'name' to all commissions.

Make all quotaholder commissions to have a name. Since, Cyclades does
not store the association of a commission with a resource, we include
the resource in the commission name.

b2791a77 07/24/2013 09:15 pm Christos Stavrakakis

cyclades: Set firewall profile per NIC

Set firewall profile per NIC, because since Floating IPs an instance can
have more than one NICs in a public network. Extend the API call to take
the NIC index as argument, and modify the Ganeti instance tags to be
formated with the NIC index.

70a0afab 07/24/2013 09:15 pm Christos Stavrakakis

cyclades-rapi: Add bulk option to GetJobs

8d325d4b 07/17/2013 02:57 pm Christos Stavrakakis

cyclades: Simple code refactoring

5aeb4e93 07/17/2013 02:57 pm Christos Stavrakakis

cyclades: Support IPv6 only networks

Make Cyclades support IPv6 only networks. Such networks will have
the 'subnet' attribute set to None, in the same way that IPv4 only
networks have the 'subnet6' attribute set to None.

Because currently Ganeti does not support IPv6 only networks, we create the...

890f3d4c 07/12/2013 01:22 pm Christos Stavrakakis

Merge branch 'hotfix-0.14.2' into develop

Conflicts:
Changelog
snf-astakos-app/astakos/im/messages.py
snf-astakos-app/astakos/im/views/target/__init__.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/logic/management/commands/reconcile-servers.py...

07602322 07/08/2013 03:05 pm Christos Stavrakakis

cyclades: Fix corner case in snf-dispatcher

Same as commit 0514662, but for servers.

527c7a7b 07/05/2013 04:52 pm Christos Stavrakakis

cyclades: Fix type error

Create network may return a list of job IDs.

05146623 07/04/2013 11:24 am Christos Stavrakakis

cyclades: Fix corner case in snf-dispatcher

Delete network from DB when job fails but the action of network is
DESTROY. This case appears when someone tries to remove a network that
has been deleted from Ganeti backend, but not yet from DB.

736b9244 07/02/2013 04:00 pm Christos Stavrakakis

cyclades: Fix exception message

Fix exception message when trying to connect an instance to a non-active
public network.

952b2a48 07/02/2013 03:00 pm Christos Stavrakakis

cyclades: Wrap diagnostic creation in transaction

Fix bug in snf-dispatcher causing wrong updates of VMs from ganeti.
Creation if instance diagnostics must run inside a transaction, because
otherwise, the lock of the vm (select_for_update) is released too early....

0c50c760 07/02/2013 12:32 pm Christos Stavrakakis

cyclades: Check if a NIC is a floating IP.

Extend NetworkInterface model with a property showing if the IPv4
address of the NIC is a floating IP or not. Also, simplify some
relevant code.

f8675683 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Various fixes regarding floating IPs

Fix some issues regarding floating ips introduced by rebasing branch.

cb66110b 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Add network policy at server create

Add 'DEFAULT_INSTANCE_NETWORKS' setting that can be used by the
administrator to define the list of networks that each server that is
created must be connected to. Each created VM will have a NIC connected
to each of these networks. This settings can include a list of network...

72dea98f 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Do not release floating IPs

Update 'release_instance_nics' backend function which is called to
process NIC modifications in the Ganeti backend, to not return a
IPv4 address back to pool if this address is a floating IP.

2c022086 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Support many NICs when creating vms

Support arbitraty number of NICs when creating Ganeti instances.

41a7fae7 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Major refactor to api and logic apps

Major refactor to cyclades api and logic apps that removes most code
from the api views and moves it to logic app, so that the same logic
code can be used by different apps (e.g. EC2 API). Code is moved into...

2cd3f389 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Add backend function for resizing vm

Add 'resize_instance' backend function for resizing a vm. This command
issues an OP_INSTANCE_MODIFY job with the specified beparams, which
currently are vcpus, minmem and maxmem.

Refs #3870

ca4d59e3 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Handle Ganeti msgs for resized VMs

Handle 'OP_INSTANCE_SET_PARAMS' notifications from Ganeti that have
'beparams' attribute, which corresponds to resized VMs. In such
case, the corresponding flavor is found an the VM in DB is updated to
have this flavor....

e77a29ab 06/28/2013 08:45 pm Christos Stavrakakis

cyclades: Fixes relative to reconciliation

Get available backends once and work only with them, since available
backends may change during reconciliation. Also temporary remove support
for communicating with each backend in a separate process.

b12ab2d0 06/25/2013 03:23 pm Christos Stavrakakis

Revert "cyclades: Speed up reconciliation"

This reverts commit cf2241c45a22dac71e31dbf32f4700c475804c12.

cf2241c4 06/25/2013 11:36 am Christos Stavrakakis

cyclades: Speed up reconciliation

Speed up reconciliation by using mulitprocessing to get in parallel
the instances from all Ganeti backends.

79f65825 06/21/2013 01:40 pm Christos Stavrakakis

cyclades: Handle errors when using multiprocessing

Check the process exitcode to determine if a process finished
successfully.

d986cb32 06/21/2013 12:54 pm Christos Stavrakakis

cyclades: Speed up reconciliation

Speed up reconciliation by using mulitprocessing to get in parallel
the instances from all Ganeti backends.

40d53b77 06/19/2013 11:07 am Christos Stavrakakis

cyclades: Fix deadlock when creating servers

Fix deadlock between snf-dispatcher and backend allocator, by making
snf-dispatcher take an exclusive lock to the backend which hosts the vm,
before trying to get the exclusive locks to the IP pools in order to...

b2222a7f 06/13/2013 05:10 pm Christos Stavrakakis

cyclades: Pass disk origin for all 'ext' flavors

bd87213f 05/21/2013 02:07 pm Christos Stavrakakis

cyclades: Sypport backends with Xen hypervisor

Support Ganeti backends with Xen hypervisor:
  • Extend 'Backend' model with the 'hypervisor' attribute, which
    currently can be 'kvm' or 'xen'. Use 'kvm' as default value to
    easy database migration. Also add the corresponding schema migration....
fd95834e 05/14/2013 01:25 pm Christos Stavrakakis

dispatcher: Handle NICs in callbacks.

Update 'update_db' callback to handle instance NICs together with
instance status.

Also update the relevant tests.

be4d8aed 05/14/2013 01:25 pm Christos Stavrakakis

cyclades: simplify comparison of NICs

acfc71ef 05/14/2013 01:25 pm Christos Stavrakakis

cyclades: Fix deadlock when connecting vms

Fix deadlock caused by 'get_or_create' method, since it acquires an
implicit shared lock. Take an exclusive lock on network to fix this
deadlock.

27cda06b 05/14/2013 01:24 pm Christos Stavrakakis

cyclades: Fixes relevant to lazy networks

99af08a4 05/13/2013 12:38 pm Christos Stavrakakis

cyclades: Implement lazy networks

Implement "lazy networks", which are not created to all Ganeti backends
upon creation, but only when an instance that resides in a Ganeti is
connected to that network.

Update 'create_network' to create the Network object in DB without...

2509ce17 04/30/2013 05:05 pm Christos Stavrakakis

cyclades: refactor code relative to commission

Refactor Cyclades code that handles commissioning with Quotaholder. This
commit removes the 'uses_commission' decorator. Instead it introduces
the 'issue_and_accept_commission' function that is responsible for...

939d71dd 03/21/2013 05:05 pm Christos Stavrakakis

Create public NIC when creating VM

Until now, a server's public NetworkInterface was not created in the DB
until the arrival of the message from Ganeti hooks. However, the IPv4
address of the NIC is reserved before sending the OP_INSTANCE_CREATE
job to Ganeti....

e18c1749 03/08/2013 04:22 pm Christos Stavrakakis

Bypass commissioning for orphan networks

Cyclades public networks do not have an owner, and since can not use
the commissioning system. Also, make snf-dispatcher instead of failing
for networks without userid to just a log a warning. This does not apply for...

8528b8ac 02/14/2013 11:03 am Christos Stavrakakis

Totally remove password from create_instance

Never pass password as an 'osparam' to Ganeti. Always use the vmapi,
even for VMs created with server-create management command.

cc92b70f 02/06/2013 02:18 pm Christos Stavrakakis

Fix pep8 warnings

3f77e63d 01/07/2013 12:22 pm Christos Stavrakakis

Merge branch 'master' into devel-0.13

Conflicts:
snf-cyclades-app/synnefo/db/models.py
snf-cyclades-app/synnefo/logic/management/commands/backend-modify.py

5480daec 12/20/2012 05:22 pm Christos Stavrakakis

Remove dummy function

e6f6627c 12/14/2012 06:22 pm Christos Stavrakakis

Misc minor bug fixes

f45a7ac4 12/12/2012 02:05 pm Christos Stavrakakis

Fix variable scope bug in process_net_status

c414bc87 12/10/2012 04:14 pm Christos Stavrakakis

Add backend-id option to reconcile-servers

Make reconcile-servers perform reconciliation only for the specified
backend.

b7d38981 12/04/2012 04:51 pm Dimitris Aragiorgis

Introduce network flavors to replace network types

Add tags and mode to network db model. Replace existing type with
flavor. Introduce 4 different flavors. Each flavor has mode, link,
mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,
MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....

006c6249 11/30/2012 12:49 pm Christos Stavrakakis

Fix typos

2a599282 11/30/2012 12:20 pm Christos Stavrakakis

Merge handling of ext_<provider> disk templates

33050118 11/29/2012 03:55 pm Georgios D. Tsoukalas

Merge branch 'origin/devel-0.13' into quota

Conflicts:
snf-cyclades-app/synnefo/api/servers.py

cb4eee84 11/26/2012 12:40 pm Christos Stavrakakis

Issue commission when deleting VM/Network

Issue commission to the Quotaholder when being notified from the Ganeti
backend that a VM/Network has been deleted. Although the resource has
already been deleting, we are interacting with the quotaholder in a
two step phase: issue commission and accept commission. It would be...

10de1102 11/26/2012 12:40 pm Christos Stavrakakis

Remove transaction from update_network_state

Remove transaction decorator from update_network_state function, since
this function is only called from process_network_status, which already runs
inside a transaction.

4d5d0b9c 11/26/2012 12:31 pm Christos Stavrakakis

Do not automatically update network state

Each modification in a BackendNetwork object, triggers an update
to the corresponding Network object. Until now this was done by
the BackendNetwork save() method. Now it is explicitly done
from the backend pross_network_status.

d7841399 11/23/2012 03:13 pm Christos Stavrakakis

Remove IGNORE_FLAVOR_DISK_SIZES setting

79b7dbb7 11/23/2012 01:50 pm Christos Stavrakakis

Use vmapi when creating servers

Use the vmapi when creating servers. Creete instance RAPI call takes the
configuration url instead all the image parametere. The id and format
of the image are still passed, just for readability and debugging.

Also renamed params_url to config-url.

b578d9e7 11/22/2012 01:26 pm Christos Stavrakakis

snf-dispatcher: Update NICs only when needed

Do not remove and recreate the NICs of a VM when NICs have not changed.

7f2dbcad 11/07/2012 12:52 pm Christos Stavrakakis

Use select_for_update for VirtualMachines

Use select_for_update method when accessing VM objects to prevent
race conditions. Also move start_action from logic to api.

fd2bdbb2 11/07/2012 11:04 am Christos Stavrakakis

Process OP_NETWORK_SET_PARAMS Ganeti jobs

Modify snf-ganeti-eventd and snf-dispatcher to handle jobs with
OP_NETWORK_SET_PARAMS opcode. Handling of these jobs is needed
to update the IP Pool of network, when reserving and releasing
IPs directly from the Ganeti backend.

2d762302 11/06/2012 12:57 pm Dimitris Aragiorgis

Minor backend fixes in networks

Add conflicts_check option when creating and add network. Check only
if a network is public.

Let tags argument in rapi calls be a list.

Signed-off-by: Dimitris Aragiorgis <>

e221ade2 11/01/2012 01:24 pm Christos Stavrakakis

Do not allow any API action on suspended VMs

Do not allow any modification of any administratively suspended VM through the
API.

1e2cc762 10/29/2012 02:28 pm Kostas Papadimitriou

Merge remote-tracking branch 'origin/devel-0.12' into ui-0.11

eeffcbea 10/23/2012 04:28 pm Kostas Papadimitriou

Merge branch 'devel-0.12' into ui-0.11

40ef487d 10/23/2012 03:50 pm Christos Stavrakakis

Fix bug, UI not updating NICs

Dummy-save the Network when processing messages from Ganeti hooks
about a VM NICs. This is needed, because the UI uses the HTTP changes-sinces
when requesting VMs and Networks, and combines the result in order to show
the new NICs. Not updating the Network object, resulted in VMs IPv4...

87920bc3 10/16/2012 08:50 pm Christos Stavrakakis

Do not give IPs to networks without dhcp

122c4019 10/15/2012 12:56 pm Christos Stavrakakis

Fix bug in reconcile-networks

Fix bug when connecting a network to a nodegroup in network
reconciliation. Use connect_network instead of obsolete
connect_network_group method. Also fix job dependency handling in this
method.

e97288bc 10/15/2012 12:55 pm Christos Stavrakakis

Refactor process op and network status callbacks

e60b4630 10/10/2012 01:36 pm Christos Stavrakakis

Fix bug when synchronously creating networks

fa454545 10/10/2012 01:05 pm Christos Stavrakakis

Release address only if a NIC has an IP

b16db49b 10/09/2012 04:27 pm Kostas Papadimitriou

Merge branch 'devel-0.11' into ui-0.11

Conflicts:
snf-cyclades-app/synnefo/db/models.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/ui/static/snf/js/models.js

6cc3a31c 10/04/2012 03:23 pm Christos Stavrakakis

Add missing ipv6 arguments to create_network

c4ce868e 10/04/2012 01:17 pm Christos Stavrakakis

Fix bug in snf-dispatcher about ignored messages

Update backendtime only for jobs that have been successfully completed,
since only those jobs update the state of the VM. Else a "race
condition" may occur when a successful job (e.g. OP_INSTANCE_REMOVE)
completes before an error job for the same VM and messages arrive in...

6afeb85d 10/03/2012 03:58 pm Christos Stavrakakis

Hide image password from log file

bf5c82dc 10/03/2012 01:09 pm Christos Stavrakakis

Add extra logging

3524241a 10/03/2012 12:22 pm Christos Stavrakakis

Pool Ganeti RAPI clients

- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.
- Create GanetiRapiClientPool for pooling GanetiRapiClient objects
- Create context manager in db.models for pool handling
- Update code using the RAPI client

27d6d48d 10/01/2012 11:13 am Stratos Psomadakis

Add support for Archipelagos/extstorage COW images

adc46059 09/28/2012 12:19 pm Christos Stavrakakis

Perform allocations in separate transactions

Add ForUpdateManager to the Backend model, in order to guarantee
exclusive order when allocating a VM.

Use manual transactions when allocating Backend and public IP for a new
VM, since these allocations need exclusive access (SELECT..FOR UPDATE)...

fdc94944 09/20/2012 12:14 pm Christos Stavrakakis

Refactor IP pools

Treat IP pool the same way as bridge and mac-prefix pools. The pool is
removed from the Network table and moved to an IPPoolTable containing the
IP pools for all networks. Also added support for external reservations.

Migration files for transfering the pool to the new tables are included....

03992c72 09/19/2012 01:16 pm Christos Stavrakakis

Refactor bridge and mac-prefix pool tables

Redesign the implementation of pools and map mac-prefix and bridge pools
to that design.

Each pool type is mapped to a table in the database, and each row of
that table is a pool of that type. All pool tables must contain the...

341c818e 09/07/2012 05:41 pm Dimitris Aragiorgis

Enable hotplug during create_instance

This passes hotplug = True to ganeti and enables hotplug per instance.
It checks GANETI_USE_HOTPLUG setting.

Signed-off-by: Dimitris Aragiorgis <>

7fede91e 09/07/2012 05:36 pm Christos Stavrakakis

Implement multiple public networks

Implement multiple public networks in order to support:
- different public networks per backend
- non-consecutive IP ranges in each backend

To support this, public networks are treated different from private
ones. Namely, they are correlated with one backend (and so have...

f8d1eaf4 08/09/2012 05:00 pm Kostas Papadimitriou

Revert commented line in logic backend

the line was uncommented by mistake in `7d43565`

7d43565f 08/08/2012 08:27 pm Kostas Papadimitriou

Logic backend api helper to create vm diagnostics

0827883e 08/06/2012 06:17 pm Nikos Skalkotos

Comply with the new monitoring system of snf-image

For now only the image-copy-progress messages are processed

296682fe 08/06/2012 11:45 am Kostas Papadimitriou

Support for ext disk providers

Extract provider from flavor disk template name and set the additional
`provider` parameter in disk options. Flavor disk_template should be
formated as `ext_<provider_name>`.

16ddd1fe 07/27/2012 07:26 pm Christos Stavrakakis

Add setting indicating whether to use hotplug

3165f027 07/27/2012 02:57 pm Christos Stavrakakis

Implement MAC Prefix Management

In order to guarantee uniqueness of MAC address accross all backends,
synnefo assignes a unique mac prefix to each network in each backend.

A mac-prefix of 20 bits, which is picked from the MacPrefixPool, is
assigned to each Network. The mac-prefix of the network in each...

aeb523a4 07/26/2012 06:29 pm Christos Stavrakakis

Get public network by public attribute

Do not find the public network by the primary key, as the initial_data.json
is obsolete. Instead, ise the public attribute of the Network Model.
Currently only one public Network is supported, and the used can not
create or delete the public Network. But this can easily change in the...

90b29b33 07/24/2012 02:27 pm Christos Stavrakakis

Fix creation of networks in new backend

2b36eda5 07/24/2012 02:26 pm Christos Stavrakakis

Fix bug in process_net_status

Check that nic has an ipv4 before trying to release it.

ad297723 07/23/2012 03:31 pm Christos Stavrakakis

Setting for using IP Pool to public network

Introduce new setting 'PUBLIC_ROUTED_USE_POOL', to indicate whether
synnefo will be responsible for the IP management of the public network,
or this responsibility will be delegated to Ganeti.

967d3f83 07/23/2012 12:16 pm Christos Stavrakakis

Transaction on create_instance