Statistics
| Branch: | Tag: | Revision:

root / snf-cyclades-app / synnefo / logic / backend.py @ 435bb7fb

History | View | Annotate | Download (40.6 kB)

# Date Author Comment
bac6ed1e 11/09/2013 07:48 pm Christos Stavrakakis

cyclades: Do not print IP pool for IPv6 only nets

Do not print IP pool of Ganeti network in case the network is an IPv6
only network. The Ganeti network has a dummy IPv5 subnet, just because
it is needed. Only change this dummy subnet to /30 to reduce space in...

a2bd0802 11/07/2013 04:31 pm Christos Stavrakakis

cyclades: Pass 'nfdhcp' tag to IPv6 networks

Pass 'nfdhcp' tag even if network is IPv6 only.

d2036274 11/07/2013 04:28 pm Christos Stavrakakis

cyclades: Code refactoring to backend module

  • Create function 'ensure_network_is_active' to check if a network
    exists in backend and create it if not.
  • Create function 'create_job_dependencies' to transaform a list of job
    IDs to the 'depends' attribute as expected by Ganeti....
bfb3f9c2 11/06/2013 05:50 pm Christos Stavrakakis

cyclades: Create IPv6 Addresses from dispatcher

When ports are created from the API they do not have an IPv6 address.
The IPv6 address is only available after the MAC address of the
NIC is known which is only available after the message from the
Ganeti backend has been received. However, dispatcher did not created...

8d5795b4 11/06/2013 02:00 pm Christos Stavrakakis

Cyclades: Rename 'Can not' to 'Cannot' to all msgs

0292883e 11/04/2013 03:52 pm Christos Stavrakakis

cyclades: Refactor code

Move code relative with allocating ips and floating IPs to 'logic.ips'
module.

c82f57ad 11/04/2013 12:18 pm Christos Stavrakakis

cyclades: Series of fixes regarding networks & ips

  • Do not delete the network from the DB if it has associated floating
    IPs. This should never be the case because a check is made before
    deleting the network, however this check is added for safety.
  • Do not allow deleting a network if there are still ports configured on...
e8234183 11/04/2013 12:18 pm Christos Stavrakakis

cyclades: Delete backend networks with network

Delete backend network when network is deleted (marked as deleted=True)
in DB. Backend networks are useless as history and they may be recreated
if needed. Also fix some tests.

fae6e5f0 11/01/2013 04:17 pm Christos Stavrakakis

cyclades: Create ports without 'device_id'

Allow creation of ports that are not connected to any instance. Such
ports will be in 'DOWN' state. To allow this, 'machine' attribute of
NetworkInterface model has been made to allow null values. Finally, the
'logic.ports' module has been remove, and the corresponding code has...

ff863a80 11/01/2013 12:45 am Christos Stavrakakis

cyclades: Add accounting for IP Addresses

Create IPAddressLog DB table to hold information about allocation of IP
addresses to virtual servers. The information that is currently saved
contains the IP address, the ID of the virtual server, the ID of the
virtual network along with the datetime that the address has been...

96feddae 10/30/2013 12:56 pm Christos Stavrakakis

cyclades: Update state of NICs if VM creation fail

Set the state of NICs(ports) to ERROR if the OP_INSTANCE_CREATE job
fails.

3a6be177 10/30/2013 12:51 pm Christos Stavrakakis

cyclades: Rename Port state from BUILDING to BUILD

Neutron API defines the 'BUILD' status. This commit renames BUILDING
state of NIC to BUILD.
437e1eee 10/29/2013 02:42 pm Christos Stavrakakis

cyclades: Create network in backend if missing

Automatically create a network in a Ganeti backend if missing.
Currently, this was done only for private networks. This commit changes
this to be done for any kind of networks.

fe3b2809 10/18/2013 02:50 pm Christos Stavrakakis

cyclades: Fix bug when creating networks

3278725f 10/14/2013 05:56 pm Christos Stavrakakis

cyclades: Refactor backend code for new networks

  • Optimize DB queries by using prefetch related for NIC IPs.
  • Do not try to construct new NIC if it does not exist in Cyclades DB.
  • Update tests
a96e84cf 10/10/2013 08:08 pm Christos Stavrakakis

cyclades: Fix code for networks logic and API

8764d304 10/10/2013 04:35 pm Christos Stavrakakis

cyclades: Make logic tests pass

Refactor Cyclades code so that it follows the new models for Networks,
Subnets and IPAddresses. This refactor is not complete. Instead is the
necessary refactor for making tests for server API calls succeed.

92d2d1ce 10/10/2013 01:56 am Christos Stavrakakis

cyclades: Make API server tests pass

Refactor Cyclades code so that it follows the new models for Networks,
Subnets and IPAddresses. This refactor is not complete. Instead is the
necessary refactor for making tests for server API calls succeed.

326c3ec8 10/09/2013 07:36 pm Christos Stavrakakis

cyclades: Replace FloatingIP with IPAddress

This commit just replaces all references to FloatingIP objects with
references to IPAddress object, in order to be able to just load the
code. But the code is left unchanged, so all tests will fail.

bfd04b01 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Include 'index' when comparing NICs

cb7b1c23 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Clear FloatingIP when deleting NIC

Override 'delete' method of NetworkInterface model, to clear the
'machine' flag of the corresponding NIC, if any.

0d069390 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Ignore Ganeti NICs with unknown ID

Do not process Ganeti NICs that do not have a valid ID and so are
unknown to Cyclades, because processing these NICs would require to
create them in DB and then rename them in Ganeti with the corresponding
identifier....

d0545590 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Use NIC UUID in firewall tags

Use NICs UUID instead of index to generate firewall tags. Make the
'set_firewall_profile' action to take the NICs ID as argument. Also,
update the firewall settings.

121b8921 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Fix case where 'dfree' is None

If Ganeti has not enabled 'plain' or 'drbd' disk template, then it
reports 'dfree' as None. Fix this corner case by treating None as 0.

a1baa42b 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Use UUIDs to update VM NICs

Callback functions of snf-dispatcher updated the NICs of a VM by
deleting all NICs from DB, and creating the new NICs that were described
in the message. However, since NICs have UUIDs, this is no longer
required. This commit update callbacks to update the NICs based on their...

e6fbada1 10/09/2013 11:40 am Christos Stavrakakis

gtools: Make eventd send job fields

Make snf-ganeti-eventd send 'job_fields' for instance and network
opcodes, and modify the corresponding backend functions and
snf-dispatcher callbacks.

3c52a9df 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Fix getting action from Ganeti opcode

Parse the fields of the Ganeti command, to distinquish between
'CONNECT' and 'DISCONNECT' in case of 'OP_INSTANCE_SET_PARAMS'.

d7ff7f5a 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Make reconciliation timeouts constants

Remove hard-coded timeouts from code and make them constants. Also do
not perform reconciliation for recent building NICs.

cd7ed999 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Use NIC uuid when deleting NIC

7c714455 10/09/2013 11:40 am Christos Stavrakakis

cyclades: Use Ganeti NIC UUIDs

Since Ganeti 2.8, it is supported to refer to devices (NICs and Disks)
not only by their index but also by their name or UUID. This commit
updates Synnefo to refer to devices by their name. Synnefo will set the
name of the devices which will be a UUID. We do not use Ganeti's UUIDs...

e9d59f5e 10/09/2013 11:40 am Christos Stavrakakis

Use opportunistic locking

Use opportunistic locking, introduced in Ganeti 2.7, to allow parallel creation
of VMs in the Ganeti backend. When this option is set, Ganeti iallocator
will allocate the VM only to nodes that not locked (e.g. by other VM
creation). This may result in unbalanced clusters, but `hbal` can be...

bc37fa88 10/09/2013 11:40 am Dimitris Aragiorgis

Remove deprecated network type in ganeti

Signed-off-by: Dimitris Aragiorgis <>

700b85be 10/09/2013 11:40 am Dimitris Aragiorgis

Pass network type info in tags during create

Signed-off-by: Dimitris Aragiorgis <>

89e92ee5 10/09/2013 11:40 am Dimitris Aragiorgis

Remove hotplug from InstanceCreate

Signed-off-by: Dimitris Aragiorgis <>

1d74f135 10/09/2013 11:40 am Dimitris Aragiorgis

New style format for modifying a NIC

Signed-off-by: Dimitris Aragiorgis <>

fc56ae0f 10/07/2013 02:50 pm Christos Stavrakakis

cyclades: Do not release externally reserved IPs

Do not automatically release an externally reserved IP if it is released
from a Ganeti backend. The IP should be released with snf-manage
network-modify management commands.

a8ae6989 10/01/2013 08:27 pm Christos Stavrakakis

cyclades: Make disk_templates work with Ganeti<2.8

'enabled_disk_templates' cluster attribute has been added in Ganeti 2.8.
If this attribute is missing, return the ipolicy disk templates.

1da50fe3 10/01/2013 11:37 am Christos Stavrakakis

cyclades: Refactor backend-update-status cmd

The command has been changed to no longer take any arguments or options.
Instead, this command always updates the resources and the enabled disk
templates of all online Ganeti backends.

88fd91af 09/20/2013 03:46 pm Christos Stavrakakis

cyclades: Fix wrong quotas when creating server

Catch exception when talking to Ganeti inside 'server_command',
otherwise the pending commission will be rejected. Also, accept the
commission in case of build, because the VM will be created in DB.

5c8076b6 09/20/2013 01:45 pm Christos Stavrakakis

cyclades: Helper function for commissioning

Create 'quotas.handle_resource_commission' function for handling
commissions for VMs and Networks.

562bf712 09/19/2013 07:50 pm Christos Stavrakakis

cyclades: refactor server creation command

Split the server creation command in more steps in order to be
decorated with the 'server_command' function and follow the logic of all
other server commands. The main change is that the commission job is not
accepted when the VM is stored in DB, but when the OP_INSTANCE_CREATE...

ed2064f8 09/18/2013 05:17 pm Christos Stavrakakis

Merge branch 'hotfix-0.14.6' into develop

Conflicts:
Changelog
docs/admin-guide.rst
snf-cyclades-app/synnefo/api/management/commands/network-remove.py
snf-cyclades-app/synnefo/api/servers.py
snf-cyclades-app/synnefo/api/tests/servers.py
snf-cyclades-app/synnefo/logic/backend.py...

25891abc 09/13/2013 04:37 pm Christos Stavrakakis

cyclades: Simplify code

Simplify code by removing unnecessary temp variable.

db3037f1 09/12/2013 06:29 pm Giorgos Korfiatis

cyclades: Do not release resources if already deleted

Ensure that a commission to release vm resources is issued only once.
If a vm is marked `deleted', then resources must have already been
released.

32a0b855 09/12/2013 06:29 pm Giorgos Korfiatis

cyclades: Do not save stale objects after a commit

Call to issue_and_accept_commission saves the resource object (eg a vm)
and commits the transaction. This has the side-effect that the resource
object becomes stale: its lock is released and another thread may acquire...

90858bda 09/10/2013 03:29 pm Christos Stavrakakis

cyclades: Update NICs even if job failed

Make snf-ganeti-eventd send the NICs of the instance not only on
success, but also if the job has failed or has been canceled. Also,
update snf-dispatcher to handle those messages. This step is required to
clear BUILDING NICs that fail.

2a2b01e5 09/10/2013 03:29 pm Christos Stavrakakis

cyclades: Create NIC before send job to Ganeti

Create NICs in building state before enqueuing corresponding jobs to
Ganeti. This has already been done when creating the instance, but not
when connecting it to other networks.

198d91c3 09/06/2013 12:22 pm Christos Stavrakakis

cyclades: Fix bug in snf-dispatcher logic

This commit reverts commits #07602322 and #30514662 which introduced a
bug in dispatcher's logic. These commit fixed the issue that, in a case
the Cyclades DB is unsynced with Ganeti, a resource that does not exist...

bbae3e45 08/28/2013 01:55 pm Christos Stavrakakis

cyclades: Fix bug in handling of reboot type.

'type' attribute of reboot OS API call was directly mapped to the
'reboot_type' option of Ganeti reboot job. However these two are
semantically different. The 'SOFT' and 'HARD' reboot types of OS API are
mapped to 'shutdown_timeout' option of Ganeti jobs. However, setting...

6488097c 08/27/2013 11:25 am Christos Stavrakakis

cyclades: Do not pass 'hotplug' if disabled

Fix passing of hotplug argument to GanetiRAPIClient, in case the
GANETI_USE_HOTPLUG is unset. If the Ganeti version does not support
hotplug, passing hotplug=False, will make the jobs fail at Ganeti opcode
validation. This commit fixes this by passing the argument, only when...

231b0fb6 08/20/2013 08:40 am Christos Stavrakakis

cyclades: Fix check for firewall profile

27435d42 08/19/2013 11:25 am Christos Stavrakakis

cyclades: Remove firewall tag when removing NIC

Remove firewall tag from ganeti instance when disconnecting them from
networks. Until now, this was not needed because a vm could not be
disconnected from public networks. Since floating IPs, a vm may be
dynamically be connected and disconnected to public networks, and so a...

9122ffab 07/25/2013 04:42 pm Christos Stavrakakis

cyclades: Add 'name' to all commissions.

Make all quotaholder commissions to have a name. Since, Cyclades does
not store the association of a commission with a resource, we include
the resource in the commission name.

b2791a77 07/24/2013 09:15 pm Christos Stavrakakis

cyclades: Set firewall profile per NIC

Set firewall profile per NIC, because since Floating IPs an instance can
have more than one NICs in a public network. Extend the API call to take
the NIC index as argument, and modify the Ganeti instance tags to be
formated with the NIC index.

70a0afab 07/24/2013 09:15 pm Christos Stavrakakis

cyclades-rapi: Add bulk option to GetJobs

8d325d4b 07/17/2013 02:57 pm Christos Stavrakakis

cyclades: Simple code refactoring

5aeb4e93 07/17/2013 02:57 pm Christos Stavrakakis

cyclades: Support IPv6 only networks

Make Cyclades support IPv6 only networks. Such networks will have
the 'subnet' attribute set to None, in the same way that IPv4 only
networks have the 'subnet6' attribute set to None.

Because currently Ganeti does not support IPv6 only networks, we create the...

890f3d4c 07/12/2013 01:22 pm Christos Stavrakakis

Merge branch 'hotfix-0.14.2' into develop

Conflicts:
Changelog
snf-astakos-app/astakos/im/messages.py
snf-astakos-app/astakos/im/views/target/__init__.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/logic/management/commands/reconcile-servers.py...

07602322 07/08/2013 03:05 pm Christos Stavrakakis

cyclades: Fix corner case in snf-dispatcher

Same as commit 0514662, but for servers.

527c7a7b 07/05/2013 04:52 pm Christos Stavrakakis

cyclades: Fix type error

Create network may return a list of job IDs.

05146623 07/04/2013 11:24 am Christos Stavrakakis

cyclades: Fix corner case in snf-dispatcher

Delete network from DB when job fails but the action of network is
DESTROY. This case appears when someone tries to remove a network that
has been deleted from Ganeti backend, but not yet from DB.

736b9244 07/02/2013 04:00 pm Christos Stavrakakis

cyclades: Fix exception message

Fix exception message when trying to connect an instance to a non-active
public network.

952b2a48 07/02/2013 03:00 pm Christos Stavrakakis

cyclades: Wrap diagnostic creation in transaction

Fix bug in snf-dispatcher causing wrong updates of VMs from ganeti.
Creation if instance diagnostics must run inside a transaction, because
otherwise, the lock of the vm (select_for_update) is released too early....

0c50c760 07/02/2013 12:32 pm Christos Stavrakakis

cyclades: Check if a NIC is a floating IP.

Extend NetworkInterface model with a property showing if the IPv4
address of the NIC is a floating IP or not. Also, simplify some
relevant code.

f8675683 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Various fixes regarding floating IPs

Fix some issues regarding floating ips introduced by rebasing branch.

cb66110b 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Add network policy at server create

Add 'DEFAULT_INSTANCE_NETWORKS' setting that can be used by the
administrator to define the list of networks that each server that is
created must be connected to. Each created VM will have a NIC connected
to each of these networks. This settings can include a list of network...

72dea98f 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Do not release floating IPs

Update 'release_instance_nics' backend function which is called to
process NIC modifications in the Ganeti backend, to not return a
IPv4 address back to pool if this address is a floating IP.

2c022086 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Support many NICs when creating vms

Support arbitraty number of NICs when creating Ganeti instances.

41a7fae7 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Major refactor to api and logic apps

Major refactor to cyclades api and logic apps that removes most code
from the api views and moves it to logic app, so that the same logic
code can be used by different apps (e.g. EC2 API). Code is moved into...

2cd3f389 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Add backend function for resizing vm

Add 'resize_instance' backend function for resizing a vm. This command
issues an OP_INSTANCE_MODIFY job with the specified beparams, which
currently are vcpus, minmem and maxmem.

Refs #3870

ca4d59e3 07/02/2013 12:31 pm Christos Stavrakakis

cyclades: Handle Ganeti msgs for resized VMs

Handle 'OP_INSTANCE_SET_PARAMS' notifications from Ganeti that have
'beparams' attribute, which corresponds to resized VMs. In such
case, the corresponding flavor is found an the VM in DB is updated to
have this flavor....

e77a29ab 06/28/2013 08:45 pm Christos Stavrakakis

cyclades: Fixes relative to reconciliation

Get available backends once and work only with them, since available
backends may change during reconciliation. Also temporary remove support
for communicating with each backend in a separate process.

b12ab2d0 06/25/2013 03:23 pm Christos Stavrakakis

Revert "cyclades: Speed up reconciliation"

This reverts commit cf2241c45a22dac71e31dbf32f4700c475804c12.

cf2241c4 06/25/2013 11:36 am Christos Stavrakakis

cyclades: Speed up reconciliation

Speed up reconciliation by using mulitprocessing to get in parallel
the instances from all Ganeti backends.

79f65825 06/21/2013 01:40 pm Christos Stavrakakis

cyclades: Handle errors when using multiprocessing

Check the process exitcode to determine if a process finished
successfully.

d986cb32 06/21/2013 12:54 pm Christos Stavrakakis

cyclades: Speed up reconciliation

Speed up reconciliation by using mulitprocessing to get in parallel
the instances from all Ganeti backends.

40d53b77 06/19/2013 11:07 am Christos Stavrakakis

cyclades: Fix deadlock when creating servers

Fix deadlock between snf-dispatcher and backend allocator, by making
snf-dispatcher take an exclusive lock to the backend which hosts the vm,
before trying to get the exclusive locks to the IP pools in order to...

b2222a7f 06/13/2013 05:10 pm Christos Stavrakakis

cyclades: Pass disk origin for all 'ext' flavors

bd87213f 05/21/2013 02:07 pm Christos Stavrakakis

cyclades: Sypport backends with Xen hypervisor

Support Ganeti backends with Xen hypervisor:
  • Extend 'Backend' model with the 'hypervisor' attribute, which
    currently can be 'kvm' or 'xen'. Use 'kvm' as default value to
    easy database migration. Also add the corresponding schema migration....
fd95834e 05/14/2013 01:25 pm Christos Stavrakakis

dispatcher: Handle NICs in callbacks.

Update 'update_db' callback to handle instance NICs together with
instance status.

Also update the relevant tests.

be4d8aed 05/14/2013 01:25 pm Christos Stavrakakis

cyclades: simplify comparison of NICs

acfc71ef 05/14/2013 01:25 pm Christos Stavrakakis

cyclades: Fix deadlock when connecting vms

Fix deadlock caused by 'get_or_create' method, since it acquires an
implicit shared lock. Take an exclusive lock on network to fix this
deadlock.

27cda06b 05/14/2013 01:24 pm Christos Stavrakakis

cyclades: Fixes relevant to lazy networks

99af08a4 05/13/2013 12:38 pm Christos Stavrakakis

cyclades: Implement lazy networks

Implement "lazy networks", which are not created to all Ganeti backends
upon creation, but only when an instance that resides in a Ganeti is
connected to that network.

Update 'create_network' to create the Network object in DB without...

2509ce17 04/30/2013 05:05 pm Christos Stavrakakis

cyclades: refactor code relative to commission

Refactor Cyclades code that handles commissioning with Quotaholder. This
commit removes the 'uses_commission' decorator. Instead it introduces
the 'issue_and_accept_commission' function that is responsible for...

939d71dd 03/21/2013 05:05 pm Christos Stavrakakis

Create public NIC when creating VM

Until now, a server's public NetworkInterface was not created in the DB
until the arrival of the message from Ganeti hooks. However, the IPv4
address of the NIC is reserved before sending the OP_INSTANCE_CREATE
job to Ganeti....

e18c1749 03/08/2013 04:22 pm Christos Stavrakakis

Bypass commissioning for orphan networks

Cyclades public networks do not have an owner, and since can not use
the commissioning system. Also, make snf-dispatcher instead of failing
for networks without userid to just a log a warning. This does not apply for...

8528b8ac 02/14/2013 11:03 am Christos Stavrakakis

Totally remove password from create_instance

Never pass password as an 'osparam' to Ganeti. Always use the vmapi,
even for VMs created with server-create management command.

cc92b70f 02/06/2013 02:18 pm Christos Stavrakakis

Fix pep8 warnings

3f77e63d 01/07/2013 12:22 pm Christos Stavrakakis

Merge branch 'master' into devel-0.13

Conflicts:
snf-cyclades-app/synnefo/db/models.py
snf-cyclades-app/synnefo/logic/management/commands/backend-modify.py

5480daec 12/20/2012 05:22 pm Christos Stavrakakis

Remove dummy function

e6f6627c 12/14/2012 06:22 pm Christos Stavrakakis

Misc minor bug fixes

f45a7ac4 12/12/2012 02:05 pm Christos Stavrakakis

Fix variable scope bug in process_net_status

c414bc87 12/10/2012 04:14 pm Christos Stavrakakis

Add backend-id option to reconcile-servers

Make reconcile-servers perform reconciliation only for the specified
backend.

b7d38981 12/04/2012 04:51 pm Dimitris Aragiorgis

Introduce network flavors to replace network types

Add tags and mode to network db model. Replace existing type with
flavor. Introduce 4 different flavors. Each flavor has mode, link,
mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,
MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....

006c6249 11/30/2012 12:49 pm Christos Stavrakakis

Fix typos

2a599282 11/30/2012 12:20 pm Christos Stavrakakis

Merge handling of ext_<provider> disk templates

33050118 11/29/2012 03:55 pm Georgios D. Tsoukalas

Merge branch 'origin/devel-0.13' into quota

Conflicts:
snf-cyclades-app/synnefo/api/servers.py

cb4eee84 11/26/2012 12:40 pm Christos Stavrakakis

Issue commission when deleting VM/Network

Issue commission to the Quotaholder when being notified from the Ganeti
backend that a VM/Network has been deleted. Although the resource has
already been deleting, we are interacting with the quotaholder in a
two step phase: issue commission and accept commission. It would be...

10de1102 11/26/2012 12:40 pm Christos Stavrakakis

Remove transaction from update_network_state

Remove transaction decorator from update_network_state function, since
this function is only called from process_network_status, which already runs
inside a transaction.