Synnefo

cyclades: Support IPv6 only networks

Make Cyclades support IPv6 only networks. Such networks will have
the 'subnet' attribute set to None, in the same way that IPv4 only
networks have the 'subnet6' attribute set to None.

Because currently Ganeti does not support IPv6 only networks, we create the...

cyclades: Fix corner case in snf-dispatcher

Same as commit 0514662, but for servers.

cyclades: Fix corner case in snf-dispatcher

Delete network from DB when job fails but the action of network is
DESTROY. This case appears when someone tries to remove a network that
has been deleted from Ganeti backend, but not yet from DB.

cyclades: Wrap diagnostic creation in transaction

Fix bug in snf-dispatcher causing wrong updates of VMs from ganeti.
Creation if instance diagnostics must run inside a transaction, because
otherwise, the lock of the vm (select_for_update) is released too early....

cyclades: Various fixes regarding floating IPs

Fix some issues regarding floating ips introduced by rebasing branch.

cyclades: Do not release floating IPs

Update 'release_instance_nics' backend function which is called to
process NIC modifications in the Ganeti backend, to not return a
IPv4 address back to pool if this address is a floating IP.

cyclades: Major refactor to api and logic apps

Major refactor to cyclades api and logic apps that removes most code
from the api views and moves it to logic app, so that the same logic
code can be used by different apps (e.g. EC2 API). Code is moved into...

cyclades: Handle Ganeti msgs for resized VMs

Handle 'OP_INSTANCE_SET_PARAMS' notifications from Ganeti that have
'beparams' attribute, which corresponds to resized VMs. In such
case, the corresponding flavor is found an the VM in DB is updated to
have this flavor....

Revert "cyclades: Speed up reconciliation"

This reverts commit cf2241c45a22dac71e31dbf32f4700c475804c12.

cyclades: Handle errors when using multiprocessing

Check the process exitcode to determine if a process finished
successfully.

cyclades: Fix deadlock when creating servers

Fix deadlock between snf-dispatcher and backend allocator, by making
snf-dispatcher take an exclusive lock to the backend which hosts the vm,
before trying to get the exclusive locks to the IP pools in order to...

cyclades: Sypport backends with Xen hypervisor

• Extend 'Backend' model with the 'hypervisor' attribute, which
  currently can be 'kvm' or 'xen'. Use 'kvm' as default value to
  easy database migration. Also add the corresponding schema migration....

cyclades: simplify comparison of NICs

cyclades: Fixes relevant to lazy networks

cyclades: refactor code relative to commission

Refactor Cyclades code that handles commissioning with Quotaholder. This
commit removes the 'uses_commission' decorator. Instead it introduces
the 'issue_and_accept_commission' function that is responsible for...

Bypass commissioning for orphan networks

Cyclades public networks do not have an owner, and since can not use
the commissioning system. Also, make snf-dispatcher instead of failing
for networks without userid to just a log a warning. This does not apply for...

Fix pep8 warnings

Remove dummy function

Fix variable scope bug in process_net_status

Introduce network flavors to replace network types

Add tags and mode to network db model. Replace existing type with
flavor. Introduce 4 different flavors. Each flavor has mode, link,
mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,
MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....

Merge handling of ext_<provider> disk templates

Issue commission when deleting VM/Network

Issue commission to the Quotaholder when being notified from the Ganeti
backend that a VM/Network has been deleted. Although the resource has
already been deleting, we are interacting with the quotaholder in a
two step phase: issue commission and accept commission. It would be...

Do not automatically update network state

Each modification in a BackendNetwork object, triggers an update
to the corresponding Network object. Until now this was done by
the BackendNetwork save() method. Now it is explicitly done
from the backend pross_network_status.

Use vmapi when creating servers

Use the vmapi when creating servers. Creete instance RAPI call takes the
configuration url instead all the image parametere. The id and format
of the image are still passed, just for readability and debugging.

Also renamed params_url to config-url.

Use select_for_update for VirtualMachines

Use select_for_update method when accessing VM objects to prevent
race conditions. Also move start_action from logic to api.

Minor backend fixes in networks

Add conflicts_check option when creating and add network. Check only
if a network is public.

Let tags argument in rapi calls be a list.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

Merge remote-tracking branch 'origin/devel-0.12' into ui-0.11

Fix bug, UI not updating NICs

Dummy-save the Network when processing messages from Ganeti hooks
about a VM NICs. This is needed, because the UI uses the HTTP changes-sinces
when requesting VMs and Networks, and combines the result in order to show
the new NICs. Not updating the Network object, resulted in VMs IPv4...

Fix bug in reconcile-networks

Fix bug when connecting a network to a nodegroup in network
reconciliation. Use connect_network instead of obsolete
connect_network_group method. Also fix job dependency handling in this
method.

Fix bug when synchronously creating networks

Merge branch 'devel-0.11' into ui-0.11

Conflicts:
snf-cyclades-app/synnefo/db/models.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/ui/static/snf/js/models.js

Fix bug in snf-dispatcher about ignored messages

Update backendtime only for jobs that have been successfully completed,
since only those jobs update the state of the VM. Else a "race
condition" may occur when a successful job (e.g. OP_INSTANCE_REMOVE)
completes before an error job for the same VM and messages arrive in...

Add extra logging

Add support for Archipelagos/extstorage COW images

Refactor IP pools

Treat IP pool the same way as bridge and mac-prefix pools. The pool is
removed from the Network table and moved to an IPPoolTable containing the
IP pools for all networks. Also added support for external reservations.

Migration files for transfering the pool to the new tables are included....

Enable hotplug during create_instance

This passes hotplug = True to ganeti and enables hotplug per instance.
It checks GANETI_USE_HOTPLUG setting.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

Revert commented line in logic backend

the line was uncommented by mistake in `7d43565`

Comply with the new monitoring system of snf-image

For now only the image-copy-progress messages are processed

Add setting indicating whether to use hotplug

Get public network by public attribute

Do not find the public network by the primary key, as the initial_data.json
is obsolete. Instead, ise the public attribute of the Network Model.
Currently only one public Network is supported, and the used can not
create or delete the public Network. But this can easily change in the...

Fix bug in process_net_status

Check that nic has an ipv4 before trying to release it.

Transaction on create_instance

Fix IP pool exceptions

Fix error state of BackendNetwork

Update network settings

Revert "Fix bug in dispatcher callback"

This reverts commit 5d40a128fb593ed641400c462f0e00a1b457f76e.

Fix bug in dispatcher callback

During handling of ganeti-op-status messages, state was set to error
only if the the opcode was OP_INSTANCE_CREATE. This resulted to
vm's never being deleted, if they did not exist in Ganeti.

Modification in logic for new-style networks

Implement BackendNetwork model

Create a new model, to represent the state of each network in each
backend.

New management commands concerning backends

- Create command to create a backend (snf-manage backend_create)
- Create command to modify a backend (snf-manage backend_list)
- Create command to list backends (snf-manage backend_modify)
- Modify listservers command to display the backend that hosts the VM