Synnefo

cyclades: Replace FloatingIP with IPAddress

This commit just replaces all references to FloatingIP objects with
references to IPAddress object, in order to be able to just load the
code. But the code is left unchanged, so all tests will fail.

cyclades: Clear FloatingIP when deleting NIC

Override 'delete' method of NetworkInterface model, to clear the
'machine' flag of the corresponding NIC, if any.

cyclades: Use NIC UUID in firewall tags

Use NICs UUID instead of index to generate firewall tags. Make the
'set_firewall_profile' action to take the NICs ID as argument. Also,
update the firewall settings.

cyclades: Use UUIDs to update VM NICs

Callback functions of snf-dispatcher updated the NICs of a VM by
deleting all NICs from DB, and creating the new NICs that were described
in the message. However, since NICs have UUIDs, this is no longer
required. This commit update callbacks to update the NICs based on their...

cyclades: Fix getting action from Ganeti opcode

Parse the fields of the Ganeti command, to distinquish between
'CONNECT' and 'DISCONNECT' in case of 'OP_INSTANCE_SET_PARAMS'.

cyclades: Use NIC uuid when deleting NIC

Use opportunistic locking

Use opportunistic locking, introduced in Ganeti 2.7, to allow parallel creation
of VMs in the Ganeti backend. When this option is set, Ganeti iallocator
will allocate the VM only to nodes that not locked (e.g. by other VM
creation). This may result in unbalanced clusters, but `hbal` can be...

Pass network type info in tags during create

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

New style format for modifying a NIC

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

cyclades: Make disk_templates work with Ganeti<2.8

'enabled_disk_templates' cluster attribute has been added in Ganeti 2.8.
If this attribute is missing, return the ipolicy disk templates.

cyclades: Fix wrong quotas when creating server

Catch exception when talking to Ganeti inside 'server_command',
otherwise the pending commission will be rejected. Also, accept the
commission in case of build, because the VM will be created in DB.

cyclades: refactor server creation command

Split the server creation command in more steps in order to be
decorated with the 'server_command' function and follow the logic of all
other server commands. The main change is that the commission job is not
accepted when the VM is stored in DB, but when the OP_INSTANCE_CREATE...

cyclades: Simplify code

Simplify code by removing unnecessary temp variable.

cyclades: Do not save stale objects after a commit

Call to issue_and_accept_commission saves the resource object (eg a vm)
and commits the transaction. This has the side-effect that the resource
object becomes stale: its lock is released and another thread may acquire...

cyclades: Create NIC before send job to Ganeti

Create NICs in building state before enqueuing corresponding jobs to
Ganeti. This has already been done when creating the instance, but not
when connecting it to other networks.

cyclades: Fix bug in handling of reboot type.

'type' attribute of reboot OS API call was directly mapped to the
'reboot_type' option of Ganeti reboot job. However these two are
semantically different. The 'SOFT' and 'HARD' reboot types of OS API are
mapped to 'shutdown_timeout' option of Ganeti jobs. However, setting...

cyclades: Fix check for firewall profile

cyclades: Add 'name' to all commissions.

Make all quotaholder commissions to have a name. Since, Cyclades does
not store the association of a commission with a resource, we include
the resource in the commission name.

cyclades-rapi: Add bulk option to GetJobs

cyclades: Support IPv6 only networks

Make Cyclades support IPv6 only networks. Such networks will have
the 'subnet' attribute set to None, in the same way that IPv4 only
networks have the 'subnet6' attribute set to None.

Because currently Ganeti does not support IPv6 only networks, we create the...

cyclades: Fix corner case in snf-dispatcher

Same as commit 0514662, but for servers.

cyclades: Fix corner case in snf-dispatcher

Delete network from DB when job fails but the action of network is
DESTROY. This case appears when someone tries to remove a network that
has been deleted from Ganeti backend, but not yet from DB.

cyclades: Wrap diagnostic creation in transaction

Fix bug in snf-dispatcher causing wrong updates of VMs from ganeti.
Creation if instance diagnostics must run inside a transaction, because
otherwise, the lock of the vm (select_for_update) is released too early....

cyclades: Various fixes regarding floating IPs

Fix some issues regarding floating ips introduced by rebasing branch.

cyclades: Do not release floating IPs

Update 'release_instance_nics' backend function which is called to
process NIC modifications in the Ganeti backend, to not return a
IPv4 address back to pool if this address is a floating IP.

cyclades: Major refactor to api and logic apps

Major refactor to cyclades api and logic apps that removes most code
from the api views and moves it to logic app, so that the same logic
code can be used by different apps (e.g. EC2 API). Code is moved into...

cyclades: Handle Ganeti msgs for resized VMs

Handle 'OP_INSTANCE_SET_PARAMS' notifications from Ganeti that have
'beparams' attribute, which corresponds to resized VMs. In such
case, the corresponding flavor is found an the VM in DB is updated to
have this flavor....

Revert "cyclades: Speed up reconciliation"

This reverts commit cf2241c45a22dac71e31dbf32f4700c475804c12.

cyclades: Handle errors when using multiprocessing

Check the process exitcode to determine if a process finished
successfully.

cyclades: Fix deadlock when creating servers

Fix deadlock between snf-dispatcher and backend allocator, by making
snf-dispatcher take an exclusive lock to the backend which hosts the vm,
before trying to get the exclusive locks to the IP pools in order to...

cyclades: Sypport backends with Xen hypervisor

• Extend 'Backend' model with the 'hypervisor' attribute, which
  currently can be 'kvm' or 'xen'. Use 'kvm' as default value to
  easy database migration. Also add the corresponding schema migration....

cyclades: simplify comparison of NICs

cyclades: Fixes relevant to lazy networks

cyclades: refactor code relative to commission

Refactor Cyclades code that handles commissioning with Quotaholder. This
commit removes the 'uses_commission' decorator. Instead it introduces
the 'issue_and_accept_commission' function that is responsible for...

Bypass commissioning for orphan networks

Cyclades public networks do not have an owner, and since can not use
the commissioning system. Also, make snf-dispatcher instead of failing
for networks without userid to just a log a warning. This does not apply for...

Fix pep8 warnings

Remove dummy function

Fix variable scope bug in process_net_status

Introduce network flavors to replace network types

Add tags and mode to network db model. Replace existing type with
flavor. Introduce 4 different flavors. Each flavor has mode, link,
mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,
MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....

Merge handling of ext_<provider> disk templates

Issue commission when deleting VM/Network

Issue commission to the Quotaholder when being notified from the Ganeti
backend that a VM/Network has been deleted. Although the resource has
already been deleting, we are interacting with the quotaholder in a
two step phase: issue commission and accept commission. It would be...

Do not automatically update network state

Each modification in a BackendNetwork object, triggers an update
to the corresponding Network object. Until now this was done by
the BackendNetwork save() method. Now it is explicitly done
from the backend pross_network_status.

Use vmapi when creating servers

Use the vmapi when creating servers. Creete instance RAPI call takes the
configuration url instead all the image parametere. The id and format
of the image are still passed, just for readability and debugging.

Also renamed params_url to config-url.

Use select_for_update for VirtualMachines

Use select_for_update method when accessing VM objects to prevent
race conditions. Also move start_action from logic to api.

Minor backend fixes in networks

Add conflicts_check option when creating and add network. Check only
if a network is public.

Let tags argument in rapi calls be a list.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

Merge remote-tracking branch 'origin/devel-0.12' into ui-0.11

Fix bug, UI not updating NICs

Dummy-save the Network when processing messages from Ganeti hooks
about a VM NICs. This is needed, because the UI uses the HTTP changes-sinces
when requesting VMs and Networks, and combines the result in order to show
the new NICs. Not updating the Network object, resulted in VMs IPv4...

Fix bug in reconcile-networks

Fix bug when connecting a network to a nodegroup in network
reconciliation. Use connect_network instead of obsolete
connect_network_group method. Also fix job dependency handling in this
method.

Fix bug when synchronously creating networks

Merge branch 'devel-0.11' into ui-0.11

Conflicts:
snf-cyclades-app/synnefo/db/models.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/ui/static/snf/js/models.js