Synnefo

cyclades: Make snf-dispatcher handle volumes

Update snf-dispatcher to handle messages from snf-ganeti-eventd about
the instance disks(volumes). Handling of disks is done almost exactly as
with NICs.

cyclades: Ignore messages about deleted resources

Ignore messages from Ganeti about servers and networks that have already
been marked as deleted in the Cyclades DB. Processing of these messages
may asynchronize Cyclades with Astakos quotas.

cyclades: Do not retry all Ganeti failed creations

Make snf-dispatcher retry only the OP_INSTANCE_CREATE jobs that failed
due to opportunistic locking. Such jobs include the 'ECODE_TEMP_NORES'
error code in the opcode result. This commit makes snf-ganeti-eventd...

gtools: Fix Ganeti allocation parameters

When retrying a failed job, remove 'pnode' and 'snode' fields if they
were implicitly set by Ganeti iallocator. Ganeti will fail if both
iallocator and node are specified in the job.

cyclades: Retry failed instance creation

Handle special case where an OP_INSTANCE_CREATE job which uses the
opportunistic locking feature fails. This failure is probably due to the
fact that all nodes may be already locked. In this case, make
snf-dispatcher retry the failed job with opportunistic locking turned...

Cyclades: Rename 'Can not' to 'Cannot' to all msgs

cyclades: fix typo in snf-dispatcher

cyclades: Fix bug in update_cluster callback

Decorate 'update_cluster' callback with 'handle_message_delivery' to
handle acknowledgement of messages.

cyclades: Callback for cluster modifications

Extend snf-dispatcher callbacks with one to run every time an
OP_CLUSTER_ opcode is executed in a Ganeti backend.

• make snf-ganeti-eventd send messages on cluster modifications. These
  messages do not contain any description about the job....

dispatcher: Handle NICs in callbacks.

Update 'update_db' callback to handle instance NICs together with
instance status.

Also update the relevant tests.

cyclades: Implement lazy networks

Implement "lazy networks", which are not created to all Ganeti backends
upon creation, but only when an instance that resides in a Ganeti is
connected to that network.

Update 'create_network' to create the Network object in DB without...

Fix progress messages handling

proper handling of missing `messages` key progress messages.

Fix dispatcher logging

Process OP_NETWORK_SET_PARAMS Ganeti jobs

Modify snf-ganeti-eventd and snf-dispatcher to handle jobs with
OP_NETWORK_SET_PARAMS opcode. Handling of these jobs is needed
to update the IP Pool of network, when reserving and releasing
IPs directly from the Ganeti backend.

Use basic_reject in snf-dispatcher callbacks

Modify snf-dispatcher callbacks in order to reject messages that caused
an unknown exception. Because a dead-letter-exchange is associated
with the queues that snf-dispatcher is using, these messages will be
transfered to the dead-letter-exchange.

Take care for messages without timestamp

Catch exception produced in snf-dispatcher when receiving messages without
proper timestamp.

Additional build progress callback handlers

Handle image-{error, info, warning, helper} messages and create the
corresponding virtual machine diagnostics.

Comply with the new monitoring system of snf-image

For now only the image-copy-progress messages are processed

Modification in logic for new-style networks

Refactor common code in dispatcher's callbacks

Remove unused code

Remove old code used for reconciliation, as the reconciliation mechanism has
changed

Remove legacy code from dispatcher

Renamed snf-app to snf-cyclades-app