Synnefo

Remove transaction from 'backend-add' command

Do not run 'backend-add' command inside transaction, because if
'snf-dispatcher' is running, it will receive messages that will arrive
from a Ganeti backend that does not exist in the DB and will not be able
to process them.

Use 'python-objpool' instead of 'synnefo.lib.pool'

Use the python-objpool package for pooling resources as object, instead
of the pooling code that existed in snf-common ('synnefo.lib.pool')

Refs #3524

Create public NIC when creating VM

Until now, a server's public NetworkInterface was not created in the DB
until the arrival of the message from Ganeti hooks. However, the IPv4
address of the NIC is reserved before sending the OP_INSTANCE_CREATE
job to Ganeti....

Move common code for mngmt commands to webproject

Move cyclades functions that are useful for all managament commands, like
pprint_table or parse_filters, to snf-webproject.

Merge branch 'release-0.13' into develop

Conflicts:
version

Merge branch 'release-0.13' into develop

Conflicts:
snf-cyclades-app/synnefo/api/management/commands/cyclades-export-quota.py
snf-cyclades-app/synnefo/api/management/commands/network-list.py
snf-cyclades-app/synnefo/api/management/commands/server-inspect.py...

Improve perf for snf-manage cyclades commands

When the snf-manage {server, network}-list commands are invoked with the
--displayname option, batch the UUID-to-displayname requests to astakos,
instead of issuing a request for each (unique) UUID.

Refs #3375

reconciliation: handle missing network from db

Continue reconciliation even if an orphan network is missing from DB.

pep8 convension

Add --uuids option to snf-manage cmds

Fix reconciliation for building VMs

Since commit 482c645, a VM exists in the Cyclades DB before the
corresponding job is enqueued in the Ganeti backend. Therefore, there is
a corner case, where Cyclades can not enqueue the job to Ganeti and
Cyclades crashes before deleting the VM from the DB, which results in a...

Fix pep8 issues

Fix bug in network reconciliation

Fix bug in network reconciliation caused by wrong variable scope.

Rename ARCHIPELAGOS to ARCHIPELAGO

Rename ARCHIPELAGOS_BACKENDS setting to ARCHIPELAGO_BACKEND.

Network field type has been renamed to flavor

Add warning message for backend without networks

Remove unused imports from tests

Refactor tests

Add more tests for db, logic and api

Add more tests for db and logic

Misc minor bug fixes

Fix variable scope bug in process_net_status

Add backend-id option to reconcile-servers

Make reconcile-servers perform reconciliation only for the specified
backend.

Modify backend-modify --drained and --offline args

Make backend-modify drained argument as --drained=True|False instead
of --drained and --no-drained. The same for offline.

Rename reconcile command to reconcile-servers

Rename `snf-manage reconcile` management command to `snf-manage
reconcile-servers`, to be consistent with `snf-manage
reconcile-networks`. Also, treat '--detect-all' option as default
when no other '--detect-*' is specified.

Implement quotas-init and quotas-verify

Implement quotas-init and quotas-verify management commands.
- quotas-init sets quotas of Quotaholder to the state of quotas in
Cyclades DB
- quotas-verify detects incosistencies between quotas in Quotaholder and...

Fix typos

Merge branch 'origin/devel-0.13' into quota

Conflicts:
snf-cyclades-app/synnefo/api/servers.py

Implement reconciliation for pending quotas

Do not automatically update network state

Each modification in a BackendNetwork object, triggers an update
to the corresponding Network object. Until now this was done by
the BackendNetwork save() method. Now it is explicitly done
from the backend pross_network_status.

Use vmapi when creating servers

Use the vmapi when creating servers. Creete instance RAPI call takes the
configuration url instead all the image parametere. The id and format
of the image are still passed, just for readability and debugging.

Also renamed params_url to config-url.

Implement resizing of pools

Extend PoolManager to support extending and shrinking of Pools. Also
add extra option '--size' to pool-modify command.

Make amqp_puka get logger as argument

Add check for clustername when adding backend

Fix typo

Improve management commands

Fix dispatcher logging

Process OP_NETWORK_SET_PARAMS Ganeti jobs

Modify snf-ganeti-eventd and snf-dispatcher to handle jobs with
OP_NETWORK_SET_PARAMS opcode. Handling of these jobs is needed
to update the IP Pool of network, when reserving and releasing
IPs directly from the Ganeti backend.

Add conflicts_check option in network rapi calls

Ganeti by default checks for conflicting IPs when creating/connecting
networks. This feature can be bypassed for private networks. We do
not care for IP uniqueness in case of private networks as long as they...

Minor backend fixes in networks

Add conflicts_check option when creating and add network. Check only
if a network is public.

Let tags argument in rapi calls be a list.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

Allow custom link, mac_prefix in network-create

Add extra command line arguments, to allow the user specify the
link and mac_prefix of the new network if needed.

Do not allow any API action on suspended VMs

Do not allow any modification of any administratively suspended VM through the
API.

Allow custom link, mac_prefix in network-create

Add extra command line arguments, to allow the user specify the
link and mac_prefix of the new network if needed.

Merge branch 'master' into devel-0.12

Conflicts:
snf-cyclades-app/setup.py

Implement queue-retry management command

Implement a new command which consumes all the messages from the
dead-letter queues and re-sends the messages back to the snf-dispatcher
in order to handle them again.

Messages are annotated with "from-dead-letter" field in order to detect...

Declare dead-letter exchanges and queues

Modify snf-dispatcher in order to declare the dead-letter exchanges and
queues

Add mac2eui64 to snf-cyclades-app

Copy mac2eui64 from snf-network to snf-cyclades-app, since it is
now used in the reconciliation process to caclculate the IPv6
address of the VMs.

Detect and handle connection problems in amqp_puka

Use the TCP keepalive feature in amqp_puka in order to detect connection
errors to the RabbitMQ broker, and retry to a different one. Also setup
the keepalive parameters to relative small values, to detect failures...

Do not give IPs to networks without dhcp

Use - instead of _ in backend-update-status