Synnefo

cyclades: Fix broken tests

Fix tests that were broken by commit #7ef05bd.

cyclades: Fix corner case in reconciliation

Do not perform reconciliation for building servers that the
OP_INSTANCE_CREATE job is completed after quering Ganeti about the state
of servers.

cyclades: Fix passing arg shutdown_timeout in logic

cyclades: Fix a bug in subnet creation

Since commit ba6ad3460931223d262badebcecbb4124ac136d8 subnets were
always created with an empty name field due to a bug.

cyclades: Lock objects in reconciliation

Make reconciliation mechanism take an exclusive lock on objects before
issuing messages for reconciliation events. This will make
reconciliation mechanism more slow, but will ensure correct quotas when
reconciliation is run while the service is online.

cyclades: Add 'userid' and 'public' to subnets

Extend 'Subnet' model with 'userid' and 'public' attributes. This is
required in order to avoid an expensive join when listing subnets. This
commit also includes a database migration for existing subnets.

cyclades: Optimize dispatcher DB queries

cyclades: Update help msg on snf-manage commands

cyclades: Do not prefetch related in get()

cyclades: Add a function to check name legnth

• Add a function logic/utils.py to check the length of a string and
  raise an API fault with the corresponding message.
• Check the length of the name of servers/ports/networks/subnets

cyclades: Fix typo in stats-cyclades cmd

cyclades: Fix bug in network link validation

cyclades: Fix error msg when deleting floating IPs

Vocabulary fixes: Dettach -> Detach

cyclades: Setting for max NICs per Ganeti instance

Add new 'GANETI_MAX_NICS_PER_INSTANCE' setting which represents the
maximum number of NICs that is allowed for each Ganeti instance. The
value of this setting must be less than 'max:nic-count` option of
Ganeti's ipolicy.

cyclades: Ignore messages about deleted resources

Ignore messages from Ganeti about servers and networks that have already
been marked as deleted in the Cyclades DB. Processing of these messages
may asynchronize Cyclades with Astakos quotas.

cyclades: Remove testing try/exception statement

cyclades: Fix bug in getting a free floating IP

Fix wrong query which resulted in getting non-floating IPs.

cyclades: Fix DB locking issues causing deadlocks

Fix deadlock issues that is caused by Postgresql's locks on foreign
keys. For example, creation of a BackendNetwork row will result in a
shared lock in the 'backend_id' rows. Because updating a VM's NICs
acquired an exclusive lock on the backend that the VM lived, this...

cyclades: Require subnet for IP_LESS_ROUTED nets

cyclades: Add --drained option to network-create

cyclades: Optimize server-list command

Optimize server-list command by prefetching related fields if needed.

cyclades: Fix wrong state of deleted networks

Before deleting a network, the network is marked as drained to be
excluded from automatic IP allocation. The state of the network that is
exposed in the API is always 'SNF:DRAINED' if the drained flag is set....

cyclades: Lock objects in modifying commands

Add some missing Copyright notices

Also add 2013 in package COPYRIGHT files.

cyclades: Fix bug by wrong indentation

cyclades: Include 'vlmc' and 'archipelago'

Fix image ID for both 'vlmc' and 'archipelago' ext providers.

cyclades: Fix a bug in logic/subnets.py

• Don't allow creation of subnets on a network that is initilized in a
  backend

cyclades: Update some snf-manage commands

• Update network-remove, server-remove, floating-ip-remove, port-remove
  to extend RemoveCommand class added in commit
  ec15e2ab68f90dcc7310863c9293708055fbfb7d

cyclades: Fix a bug in logic/subnets.py

• Fix a potential crash if the gateway IP was the last available IP of
  the subnet
• Create the corresponding test

cyclades: Map reboot type to shutdown timeout

Fix handling of reboot type option. If 'hard' reboot type is specified
this is mapped to zero shutdown timeout of OP_INSTANCE_REBOOT Ganeti
job. This is only supported from Ganeti>=2.10 and snf-ganeti. In other...

cyclades: Fix removal of stale NICs in BUILD state

Until now, NICs that did not exist in the Ganeti backend and were in
BUILD state were removed after a timeout (BUILDING_NIC_TIMEOUT). This
was dangerous, because if an OP_INSTANCE_CONNECT job had been "stuck"...

cyclades: Command to export Cyclades statistics

Add new 'snf-manage stats-cyclades' management command to export basic
statistics about Cyclades, like number of servers, network and resource
utilization. The command can either export the statistics either in JSON...

cyclades: Fix bugs relative with quotas

• Reject previous unaccepted serials if action is 'DESTROY'. This is
  required because multiple DESTROY actions may be pending for the same
  resource (e.g. VM). The one who succeeds will be accepted, while all
  others will be rejected. This isn't optimal since it causes circular...

gtools: Fix Ganeti allocation parameters

When retrying a failed job, remove 'pnode' and 'snode' fields if they
were implicitly set by Ganeti iallocator. Ganeti will fail if both
iallocator and node are specified in the job.

cyclades: Generalize issue_and_accept_commission

Use issue_and_accept_commission for any action where one needs to
automatically accept the commission after issuing and committing.

cyclades: Resolve undecided serial in force mode

Reject a serial with pending=True only when we need to force a new
commission on a given resource.

Also improve handling of serial state: make sure pending is set to False
when we decide whether to accept or reject a serial. Mark a serial to...

cyclades: Allow deletion of ports from public nets

Move check about deleting a port from a public network. This check is
only enforced in the API, but is allowed in logic layer, e.g. snf-manage
commands.

cyclades: Update disk_templates if empty

Make backend_allocator to refresh the available disk templates of a
backend if there are empty.

cyclades: Check for all finalized states

Merge branch 'feature-cyclades-networks' into develop

Conflicts:
snf-cyclades-app/synnefo/api/management/commands/server-inspect.py
snf-cyclades-app/synnefo/app_settings/urls.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/logic/backend_allocator.py...

cyclades: Fix a bug in network and subnet create

• Fix a bug if the user specified no allocation pools
• Fix a wrong error message in logic/subnets.py

cyclades: Retry failed instance creation

Handle special case where an OP_INSTANCE_CREATE job which uses the
opportunistic locking feature fails. This failure is probably due to the
fact that all nodes may be already locked. In this case, make
snf-dispatcher retry the failed job with opportunistic locking turned...

cyclades: fix typo in snf-dispatcher

cyclades: Small fixes to network reconciliation

• Pass missing argument to function
• Fix corner case where there are no externally reserved IPs.

cyclades: Remove hotplug from InstanceCreate

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

cyclades: Remove deprecated network type in ganeti

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

gtools: Fix compatibility issues with ganeti-2.7

• SimpleConfigReader has been removed. Use SimpleStore
• QUEUE_DIR has been moven from ganeti.constants to ganeti.pathutils
• _QueuedJob.Restore takes 2 more arguments
• cyclades: Group info is returned as list instead of string

Merge branch 'master' into develop

Conflicts:
Changelog
astakosclient/astakosclient/keypath.py
ci/utils.py
docs/admin-guide.rst
snf-astakos-app/astakos/im/cookie.py
snf-astakos-app/astakos/im/functions.py
snf-astakos-app/astakos/im/quotas.py
snf-astakos-app/astakos/im/views/projects.py...

cyclades: Do not print IP pool for IPv6 only nets

Do not print IP pool of Ganeti network in case the network is an IPv6
only network. The Ganeti network has a dummy IPv5 subnet, just because
it is needed. Only change this dummy subnet to /30 to reduce space in...