Synnefo

cyclades: Better handling of deleted subnets

• Fix a bug where snf-manage subnet-list printed deleted subnets too
• Fix a query, that also returned deleted subnets

cyclades: Map reboot type to shutdown timeout

Fix handling of reboot type option. If 'hard' reboot type is specified
this is mapped to zero shutdown timeout of OP_INSTANCE_REBOOT Ganeti
job. This is only supported from Ganeti>=2.10 and snf-ganeti. In other...

cyclades: Fix removal of stale NICs in BUILD state

Until now, NICs that did not exist in the Ganeti backend and were in
BUILD state were removed after a timeout (BUILDING_NIC_TIMEOUT). This
was dangerous, because if an OP_INSTANCE_CONNECT job had been "stuck"...

cyclades: Export Cyclades detailed stats from API

Add '/admin/stats/detail' API endpoint to export basic statistics about
Cyclades. Access to this view is only allowed to the users that belong
to specific Astakos groups, as specified in the new setting
'ADMIN_STATS_PERMITTED_GROUPS.' Also, added the new 'user_in_groups' API...

gtools: Fix Ganeti allocation parameters

When retrying a failed job, remove 'pnode' and 'snode' fields if they
were implicitly set by Ganeti iallocator. Ganeti will fail if both
iallocator and node are specified in the job.

cyclades: Issue commission based on action

Make issue_commission() the single point where provisions are computed
and the appropriate astakosclient call is issued.

cyclades: Pass action params in server_command

Commission to be issued depends on extra parameters for some actions,
e.g., on the flavor when resizing a vm.

Merge branch 'hotfix-0.14.10' into develop

Conflicts:
Changelog
snf-cyclades-app/synnefo/api/servers.py
snf-cyclades-app/synnefo/logic/backend.py
snf-cyclades-app/synnefo/logic/callbacks.py
snf-cyclades-app/synnefo/logic/management/commands/reconcile-networks.py...

cyclades: Check drained only if not destroying

Check drained flag only if network actions is not destroy. When network
is being destroyed the network is automatically marked as drained to
exclude it from IP allocations.

cyclades: Handle snf-ganeti bug

Create IPv6 only networks as /29 because snf-ganeti can not create /30
networks.

cyclades: Fix some broking tests

cyclades: Add a new function in logic/subnets.py

• Add a function in logic/subnets.py that parses allocation pools
• Add a new option in snf-manage network-create for allocation pools
• Update subnet-create to use the new function

cyclades: Fix compatibility issues with networks

From version 2.7, Ganeti returns the network's UUID field in nics
'networks' attribute. Instead use 'networks.names' attribute to get
the network name.

cyclades: Fix case where 'dfree' is None

If Ganeti has not enabled 'plain' or 'drbd' disk template, then it
reports 'dfree' as None. Fix this corner case by treating None as 0.

cyclades: New style format for modifying a NIC

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

cyclades: Pass network type info as network tag

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

cyclades: Use opportunistic locking

Use opportunistic locking, introduced in Ganeti 2.7, to allow parallel creation
of VMs in the Ganeti backend. When this option is set, Ganeti iallocator
will allocate the VM only to nodes that not locked (e.g. by other VM...

cyclades: Fix bug in network reconciliation

Skip reconciliation for a network that does not exist in the DB, if
reconciliation is running without the '--fix' option.

cyclades: Simplify code to 'logic.subnets' module

cyclades: Handle 'drained' flag of networks

• Export 'SNF:DRAINED' status for drained networks.
• Do not allow creation of new ports in drained networks.
• Do not allow allocation of new IPs in drained networks.

cyclades: Fix reconciliation of floating IP pools

Remove special handling of floating IP pools in reconciliation
mechanism since floating IP pools are treated the same way with other
networks.

cyclades: Optimize *-list commands

Use prefetch_related and select-related to importive the DB queries that
are generated by *-list snf-manage commands.

cyclades: Fix init of networks in backend-add

Fix which networks are going to be initialized in the new backend and
how the are printed in the output!

cyclades: Fix bugs in subnets and ports

• Return subnets from public networks, in GET subnets
• Fix a bug in list ports, where only attached ports to servers were
  listed

cyclades: Merge subnet-create with network-create

• Merge snf-manage subnet-create into snf-manage network-create

cyclades: Do not allow any action on 'ERROR' vms

Do not allow any action, other than 'DESTROY' in servers that are in
'ERROR' state. If the server is in 'ERROR' state then the corresponding
Ganeti instance may not exist, so all actions will fail.

cyclades: Add missing 'ips' module

cyclades: Workaround for race condition in Ganeti

Workaround for race where OP_INSTANCE_REMOVE starts executing on Ganeti
before OP_INSTANCE_CREATE. If this is the case, OP_INSTANCE_REMOVE will
not wait for locks, instead it will fail because the instance has not...

cyclades: Create server in public net with FIP

Remove 'floating_ips' extension attribute to POST /servers API call.
Instead use the 'fixed_ip' attribute of 'networks' attribute to specify
the IPv4 address of a floating IP. So in order to create a server
connected to a private network and a public network, the 'networks'...

cyclades: Delete backend networks with network

Delete backend network when network is deleted (marked as deleted=True)
in DB. Backend networks are useless as history and they may be recreated
if needed. Also fix some tests.

cyclades: Fix permissions when looking up ports

Look up of ports should check the port userid and not the network
userid.

cyclades: Fix double double creation of IP log

The port creation command called 'associate_port_with_machine' function
in order to associate the port with the machine and create an
IPAddressLog entry. However, this is already done by the connect
function which sends the job in the Ganeti backend. This resulted in...

cyclades: Create ports without 'device_id'

Allow creation of ports that are not connected to any instance. Such
ports will be in 'DOWN' state. To allow this, 'machine' attribute of
NetworkInterface model has been made to allow null values. Finally, the
'logic.ports' module has been remove, and the corresponding code has...

cyclades: Add accounting for IP Addresses

Create IPAddressLog DB table to hold information about allocation of IP
addresses to virtual servers. The information that is currently saved
contains the IP address, the ID of the virtual server, the ID of the
virtual network along with the datetime that the address has been...

cyclades: Fix bug in port-create

Check if floating-ip belongs to the given network

cyclades: Various refactors in subnets

• Remove unused functions and imports from logic/subnets.py
• Remove --network-id option from snf-manage subnet-modify and make
  it an argument
• Fix --ip-version, --dhcp and --gateway options in snf-manage
  subnet-create...

cyclades: Update state of NICs if VM creation fail

Set the state of NICs(ports) to ERROR if the OP_INSTANCE_CREATE job
fails.

cyclades: Common module for pretty-printing

Create common module for pretty-printing database objects. Currently it
contains only functions for pretty-printing networks and ports.

cyclades: Various fixes to networks/subnets/pools

• Fix IP pools of subnets to take into account the 'offset' when searching
  if a pool contains an IP address and when converting from value to index
  and from index to value.
• Fix the way subnets are created from 'logic/networks'...

cyclades: Small bug fixes in logic/subnets.py

cyclades: Fix some bugs in subnets

• Fix a bug, where a user could change the name of a subnet without
  being the owner of said subnet.
• Fix two bugs in snf-manage subnet-create. Fix no-dhcp option and a
  potential crash if the user specified an invalid IP version type....

cyclades: Remove stale references to vm.router

cyclades: Create network in backend if missing

Automatically create a network in a Ganeti backend if missing.
Currently, this was done only for private networks. This commit changes
this to be done for any kind of networks.

cyclades: Use existing IP when creating port

Extend port creation with the option to use an existing IPAddress. This
will be used to create a port with an existing Floating IP. Also, extend
port-creata management command with the '--floating-ip' option.

cyclades: PEP8 fix

cyclades: snf-manage floating-ip-attach

snf-manage floating-ip-attach <address> --machine <server_id/router_id>

cyclades: move subnets.py from api to logic

Move api/subnets.py to logic/subnets.py, for further code refactoring

cyclades: Fix bug when deleting instances

If OP_INSTANCE_REMOVE runs before OP_INSTANCE_CREATE the job will fail.
Dispatcher will ask Ganeti if the instance exists, and if not it will
release it's IPv4 address. However the instance may not exist because it...

cyclades: Add backend code to Ports API/logic

Add backend code to create and remove ports from Ganeti backends. Also
add some imporovements and fixes to logic, API code and management
commands.

Resolve some undefined names

Add missing imports; remove call to sqlite3_changes; set key and val
in render_meta.