cyclades: Refactor backend-update-status cmd
The command has been changed to no longer take any arguments or options.Instead, this command always updates the resources and the enabled disktemplates of all online Ganeti backends.
Allocate VMs based on enabled disk-templates
Remove the ARCHIPELAGO_BACKENDS setting. Instead allocate VMs based onthe disk-templates that are enabled and allowed in ipolicy for eachbackend. The command 'snf-manage backend-update-status' can be used to...
cyclades: Display disk-templates in backend-list
cyclades: Return the newest vm object upon creation
The server_command decorator commits the just created object and retrievesit back from the database. Return the latter, because the former is stalein memory.
cyclades: Fix wrong quotas when creating server
Catch exception when talking to Ganeti inside 'server_command',otherwise the pending commission will be rejected. Also, accept thecommission in case of build, because the VM will be created in DB.
cyclades: Helper function for commissioning
Create 'quotas.handle_resource_commission' function for handlingcommissions for VMs and Networks.
cyclades: Handle errors during vm creation
Commit #f2080d1 mocked an OP_INSTANCE_REMOVE when enqueuing anOP_INSTANCE_CREATE job to Ganeti failed. However, this may result inreleasing IPs that may get reserved in Ganeti, because we can neverreally know if the job has reached Ganeti or not (e.g. timeout). For...
cyclades: refactor server creation command
Split the server creation command in more steps in order to bedecorated with the 'server_command' function and follow the logic of allother server commands. The main change is that the commission job is notaccepted when the VM is stored in DB, but when the OP_INSTANCE_CREATE...
cyclades: Apply old commits to moved files
Code from file 'api/servers.py' has been moved to 'logic/servers.py'.During the merge of hotfix-0.14.6 the develop, this code was lost. Thiscommit applies commits #f2080d1, #ea9b0fb, #32a0b85 to the new file.
Merge branch 'hotfix-0.14.6' into develop
Conflicts: Changelog docs/admin-guide.rst snf-cyclades-app/synnefo/api/management/commands/network-remove.py snf-cyclades-app/synnefo/api/servers.py snf-cyclades-app/synnefo/api/tests/servers.py snf-cyclades-app/synnefo/logic/backend.py...
cyclades: Make GanetiRapiClient work with gevent
GanetiRapiClient uses PyCurl, a Python interface to libcurl, andperforms blocking requests, so does not work well with gevent. Thiscommit makes GanetiRapiClient to use 'Requests' HTTP library that isgreenlet-safe.
cyclades: Simplify code
Simplify code by removing unnecessary temp variable.
cyclades: Do not release resources if already deleted
Ensure that a commission to release vm resources is issued only once.If a vm is marked `deleted', then resources must have already beenreleased.
cyclades: Do not save stale objects after a commit
Call to issue_and_accept_commission saves the resource object (eg a vm)and commits the transaction. This has the side-effect that the resourceobject becomes stale: its lock is released and another thread may acquire...
cyclades: fix typos
cyclades: Update NICs even if job failed
Make snf-ganeti-eventd send the NICs of the instance not only onsuccess, but also if the job has failed or has been canceled. Also,update snf-dispatcher to handle those messages. This step is required toclear BUILDING NICs that fail.
cyclades: Store action to the VM
cyclades: Fix bug in reconciliation
VMs at building error that do not exist in Ganeti must not be reportedas stale. The VM is stale only if the user has asked to destroy the VM,and so the action is DESTROY.
cyclades: Create NIC before send job to Ganeti
Create NICs in building state before enqueuing corresponding jobs toGaneti. This has already been done when creating the instance, but notwhen connecting it to other networks.
cyclades: Fix bug in snf-dispatcher logic
This commit reverts commits #07602322 and #30514662 which introduced abug in dispatcher's logic. These commit fixed the issue that, in a casethe Cyclades DB is unsynced with Ganeti, a resource that does not exist...
cyclades: Fix bug in handling of reboot type.
'type' attribute of reboot OS API call was directly mapped to the'reboot_type' option of Ganeti reboot job. However these two aresemantically different. The 'SOFT' and 'HARD' reboot types of OS API aremapped to 'shutdown_timeout' option of Ganeti jobs. However, setting...
cyclades: Do not pass 'hotplug' if disabled
Fix passing of hotplug argument to GanetiRAPIClient, in case theGANETI_USE_HOTPLUG is unset. If the Ganeti version does not supporthotplug, passing hotplug=False, will make the jobs fail at Ganeti opcodevalidation. This commit fixes this by passing the argument, only when...
cyclades: Add check for net action with no hotplug
In case hotplug is disabled, via GANETI_USE_HOTPLUG setting, networkactions should be allowed only on STOPPED vms. This commit adds thischeck and a unittest for this case. Also it makes some other checks...
cyclades: Fix check for firewall profile
cyclades: Remove firewall tag when removing NIC
Remove firewall tag from ganeti instance when disconnecting them fromnetworks. Until now, this was not needed because a vm could not bedisconnected from public networks. Since floating IPs, a vm may bedynamically be connected and disconnected to public networks, and so a...
cyclades: Fix bug in servers tests
Import settings from django.conf instead of synnefo.
cyclades: Import settings from django.conf
cyclades: init dispatcher with DJANGO_SETINGS_MODULE
cyclades: Fix typo
cyclades: Fix condition for network netmask size
cyclades: Check that IPv6 network netmask is /64
Check that IPv6 network netmask is /64, because this is imposed by the'mac2eui64' utility that generates EUI-64 addresses from 48-bit MACaddresses.
Refs #4201
cyclades: Move code for validating network values
Move code for network parameters validation from 'api' module to 'logic'.
Fix stale import statements
Commit 7cfc0ce moved code from webproject to snf_django. This commitfixes some stale imports that skipped renaming.
snf-django: Move ListCommand to snf-django
snf-webproject contained the generic ListCommand, and because of this ithad a dependency from snf-django-lib. This commit moves this command fromsnf-webproject to snf-django-lib and removes this dependency. Also,...
cyclades: Add 'name' to all commissions.
Make all quotaholder commissions to have a name. Since, Cyclades doesnot store the association of a commission with a resource, we includethe resource in the commission name.
cyclades: Add network to multiple backends
Update snf-manage network-create command to take a list of commaseperated backend IDs to create the network to.
cyclades: Add check for link during network create
Add a check when creating IP_LESS_ROUTED networks, that the connectivitylink is not already used by another network.
cyclades: Fixes in network management commands
Merge code from logic/networks.py and network management commands.
cyclades: Set firewall profile per NIC
Set firewall profile per NIC, because since Floating IPs an instance canhave more than one NICs in a public network. Extend the API call to takethe NIC index as argument, and modify the Ganeti instance tags to beformated with the NIC index.
cyclades-rapi: Add bulk option to GetJobs
cyclades: Reconciliation for pending vm tasks
Extend reconciliation mechanism, to detect instances with stale pendingtasks, by looking if the corresponding job, 'vm.task_job_id', is stillrunning in the Ganeti backend.
Also refactor code that performed reconciliation for buildign vms, to...
cyclades: Minor improvements in manage commands
cyclades: Fix import of renamed settings
cyclades: No commission for public networks
Do not issue commission for public networks, since they do not have anowner.
cyclades: Consider IPv6 only networks in allocator
Backend allocator excludes backends that do not have a free IPv4address. Since, from now on, not all instances are connected to a publicnetwork, this check is only needed when 'SNF:ANY_PUBIC' network is used.
cyclades: Implement NetworkReconciler class
Refactory network reconciliation. Implement NetworkReconciler class tohold all reconciliation functions.
cyclades: Move NetworkReconciler to reconciliation
Move NetworkReconciler class from network-create command toreconciliation module.
cyclades: Add tests for network reconciliation
cyclades: Simple code refactoring
cyclades: Support IPv6 only networks
Make Cyclades support IPv6 only networks. Such networks will havethe 'subnet' attribute set to None, in the same way that IPv4 onlynetworks have the 'subnet6' attribute set to None.
Because currently Ganeti does not support IPv6 only networks, we create the...
cyclades: Add tests for IPv6 only networks
Add tests to check creation and usage of IPv6 only networks. Tests alsocover creation of other type of networks.
cyclades: Refactor code relative to networks
Move code checks from network-create management command tologic/networks.py.
Also create common decorator for management commands to convert apifaults to command errors.
cyclades: Refactor logic tests
Create tests module to hold all tests for logic.
Merge branch 'hotfix-0.14.2' into develop
Conflicts: Changelog snf-astakos-app/astakos/im/messages.py snf-astakos-app/astakos/im/views/target/__init__.py snf-cyclades-app/synnefo/logic/backend.py snf-cyclades-app/synnefo/logic/management/commands/reconcile-servers.py...
cyclades: Fix reconciliation of building vms
When reconciling a building vm, you first have to reconciliation it'screation and then it's operstate. Otherwise it is possible to end uphaving wrong quotas.
cyclades: Fix corner case in snf-dispatcher
Same as commit 0514662, but for servers.
cyclades: Fix typo bug in management command
cyclades: Fix type error
Create network may return a list of job IDs.
cyclades: Fix wrong unit of ram
cyclades: Parallel server reconciliation
Major refactor to reconcile-servers management command to make it workin parallel for each backend. Also, removed all --detect options, anddetect all incosistencies by default.
cyclades: Add tests for server reconciliation
Delete network from DB when job fails but the action of network isDESTROY. This case appears when someone tries to remove a network thathas been deleted from Ganeti backend, but not yet from DB.
cyclades: Fix typo in reconciliation
reconciliation: Strip Ganeti IP addresses
Strip spaces from Ganeti IP addresses, since they are not automaticallyignored in newer versions of ipaddr.
reconciliation: Get servers from DB ater ganeti
Change server reconciliation, to get the instances from DB after gettinginstances from Ganeti, to decrease the possibility for races.
cyclades: Fix exception message
Fix exception message when trying to connect an instance to a non-activepublic network.
cyclades: Wrap diagnostic creation in transaction
Fix bug in snf-dispatcher causing wrong updates of VMs from ganeti.Creation if instance diagnostics must run inside a transaction, becauseotherwise, the lock of the vm (select_for_update) is released too early....
cyclades: Rename "public" to "SNF:ANY_PUBLIC"
Rename reserved work for automatically choosing a public network from"public" to "SNF:ANY_PUBLIC". Also add the setting description to conffiles.
cyclades: Remove PUBLIC_USE_POOL setting
Remove PUBLIC_USE_POOL setting, since in order to use floating IPs, thecyclades must be responsible for managing all IP pools.
cyclades: Check if a NIC is a floating IP.
Extend NetworkInterface model with a property showing if the IPv4address of the NIC is a floating IP or not. Also, simplify somerelevant code.
cyclades: Choose floating-ips when creating server
Extend POST /servers API call, to take an optional attribute'floating_ips', which is a list of floating IP addresses. The serverwill have one NIC for each of this addresses after the'DEFAULT_INSTANCE_NETWORKS' and before any private networks that the...
Distinquish between networks and Floating IP pools
Extend Network model with the 'floating_ip_pool' boolean field, toindicate whether a Network can be a floating IP pool or not. Networksthat are floating IP pools must exist in all Backends, and so'snf-manage backend-add' should try to add them....
cyclades: Various fixes regarding floating IPs
Fix some issues regarding floating ips introduced by rebasing branch.
Fixes to tests
cyclades: Separate logic for networks from API
Remove code handling networks from network API views to logic.networks
cyclades: Add 'floating-ip-list' management cmd
cyclades: Add network policy at server create
Add 'DEFAULT_INSTANCE_NETWORKS' setting that can be used by theadministrator to define the list of networks that each server that iscreated must be connected to. Each created VM will have a NIC connectedto each of these networks. This settings can include a list of network...
cyclades: Do not release floating IPs
Update 'release_instance_nics' backend function which is called toprocess NIC modifications in the Ganeti backend, to not return aIPv4 address back to pool if this address is a floating IP.
cyclades: Update reconcilliation for floating IPs
Update network reconcilliation to consider floating IPs reconcilling IPpools. Floating IPs that are reserved and usused, will appear asasynced. This commit fixes this, by considering unused floating IPs as...
cyclades: Check if network exists in backend
Raise 503 if a user tries to add a floating IP to a VM that resides in aGaneti backend that the coressponding network does not exist.
cyclades: Support many NICs when creating vms
Support arbitraty number of NICs when creating Ganeti instances.
cyclades: Extend /servers with floating IP actions
Extend '/servers/action' API endpoint, to support 'addFloatingIP'and 'removeFloatingIP' actions.
cyclades: refactor reconciliation
Refactor reconciliation code to use namedtuples and improve performancefor retrieving info from database.
cyclades: reconciliation for resizing VMs
cyclades: Major refactor to api and logic apps
Major refactor to cyclades api and logic apps that removes most codefrom the api views and moves it to logic app, so that the same logiccode can be used by different apps (e.g. EC2 API). Code is moved into...
cyclades: Handle 'EmptyPool' in util method
Refactor cyclades code to handle 'EmptyPool' exception in'get_network_free_address' function.
cyclades: Add backend function for resizing vm
Add 'resize_instance' backend function for resizing a vm. This commandissues an OP_INSTANCE_MODIFY job with the specified beparams, whichcurrently are vcpus, minmem and maxmem.
Refs #3870
cyclades: Handle Ganeti msgs for resized VMs
Handle 'OP_INSTANCE_SET_PARAMS' notifications from Ganeti that have'beparams' attribute, which corresponds to resized VMs. In suchcase, the corresponding flavor is found an the VM in DB is updated tohave this flavor....
cyclades: Fix assertion on mocked client
cyclades: Fix bug in testcase
Merge branch 'release-0.14' into develop
Conflicts: snf-astakos-app/astakos/im/views/target/redirect.py snf-cyclades-app/synnefo/logic/backend.py version
cyclades: Fixes relative to reconciliation
Get available backends once and work only with them, since availablebackends may change during reconciliation. Also temporary remove supportfor communicating with each backend in a separate process.
Revert "cyclades: Speed up reconciliation"
This reverts commit cf2241c45a22dac71e31dbf32f4700c475804c12.
cyclades: Speed up reconciliation
Speed up reconciliation by using mulitprocessing to get in parallelthe instances from all Ganeti backends.
cyclades: Fix bug in allocating archipelago VMs
This commit fixes a bug that caused backend allocator to sendarchipelago VMs backends to backends not in ARCHIPELAGO_BACKENDSsetting. The flavor disk template is parsed before the backendallocation, so the disk_template at this point is ext and not ext_vlmc,...
cyclades: Handle errors when using multiprocessing
Check the process exitcode to determine if a process finishedsuccessfully.
cyclades: Speed up server reconciliation
Speed up reconciliation, by increasing the time interval to get Ganetijob status for VMs in building state.
cyclades: Fix deadlock when creating servers
Fix deadlock between snf-dispatcher and backend allocator, by makingsnf-dispatcher take an exclusive lock to the backend which hosts the vm,before trying to get the exclusive locks to the IP pools in order to...