cyclades: Simple code refactoring
cyclades: Support IPv6 only networks
Make Cyclades support IPv6 only networks. Such networks will havethe 'subnet' attribute set to None, in the same way that IPv4 onlynetworks have the 'subnet6' attribute set to None.
Because currently Ganeti does not support IPv6 only networks, we create the...
cyclades: Add tests for IPv6 only networks
Add tests to check creation and usage of IPv6 only networks. Tests alsocover creation of other type of networks.
cyclades: Refactor code relative to networks
Move code checks from network-create management command tologic/networks.py.
Also create common decorator for management commands to convert apifaults to command errors.
Merge branch 'hotfix-0.14.2' into develop
Conflicts: Changelog snf-astakos-app/astakos/im/messages.py snf-astakos-app/astakos/im/views/target/__init__.py snf-cyclades-app/synnefo/logic/backend.py snf-cyclades-app/synnefo/logic/management/commands/reconcile-servers.py...
cyclades: Fix reconciliation of building vms
When reconciling a building vm, you first have to reconciliation it'screation and then it's operstate. Otherwise it is possible to end uphaving wrong quotas.
cyclades: Fix corner case in snf-dispatcher
Same as commit 0514662, but for servers.
cyclades: Fix typo bug in management command
cyclades: Fix type error
Create network may return a list of job IDs.
cyclades: Fix wrong unit of ram
cyclades: Parallel server reconciliation
Major refactor to reconcile-servers management command to make it workin parallel for each backend. Also, removed all --detect options, anddetect all incosistencies by default.
cyclades: Add tests for server reconciliation
Delete network from DB when job fails but the action of network isDESTROY. This case appears when someone tries to remove a network thathas been deleted from Ganeti backend, but not yet from DB.
cyclades: Fix typo in reconciliation
reconciliation: Strip Ganeti IP addresses
Strip spaces from Ganeti IP addresses, since they are not automaticallyignored in newer versions of ipaddr.
reconciliation: Get servers from DB ater ganeti
Change server reconciliation, to get the instances from DB after gettinginstances from Ganeti, to decrease the possibility for races.
cyclades: Fix exception message
Fix exception message when trying to connect an instance to a non-activepublic network.
cyclades: Wrap diagnostic creation in transaction
Fix bug in snf-dispatcher causing wrong updates of VMs from ganeti.Creation if instance diagnostics must run inside a transaction, becauseotherwise, the lock of the vm (select_for_update) is released too early....
cyclades: Rename "public" to "SNF:ANY_PUBLIC"
Rename reserved work for automatically choosing a public network from"public" to "SNF:ANY_PUBLIC". Also add the setting description to conffiles.
cyclades: Remove PUBLIC_USE_POOL setting
Remove PUBLIC_USE_POOL setting, since in order to use floating IPs, thecyclades must be responsible for managing all IP pools.
cyclades: Check if a NIC is a floating IP.
Extend NetworkInterface model with a property showing if the IPv4address of the NIC is a floating IP or not. Also, simplify somerelevant code.
cyclades: Choose floating-ips when creating server
Extend POST /servers API call, to take an optional attribute'floating_ips', which is a list of floating IP addresses. The serverwill have one NIC for each of this addresses after the'DEFAULT_INSTANCE_NETWORKS' and before any private networks that the...
Distinquish between networks and Floating IP pools
Extend Network model with the 'floating_ip_pool' boolean field, toindicate whether a Network can be a floating IP pool or not. Networksthat are floating IP pools must exist in all Backends, and so'snf-manage backend-add' should try to add them....
cyclades: Various fixes regarding floating IPs
Fix some issues regarding floating ips introduced by rebasing branch.
Fixes to tests
cyclades: Separate logic for networks from API
Remove code handling networks from network API views to logic.networks
cyclades: Add 'floating-ip-list' management cmd
cyclades: Add network policy at server create
Add 'DEFAULT_INSTANCE_NETWORKS' setting that can be used by theadministrator to define the list of networks that each server that iscreated must be connected to. Each created VM will have a NIC connectedto each of these networks. This settings can include a list of network...
cyclades: Do not release floating IPs
Update 'release_instance_nics' backend function which is called toprocess NIC modifications in the Ganeti backend, to not return aIPv4 address back to pool if this address is a floating IP.
cyclades: Update reconcilliation for floating IPs
Update network reconcilliation to consider floating IPs reconcilling IPpools. Floating IPs that are reserved and usused, will appear asasynced. This commit fixes this, by considering unused floating IPs as...
cyclades: Check if network exists in backend
Raise 503 if a user tries to add a floating IP to a VM that resides in aGaneti backend that the coressponding network does not exist.
cyclades: Support many NICs when creating vms
Support arbitraty number of NICs when creating Ganeti instances.
cyclades: Extend /servers with floating IP actions
Extend '/servers/action' API endpoint, to support 'addFloatingIP'and 'removeFloatingIP' actions.
cyclades: refactor reconciliation
Refactor reconciliation code to use namedtuples and improve performancefor retrieving info from database.
cyclades: reconciliation for resizing VMs
cyclades: Major refactor to api and logic apps
Major refactor to cyclades api and logic apps that removes most codefrom the api views and moves it to logic app, so that the same logiccode can be used by different apps (e.g. EC2 API). Code is moved into...
cyclades: Handle 'EmptyPool' in util method
Refactor cyclades code to handle 'EmptyPool' exception in'get_network_free_address' function.
cyclades: Add backend function for resizing vm
Add 'resize_instance' backend function for resizing a vm. This commandissues an OP_INSTANCE_MODIFY job with the specified beparams, whichcurrently are vcpus, minmem and maxmem.
Refs #3870
cyclades: Handle Ganeti msgs for resized VMs
Handle 'OP_INSTANCE_SET_PARAMS' notifications from Ganeti that have'beparams' attribute, which corresponds to resized VMs. In suchcase, the corresponding flavor is found an the VM in DB is updated tohave this flavor....
cyclades: Fix assertion on mocked client
cyclades: Fix bug in testcase
Merge branch 'release-0.14' into develop
Conflicts: snf-astakos-app/astakos/im/views/target/redirect.py snf-cyclades-app/synnefo/logic/backend.py version
cyclades: Fixes relative to reconciliation
Get available backends once and work only with them, since availablebackends may change during reconciliation. Also temporary remove supportfor communicating with each backend in a separate process.
Revert "cyclades: Speed up reconciliation"
This reverts commit cf2241c45a22dac71e31dbf32f4700c475804c12.
cyclades: Speed up reconciliation
Speed up reconciliation by using mulitprocessing to get in parallelthe instances from all Ganeti backends.
cyclades: Fix bug in allocating archipelago VMs
This commit fixes a bug that caused backend allocator to sendarchipelago VMs backends to backends not in ARCHIPELAGO_BACKENDSsetting. The flavor disk template is parsed before the backendallocation, so the disk_template at this point is ext and not ext_vlmc,...
cyclades: Handle errors when using multiprocessing
Check the process exitcode to determine if a process finishedsuccessfully.
cyclades: Speed up server reconciliation
Speed up reconciliation, by increasing the time interval to get Ganetijob status for VMs in building state.
cyclades: Fix deadlock when creating servers
Fix deadlock between snf-dispatcher and backend allocator, by makingsnf-dispatcher take an exclusive lock to the backend which hosts the vm,before trying to get the exclusive locks to the IP pools in order to...
cyclades: Validate MAC prefix in pool-create cmd
cyclades: Pass disk origin for all 'ext' flavors
cyclades: Rename service token
Rename CYCLADES_ASTAKOS_SERVICE_TOKEN to CYCLADES_SERVICE_TOKEN foruniformity.
make synnefo URLs consistent and configurable
https://service.deployment.vendor.tld/service-base/api-prefix/hard/path |___________________________________| |__________| |________| |_______| BASE_HOST BASE_PATH PREFIX Hardcoded...
cyclades: Add 'drained' flag to Network
Extend Network model with the 'drained' attribute, to indicate whether anetwork should be used in public IP allocation when creating servers.This option is only useful for public networks.
cyclades: Export hypervisor to backend-list cmd
cyclades: Add --hypervisor option to backend cmds
Add '--hypervisor' option to 'backend-add' and 'backend-modify'management commands.
Refs #3772
cyclades: Sypport backends with Xen hypervisor
cyclades: Various fixes to tests
dispatcher: Handle NICs in callbacks.
Update 'update_db' callback to handle instance NICs together withinstance status.
Also update the relevant tests.
cyclades: simplify comparison of NICs
Remove 'snf-events-net' queue and binding
Remove 'snf-events-net' queue (together with the corresponding routingkey and binding) from the queues that snf-dispatcher is consumingmessages. This queue is no longer needed, because messages for instanceNICs will be sended by snf-ganeti-eventd to 'snf-events-op' queue....
cyclades: Fix deadlock when connecting vms
Fix deadlock caused by 'get_or_create' method, since it acquires animplicit shared lock. Take an exclusive lock on network to fix thisdeadlock.
cyclades: Fixes relevant to lazy networks
cyclades: Simplify reconcile-networks checks
Refactor and simplify checks of reconcile-networks to take advantageof lazy networks.
cyclades: update tests for lazy networks
cyclades: Update 'backend-add' for lazy networks
Do not create private networks when adding a new backend. Since,"lazy networks" this is no longer needed.
cyclades: Implement lazy networks
Implement "lazy networks", which are not created to all Ganeti backendsupon creation, but only when an instance that resides in a Ganeti isconnected to that network.
Update 'create_network' to create the Network object in DB without...
cyclades: refactor code relative to commission
Refactor Cyclades code that handles commissioning with Quotaholder. Thiscommit removes the 'uses_commission' decorator. Instead it introducesthe 'issue_and_accept_commission' function that is responsible for...
Mock quotaholder for Cyclades tests
Create 'mocked_quotaholder' context manager to bypass commission and useit in Cyclades tests. Also this commit does some various fixes to othertests.
Remove transaction from 'backend-add' command
Do not run 'backend-add' command inside transaction, because if'snf-dispatcher' is running, it will receive messages that will arrivefrom a Ganeti backend that does not exist in the DB and will not be ableto process them.
Move astakos lib from snf-common to snf-django-lib
Use 'python-objpool' instead of 'synnefo.lib.pool'
Use the python-objpool package for pooling resources as object, insteadof the pooling code that existed in snf-common ('synnefo.lib.pool')
Refs #3524
Use pprint_table in server-show command
Create public NIC when creating VM
Until now, a server's public NetworkInterface was not created in the DBuntil the arrival of the message from Ganeti hooks. However, the IPv4address of the NIC is reserved before sending the OP_INSTANCE_CREATEjob to Ganeti....
Move UserCache from cyclades.common to lib.astakos
Move UserCache from synnefo.management.common to synnefo.lib.astakos.Also, update ListCommand with 'astakos_url' and 'astakos_token' classattributes, which must be declared only when 'user_uuid_field' is used.
Move common code for mngmt commands to webproject
Move cyclades functions that are useful for all managament commands, likepprint_table or parse_filters, to snf-webproject.
Add ListCommand to be used for all *-list commands
Create a generic ListCommand to be user for all *-list managementcommands. ListCommand handles must tasks that are common to allmanagement commands, like retrieving objects from db, filteringresults and formating output to a pretty table, json or csv....
Merge branch 'release-0.13' into develop
Conflicts: version
Bypass commissioning for orphan networks
Cyclades public networks do not have an owner, and since can not usethe commissioning system. Also, make snf-dispatcher instead of failingfor networks without userid to just a log a warning. This does not apply for...
Conflicts: snf-cyclades-app/synnefo/api/management/commands/cyclades-export-quota.py snf-cyclades-app/synnefo/api/management/commands/network-list.py snf-cyclades-app/synnefo/api/management/commands/server-inspect.py...
Improve server-list usability
Refs #3375
Improve perf for snf-manage cyclades commands
When the snf-manage {server, network}-list commands are invoked with the--displayname option, batch the UUID-to-displayname requests to astakos,instead of issuing a request for each (unique) UUID.
reconciliation: handle missing network from db
Continue reconciliation even if an orphan network is missing from DB.
Fix typo
pep8 convension
pep8 convension to cyclades management commands
Add --uuids option to snf-manage cmds
Totally remove password from create_instance
Never pass password as an 'osparam' to Ganeti. Always use the vmapi,even for VMs created with server-create management command.
Fix reconciliation for building VMs
Since commit 482c645, a VM exists in the Cyclades DB before thecorresponding job is enqueued in the Ganeti backend. Therefore, there isa corner case, where Cyclades can not enqueue the job to Ganeti andCyclades crashes before deleting the VM from the DB, which results in a...
Remove mac2eui64 dependency
Import mac2eui64 code which is already shipped under util/mac2eui64 insteadof shipping it as a seperate script.
Fix pep8 issues
Fix pep8 warnings
Fix bug in network reconciliation
Fix bug in network reconciliation caused by wrong variable scope.
pep8 fixes
Rename ARCHIPELAGOS to ARCHIPELAGO
Rename ARCHIPELAGOS_BACKENDS setting to ARCHIPELAGO_BACKEND.
Fix progress messages handling
proper handling of missing `messages` key progress messages.
Network field type has been renamed to flavor
Fix typo bug in reconcile ip pools