cyclades: Optimize *-list commands
Use prefetch_related and select-related to importive the DB queries thatare generated by *-list snf-manage commands.
cyclades: Fix init of networks in backend-add
Fix which networks are going to be initialized in the new backend andhow the are printed in the output!
Cyclades: Rename 'Can not' to 'Cannot' to all msgs
cyclades: Add snf-manage subnet-inspect
cyclades: Fix bug in floating-ip-list
cyclades: Fix stale references to moved function
cyclades: Common module for pretty-printing
Create common module for pretty-printing database objects. Currently itcontains only functions for pretty-printing networks and ports.
cyclades: Fix output of network-inspect command
cyclades: Fix snf-manage floating-ip-list
A minor fix to the snf-manage floating-ip-list command to complyto the new models
cyclades: Fix network inspect management command
cyclades: Add method to get ip stats to network
Extend Network model with a method to get the number of free and totalIPv4 addresses of the network. Also, add method to IPAddress model torelease its IPv4 address if any. Finally, update backend-list management...
cyclades: Make API server tests pass
Refactor Cyclades code so that it follows the new models for Networks,Subnets and IPAddresses. This refactor is not complete. Instead is thenecessary refactor for making tests for server API calls succeed.
cyclades: Replace FloatingIP with IPAddress
This commit just replaces all references to FloatingIP objects withreferences to IPAddress object, in order to be able to just load thecode. But the code is left unchanged, so all tests will fail.
cyclades: Refactor reconciliation of pools
Move reconciliation of IP pools from 'snf-manage reconcile-networks'to 'snf-manage reconcile-pools'. Also change the way reconciliationworks: Instead of reconciling the IP pools in DB with the IP pools inGaneti, the new reconciliation just checks that:...
cyclades: Remove custom ProtectedDeletedManager
Remove custom ProtectedDeletedManager. Instead use theForeignKey.on_delete attribute that has been introduced in Django 1.4.
cyclades: Callback for cluster modifications
Extend snf-dispatcher callbacks with one to run every time anOP_CLUSTER_ opcode is executed in a Ganeti backend.
cyclades: Refactor backend-update-status cmd
The command has been changed to no longer take any arguments or options.Instead, this command always updates the resources and the enabled disktemplates of all online Ganeti backends.
cyclades: Display disk-templates in backend-list
cyclades: Import settings from django.conf
Fix stale import statements
Commit 7cfc0ce moved code from webproject to snf_django. This commitfixes some stale imports that skipped renaming.
snf-django: Move ListCommand to snf-django
snf-webproject contained the generic ListCommand, and because of this ithad a dependency from snf-django-lib. This commit moves this command fromsnf-webproject to snf-django-lib and removes this dependency. Also,...
cyclades: Reconciliation for pending vm tasks
Extend reconciliation mechanism, to detect instances with stale pendingtasks, by looking if the corresponding job, 'vm.task_job_id', is stillrunning in the Ganeti backend.
Also refactor code that performed reconciliation for buildign vms, to...
cyclades: Minor improvements in manage commands
cyclades: Fix import of renamed settings
cyclades: Implement NetworkReconciler class
Refactory network reconciliation. Implement NetworkReconciler class tohold all reconciliation functions.
cyclades: Move NetworkReconciler to reconciliation
Move NetworkReconciler class from network-create command toreconciliation module.
cyclades: Support IPv6 only networks
Make Cyclades support IPv6 only networks. Such networks will havethe 'subnet' attribute set to None, in the same way that IPv4 onlynetworks have the 'subnet6' attribute set to None.
Because currently Ganeti does not support IPv6 only networks, we create the...
cyclades: Parallel server reconciliation
Major refactor to reconcile-servers management command to make it workin parallel for each backend. Also, removed all --detect options, anddetect all incosistencies by default.
cyclades: Fix typo in reconciliation
reconciliation: Strip Ganeti IP addresses
Strip spaces from Ganeti IP addresses, since they are not automaticallyignored in newer versions of ipaddr.
cyclades: Remove PUBLIC_USE_POOL setting
Remove PUBLIC_USE_POOL setting, since in order to use floating IPs, thecyclades must be responsible for managing all IP pools.
Distinquish between networks and Floating IP pools
Extend Network model with the 'floating_ip_pool' boolean field, toindicate whether a Network can be a floating IP pool or not. Networksthat are floating IP pools must exist in all Backends, and so'snf-manage backend-add' should try to add them....
cyclades: Add 'floating-ip-list' management cmd
cyclades: Update reconcilliation for floating IPs
Update network reconcilliation to consider floating IPs reconcilling IPpools. Floating IPs that are reserved and usused, will appear asasynced. This commit fixes this, by considering unused floating IPs as...
cyclades: refactor reconciliation
Refactor reconciliation code to use namedtuples and improve performancefor retrieving info from database.
cyclades: reconciliation for resizing VMs
cyclades: Fixes relative to reconciliation
Get available backends once and work only with them, since availablebackends may change during reconciliation. Also temporary remove supportfor communicating with each backend in a separate process.
cyclades: Validate MAC prefix in pool-create cmd
cyclades: Rename service token
Rename CYCLADES_ASTAKOS_SERVICE_TOKEN to CYCLADES_SERVICE_TOKEN foruniformity.
make synnefo URLs consistent and configurable
https://service.deployment.vendor.tld/service-base/api-prefix/hard/path |___________________________________| |__________| |________| |_______| BASE_HOST BASE_PATH PREFIX Hardcoded...
cyclades: Add 'drained' flag to Network
Extend Network model with the 'drained' attribute, to indicate whether anetwork should be used in public IP allocation when creating servers.This option is only useful for public networks.
cyclades: Export hypervisor to backend-list cmd
cyclades: Add --hypervisor option to backend cmds
Add '--hypervisor' option to 'backend-add' and 'backend-modify'management commands.
Refs #3772
cyclades: Simplify reconcile-networks checks
Refactor and simplify checks of reconcile-networks to take advantageof lazy networks.
cyclades: Update 'backend-add' for lazy networks
Do not create private networks when adding a new backend. Since,"lazy networks" this is no longer needed.
cyclades: Implement lazy networks
Implement "lazy networks", which are not created to all Ganeti backendsupon creation, but only when an instance that resides in a Ganeti isconnected to that network.
Update 'create_network' to create the Network object in DB without...
Remove transaction from 'backend-add' command
Do not run 'backend-add' command inside transaction, because if'snf-dispatcher' is running, it will receive messages that will arrivefrom a Ganeti backend that does not exist in the DB and will not be ableto process them.
Move astakos lib from snf-common to snf-django-lib
Use pprint_table in server-show command
Move UserCache from cyclades.common to lib.astakos
Move UserCache from synnefo.management.common to synnefo.lib.astakos.Also, update ListCommand with 'astakos_url' and 'astakos_token' classattributes, which must be declared only when 'user_uuid_field' is used.
Move common code for mngmt commands to webproject
Move cyclades functions that are useful for all managament commands, likepprint_table or parse_filters, to snf-webproject.
Add ListCommand to be used for all *-list commands
Create a generic ListCommand to be user for all *-list managementcommands. ListCommand handles must tasks that are common to allmanagement commands, like retrieving objects from db, filteringresults and formating output to a pretty table, json or csv....
Merge branch 'release-0.13' into develop
Conflicts: snf-cyclades-app/synnefo/api/management/commands/cyclades-export-quota.py snf-cyclades-app/synnefo/api/management/commands/network-list.py snf-cyclades-app/synnefo/api/management/commands/server-inspect.py...
Improve server-list usability
Refs #3375
Improve perf for snf-manage cyclades commands
When the snf-manage {server, network}-list commands are invoked with the--displayname option, batch the UUID-to-displayname requests to astakos,instead of issuing a request for each (unique) UUID.
Conflicts: version
reconciliation: handle missing network from db
Continue reconciliation even if an orphan network is missing from DB.
Fix typo
pep8 convension to cyclades management commands
Add --uuids option to snf-manage cmds
Remove mac2eui64 dependency
Import mac2eui64 code which is already shipped under util/mac2eui64 insteadof shipping it as a seperate script.
Fix pep8 issues
Fix pep8 warnings
Fix bug in network reconciliation
Fix bug in network reconciliation caused by wrong variable scope.
Network field type has been renamed to flavor
Fix typo bug in reconcile ip pools
Refactor network reconciliation
Misc minor bug fixes
Do not allow creation of more than one pools
Do not allow creation of more than one Bridge or MacPrefix pool, sincewe can not handle them yet.
Add backend-id option to reconcile-servers
Make reconcile-servers perform reconciliation only for the specifiedbackend.
reconciliation:avoid double asking Ganeti
Modify reconcile-servers to not communicate 2 times with Ganetito get the operstate and the NICs of the VMs. Merge this two onecall.
Modify backend-modify --drained and --offline args
Make backend-modify drained argument as --drained=True|False insteadof --drained and --no-drained. The same for offline.
Mark as drained newly added backends
Rename reconcile command to reconcile-servers
Rename `snf-manage reconcile` management command to `snf-managereconcile-servers`, to be consistent with `snf-managereconcile-networks`. Also, treat '--detect-all' option as defaultwhen no other '--detect-*' is specified.
Introduce network flavors to replace network types
Add tags and mode to network db model. Replace existing type withflavor. Introduce 4 different flavors. Each flavor has mode, link,mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....
Implement quotas-init and quotas-verify
Implement quotas-init and quotas-verify management commands.- quotas-init sets quotas of Quotaholder to the state of quotas in Cyclades DB- quotas-verify detects incosistencies between quotas in Quotaholder and...
Implement reconciliation for pending quotas
Implement resizing of pools
Extend PoolManager to support extending and shrinking of Pools. Alsoadd extra option '--size' to pool-modify command.
Add function for pretty printing tables
Check clustername when modifying backend
Add check for clustername when adding backend
Fix lectical in pool-modify
Refactor code in management commands
Improve management commands
Merge branch 'master' into devel-0.12
Conflicts: snf-cyclades-app/synnefo/api/management/commands/flavor-list.py snf-cyclades-app/synnefo/api/management/commands/server-list.py snf-cyclades-app/synnefo/api/networks.py snf-cyclades-app/synnefo/logic/rapi.py
PUBLIC_ROUTED minor changes
Do not allow api to create public networks and type otherthan PRIVATE_MAC_FILTERED/PRIVATE_PHYSICAL_VLAN.
Rename PUBLIC_ROUTED_USE_POOL setting to PUBLIC_USE_POOL.
Let synnefo do IP pool management only if network is private or...
Display VM build percentage in management commands
Implement queue-retry management command
Implement a new command which consumes all the messages from thedead-letter queues and re-sends the messages back to the snf-dispatcherin order to handle them again.
Messages are annotated with "from-dead-letter" field in order to detect...
Fix bug in reconcile-networks
When only detecting, reconciliation can not continue, if theBackendNetwork object does not exist.
Fix typos
Use - instead of _ in backend-update-status
Fix bug when connecting a network to a nodegroup in networkreconciliation. Use connect_network instead of obsoleteconnect_network_group method. Also fix job dependency handling in thismethod.
Pool Ganeti RAPI clients
- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.- Create GanetiRapiClientPool for pooling GanetiRapiClient objects- Create context manager in db.models for pool handling- Update code using the RAPI client
Do not print public networks when adding backend
Fix white space
Fix bugs in reconcile-networks
Perform logical and instead of or, when merging backend ip maps, sincenow the maps are inverted. Also do not report as stale backend networksthat are already deleted.
Catch corner case in reconciliation
Catch error produced when the DB entry for an orphan VM in Ganeti doesnot exist. This may happen because either an entry from DB was removedor because a VM in Ganeti with the synnefo prefix was created.