cyclades: Import settings from django.conf
Fix stale import statements
Commit 7cfc0ce moved code from webproject to snf_django. This commitfixes some stale imports that skipped renaming.
snf-django: Move ListCommand to snf-django
snf-webproject contained the generic ListCommand, and because of this ithad a dependency from snf-django-lib. This commit moves this command fromsnf-webproject to snf-django-lib and removes this dependency. Also,...
cyclades: Reconciliation for pending vm tasks
Extend reconciliation mechanism, to detect instances with stale pendingtasks, by looking if the corresponding job, 'vm.task_job_id', is stillrunning in the Ganeti backend.
Also refactor code that performed reconciliation for buildign vms, to...
cyclades: Minor improvements in manage commands
cyclades: Fix import of renamed settings
cyclades: Implement NetworkReconciler class
Refactory network reconciliation. Implement NetworkReconciler class tohold all reconciliation functions.
cyclades: Move NetworkReconciler to reconciliation
Move NetworkReconciler class from network-create command toreconciliation module.
cyclades: Support IPv6 only networks
Make Cyclades support IPv6 only networks. Such networks will havethe 'subnet' attribute set to None, in the same way that IPv4 onlynetworks have the 'subnet6' attribute set to None.
Because currently Ganeti does not support IPv6 only networks, we create the...
cyclades: Parallel server reconciliation
Major refactor to reconcile-servers management command to make it workin parallel for each backend. Also, removed all --detect options, anddetect all incosistencies by default.
cyclades: Fix typo in reconciliation
reconciliation: Strip Ganeti IP addresses
Strip spaces from Ganeti IP addresses, since they are not automaticallyignored in newer versions of ipaddr.
cyclades: Remove PUBLIC_USE_POOL setting
Remove PUBLIC_USE_POOL setting, since in order to use floating IPs, thecyclades must be responsible for managing all IP pools.
Distinquish between networks and Floating IP pools
Extend Network model with the 'floating_ip_pool' boolean field, toindicate whether a Network can be a floating IP pool or not. Networksthat are floating IP pools must exist in all Backends, and so'snf-manage backend-add' should try to add them....
cyclades: Add 'floating-ip-list' management cmd
cyclades: Update reconcilliation for floating IPs
Update network reconcilliation to consider floating IPs reconcilling IPpools. Floating IPs that are reserved and usused, will appear asasynced. This commit fixes this, by considering unused floating IPs as...
cyclades: refactor reconciliation
Refactor reconciliation code to use namedtuples and improve performancefor retrieving info from database.
cyclades: reconciliation for resizing VMs
cyclades: Fixes relative to reconciliation
Get available backends once and work only with them, since availablebackends may change during reconciliation. Also temporary remove supportfor communicating with each backend in a separate process.
cyclades: Validate MAC prefix in pool-create cmd
cyclades: Rename service token
Rename CYCLADES_ASTAKOS_SERVICE_TOKEN to CYCLADES_SERVICE_TOKEN foruniformity.
make synnefo URLs consistent and configurable
https://service.deployment.vendor.tld/service-base/api-prefix/hard/path |___________________________________| |__________| |________| |_______| BASE_HOST BASE_PATH PREFIX Hardcoded...
cyclades: Add 'drained' flag to Network
Extend Network model with the 'drained' attribute, to indicate whether anetwork should be used in public IP allocation when creating servers.This option is only useful for public networks.
cyclades: Export hypervisor to backend-list cmd
cyclades: Add --hypervisor option to backend cmds
Add '--hypervisor' option to 'backend-add' and 'backend-modify'management commands.
Refs #3772
cyclades: Simplify reconcile-networks checks
Refactor and simplify checks of reconcile-networks to take advantageof lazy networks.
cyclades: Update 'backend-add' for lazy networks
Do not create private networks when adding a new backend. Since,"lazy networks" this is no longer needed.
cyclades: Implement lazy networks
Implement "lazy networks", which are not created to all Ganeti backendsupon creation, but only when an instance that resides in a Ganeti isconnected to that network.
Update 'create_network' to create the Network object in DB without...
Remove transaction from 'backend-add' command
Do not run 'backend-add' command inside transaction, because if'snf-dispatcher' is running, it will receive messages that will arrivefrom a Ganeti backend that does not exist in the DB and will not be ableto process them.
Move astakos lib from snf-common to snf-django-lib
Use pprint_table in server-show command
Move UserCache from cyclades.common to lib.astakos
Move UserCache from synnefo.management.common to synnefo.lib.astakos.Also, update ListCommand with 'astakos_url' and 'astakos_token' classattributes, which must be declared only when 'user_uuid_field' is used.
Move common code for mngmt commands to webproject
Move cyclades functions that are useful for all managament commands, likepprint_table or parse_filters, to snf-webproject.
Add ListCommand to be used for all *-list commands
Create a generic ListCommand to be user for all *-list managementcommands. ListCommand handles must tasks that are common to allmanagement commands, like retrieving objects from db, filteringresults and formating output to a pretty table, json or csv....
Merge branch 'release-0.13' into develop
Conflicts: snf-cyclades-app/synnefo/api/management/commands/cyclades-export-quota.py snf-cyclades-app/synnefo/api/management/commands/network-list.py snf-cyclades-app/synnefo/api/management/commands/server-inspect.py...
Improve server-list usability
Refs #3375
Improve perf for snf-manage cyclades commands
When the snf-manage {server, network}-list commands are invoked with the--displayname option, batch the UUID-to-displayname requests to astakos,instead of issuing a request for each (unique) UUID.
Conflicts: version
reconciliation: handle missing network from db
Continue reconciliation even if an orphan network is missing from DB.
Fix typo
pep8 convension to cyclades management commands
Add --uuids option to snf-manage cmds
Remove mac2eui64 dependency
Import mac2eui64 code which is already shipped under util/mac2eui64 insteadof shipping it as a seperate script.
Fix pep8 issues
Fix pep8 warnings
Fix bug in network reconciliation
Fix bug in network reconciliation caused by wrong variable scope.
Network field type has been renamed to flavor
Fix typo bug in reconcile ip pools
Refactor network reconciliation
Misc minor bug fixes
Do not allow creation of more than one pools
Do not allow creation of more than one Bridge or MacPrefix pool, sincewe can not handle them yet.
Add backend-id option to reconcile-servers
Make reconcile-servers perform reconciliation only for the specifiedbackend.
reconciliation:avoid double asking Ganeti
Modify reconcile-servers to not communicate 2 times with Ganetito get the operstate and the NICs of the VMs. Merge this two onecall.
Modify backend-modify --drained and --offline args
Make backend-modify drained argument as --drained=True|False insteadof --drained and --no-drained. The same for offline.
Mark as drained newly added backends
Rename reconcile command to reconcile-servers
Rename `snf-manage reconcile` management command to `snf-managereconcile-servers`, to be consistent with `snf-managereconcile-networks`. Also, treat '--detect-all' option as defaultwhen no other '--detect-*' is specified.
Introduce network flavors to replace network types
Add tags and mode to network db model. Replace existing type withflavor. Introduce 4 different flavors. Each flavor has mode, link,mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....
Implement quotas-init and quotas-verify
Implement quotas-init and quotas-verify management commands.- quotas-init sets quotas of Quotaholder to the state of quotas in Cyclades DB- quotas-verify detects incosistencies between quotas in Quotaholder and...
Implement reconciliation for pending quotas
Implement resizing of pools
Extend PoolManager to support extending and shrinking of Pools. Alsoadd extra option '--size' to pool-modify command.
Add function for pretty printing tables
Check clustername when modifying backend
Add check for clustername when adding backend
Fix lectical in pool-modify
Refactor code in management commands
Improve management commands
Merge branch 'master' into devel-0.12
Conflicts: snf-cyclades-app/synnefo/api/management/commands/flavor-list.py snf-cyclades-app/synnefo/api/management/commands/server-list.py snf-cyclades-app/synnefo/api/networks.py snf-cyclades-app/synnefo/logic/rapi.py
PUBLIC_ROUTED minor changes
Do not allow api to create public networks and type otherthan PRIVATE_MAC_FILTERED/PRIVATE_PHYSICAL_VLAN.
Rename PUBLIC_ROUTED_USE_POOL setting to PUBLIC_USE_POOL.
Let synnefo do IP pool management only if network is private or...
Display VM build percentage in management commands
Implement queue-retry management command
Implement a new command which consumes all the messages from thedead-letter queues and re-sends the messages back to the snf-dispatcherin order to handle them again.
Messages are annotated with "from-dead-letter" field in order to detect...
Fix bug in reconcile-networks
When only detecting, reconciliation can not continue, if theBackendNetwork object does not exist.
Fix typos
Use - instead of _ in backend-update-status
Fix bug when connecting a network to a nodegroup in networkreconciliation. Use connect_network instead of obsoleteconnect_network_group method. Also fix job dependency handling in thismethod.
Pool Ganeti RAPI clients
- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.- Create GanetiRapiClientPool for pooling GanetiRapiClient objects- Create context manager in db.models for pool handling- Update code using the RAPI client
Do not print public networks when adding backend
Fix white space
Fix bugs in reconcile-networks
Perform logical and instead of or, when merging backend ip maps, sincenow the maps are inverted. Also do not report as stale backend networksthat are already deleted.
Add pool-create and pool-remove commands
Fix presentation of pools
Fix two bugs in network reconciliation
Catch corner case in reconciliation
Catch error produced when the DB entry for an orphan VM in Ganeti doesnot exist. This may happen because either an entry from DB was removedor because a VM in Ganeti with the synnefo prefix was created.
Refactor IP pools
Treat IP pool the same way as bridge and mac-prefix pools. The pool isremoved from the Network table and moved to an IPPoolTable containing theIP pools for all networks. Also added support for external reservations.
Migration files for transfering the pool to the new tables are included....
Handle case of no pools in reconcile-pools
Management commands for pools
Create three new management commands for listing, showing and modifyinga pool.
Unique mac-prefix only for mac-filtered networks
Refactor bridge and mac-prefix pool tables
Redesign the implementation of pools and map mac-prefix and bridge poolsto that design.
Each pool type is mapped to a table in the database, and each row ofthat table is a pool of that type. All pool tables must contain the...
Fix bug in server-inspect
New method for creating BackendNetwork entries
Create a new method to the Network model, in order to create theBackendNetwork entries tha correspond to this network. Also removethe custom save method, since when multiple public network will besupported, we want them to be correlated with specific backends.
Implement multiple public networks
Implement multiple public networks in order to support:- different public networks per backend- non-consecutive IP ranges in each backend
To support this, public networks are treated different from privateones. Namely, they are correlated with one backend (and so have...
Management command for removing backend
Add command for checking pool consistency
Add new management command 'reconcile-pools' which checks the consistency of thepools and the uniqueness of the associated resources. Currently thiscommands detects the inconcistencies, without trying to resolve them.
Fix bug when adding a new backend
Fix wrong reporting messages when adding a new backend.
Fix creation of networks in new backend
Setting for using IP Pool to public network
Introduce new setting 'PUBLIC_ROUTED_USE_POOL', to indicate whethersynnefo will be responsible for the IP management of the public network,or this responsibility will be delegated to Ganeti.
Fix case of empty address pool
Display IP pool map in network-inspect command