cyclades: Rename service token
Rename CYCLADES_ASTAKOS_SERVICE_TOKEN to CYCLADES_SERVICE_TOKEN foruniformity.
make synnefo URLs consistent and configurable
https://service.deployment.vendor.tld/service-base/api-prefix/hard/path |___________________________________| |__________| |________| |_______| BASE_HOST BASE_PATH PREFIX Hardcoded...
cyclades: Add 'drained' flag to Network
Extend Network model with the 'drained' attribute, to indicate whether anetwork should be used in public IP allocation when creating servers.This option is only useful for public networks.
cyclades: Export hypervisor to backend-list cmd
cyclades: Add --hypervisor option to backend cmds
Add '--hypervisor' option to 'backend-add' and 'backend-modify'management commands.
Refs #3772
cyclades: Sypport backends with Xen hypervisor
cyclades: Various fixes to tests
dispatcher: Handle NICs in callbacks.
Update 'update_db' callback to handle instance NICs together withinstance status.
Also update the relevant tests.
cyclades: simplify comparison of NICs
Remove 'snf-events-net' queue and binding
Remove 'snf-events-net' queue (together with the corresponding routingkey and binding) from the queues that snf-dispatcher is consumingmessages. This queue is no longer needed, because messages for instanceNICs will be sended by snf-ganeti-eventd to 'snf-events-op' queue....
cyclades: Fix deadlock when connecting vms
Fix deadlock caused by 'get_or_create' method, since it acquires animplicit shared lock. Take an exclusive lock on network to fix thisdeadlock.
cyclades: Fixes relevant to lazy networks
cyclades: Simplify reconcile-networks checks
Refactor and simplify checks of reconcile-networks to take advantageof lazy networks.
cyclades: update tests for lazy networks
cyclades: Update 'backend-add' for lazy networks
Do not create private networks when adding a new backend. Since,"lazy networks" this is no longer needed.
cyclades: Implement lazy networks
Implement "lazy networks", which are not created to all Ganeti backendsupon creation, but only when an instance that resides in a Ganeti isconnected to that network.
Update 'create_network' to create the Network object in DB without...
cyclades: refactor code relative to commission
Refactor Cyclades code that handles commissioning with Quotaholder. Thiscommit removes the 'uses_commission' decorator. Instead it introducesthe 'issue_and_accept_commission' function that is responsible for...
Mock quotaholder for Cyclades tests
Create 'mocked_quotaholder' context manager to bypass commission and useit in Cyclades tests. Also this commit does some various fixes to othertests.
Remove transaction from 'backend-add' command
Do not run 'backend-add' command inside transaction, because if'snf-dispatcher' is running, it will receive messages that will arrivefrom a Ganeti backend that does not exist in the DB and will not be ableto process them.
Move astakos lib from snf-common to snf-django-lib
Use 'python-objpool' instead of 'synnefo.lib.pool'
Use the python-objpool package for pooling resources as object, insteadof the pooling code that existed in snf-common ('synnefo.lib.pool')
Refs #3524
Use pprint_table in server-show command
Create public NIC when creating VM
Until now, a server's public NetworkInterface was not created in the DBuntil the arrival of the message from Ganeti hooks. However, the IPv4address of the NIC is reserved before sending the OP_INSTANCE_CREATEjob to Ganeti....
Move UserCache from cyclades.common to lib.astakos
Move UserCache from synnefo.management.common to synnefo.lib.astakos.Also, update ListCommand with 'astakos_url' and 'astakos_token' classattributes, which must be declared only when 'user_uuid_field' is used.
Move common code for mngmt commands to webproject
Move cyclades functions that are useful for all managament commands, likepprint_table or parse_filters, to snf-webproject.
Add ListCommand to be used for all *-list commands
Create a generic ListCommand to be user for all *-list managementcommands. ListCommand handles must tasks that are common to allmanagement commands, like retrieving objects from db, filteringresults and formating output to a pretty table, json or csv....
Merge branch 'release-0.13' into develop
Conflicts: version
Bypass commissioning for orphan networks
Cyclades public networks do not have an owner, and since can not usethe commissioning system. Also, make snf-dispatcher instead of failingfor networks without userid to just a log a warning. This does not apply for...
Conflicts: snf-cyclades-app/synnefo/api/management/commands/cyclades-export-quota.py snf-cyclades-app/synnefo/api/management/commands/network-list.py snf-cyclades-app/synnefo/api/management/commands/server-inspect.py...
Improve server-list usability
Refs #3375
Improve perf for snf-manage cyclades commands
When the snf-manage {server, network}-list commands are invoked with the--displayname option, batch the UUID-to-displayname requests to astakos,instead of issuing a request for each (unique) UUID.
reconciliation: handle missing network from db
Continue reconciliation even if an orphan network is missing from DB.
Fix typo
pep8 convension
pep8 convension to cyclades management commands
Add --uuids option to snf-manage cmds
Totally remove password from create_instance
Never pass password as an 'osparam' to Ganeti. Always use the vmapi,even for VMs created with server-create management command.
Fix reconciliation for building VMs
Since commit 482c645, a VM exists in the Cyclades DB before thecorresponding job is enqueued in the Ganeti backend. Therefore, there isa corner case, where Cyclades can not enqueue the job to Ganeti andCyclades crashes before deleting the VM from the DB, which results in a...
Remove mac2eui64 dependency
Import mac2eui64 code which is already shipped under util/mac2eui64 insteadof shipping it as a seperate script.
Fix pep8 issues
Fix pep8 warnings
Fix bug in network reconciliation
Fix bug in network reconciliation caused by wrong variable scope.
pep8 fixes
Rename ARCHIPELAGOS to ARCHIPELAGO
Rename ARCHIPELAGOS_BACKENDS setting to ARCHIPELAGO_BACKEND.
Fix progress messages handling
proper handling of missing `messages` key progress messages.
Network field type has been renamed to flavor
Fix typo bug in reconcile ip pools
Add warning message for backend without networks
Refactor network reconciliation
Remove unused imports from tests
Merge branch 'master' into devel-0.13
Conflicts: snf-cyclades-app/synnefo/db/models.py snf-cyclades-app/synnefo/logic/management/commands/backend-modify.py
Refactor tests
Fix bug about networks and offline backends
Ignore offline backends when computing the state of the network, sinceno notification will be received from them. Also, create noBackendNetwork for an offline backend. This will be created byreconcilliation, when the backend gets online again.
Add more tests for db, logic and api
Remove dummy function
Add more tests for db and logic
Refactor some test code
Misc minor bug fixes
Fix current db, logic and api tests
Fix variable scope bug in process_net_status
Do not allow creation of more than one pools
Do not allow creation of more than one Bridge or MacPrefix pool, sincewe can not handle them yet.
Add backend-id option to reconcile-servers
Make reconcile-servers perform reconciliation only for the specifiedbackend.
reconciliation:avoid double asking Ganeti
Modify reconcile-servers to not communicate 2 times with Ganetito get the operstate and the NICs of the VMs. Merge this two onecall.
Modify backend-modify --drained and --offline args
Make backend-modify drained argument as --drained=True|False insteadof --drained and --no-drained. The same for offline.
Mark as drained newly added backends
Rename reconcile command to reconcile-servers
Rename `snf-manage reconcile` management command to `snf-managereconcile-servers`, to be consistent with `snf-managereconcile-networks`. Also, treat '--detect-all' option as defaultwhen no other '--detect-*' is specified.
Introduce network flavors to replace network types
Add tags and mode to network db model. Replace existing type withflavor. Introduce 4 different flavors. Each flavor has mode, link,mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....
Implement quotas-init and quotas-verify
Implement quotas-init and quotas-verify management commands.- quotas-init sets quotas of Quotaholder to the state of quotas in Cyclades DB- quotas-verify detects incosistencies between quotas in Quotaholder and...
Temporary fix for allocating archipelagos VMs
Fix typos
Merge handling of ext_<provider> disk templates
Merge branch 'origin/devel-0.13' into quota
Conflicts: snf-cyclades-app/synnefo/api/servers.py
Issue commission when deleting VM/Network
Issue commission to the Quotaholder when being notified from the Ganetibackend that a VM/Network has been deleted. Although the resource hasalready been deleting, we are interacting with the quotaholder in atwo step phase: issue commission and accept commission. It would be...
Implement reconciliation for pending quotas
Remove transaction from update_network_state
Remove transaction decorator from update_network_state function, sincethis function is only called from process_network_status, which already runsinside a transaction.
Do not automatically update network state
Each modification in a BackendNetwork object, triggers an updateto the corresponding Network object. Until now this was done bythe BackendNetwork save() method. Now it is explicitly donefrom the backend pross_network_status.
Remove IGNORE_FLAVOR_DISK_SIZES setting
Use vmapi when creating servers
Use the vmapi when creating servers. Creete instance RAPI call takes theconfiguration url instead all the image parametere. The id and formatof the image are still passed, just for readability and debugging.
Also renamed params_url to config-url.
snf-dispatcher: Update NICs only when needed
Do not remove and recreate the NICs of a VM when NICs have not changed.
Implement resizing of pools
Extend PoolManager to support extending and shrinking of Pools. Alsoadd extra option '--size' to pool-modify command.
Add function for pretty printing tables
Make amqp_puka get logger as argument
Check clustername when modifying backend
Add check for clustername when adding backend
Fix lectical in pool-modify
Refactor code in management commands
Improve management commands
Merge branch 'master' into devel-0.12
Conflicts: snf-cyclades-app/synnefo/api/management/commands/flavor-list.py snf-cyclades-app/synnefo/api/management/commands/server-list.py snf-cyclades-app/synnefo/api/networks.py snf-cyclades-app/synnefo/logic/rapi.py
Fix dispatcher logging
Use select_for_update for VirtualMachines
Use select_for_update method when accessing VM objects to preventrace conditions. Also move start_action from logic to api.
Process OP_NETWORK_SET_PARAMS Ganeti jobs
Modify snf-ganeti-eventd and snf-dispatcher to handle jobs withOP_NETWORK_SET_PARAMS opcode. Handling of these jobs is neededto update the IP Pool of network, when reserving and releasingIPs directly from the Ganeti backend.
PUBLIC_ROUTED minor changes
Do not allow api to create public networks and type otherthan PRIVATE_MAC_FILTERED/PRIVATE_PHYSICAL_VLAN.
Rename PUBLIC_ROUTED_USE_POOL setting to PUBLIC_USE_POOL.
Let synnefo do IP pool management only if network is private or...
Add conflicts_check option in network rapi calls
Ganeti by default checks for conflicting IPs when creating/connectingnetworks. This feature can be bypassed for private networks. We donot care for IP uniqueness in case of private networks as long as they...
Update rapi client
Get rapi client from snf-ganeti. Add depends argument (for chain jobs)where needed.
Argument tags must be a list of strings and arguments add_reserved_ips andremove_reserved_ips must be None or a list of strings.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Minor backend fixes in networks
Add conflicts_check option when creating and add network. Check onlyif a network is public.
Let tags argument in rapi calls be a list.
Display VM build percentage in management commands
Allow custom link, mac_prefix in network-create
Add extra command line arguments, to allow the user specify thelink and mac_prefix of the new network if needed.
Introduce per-user allocation of VMs to backends
Add BACKEND_PER_USER setting, a mapping from users to backends. This settingcan by used in order to bypass the default backend allocation process, andallocate all the new VMs of a specific user to a specific Backend.