History | View | Annotate | Download (27 kB)
cyclades: Fix bug in snf-dispatcher logic
This commit reverts commits #07602322 and #30514662 which introduced abug in dispatcher's logic. These commit fixed the issue that, in a casethe Cyclades DB is unsynced with Ganeti, a resource that does not exist...
cyclades: Fix corner case in snf-dispatcher
Same as commit 0514662, but for servers.
Delete network from DB when job fails but the action of network isDESTROY. This case appears when someone tries to remove a network thathas been deleted from Ganeti backend, but not yet from DB.
cyclades: Wrap diagnostic creation in transaction
Fix bug in snf-dispatcher causing wrong updates of VMs from ganeti.Creation if instance diagnostics must run inside a transaction, becauseotherwise, the lock of the vm (select_for_update) is released too early....
Revert "cyclades: Speed up reconciliation"
This reverts commit cf2241c45a22dac71e31dbf32f4700c475804c12.
cyclades: Speed up reconciliation
Speed up reconciliation by using mulitprocessing to get in parallelthe instances from all Ganeti backends.
cyclades: Fix deadlock when creating servers
Fix deadlock between snf-dispatcher and backend allocator, by makingsnf-dispatcher take an exclusive lock to the backend which hosts the vm,before trying to get the exclusive locks to the IP pools in order to...
cyclades: Pass disk origin for all 'ext' flavors
cyclades: Sypport backends with Xen hypervisor
dispatcher: Handle NICs in callbacks.
Update 'update_db' callback to handle instance NICs together withinstance status.
Also update the relevant tests.
cyclades: simplify comparison of NICs
cyclades: Fix deadlock when connecting vms
Fix deadlock caused by 'get_or_create' method, since it acquires animplicit shared lock. Take an exclusive lock on network to fix thisdeadlock.
cyclades: Fixes relevant to lazy networks
cyclades: Implement lazy networks
Implement "lazy networks", which are not created to all Ganeti backendsupon creation, but only when an instance that resides in a Ganeti isconnected to that network.
Update 'create_network' to create the Network object in DB without...
cyclades: refactor code relative to commission
Refactor Cyclades code that handles commissioning with Quotaholder. Thiscommit removes the 'uses_commission' decorator. Instead it introducesthe 'issue_and_accept_commission' function that is responsible for...
Create public NIC when creating VM
Until now, a server's public NetworkInterface was not created in the DBuntil the arrival of the message from Ganeti hooks. However, the IPv4address of the NIC is reserved before sending the OP_INSTANCE_CREATEjob to Ganeti....
Bypass commissioning for orphan networks
Cyclades public networks do not have an owner, and since can not usethe commissioning system. Also, make snf-dispatcher instead of failingfor networks without userid to just a log a warning. This does not apply for...
Totally remove password from create_instance
Never pass password as an 'osparam' to Ganeti. Always use the vmapi,even for VMs created with server-create management command.
Fix pep8 warnings
Merge branch 'master' into devel-0.13
Conflicts: snf-cyclades-app/synnefo/db/models.py snf-cyclades-app/synnefo/logic/management/commands/backend-modify.py
Remove dummy function
Misc minor bug fixes
Fix variable scope bug in process_net_status
Add backend-id option to reconcile-servers
Make reconcile-servers perform reconciliation only for the specifiedbackend.
Introduce network flavors to replace network types
Add tags and mode to network db model. Replace existing type withflavor. Introduce 4 different flavors. Each flavor has mode, link,mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....
Fix typos
Merge handling of ext_<provider> disk templates
Merge branch 'origin/devel-0.13' into quota
Conflicts: snf-cyclades-app/synnefo/api/servers.py
Issue commission when deleting VM/Network
Issue commission to the Quotaholder when being notified from the Ganetibackend that a VM/Network has been deleted. Although the resource hasalready been deleting, we are interacting with the quotaholder in atwo step phase: issue commission and accept commission. It would be...
Remove transaction from update_network_state
Remove transaction decorator from update_network_state function, sincethis function is only called from process_network_status, which already runsinside a transaction.
Do not automatically update network state
Each modification in a BackendNetwork object, triggers an updateto the corresponding Network object. Until now this was done bythe BackendNetwork save() method. Now it is explicitly donefrom the backend pross_network_status.
Remove IGNORE_FLAVOR_DISK_SIZES setting
Use vmapi when creating servers
Use the vmapi when creating servers. Creete instance RAPI call takes theconfiguration url instead all the image parametere. The id and formatof the image are still passed, just for readability and debugging.
Also renamed params_url to config-url.
snf-dispatcher: Update NICs only when needed
Do not remove and recreate the NICs of a VM when NICs have not changed.
Use select_for_update for VirtualMachines
Use select_for_update method when accessing VM objects to preventrace conditions. Also move start_action from logic to api.
Process OP_NETWORK_SET_PARAMS Ganeti jobs
Modify snf-ganeti-eventd and snf-dispatcher to handle jobs withOP_NETWORK_SET_PARAMS opcode. Handling of these jobs is neededto update the IP Pool of network, when reserving and releasingIPs directly from the Ganeti backend.
Minor backend fixes in networks
Add conflicts_check option when creating and add network. Check onlyif a network is public.
Let tags argument in rapi calls be a list.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Do not allow any API action on suspended VMs
Do not allow any modification of any administratively suspended VM through theAPI.
Merge remote-tracking branch 'origin/devel-0.12' into ui-0.11
Merge branch 'devel-0.12' into ui-0.11
Fix bug, UI not updating NICs
Dummy-save the Network when processing messages from Ganeti hooksabout a VM NICs. This is needed, because the UI uses the HTTP changes-sinceswhen requesting VMs and Networks, and combines the result in order to showthe new NICs. Not updating the Network object, resulted in VMs IPv4...
Do not give IPs to networks without dhcp
Fix bug in reconcile-networks
Fix bug when connecting a network to a nodegroup in networkreconciliation. Use connect_network instead of obsoleteconnect_network_group method. Also fix job dependency handling in thismethod.
Refactor process op and network status callbacks
Fix bug when synchronously creating networks
Release address only if a NIC has an IP
Merge branch 'devel-0.11' into ui-0.11
Conflicts: snf-cyclades-app/synnefo/db/models.py snf-cyclades-app/synnefo/logic/backend.py snf-cyclades-app/synnefo/ui/static/snf/js/models.js
Add missing ipv6 arguments to create_network
Fix bug in snf-dispatcher about ignored messages
Update backendtime only for jobs that have been successfully completed,since only those jobs update the state of the VM. Else a "racecondition" may occur when a successful job (e.g. OP_INSTANCE_REMOVE)completes before an error job for the same VM and messages arrive in...
Hide image password from log file
Add extra logging
Pool Ganeti RAPI clients
- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.- Create GanetiRapiClientPool for pooling GanetiRapiClient objects- Create context manager in db.models for pool handling- Update code using the RAPI client
Add support for Archipelagos/extstorage COW images
Perform allocations in separate transactions
Add ForUpdateManager to the Backend model, in order to guaranteeexclusive order when allocating a VM.
Use manual transactions when allocating Backend and public IP for a newVM, since these allocations need exclusive access (SELECT..FOR UPDATE)...
Refactor IP pools
Treat IP pool the same way as bridge and mac-prefix pools. The pool isremoved from the Network table and moved to an IPPoolTable containing theIP pools for all networks. Also added support for external reservations.
Migration files for transfering the pool to the new tables are included....
Refactor bridge and mac-prefix pool tables
Redesign the implementation of pools and map mac-prefix and bridge poolsto that design.
Each pool type is mapped to a table in the database, and each row ofthat table is a pool of that type. All pool tables must contain the...
Enable hotplug during create_instance
This passes hotplug = True to ganeti and enables hotplug per instance.It checks GANETI_USE_HOTPLUG setting.
Implement multiple public networks
Implement multiple public networks in order to support:- different public networks per backend- non-consecutive IP ranges in each backend
To support this, public networks are treated different from privateones. Namely, they are correlated with one backend (and so have...
Revert commented line in logic backend
the line was uncommented by mistake in `7d43565`
Logic backend api helper to create vm diagnostics
Comply with the new monitoring system of snf-image
For now only the image-copy-progress messages are processed
Support for ext disk providers
Extract provider from flavor disk template name and set the additional`provider` parameter in disk options. Flavor disk_template should beformated as `ext_<provider_name>`.
Add setting indicating whether to use hotplug
Implement MAC Prefix Management
In order to guarantee uniqueness of MAC address accross all backends,synnefo assignes a unique mac prefix to each network in each backend.
A mac-prefix of 20 bits, which is picked from the MacPrefixPool, isassigned to each Network. The mac-prefix of the network in each...
Get public network by public attribute
Do not find the public network by the primary key, as the initial_data.jsonis obsolete. Instead, ise the public attribute of the Network Model.Currently only one public Network is supported, and the used can notcreate or delete the public Network. But this can easily change in the...
Fix creation of networks in new backend
Fix bug in process_net_status
Check that nic has an ipv4 before trying to release it.
Setting for using IP Pool to public network
Introduce new setting 'PUBLIC_ROUTED_USE_POOL', to indicate whethersynnefo will be responsible for the IP management of the public network,or this responsibility will be delegated to Ganeti.
Transaction on create_instance
Implement model manager using row-level db locking
Implement ForUpdateManager that uses SELECT .. FOR UPDATE statement inorder to acquire row-level exclusive locks to DB. Essentially thismanager backports the select_for_update() method that is implemented in...
Fix IP pool exceptions
Release IP addresses when nics are destroyed
Fix error state of BackendNetwork
Implement IP management
Delegate IP management to synnefo, in order to be able to allocate IPaddresses accross multiple Ganeti backends. This is achieved by creating aPool of IPs for each Network, in the form of a bitarray, which is storedin the DB as a base64 string....
Update network settings
Add dirty bit to NetworkInterface
Extend NetworkInterface model with a dirty bit, indicating whetherthe index of the NIC is consistent with the one in Ganeti. This flag isset when deleting a NIC, and is switched off after processing thenotification from Ganeti. Removing a NIC with the dirty bit on is not...
Revert "Fix bug in dispatcher callback"
This reverts commit 5d40a128fb593ed641400c462f0e00a1b457f76e.
Remove machine if ganeti job fails
Set the VM deleted flag when receiving a failure notification for aGaneti OP_INSTANCE_REMOVE operation, even if the VM operstate is notERROR. The server may not exist in the backend not only because thecreation failed, but also because a remove notification was lost.
Fix bug in dispatcher callback
During handling of ganeti-op-status messages, state was set to erroronly if the the opcode was OP_INSTANCE_CREATE. This resulted tovm's never being deleted, if they did not exist in Ganeti.
Add backend method for removing a vm's nic
Modification in logic for new-style networks
Synchronize networks when adding a new backend
Implement BackendNetwork model
Create a new model, to represent the state of each network in eachbackend.
Implement backend allocator and management command
New module implementing instance allocation to one of the availablebackends. Allocation is performed based on memory and disk utilization.Statistics for each backend are stored in DB and updated periodically....
Use the Backend client instead of global RAPI
Replace use of global RAPI client, with the cached client from theVirtualMachine or Backend model.
Dispatch network actions to all backends.
New management commands concerning backends
- Create command to create a backend (snf-manage backend_create)- Create command to modify a backend (snf-manage backend_list)- Create command to list backends (snf-manage backend_modify)- Modify listservers command to display the backend that hosts the VM
Rename backend_id to backend_vm_id
Rename backend_id field of VirtualMachine model to avoid conflict between thebackend that hosts the VM and the id that the VM will have on the backend.
Merge branch 'simplequotas' into queues+minor_net
Simple per user quota mechanism for networks/vms
- New dict setting VMS_USER_QUOTA to allow setting per user quota (overrides MAX_VMS_PER_USER setting)- Private networks quota mechanism (based on MAX_NETWORKS_PER_USER and NETWORKS_USER_QUOTA settings)...
Update modules interacting with AMQP
Update all modules which are interacting with the RabbitMQ messaging system touse the AMQPClient in order to use highly available queues in active/activemode.
Extend messages to include a unique identifier in order to handle...
Minor change in rapi call
Ganeti 2.5 needs a tupple of length 3 for nics modifications.So in ModifyInstance rapi call we pass (add, -1, {..}) for nis.
Take advantage of ganeti hotplug
No race condition. No need for ther NetworkInterface to have anoperstate. Hook runs during instance-modify and depicts the currentlyexisting nics. No reboot needed modification to take place.
Later in mutliple backend mac prefix handling should be done by synnefo....
Delete VM nics on update path
Delete VM nics when a message of type OP_INSTANCE_REMOVE has been receivedfrom ganeti, and not when doing the RAPI call.
Simplify disconnecting vm from network
Use the new option of Ganeti, to remove a specific nic.
Use 'network' instead of 'link'
- Rename GANETI_PUBLIC_LINK to GANETI_PUBLIC_NETWORK- modify create_instance of backend module
Switch to the logging middleware
Merge remote-tracking branch 'origin/master' into astakos-integration
Conflicts: runtests.sh snf-cyclades-app/synnefo/aai/__init__.py snf-cyclades-app/synnefo/aai/fixtures/auth_test_data.json snf-cyclades-app/synnefo/aai/middleware.py snf-cyclades-app/synnefo/aai/models.py...
Renamed snf-app to snf-cyclades-app