Fix typos
Use - instead of _ in backend-update-status
Fix bug in reconcile-networks
Fix bug when connecting a network to a nodegroup in networkreconciliation. Use connect_network instead of obsoleteconnect_network_group method. Also fix job dependency handling in thismethod.
Refactor process op and network status callbacks
Various fixes about logging
Fix logging from snf-manage commands. Modify snf-manage implementationin order to setup the loggers for all snf-manage commands. Configurationis based on SNF_MANAGE_LOGGING_SETUP setting from snf-common or onbasicConfig if that setting is missing....
Fix bug when synchronously creating networks
Release address only if a NIC has an IP
Give more weight to cpu_ratio
Change how scores are computed in default_allocator so that vcpuratio has more importance the free memory and disk ratio
Change eventd and dispatcher AMQP retries
Set unlimited connection retries to AMQP hosts for snf-ganeti-eventd,since death of snf-ganeti-eventd results in loss of events.Also increase retries of snf-dispatcher.
Fix bug about RAPI client pooling
Modify the pooled_rapi_client context manager to handle exceptionsand release the client in any case. Also modify uses of reconciliationto handle exceptions inside the scope of the context manager
Add missing ipv6 arguments to create_network
Fix bug when hidding image password
Use deepcopy instead of copy, since the dictionaryis nested
Fix bug in snf-dispatcher about ignored messages
Update backendtime only for jobs that have been successfully completed,since only those jobs update the state of the VM. Else a "racecondition" may occur when a successful job (e.g. OP_INSTANCE_REMOVE)completes before an error job for the same VM and messages arrive in...
Hide image password from log file
Garbage collect GanetiRapiClient pools
Modify the rapi_pool to delete unused GanetiRapiClient pools. Thisclients become unused when the credentials of a backend change.
Add extra logging
Refresh backends in locked mode
Pool Ganeti RAPI clients
- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.- Create GanetiRapiClientPool for pooling GanetiRapiClient objects- Create context manager in db.models for pool handling- Update code using the RAPI client
Do not print public networks when adding backend
Fix white space
Add support for Archipelagos/extstorage COW images
Move check for free IP to backend_allocator
Filter out the backends that do no have a free IP in BackendAllocatorinstead of the default_allocator, in order to remove imports fromsynnefo in the allocation strategy.
Fix bugs in reconcile-networks
Perform logical and instead of or, when merging backend ip maps, sincenow the maps are inverted. Also do not report as stale backend networksthat are already deleted.
Take care for messages without timestamp
Catch exception produced in snf-dispatcher when receiving messages withoutproper timestamp.
Perform allocations in separate transactions
Add ForUpdateManager to the Backend model, in order to guaranteeexclusive order when allocating a VM.
Use manual transactions when allocating Backend and public IP for a newVM, since these allocations need exclusive access (SELECT..FOR UPDATE)...
Fix typo
Merge branch 'public_networks' into devel-0.11
Conflicts: snf-cyclades-app/synnefo/logic/dispatcher.py
Close DB conn in dispatcher before every message
Have the dispatcher close the Django DB connection before processingan incoming message. This allows the actual, underlying DBAPI connectionto return to the DB connection pool, if used, and enables the dispatcher...
Make snf-dispatcher work with gevent
Remove workers from snf-dispatcher in order to make it compatible with geventand specifically the green, monkey patched version of psycopg2(synnefo.lib.db.psyco_gevent). Also after daemonizing, gevent must bereinitialized because the deamon context forks.
Add pool-create and pool-remove commands
Change process title to simplify 'ps' display
Use python-setproctitle to change the process title of the runningdaemon. Make it appear as a native executable to simplifyadministration. Only set executable name, because cannot set individualvalues for arguments.
Fix presentation of pools
Fix two bugs in network reconciliation
Fix race conditions when updating network objects
Use for_update() object manager in order to exclude race conditionswhen updating network objects.
Fix double logging of dispatcher
Dispatcher had as default handlers the console and the file handler.However, when daemonizing, we redirect the stdout and stderr to the filehandler, which resulted in the log messages to be printed twice. So weremove the console handler from the dict configuration, and we only add...
Catch corner case in reconciliation
Catch error produced when the DB entry for an orphan VM in Ganeti doesnot exist. This may happen because either an entry from DB was removedor because a VM in Ganeti with the synnefo prefix was created.
Add some logging to backend allocator
Refactor IP pools
Treat IP pool the same way as bridge and mac-prefix pools. The pool isremoved from the Network table and moved to an IPPoolTable containing theIP pools for all networks. Also added support for external reservations.
Migration files for transfering the pool to the new tables are included....
Handle case of no pools in reconcile-pools
Management commands for pools
Create three new management commands for listing, showing and modifyinga pool.
Unique mac-prefix only for mac-filtered networks
Refactor bridge and mac-prefix pool tables
Redesign the implementation of pools and map mac-prefix and bridge poolsto that design.
Each pool type is mapped to a table in the database, and each row ofthat table is a pool of that type. All pool tables must contain the...
Fix bug in server-inspect
Enable hotplug during create_instance
This passes hotplug = True to ganeti and enables hotplug per instance.It checks GANETI_USE_HOTPLUG setting.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
New method for creating BackendNetwork entries
Create a new method to the Network model, in order to create theBackendNetwork entries tha correspond to this network. Also removethe custom save method, since when multiple public network will besupported, we want them to be correlated with specific backends.
Remove dead code
Implement multiple public networks
Implement multiple public networks in order to support:- different public networks per backend- non-consecutive IP ranges in each backend
To support this, public networks are treated different from privateones. Namely, they are correlated with one backend (and so have...
Reserve gateway's IP when creating an IP pool
Support for ext disk providers
Extract provider from flavor disk template name and set the additional`provider` parameter in disk options. Flavor disk_template should beformated as `ext_<provider_name>`.
Add setting indicating whether to use hotplug
Management command for removing backend
Add command for checking pool consistency
Add new management command 'reconcile-pools' which checks the consistency of thepools and the uniqueness of the associated resources. Currently thiscommands detects the inconcistencies, without trying to resolve them.
Fix serious bug in reconciliation
Do not perform reconciliation for VMs that are hosted in Backendsthat are currently offline, since it is impossible to get thestate of these VMs from the backend. Before this patch, thereconciliation mechanism would report these VMs as stale (and with...
Implement MAC Prefix Management
In order to guarantee uniqueness of MAC address accross all backends,synnefo assignes a unique mac prefix to each network in each backend.
A mac-prefix of 20 bits, which is picked from the MacPrefixPool, isassigned to each Network. The mac-prefix of the network in each...
Fix bug when adding a new backend
Fix wrong reporting messages when adding a new backend.
Get public network by public attribute
Do not find the public network by the primary key, as the initial_data.jsonis obsolete. Instead, ise the public attribute of the Network Model.Currently only one public Network is supported, and the used can notcreate or delete the public Network. But this can easily change in the...
Fix creation of networks in new backend
Fix bug in process_net_status
Check that nic has an ipv4 before trying to release it.
Setting for using IP Pool to public network
Introduce new setting 'PUBLIC_ROUTED_USE_POOL', to indicate whethersynnefo will be responsible for the IP management of the public network,or this responsibility will be delegated to Ganeti.
Consider cpu ratio when allocating vm's
Fix case of empty address pool
Transaction on create_instance
Reconciliation for network IP pool management
Reconcile the contents of each networks IP pool by comparing itwith combination of IP pools of all ganeti backends.
Add extra option for detecting conflicting IPs.
Implement model manager using row-level db locking
Implement ForUpdateManager that uses SELECT .. FOR UPDATE statement inorder to acquire row-level exclusive locks to DB. Essentially thismanager backports the select_for_update() method that is implemented in...
Fix IP pool exceptions
Display IP pool map in network-inspect command
Release IP addresses when nics are destroyed
Add support for IPv6
- Extend the Network model with fields for holding the IPv6 subnet and gateway.- Update network API methods.- Modify ganeti hooks to return IPv6 address for all network interfaces. IPv6 is now derived from the IPv6 subnet of the network in Ganeti,...
Fix error state of BackendNetwork
Implement IP management
Delegate IP management to synnefo, in order to be able to allocate IPaddresses accross multiple Ganeti backends. This is achieved by creating aPool of IPs for each Network, in the form of a bitarray, which is storedin the DB as a base64 string....
Update network settings
Rename management commands
Add dirty bit to NetworkInterface
Extend NetworkInterface model with a dirty bit, indicating whetherthe index of the NIC is consistent with the one in Ganeti. This flag isset when deleting a NIC, and is switched off after processing thenotification from Ganeti. Removing a NIC with the dirty bit on is not...
Revert "Fix bug in dispatcher callback"
This reverts commit 5d40a128fb593ed641400c462f0e00a1b457f76e.
Remove machine if ganeti job fails
Set the VM deleted flag when receiving a failure notification for aGaneti OP_INSTANCE_REMOVE operation, even if the VM operstate is notERROR. The server may not exist in the backend not only because thecreation failed, but also because a remove notification was lost.
Implement network reconciliation
New management commands to inspect network/server
Common functions for instance/network backend name
Fix use of network name prefix
Fix bug about unknown network in instance's nics
Fix management commands
- Fix bug in backend_add- Display backend_name in network_inspect command
Bug fixes concerning networks
Fix bug in dispatcher callback
During handling of ganeti-op-status messages, state was set to erroronly if the the opcode was OP_INSTANCE_CREATE. This resulted tovm's never being deleted, if they did not exist in Ganeti.
Add backend method for removing a vm's nic
Modification in logic for new-style networks
Synchronize networks when adding a new backend
Rename backend_create to backend_add
Implement BackendNetwork model
Create a new model, to represent the state of each network in eachbackend.
Create network queues and refactor dispatcher
Implement backend allocator and management command
New module implementing instance allocation to one of the availablebackends. Allocation is performed based on memory and disk utilization.Statistics for each backend are stored in DB and updated periodically....
Fix issues concerning reconciliation
Use the Backend client instead of global RAPI
Replace use of global RAPI client, with the cached client from theVirtualMachine or Backend model.
Dispatch network actions to all backends.
New management commands concerning backends
- Create command to create a backend (snf-manage backend_create)- Create command to modify a backend (snf-manage backend_list)- Create command to list backends (snf-manage backend_modify)- Modify listservers command to display the backend that hosts the VM
Rename backend_id to backend_vm_id
Rename backend_id field of VirtualMachine model to avoid conflict between thebackend that hosts the VM and the id that the VM will have on the backend.
Remove stale pid lockfiles on daemon invocation
Rename management command
Merge branch 'simplequotas' into queues+minor_net