Remove IGNORE_FLAVOR_DISK_SIZES setting
Use vmapi when creating servers
Use the vmapi when creating servers. Creete instance RAPI call takes theconfiguration url instead all the image parametere. The id and formatof the image are still passed, just for readability and debugging.
Also renamed params_url to config-url.
snf-dispatcher: Update NICs only when needed
Do not remove and recreate the NICs of a VM when NICs have not changed.
Implement resizing of pools
Extend PoolManager to support extending and shrinking of Pools. Alsoadd extra option '--size' to pool-modify command.
Add function for pretty printing tables
Make amqp_puka get logger as argument
Check clustername when modifying backend
Add check for clustername when adding backend
Fix lectical in pool-modify
Fix typo
Refactor code in management commands
Improve management commands
Merge branch 'master' into devel-0.12
Conflicts: snf-cyclades-app/synnefo/api/management/commands/flavor-list.py snf-cyclades-app/synnefo/api/management/commands/server-list.py snf-cyclades-app/synnefo/api/networks.py snf-cyclades-app/synnefo/logic/rapi.py
Fix dispatcher logging
Use select_for_update for VirtualMachines
Use select_for_update method when accessing VM objects to preventrace conditions. Also move start_action from logic to api.
Process OP_NETWORK_SET_PARAMS Ganeti jobs
Modify snf-ganeti-eventd and snf-dispatcher to handle jobs withOP_NETWORK_SET_PARAMS opcode. Handling of these jobs is neededto update the IP Pool of network, when reserving and releasingIPs directly from the Ganeti backend.
PUBLIC_ROUTED minor changes
Do not allow api to create public networks and type otherthan PRIVATE_MAC_FILTERED/PRIVATE_PHYSICAL_VLAN.
Rename PUBLIC_ROUTED_USE_POOL setting to PUBLIC_USE_POOL.
Let synnefo do IP pool management only if network is private or...
Add conflicts_check option in network rapi calls
Ganeti by default checks for conflicting IPs when creating/connectingnetworks. This feature can be bypassed for private networks. We donot care for IP uniqueness in case of private networks as long as they...
Update rapi client
Get rapi client from snf-ganeti. Add depends argument (for chain jobs)where needed.
Argument tags must be a list of strings and arguments add_reserved_ips andremove_reserved_ips must be None or a list of strings.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Minor backend fixes in networks
Add conflicts_check option when creating and add network. Check onlyif a network is public.
Let tags argument in rapi calls be a list.
Display VM build percentage in management commands
Allow custom link, mac_prefix in network-create
Add extra command line arguments, to allow the user specify thelink and mac_prefix of the new network if needed.
Introduce per-user allocation of VMs to backends
Add BACKEND_PER_USER setting, a mapping from users to backends. This settingcan by used in order to bypass the default backend allocation process, andallocate all the new VMs of a specific user to a specific Backend.
Do not allow any API action on suspended VMs
Do not allow any modification of any administratively suspended VM through theAPI.
Merge branch 'ui-0.11' into devel-0.12
Merge remote-tracking branch 'origin/devel-0.12' into ui-0.11
Conflicts: snf-cyclades-app/setup.py
Merge branch 'devel-0.12' into ui-0.11
Implement queue-retry management command
Implement a new command which consumes all the messages from thedead-letter queues and re-sends the messages back to the snf-dispatcherin order to handle them again.
Messages are annotated with "from-dead-letter" field in order to detect...
Fix bug, UI not updating NICs
Dummy-save the Network when processing messages from Ganeti hooksabout a VM NICs. This is needed, because the UI uses the HTTP changes-sinceswhen requesting VMs and Networks, and combines the result in order to showthe new NICs. Not updating the Network object, resulted in VMs IPv4...
Declare dead-letter exchanges and queues
Modify snf-dispatcher in order to declare the dead-letter exchanges andqueues
Use basic_reject in snf-dispatcher callbacks
Modify snf-dispatcher callbacks in order to reject messages that causedan unknown exception. Because a dead-letter-exchange is associatedwith the queues that snf-dispatcher is using, these messages will betransfered to the dead-letter-exchange.
Add mac2eui64 to snf-cyclades-app
Copy mac2eui64 from snf-network to snf-cyclades-app, since it isnow used in the reconciliation process to caclculate the IPv6address of the VMs.
Reconnect to different Broker if idle for too long
Modify snf-dispatcher and add a timeout option to basic_wait method. Iftimeout is reached, the connection to the current RabbitMQ broker isclosed and dispatcher tries to connect to a different one.
This commit is supplementary of 6d27ead, which detects connection...
Detect and handle connection problems in amqp_puka
Use the TCP keepalive feature in amqp_puka in order to detect connectionerrors to the RabbitMQ broker, and retry to a different one. Also setupthe keepalive parameters to relative small values, to detect failures...
Fix bug in reconcile-networks
When only detecting, reconciliation can not continue, if theBackendNetwork object does not exist.
Do not give IPs to networks without dhcp
Fix typos
Use - instead of _ in backend-update-status
Fix bug when connecting a network to a nodegroup in networkreconciliation. Use connect_network instead of obsoleteconnect_network_group method. Also fix job dependency handling in thismethod.
Refactor process op and network status callbacks
Various fixes about logging
Fix logging from snf-manage commands. Modify snf-manage implementationin order to setup the loggers for all snf-manage commands. Configurationis based on SNF_MANAGE_LOGGING_SETUP setting from snf-common or onbasicConfig if that setting is missing....
Fix bug when synchronously creating networks
Release address only if a NIC has an IP
Give more weight to cpu_ratio
Change how scores are computed in default_allocator so that vcpuratio has more importance the free memory and disk ratio
Merge branch 'devel-0.11' into ui-0.11
Conflicts: snf-cyclades-app/synnefo/db/models.py snf-cyclades-app/synnefo/logic/backend.py snf-cyclades-app/synnefo/ui/static/snf/js/models.js
Change eventd and dispatcher AMQP retries
Set unlimited connection retries to AMQP hosts for snf-ganeti-eventd,since death of snf-ganeti-eventd results in loss of events.Also increase retries of snf-dispatcher.
Fix bug about RAPI client pooling
Modify the pooled_rapi_client context manager to handle exceptionsand release the client in any case. Also modify uses of reconciliationto handle exceptions inside the scope of the context manager
Add missing ipv6 arguments to create_network
Fix bug when hidding image password
Use deepcopy instead of copy, since the dictionaryis nested
Fix bug in snf-dispatcher about ignored messages
Update backendtime only for jobs that have been successfully completed,since only those jobs update the state of the VM. Else a "racecondition" may occur when a successful job (e.g. OP_INSTANCE_REMOVE)completes before an error job for the same VM and messages arrive in...
Hide image password from log file
Garbage collect GanetiRapiClient pools
Modify the rapi_pool to delete unused GanetiRapiClient pools. Thisclients become unused when the credentials of a backend change.
Add extra logging
Refresh backends in locked mode
Pool Ganeti RAPI clients
- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.- Create GanetiRapiClientPool for pooling GanetiRapiClient objects- Create context manager in db.models for pool handling- Update code using the RAPI client
Do not print public networks when adding backend
Fix white space
Add support for Archipelagos/extstorage COW images
Move check for free IP to backend_allocator
Filter out the backends that do no have a free IP in BackendAllocatorinstead of the default_allocator, in order to remove imports fromsynnefo in the allocation strategy.
Fix bugs in reconcile-networks
Perform logical and instead of or, when merging backend ip maps, sincenow the maps are inverted. Also do not report as stale backend networksthat are already deleted.
Take care for messages without timestamp
Catch exception produced in snf-dispatcher when receiving messages withoutproper timestamp.
Perform allocations in separate transactions
Add ForUpdateManager to the Backend model, in order to guaranteeexclusive order when allocating a VM.
Use manual transactions when allocating Backend and public IP for a newVM, since these allocations need exclusive access (SELECT..FOR UPDATE)...
Merge branch 'public_networks' into devel-0.11
Conflicts: snf-cyclades-app/synnefo/logic/dispatcher.py
Close DB conn in dispatcher before every message
Have the dispatcher close the Django DB connection before processingan incoming message. This allows the actual, underlying DBAPI connectionto return to the DB connection pool, if used, and enables the dispatcher...
Make snf-dispatcher work with gevent
Remove workers from snf-dispatcher in order to make it compatible with geventand specifically the green, monkey patched version of psycopg2(synnefo.lib.db.psyco_gevent). Also after daemonizing, gevent must bereinitialized because the deamon context forks.
Add pool-create and pool-remove commands
Change process title to simplify 'ps' display
Use python-setproctitle to change the process title of the runningdaemon. Make it appear as a native executable to simplifyadministration. Only set executable name, because cannot set individualvalues for arguments.
Fix presentation of pools
Fix two bugs in network reconciliation
Fix race conditions when updating network objects
Use for_update() object manager in order to exclude race conditionswhen updating network objects.
Fix double logging of dispatcher
Dispatcher had as default handlers the console and the file handler.However, when daemonizing, we redirect the stdout and stderr to the filehandler, which resulted in the log messages to be printed twice. So weremove the console handler from the dict configuration, and we only add...
Catch corner case in reconciliation
Catch error produced when the DB entry for an orphan VM in Ganeti doesnot exist. This may happen because either an entry from DB was removedor because a VM in Ganeti with the synnefo prefix was created.
Add some logging to backend allocator
Refactor IP pools
Treat IP pool the same way as bridge and mac-prefix pools. The pool isremoved from the Network table and moved to an IPPoolTable containing theIP pools for all networks. Also added support for external reservations.
Migration files for transfering the pool to the new tables are included....
Handle case of no pools in reconcile-pools
Management commands for pools
Create three new management commands for listing, showing and modifyinga pool.
Unique mac-prefix only for mac-filtered networks
Refactor bridge and mac-prefix pool tables
Redesign the implementation of pools and map mac-prefix and bridge poolsto that design.
Each pool type is mapped to a table in the database, and each row ofthat table is a pool of that type. All pool tables must contain the...
Fix bug in server-inspect
Enable hotplug during create_instance
This passes hotplug = True to ganeti and enables hotplug per instance.It checks GANETI_USE_HOTPLUG setting.
New method for creating BackendNetwork entries
Create a new method to the Network model, in order to create theBackendNetwork entries tha correspond to this network. Also removethe custom save method, since when multiple public network will besupported, we want them to be correlated with specific backends.
Remove dead code
Implement multiple public networks
Implement multiple public networks in order to support:- different public networks per backend- non-consecutive IP ranges in each backend
To support this, public networks are treated different from privateones. Namely, they are correlated with one backend (and so have...
Reserve gateway's IP when creating an IP pool
Revert commented line in logic backend
the line was uncommented by mistake in `7d43565`
Additional build progress callback handlers
Handle image-{error, info, warning, helper} messages and create thecorresponding virtual machine diagnostics.
Logic backend api helper to create vm diagnostics
Make dispatcher aware of the new monitoring msgs
Make dispatcher aware of image-warning and image-info messages
Comply with the new monitoring system of snf-image
For now only the image-copy-progress messages are processed
Support for ext disk providers
Extract provider from flavor disk template name and set the additional`provider` parameter in disk options. Flavor disk_template should beformated as `ext_<provider_name>`.
Add setting indicating whether to use hotplug
Management command for removing backend
Add command for checking pool consistency
Add new management command 'reconcile-pools' which checks the consistency of thepools and the uniqueness of the associated resources. Currently thiscommands detects the inconcistencies, without trying to resolve them.
Fix serious bug in reconciliation
Do not perform reconciliation for VMs that are hosted in Backendsthat are currently offline, since it is impossible to get thestate of these VMs from the backend. Before this patch, thereconciliation mechanism would report these VMs as stale (and with...