cyclades: Various fixes to tests
dispatcher: Handle NICs in callbacks.
Update 'update_db' callback to handle instance NICs together withinstance status.
Also update the relevant tests.
cyclades: simplify comparison of NICs
Remove 'snf-events-net' queue and binding
Remove 'snf-events-net' queue (together with the corresponding routingkey and binding) from the queues that snf-dispatcher is consumingmessages. This queue is no longer needed, because messages for instanceNICs will be sended by snf-ganeti-eventd to 'snf-events-op' queue....
cyclades: Fix deadlock when connecting vms
Fix deadlock caused by 'get_or_create' method, since it acquires animplicit shared lock. Take an exclusive lock on network to fix thisdeadlock.
cyclades: Fix stale 'DeletedError' exception
cyclades: Fix typo in comment
cyclades: Fixes relevant to lazy networks
cyclades: Simplify reconcile-networks checks
Refactor and simplify checks of reconcile-networks to take advantageof lazy networks.
cyclades: Fix bug in management command
Do not try to issue commission for networks without user (e.g. publicnetworks), when deleting them.
cyclades: update tests for lazy networks
cyclades: Update 'backend-add' for lazy networks
Do not create private networks when adding a new backend. Since,"lazy networks" this is no longer needed.
cyclades: Implement lazy networks
Implement "lazy networks", which are not created to all Ganeti backendsupon creation, but only when an instance that resides in a Ganeti isconnected to that network.
Update 'create_network' to create the Network object in DB without...
Merge branch 'feature-quotas' into develop
Conflicts: docs/dev-guide.rst
cyclades: add commissioning to create/remove cmds
Update create/remove management commands to issue commission whenneeded.
astakos: WIP move all api calls under ``/astakos/api``
Refs: #3340
Add 'ASTAKOS_POOLSIZE' setting.
helpdesk: Fix helpdesk tests
cyclades: Fix vmapi test
Use 'urljoin' to compute the 'config_url' parameter.
Merge branch 'feature-api-docs' into develop
Minor Conflict: docs/cyclades-api-guide.rst
cyclades ui: Remove deprecated setting
cyclades ui: Expose and use the new astakos quota api
- Removed /userdata/quota view in respect of astakos quotas api delegate views served in the same urls used by astakos api (/astakos/api/quotas, /astakos/api/resources). Base url is configurable using the...
pithos, cyclades: Name reconciling commissions
Use commission name "RECONCILE" to distinguish such operationsin the quotaholder provision log.
cyclades: Fix import of urlparse
Merge branch 'feature-quotas-pithos' into feature-quotas
Conflicts: snf-cyclades-app/synnefo/plankton/backend.py snf-pithos-app/README snf-pithos-app/pithos/api/util.py
pithos: Rename backend quotaholder properties
Refer to astakosclient instead of quotaholder
WIP astakosclient for quotas in pithos
Fix typo in network-list command
cyclades: Rename 'qh_resolve_commissions' function
Rename 'qh_resolve_commissions' function to 'resolve_commissions'.
cyclades: Remove obsolete code for static quotas
Remove the following obsolete settings, since their usage is coveredby Quotaholder:- MAX_VMS_PER_USER- MAX_NETWORKS_PER_USER- VMS_USER_QUOTA- NETWORKS_USER_QUOTA
Also remove 'cyclades-export-quota' management command.
Implement resource-export-cyclades command
Implement managemtn command to export cyclades resources in JSON format,in order to be imported to Astakos.
Cyclades: remove all quotaholder settings
Remove from cyclades all settings that were relative with Quotaholder.Remove CYCLADES_USE_QUOTAHOLDER settings, since Cyclades can not workany more without quotas.Remove CYCLADES_QUOTAHOLDER_URL and CYCLADES_QUOTAHOLDER_TOKEN since...
Varous fixes to cyclades-usage-verify command
Extend 'cyclades-usage-verify' management command to cover the usage of'cyclades-reset-usage' and remove the second one. The new commandreconciles resources(usage) by detecting inconsistencies between theusage of resources in Astakos and Cyclades DB and fixing them by issuing...
cyclades: Rename quotas management commands
Rename 'cyclades-reconcile-commissions' to 'reconcile-commissions-cyclades.'Rename 'cyclades-usage-verify' to 'reconcile-resources-cyclades.'
reconcile-resources: check that user exists in QH
Do not consider 0 usage for users and resources that are not included inQuotaholder resources list, since Quotaholder returns zero usage for allusers and all resources. If a user/resource does not belong to QH...
Refactor QuotaHolder serial model
Fix info message of reconcile-resources-cyclades
cyclades: refactor code relative to commission
Refactor Cyclades code that handles commissioning with Quotaholder. Thiscommit removes the 'uses_commission' decorator. Instead it introducesthe 'issue_and_accept_commission' function that is responsible for...
cyclades: resolve previous commissions
Extend 'issue_and_accept_commission' function to resolve previousunresolved commission, if exists. In order to resolve it, thecommission must not be pending, because if it is there is notenough information to resolve it.
Mock quotaholder for Cyclades tests
Create 'mocked_quotaholder' context manager to bypass commission and useit in Cyclades tests. Also this commit does some various fixes to othertests.
cyclades: Check qh resolve commission response
Check Quotaholder response to see if the commissions were accepted orrejected as requested, or failed for some reason.
Accepting or rejecting a commission will fail if the commission wasalready resolved. Since this is sometimes accepted, like when...
Restore quotas if enqueuing job to Ganeti fails
When creating a server, we commit the VM model in DB and reserve thecorresponding quotas in Quotaholder before enqueuing theOP_INSTANCE_CREATE job (because messages from dispatcher may arrivebefore committing). In case enqueuing job to Ganeti fails we must...
cyclades: Use astakosclient for quotas and commissions
Remove unused 'models' file
Various fixes to quotas
snf-cyclades-app: Use astakosclient
cyclades: remove redundant settings
Remove 'CYCLADES_USER_CATALOG_URL' and 'CYCLADES_USER_FEEDBACK_URL'settings, since both of them can be derived from 'ASTAKOS_URL' setting.
plankton: Remove 'TRANSLATE_UUIDS' setting
Remove 'TRANSLATE_UUIDS' from plankton setting.
PEP8 style fixes
cyclades: fix bug in server-create command
Make --flavor-id mandatory in 'snf-manage server-create' managementcommand.
cyclades: Require user when creating private net
Update 'network-create' management command, to require an owner whencreating a private network.
cyclades: Fix typo in create_server
State of public NIC should be 'BUILDING'
vmapi: Remove extra '/' from config_url
Use urlparse.urljoin create the config url in order to avoid extra '/'.
Fix deadlock when creating server
Fix deadlock when creating a server that was introduced by commit501053a. Allocating the public IP in the same transaction with creatingthe server resulted in a deadlock with backend allocator, becauseinserting the new VM in the database waits on an exclusive lock of the...
Fix help message of management command
Cyclades: reserve public IP after commission
When creating a server, do not reserve a public IP address beforeissuing commission to quotaholder. This bug resulted in unsynchronizedIP pools between Cyclades and Ganeti, and needed externalreconciliation to synchronize them....
plankton: Cleanup and improve code
Improve performance in listing images
Use '_get_domain_objects' function of Pithos backend, to list allobjects in a 'plankton' domain, instead of looping over all accessibleaccounts, containers and objects. This function also returns metadataand permissions for each object in order to avoid two extra calls for...
List all images in snf-manage image-list
Until now 'snf-manage image-list' management command printed only imagesthat were public, or accessible by a specific user. This commit changesthis command to print all images (regardless of the owner).
plankton: raise BadRequest for invalid location
Modify 'add_image' plankton API method, to check that location parameteris a valid Pithos url, and raise BadRequest if not.
plankton: Correct image unregister
Correct usage of 'update_object_meta' function to remove all metadatafrom plankton domain. Use an empty dictionary with replace=True.
Refs #3522
plankton: add logging to backend functions
Bypass authentication in image-show command
Make image-show management command to show information about an imagewithout passing a userid. Use 'image_list' command of ImageBackendinstead of 'get_image' because the second one will raise Forbidden fornon-public images....
Remove transaction from 'backend-add' command
Do not run 'backend-add' command inside transaction, because if'snf-dispatcher' is running, it will receive messages that will arrivefrom a Ganeti backend that does not exist in the DB and will not be ableto process them.
Merge branch 'develop' into feature-api-docs
Fix small typo in import statements
Improve settings related to ASTAKOS url
Rename 'PITHOS_AUTHENTICATION_URL' setting to 'ASTAKOS_URL'. Also remove'PITHOS_AUTHENTICATION_USERS' from docs (it was removed from code incommit 8df4fae).
Modify 'ASTAKOS_URL' to point to the node hosting the Astakos service...
Fix small typo in import statement
Merge branch 'feature-snf-django' into develop
Conflicts: docs/older/Changelog.pithos-app snf-astakos-app/astakos/im/api/__init__.py snf-pithos-app/pithos/api/util.py
Merge branch 'release-0.13' into develop
Conflicts: snf-astakos-app/astakos/im/messages.py
Fix create vm view images filtering in cyclades ui
use the new synnefo.user method `get_username` to populatepersonal/shared images.
Add Server Create documentation
Fix typo
Move astakos lib from snf-common to snf-django-lib
Use 'python-objpool' instead of 'synnefo.lib.pool'
Use the python-objpool package for pooling resources as object, insteadof the pooling code that existed in snf-common ('synnefo.lib.pool')
Refs #3524
Move common test code to snf_django.lib.testing
Remove VM/Network DeletedError and BuildingError
Handle these errors explicitly in the code instead of the "api_method" decorator.
Remove ImageBackend NotAllowedError
Refs #3359
Use the common 'api_method' decorator in plankton
Use the common 'api_method' decorator in all plankton API methods.Modify the 'plankton_method' decorator to only create and closethe ImageBackend.
Use common 'api_method' decorator to cyclades api
Update Cyclades API methods to use the new common 'api_method'decorator. Also, update all the relevant tests.
Use common 'api_method' decorator in pithos api
Remove dead code
Move cloud faults to 'snf_django.lib.api.faults'
Gather all faults from all synnefo projects into common'snf_django.lib.api.faults'.
Use pprint_table in server-show command
Simple tests for synnefo.quotas
Make 'type' parameter for network-create mandatory
Do not use a default value (was "MAC_FILTERED") for "type" parameter of"network_create" API method. The client must explicitly choose thenetwork type.
Delete Image without deleting file from Pithos
Until now, deleting an image resulted in deletion of the correspondingfile from Pithos. Plankton Images are files in Pithos with specialmetadata on the 'plankton' metadata domain. Adding an Image to plankton...
Fix plankton ItemNotFound exception
Plankton backend raised AssertionError when an Image was not found.Change this to raise faults.ItemNotFound error. Temporary commituntil common API decorator is implemented.
Refs #3356
Conflicts: snf-astakos-app/astakos/im/messages.py snf-astakos-app/astakos/im/views.py version
Allow helpdesk users to shutdown/start vms
Include an additional action to helpdesk view. Based on the currentstatus of the virtual machine helpdesk user is allowed to start it orshut it down.
This comes handy since in most cases vm suspension needs to be...
Remove ui debug statement
Update cyclades webproject settings
remove deprecated admin app entries
Merge branch 'tmp-common-email-recipients' into develop
Merge branch 'feature-astakos-authpolicies' into develop
Conflicts: snf-cyclades-app/synnefo/api/management/commands/flavor-create.py version
UI auth module typo fix
fix get_username to return user uuid instead of user token. The changefix the resolving of image ownership of the current ui user.
UI create vm errors fix
never forget to pass vm create callback arguments
Delegate cyclades ui feedback calls to astakos
Mimic pithos app/ui feedback handling to avoid duplicating boilerplatecode.
New settings:CYCLADES_USER_FEEDBACK_URL
Deprecated (removed) settings:FEEDBACK_CONTACTSFEEDBACK_EMAIL_FROM
pool context manager, http pool AssertionError fix
Refs #3492
- Introduce a generic PooledObject class to act both as a context manager for getting and putting back an object from a pool.
- Implement a class PooledHTTPConnection(PooledObject) as a sublcass of the one in (1)...
Display additional vm backend info in helpdesk app
Refs #3452