cyclades: Fix bug in snf-dispatcher logic
This commit reverts commits #07602322 and #30514662 which introduced abug in dispatcher's logic. These commit fixed the issue that, in a casethe Cyclades DB is unsynced with Ganeti, a resource that does not exist...
cyclades: Add disk template to flavors fixture
cyclades: Remove obsolete settings
Remove obsolete GANETI_DISK_TEMPLATES and DEFAULT_GANETI_DISK_TEMPLATEsettings.
Fix compatibility issues with factory_boy
Update models_factory to use factory.DjangoModelFactory instead offactory.Factory.
cyclades: Add 'drained' flag to Network
Extend Network model with the 'drained' attribute, to indicate whether anetwork should be used in public IP allocation when creating servers.This option is only useful for public networks.
cyclades: Add --hypervisor option to backend cmds
Add '--hypervisor' option to 'backend-add' and 'backend-modify'management commands.
Refs #3772
cyclades: Sypport backends with Xen hypervisor
cyclades: fix db migration
Migration 0066 should not run on a fresh installation.
cyclades: Make cyclades work with pycrypto==2.6
In PyCrypto 2.6, it is mandatory to use an IV of 16 bytes when using AESencryption. This commit updates 'aes_encrypt' module to use a random IV.Also, a data migration is required to re-encrypt the old passwords, as...
cyclades: Implement lazy networks
Implement "lazy networks", which are not created to all Ganeti backendsupon creation, but only when an instance that resides in a Ganeti isconnected to that network.
Update 'create_network' to create the Network object in DB without...
Refactor QuotaHolder serial model
Remove VM/Network DeletedError and BuildingError
Handle these errors explicitly in the code instead of the "api_method" decorator.
Move cloud faults to 'snf_django.lib.api.faults'
Gather all faults from all synnefo projects into common'snf_django.lib.api.faults'.
Simple tests for synnefo.quotas
Fix NIC's default state
Use "ACTIVE" as NIC's default state, to avoid addding a data migration.
Create public NIC when creating VM
Until now, a server's public NetworkInterface was not created in the DBuntil the arrival of the message from Ganeti hooks. However, the IPv4address of the NIC is reserved before sending the OP_INSTANCE_CREATEjob to Ganeti....
Bypass commissioning for orphan networks
Cyclades public networks do not have an owner, and since can not usethe commissioning system. Also, make snf-dispatcher instead of failingfor networks without userid to just a log a warning. This does not apply for...
Fix BridgePoolTable testcase
Fix whitespace, Licence
Re-implement select_for_update
Previous implementation of 'select_for_update' method was based on anattribute of the 'ForUpdateManager'. However, the model manager is sharedbetween all threads, which lead to race conditions. This commitreimplements 'select_for_update', by making it return a custom QuerySet...
Fix bug in PoolTable
Catch correct exception
Remove unused module
pep8 fixes
Add disk_template to Flavor name
Merge branch 'master' into devel-0.13
Conflicts: snf-cyclades-app/synnefo/db/models.py snf-cyclades-app/synnefo/logic/management/commands/backend-modify.py
Fix bug about networks and offline backends
Ignore offline backends when computing the state of the network, sinceno notification will be received from them. Also, create noBackendNetwork for an offline backend. This will be created byreconcilliation, when the backend gets online again.
Add more tests for db, logic and api
Fix wrong count of reserved pool entries
Padding should not be considered when counting unavailable entries.
Fix bugs in models methods
Add more tests for db and logic
Add object factories for most db models
Use 'factory_boy' library for creating Django ORM objects as test data.These factories can be used as fixtures replacement since they areeasier to write and maintain.
Include Pool tests to Django testsuite
Fix current db, logic and api tests
Do not allow creation of more than one pools
Do not allow creation of more than one Bridge or MacPrefix pool, sincewe can not handle them yet.
Introduce network flavors to replace network types
Add tags and mode to network db model. Replace existing type withflavor. Introduce 4 different flavors. Each flavor has mode, link,mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....
Migration files for network types
Fix bugs from merging
Add migration file for quotaholderserial
Reorder code
Create QuotaHolderSerial table
Add QuotaHolderSerial column to VM and Network
Correlate the VirtualMachine and Network models with a QuotaHolderSerialentry, holding the serial number returned by the QuotaHolder for thelast allocation for these resources.
Do not automatically update network state
Each modification in a BackendNetwork object, triggers an updateto the corresponding Network object. Until now this was done bythe BackendNetwork save() method. Now it is explicitly donefrom the backend pross_network_status.
Implement resizing of pools
Extend PoolManager to support extending and shrinking of Pools. Alsoadd extra option '--size' to pool-modify command.
Add indexes to VirtualMachine and Network
Add indexes to VirtualMachine userid and deleted columnsAdd indexes to Network userid, deleted and public columns
Use select_for_update for VirtualMachines
Use select_for_update method when accessing VM objects to preventrace conditions. Also move start_action from logic to api.
Delete BackendNetworks when deleting a Backend
Merge branch 'ui-0.11' into devel-0.12
Do not allow actions to deleted networks
Remove depricated settings and update migrations
Merge remote-tracking branch 'origin/devel-0.12' into ui-0.11
Fix compatibility issues with python-bitarray
Reapply diagnostics migrations
Merge branch 'devel-0.12' into ui-0.11
Do not create bridge for public network
Bridge indexes must start from 1
Restore IP Pool creation for corner cases
Create IPPool when creating Network
Create the IPPool associated with each Network, when creating it. Untilnow, IPPools were created on first demand. However, because the Networkobject is not accessed in exclusive mode, concurrent access to theNetwork model resulted in a race condition. More than one workers may...
Fix typos
Release mac_prefix only if network is MAC_FILTERED
Change and fix migration files
Change network migration, so that it does not allocate a new bridge toexisting network. Instead it keeps the bridges from the old NetworkLinkpool.
Remove network_mac_prefix migration because there are no networks withmac prefixes....
Merge branch 'devel-0.11' into ui-0.11
Conflicts: snf-cyclades-app/synnefo/db/models.py snf-cyclades-app/synnefo/logic/backend.py snf-cyclades-app/synnefo/ui/static/snf/js/models.js
Fix bug about RAPI client pooling
Modify the pooled_rapi_client context manager to handle exceptionsand release the client in any case. Also modify uses of reconciliationto handle exceptions inside the scope of the context manager
Fix wrong call in VirtualMachine get_client
Add missing arguments in Backend's get_client
Remove user provided names from models unicode
Remove user provided names from models representation, as the usermay provide a value that can affect the admin terminal. Fix suggestedby vkoukis.
Add extra logging
Pool Ganeti RAPI clients
- Move Ganeti RAPI client from snf-common to snf-cyclades-app/logic.- Create GanetiRapiClientPool for pooling GanetiRapiClient objects- Create context manager in db.models for pool handling- Update code using the RAPI client
Perform allocations in separate transactions
Add ForUpdateManager to the Backend model, in order to guaranteeexclusive order when allocating a VM.
Use manual transactions when allocating Backend and public IP for a newVM, since these allocations need exclusive access (SELECT..FOR UPDATE)...
Refactor IP pools
Treat IP pool the same way as bridge and mac-prefix pools. The pool isremoved from the Network table and moved to an IPPoolTable containing theIP pools for all networks. Also added support for external reservations.
Migration files for transfering the pool to the new tables are included....
Add tests for PoolManager(s)
Fix bugs in IPPool
- Do not always reserve gateway- Initial reservations must be in reserved map
Unique mac-prefix only for mac-filtered networks
Add logging to synefo.db.models
Refactor bridge and mac-prefix pool tables
Redesign the implementation of pools and map mac-prefix and bridge poolsto that design.
Each pool type is mapped to a table in the database, and each row ofthat table is a pool of that type. All pool tables must contain the...
New method for creating BackendNetwork entries
Create a new method to the Network model, in order to create theBackendNetwork entries tha correspond to this network. Also removethe custom save method, since when multiple public network will besupported, we want them to be correlated with specific backends.
Virtual machine diagnostics models and migrations
Initial commit for VirtualMachineDiagnostics model and the correspondingsouth migrations script.
Add check for maximum secret encryption key
SECRET_ENCRYPTION_KEY can be up to 32 bytes long, since bigger keys arenot supported by AES encryption. Add comment to the setting and bettermessage to exception.
Fix bug in migration file
Do not allocate a bridge from the pool for deleted networks.
Rename setting
Rename ENCRYPTION_KEY to SECRET_ENCRYPTION_KEY to prevent from beingdisplayed at django debug traces.
Revert "Start pool indexing from 0"
This reverts commit 2f998a3756c6fc7ddce9e0f83dea738e473320b8.
Start pool indexing from 0
Fix migration files
Fix south migration files. In data migrations, access the Models fromthe orm, instead importing Models from synnefo.db. This is necessary asthe orm provides access to the version of the models that existed whenthe migration file was created.
Implement MAC Prefix Management
In order to guarantee uniqueness of MAC address accross all backends,synnefo assignes a unique mac prefix to each network in each backend.
A mac-prefix of 20 bits, which is picked from the MacPrefixPool, isassigned to each Network. The mac-prefix of the network in each...
Warning about missing setting to migration file
Release resources of Network only on deletion
Assert that this is an 'real' deletion of a Network before releasingit's resources. Without this patch, a resource used by an other Networkcould be released, if someone called the Network.update_state function...
Fix bugs in migration files
Implement model manager using row-level db locking
Implement ForUpdateManager that uses SELECT .. FOR UPDATE statement inorder to acquire row-level exclusive locks to DB. Essentially thismanager backports the select_for_update() method that is implemented in...
Increase the size of backend password_hash
Use ForUpdateManager to Pool objects
Add support for IPv6
- Extend the Network model with fields for holding the IPv6 subnet and gateway.- Update network API methods.- Modify ganeti hooks to return IPv6 address for all network interfaces. IPv6 is now derived from the IPv6 subnet of the network in Ganeti,...
New management command for creating a network
Create a management command for creating networks. Also removethe initial_data.json, since it is currently not necessary.
Implement IP management
Delegate IP management to synnefo, in order to be able to allocate IPaddresses accross multiple Ganeti backends. This is achieved by creating aPool of IPs for each Network, in the form of a bitarray, which is storedin the DB as a base64 string....
Update network settings
Encrypt Backend password
Use cipher to encrypt the passwords of the Ganeti backends in the DB.Encryption/decryption is based on a configurable key, defined by theENCRYPTION_KEY setting. After modifying this setting, synnefo will beunable to decrypt already stored passwords, and passwords must be...
Add dirty bit to NetworkInterface
Extend NetworkInterface model with a dirty bit, indicating whetherthe index of the NIC is consistent with the one in Ganeti. This flag isset when deleting a NIC, and is switched off after processing thenotification from Ganeti. Removing a NIC with the dirty bit on is not...
Common functions for instance/network backend name
Release resources when deleting network
Set as available in the pool the bridge and mac prefix that wasused by the deleted network.
Bug fixes concerning networks
Print warning about subnet value during migration
Set index of NetworkInterface model to not null