Remove transaction from 'backend-add' command
Do not run 'backend-add' command inside transaction, because if'snf-dispatcher' is running, it will receive messages that will arrivefrom a Ganeti backend that does not exist in the DB and will not be ableto process them.
Move astakos lib from snf-common to snf-django-lib
Use 'python-objpool' instead of 'synnefo.lib.pool'
Use the python-objpool package for pooling resources as object, insteadof the pooling code that existed in snf-common ('synnefo.lib.pool')
Refs #3524
Use pprint_table in server-show command
Create public NIC when creating VM
Until now, a server's public NetworkInterface was not created in the DBuntil the arrival of the message from Ganeti hooks. However, the IPv4address of the NIC is reserved before sending the OP_INSTANCE_CREATEjob to Ganeti....
Move UserCache from cyclades.common to lib.astakos
Move UserCache from synnefo.management.common to synnefo.lib.astakos.Also, update ListCommand with 'astakos_url' and 'astakos_token' classattributes, which must be declared only when 'user_uuid_field' is used.
Move common code for mngmt commands to webproject
Move cyclades functions that are useful for all managament commands, likepprint_table or parse_filters, to snf-webproject.
Add ListCommand to be used for all *-list commands
Create a generic ListCommand to be user for all *-list managementcommands. ListCommand handles must tasks that are common to allmanagement commands, like retrieving objects from db, filteringresults and formating output to a pretty table, json or csv....
Merge branch 'release-0.13' into develop
Conflicts: version
Bypass commissioning for orphan networks
Cyclades public networks do not have an owner, and since can not usethe commissioning system. Also, make snf-dispatcher instead of failingfor networks without userid to just a log a warning. This does not apply for...
Conflicts: snf-cyclades-app/synnefo/api/management/commands/cyclades-export-quota.py snf-cyclades-app/synnefo/api/management/commands/network-list.py snf-cyclades-app/synnefo/api/management/commands/server-inspect.py...
Improve server-list usability
Refs #3375
Improve perf for snf-manage cyclades commands
When the snf-manage {server, network}-list commands are invoked with the--displayname option, batch the UUID-to-displayname requests to astakos,instead of issuing a request for each (unique) UUID.
reconciliation: handle missing network from db
Continue reconciliation even if an orphan network is missing from DB.
Fix typo
pep8 convension
pep8 convension to cyclades management commands
Add --uuids option to snf-manage cmds
Totally remove password from create_instance
Never pass password as an 'osparam' to Ganeti. Always use the vmapi,even for VMs created with server-create management command.
Fix reconciliation for building VMs
Since commit 482c645, a VM exists in the Cyclades DB before thecorresponding job is enqueued in the Ganeti backend. Therefore, there isa corner case, where Cyclades can not enqueue the job to Ganeti andCyclades crashes before deleting the VM from the DB, which results in a...
Remove mac2eui64 dependency
Import mac2eui64 code which is already shipped under util/mac2eui64 insteadof shipping it as a seperate script.
Fix pep8 issues
Fix pep8 warnings
Fix bug in network reconciliation
Fix bug in network reconciliation caused by wrong variable scope.
pep8 fixes
Rename ARCHIPELAGOS to ARCHIPELAGO
Rename ARCHIPELAGOS_BACKENDS setting to ARCHIPELAGO_BACKEND.
Fix progress messages handling
proper handling of missing `messages` key progress messages.
Network field type has been renamed to flavor
Fix typo bug in reconcile ip pools
Add warning message for backend without networks
Refactor network reconciliation
Remove unused imports from tests
Merge branch 'master' into devel-0.13
Conflicts: snf-cyclades-app/synnefo/db/models.py snf-cyclades-app/synnefo/logic/management/commands/backend-modify.py
Refactor tests
Fix bug about networks and offline backends
Ignore offline backends when computing the state of the network, sinceno notification will be received from them. Also, create noBackendNetwork for an offline backend. This will be created byreconcilliation, when the backend gets online again.
Add more tests for db, logic and api
Remove dummy function
Add more tests for db and logic
Refactor some test code
Misc minor bug fixes
Fix current db, logic and api tests
Fix variable scope bug in process_net_status
Do not allow creation of more than one pools
Do not allow creation of more than one Bridge or MacPrefix pool, sincewe can not handle them yet.
Add backend-id option to reconcile-servers
Make reconcile-servers perform reconciliation only for the specifiedbackend.
reconciliation:avoid double asking Ganeti
Modify reconcile-servers to not communicate 2 times with Ganetito get the operstate and the NICs of the VMs. Merge this two onecall.
Modify backend-modify --drained and --offline args
Make backend-modify drained argument as --drained=True|False insteadof --drained and --no-drained. The same for offline.
Mark as drained newly added backends
Rename reconcile command to reconcile-servers
Rename `snf-manage reconcile` management command to `snf-managereconcile-servers`, to be consistent with `snf-managereconcile-networks`. Also, treat '--detect-all' option as defaultwhen no other '--detect-*' is specified.
Introduce network flavors to replace network types
Add tags and mode to network db model. Replace existing type withflavor. Introduce 4 different flavors. Each flavor has mode, link,mac_prefix, tags. Supported flavors are CUSTOM, IP_LESS_ROUTED,MAC_FILTERED, PHYSICAL_VLAN and are hardcoded in db models....
Implement quotas-init and quotas-verify
Implement quotas-init and quotas-verify management commands.- quotas-init sets quotas of Quotaholder to the state of quotas in Cyclades DB- quotas-verify detects incosistencies between quotas in Quotaholder and...
Temporary fix for allocating archipelagos VMs
Fix typos
Merge handling of ext_<provider> disk templates
Merge branch 'origin/devel-0.13' into quota
Conflicts: snf-cyclades-app/synnefo/api/servers.py
Issue commission when deleting VM/Network
Issue commission to the Quotaholder when being notified from the Ganetibackend that a VM/Network has been deleted. Although the resource hasalready been deleting, we are interacting with the quotaholder in atwo step phase: issue commission and accept commission. It would be...
Implement reconciliation for pending quotas
Remove transaction from update_network_state
Remove transaction decorator from update_network_state function, sincethis function is only called from process_network_status, which already runsinside a transaction.
Do not automatically update network state
Each modification in a BackendNetwork object, triggers an updateto the corresponding Network object. Until now this was done bythe BackendNetwork save() method. Now it is explicitly donefrom the backend pross_network_status.
Remove IGNORE_FLAVOR_DISK_SIZES setting
Use vmapi when creating servers
Use the vmapi when creating servers. Creete instance RAPI call takes theconfiguration url instead all the image parametere. The id and formatof the image are still passed, just for readability and debugging.
Also renamed params_url to config-url.
snf-dispatcher: Update NICs only when needed
Do not remove and recreate the NICs of a VM when NICs have not changed.
Implement resizing of pools
Extend PoolManager to support extending and shrinking of Pools. Alsoadd extra option '--size' to pool-modify command.
Add function for pretty printing tables
Make amqp_puka get logger as argument
Check clustername when modifying backend
Add check for clustername when adding backend
Fix lectical in pool-modify
Refactor code in management commands
Improve management commands
Merge branch 'master' into devel-0.12
Conflicts: snf-cyclades-app/synnefo/api/management/commands/flavor-list.py snf-cyclades-app/synnefo/api/management/commands/server-list.py snf-cyclades-app/synnefo/api/networks.py snf-cyclades-app/synnefo/logic/rapi.py
Fix dispatcher logging
Use select_for_update for VirtualMachines
Use select_for_update method when accessing VM objects to preventrace conditions. Also move start_action from logic to api.
Process OP_NETWORK_SET_PARAMS Ganeti jobs
Modify snf-ganeti-eventd and snf-dispatcher to handle jobs withOP_NETWORK_SET_PARAMS opcode. Handling of these jobs is neededto update the IP Pool of network, when reserving and releasingIPs directly from the Ganeti backend.
PUBLIC_ROUTED minor changes
Do not allow api to create public networks and type otherthan PRIVATE_MAC_FILTERED/PRIVATE_PHYSICAL_VLAN.
Rename PUBLIC_ROUTED_USE_POOL setting to PUBLIC_USE_POOL.
Let synnefo do IP pool management only if network is private or...
Add conflicts_check option in network rapi calls
Ganeti by default checks for conflicting IPs when creating/connectingnetworks. This feature can be bypassed for private networks. We donot care for IP uniqueness in case of private networks as long as they...
Update rapi client
Get rapi client from snf-ganeti. Add depends argument (for chain jobs)where needed.
Argument tags must be a list of strings and arguments add_reserved_ips andremove_reserved_ips must be None or a list of strings.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Minor backend fixes in networks
Add conflicts_check option when creating and add network. Check onlyif a network is public.
Let tags argument in rapi calls be a list.
Display VM build percentage in management commands
Allow custom link, mac_prefix in network-create
Add extra command line arguments, to allow the user specify thelink and mac_prefix of the new network if needed.
Introduce per-user allocation of VMs to backends
Add BACKEND_PER_USER setting, a mapping from users to backends. This settingcan by used in order to bypass the default backend allocation process, andallocate all the new VMs of a specific user to a specific Backend.
Do not allow any API action on suspended VMs
Do not allow any modification of any administratively suspended VM through theAPI.
Merge branch 'ui-0.11' into devel-0.12
Merge remote-tracking branch 'origin/devel-0.12' into ui-0.11
Conflicts: snf-cyclades-app/setup.py
Merge branch 'devel-0.12' into ui-0.11
Implement queue-retry management command
Implement a new command which consumes all the messages from thedead-letter queues and re-sends the messages back to the snf-dispatcherin order to handle them again.
Messages are annotated with "from-dead-letter" field in order to detect...
Fix bug, UI not updating NICs
Dummy-save the Network when processing messages from Ganeti hooksabout a VM NICs. This is needed, because the UI uses the HTTP changes-sinceswhen requesting VMs and Networks, and combines the result in order to showthe new NICs. Not updating the Network object, resulted in VMs IPv4...
Declare dead-letter exchanges and queues
Modify snf-dispatcher in order to declare the dead-letter exchanges andqueues
Use basic_reject in snf-dispatcher callbacks
Modify snf-dispatcher callbacks in order to reject messages that causedan unknown exception. Because a dead-letter-exchange is associatedwith the queues that snf-dispatcher is using, these messages will betransfered to the dead-letter-exchange.
Add mac2eui64 to snf-cyclades-app
Copy mac2eui64 from snf-network to snf-cyclades-app, since it isnow used in the reconciliation process to caclculate the IPv6address of the VMs.
Reconnect to different Broker if idle for too long
Modify snf-dispatcher and add a timeout option to basic_wait method. Iftimeout is reached, the connection to the current RabbitMQ broker isclosed and dispatcher tries to connect to a different one.
This commit is supplementary of 6d27ead, which detects connection...
Detect and handle connection problems in amqp_puka
Use the TCP keepalive feature in amqp_puka in order to detect connectionerrors to the RabbitMQ broker, and retry to a different one. Also setupthe keepalive parameters to relative small values, to detect failures...
Fix bug in reconcile-networks
When only detecting, reconciliation can not continue, if theBackendNetwork object does not exist.
Do not give IPs to networks without dhcp
Use - instead of _ in backend-update-status
Fix bug when connecting a network to a nodegroup in networkreconciliation. Use connect_network instead of obsoleteconnect_network_group method. Also fix job dependency handling in thismethod.