Split starting and stopping master IP and daemons
Adapt non-KVM hypervisors to new migration RPCs
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add memory transfer progress info to migration
Make migration RPC non-blocking
To add status reporting for the KVM migration, the instance_migrate RPCmust be non-blocking. Moreover, there must be a way to represent themigration status and a way to fetch it.
Move _TimeoutExpired to utils
Add an allocation limit to hspace
This is very useful for testing/benchmarking.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Small simplification in tryAlloc
Change how node pairs are generated/used
Currently, the node pairs used for allocation are a simple [(primary,secondary)] list of tuples, as this is how they were used before theprevious patch. However, for that patch, we use them separately perprimary node, and we have to unpack this list right after generation....
Parallelise instance allocation/capacity computation
This patch finally enables parallelisation in instance placement.
My original try for enabling this didn't work well, but it took awhile (and liberal use of threadscope) to understand why. The attempt...
Abstract comparison of AllocElements
This is moved outside of the concatAllocs as it will be needed inanother place in the future.
Change type of Cluster.AllocSolution
Originally, this data type was used both by instance allocation (1result), and by instance relocation (many results, one perinstance). As such, the field 'asSolutions' was a list, and thevarious code paths checked whether the length of the list matches the...
http.client: Show pending requests as “owner”
In the context of the lock monitor a “pending” item does not yet own therequested resource. Since these HTTP requests are already undergoingthey should be shown as owners.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
http.client: Add nice name to requests
With this change a node name instead of the IP address can be shown forpending RPC requests:Name Pendingrpc/node18.example.com/test_delay thread:Jq1/Job692/TEST_DELAY
rpc/http: Show pending RPC requests in lock monitor
Not all requests use an instance of RpcRunner yet and therefore won'tshow up (only instances have access to the global Ganeti context).Currently only the IP address is accessible. Another patch will add a...
http.client: Factorize code interacting with cURL
This simplifies HttpClientPool.ProcessRequests significantly and will behandy for showing pending RPC requests in the lock monitor.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Adding qemu-img dependency to INSTALL
Signed-off-by: Agata Murawska <agatamurawska@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
http.client: Reduce performance impact by assertion
Call dict.values once instead of N times.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
rpc: Overhaul client structure
- Clearly separate node name to IP address resolution into separate functions- Simplified code structure (one code path instead of several)- Fully unittested- Preparation for more RPC improvements
rpc: Make compression function module-global
No need to keep it in the class.
Keep only one global RPC runner in Ganeti context
Instead of having one RPC runner per mcpu processor this will keep onlyone instance as part of the masterd-wide Ganeti context. Upcomingpatches will change the RPC runner to report pending requests to the...
Update INSTALL with ovfconverter requirements
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
TemporaryFilesManager implementation
Export: unittests
Export: documentation
Export: saving data to ovf file
Export: parsing data from config file
Export: initial commit - manifest, ova creation etc
Import: unittests
Import: backend, hypervisor and os
Import: networks
Import: disk conversion
Import: reading ovf file
Initial commit for ovfconverter tool
doc: sphinx config file changes
I wanted to just enable another extension (the graphviz one), but thenI went and did a lot of changes:
- replaced ' with " for consistency with our style guide- imported new settings (commented out) that current python-sphinx...
doc: re-wrap design-oob to 72 chars
I started with just adding some :term:`SoW` and similar to design-oob,but then I realised this was 80-chars wrapped, not 72-chars. So I wentand re-wrapped most of it, plus adding the glossary references.
Signed-off-by: Iustin Pop <iustin@google.com>...
doc: glossary improvements
These will be used to remove some inline definitions and replace themwith :term:`foo`.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
serializer: Add comment about simplejson vs. built-in json
Revert "Fail if dictionary uses invalid keys" and "Support newer “json” module"
This reverts commit fd0351aef246f5d36e641209429e2ec093d325f8 and9869e771704ada62bab001e729c52a36525ef081. The built-in module is a lotslower in Python 2.6.
serializer: Fail if dictionary uses invalid keys
JSON only supports a very restricted set of types for dictionary keys,among them strings, booleans and “null”. Integers and floats areconverted to strings. Since this can cause a lot of confusion in Python,...
serializer: Support newer “json” module
This module is included from Python 2.6 and is based onsimplejson.
htools: man page improvements
This patch moves all the backend options into the main htools manpage, and it adds documentation for the -t option, which so far wasnot documented w.r.t. the file structure.
hspace: add short forms for the group policy
This adds a shortened versions of the allocation policies, as writingout the whole name in the command line can become tedious.
Fix interaction between CPU pinning and KVM migration
CPU pinning requires the KVM hypervisor to start in the paused state, inorder to retrieve information, and immediately unpauses it.
This does not play well with live migration, as the unpausing was done...
htools: add a MonadPlus instance for Result
This will be used to implement more easily 'choice' parsing of inputdata, without resorting to syntax (case … of Bad _ -> …).
Merge branch 'devel-2.5'
RAPI: Add resource to powercycle node
Add tls_ciphers and use_vdagent options
Updated man pages with new SPICE TLS options
Implementation of TLS-protected SPICE connections
Added support for TLS-protected SPICE connections:
Added SPICE TLS option and related cert paths
Fix OS creation's error handling when pausing sync
Commit 41e1e79 introduced a feature in which when wait_for_sync is notset, DRBD sync is paused during the OS installation.
Doing so, however, broke OS creation's error handling: the result valuefrom the instance_os_add RPC call was overwritten by the one of the...
Fix two pylint errors
- hv_kvm.py: silence F0401, that is raised if pylint does not find the affinity module- rlib2.py: change disable-msg to disable
Fix backend.MigrateInstance docs
The MigrateInstance function does not return anything, so the relevantlines are removed from the documentation. Instead, the raised exceptionis documented.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
hv_kvm: bugfix
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Import: further doc updates
Signed-off-by: Agata Murawska <agatamurawska@google.com>Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
RAPI: Add resource to recreate instance's disks
This was still missing from RAPI.
Adding an updated design doc for the caching mechanism
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_xen: fix use of CPU pinning constants
… to be consistent with hv_kvm
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_kvm: fix hardcoded KVM command string
hv_kvm: support for CPU pinning
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: Iustin Pop <iustin@google.com>[iustin@google.com: fixed some small code and style issues]Reviewed-by: Iustin Pop <iustin@google.com>
constants: support for CPU pinning under KVM
Make affinity module optional
This was overlooked in the review, sorry.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
configure.ac: changes for CPU pinning
Also added optional affinity Python package for CPU pinning under KVMto the INSTALL doc.
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
docs unittest: Add verification of opcodes covered by RAPI
All opcodes which are not yet covered or can't with the current design(e.g. cluster initialization) are excluded. This test is added to thedocs unittest since here the code already has access to a list of all...
docs unittest: Split into two separate test cases
The RAPI and hooks documentation aren't related at all.
docs unittest: Make _ReadDocFile a global function
rlib2: Declare all opcodes and equivalents
By declaring all used opcodes or opcodes equivalent to the operationsexecuted in a resource we will be able to ensure all opcodes are coveredby RAPI (with some exceptions).
baserlib: Function to retrieve opcodes used by handler
This will be used to verify opcodes used by RAPI.
htools: remove dead code
The tryEvac/evacuateInstance functions are no longer used in the newmulti-group world order, so we remove them and change the unit-test totest the actual IAllocator function.
hail: don't select the primary as new secondary
This just adds the primary node of the instance as 'non-allocable'during the choosing of the new secondary.
hail: add an extra safety check in relocate
If we select the primary as new secondary, better to fail than returnwrong data to Ganeti.
Fix RAPI documentation for gnt-instance console
Fix a failing pyassert in the RAPI docs and update it to reflect theaddition of SPICE to gnt-instance console.
Add SPICE compression and streaming options
Add the following SPICE audio/image compression and video streamingdetection hypervisor options:
Add SPICE support to gnt-instance console
Also update related unit tests.
Fix unittest breakage from commit 132cdb87159e
rapi: Re-add “/2” resource
Like “/”, it'll just return a JSON null.
Wrap long line in ganeti.rapi.rlib2_unittest.py
rlib2: Improve test for _ParseInstanceReinstallRequest
rlib2: Add test for /2/info
rlib2: Add tests for /, /version and /2/features
rlib2: Convert /2/instances to OpcodeResource
rlib2: Convert /2/*/tags to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/storage/repair to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/storage/modify to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/storage to OpcodeResource
rlib2: Convert /2/groups/[group_name] to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/role to OpcodeResource
Also fix a typo in a constant name.
rlib2: Convert /2/instances/[inst]/disk/[idx]/grow to OpcodeResource
rlib2: Convert /2/instances/[inst]/modify to OpcodeResource
rlib2: Convert /2/instances/[inst]/rename to OpcodeResource
rlib2: Convert /2/instances/[inst]/failover to OpcodeResource
rlib2: Convert /2/instances/[inst]/migrate to OpcodeResource
rlib2: Convert /2/instances/[inst]/export to OpcodeResource
rlib2: Convert /2/instances/[inst]/prepare-export to OpcodeResource
rlib2: Convert /2/instances/[inst]/deactivate-disks to OpcodeResource
rlib2: Convert /2/instances/[inst]/activate-disks to OpcodeResource
rlib2: Convert /2/instances/[inst]/replace-disks to OpcodeResource
rlib2: Convert /2/instances/[inst]/shutdown to OpcodeResource
rlib2: Convert /2/instances/[inst]/startup to OpcodeResource