A few minor test improvements
This patch adds a few niceties to the test suite:
- allows matching test groups case insensitive and emit warnings when we give test group names that don't match anything- add a new operator that is similar to assertEqual in Python: it...
Use TemplateHaskell to decorate tests with names
This makes error message change from "Test 4 failed …" to "Testprop_Loader_mergeData failed", which is much more readable. It alsoremoves the duplication of test suite names in the test.hs file.
Signed-off-by: Iustin Pop <iustin@google.com>...
Use TemplateHaskell to generate opcode serialisation
This replaces the hand-coded opcode serialisation code withauto-generation based on TemplateHaskell.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Use TemplateHaskell to build the opID function
This replaces the hand-coded opID with one automatically generatedfrom the constructor names, similar to the way Python does it, exceptit's done at compilation time as opposed to runtime.
Again, the code line delta does not favour this patch, but this...
Use TemplateHaskell instead of hand-coded instances
This patch replaces the current hard-coded JSON instances (all alike,just manual conversion to/from string) with auto-generated code basedon Template Haskell(http://www.haskell.org/haskellwiki/Template_Haskell)....
Rename some helper functions for consistency
This changes the names for some helper functions so that futurepatches are touching less unrelated code. The change replacesshortened prefixes with the full type name.
Split part of Utils.hs into JSON.hs
Utils is a bit big, let's split the JSON stuff (not all of it) into aseparate module that doesn't have any other dependencies.
Merge branch 'devel-2.5'
Merge branch 'stable-2.5' into devel-2.5
Use --yes to deactivate master ip in cluster merge
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
Use deactivate-master-ip in cluster-merge
Use the gnt-cluster deactivate-master-ip command in cluster-merge todisable the master IP.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...
Add gnt-cluster commands to toggle the master IP
Split starting and stopping master IP and daemons
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
utils.log: Write error messages to stderr
When “gnt-cluster copyfile” failed it would only print “Copy of file …to node … failed”. A detailed message is written using logging.error.Writing error messages to stderr can be helpful in figuring out whatwent wrong (the messages also go to the log file, but not everyone might...
ssh: Quote strings in error message
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
listrunner: Don't pass arguments if there are none
If no arguments were specified the “exec_args” variable was “None”,leading to the command being run as “… ./… None”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Add signal handling doc to hbal man page
Also remove a bug note, since hbal can now for a long time directlyexecute jobs.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Adapt non-KVM hypervisors to new migration RPCs
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add memory transfer progress info to migration
Make migration RPC non-blocking
To add status reporting for the KVM migration, the instance_migrate RPCmust be non-blocking. Moreover, there must be a way to represent themigration status and a way to fetch it.
Move _TimeoutExpired to utils
Add an allocation limit to hspace
This is very useful for testing/benchmarking.
Small simplification in tryAlloc
Change how node pairs are generated/used
Currently, the node pairs used for allocation are a simple [(primary,secondary)] list of tuples, as this is how they were used before theprevious patch. However, for that patch, we use them separately perprimary node, and we have to unpack this list right after generation....
Parallelise instance allocation/capacity computation
This patch finally enables parallelisation in instance placement.
My original try for enabling this didn't work well, but it took awhile (and liberal use of threadscope) to understand why. The attempt...
Abstract comparison of AllocElements
This is moved outside of the concatAllocs as it will be needed inanother place in the future.
Change type of Cluster.AllocSolution
Originally, this data type was used both by instance allocation (1result), and by instance relocation (many results, one perinstance). As such, the field 'asSolutions' was a list, and thevarious code paths checked whether the length of the list matches the...
Migration: warn the user about hv version mismatch
Fix handling of cluster verify hooks
The change to enforce boolean results for cluster verify group opcodemissed the HooksCallBack, which uses a very ugly 1/0logic. Furthermore, the logic is wrong, since it unconditionallyresets the verify result to true....
http.client: Show pending requests as “owner”
In the context of the lock monitor a “pending” item does not yet own therequested resource. Since these HTTP requests are already undergoingthey should be shown as owners.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
rpc/http: Show pending RPC requests in lock monitor
Not all requests use an instance of RpcRunner yet and therefore won'tshow up (only instances have access to the global Ganeti context).Currently only the IP address is accessible. Another patch will add a...
http.client: Add nice name to requests
With this change a node name instead of the IP address can be shown forpending RPC requests:Name Pendingrpc/node18.example.com/test_delay thread:Jq1/Job692/TEST_DELAY
http.client: Factorize code interacting with cURL
This simplifies HttpClientPool.ProcessRequests significantly and will behandy for showing pending RPC requests in the lock monitor.
Redistribute the RAPI certificate
This reverts to the old behaviour in Ganeti 2.4 and before.
Adding qemu-img dependency to INSTALL
Signed-off-by: Agata Murawska <agatamurawska@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
http.client: Reduce performance impact by assertion
Call dict.values once instead of N times.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
rpc: Overhaul client structure
- Clearly separate node name to IP address resolution into separate functions- Simplified code structure (one code path instead of several)- Fully unittested- Preparation for more RPC improvements
rpc: Make compression function module-global
No need to keep it in the class.
Keep only one global RPC runner in Ganeti context
Instead of having one RPC runner per mcpu processor this will keep onlyone instance as part of the masterd-wide Ganeti context. Upcomingpatches will change the RPC runner to report pending requests to the...
Update INSTALL with ovfconverter requirements
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
TemporaryFilesManager implementation
Export: unittests
Export: documentation
Export: saving data to ovf file
Export: parsing data from config file
Export: initial commit - manifest, ova creation etc
Import: unittests
Import: backend, hypervisor and os
Import: networks
Import: disk conversion
Import: reading ovf file
Initial commit for ovfconverter tool
doc: glossary improvements
These will be used to remove some inline definitions and replace themwith :term:`foo`.
doc: re-wrap design-oob to 72 chars
I started with just adding some :term:`SoW` and similar to design-oob,but then I realised this was 80-chars wrapped, not 72-chars. So I wentand re-wrapped most of it, plus adding the glossary references.
doc: sphinx config file changes
I wanted to just enable another extension (the graphviz one), but thenI went and did a lot of changes:
- replaced ' with " for consistency with our style guide- imported new settings (commented out) that current python-sphinx...
serializer: Add comment about simplejson vs. built-in json
Revert "Fail if dictionary uses invalid keys" and "Support newer “json” module"
This reverts commit fd0351aef246f5d36e641209429e2ec093d325f8 and9869e771704ada62bab001e729c52a36525ef081. The built-in module is a lotslower in Python 2.6.
serializer: Fail if dictionary uses invalid keys
JSON only supports a very restricted set of types for dictionary keys,among them strings, booleans and “null”. Integers and floats areconverted to strings. Since this can cause a lot of confusion in Python,...
serializer: Support newer “json” module
This module is included from Python 2.6 and is based onsimplejson.
htools: man page improvements
This patch moves all the backend options into the main htools manpage, and it adds documentation for the -t option, which so far wasnot documented w.r.t. the file structure.
hspace: add short forms for the group policy
This adds a shortened versions of the allocation policies, as writingout the whole name in the command line can become tedious.
Fix interaction between CPU pinning and KVM migration
CPU pinning requires the KVM hypervisor to start in the paused state, inorder to retrieve information, and immediately unpauses it.
This does not play well with live migration, as the unpausing was done...
QA: Add tests for instance start/stop via RAPI
This would have detected the issue fixed in the previous patch.
RAPI: Fix wrong check on instance shutdown
Commit 7fa310f6d84 (April 1st, 2011) converted the RAPI resource forshutting down an instance to FillOpCode. Unfortunately it missed thefact that the shutdown resource gets its parameters as query arguments.
baserlib: Accept empty body in FillOpcode
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>(cherry picked from commit c6e1a3eef05674d637570c39f25a799cec7ba187)
Signed-off-by: Michael Hanselmann <hansmi@google.com>
htools: add a MonadPlus instance for Result
This will be used to implement more easily 'choice' parsing of inputdata, without resorting to syntax (case … of Bad _ -> …).
RAPI: Add resource to powercycle node
Add tls_ciphers and use_vdagent options
Added SPICE TLS option and related cert paths
Implementation of TLS-protected SPICE connections
Added support for TLS-protected SPICE connections:
Updated man pages with new SPICE TLS options
Fix OS creation's error handling when pausing sync
Commit 41e1e79 introduced a feature in which when wait_for_sync is notset, DRBD sync is paused during the OS installation.
Doing so, however, broke OS creation's error handling: the result valuefrom the instance_os_add RPC call was overwritten by the one of the...
Fix two pylint errors
- hv_kvm.py: silence F0401, that is raised if pylint does not find the affinity module- rlib2.py: change disable-msg to disable
Fix backend.MigrateInstance docs
The MigrateInstance function does not return anything, so the relevantlines are removed from the documentation. Instead, the raised exceptionis documented.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
hv_kvm: bugfix
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Import: further doc updates
Signed-off-by: Agata Murawska <agatamurawska@google.com>Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
RAPI: Add resource to recreate instance's disks
This was still missing from RAPI.
Adding an updated design doc for the caching mechanism
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_kvm: support for CPU pinning
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: Iustin Pop <iustin@google.com>[iustin@google.com: fixed some small code and style issues]Reviewed-by: Iustin Pop <iustin@google.com>
hv_kvm: fix hardcoded KVM command string
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_xen: fix use of CPU pinning constants
… to be consistent with hv_kvm
constants: support for CPU pinning under KVM
Make affinity module optional
This was overlooked in the review, sorry.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
configure.ac: changes for CPU pinning
Also added optional affinity Python package for CPU pinning under KVMto the INSTALL doc.
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
docs unittest: Add verification of opcodes covered by RAPI
All opcodes which are not yet covered or can't with the current design(e.g. cluster initialization) are excluded. This test is added to thedocs unittest since here the code already has access to a list of all...
docs unittest: Split into two separate test cases
The RAPI and hooks documentation aren't related at all.
docs unittest: Make _ReadDocFile a global function
rlib2: Declare all opcodes and equivalents
By declaring all used opcodes or opcodes equivalent to the operationsexecuted in a resource we will be able to ensure all opcodes are coveredby RAPI (with some exceptions).
baserlib: Function to retrieve opcodes used by handler
This will be used to verify opcodes used by RAPI.
htools: remove dead code
The tryEvac/evacuateInstance functions are no longer used in the newmulti-group world order, so we remove them and change the unit-test totest the actual IAllocator function.
hail: don't select the primary as new secondary
This just adds the primary node of the instance as 'non-allocable'during the choosing of the new secondary.
hail: add an extra safety check in relocate
If we select the primary as new secondary, better to fail than returnwrong data to Ganeti.
Fix RAPI documentation for gnt-instance console
Fix a failing pyassert in the RAPI docs and update it to reflect theaddition of SPICE to gnt-instance console.
Add SPICE compression and streaming options
Add the following SPICE audio/image compression and video streamingdetection hypervisor options:
Add SPICE support to gnt-instance console
Also update related unit tests.