utils.cfunc: Cleanup, more flexibility
- Split code using ctypes directly into a helper class- Don't load “libc.so.6”, but use handle for main program instead (see comment in code)- Clarify comment on errno with older ctypes versions- Rename unittest since it can't be used for other functions (modifies...
Rename utils.mlock to utils.cfunc
Renaming so that more code using ctypes could be added to the same file.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
constants: Verify exported names
The “constants” module is a bit special in the sense that we don't wantto export random stuff from it. This unittest checks the namingconvention and removes imported modules from the module's namespace.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
http.client: Remove HTTP client pool code
This patch removes all remains of the HTTP client pool. Newly added unittestsprovide 96% coverage on http.client.
rpc: Remove thread-local storage with HTTP pool
The HTTP pool is no longer used.
Merge branch 'devel-2.5'
Tiny optimisation related to filter parsing
Currently, we get a luxi Client, then parse the filter, then executethe query. If parsing the filter fails, we connected to the masterdneedlessly.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Merge branch 'stable-2.5' into devel-2.5
Standardise LUXI call argument types
Currently, we have 4 types of arguments in LUXI calls:
- most common, a list of values- a single argument that is sent as a list of one element- a single argument that is sent by itself- a dictionary (only Query and QueryFields)...
Rename filter and filter_ to qfilter
We currently use 'filter' as the OpCode, QueryRequest and RAPI fieldname for representing a query filter. However, since 'filter' is abuilt-in function, we actually have to use filter_ throughout the codein order to not override the built-in function....
Merge branch 'devel-2.4' into stable-2.5
rpc: Disable HTTP client pool and reduce memory consumption
We noticed that “ganeti-masterd” can use large amounts of memory,especially on large clusters. Measurements showed a single PycURL clientusing about 500 kB of heap memory (the actual usage depends on versions,...
Don't send gratuitous ARP if master IP setup fails
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add error codes documentation
Generalize docpp and sphinx_ext
Fixes for ovfconverter + vmware
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Demote to warnings the errors in --ignore-errors
Treat the gnt-cluster verify errors identified by the error codes in--ignore-errors as warnings; just print a warning message for the user.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add --ignore-errors parameter to cluster verify
lib/cli.py- add IGNORE_ERROR_OPT;
client/gnt_cluster.py- pass the ignore_errors parameter to the opcodes
lib/opcode.py- update OpClusterVerifyConfig, OpClusterVerify and OpClusterVerifyGroup to accept the ignore_errors parameter...
Move cluster verify error codes to constants
- move the cluster verify error codes from cmdlib._VerifyErrors to constants;- add to each of them the CV (Cluster Verify) prefix;- add the CV_ALL_ECODES and CV_ALL_ECODES_STRINGS constants;- wrap the lines that exceed 80 characters after changing the error...
Restore backend.GetMasterInfo return values order
Change 5a8648eb609f7e3a8d7ad7f82e93cfdd467a8fb5 changed the order of thereturn values of backend.GetMasterInfo(). This broke the users of themaster_info RPC.
This change restores the original order, and adds a comment in...
Add cluster netmask parameter
Add the master_netmask cluster parameter, that represents the netmask ofthe master IP, encoded as a CIDR suffix.
This parameter can be set via the --master-netmask of gnt-cluster initand gnt-cluster modify. The default behaviour is to be consistent with...
Add ValidateNetmask and GetClass IPAddress methods
Also, add related tests to the test suite.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>...
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix issue when verifying cluster files
If a cluster has any non-master-candidate nodes, those don't contain allfiles (e.g. config.data). With commit aef59ae764dc (March 31st, 2011)the logic was changed and subsequently verifying a cluster with non-mcnodes would complain....
Revert "utils.log: Write error messages to stderr"
This reverts commit 34aa8b7c4bb6f5e2e788108e024c9cd70bdb3431. Writingerror messages to stderr would also include backtraces, something wetried to avoid in the past.
Fix adding nodes after commit 64c7b3831dc
Commit 64c7b3831dc changed the RPC call for verifying SSH connections.Unfortunately this case in adding nodes was missed.
LUClusterVerifyGroup: Spread SSH checks over more nodes
When verifying a group the code would always check SSH to all nodes inthe same group, as well as the first node for every other group. On bigclusters this can cause issues since many nodes will try to connect to...
Optimise cli.JobExecutor with many pending jobs
In the case we submit many pending jobs (> 100) to the masterd, theJobExecutor 'spams' the master daemon with status requests for thestatus of all the jobs, even though in the end it will only choose asingle job for polling....
Add gnt-cluster commands to toggle the master IP
Split starting and stopping master IP and daemons
ssh: Quote strings in error message
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
utils.log: Write error messages to stderr
When “gnt-cluster copyfile” failed it would only print “Copy of file …to node … failed”. A detailed message is written using logging.error.Writing error messages to stderr can be helpful in figuring out whatwent wrong (the messages also go to the log file, but not everyone might...
Adapt non-KVM hypervisors to new migration RPCs
Add memory transfer progress info to migration
Make migration RPC non-blocking
To add status reporting for the KVM migration, the instance_migrate RPCmust be non-blocking. Moreover, there must be a way to represent themigration status and a way to fetch it.
Move _TimeoutExpired to utils
Migration: warn the user about hv version mismatch
Fix handling of cluster verify hooks
The change to enforce boolean results for cluster verify group opcodemissed the HooksCallBack, which uses a very ugly 1/0logic. Furthermore, the logic is wrong, since it unconditionallyresets the verify result to true....
http.client: Show pending requests as “owner”
In the context of the lock monitor a “pending” item does not yet own therequested resource. Since these HTTP requests are already undergoingthey should be shown as owners.
http.client: Add nice name to requests
With this change a node name instead of the IP address can be shown forpending RPC requests:Name Pendingrpc/node18.example.com/test_delay thread:Jq1/Job692/TEST_DELAY
rpc/http: Show pending RPC requests in lock monitor
Not all requests use an instance of RpcRunner yet and therefore won'tshow up (only instances have access to the global Ganeti context).Currently only the IP address is accessible. Another patch will add a...
http.client: Factorize code interacting with cURL
This simplifies HttpClientPool.ProcessRequests significantly and will behandy for showing pending RPC requests in the lock monitor.
Redistribute the RAPI certificate
This reverts to the old behaviour in Ganeti 2.4 and before.
http.client: Reduce performance impact by assertion
Call dict.values once instead of N times.
rpc: Overhaul client structure
- Clearly separate node name to IP address resolution into separate functions- Simplified code structure (one code path instead of several)- Fully unittested- Preparation for more RPC improvements
rpc: Make compression function module-global
No need to keep it in the class.
Keep only one global RPC runner in Ganeti context
Instead of having one RPC runner per mcpu processor this will keep onlyone instance as part of the masterd-wide Ganeti context. Upcomingpatches will change the RPC runner to report pending requests to the...
TemporaryFilesManager implementation
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Export: saving data to ovf file
Export: parsing data from config file
Export: initial commit - manifest, ova creation etc
Import: backend, hypervisor and os
Import: networks
Import: disk conversion
Import: reading ovf file
Initial commit for ovfconverter tool
serializer: Add comment about simplejson vs. built-in json
Revert "Fail if dictionary uses invalid keys" and "Support newer “json” module"
This reverts commit fd0351aef246f5d36e641209429e2ec093d325f8 and9869e771704ada62bab001e729c52a36525ef081. The built-in module is a lotslower in Python 2.6.
serializer: Fail if dictionary uses invalid keys
JSON only supports a very restricted set of types for dictionary keys,among them strings, booleans and “null”. Integers and floats areconverted to strings. Since this can cause a lot of confusion in Python,...
serializer: Support newer “json” module
This module is included from Python 2.6 and is based onsimplejson.
Fix interaction between CPU pinning and KVM migration
CPU pinning requires the KVM hypervisor to start in the paused state, inorder to retrieve information, and immediately unpauses it.
This does not play well with live migration, as the unpausing was done...
RAPI: Fix wrong check on instance shutdown
Commit 7fa310f6d84 (April 1st, 2011) converted the RAPI resource forshutting down an instance to FillOpCode. Unfortunately it missed thefact that the shutdown resource gets its parameters as query arguments.
baserlib: Accept empty body in FillOpcode
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>(cherry picked from commit c6e1a3eef05674d637570c39f25a799cec7ba187)
Signed-off-by: Michael Hanselmann <hansmi@google.com>
RAPI: Add resource to powercycle node
Add tls_ciphers and use_vdagent options
Implementation of TLS-protected SPICE connections
Added support for TLS-protected SPICE connections:
Added SPICE TLS option and related cert paths
Fix OS creation's error handling when pausing sync
Commit 41e1e79 introduced a feature in which when wait_for_sync is notset, DRBD sync is paused during the OS installation.
Doing so, however, broke OS creation's error handling: the result valuefrom the instance_os_add RPC call was overwritten by the one of the...
Fix two pylint errors
- hv_kvm.py: silence F0401, that is raised if pylint does not find the affinity module- rlib2.py: change disable-msg to disable
Fix backend.MigrateInstance docs
The MigrateInstance function does not return anything, so the relevantlines are removed from the documentation. Instead, the raised exceptionis documented.
hv_kvm: bugfix
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
RAPI: Add resource to recreate instance's disks
This was still missing from RAPI.
hv_xen: fix use of CPU pinning constants
… to be consistent with hv_kvm
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hv_kvm: fix hardcoded KVM command string
hv_kvm: support for CPU pinning
Signed-off-by: Tsachy Shacham <tsachy@google.com>Signed-off-by: Iustin Pop <iustin@google.com>[iustin@google.com: fixed some small code and style issues]Reviewed-by: Iustin Pop <iustin@google.com>
constants: support for CPU pinning under KVM
rlib2: Declare all opcodes and equivalents
By declaring all used opcodes or opcodes equivalent to the operationsexecuted in a resource we will be able to ensure all opcodes are coveredby RAPI (with some exceptions).
baserlib: Function to retrieve opcodes used by handler
This will be used to verify opcodes used by RAPI.
Add SPICE compression and streaming options
Add the following SPICE audio/image compression and video streamingdetection hypervisor options:
Add SPICE support to gnt-instance console
Also update related unit tests.
rapi: Re-add “/2” resource
Like “/”, it'll just return a JSON null.
rlib2: Convert /2/instances to OpcodeResource
rlib2: Convert /2/*/tags to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/storage/repair to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/storage/modify to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/storage to OpcodeResource
rlib2: Convert /2/groups/[group_name] to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/role to OpcodeResource
Also fix a typo in a constant name.
rlib2: Convert /2/instances/[inst]/disk/[idx]/grow to OpcodeResource
rlib2: Convert /2/instances/[inst]/modify to OpcodeResource
rlib2: Convert /2/instances/[inst]/rename to OpcodeResource
rlib2: Convert /2/instances/[inst]/failover to OpcodeResource
rlib2: Convert /2/instances/[inst]/migrate to OpcodeResource