config: Add unlocked function to get all node objects
This will be used for resolving node names.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
rpc: Remove _Static* and other unused functions
These are no longer needed.
rpc: Convert last two calls to generated code
These two calls, “upload_file” and “write_ssconf_files” are treatedseparated as they're used by the configuration, where we can't use thenormal resolver.
There's still some duplicated code in rpc.py, but that will be taken...
rpc: Convert call for HV parameter validation
Instead of filling the parameters in the RPC layer, that is now donebefore the wrapper is called, thereby simplifying the wrapper.
rpc: Convert two more instance-specific calls
Interface changes were necessary as these took more parameters than wereactually passed over the wire. Those parameters were just passed to theinstance serialization function.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Move bootstrap-related RPC to generated wrappers
With this patch, only 6 RPC are left as old-style code.
Convert job queue's RPC to generated code
With these changes job queue RPC will finally show up on the lockmonitor. See below for an example. A job queue-specific class is used torestrict the use of a static list for name resolution to the job queue.Further improvements can be made to not re-create the whole RPC client...
rpc: Convert wrappers for starting import/export daemons
rpc: Convert storage-related calls
rpc: Convert blockdev-related calls
rpc: Convert instance-related calls
rpc: Convert misc calls
rpc: Convert file-storage-related calls
rpc: Convert node-related calls
rpc: Convert OS-related calls
rpc: Convert X509 calls
rpc: Convert import/export functions
rpc: Add helpers
These will be used by the generated RPC wrappers.
rpc: Add support for generated RPC wrappers
Add RPC definitions
These will be used for replacing the tons and tons of boilerplate codein rpc.py.
rpc_definitions: Add helpers
These helpers will be used to convert incoming parameters toJSON-compatible types.
rpc: Disable timeout check
This is in preparation for the definition-based RPC model, wherethe timeout will be stored within the definition.
Add script to generate RPC wrappers
A new script, “autotools/build-rpc”, will be used to generate code forRPC client wrappers. This is done because “lib/rpc.py” contains lots andlots of boilerplate code. Forthcoming patches will start convertingRPC wrappers....
cmdlib: Use constant for DRBD meta device size
… instead of a hardcoded value.
Fix parameters to RPC "blockdev_rename"
Before this patch it would take a variable number of arguments, whichisn't very good for a definition-based approach.
Fix parameters to RPC "os_validate"
All other RPC wrappers take the node name(s) as the first parameter.
Un-revert comments in utils.mlock
These updates and clarifications are still useful.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Remove “node_tcp_ping” RPC
It is not used anywhere.
utils.ShellWriter: Don't indent empty lines
Empty lines shouldn't get indented. Unittest included.
http.client: Remove use of PycURL's “reset” function
We don't re-use cURL objects anymore, so there's no need to reset them.PycURL 7.19.0 has a reference counting bug leading to a crash aftera certain number of performed requests.
Since a unittest depended on this function for a test, it is replaced...
ovfconverter: use qemu-img path from constants
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge branch 'devel-2.5'
Conflicts: lib/cmdlib.py - trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix for buildbot pylint failure
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Allow per-hypervisor optional files
Rather than just allowing files for all nodes to be optional, we allowoptional files to be per-category. The way this works is that they mustbe included in both lists (the new code checks for this).
The code also removes a duplicate assert (present both in verify and...
Add hypervisors ancillary files list
These lists will be used to declare some of the files not necessarilyneeded on all nodes. The files selected are files without which thevarious hypervisors can still work, but that when they are presentshould be synchronized across the cluster (or node group)....
xen: abstract a few hardcoded strings as constants
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Check for qemu-img in configure and Makefile
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Upload spice files in redist-conf
Revert "Rename utils.mlock to utils.cfunc"
The rename is not needed either, since we're not adding more code as ofnow.This reverts commit 57ca011e1cd2681948969724e2646edaac22da28.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Revert "utils.cfunc: Cleanup, more flexibility"
We discussed that this is not needed right now, and it breaks existingfunctionality and unittests.
This reverts commit 6915fe26da8dce41fc967d761f005390aa956161.
Fix unittest failures with python 2.7
In python 2.7 the ovf unittests fail because OVFReader expectsElementTree.parse() of an erroneous document to throw anxml.parsers.expat.ExpatError while instead it throws anElementTree.ParseError.
The solution is to "except" for both errors, with the catch that...
Revert "rapi.client.ModifyNode should PUT rather than POST"
This was a mistake on my side because ModifyGroup and ModifyInstancewere PUT, and I was not aware of the discussion and the rationale whythis one had to be POST.
This reverts commit 55ef0cf6497c570aaab9413851435a7ee744222e....
Revert "Added SPICE TLS option and related cert paths"
This reverts commit bfe86c763a9ff1b481d799537ff0f0cf6740dfd1.This commit will be readded on master.
Revert "Implementation of TLS-protected SPICE connections"
This reverts commit b6267745ede04b3c943bc02e004bdb9347e0f564.This commit will be readded on master.
Revert "Add tls_ciphers and use_vdagent options"
This reverts commit 3e40b5879fa0070d6dd0e689dcfc31f20198a5a8.This commit will be readded on master.
rapi.client.ModifyNode should PUT rather than POST
This was caught (albeit in a sibylline manner) by unittests on masterwhich are not present in 2.5.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix RAPI node modify client and server calls
rapi.client.ModifyNode accepts a "group" and not a "node" param.(this bug is invisible but still not nice)
rlib2.R_2_nodes_name_modify submits the opcode with instance_name ratherthan node_name as a param. This would break the call....
xen: changes to facilitate "xl" support (xen 4.1)
- Copy the xl config file, in case there's any- Start instances by config file, not name (also xm compatible)- Start paused domains with p and not --paused (also xm compatible) Add a fixme for migration (changes are not xm compatible)...
xen: abstract instance config file naming
Abstract xen's 'xm' command as a constant
utils.cfunc: Cleanup, more flexibility
- Split code using ctypes directly into a helper class- Don't load “libc.so.6”, but use handle for main program instead (see comment in code)- Clarify comment on errno with older ctypes versions- Rename unittest since it can't be used for other functions (modifies...
Rename utils.mlock to utils.cfunc
Renaming so that more code using ctypes could be added to the same file.
rapi: Allow auto-promotion on node role change
rapi: Add resource for modifying node
A separate patch will add “auto-promote” through“/2/nodes/[node_name]/role”.
opcodes: Add comment to *SetParams result description
Explicitely say that the second element of the tuple is the new value.
constants: Verify exported names
The “constants” module is a bit special in the sense that we don't wantto export random stuff from it. This unittest checks the namingconvention and removes imported modules from the module's namespace.
http.client: Remove HTTP client pool code
This patch removes all remains of the HTTP client pool. Newly added unittestsprovide 96% coverage on http.client.
rpc: Remove thread-local storage with HTTP pool
The HTTP pool is no longer used.
Tiny optimisation related to filter parsing
Currently, we get a luxi Client, then parse the filter, then executethe query. If parsing the filter fails, we connected to the masterdneedlessly.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Merge branch 'stable-2.5' into devel-2.5
Standardise LUXI call argument types
Currently, we have 4 types of arguments in LUXI calls:
- most common, a list of values- a single argument that is sent as a list of one element- a single argument that is sent by itself- a dictionary (only Query and QueryFields)...
Rename filter and filter_ to qfilter
We currently use 'filter' as the OpCode, QueryRequest and RAPI fieldname for representing a query filter. However, since 'filter' is abuilt-in function, we actually have to use filter_ throughout the codein order to not override the built-in function....
Merge branch 'devel-2.4' into stable-2.5
rpc: Disable HTTP client pool and reduce memory consumption
We noticed that “ganeti-masterd” can use large amounts of memory,especially on large clusters. Measurements showed a single PycURL clientusing about 500 kB of heap memory (the actual usage depends on versions,...
Don't send gratuitous ARP if master IP setup fails
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add error codes documentation
Generalize docpp and sphinx_ext
Fixes for ovfconverter + vmware
Demote to warnings the errors in --ignore-errors
Treat the gnt-cluster verify errors identified by the error codes in--ignore-errors as warnings; just print a warning message for the user.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add --ignore-errors parameter to cluster verify
lib/cli.py- add IGNORE_ERROR_OPT;
client/gnt_cluster.py- pass the ignore_errors parameter to the opcodes
lib/opcode.py- update OpClusterVerifyConfig, OpClusterVerify and OpClusterVerifyGroup to accept the ignore_errors parameter...
Move cluster verify error codes to constants
- move the cluster verify error codes from cmdlib._VerifyErrors to constants;- add to each of them the CV (Cluster Verify) prefix;- add the CV_ALL_ECODES and CV_ALL_ECODES_STRINGS constants;- wrap the lines that exceed 80 characters after changing the error...
Restore backend.GetMasterInfo return values order
Change 5a8648eb609f7e3a8d7ad7f82e93cfdd467a8fb5 changed the order of thereturn values of backend.GetMasterInfo(). This broke the users of themaster_info RPC.
This change restores the original order, and adds a comment in...
Add cluster netmask parameter
Add the master_netmask cluster parameter, that represents the netmask ofthe master IP, encoded as a CIDR suffix.
This parameter can be set via the --master-netmask of gnt-cluster initand gnt-cluster modify. The default behaviour is to be consistent with...
Add ValidateNetmask and GetClass IPAddress methods
Also, add related tests to the test suite.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>...
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix issue when verifying cluster files
If a cluster has any non-master-candidate nodes, those don't contain allfiles (e.g. config.data). With commit aef59ae764dc (March 31st, 2011)the logic was changed and subsequently verifying a cluster with non-mcnodes would complain....
Revert "utils.log: Write error messages to stderr"
This reverts commit 34aa8b7c4bb6f5e2e788108e024c9cd70bdb3431. Writingerror messages to stderr would also include backtraces, something wetried to avoid in the past.
Fix adding nodes after commit 64c7b3831dc
Commit 64c7b3831dc changed the RPC call for verifying SSH connections.Unfortunately this case in adding nodes was missed.
LUClusterVerifyGroup: Spread SSH checks over more nodes
When verifying a group the code would always check SSH to all nodes inthe same group, as well as the first node for every other group. On bigclusters this can cause issues since many nodes will try to connect to...
Optimise cli.JobExecutor with many pending jobs
In the case we submit many pending jobs (> 100) to the masterd, theJobExecutor 'spams' the master daemon with status requests for thestatus of all the jobs, even though in the end it will only choose asingle job for polling....
Add gnt-cluster commands to toggle the master IP
Split starting and stopping master IP and daemons
ssh: Quote strings in error message
utils.log: Write error messages to stderr
When “gnt-cluster copyfile” failed it would only print “Copy of file …to node … failed”. A detailed message is written using logging.error.Writing error messages to stderr can be helpful in figuring out whatwent wrong (the messages also go to the log file, but not everyone might...
Adapt non-KVM hypervisors to new migration RPCs
Add memory transfer progress info to migration
Make migration RPC non-blocking
To add status reporting for the KVM migration, the instance_migrate RPCmust be non-blocking. Moreover, there must be a way to represent themigration status and a way to fetch it.
Move _TimeoutExpired to utils
Migration: warn the user about hv version mismatch
Fix handling of cluster verify hooks
The change to enforce boolean results for cluster verify group opcodemissed the HooksCallBack, which uses a very ugly 1/0logic. Furthermore, the logic is wrong, since it unconditionallyresets the verify result to true....
http.client: Show pending requests as “owner”
In the context of the lock monitor a “pending” item does not yet own therequested resource. Since these HTTP requests are already undergoingthey should be shown as owners.
http.client: Add nice name to requests
With this change a node name instead of the IP address can be shown forpending RPC requests:Name Pendingrpc/node18.example.com/test_delay thread:Jq1/Job692/TEST_DELAY