Show RPC calls from config in lock monitor
With this patch all RPC calls at runtime of masterd will show up in thelock monitor. There is a chicken-and-egg issue with initializing theconfiguration with a context since the lock manager, containing themonitor, requires the configuration. This is worked around by setting...
Derive IP hooks env variables from RPC parameter
Let the environment variables of the master IP turnup/turndown bederived from the parameter of the RPC itself (that is of typeobjects.MasterNetworkParameters in both cases).
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>...
Pass MasterNetworkParameters instances in RPCs
Pass instances of objects.MasterNetworkParameters when calling RPCs foractivation and deactivation of master IP.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Use MasterNetworkParameters attributes for RPC
Instead of manually unpacking the return values ofcfg.GetMasterNetworkParameters, let it return an instance ofobjects.MasterNetworkParameters and pass its attributes.
Add object representing the master network params
Uniform master IP activation and deactivation
Add the master IP family parameter to the master IP deactivation RPCs,so that the activation and deactivation interfaces are uniform.
Make it possible to pass in flags using ENV variables
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
utils.algo: Add a function to insert a list into a list
Fix overriding of ActivateMasterIp params
In the process of reworking the patches for master, I forgot to remove acall to GetMasterInfo() that overrides the parameters passed via RPC.This patch fixes this issue.
Add netmask and IP version to master IP hooks env
Add the master netmask and master IP version parameters to the hooks anddocument them. In this way, the hook environment building function canbe reused for the planned master IP turnup changes.
Explicitly pass params to activate_master_ip
To remove the usage of ssconf in the backend, the master needs to pushthe parameters of activate_master_ip to the backend.
This patch changes the entire call path of activate_master_ip to use thenew interface....
Change the semantics of stop_master RPC
To avoid polluting the stop_master RPC interface with networkparameters, this patch removes the deactivation of the master IP fromthis RPC.
The call to deactivate_master_ip is then added before every stop_master...
Explicitly pass params to deactivate_master_ip
Make the master explicitly pass the parameters to thedeactivate_master_ip RPC, and change all the call flow to use the newinterface.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Explicitly pass params to change_master_netmask
Make the master explicitly pass the parameters to thechange_master_netmask RPC, and change all the call flow to use the newinterface.
locking: Make level names consistent, add check
- Change all lock level names to their singular form- Add assertion checking consistency between level names and lockset name
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add new lock level for node resource modifications
This is in preparation for implementing a new resource model.
Add master IP turnup and turndown hooks
Add RunLocalHooks decorator
Add the RunLocalHooks decorator, that allows the execution of hookslocally. Also, add a RunLocalHooks method to HooksRunner, to adapt thesignature of HooksRunner.RunHooks to the one expected by HooksMaster,and also to check that the hooks are being executed locally....
Generalize HooksMaster
- remove any dependence on Logical Units from the HooksMaster;- add a new function parameter to the constructor, a function that is expected to convert the results of the hooks execution in a format understood by the HooksMaster;...
RAPI: Fix issue when replacing disks
This is another fallout from a suboptimal merge.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
rpc: Improve argument definitions
Instead of in-place code now each argument has an (optional) kindassigned. The RPC client will then take care of encoding each argumentaccording to its definition. A new base (_RpcClientBase) is added totake care of encoding and dispatching calls....
rpc: Change {import,export}_start to take source/dest in single argument
This simplifies the RPC argument encoding code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Rename lib/build/rpc_definitions.py to lib/rpc_defs.py
It will also be used at runtime, not just for building.
Merge branch 'devel-2.5'
Add missing get_migration_status RPC definition
Add the instance_get_migration_status RPC definition and the neededhelper.
cmdlib: Fix issue when marking node as online
When a node is marked as online (“gnt-node modify -O no …”), an RPC ismade to the node to check whether the node daemon is running. My recentRPC changes led to offline nodes being ignored before the actual call is...
Add shortcut for network-related cluster params
Use GetClassFromIpFamily in ActivateMasterIp
rpc: Remove leftovers from timeout table
config: Add unlocked function to get all node objects
This will be used for resolving node names.
rpc: Remove _Static* and other unused functions
These are no longer needed.
rpc: Convert last two calls to generated code
These two calls, “upload_file” and “write_ssconf_files” are treatedseparated as they're used by the configuration, where we can't use thenormal resolver.
There's still some duplicated code in rpc.py, but that will be taken...
rpc: Convert call for HV parameter validation
Instead of filling the parameters in the RPC layer, that is now donebefore the wrapper is called, thereby simplifying the wrapper.
rpc: Convert two more instance-specific calls
Interface changes were necessary as these took more parameters than wereactually passed over the wire. Those parameters were just passed to theinstance serialization function.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Move bootstrap-related RPC to generated wrappers
With this patch, only 6 RPC are left as old-style code.
Convert job queue's RPC to generated code
With these changes job queue RPC will finally show up on the lockmonitor. See below for an example. A job queue-specific class is used torestrict the use of a static list for name resolution to the job queue.Further improvements can be made to not re-create the whole RPC client...
rpc: Convert wrappers for starting import/export daemons
rpc: Convert storage-related calls
rpc: Convert blockdev-related calls
rpc: Convert instance-related calls
rpc: Convert misc calls
rpc: Convert file-storage-related calls
rpc: Convert node-related calls
rpc: Convert OS-related calls
rpc: Convert X509 calls
rpc: Convert import/export functions
rpc: Add helpers
These will be used by the generated RPC wrappers.
rpc: Add support for generated RPC wrappers
Add RPC definitions
These will be used for replacing the tons and tons of boilerplate codein rpc.py.
rpc_definitions: Add helpers
These helpers will be used to convert incoming parameters toJSON-compatible types.
rpc: Disable timeout check
This is in preparation for the definition-based RPC model, wherethe timeout will be stored within the definition.
Add script to generate RPC wrappers
A new script, “autotools/build-rpc”, will be used to generate code forRPC client wrappers. This is done because “lib/rpc.py” contains lots andlots of boilerplate code. Forthcoming patches will start convertingRPC wrappers....
Move RenameFile to the new functions
ensure_dirs: Move some useful functions into utils.
With this change we can easily reuse this functionality where it makessense on other parts of Ganeti.
cmdlib: Use constant for DRBD meta device size
… instead of a hardcoded value.
Use JoinDisjointDicts in mcpu
Add the JoinDisjointDicts function to utils.algo
Add a function that joins two dictionaries, enforcing the constraintthat the two key sets should be disjoint. Also, add unit tests for thisfunction.
Fix parameters to RPC "blockdev_rename"
Before this patch it would take a variable number of arguments, whichisn't very good for a definition-based approach.
Fix parameters to RPC "os_validate"
All other RPC wrappers take the node name(s) as the first parameter.
Un-revert comments in utils.mlock
These updates and clarifications are still useful.
Remove “node_tcp_ping” RPC
It is not used anywhere.
utils.ShellWriter: Don't indent empty lines
Empty lines shouldn't get indented. Unittest included.
Merge branch 'stable-2.5' into devel-2.5
Merge branch 'devel-2.4' into stable-2.5
http.client: Remove use of PycURL's “reset” function
We don't re-use cURL objects anymore, so there's no need to reset them.PycURL 7.19.0 has a reference counting bug leading to a crash aftera certain number of performed requests.
Since a unittest depended on this function for a test, it is replaced...
ovfconverter: use qemu-img path from constants
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Conflicts: lib/cmdlib.py - trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix queue archive creation with wrong permissions
On a master failover some of the archive dirs might have wrongpermissions in the non-root model. This is due to the nature of nodedstill running as root and the job queue is synced that way. This patchwill fix this behaviour by setting the permissions accordingly....
Fix for buildbot pylint failure
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Allow per-hypervisor optional files
Rather than just allowing files for all nodes to be optional, we allowoptional files to be per-category. The way this works is that they mustbe included in both lists (the new code checks for this).
The code also removes a duplicate assert (present both in verify and...
Add hypervisors ancillary files list
These lists will be used to declare some of the files not necessarilyneeded on all nodes. The files selected are files without which thevarious hypervisors can still work, but that when they are presentshould be synchronized across the cluster (or node group)....
xen: abstract a few hardcoded strings as constants
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Check for qemu-img in configure and Makefile
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Upload spice files in redist-conf
Revert "Rename utils.mlock to utils.cfunc"
The rename is not needed either, since we're not adding more code as ofnow.This reverts commit 57ca011e1cd2681948969724e2646edaac22da28.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Revert "utils.cfunc: Cleanup, more flexibility"
We discussed that this is not needed right now, and it breaks existingfunctionality and unittests.
This reverts commit 6915fe26da8dce41fc967d761f005390aa956161.
Fix unittest failures with python 2.7
In python 2.7 the ovf unittests fail because OVFReader expectsElementTree.parse() of an erroneous document to throw anxml.parsers.expat.ExpatError while instead it throws anElementTree.ParseError.
The solution is to "except" for both errors, with the catch that...
Ensure permission on the job queue version file
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
OpGroupVerifyDisks: Fix wrong result type declaration
If an instance had actually a missing disk, the type check would fail.
RAPI: Make node evacuation actually work
Commit e1f23243 changed te LU and opcode for node evacuation to receivea “mode” parameter (among other things). Commit de40437a changed theRAPI code accordingly, but did so for an earlier version of the firstpatch. Obviously this couldn't work, so here's the fix....
Revert "rapi.client.ModifyNode should PUT rather than POST"
This was a mistake on my side because ModifyGroup and ModifyInstancewere PUT, and I was not aware of the discussion and the rationale whythis one had to be POST.
This reverts commit 55ef0cf6497c570aaab9413851435a7ee744222e....
Revert "Implementation of TLS-protected SPICE connections"
This reverts commit b6267745ede04b3c943bc02e004bdb9347e0f564.This commit will be readded on master.
Revert "Added SPICE TLS option and related cert paths"
This reverts commit bfe86c763a9ff1b481d799537ff0f0cf6740dfd1.This commit will be readded on master.
Revert "Add tls_ciphers and use_vdagent options"
This reverts commit 3e40b5879fa0070d6dd0e689dcfc31f20198a5a8.This commit will be readded on master.
rapi.client.ModifyNode should PUT rather than POST
This was caught (albeit in a sibylline manner) by unittests on masterwhich are not present in 2.5.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix RAPI node modify client and server calls
rapi.client.ModifyNode accepts a "group" and not a "node" param.(this bug is invisible but still not nice)
rlib2.R_2_nodes_name_modify submits the opcode with instance_name ratherthan node_name as a param. This would break the call....
xen: changes to facilitate "xl" support (xen 4.1)
- Copy the xl config file, in case there's any- Start instances by config file, not name (also xm compatible)- Start paused domains with p and not --paused (also xm compatible) Add a fixme for migration (changes are not xm compatible)...
Abstract xen's 'xm' command as a constant
xen: abstract instance config file naming
utils.cfunc: Cleanup, more flexibility
- Split code using ctypes directly into a helper class- Don't load “libc.so.6”, but use handle for main program instead (see comment in code)- Clarify comment on errno with older ctypes versions- Rename unittest since it can't be used for other functions (modifies...
Rename utils.mlock to utils.cfunc
Renaming so that more code using ctypes could be added to the same file.
RAPI: Fix resource for replacing disks
Commit d1c172deb4f inadvertently changes the“/2/instances/[instance_name]/replace-disks” resource to use bodyparameters. There were no QA tests and the issue wasn't noticed.
This patch re-introduces support for query parameters and adds a QA...
rapi: Allow auto-promotion on node role change
rapi: Add resource for modifying node
A separate patch will add “auto-promote” through“/2/nodes/[node_name]/role”.
opcodes: Add comment to *SetParams result description
Explicitely say that the second element of the tuple is the new value.
constants: Verify exported names
The “constants” module is a bit special in the sense that we don't wantto export random stuff from it. This unittest checks the namingconvention and removes imported modules from the module's namespace.
http.client: Remove HTTP client pool code
This patch removes all remains of the HTTP client pool. Newly added unittestsprovide 96% coverage on http.client.
rpc: Remove thread-local storage with HTTP pool
The HTTP pool is no longer used.