History | View | Annotate | Download (46.4 kB)
Fix RPC mismatch in blockdev_getsize[s]
Commit 92fd2250 added consistency checks in the RPC layer, which brokethe call_blockdev_getsizes RPC call (declared with 's' at the end inrpc.py, without 's' in the node daemon).
The immediate fix is to correct the rpc function name, the long term...
Re-create instance disk symlinks on activate
This patch implements recreation of instance disk symlinks when theactivate-disks operation is run. Until now, it was not possible tore-create these symlinks without stopping and starting or migrating aninstance as the RPC call where this is done was in instance startup...
Add RPC calls for pause/resume sync of devices
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RPC: mark jobqueue functions as URGENT
Recently, we've seen more and more cases of a specific breakagepattern in Ganeti: master candidates which are semi-alive (as in, theyrespond to ping, they can complete a TCP/SSL handshake, but otherwisethe root filesystem is broken) cause lots of confusion within masterd....
RpcResult: simplify some asserts
data ≫ code, eom.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Adding missing timeout parameter to rpc interface
Adding RPC infrastructure to call oob
Fix disk checks in “gnt-cluster verify”
Tests have shown that the changes in commit b8d26c6e5 don't work aswanted. If any disk wasn't found on the node, all disks located on thesame node would show as faulty. The cause was incorrect exceptionhandling on the node....
LUClusterVerify: Complain if disk is marked faulty
This will show a warning if, for example, one side of a DRBDdisk becomes unavailable. The data is collected separatelyfrom the other verification data.
Example output:
rpc: Work around epydoc warning
Aliasing the “threading” module allows us to avoid the “No informationavailable for ganeti.rpc._RpcThreadLocal's base threading.local” warningby epydoc.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Adding RPC call for blockdev_wipe
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
gnt-instance reinstall: Allow overriding OS parameters
This allows OS installation scripts to make use of special parameters,e.g. to retain some data on reinstallation.
The RAPI resource is not updated as it takes all parameters via thequery string and encoding arbitrary data in a query string is tricky....
Merge branch 'devel-2.2'
Conflicts: lib/rpc.py (trivial, copyright header)
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
RPC: disable curl's Expect header
This patch solves the very slow (~8-9 seconds) gnt-instance modifybehaviour. Well, it solves in general the slow RPC behaviour, but it wasmost visible in that LU.
It seems that curl's behaviour with regard to file uploads (via PUT) and...
Add RPC calls to update /etc/hosts
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix a few epydoc docstrings
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Removing all ssh setup code from the core
Support IPv6 node add
Support for resolving hostnames to IPv6 addresses
This patch enables IPv6 name resolution by using socket.getaddrinfoinstead of socket.gethostbyname_ex.
It renames the HostInfo class to Hostname and unifies its use throughoutthe code. This is achieved by using static calls where no object is...
Always use address instead of hostname in rpc.Client
In light of the upcoming IPv6 support, this patch enables the rpc.Clientto always use a node's address to connect to it. This is necessary as wedo not want to rely on name resolution to connect to the correct IP...
Move the UsesRPC decorator from cli to rpc
This is needed because not just the cli scripts need this decorator, butthe master daemon too (and it already duplicated the code once).
In cli.py we just leave a stub, so that we don't have to modify all thescripts to import rpc.py....
Convert RPC client to PycURL
Instead of using our custom HTTP client, using PycURL's multiinterface allows us to get rid of the HTTP client threadpool.The majority of the code is still in the ganeti.http.clientmodule.
A simple per-thread HTTP client pool gives cURL a chance to...
Introduce lib/netutils.py
This patch moves network utility functions to a dedicated module.
Add drbd_helper rpc call
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add OS parameters to cluster and instance objects
The patch also modifies the instance RPC calls to fill the osparameterscorrectly with the cluster defaults, and exports the OS parameters inthe instance/OS environment.
Signed-off-by: Iustin Pop <iustin@google.com>...
Introduce an RPC call for OS parameters validation
While we only support the 'parameters' check today, the RPC call isgeneric enough that will be able to support other checks in the future.The backend function will both validate the parameters list (so as to...
Merge branch 'devel-2.1' into master
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Fix RpcResult.Raise error code
A typo in the Raise() method of rpc.RpcResult means that any remoteerrors will lack an appropriate error code; this will confuse e.g. RAPIusers.
Remove the job queue drain rpc call
This call was introduced but never used. In two years.Since it's just creating/removing a file it can also be in simpler ways,without a special rpc call, if/when we need it again. In the meantime,let's give it to history....
Introduce harcdoded timeouts for each RPC call
This patch adds a table with per-opcode timeouts. They were chosen in anempiric, rather than scientific, way - see the comments in lib/rpc.py.
The patch also shows how custom timeouts can be used - call_test_delay...
http client: support per-request read timeout
Currently, the read timeout is hardcoded in theHttpClientRequestExecutor class. The patch changes the timeout so thatit's a per-request property, and makes the rpc.Client class pass oneexplicitly in. Furthermore, we modify the rpc.RpcRunner class to support...
Put common import/export daemon options into object
The X509 key name and CA are passed from cmdlib all the way tothe backend import/export daemon. With the addition of an optionto choose the compression method, another parameter would haveto be passed all the way. By moving these options to a separate...
Remove two unused RPC functions
Both of these functions, “snapshot_export” and “instance_os_import”,have been replaced by the instance import/export daemon.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Complete what was begun in commit ef40fbfb4
The X509 certificate RPC functions were also renamed, but Iforgot to rename them in rpc.py. Fortunately they aren't usedyet.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add RPC call to send SIGTERM to import/export daemon
This will be used to stop the daemon without doing complete cleanup (yet).
Rename import/export RPC calls to match others
Add RPC calls to import and export instance data
These RPC calls can be used to start, monitor and stop the instance dataimport/export daemon.
Add RPC calls to create and remove X509 certificates
Certificates and keys generated using these functions will be used forinter-cluster instance moves. As per design, the private key should neverleave the node.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Rename SSL_CERT_FILE to NODED_CERT_FILE
To be consistent with RAPI_CERT_FILE, the rather generic named“SSL_CERT_FILE” constant is renamed to “NODED_CERT_FILE”. The actual filename is not changed.
Implement debug level across OS-related RPC calls
This doesn't implement the full functionality, we need to add the debuglevel to the opcodes too, but at least won't require changing the RPCcalls during the 2.1 series.
Add targeted pylint disables
This patch should have only:
- pylint disables- docstring changes- whitespace changes
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Merge branch 'devel-2.0' into devel-2.1
Add targetted pylint disables
This patch adds targeted pylint disables, where it makes sense (eitherdue to limitations in pylint or due to historical usage), and also a fewblanket ones in rapi where all the names are… “different”.
Add check for OpenSSL entropy status
By checking for this explicitly, the errors (SSLEAY_RAND_BYTES, “PRNGnot seeded”) will happen in the start-up phase of the daemon and notonly when executing remote procedure calls.
Add ecode to rpc.py's RpcResult.Raise()
This patch adds a new ecode argument to RpcResult.Raise(). This allowsspecifying the error code (for both OpExec and OpPrereq errors).
Note that this patch also makes the OpExecError exceptions raised from_FindFaultInstanceDisks have the error code classification....
Make cluster initialization more reliable
There was a race condition between starting the node daemonand sending requests to write the ssconf files. With thispatch, the initialization waits up to ten seconds for thenode daemon to become responsive.
Adding '--no-ssh-init' option to 'gnt-cluster init'.
Allows the initialization of a cluster without the creation or distributionof SSH key pairs. Includes changes for LeaveCluster and RPC.
Signed-off-by: Ken Wehr <ksw@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>...
Ensure RpcResult has “payload” attribute
Also add assertions to avoid missing attributes in the future.They won't be included in optimized bytecode.
Add timeout options to other LUs
All the LUs that shut down the instance need to be able too pass thetimeout parameter as well.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Accept shutdown timeout from the user
Using the new --timeout option:
- gnt-instance shutdown is changed to accept a timeout- the opcode is changed to hold one- the LU is changed to optionally get one- the rpc is changed to carry one- the backend is changed to take it as a parameter rather than...
Fix rpc.call_os_get to actually return the OS
Since nobody ever read the actual OS object, this bug was introduced inthe rpc conversion.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Remove RpcResult.RemoteFailMsg completely
Remove RpcResult.failed attribute
It's redundant with RpcResult.fail_msg. This change survived QA, burnin andunittests.
RpcResult: Fix cases where “failed” wouldn't be set to True
This broke “gnt-instance replace-disks --auto” when the instanceis down.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Add disk copy support at backend and the rpc level
This uses a simple 'dd if=… | ssh $target dd of=…' method, like theExportSnapshot (which uses the OS export; here we want full disk-levelcopy and not any FS-level changes).
Add RPC call for storage operations
Merge commit 'origin/next' into branch-2.1
Merge branch 'master' into next
Use objects for blockdev_getmirrorstatus RPC call result
This patch changes the return type for backend.BlockdevGetmirrorstatus froma list of tuples to a list of objects.BlockDevStatus instances.
Use object for blockdev_find RPC call result
This patch changes the return type for backend.BlockdevFind to an object(objects.BlockDevStatus). Before a tuple was used. Adding more values tothis tuple causes a lot of work. Converting the result to an object with...
rpc: add rpc call for getting disk size
Note that this exports the disk size as bdev returns it, in bytes. Thevalue will be converted to MiB in cmdlib.
Add RPC calls to modify storage fields
Add RPC calls for storage unit list
Extend call_node_start_master rpc with no_voting
When the parameter is set to True and start_daemons is also True,ganeti-masterd will be started with the new --no-voting --yes-do-itoptions.
This new option is set to True only on masterfailover, when no_voting is...
Change GetNodeDaemonPort to GetDaemonPort in utils
GetNodeDaemonPort is used to lookup the node daemon port in the servicesfile, and if not found to return the default one. We make it a genericfunction, which accepts the daemon name in input, so that it can be used...
Conflicts: lib/cli.py: trivial extra empty line
Fix a couple of epydoc warnings
It seems epydoc needs fully-qualified references, and doesn't deal withrelative ones (not even in the current module) if there are anyambiguities.
There are other epydoc warnings, in the rapi docstrings, but those areleft as-is as they're removed in 2.1....
Merge branch 'next' into branch-2.1
Fix pylint warnings
Fix some typos
Rename the volume_list RPC call to lv_list
There are volume-related rpc calls. This patch renames the ‘volume_list’call to ‘lv_list’ to make more clear its purpose.
Simplify RPC call result check in cmdlib.py
Now that all results are the same, we can even more simplify thehandling in cmdlib more. Almost all if result.RemoteFailMsg()…constructs are similar, and we resurect the RpcResult.Raise() functionto take a message argument, which it will process and raise an...
Pre-compute error status in RpcResult
Since now all RPCs have the new style result type, we can pre-computethe error message at RpcResult() init time and not wait until it'srequested.
Convert os_diagnose rpc to new style result
This also removes custom post-processing from rpc.py; since this callhas only one user, it was simple to move it back to the caller.
Convert node_info rpc to new style result
This patch also does some cleanup and enforces valid results (withproper type, i.e. int for memory/disk values) from remote node,otherwise we handle the result as failure.
We do this so that we can remove custom processing in rpc.py which is...
Convert export_info rpc to new style result
This also removes some code from ganeti-noded and rpc.py, which shouldnot do such processing of data (and be simply glue code). (Oralternatively they could, if we had better infrastructure).
rpc: fill in NIC parameters in instances
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Move FillDict at module level
This way it can be also used by scripts and other object types.
Fix two issues with exports and snapshot errors
This patch fixes two issues related to failed snapshots during exports: - first, the error messages used disk.logical_id1, which is a node name for DRBD, and it resulted in strange error messages like...
Add a node powercycle command
This (somewhat big) patch adds support for remotely rebooting the nodesvia whatever support the hypervisor has for such a concept.
For KVM/fake (and containers in the future) this just uses sysrq plus a‘reboot’ call if the sysrq method failed. For Xen, it first tries the...
call_instance_start: add optional hv/be parameters
This patch modifies the rpc.call_instance_start - the master side - totake optional hv/be parameters. The noded side is unchanged andoblivious to the change.
This will allow implementation of single-user capability and such on...
Inform the OS create script of reinstalls
Sometimes reinstalls are slightly different than new installs. Forexample certain partitions may need to be preserved accross reinstalls.In order to do that on a per-os basis we pass in the INSTANCE_REINSTALLvariable to inform the create script about when a reinstall is...
Fix some epydoc style issues
99% of the epydoc return tags are "@return:", but each of the modified fileshad one "@returns:" line. We fix this for consistency.
Reviewed-by: imsnah
Remove the extra_args parameter in instance start
This patch removes the extra_args parameter and instead switches theinstance to the HV_KERNEL_ARGS hypervisor option.
This is a big change, but it's a needed cleanup, this extra parameter onall RPC calls is not generic and we also need to have a persistent value...
RpcResult: add a new payload field
For results which use the (status, payload) response type, it's easierto define a ‘payload’ field on the result holding the payload than toextract it using “data1” in the caller code.
Reviewed-by: ultrotter
Fill the 'call' attribute of offline rpc results
When creating ‘fake’ results for offline nodes, we currently don't passthe call attribute. This complicates debugging, so even though thisshould not matter in practice, it's better to fix it.
Log the rpc call name in the RPC errors message
Currently the rpc module logs the error description and target node inrpc calls logging, as such:
2009-01-21 00:50:01,456: pid=1051/Thread-21 ERROR RPC error from node node1.example.com: Connection failed (111: Connection...
Add calls in the intra-node migration protocol
Currently the hypervisor is expected to do all the migration from thesource side. With this patch we also add the option of passing someinformation to the target side, and starting some operation there.
As a bonus, a function to cleanup any started operation is included....
Forward-port DrbdNetReconfig
This is a modified forward-port of DrbdNetReconfig and their associatedRPCs. In Ganeti 2.0, these functions will be used for two things: - live migration (as in 1.2) - and for other network reconfiguration tasks, since DRBD8.Attach()...
Silence warning on node list for offline nodes
The warning in node list is meant for nodes that return wronginformation, but for offline nodes this case is normal.
rpc: Add a method for easy check of remote results
The patch adds a new method to the rpc.RpcResult class called"RemoteFailMsg" which is useful for the RPC calls which return a(status, payload) style result.
Add an instance_migratable rpc call
This is a forward-port of commit 1194 on the 1.2 branch:
This call will check whether an instance is up on its primary, and that it has been started with symlinks. We currently have no on-secondary checks, nor any hypervisor specific call....
Pass instance name to rpc call blockdev_close
This is an extract of commit 1166 on the 1.2 branch (Add a rpc call fordrbd network reconfiguration), but only the blockdev_close part.
The patch changes the blockdev_close call to take the instance so that...
Job queue: Allow more than one file rename per RPC call
cleanup: fix 'variable unused' warning
In the iteration we don't care about the node names, so we change thefor loop to be over the values (and not itervalues).
Reviewed-by: amishchenko
Fix epydoc format warnings
This patch should fix all outstanding epydoc parsing errors; as such, weswitch epydoc into verbose mode so that any new errors will be visible.
RPC: Compress file upload data
Adding compression to larger amounts of data is more efficient thantransferring it (len(nodes) - 1) times over the network withoutcompression. We were able to compress a 800KB config file to about30 KB, which is about 40 KB with Base64 encoding (required due to...