rpc._RpcProcessor: Make read timeout a non-keyword argument
It is always used and can be set to “None” if intended.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
rpc: Pass resolver options to actual resolver
Merge branch 'devel-2.5'
Merge branch 'stable-2.5' into devel-2.5
test/lockperf: Use a list of integers, not booleans
Add lock performance utility
I had an idea for improving locking performance. To see if it worked Iwrote this tool. Unfortunately the idea didn't quite work (brokeunittests left and right), but the tool is still handy for evaluatingfuture changes to the “SharedLock” class....
jqueue: Factorize checking job processor's result
This allows for more unittesting.
jqueue unittest: Rename simple fake-job class
Also add a parameter for priority, to be used in an upcomingpatch.
unit tests: Add tests for file mode handling in utils.WriteFile
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
unit tests: Add tests for uid and gid handling in utils.WriteFile
These tests need fakeroot. If it's not present, they are not run (theydon't fail).
noded: Fix /etc/hosts group ownership
When Ganeti was compiled with the option of running as a differentuser/group, the group ownership of /etc/hosts was set to gnt-daemons. Nowpermissions for /etc/hosts are preserved (or set correctly).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Change internal RPC client body values
Currently, all RPC payloads sent by the client to the remote nodedaemons must be identical, due to how the data is passedinternally. This is deficient in both use (from the programmer's pointof view) and from the network traffic (cluster verify/disk data...
test/hv_kvm: Code style, additional tests
Change code style to match rest of Ganeti code and addtwo tests checking for single-line messages.
serializer: Remove JSON indentation and dict key sorting
Serializing to JSON using “simplejson” is significantly slower whenindentation and/or sorting of dictionary keys is used. In simplejson 1.xthe difference isn't that big, but with simplejson 2.x the difference...
jqueue: Fix deadlock between job queue and dependency manager
When an opcode is about to be processed its dependencies areevaluated using “_JobDependencyManager.CheckAndRegister”. Dueto its nature that function requires a lock on the manager'sinternal structures. All of this happens while the job queue...
Fix unittests for daemon_util with disabled confd
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
RAPI client: Export feature constants
Users of the RAPI client may want to use feature strings too, thereforethey should be explicitely public.
Group query now calculates ipolicy
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
utils.io.WritePidFile: Improve error reporting
If the PID file is already locked by another process, try to readthe content and report it as part of the error message.
utils.ListVisibleFiles: Hide “lost+found” directories
If a “lost+found” directory is found at a filesystem's root path it isignored. In all other cases directory entries named “lost+found” aretreated normally. Unittests are included. Fixes issue 153.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add tests for parameter "always_failover"
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
cmdlib: Adding hv/disk state dict helper functions
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
cmdlib: Adding _UpdateAndVerifySubDict helper
This helps with 2 dimensional dicts.For example the hv_state and the disk_state dicts.
Add DRBD barriers disk parameters
Add the disk-barriers and meta-barriers parameters described in thedesign doc.
objects.Cluster: Add property for primary hypervisor
This is useful for working with a node's hypervisor state, where onlythe primary hypervisor will be authoritative.
Add basic support for disk parameters
objects.py: * add disk parameters to Disk, Cluster, NodeGroup.
constants.py: * add dictionaries that will hold types and default values for disk parameters (for now, empty).
test/ganeti.constants_unittest.py:...
Fix race condition in test for *FileID functions
In this test the “file ID” of a temporary file is compared against thefile ID gathered via an open file descriptor to the same file. Forreasons unknown to me utime(2) is called in-between to update theinode's a- and mtime. Depending on the file system's timestamp...
Add objects for disk/hv state
- Data objects- Serialization/deserialization- Unittests
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
locking: Allow checking if lock is owned in certain mode
With this patch the “LockSet” and “GanetiLockManager” classes have a newfunction to check if a single or a group of locks (at a certain level)have been acquired in a specific mode. This will be used for additional...
LUGroupAssignNodes: Fix node membership corruption
Note: This bug only manifests itself in Ganeti 2.5, but since theproblematic code also exists in 2.4, I decided to fix it there.
If a node was assigned to a new group using “gnt-group assign-nodes” the...
Separate OpNodeEvacuate.mode from iallocator
Until now the iallocator constants for node evacuation(IALLOCATOR_NEVAC_*) were also used for the opcode. However, it turnedout this was due to a misunderstanding and is incorrect. This patch addsnew constants (with the same values) and changes the affected places....
unittests: use max/min memory
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
workerpool: Allow processing of new tasks to be stopped
This is different from “Quiesce” in the sense that this function justchanges an internal flag and doesn't wait for the queue to be empty.Tasks already being processed continue normally, but no new tasks will...
Introduce admin_state as 3-values data type
Rename admin_up to admin_state
algo: Make a dict from an flat list
This is in preparation to take deeper dict constructs from the commandline. You can feed the optionslist directly constructed of type"identkeyval" to it and it returns a fully deflated dict.
This is mainly needed for the resource model changes where we have to...
locking: Make some aliased methods public
Some methods, such as “_is_owned” and “list_owned”, have been aliased tomake them public for a while now. This patch makes the actualimplementation public.
SharedLock's “is_owned” needs to be aliased to “_is_owned” to remain...
check-python-code: Detect old-style pylint disable-msg lines
Commit b459a848d was supposed to replace all, but one was missed. Add acheck to autotools/check-python-code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
Fail if node/group evacuation can't evacuate instances
If an instance can't be evacuated, only a message would be printed. Withthis change the operation always aborts. Newly added unittests check forthis behaviour.
query: Use new SequenceToDict utility
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
utils.algo: Add utility to convert sequence to dictionary
Useful for converting list of query fields to a dictionary and toconvert RPC definitions. Includes duplicate detection.
utils.algo: Add a function to insert a list into a list
Generalize HooksMaster
- remove any dependence on Logical Units from the HooksMaster;- add a new function parameter to the constructor, a function that is expected to convert the results of the hooks execution in a format understood by the HooksMaster;...
Clean up incorrect merge 7dcf333d
The unittests weren't corrected for the master branch.
Fix wrong headers and licences
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Stephen Shirley <diamond@google.com>
Fix parameters of RpcResult in hooks unit tests
In FakeHooksRpcSuccess, the data parameter of the RpcResult constructorwas not enclosed in a tuple. While this does not make the test fail, itmust be fixed.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>...
Fix RPC unittest
Add missing “read_timeout” value.
Fix a too long line.
That's what you get for not running make lint :(
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Move bootstrap-related RPC to generated wrappers
With this patch, only 6 RPC are left as old-style code.
rpc: Disable timeout check
This is in preparation for the definition-based RPC model, wherethe timeout will be stored within the definition.
Move RenameFile to the new functions
ensure_dirs: Move some useful functions into utils.
With this change we can easily reuse this functionality where it makessense on other parts of Ganeti.
Add the JoinDisjointDicts function to utils.algo
Add a function that joins two dictionaries, enforcing the constraintthat the two key sets should be disjoint. Also, add unit tests for thisfunction.
utils.ShellWriter: Don't indent empty lines
Empty lines shouldn't get indented. Unittest included.
http.client: Remove use of PycURL's “reset” function
We don't re-use cURL objects anymore, so there's no need to reset them.PycURL 7.19.0 has a reference counting bug leading to a crash aftera certain number of performed requests.
Since a unittest depended on this function for a test, it is replaced...
Add unittest for PycURL 7.19.0's reset issue
While we won't use this function in Ganeti (yet), users should be awareof the potential crashes caused by this functioni (discussed offline).If the installed version of PycURL is not affected, this test is a...
Conflicts: lib/cmdlib.py - trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Allow per-hypervisor optional files
Rather than just allowing files for all nodes to be optional, we allowoptional files to be per-category. The way this works is that they mustbe included in both lists (the new code checks for this).
The code also removes a duplicate assert (present both in verify and...
Revert "Rename utils.mlock to utils.cfunc"
The rename is not needed either, since we're not adding more code as ofnow.This reverts commit 57ca011e1cd2681948969724e2646edaac22da28.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Revert "utils.cfunc: Cleanup, more flexibility"
We discussed that this is not needed right now, and it breaks existingfunctionality and unittests.
This reverts commit 6915fe26da8dce41fc967d761f005390aa956161.
RAPI: Make node evacuation actually work
Commit e1f23243 changed te LU and opcode for node evacuation to receivea “mode” parameter (among other things). Commit de40437a changed theRAPI code accordingly, but did so for an earlier version of the firstpatch. Obviously this couldn't work, so here's the fix....
utils.cfunc: Cleanup, more flexibility
- Split code using ctypes directly into a helper class- Don't load “libc.so.6”, but use handle for main program instead (see comment in code)- Clarify comment on errno with older ctypes versions- Rename unittest since it can't be used for other functions (modifies...
Rename utils.mlock to utils.cfunc
Renaming so that more code using ctypes could be added to the same file.
RAPI: Fix resource for replacing disks
Commit d1c172deb4f inadvertently changes the“/2/instances/[instance_name]/replace-disks” resource to use bodyparameters. There were no QA tests and the issue wasn't noticed.
This patch re-introduces support for query parameters and adds a QA...
rapi: Add resource for modifying node
A separate patch will add “auto-promote” through“/2/nodes/[node_name]/role”.
constants: Verify exported names
The “constants” module is a bit special in the sense that we don't wantto export random stuff from it. This unittest checks the namingconvention and removes imported modules from the module's namespace.
http.client: Remove HTTP client pool code
This patch removes all remains of the HTTP client pool. Newly added unittestsprovide 96% coverage on http.client.
Rename filter and filter_ to qfilter
We currently use 'filter' as the OpCode, QueryRequest and RAPI fieldname for representing a query filter. However, since 'filter' is abuilt-in function, we actually have to use filter_ throughout the codein order to not override the built-in function....
Fixes for ovfconverter + vmware
Move cluster verify error codes to constants
- move the cluster verify error codes from cmdlib._VerifyErrors to constants;- add to each of them the CV (Cluster Verify) prefix;- add the CV_ALL_ECODES and CV_ALL_ECODES_STRINGS constants;- wrap the lines that exceed 80 characters after changing the error...
Add ValidateNetmask and GetClass IPAddress methods
Also, add related tests to the test suite.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix issue when verifying cluster files
If a cluster has any non-master-candidate nodes, those don't contain allfiles (e.g. config.data). With commit aef59ae764dc (March 31st, 2011)the logic was changed and subsequently verifying a cluster with non-mcnodes would complain....
LUClusterVerifyGroup: Spread SSH checks over more nodes
When verifying a group the code would always check SSH to all nodes inthe same group, as well as the first node for every other group. On bigclusters this can cause issues since many nodes will try to connect to...
Add gnt-cluster commands to toggle the master IP
Move _TimeoutExpired to utils
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
rpc/http: Show pending RPC requests in lock monitor
Not all requests use an instance of RpcRunner yet and therefore won'tshow up (only instances have access to the global Ganeti context).Currently only the IP address is accessible. Another patch will add a...
rpc: Overhaul client structure
- Clearly separate node name to IP address resolution into separate functions- Simplified code structure (one code path instead of several)- Fully unittested- Preparation for more RPC improvements
Export: unittests
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Import: unittests
Revert "Fail if dictionary uses invalid keys" and "Support newer “json” module"
This reverts commit fd0351aef246f5d36e641209429e2ec093d325f8 and9869e771704ada62bab001e729c52a36525ef081. The built-in module is a lotslower in Python 2.6.
serializer: Fail if dictionary uses invalid keys
JSON only supports a very restricted set of types for dictionary keys,among them strings, booleans and “null”. Integers and floats areconverted to strings. Since this can cause a lot of confusion in Python,...
RAPI: Add resource to powercycle node
RAPI: Add resource to recreate instance's disks
This was still missing from RAPI.
docs unittest: Add verification of opcodes covered by RAPI
All opcodes which are not yet covered or can't with the current design(e.g. cluster initialization) are excluded. This test is added to thedocs unittest since here the code already has access to a list of all...
docs unittest: Split into two separate test cases
The RAPI and hooks documentation aren't related at all.
docs unittest: Make _ReadDocFile a global function
rlib2: Declare all opcodes and equivalents
By declaring all used opcodes or opcodes equivalent to the operationsexecuted in a resource we will be able to ensure all opcodes are coveredby RAPI (with some exceptions).
Add SPICE support to gnt-instance console
Also update related unit tests.
Fix unittest breakage from commit 132cdb87159e