Correct capitalisation of two Luxi calls
Two Luxi calls have inconsistent an name/value mapping (in the Pythoncode):
- REQ_AUTOARCHIVE_JOBS versus AutoArchiveJobs (versus AutoarchiveJobs)- REQ_QUEUE_SET_DRAIN_FLAG versus SetDrainFlag (no Queue)
While these are only a consistency issue, let's fix them so that the...
QA: Enable use of OR conditions in test checks
Until now “TestRunIf” and “TestEnabled” could only handle AND. With thispatch a new class named “Either” is added to “qa_config” and allows OR.The name “Either” was chosen instead of “Or” as the latter is very close...
Fix rapi.testutils unittest
Since we use a testutils.InputTestClient(), then the actual errorexpected is VerificationError, and not GanetiApiError (which is usedat real run-time).
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Add more RAPI test utilities
This patch adds a mock RAPI client to test input values to methods. Allmethods either raise an exception if there was a problem or return None.Third-party code can use this to test their input values to the RAPIclient.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
rapi.testutils.FakeCurl: Add header support
With this patch headers are constructed from the PycURL optionsand passed to the mock implementation.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Move _FakeCurl from tests/ganeti.rapi.client to ganeti.rapi.testutils
This is preparation for the mock system, where we need the same cURLmock.
Signed-off-by: René Nussbaumer <rn@google.com>Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Convert listing exports to query2
This solves one case where locks are acquired during LUXI queries.Pretty late into the transition I noticed that OpBackupQuery had a“use_locking” parameter for a long time, but didn't use it. Sincemost of the other changes were already and this allows exports to...
Copy debug level, priority and set comment for LU-generated opcodes
Before this patch, a node evacuation submitted with high priority wouldonly compute the solution at that priority, but the actual evacuationran at normal priority.
Merge branch 'devel-2.5'
Fix query unittests after converting jobs to query2
I missed these among some shelltest-related failures.
jqueue: Convert GetInfo to query2
This rather inefficient implementation (fields are evaluated on everycall to GetInfo) is not good for WaitForJobChanges and doesn't supportfilters, but that will be rectified in later patches.
qlang.MakeFilter: Enable use of different name field
Jobs don't have a “name” field, so we must be able to controlthe field used for simple filters.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge cli.FormatTimestamp and utils.FormatTime
… to some degree at least. Unittests are included.
constants: Don't hardcode priorities for LOCK_ATTEMPTS_TIMEOUT
Also include unittest for LOCK_ATTEMPTS_TIMEOUT.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Add whitelist for opcodes using BGL
RPC: Add a new client type for DNS only
This patch moves the “call_version” to a new RPC client definition andthen adds a new runner using the DNS resolver for getting the hostaddress.
The standard “BootstrapRunner”, where the call was before, tries to...
Add trivial tests for gnt-* cli
While testing some other stuff, I realised that the gnt-* commandscould be broken (as in, the script fails with syntax errors), but makecheck doesn't detect it. Since we have shelltest, we can now addtrivial tests for this case....
cmdlib: Stop forking in LUClusterQuery
While debugging another issue we realized that LUClusterQuery forks.This turned out to be the “platform.architecture” function from thePython library. It uses the “file” command to determine the architectureof the Python binary....
Convert manual shell tests to shelltestrunner
This is more of a RFC… Basically most of the shell-based tests areconverted from exec+grep to shelltestrunner.
Things are not all fine and nice though:
- we have dependencies between tests, as some generate some data files...
locking: Handle spurious notifications on lock acquire
This was already a TODO since the implementation of lock priorities inSeptember 2010. Under certain conditions a waiting acquire can benotified at a time when it can't actually get the lock. In this case it...
locking: Fix lock deletion with timeout
While working on another SharedLock fix I realized timeouts on lockdeletion don't work very well if the timeout actually expires. Thispatch fixes the issue and adds a new unittest.
Move _TimeoutExpired to utils
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
(cherry picked from commit f8326fcaac87958241d78526e5868d23d78ac286)
Add new hail (shell) tests
Using some hand-crafted files (manually modified from a real clusterrunning master branch), we test that hail behaves as we expect:doesn't load obvious wrong data, can do relocations/allocations, etc.
Signed-off-by: Iustin Pop <iustin@google.com>...
Add RAPI shell-level unittest
Add tests for custom text files
Rename spindle_usage to spindle_use on Ganeti side
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
ipolicy: Keep track of spindle usage
Fix lint error in commit 035b33e2
Commit 035b33e2 forgot one blank space, and current pylint in DebianSid doesn't run on our code… so I didn't realise this until aftercommit, sorry.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix tempfile reset code & test on newer Python
Python 2.7.3 (rc status) and 3.2.3/3.3 (rc, respectively alpha status)have fixed http://bugs.python.org/issue12856 which we worked aroundourselves.
This means two things:
- we don't need to manually reset the module...
Merge remote-tracking branch 'origin/devel-2.5'
Conflicts: Makefile.am (trivial, test data files added on both branches) lib/opcodes.py (trivial, master renamed filter→qfilter, 2.5 fixed the type of the parameter)...
Add result checks for OpTags*
opcodes: Annotate the OP_RESULT of query operations
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix asyncnotifier unit tests
On my workstation, if I run the following two tests in this exactorder:
- TestSingleFileEventHandler.testError- TestSingleFileEventHandler.testReplace
the second test will fail, because there is no "unregister" ofprevious tests (and the poll() done by asyncore will fire on the...
Add ndp/* fields to group/node query
Also change the meaning of group ndparams to mean the actual (notcustom) nd params, and add custom_ndparams for the current meaning ofndparams.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add new spindle_count node parameter
Currently this is not handled by Ganeti, just recorded.
opcodes: Add result checks for OpBackup*
Improve opcode result check test to have a whitelist
Only whitelisted opcodes may not have a result check.
config: Add check for disk's “iv_name”
This check verifies “iv_name” of all instance disks. If one is wrong(which shouldn't happen in the first place), cluster verification willwarn:
“ERROR: cluster: Instance 'inst.example.com' has wrongly named disks:...
cli: Handle negative numbers when parsing key-value assignments
To remove the last disk, or to add a NIC to the end, one can use theindex 1. This wouldn't work as intended as “” is a special prefix.
utils.text: Add function to truncate string
The function adds an ellipse if the string was actually truncated. Alsostart using it in mcpu for result checks (where the message is alsoslightly changed to use a colon).
LUInstanceSetParams: Update disk's “iv_name”
When modifications are made, disks may not have the same index anymore.Updating all disks fixes this.
ApplyContainerMods: Fix issues with indices
When adding an item the index given to the callback function would beincorrect under certain conditions. This patch also adds assertions andmore tests.
gnt-instance modify: Support new-style NIC/disk modifications
This patch adds support for adding/removing NICs/disks at arbitraryindices on the command line. To add a disk at a specified index, use“--disk 3:size=16G”. To remove the second disk, use “--disk 2:remove”....
Workaround changed LVM behaviour
The vgreduce command has changed behaviour from when we initiallywrote the code (2.02.02 versus 2.02.66, 4 years delta):
- if there are LVs which will be impacted, it requires --force- otherwise refuses to proceed, but it still returns exit code 0...
Add unittest for cmdlib._GenerateDiskTemplate
This is in preparation to de-duplicating significant chunks of code incmdlib._GenerateDiskTemplate.
rapi.testutils: Add exported functions to verify opcode input/result
These can be used by third-party code to verify mock code. Further workon mocks is forthcoming, so this is only a start.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
ApplyContainerMods: Return changes from callbacks
… instead of passing the list of changes as a parameter.
cmdlib: Add generic algorithm for modifying NICs/disks
While preparing this patch series I identified at least three differentimplementations of the algorithm for adding/removing/changingNICs/disks. These two functions and corresponding unittests provide a...
Fix unittest breakage after commit a2aadb34b
Thou shalt run “make check” before sending patches.
rapi.client: Removing constants not needed anymore
They were not referenced through the whole code and were marked internalonly.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add stricter checks for OpInstanceSetParams.{nics,disks}
Stricter check for OS modifications passed to OpClusterSetParams
Don't just check the first element of each item, but also make sure theOS name is a string and not empty.
Implement rbd disk template
Introduce the rbd disk template, which handles provisioning andmanagement of instance disks as block devices mapped to rbd volumeson a RADOS cluster.
The rbd template is of type DTS_EXT_MIRROR, meaning rbd-basedinstances can failover or migrate to any node, without...
Add UnescapeAndSplit unittest for multi-escapes
This would have caught the bug in the first place. Argh,hand-generated test cases!
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
cmdlib._ComputeMinMaxSpec: Add unittest for this function
cmdlib: Rename _CheckMinMaxSpecs to _ComputeMinMaxSpec
This is to reflect the fact that _Check functions do raise Op*Errorwhich is not the case for _CheckMinMaxSpec.
_CheckTargetNodeIPolicy: instance.primary_node is not objects.Node
rpc: More unittests
- Fix: Don't check st_atime for temporary file for upload tests- Add tests for five encoders
cmdlib: Adding helper for instance policy
rpc.ConfigRunner: Fix uploading file
Commit 601dfcb made some changes to the default encoders. This made“ConfigRunner.call_upload_file” fail due to a missing encoder. Thispatch applies the necessary changes to ConfigRunner and updates theunittest for “call_upload_file”....
Conflicts: NEWS: trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add unittest for uploading file via RPC
After some preparing patches this unittest can finally be added.
Add unittests for RPC client
This patch adds a number of unittests for the RPC client base class.Some small changes were necessary in “rpc.py” to allow for bettertesting.
Add unittest for RPC compression
Merge branch 'stable-2.5' into devel-2.5
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
KVM: support version reported by 1.0
This of course was working for all the rcs, but broke with 1.0 itself.
In addition: - split between running kvm --version and parsing its output - unittest parsing for various known --help outputs - updated NEWS file...
rpc._NodeConfigResolver: Support resolving offline nodes
This is needed to powercycle a node marked offline.
rpc._RpcProcessor: Make read timeout a non-keyword argument
It is always used and can be set to “None” if intended.
rpc: Pass resolver options to actual resolver
test/lockperf: Use a list of integers, not booleans
Add lock performance utility
I had an idea for improving locking performance. To see if it worked Iwrote this tool. Unfortunately the idea didn't quite work (brokeunittests left and right), but the tool is still handy for evaluatingfuture changes to the “SharedLock” class....
jqueue: Factorize checking job processor's result
This allows for more unittesting.
jqueue unittest: Rename simple fake-job class
Also add a parameter for priority, to be used in an upcomingpatch.
unit tests: Add tests for file mode handling in utils.WriteFile
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
unit tests: Add tests for uid and gid handling in utils.WriteFile
These tests need fakeroot. If it's not present, they are not run (theydon't fail).
noded: Fix /etc/hosts group ownership
When Ganeti was compiled with the option of running as a differentuser/group, the group ownership of /etc/hosts was set to gnt-daemons. Nowpermissions for /etc/hosts are preserved (or set correctly).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Change internal RPC client body values
Currently, all RPC payloads sent by the client to the remote nodedaemons must be identical, due to how the data is passedinternally. This is deficient in both use (from the programmer's pointof view) and from the network traffic (cluster verify/disk data...
test/hv_kvm: Code style, additional tests
Change code style to match rest of Ganeti code and addtwo tests checking for single-line messages.
serializer: Remove JSON indentation and dict key sorting
Serializing to JSON using “simplejson” is significantly slower whenindentation and/or sorting of dictionary keys is used. In simplejson 1.xthe difference isn't that big, but with simplejson 2.x the difference...
jqueue: Fix deadlock between job queue and dependency manager
When an opcode is about to be processed its dependencies areevaluated using “_JobDependencyManager.CheckAndRegister”. Dueto its nature that function requires a lock on the manager'sinternal structures. All of this happens while the job queue...
Fix unittests for daemon_util with disabled confd
RAPI client: Export feature constants
Users of the RAPI client may want to use feature strings too, thereforethey should be explicitely public.
Group query now calculates ipolicy
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
utils.io.WritePidFile: Improve error reporting
If the PID file is already locked by another process, try to readthe content and report it as part of the error message.
utils.ListVisibleFiles: Hide “lost+found” directories
If a “lost+found” directory is found at a filesystem's root path it isignored. In all other cases directory entries named “lost+found” aretreated normally. Unittests are included. Fixes issue 153.
Add tests for parameter "always_failover"
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
cmdlib: Adding hv/disk state dict helper functions
cmdlib: Adding _UpdateAndVerifySubDict helper
This helps with 2 dimensional dicts.For example the hv_state and the disk_state dicts.
Add DRBD barriers disk parameters
Add the disk-barriers and meta-barriers parameters described in thedesign doc.
objects.Cluster: Add property for primary hypervisor
This is useful for working with a node's hypervisor state, where onlythe primary hypervisor will be authoritative.
Add basic support for disk parameters
objects.py: * add disk parameters to Disk, Cluster, NodeGroup.
constants.py: * add dictionaries that will hold types and default values for disk parameters (for now, empty).
test/ganeti.constants_unittest.py:...
Fix race condition in test for *FileID functions
In this test the “file ID” of a temporary file is compared against thefile ID gathered via an open file descriptor to the same file. Forreasons unknown to me utime(2) is called in-between to update theinode's a- and mtime. Depending on the file system's timestamp...
Add objects for disk/hv state
- Data objects- Serialization/deserialization- Unittests
locking: Allow checking if lock is owned in certain mode
With this patch the “LockSet” and “GanetiLockManager” classes have a newfunction to check if a single or a group of locks (at a certain level)have been acquired in a specific mode. This will be used for additional...
LUGroupAssignNodes: Fix node membership corruption
Note: This bug only manifests itself in Ganeti 2.5, but since theproblematic code also exists in 2.4, I decided to fix it there.
If a node was assigned to a new group using “gnt-group assign-nodes” the...