Add new hail (shell) tests
Using some hand-crafted files (manually modified from a real clusterrunning master branch), we test that hail behaves as we expect:doesn't load obvious wrong data, can do relocations/allocations, etc.
Signed-off-by: Iustin Pop <iustin@google.com>...
Rename spindle_usage to spindle_use on Ganeti side
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
ipolicy: Keep track of spindle usage
Fix lint error in commit 035b33e2
Commit 035b33e2 forgot one blank space, and current pylint in DebianSid doesn't run on our code… so I didn't realise this until aftercommit, sorry.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix tempfile reset code & test on newer Python
Python 2.7.3 (rc status) and 3.2.3/3.3 (rc, respectively alpha status)have fixed http://bugs.python.org/issue12856 which we worked aroundourselves.
This means two things:
- we don't need to manually reset the module...
Merge remote-tracking branch 'origin/devel-2.5'
Conflicts: Makefile.am (trivial, test data files added on both branches) lib/opcodes.py (trivial, master renamed filter→qfilter, 2.5 fixed the type of the parameter)...
Add result checks for OpTags*
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
opcodes: Annotate the OP_RESULT of query operations
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix asyncnotifier unit tests
On my workstation, if I run the following two tests in this exactorder:
- TestSingleFileEventHandler.testError- TestSingleFileEventHandler.testReplace
the second test will fail, because there is no "unregister" ofprevious tests (and the poll() done by asyncore will fire on the...
Add ndp/* fields to group/node query
Also change the meaning of group ndparams to mean the actual (notcustom) nd params, and add custom_ndparams for the current meaning ofndparams.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add new spindle_count node parameter
Currently this is not handled by Ganeti, just recorded.
opcodes: Add result checks for OpBackup*
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Improve opcode result check test to have a whitelist
Only whitelisted opcodes may not have a result check.
config: Add check for disk's “iv_name”
This check verifies “iv_name” of all instance disks. If one is wrong(which shouldn't happen in the first place), cluster verification willwarn:
“ERROR: cluster: Instance 'inst.example.com' has wrongly named disks:...
cli: Handle negative numbers when parsing key-value assignments
To remove the last disk, or to add a NIC to the end, one can use theindex 1. This wouldn't work as intended as “” is a special prefix.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
utils.text: Add function to truncate string
The function adds an ellipse if the string was actually truncated. Alsostart using it in mcpu for result checks (where the message is alsoslightly changed to use a colon).
LUInstanceSetParams: Update disk's “iv_name”
When modifications are made, disks may not have the same index anymore.Updating all disks fixes this.
ApplyContainerMods: Fix issues with indices
When adding an item the index given to the callback function would beincorrect under certain conditions. This patch also adds assertions andmore tests.
gnt-instance modify: Support new-style NIC/disk modifications
This patch adds support for adding/removing NICs/disks at arbitraryindices on the command line. To add a disk at a specified index, use“--disk 3:size=16G”. To remove the second disk, use “--disk 2:remove”....
Workaround changed LVM behaviour
The vgreduce command has changed behaviour from when we initiallywrote the code (2.02.02 versus 2.02.66, 4 years delta):
- if there are LVs which will be impacted, it requires --force- otherwise refuses to proceed, but it still returns exit code 0...
Add unittest for cmdlib._GenerateDiskTemplate
This is in preparation to de-duplicating significant chunks of code incmdlib._GenerateDiskTemplate.
rapi.testutils: Add exported functions to verify opcode input/result
These can be used by third-party code to verify mock code. Further workon mocks is forthcoming, so this is only a start.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
ApplyContainerMods: Return changes from callbacks
… instead of passing the list of changes as a parameter.
cmdlib: Add generic algorithm for modifying NICs/disks
While preparing this patch series I identified at least three differentimplementations of the algorithm for adding/removing/changingNICs/disks. These two functions and corresponding unittests provide a...
Fix unittest breakage after commit a2aadb34b
Thou shalt run “make check” before sending patches.
rapi.client: Removing constants not needed anymore
They were not referenced through the whole code and were marked internalonly.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add stricter checks for OpInstanceSetParams.{nics,disks}
Stricter check for OS modifications passed to OpClusterSetParams
Don't just check the first element of each item, but also make sure theOS name is a string and not empty.
Implement rbd disk template
Introduce the rbd disk template, which handles provisioning andmanagement of instance disks as block devices mapped to rbd volumeson a RADOS cluster.
The rbd template is of type DTS_EXT_MIRROR, meaning rbd-basedinstances can failover or migrate to any node, without...
Merge branch 'devel-2.5'
Add UnescapeAndSplit unittest for multi-escapes
This would have caught the bug in the first place. Argh,hand-generated test cases!
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
cmdlib._ComputeMinMaxSpec: Add unittest for this function
cmdlib: Rename _CheckMinMaxSpecs to _ComputeMinMaxSpec
This is to reflect the fact that _Check functions do raise Op*Errorwhich is not the case for _CheckMinMaxSpec.
_CheckTargetNodeIPolicy: instance.primary_node is not objects.Node
rpc: More unittests
- Fix: Don't check st_atime for temporary file for upload tests- Add tests for five encoders
cmdlib: Adding helper for instance policy
rpc.ConfigRunner: Fix uploading file
Commit 601dfcb made some changes to the default encoders. This made“ConfigRunner.call_upload_file” fail due to a missing encoder. Thispatch applies the necessary changes to ConfigRunner and updates theunittest for “call_upload_file”....
Conflicts: NEWS: trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add unittest for uploading file via RPC
After some preparing patches this unittest can finally be added.
Add unittests for RPC client
This patch adds a number of unittests for the RPC client base class.Some small changes were necessary in “rpc.py” to allow for bettertesting.
Add unittest for RPC compression
Merge branch 'stable-2.5' into devel-2.5
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
KVM: support version reported by 1.0
This of course was working for all the rcs, but broke with 1.0 itself.
In addition: - split between running kvm --version and parsing its output - unittest parsing for various known --help outputs - updated NEWS file...
rpc._NodeConfigResolver: Support resolving offline nodes
This is needed to powercycle a node marked offline.
rpc._RpcProcessor: Make read timeout a non-keyword argument
It is always used and can be set to “None” if intended.
rpc: Pass resolver options to actual resolver
test/lockperf: Use a list of integers, not booleans
Add lock performance utility
I had an idea for improving locking performance. To see if it worked Iwrote this tool. Unfortunately the idea didn't quite work (brokeunittests left and right), but the tool is still handy for evaluatingfuture changes to the “SharedLock” class....
jqueue: Factorize checking job processor's result
This allows for more unittesting.
jqueue unittest: Rename simple fake-job class
Also add a parameter for priority, to be used in an upcomingpatch.
unit tests: Add tests for file mode handling in utils.WriteFile
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
unit tests: Add tests for uid and gid handling in utils.WriteFile
These tests need fakeroot. If it's not present, they are not run (theydon't fail).
noded: Fix /etc/hosts group ownership
When Ganeti was compiled with the option of running as a differentuser/group, the group ownership of /etc/hosts was set to gnt-daemons. Nowpermissions for /etc/hosts are preserved (or set correctly).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Change internal RPC client body values
Currently, all RPC payloads sent by the client to the remote nodedaemons must be identical, due to how the data is passedinternally. This is deficient in both use (from the programmer's pointof view) and from the network traffic (cluster verify/disk data...
test/hv_kvm: Code style, additional tests
Change code style to match rest of Ganeti code and addtwo tests checking for single-line messages.
serializer: Remove JSON indentation and dict key sorting
Serializing to JSON using “simplejson” is significantly slower whenindentation and/or sorting of dictionary keys is used. In simplejson 1.xthe difference isn't that big, but with simplejson 2.x the difference...
jqueue: Fix deadlock between job queue and dependency manager
When an opcode is about to be processed its dependencies areevaluated using “_JobDependencyManager.CheckAndRegister”. Dueto its nature that function requires a lock on the manager'sinternal structures. All of this happens while the job queue...
Fix unittests for daemon_util with disabled confd
RAPI client: Export feature constants
Users of the RAPI client may want to use feature strings too, thereforethey should be explicitely public.
Group query now calculates ipolicy
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
utils.io.WritePidFile: Improve error reporting
If the PID file is already locked by another process, try to readthe content and report it as part of the error message.
utils.ListVisibleFiles: Hide “lost+found” directories
If a “lost+found” directory is found at a filesystem's root path it isignored. In all other cases directory entries named “lost+found” aretreated normally. Unittests are included. Fixes issue 153.
Add tests for parameter "always_failover"
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
cmdlib: Adding hv/disk state dict helper functions
cmdlib: Adding _UpdateAndVerifySubDict helper
This helps with 2 dimensional dicts.For example the hv_state and the disk_state dicts.
Add DRBD barriers disk parameters
Add the disk-barriers and meta-barriers parameters described in thedesign doc.
objects.Cluster: Add property for primary hypervisor
This is useful for working with a node's hypervisor state, where onlythe primary hypervisor will be authoritative.
Add basic support for disk parameters
objects.py: * add disk parameters to Disk, Cluster, NodeGroup.
constants.py: * add dictionaries that will hold types and default values for disk parameters (for now, empty).
test/ganeti.constants_unittest.py:...
Fix race condition in test for *FileID functions
In this test the “file ID” of a temporary file is compared against thefile ID gathered via an open file descriptor to the same file. Forreasons unknown to me utime(2) is called in-between to update theinode's a- and mtime. Depending on the file system's timestamp...
Add objects for disk/hv state
- Data objects- Serialization/deserialization- Unittests
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
locking: Allow checking if lock is owned in certain mode
With this patch the “LockSet” and “GanetiLockManager” classes have a newfunction to check if a single or a group of locks (at a certain level)have been acquired in a specific mode. This will be used for additional...
LUGroupAssignNodes: Fix node membership corruption
Note: This bug only manifests itself in Ganeti 2.5, but since theproblematic code also exists in 2.4, I decided to fix it there.
If a node was assigned to a new group using “gnt-group assign-nodes” the...
Separate OpNodeEvacuate.mode from iallocator
Until now the iallocator constants for node evacuation(IALLOCATOR_NEVAC_*) were also used for the opcode. However, it turnedout this was due to a misunderstanding and is incorrect. This patch addsnew constants (with the same values) and changes the affected places....
unittests: use max/min memory
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
workerpool: Allow processing of new tasks to be stopped
This is different from “Quiesce” in the sense that this function justchanges an internal flag and doesn't wait for the queue to be empty.Tasks already being processed continue normally, but no new tasks will...
Introduce admin_state as 3-values data type
Rename admin_up to admin_state
algo: Make a dict from an flat list
This is in preparation to take deeper dict constructs from the commandline. You can feed the optionslist directly constructed of type"identkeyval" to it and it returns a fully deflated dict.
This is mainly needed for the resource model changes where we have to...
locking: Make some aliased methods public
Some methods, such as “_is_owned” and “list_owned”, have been aliased tomake them public for a while now. This patch makes the actualimplementation public.
SharedLock's “is_owned” needs to be aliased to “_is_owned” to remain...
check-python-code: Detect old-style pylint disable-msg lines
Commit b459a848d was supposed to replace all, but one was missed. Add acheck to autotools/check-python-code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
Fail if node/group evacuation can't evacuate instances
If an instance can't be evacuated, only a message would be printed. Withthis change the operation always aborts. Newly added unittests check forthis behaviour.
query: Use new SequenceToDict utility
utils.algo: Add utility to convert sequence to dictionary
Useful for converting list of query fields to a dictionary and toconvert RPC definitions. Includes duplicate detection.
utils.algo: Add a function to insert a list into a list
Generalize HooksMaster
- remove any dependence on Logical Units from the HooksMaster;- add a new function parameter to the constructor, a function that is expected to convert the results of the hooks execution in a format understood by the HooksMaster;...
Clean up incorrect merge 7dcf333d
The unittests weren't corrected for the master branch.
Fix wrong headers and licences
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Stephen Shirley <diamond@google.com>
Fix parameters of RpcResult in hooks unit tests
In FakeHooksRpcSuccess, the data parameter of the RpcResult constructorwas not enclosed in a tuple. While this does not make the test fail, itmust be fixed.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>...
Fix RPC unittest
Add missing “read_timeout” value.
Fix a too long line.
That's what you get for not running make lint :(
Move bootstrap-related RPC to generated wrappers
With this patch, only 6 RPC are left as old-style code.
rpc: Disable timeout check
This is in preparation for the definition-based RPC model, wherethe timeout will be stored within the definition.