Add unittests for confd constants
Checking that the fourcc code is indeed a fourcc code, and that requestcodes and reply statuses are distinct.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Wrap lines over 80 characters
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Luca Bigliardi <shammash@google.com>
Move LoadModule function to ganeti.build
It should only be used at build-time.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add simple unittest for manpages
Add simple unittest for remote API docs
Add simple unittest for hooks documentation
Run unittests in a temporary directory
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove extra argument from HooksMaster class
The mcpu.py:HooksMaster class needs to have a proc attribute/argument toinit in ordet to call its LogWarning method. However, this is availablefrom the 'lu' attribute, so we can remove this dependency.
Signed-off-by: Iustin Pop <iustin@google.com>...
Add utils.FormatTime and a simple unittest
We don't format with subsecond-precision.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Serializer, remove salt_verifier functionality
The salt needs to be returned anyway, so we don't have to add anotherkey for the sender to recognize which request an answer is answering, soall that infrastructure is useless. :(
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Add constants for local disk status
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
TestParameterNames: also check nic parameters
ConfigObject.ToDict() only export non-None values
The method is changed to a normal loop, to avoid calling getattr()twice. Also getstate is changed to just use ToDict() by default.
This should also make getstate work for objects which have tooverride the ToDict function because they contain other objects....
Fix unittests broken by commit 2bb5c9115f
File "../test/ganeti.hooks_unittest.py", line 239, in setUp self.lu = FakeLU(FakeProc(), self.op, self.context, None)File "…/ganeti/cmdlib.py", line 92, in init self.LogStep = processor.LogStepAttributeError: FakeProc instance has no attribute 'LogStep'...
Get rid of the default_hypervisor slot
Currently we have both a default_hypervisor and an enabled_hypervisorslist. The former is only settable at cluster init time, while the lattercan be changed with cluster modify.
This becomes cumbersome in a few ways: at cluster init time for example...
HMAC authenticated json messages
This patch includes HMAC authenticated json messages to the serializer.The new interface works on any json-encodable data type, and can sign itwith a private key and an optional salt. The same private key must beused upon message loading to verify the message....
check_ident_key_val, handle no_ and - prefixes
If an ident member of an IdentKeyVal relationship starts with no_ or -,handle it the same way we do for a key. Some unittests are added tocheck that check_ident_key_val behaves as expected.
This patch also changes ForceDictType to, for now, fail on such an...
_SplitKeyVal with no data return an empty dict
If an empty string is passed to _SplitKeyVal, we should return {},rather than {'': True}. Also test for the correct behavior.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'next' into branch-2.1
Fix hooks unittests after RPC result changes
Patch "Simplify the RPC result framework in backend.py" changed allRPCs, and thus the hooks results where also changed. This needs changesto the hooks unittests too.
Convert hooks_runner rpc to new style result
This also converts (and fixes) unittests and mock objects to deal withthis change, and the custom hook verifier in cmdlib.LUClusterVerify.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Remove utils.CheckDict since it's no longer used
Only the custom handling in rpc.py for call_node_info used this, so nowit can be removed.
config unittests: use nic params
The target test cluster should have nic params, rather than a defaultbridge.
Add NIC.CheckParameterSyntax
This function will be used to check the NIC parameters for validity.Unittests are included.
IsNormAbsPath and users, use "normalized" term
We used to refer to normalized paths as "normal" which might beconfusing. This fixes the syntax in all current IsNormAbsPath users andin the docstring.
Add utils.IsNormAbsPath function
Currently most of the time we check for absolute path, but that doesn'tprotect us from some invalid paths. In some places we should be morestrict, and this function should help us to.
Export more instance information in hooks
Currently we miss in hooks the instance's hypervisor, hypervisorparameters and backend parameters. This forces hooks to query back intoganeti, which is dangerous due to possible luxi sockets exhaustion.
This patch adds these three as INSTANCE_HYPERVISOR, INSTANCE_HV_*,...
Fix the SafeEncoding behaviour
Currently we have bad behaviour in SafeEncode: - binary strings are actually not handled correctly (ahem) - the encoding is not stable, due to use of string_escape
For this reason, we replace the use of string_escape with part of the...
Fix compatibility with DRBD 8.3
DRBD 8.3 changes two more things compared to 8.2: - /proc/drbd format changed in multiple ways; the part we're interested is the ‘st:’ to ‘ro:‘ change (in the changelog named as “Renamed 'state' to 'role'” - “drbdsetup /dev/drbdN show” changed the ‘device’ stanza from:...
Convert IOErrors for /proc/drbd into our errors
If /proc/drbd can't be opened, this raises an IOError, but all theerror-handling behaviour in backend treats only BlockDeviceErrors. Thiscreates a plain failure in cluster verify and in other RPC calls.
This patch simply converts EnvironmentErrors into BlockDeviceErrors, and...
Apply the right permissions to /etc/hosts
In the current Ganeti version when modifying /etc/hosts we mistakenlygive it the permissions of the temporary file we create to define itscontent, which is by default 0600. This breaks most non-rootapplications, and thus must be corrected. This patch forces the mode to...
Instance parameters: force typing
We want all the hv/be parameters to have a known type, rather than arandom mix of empty string, boolean values, and None, so we declare thetype of each variable and we enforce/convert it.
- Add some new constants for enforceable value types...
Fix unittest encoding breakage
Due to the fact that we sanitize now the output from environmentscripts, the unittest needs to be adjusted. My bad for not checking it.
Reviewed-by: imsnah
RAPI: rlib1 removal
The resources we still need moved to rlib2.
Reviewed-by: iustinp
RAPI: Implement /2 resource
RAPI: Deprecate version Rapi version1
It is impossible to keep backward compatibility due tosignificant changes in the Ganeti core.
Introduce more configuration consistency checks
This patch enhances the duplicate DRBD minors checks (currently just afew) and adds automatic checks of configuration consistency atconfiguration file writing time.
In order to do so and show meaningful error messages, the...
DRBD: check for in-use minor during Create
In order to prevent errors with old, in-use DRBD minors, we check andabort at create time if our minor is already in use. For this we need toalso modify DRBD8Status to be able to parse cs:Unconfigured devices....
Add a TailFile function
This patch adds a tail file function, to be used for parsing and returning inthe job log OS installation failures.
Reviewed-by: ultrotter
Unify some unittest functions
This patch adds unified temporary file handling to thetestutils.GanetiTestCase class, which adds easy creation and automatedcleanup of temporary files.
The patch allows a simpler handling in a couple of test cases butrequires all child classes to call the parent setUp and tearDown...
ganeti.http: Add support for private data in HTTP requests
Reviewed-by: amishchenko
Add rename function automatically creating directories if needed
Unfortunately, os.makedirs in Python 2.4 is not safe against multipleprocesses creating the same directory tree at the same time. This isonly fixed in Python 2.5 and up. Adding more checks in our code doesn't...
Add simple unittests for ganeti.http
More complex unittests will need some refactoring in the HTTP code.
Rename all HTTP classes to camel case
It should be consistent.
ganeti.http: Remove ApacheLogfile class
We don't need it anymore and it wouldn't work as it is, anyway.
Fix hooks_unittest with new rpc call structure
RAPI:Fix root list and unittest for it.
Enable auto-unit formatting in script output
This patch enables by default the old 'human-readable' option, but in aslightly different model.
The option is now called "units" and takes either: - 'h' for automatic formatting - 'm', 'g' or 't' for mebi/gibi/tebibytes...
Move FieldSet class to utils.py
Since we can use the FieldSet class in cli.py to nicely format disksizes and such, we move it to utils.py and also move its associatedunittest. I didn't remove the cmdlib.py unittest file as that's not thegood direction :)...
Skip another testWaitingXblocksY
The waiting sharer blocks exclusive is another not-possible to testright now.
Sorry for missing it the first time; there are no other testWaiting...tests.
Add a FieldSet class for variable parameter sets
This patch adds a _FieldSet class that can be used for the new variableparameter sets: e.g. the sda_size will change to disk/0.size (orsimilar) and we need to both check validity and extract the index of the...
Skip the waitingExclusiveBlockSharer test
Until we have a way to properly test this, we'll skip the test.
Convert the locking unittests to repetition-test
Currently the locking tests are using timeouts to ensure an event will'never happen'. However, this is suboptimal.
The patch converts all of these to instead: not wait, but sequence theoperations logically and expect that they execute as such. In case of...
Remove zombie special case from IsProcessAlive
Based on the discussion on the mailing list, we remove the specialcasing of zombie processes.
Note we don't use kill -0 since that has a different meaning than "checkprocess is alive", so this function is still linux-specific....
Add cleanup of processes to utils.KillProcess
In case the process we want to signal is our own process, and the signalwe send is a deadly one, we should also cleanup after the process.
This patch adds a new parameter waitpid to this function that does this,...
Implement working directory for utils.RunCmd
In order to replace usage of the shell just for 'cd %s; ...' we need tobe able to specify the working directory for childs.
The patch also changes the default working directory (when notspecified) to "/", as opposed to the current dir; since the daemos run...
Modify utils.RunCmd to write output to file
Currently we launch processes via the shell in a few places only toredirect standard output and error to a log file ("&> $file"). It isbetter to do such redirection from within RunCmd itself.
This patch splits RunCmd in two parts, the setup and the execution part,...
Move some LU logging to use proc.Log*
We change some messages which are obviously intended for the user to usethe proc.Log* functions instead of (only) the logging ones.
We also fix some wrong uses of feedback_fn.
Generalize the reading of test file data
Currently we have two methods in ganeti.bdev_unittest.py of computingthe test data file name - and, of course, they don't give the sameresults.
The patch moves the functions to compute the test file name and reading...
Set default hypervisor at cluster init
During cluster init, set the default hypervisor to be used for instances.Ensure that the default hypervisor belongs to the set enabled hypervisorsfor this cluster. Also fix a small bug with setting the default enabled...
Convert cli.py to logging
We also add two function for printing messages, so that scripts won'thave to import logger to get these. They are a simple extension over thelogger ones, as they accept the call style from logging: ToStdout("Message: %s", msg)...
Remove old HTTP server code
All users of this code have been migrated to the new and shiny HttpServerclass.
It also fixes a typo in the ApacheLogfile unittests. It has not yet beendecided whether we should keep ApacheLogfile or not, hence leaving it in....
Fix remote API unittest
I broke it with rev 1864. This patch also removes whitespace atline endings.
Really fix with the zombie test (hopefully)
So, instead of any timeouts for synchronization, we move to actualevents. We still have one (huge) timeout for pathological cases (just tocleanup), but we don't sleep at all in normal usage.
The patch moves the zombie/cleaned up child creation out from setUp(),...
Implement parameter removal in SplitKeyVal
This patch adds paramter removal in SplitKeyVal, by prefixing avalue-less key with "-"; this is needed in resetting parameters back tocluster defaults, but care must be applied now that None can come fromthe parser....
Add two new options types for CLI usage
For the new 2.0-style command line options, we need to parse strings ofthe type: ident:key=val[,...]and key=val[,...]
This patch adds two new option builders for these two, which return(ident, {key=val,}) and {key=val,} for the above two formats. It also...
Abstract checking own address into a function
Currently, we check if we have a given ip address (i.e. it's alive onone of our interfaces) but manually calling TcpPing(source=localhost).This works, but having it spread all over the code makes it hard to...
Convert rpc module to RpcRunner
This big patch changes the call model used in internode-rpc fromstandalong function calls in the rpc module to via a RpcRunner class,that holds all the methods. This can be used in the future to enablesmarter processing in the RPC layer itself (some quick examples are not...
Ajust config unittest.
Add a simple timespec parsing function
This function will be used for auto-archiving jobs via the command line.The function is pretty simple, we only support up to weeks since monthsand higher are not 'precise' entities, and dealing with them wouldrequire us to start using calendar functions....
Get rid of ssconf
Remove leftovers from ssconf.
Convert ssh.py
Get rid of ssconf and convert to configuration instead.
Convert mcpu.py
Replacing ssconf with configuration.
Fix unittests broken by revision 1727
Add cluster options from ssconf to configuration
ssconf will become write-only from ganeti-masterd's point of view,therefore all settings in there need to go into the main configurationfile.
Move instantiation of config into bootstrap.py
Future patches will add even more variables to the cluster config.Adding more parameters wouldn't make the function easier to use andit doesn't make sense to pass them to another function, as it'sonly done once in bootstrap.py on cluster initialization....
LockSet: forbid add() on a partially owned set
This patch bans add() on a half-acquired set. This behavior waspreviously possible, but created a deadlock if someone tried to acquirethe set-lock in the meantime, and thus is now forbidden. ThetestAddRemove unit test is fixed for this new behavior, and includes a...
Fix LockSet._names() to work with the set-lock
If the set-lock is acquired, currently, the _names function will fail ona double acquire of a non-recursive lock. This patch fixes the behavior,and some lines of code added to the testAcquireSetLock test check that...
utils.SplitTime: More rounding fixes
SplitTime didn't round the same on different platforms. This patch changesit to use microseconds and not care about rounding.
utils.SplitTime: Fix rounding of milliseconds
Reported by Iustin.
It used to return this:
utils.SplitTime(1234.999999999999)
(1234, 1000)
while it should've returned this:
(1235, 0)
A few more locking unit tests
A few more tests written while bug-hunting. One of them shows a realissue, at last. :)
Add lock-all-through-GLM unit test
I was hunting for a bug in my code and thought the culprit was in thelocking library, so I added a test to check. Unfortunately turns out itwasn't. :( Committing the test anyway, while still trying to figure outwhat's wrong......
Add functions to split time into tuple and merge it back
These will be used for job logs.
Add FileLock utility class
This class is a wrapper around fcntl.flock and abstracts opening andclosing the lockfile. It'll used for the job queue.
(The patch also removes a duplicate import of tempfile into the unittest)
First write operation (add tag) for Ganeti RAPI
Add instance tag handling, improved error logging....oh, yes adopt instance listing for RAPI2!
Expose utils.DaemonPidFileName
Since we need to compute this from outside utils.py, we change this to apublic function.
Add a KillProcess function
We cannot depend on all environments to have a start-stop-daemon orsimilar tool. We instead implement a KillProcess function that behavessimilar to “start-stop-daemon --retry”.
Note that the attached unittest can hang in foreground if the child...
Change IsPidFileAlive into ReadPidFile
We already have a function to test if a PID is alive, so it makes moresense to use function composition that force calling (since we need toread PIDs from files in other places too). Now IsProcessAlive returnsFalse for PIDs <= 0, since this is the error return from ReadPidFile....
Fix unittests for ganeti-rapi
The RESTHTTPServer module went the way of the dodo.
Switch RAPI to ganeti.http module
Add utils unittests for new functions
The submitted WritePidFile, RemovePidfile and IsPidFileAlive functionsmiss unit tests. Adding a simple one which covers their basicfunctionality.
Invert nodes/instances locking order
An implementation mistake from the original design caused nodes to belocked before instances, rather than after. This patch inverts the levelnumbering, changing also the relevant unittests and the recursivelocking function starting point....
Split RAPI resources to pieces
Copy the rest of the Restful-API files to trunk
Fix utils.py unittest
Remove the old locking functions
This removes (hopefully) all traces of the old locking functions anduses.
Add a more comment lines to testLockingConstants
This is to discourage even more whoever may think that this requirementis not really useful and can be lifted, and to at least know where it'sused before trying to break it.
Add a new LockSet unittest
This test checks the LockSet behaviour when an empty list is passed.The current behaviour is expected, but since this is a corner case,we're safer to keep it under a check, and if we need a different onemonitor that everything is as we expect it to be....
Pass context to LUs
Rather than passing a ConfigWriter to the LUs we'll pass the wholecontext, from which a ConfigWriter can be extracted, but we can alsoaccess the GanetiLockManager. This also fixes the places where a FakeLUis created.
mocks: create a FakeContext object
This will be passed to FakeLUs