History | View | Annotate | Download (10.6 kB)
Added function for parsing multiple CPU pinning mask
Signed-off-by: Tsachy Shacham <tsachy@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
utils: Move NewUUID to utils.io
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
utils: Move more text-related code
utils: Move process-related code into separate file
utils: Move RunningTimeout to utils.algo
utils: Move code manipulating /etc/hosts to separate file
utils: Move X509-related code into separate file
utils: Move I/O-related code into separate file
utils: Move code related to file locking into separate file
utils: Move wrappers into separate file
utils: Use function to disable fork
Use a function instead of a variable written by another module.
utils: Change RemovePidFile to take path, not name
This avoids having to monkey-patch the utils module for unittests.Monkey patching is evil and caused a bug while moving code around.
utils: Move hashing-related code into separate file
utils: Split algorithms into separate file
utils: Split Retry & co. into separate file
utils: Move text-related functions into separate file
Remove utils.FormatTimestampWithTZ
Long story short: time.strftime("%Z", time.localtime()) doesn't work,even though it's documented to be equivalent to time.strftime("%Z").
$ TZ=America/Sao_Paulo python -c 'import time; printtime.strftime("%Z"), time.strftime("%Z", time.localtime())'...
Ensure temp files from RunCmd tests are removed
utils: Timezone fixes and tests
- Update docstrings to explicitely mention Epoch- Fix timezone bug in FormatTimestampWithTZ, where it would use GMT/UTC when it should use the local timezone- Add unittests for time formatting functions
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
utils.NiceSort: Use sorted(), add keyfunc, unittests
This patch changes utils.NiceSort to use the built-in “sorted()” andgets rid of the intermediate list. Instead of wrapping the itemsourselves, a key function is used. The caller can specify another key...
utils: Add function to find duplicates in sequence
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
Improve unittests for the utils module
This just a random collection of unittest improvements. Coverageincreases from 73% to 76%.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix utils unittest TestRunCmd.testTimeoutKill
Currently the test executes "/bin/sh -c 'trap "" TERM …" via the shell,which means we have two shells involved (the parent one and the childone, which does the trap). For some weird reason, this works for older...
Temporarily remove the interactive tests
make check < /dev/null breaks these tests. Until we have change RunCmdto override stdin, we need to remove them.
Support timeouts in RunCmd
Further investigations have to be done for merging some of these bitstogether with import-export daemon which uses similiar logic.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Adding unittests for RunningTimeout
Add a "safe" file wrapper over WriteFile
Add functions to read and compare file 'ID's
utils: Add function to find items in dictionary using regex
This basically extracts a small piece of code from ganeti-rapi and putsit into a utility function. RAPI resources are found using a dictionaryin which the keys can either be static strings or compiled regular...
Use only one version of WritePidFile
This patch merges the pid file handling used for ganeti-* daemons andimpexp daemons. The latter version is used, since it's more reliable:uses locked pid files as opposed to checking 'live' processes.
Signed-off-by: Iustin Pop <iustin@google.com>...
Merge branch 'devel-2.2'
Add simple unittest for utils.CommaJoin
Move the uuid regexp to utils.py
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add new parameter type “maybe string”
Before strict checking was implemented, NIC IP addresses could be setto “None”. Commit bd061c35 added more strict checking, includingenforcing the IP address to be a string. With this new type, itcan again be set to None....
Add ParseCpuMask() utility function
Also adds a generic ParseError exception.
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Move ShellWriter class to utils
Also add unittest.
Rename test for utils.IgnoreProcessNotFound
Usually our tests are named “Test…”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>
Use reserved documentation IPs and domains
Use RFC 5737 IP addresses and RFC 2606 domain names in allunittests, docs, qa and docstrings.
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Introduce lib/netutils.py
This patch moves network utility functions to a dedicated module.
Add utils.GetMounts()
IPv6 support for utils.TcpPing()
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add function to retrieve family of an ip address
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Introduce utils.IsValidIP{4,6}()
This patch introduces functions to check for valid IPv4 and IPv6addresses and converts IsValidIP() to return True if it is either a IPv4or a IPv6 address.
For now we do not change the functional behavior and replace IsValidIP...
User assertFalse instead of assert_(not ...)
Rename some constants to facilitate IPv6 support
ListVisibleFiles: do not sort output
Among all users, turns out just one may need the output to be sorted.All the others can cope without.
utils: Add function to validate service name
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Handle ESRCH when sending signals
Upon sending signals, ESRCH can be reported when the target nolonger exists.
ListVisibleFiles: do optional sorting
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
utils: Add function to format seconds
utils: Add function to check whether process handles a signal
This will be used to avoid a race condition between starting a program (ddfor import/export) and sending signals to it.
Merge branch 'devel-2.1'
Fix IgnoreSignals on socket.error
Some confusion arose handling EINTR on this function: in python 2.6socket.error is an IOError, and thus: - It's an EnvironmentError - It has an .errno member
In 2.4 and 2.5 it's not, and so its errno variable must be extracted...
Move hash functions to the compat module
Since the hash functions' changed their module name between python 2.4and 2.6, and we have to do an try/import/except trick, we'll do it justonce, for both hash functions, and in compat.py. This also fixes a use...
Fix {Ignore, RetryOn}Signals on socket.error
Some confusion arose handling EINTR on those functions: in python 2.6socket.error is an IOError, and thus: - It's an EnvironmentError - It has an .errno member
Make utils.EnsureDirs() ignore umask
EnsureDirs() should create directories with the exact mode requestedin the arguments, but it currently applies the umask.This patch makes it independent from the umask.
Signed-off-by: Balazs Lecz <leczb@google.com>...
Add checks for master IP in cluster verify
This also updates a comment in the unittest for utils.py. We unittestthe new function for two things: correct reporting on real case (forlocalhost), and correct reporting with a mocked-out TcpPing that returns...
utils.IgnoreSignals
Remove duplicate code between a couple of asyncore related function byhaving a function in charge of handling EINTR errors. Unittests included.
utils.ReadOneLineFile()
Read the first non-empty file line. When strict is set, abort if morethan one line is non-empty. Some unittests inspired by the reverted onesfrom commit b774bb106cc28d008e790ad2666eb64c76866fa0, and some new ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Remove oneline= parameter from utils.ReadFile
This partially reverts commit b774bb106cc28d008e790ad2666eb64c76866fa0.Unittests unrelated to that particular functionality but introduced inthat commit are left untouched. Since the temporary directory is now...
utils.ReadFile: Add parameter to read only one line
Reading only one line is useful for reading passwords from files. Thisalso adds unittests for utils.ReadFile.
Fix a unittest name and docstring
The docstring has been copied by the name, but the name reports amisspell of the function it is actually testing.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Fix a unittest docstring
utils.Retry: pass up timeout arguments
If Retry has to fail with RetryTimeout, it might be useful to pass theRetry argument to RetryTimeout, to help debugging outside the Retrycycle. We also define a RetryTimeout.RaiseInner() helper, to re-raise anexception passed to RetryAgain. All served with a side of unit tests....
Add a few Retry unittests
Check that Retry actually times out, and actually retries.
Add optional “salt” argument to utils.{,Verify}Sha1Hmac
We're using salted hashes all over the place.
Use utility functions for HMAC
HMAC will be used in more places. Centralizing some parts can't hurt.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Move cmdlib._VerifyCertificate to utils
This function will also be useful for inter-cluster instancemoves for verifying certificates.
Conflicts: lib/utils.py: Trivial
masterd: Log PID, UID and GID of connected client
This can be very useful if client programs run as non-root.
Add separate module for backported language functionality
utils.py, where they were before, is already huge.
utils: Add function for partial application of function arguments
The function's code was mostly copied from Python's documentationand it's equivalent to “functools.partial” in Python 2.5 and above.
utils: Add function to read locked PID file
This is useful in combination with utils.StartDaemon and will be used forreading the import/export daemon's PID file.
utils: Add class to split string stream into lines
This will be used by the new import/export daemon to splitoutput by its child processes into lines.
Fix utils.WaitForFdCondition inner retry loop
Commit dfdc4060 added WaitForFdCondition which uses utils.Retry withouthandling timeout exceptions. This breaks any nested retry loops.
This patch fixes the above function, and also changes utils.Retry todetect and warn future similar cases. In addition, we add a few small...
utils: Move wrapper code around os.makedirs into separate function
Merge remote branch 'devel-2.1'
Remove unused import from test file
utils: Add functions to sign and verify X509 certs using HMAC
Certificates exchanged via an untrusted third party should besigned to ensure they haven't been modified.
Merge remote branch 'origin/devel-2.1'
utils.RunCmd: Test case with reset_env set and setting variables
utils.CreateBackup: Use human-readable instead of seconds since Epoch
Seconds since the Epoch are not easily readable by a human. Using aformatted timestamp makes it easier (e.g.“….backup-2010-03-12_14_02_43.…”). This patch also makes OS logfiles usethis formatted timestamp....
utils: Add function to extract X509 cert validity
X509 uses ASN1 GENERALIZEDTIME or UTCTIME to store certificate validity.pyOpenSSL 0.7 and above allow us to retrieve both “notBefore” and“notAfter” as strings. Parsing them turned out to be a challenge since...
Support passing in file object in utils.FileLock
This way we can re-use file objects opened in other places. Also add moreunittests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Support arguments in utils.RunInSeparateProcess
Add a function to validate and normalize hostnames
This differs slightly from the specification, by allowing names to startwith digits, not checking the length of individual components, andallowing underscores.
ListVisibleFiles: require normalized path names
This patch changes ListVisibleFiles to raise ProgrammerError if it'spassed a non-absolute/non-normalized path name, and adds unittests forthis behaviour.
utils: Add a PathJoin function
This will replace os.path.join since it is not safe for directorytraversal issues.
Implement utils.RunParts and use it for hooks
This function is a generic pythonic version of runparts. We currentlyuse it in the backend HooksRunner, but we'll use it for runningdifferent directories as well.
Add reset_env option to RunCmd
This allows to run a command with only the passed in environment, ratherthan just updating the default one with it.
Now with unit testing.
Add utility function to start daemon
The currently available functions in this direction (utils.RunCmd andutils.Daemonize) both can not be used to start an external daemonprocess. This function takes many steps to ensure the daemon is startedproperly and does careful error checking. Unittests are included....
Use OpenSSL module instead of binary to generate certs
This saves us one dependency and saves us from complicated handling ofexternal files if we need key and certificate separated from each other.
At the same time, the number of bits used for RSA keys is increased from...
utils: Add function to set O_NONBLOCK on file descriptor