History | View | Annotate | Download (102.2 kB)
Add new parameter type “maybe string”
Before strict checking was implemented, NIC IP addresses could be setto “None”. Commit bd061c35 added more strict checking, includingenforcing the IP address to be a string. With this new type, itcan again be set to None....
utils: Fix incorrect docstring
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Always set commonName in X509 certificates
Due to the current switch of the RPC client to PycURL, a bug with newerversions of libcurl surfaced. When the 'Subject' or 'Issuer' of'server.pem' were empty, SSL handshake failed.
This patch changes the certificate generation functions such that they...
Change AddAuthorizedKey to also allow filehandles
This is required to use this function over paramikosftp file handles.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add ParseCpuMask() utility function
Also adds a generic ParseError exception.
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Move ShellWriter class to utils
Also add unittest.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Introduce lib/netutils.py
This patch moves network utility functions to a dedicated module.
Merge branch 'devel-2.1'
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Mlockall: decrease warnings if ctypes module is not present
Node daemon prints a lot of warnings if --no-mlock option is not specified andctypes module is not present.
With the following patch the warning is printed only at noded startup.
Signed-off-by: Luca Bigliardi <shammash@google.com>...
Add utils.GetMounts()
utils.OwnIpAddress: Change try/except for Python 2.4
Python 2.4 doesn't support “except” and “finally” in the same block.
Enable utils.OwnIpAddress to work with IPv6 addresses
Currently a TcpPing using the target address as source is used todetermine wheter or not an IP address belongs to a node. This does notwork for IPv6 anymore (I'm not sure why) as binding to IPv6 localhost as...
IPv6 support for utils.TcpPing()
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add function to retrieve family of an ip address
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Introduce utils.IsValidIP{4,6}()
This patch introduces functions to check for valid IPv4 and IPv6addresses and converts IsValidIP() to return True if it is either a IPv4or a IPv6 address.
For now we do not change the functional behavior and replace IsValidIP...
Rename some constants to facilitate IPv6 support
Fix a few uncommon pylint errors in compat.py
- It's ok if the optional modules functools, roman and hashlib are not there.- It's ok to reference any and all before defining them. We're doing all this exactly because, under python 2.4, they are undefined....
Remove utils.LockedMethod
All users of this function were converted to another method, hence wecan remove it.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'stable-2.1'
Fix handling of errors from socket.gethostbyname
Socket functions can raise more than just gaierror. Most of the times,socket.gethostbyname_ex will return gaierror, but rarely it will alsoraise herror. For completeness, we catch all socket exceptions with data...
ListVisibleFiles: do not sort output
Among all users, turns out just one may need the output to be sorted.All the others can cope without.
utils: Add function to validate service name
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Handle ESRCH when sending signals
Upon sending signals, ESRCH can be reported when the target nolonger exists.
ListVisibleFiles: do optional sorting
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Make it possible to call utils.Daemonize with uid and gid to run as
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
utils: Add function to format seconds
ganeti-watcher should attempt to fix ganeti-rapi
Update ganeti-watcher so that it tests the master's RAPI port with asimple test (in this case GetVersion). If it fails, make one attemptat restarting ganeti-rapi and retest.
- daemons/ganeti-watcher: Test rapi and make one attempt at restarting it....
Add RemoveDir utility function
Backported from master, 72087dcd5b06c0127e2ec3bf8c80f7f54da3fb01
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge remote branch 'origin/devel-2.1'
Conflicts: test/ganeti.daemon_unittest.py...
Explicitely return None from IgnoreSignals
Same result, but what happens is clearer.
utils: Add function to check whether process handles a signal
This will be used to avoid a race condition between starting a program (ddfor import/export) and sending signals to it.
Fix IgnoreSignals on socket.error
Some confusion arose handling EINTR on this function: in python 2.6socket.error is an IOError, and thus: - It's an EnvironmentError - It has an .errno member
In 2.4 and 2.5 it's not, and so its errno variable must be extracted...
Move hash functions to the compat module
Since the hash functions' changed their module name between python 2.4and 2.6, and we have to do an try/import/except trick, we'll do it justonce, for both hash functions, and in compat.py. This also fixes a use...
Fix {Ignore, RetryOn}Signals on socket.error
Some confusion arose handling EINTR on those functions: in python 2.6socket.error is an IOError, and thus: - It's an EnvironmentError - It has an .errno member
Make utils.EnsureDirs() ignore umask
EnsureDirs() should create directories with the exact mode requestedin the arguments, but it currently applies the umask.This patch makes it independent from the umask.
Signed-off-by: Balazs Lecz <leczb@google.com>...
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RetryOnSignal: handle socket error as well
utils.IgnoreSignals
Remove duplicate code between a couple of asyncore related function byhaving a function in charge of handling EINTR errors. Unittests included.
Conflicts: daemons/ganeti-noded lib/daemon.py lib/rapi/baserlib.py lib/rapi/rlib2.py lib/utils.py
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix some pylint warnings
Disable warnings for:- except Exception,- use of __errno_location,- redeclaration of handleError()
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix logging string format
Fix this pylint warning:[W6501, Mlockall] Specify string format arguments as logging function parameters
Fix Filehandler / FileHandler typo
Fix typo spotted by pylint:E1101:2095:LogFileHandler.handleError: Module 'logging' has no 'Filehandler' member
Use LogFileHandler conditionally in SetupLogging
Add a parameter to SetupLogging to use LogFileHandler (default is off)
Introduce LogFileHandler class
Add a log handler class which logs to /dev/console in case of logging errors.
Introduce Mlockall()
Add Mlockall() utility to lock current process' virtual adress space into RAM.
Convert some ReadFile calls to ReadOneLineFile
For passwords we require strict oneliners, we're a bit more lax with pidand uid files.
utils.ReadOneLineFile()
Read the first non-empty file line. When strict is set, abort if morethan one line is non-empty. Some unittests inspired by the reverted onesfrom commit b774bb106cc28d008e790ad2666eb64c76866fa0, and some new ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Remove oneline= parameter from utils.ReadFile
This partially reverts commit b774bb106cc28d008e790ad2666eb64c76866fa0.Unittests unrelated to that particular functionality but introduced inthat commit are left untouched. Since the temporary directory is now...
utils.ReadFile: Add parameter to read only one line
Reading only one line is useful for reading passwords from files. Thisalso adds unittests for utils.ReadFile.
Retry{Again,Timeout}: explain reraising
IsProcessAlive: retry stat() a few times
On multiprocessor dom0 stat() on /proc can sometimes return EINVAL.Before giving up, we try a few times to get a consistent answer.
utils.Retry: pass up timeout arguments
If Retry has to fail with RetryTimeout, it might be useful to pass theRetry argument to RetryTimeout, to help debugging outside the Retrycycle. We also define a RetryTimeout.RaiseInner() helper, to re-raise anexception passed to RetryAgain. All served with a side of unit tests....
Add optional “salt” argument to utils.{,Verify}Sha1Hmac
We're using salted hashes all over the place.
Use utility functions for HMAC
HMAC will be used in more places. Centralizing some parts can't hurt.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Move cmdlib._VerifyCertificate to utils
This function will also be useful for inter-cluster instancemoves for verifying certificates.
Add wrapper class for signal.set_wrapper_fd
Managing file descriptors is always a bit tricky. Having this in a separateclass is better.
Conflicts: lib/utils.py: Trivial
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
masterd: Log PID, UID and GID of connected client
This can be very useful if client programs run as non-root.
Add separate module for backported language functionality
utils.py, where they were before, is already huge.
utils: Add function for partial application of function arguments
The function's code was mostly copied from Python's documentationand it's equivalent to “functools.partial” in Python 2.5 and above.
utils: Add function to read locked PID file
This is useful in combination with utils.StartDaemon and will be used forreading the import/export daemon's PID file.
utils: Add class to split string stream into lines
This will be used by the new import/export daemon to splitoutput by its child processes into lines.
Fix utils.WaitForFdCondition inner retry loop
Commit dfdc4060 added WaitForFdCondition which uses utils.Retry withouthandling timeout exceptions. This breaks any nested retry loops.
This patch fixes the above function, and also changes utils.Retry todetect and warn future similar cases. In addition, we add a few small...
Fix bug introduced in 76e5f8b54: mkdir mode
After commit 76e5f8b54, mkdir_mode in utils.RenameFile isno longer passed to Makedirs. This is fixed by this patch.
utils: Move wrapper code around os.makedirs into separate function
Revert "Only override any and all if not defined"
This reverts commit bd5617020a50bcd08269330638d64078c1b30b71.
Turns out our and python's any/all are not compatible.
move http.WaitForSocketCondition to utils
WaitForSocketCondition: rename, handle EINTR
- Rename WaitForSocketCondition to SingleWaitForFdCondition - Avoid potentially infinite loop, if we continue to get interrupted - Handle eintr correctly - Avoid the poller try/finally, as the poller object gets destroyed...
Only override any and all if not defined
If any or all are already defined (because we're using a new version ofpython) just link them inside "utils" rather than redefining them.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
utils: Add functions to sign and verify X509 certs using HMAC
Certificates exchanged via an untrusted third party should besigned to ensure they haven't been modified.
utils.CreateBackup: Use human-readable instead of seconds since Epoch
Seconds since the Epoch are not easily readable by a human. Using aformatted timestamp makes it easier (e.g.“….backup-2010-03-12_14_02_43.…”). This patch also makes OS logfiles usethis formatted timestamp....
utils: Add function to extract X509 cert validity
X509 uses ASN1 GENERALIZEDTIME or UTCTIME to store certificate validity.pyOpenSSL 0.7 and above allow us to retrieve both “notBefore” and“notAfter” as strings. Parsing them turned out to be a challenge since...
Add a partition function to split a list in two
This is similar to the Haskell function, except that the signature isreverse to match the 'any' and 'all' Python functions.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Support passing in file object in utils.FileLock
This way we can re-use file objects opened in other places. Also add moreunittests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Support arguments in utils.RunInSeparateProcess
Convert utils.FileLock to utils.Retry
Add a function to validate and normalize hostnames
This differs slightly from the specification, by allowing names to startwith digits, not checking the length of individual components, andallowing underscores.
Signed-off-by: Iustin Pop <iustin@google.com>...
ListVisibleFiles: require normalized path names
This patch changes ListVisibleFiles to raise ProgrammerError if it'spassed a non-absolute/non-normalized path name, and adds unittests forthis behaviour.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Replace os.path.sep.join(seq) with utils.PathJoin
This is a no-op change, but at least we concentrate the calls to pathjoins into a single function.
A use in utils.FindFile is left as-is (don't want to raise exceptionsthere, at least for now).
Switch from os.path.join to utils.PathJoin
This passes a full burnin with lots of instances, and should be safe aswe mostly to join a known root (various constants) to a run-timevariable.
utils: Add a PathJoin function
This will replace os.path.join since it is not safe for directorytraversal issues.
Move watcher's EnsureDaemon function to utils
This is going to be used from the nbma repository, to ensure that thenld daemon is running.
Implement utils.RunParts and use it for hooks
This function is a generic pythonic version of runparts. We currentlyuse it in the backend HooksRunner, but we'll use it for runningdifferent directories as well.
Add reset_env option to RunCmd
This allows to run a command with only the passed in environment, ratherthan just updating the default one with it.
Now with unit testing.
Fix lint error in StartDaemon and rename child function
This patch fixes the following pylint error:W0212:257:StartDaemon: Access to a protected member _exit of a clientclass
Additionally, it renames the _RunCmdDaemonChild function to_StartDaemonChild. _RunCmdDaemonChild was a leftover from a previous...
Add utility function to start daemon
The currently available functions in this direction (utils.RunCmd andutils.Daemonize) both can not be used to start an external daemonprocess. This function takes many steps to ensure the daemon is startedproperly and does careful error checking. Unittests are included....
Use OpenSSL module instead of binary to generate certs
This saves us one dependency and saves us from complicated handling ofexternal files if we need key and certificate separated from each other.
At the same time, the number of bits used for RSA keys is increased from...
utils.SignalHandler: Add support for handler function
This is useful if something else needs to be notified on signal arrival.
utils: Add function to set O_NONBLOCK on file descriptor