History | View | Annotate | Download (105.7 kB)
Add a "safe" file wrapper over WriteFile
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add functions to read and compare file 'ID's
utils: Add function to find items in dictionary using regex
This basically extracts a small piece of code from ganeti-rapi and putsit into a utility function. RAPI resources are found using a dictionaryin which the keys can either be static strings or compiled regular...
Fix epydoc errors
And sorry!
Fix a rare bug in StartDaemonChild and GenericMain
I've seen cases where the result from str(sys.exc_info()[1]) is ""; thisbreaks the error reporting as the parent relies on non-empty errormessages to properly detect child status (otherwise it will try to read...
Change daemon.GenericMain/utils.Daemonize workflow
This patch copies the pipe-based error reporting functionality fromutils.StartDaemon (I gave up for now on tryin to merge the two).
This patch will fix two longstanding bugs:
- if we fork, we lose all error reporting from the child to the original...
Use only one version of WritePidFile
This patch merges the pid file handling used for ganeti-* daemons andimpexp daemons. The latter version is used, since it's more reliable:uses locked pid files as opposed to checking 'live' processes.
Signed-off-by: Iustin Pop <iustin@google.com>...
Abstract daemon file descriptor setup
This does some slight changes:
- Daemonize() doesn't explicitly close the file-descriptors anymore, but only implicitly via the usage of dup2- StartDaemonChild uses separate devnull for stdin (rdonly) and stdout/stderr (wronly), or if using a log file, it uses it in append...
Abstract some daemon functionality
This patch abstracts the chdir/umask/setsid functionality, which isidentical in the code functions, just that Daemonize did the chdir/umaskin the second child; with this change it does it in the first, asStartDaemon....
Move the uuid regexp to utils.py
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.2'
Merge branch 'devel-2.1' into devel-2.2
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix mac checker regex
Currently, the mac checker regex could match a corner case of11:22:33:44:55:66: (one extra colon at the end). We fix this, and wealso move the regex compilation outside of this function, at modulelevel.
Revert "Make it possible to call utils.Daemonize with uid and gid to run as"
This reverts commit 743b53d4eb9f3de46edb5e54738dab287b1979ac.
Conflicts:
lib/daemon.py
Trivial conflict resolved. This patch reverts changes from earlier permissionsseparation stage. This is not needed anymore as start-stop-daemon takes care...
Switch to the RPC call to update /etc/hosts in LUAddNode and LURemoveNode
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
utils: Use WriteFile in {Set,Remove}EtcHostsEntry
This avoids duplicate effort and has been a TODO for a long time.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Make it possible to invoke RunCmd in interactive mode
This is needed so we can run external scripts asking for password and suchto run interactive. Downside is that we can't get the output of the programbut we can still use the exit code.
Signed-off-by: René Nussbaumer <rn@google.com>...
Support IPv6 node add
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Support for resolving hostnames to IPv6 addresses
This patch enables IPv6 name resolution by using socket.getaddrinfoinstead of socket.gethostbyname_ex.
It renames the HostInfo class to Hostname and unifies its use throughoutthe code. This is achieved by using static calls where no object is...
Add new parameter type “maybe string”
Before strict checking was implemented, NIC IP addresses could be setto “None”. Commit bd061c35 added more strict checking, includingenforcing the IP address to be a string. With this new type, itcan again be set to None....
utils: Fix incorrect docstring
Always set commonName in X509 certificates
Due to the current switch of the RPC client to PycURL, a bug with newerversions of libcurl surfaced. When the 'Subject' or 'Issuer' of'server.pem' were empty, SSL handshake failed.
This patch changes the certificate generation functions such that they...
Change AddAuthorizedKey to also allow filehandles
This is required to use this function over paramikosftp file handles.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add ParseCpuMask() utility function
Also adds a generic ParseError exception.
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Move ShellWriter class to utils
Also add unittest.
Introduce lib/netutils.py
This patch moves network utility functions to a dedicated module.
Merge branch 'devel-2.1'
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Mlockall: decrease warnings if ctypes module is not present
Node daemon prints a lot of warnings if --no-mlock option is not specified andctypes module is not present.
With the following patch the warning is printed only at noded startup.
Signed-off-by: Luca Bigliardi <shammash@google.com>...
Add utils.GetMounts()
utils.OwnIpAddress: Change try/except for Python 2.4
Python 2.4 doesn't support “except” and “finally” in the same block.
Enable utils.OwnIpAddress to work with IPv6 addresses
Currently a TcpPing using the target address as source is used todetermine wheter or not an IP address belongs to a node. This does notwork for IPv6 anymore (I'm not sure why) as binding to IPv6 localhost as...
IPv6 support for utils.TcpPing()
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add function to retrieve family of an ip address
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Introduce utils.IsValidIP{4,6}()
This patch introduces functions to check for valid IPv4 and IPv6addresses and converts IsValidIP() to return True if it is either a IPv4or a IPv6 address.
For now we do not change the functional behavior and replace IsValidIP...
Rename some constants to facilitate IPv6 support
Fix a few uncommon pylint errors in compat.py
- It's ok if the optional modules functools, roman and hashlib are not there.- It's ok to reference any and all before defining them. We're doing all this exactly because, under python 2.4, they are undefined....
Remove utils.LockedMethod
All users of this function were converted to another method, hence wecan remove it.
Merge branch 'stable-2.1'
Fix handling of errors from socket.gethostbyname
Socket functions can raise more than just gaierror. Most of the times,socket.gethostbyname_ex will return gaierror, but rarely it will alsoraise herror. For completeness, we catch all socket exceptions with data...
ListVisibleFiles: do not sort output
Among all users, turns out just one may need the output to be sorted.All the others can cope without.
utils: Add function to validate service name
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Handle ESRCH when sending signals
Upon sending signals, ESRCH can be reported when the target nolonger exists.
ListVisibleFiles: do optional sorting
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Make it possible to call utils.Daemonize with uid and gid to run as
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
utils: Add function to format seconds
ganeti-watcher should attempt to fix ganeti-rapi
Update ganeti-watcher so that it tests the master's RAPI port with asimple test (in this case GetVersion). If it fails, make one attemptat restarting ganeti-rapi and retest.
- daemons/ganeti-watcher: Test rapi and make one attempt at restarting it....
Add RemoveDir utility function
Backported from master, 72087dcd5b06c0127e2ec3bf8c80f7f54da3fb01
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge remote branch 'origin/devel-2.1'
Conflicts: test/ganeti.daemon_unittest.py...
Explicitely return None from IgnoreSignals
Same result, but what happens is clearer.
utils: Add function to check whether process handles a signal
This will be used to avoid a race condition between starting a program (ddfor import/export) and sending signals to it.
Fix IgnoreSignals on socket.error
Some confusion arose handling EINTR on this function: in python 2.6socket.error is an IOError, and thus: - It's an EnvironmentError - It has an .errno member
In 2.4 and 2.5 it's not, and so its errno variable must be extracted...
Move hash functions to the compat module
Since the hash functions' changed their module name between python 2.4and 2.6, and we have to do an try/import/except trick, we'll do it justonce, for both hash functions, and in compat.py. This also fixes a use...
Fix {Ignore, RetryOn}Signals on socket.error
Some confusion arose handling EINTR on those functions: in python 2.6socket.error is an IOError, and thus: - It's an EnvironmentError - It has an .errno member
Make utils.EnsureDirs() ignore umask
EnsureDirs() should create directories with the exact mode requestedin the arguments, but it currently applies the umask.This patch makes it independent from the umask.
Signed-off-by: Balazs Lecz <leczb@google.com>...
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RetryOnSignal: handle socket error as well
utils.IgnoreSignals
Remove duplicate code between a couple of asyncore related function byhaving a function in charge of handling EINTR errors. Unittests included.
Conflicts: daemons/ganeti-noded lib/daemon.py lib/rapi/baserlib.py lib/rapi/rlib2.py lib/utils.py
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix some pylint warnings
Disable warnings for:- except Exception,- use of __errno_location,- redeclaration of handleError()
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix logging string format
Fix this pylint warning:[W6501, Mlockall] Specify string format arguments as logging function parameters
Fix Filehandler / FileHandler typo
Fix typo spotted by pylint:E1101:2095:LogFileHandler.handleError: Module 'logging' has no 'Filehandler' member
Use LogFileHandler conditionally in SetupLogging
Add a parameter to SetupLogging to use LogFileHandler (default is off)
Introduce LogFileHandler class
Add a log handler class which logs to /dev/console in case of logging errors.
Introduce Mlockall()
Add Mlockall() utility to lock current process' virtual adress space into RAM.
Convert some ReadFile calls to ReadOneLineFile
For passwords we require strict oneliners, we're a bit more lax with pidand uid files.
utils.ReadOneLineFile()
Read the first non-empty file line. When strict is set, abort if morethan one line is non-empty. Some unittests inspired by the reverted onesfrom commit b774bb106cc28d008e790ad2666eb64c76866fa0, and some new ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Remove oneline= parameter from utils.ReadFile
This partially reverts commit b774bb106cc28d008e790ad2666eb64c76866fa0.Unittests unrelated to that particular functionality but introduced inthat commit are left untouched. Since the temporary directory is now...
utils.ReadFile: Add parameter to read only one line
Reading only one line is useful for reading passwords from files. Thisalso adds unittests for utils.ReadFile.
Retry{Again,Timeout}: explain reraising
IsProcessAlive: retry stat() a few times
On multiprocessor dom0 stat() on /proc can sometimes return EINVAL.Before giving up, we try a few times to get a consistent answer.
utils.Retry: pass up timeout arguments
If Retry has to fail with RetryTimeout, it might be useful to pass theRetry argument to RetryTimeout, to help debugging outside the Retrycycle. We also define a RetryTimeout.RaiseInner() helper, to re-raise anexception passed to RetryAgain. All served with a side of unit tests....
Add optional “salt” argument to utils.{,Verify}Sha1Hmac
We're using salted hashes all over the place.
Use utility functions for HMAC
HMAC will be used in more places. Centralizing some parts can't hurt.
Move cmdlib._VerifyCertificate to utils
This function will also be useful for inter-cluster instancemoves for verifying certificates.
Add wrapper class for signal.set_wrapper_fd
Managing file descriptors is always a bit tricky. Having this in a separateclass is better.
Conflicts: lib/utils.py: Trivial
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
masterd: Log PID, UID and GID of connected client
This can be very useful if client programs run as non-root.
Add separate module for backported language functionality
utils.py, where they were before, is already huge.
utils: Add function for partial application of function arguments
The function's code was mostly copied from Python's documentationand it's equivalent to “functools.partial” in Python 2.5 and above.
utils: Add function to read locked PID file
This is useful in combination with utils.StartDaemon and will be used forreading the import/export daemon's PID file.
utils: Add class to split string stream into lines
This will be used by the new import/export daemon to splitoutput by its child processes into lines.
Fix utils.WaitForFdCondition inner retry loop
Commit dfdc4060 added WaitForFdCondition which uses utils.Retry withouthandling timeout exceptions. This breaks any nested retry loops.
This patch fixes the above function, and also changes utils.Retry todetect and warn future similar cases. In addition, we add a few small...
Fix bug introduced in 76e5f8b54: mkdir mode
After commit 76e5f8b54, mkdir_mode in utils.RenameFile isno longer passed to Makedirs. This is fixed by this patch.
utils: Move wrapper code around os.makedirs into separate function
Revert "Only override any and all if not defined"
This reverts commit bd5617020a50bcd08269330638d64078c1b30b71.
Turns out our and python's any/all are not compatible.
move http.WaitForSocketCondition to utils
WaitForSocketCondition: rename, handle EINTR
- Rename WaitForSocketCondition to SingleWaitForFdCondition - Avoid potentially infinite loop, if we continue to get interrupted - Handle eintr correctly - Avoid the poller try/finally, as the poller object gets destroyed...
Only override any and all if not defined
If any or all are already defined (because we're using a new version ofpython) just link them inside "utils" rather than redefining them.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
utils: Add functions to sign and verify X509 certs using HMAC
Certificates exchanged via an untrusted third party should besigned to ensure they haven't been modified.