History | View | Annotate | Download (39.6 kB)
Modify utils.RunCmd to write output to file
Currently we launch processes via the shell in a few places only toredirect standard output and error to a log file ("&> $file"). It isbetter to do such redirection from within RunCmd itself.
This patch splits RunCmd in two parts, the setup and the execution part,...
Remove the logger.py module
Since now we use only one function from the logger module(SetupLogging), we move it to utils.py (which is already imported by allusers of this function), and we remove the module.
Reviewed-by: imsnah
Abstract checking own address into a function
Currently, we check if we have a given ip address (i.e. it's alive onone of our interfaces) but manually calling TcpPing(source=localhost).This works, but having it spread all over the code makes it hard to...
Move functions from ssconf.py elsewhere
These functions will be used to access config values instead of usingssconf.
Reviewed-by: iustinp
Move the pseudo-secret generation to utils.py
The bootstrap code needs a pseudo-secret and this is currently generatedinside the InitGanetiServerSetup function. Since more users will needthis, move it to utils.py
Reviewed-by: ultrotter
Add utils.ReadFile function
It abstracts exception handling and is like a complement toutils.WriteFile.
utils.FileLock: Implement timeout
The timeout can be used in ganeti-noded to be more robust againstdeadlocks.
utils.SplitTime: More rounding fixes
SplitTime didn't round the same on different platforms. This patch changesit to use microseconds and not care about rounding.
utils.SplitTime: Fix rounding of milliseconds
Reported by Iustin.
It used to return this:
utils.SplitTime(1234.999999999999)
(1234, 1000)
while it should've returned this:
(1235, 0)
Add simple lock debug output
Currently it can only be enabled by modifying utils.py, but we canadd a command line parameter later if needed.
Reviewed-by: schreiberal
RunCmd: add optional environment overriding
If the user passes an env dict to RunCmd we'll override the environmentpassed to the to-be-executed command with the values in the dict. Thisallows us to pass arbitrary environment values to commands we run....
Convert RunCmd to an epydoc docstring
Add functions to split time into tuple and merge it back
These will be used for job logs.
Log only unexpected errors in utils.FileLock
Otherwise users might be confused by errors in log files.
Add FileLock utility class
This class is a wrapper around fcntl.flock and abstracts opening andclosing the lockfile. It'll used for the job queue.
(The patch also removes a duplicate import of tempfile into the unittest)
Fix a misuse of exc_info in logging.info
This is my fault, sorry.
Fix pylint-detected issues
This is mostly: - whitespace fix (space at EOL in some files, not all, broken indentation, etc) - variable names overriding others (one is a real bug in there) - too-long-lines - cleanup of most unused imports (not all)...
Expose utils.DaemonPidFileName
Since we need to compute this from outside utils.py, we change this to apublic function.
Add a KillProcess function
We cannot depend on all environments to have a start-stop-daemon orsimilar tool. We instead implement a KillProcess function that behavessimilar to “start-stop-daemon --retry”.
Note that the attached unittest can hang in foreground if the child...
Change IsPidFileAlive into ReadPidFile
We already have a function to test if a PID is alive, so it makes moresense to use function composition that force calling (since we need toread PIDs from files in other places too). Now IsProcessAlive returnsFalse for PIDs <= 0, since this is the error return from ReadPidFile....
Fix exception class name in utils.WritePidFile
Add utils.{Write,Remove}PidFile
WritePidFile is a helper function that writes the current pid in apidfile within the ganeti run directory. RemovePidFile tries to deleteit.
Add utils.IsPidFileAlive function
This helper function reads a pid from a file containing it and checkswhether it refers to a live process.
Add signal handler class
This signal handler class abstracts some of the code previouslyused in other places. It also uninstalls its handler when Reset()is called or the class is destructed, thereby restoring theprevious behaviour.
Move watcher's LockFile function to utils
Add a simple decorator for instance methods
This is just a simple, hardcoded decorator for object methods needingsynchronization on the _lock instance attribute.
Convert utils.py to the logging module
The patch also logs all commands executed from RunCmd when we are atdebug level.
Remove the old locking functions
This removes (hopefully) all traces of the old locking functions anduses.
Add a FirstFree function to utils.py
This function will return the first unused integer based on a list ofused integers (e.g. [0, 1, 3] will return 2).
utils.WriteFile: Remove optional check_abspath parameter
cfgupgrade will not work with relative paths at all, but rather get themfrom constants.py.
Add more parameters to utils.WriteFile
- Make closing file optional: Required by ganeti-watcher to keep file open after writing it. Changes return value of utils.WriteFile if "close" parameter evaluates to True.- Pre- and post-write functions: Can be used to lock files. This...
Move cmdlib._HasValidVG to utils.CheckVolumeGroupSize
This is required for splitting the cluster initialization code.
Move {Set,Remove}EtcHostsEntry wrappers to utils.py
This is required for the split of the cluster initialization code.
Reviewed-by: iustinp, ultrotter
Remove utils.CheckDaemonAlive and use “xm info” instead
There are a couple of reasons for doing so:- /proc is not OS independent, it's only supported by Linux (there are emulations on other systems, but those might differ from the way Linux represents data)....
Disable forking in the master daemon
This patch adds a mechanism to disable utils.RunCmd in selectedprograms. This is needed in the master daemon unless we confirmthreading doesn't pose any problems.
This makes cluster init fail, but creating new trunk clusters is anyway...
Extend utils.WriteFile
Add parameters to not check the path for absoluteness, implement a dryrun mode and automatically create a backup if requested. This will beused by the cfgupgrade utility.
Move the OS search code into an abstract function
Based on the previous OS search code changes, we can now move the OSsearch code into a generic look-for-file function in utils.py. Thismeans that the allocator code can use the same function.
Allow utils.Daemonize() to not close some fds
This patch implements an optional parameter to utils.Daemonize() whichallows that function to not close some file descriptors.
This will allow the master daemon to open the listening socket beforefork - in order to be able to notify errors and return a meaningful exit...
Move the daemonize function to utils.py
Currently, in ganeti-noded we have the createDaemon function. Sincewe'll need the same in other daemons, we move this function to utils.py
With the move, a few changes were also done: - change the name to Daemonize()...
Modify utils.TcpPing to make source address optional
This patch modifies TcpPing and its callers to make the source addressselection optional. Usually, the kernel will know better whatsource address to use, just in some cases we want to enforce a given...
Codestyle fixes: adding a few empty lines
Add a test opcode that sleeps for a given duration
This can be used for testing purposes.
Reviewed-by: ultrotter,imsnah
Allow instance MAC address to be set.
Allow the MAC address of an instance to be specified optionally duringinstance creation and later to be changed via instance modify.
Make utils.RunCmd() deal with interleaved stdout/stderr
Currently, RunCmd is written with the assumption that programs will havea small stderr output, therefore we read the child's stdout (which canbe big, so we don't want to block the child) and then the stderr (which...
Replace hardcoded lock dir
This patch replaces the hardcoded ‘/var/lock/’ directory with one based onLOCALSTATEDIR.
Make utils.RunCmd log failures when using debug
This patch adds logging of command failures to the debug log in case theuser either started the command (gnt-*) or the node daemon with thedebug flag.
Sort list returned by ListVisibleFiles.
If we didn't sort this list, files would be in arbitrary order.
Make sure aliases written to /etc/hosts are unique.
Add function to return list with unique elements.
Fix another breakage in SetEtcHostsEntry
The code assumes all lines have at least two elements which arewhitespace separated - i.e. it does not deal with empty lines or commentlines with no spaces. This patch fixes this.
Also, the patch replaces the blank between IP and the canonical name...
Call fsync() after modifying /etc/hosts.
This is a critical file. Breaking it would be very bad. Thus, if the systemcrashes before the data is synched to the disk, it should have the completefile afterwards.
Fix functions for /etc/hosts.
- Combine hostname and aliases on one line- Fix bug with wrongfully removed newline characters- Use wrapper for SetEtcHostsEntry in cmdlib
Reviewed-by: iustin
Rollback commit 356.
Add functions for known_hosts files.
Reviewed-by: TODO
Add missing descriptions to {Add,Remove}EtcHostsEntry.
Use new functions to modify /etc/hosts.
Add functions to modify /etc/hosts.
Cleanup temporary file in RemoveAuthorizedKey.
Miscellaneous style fixes
This patch fixes some minor pylint warnings (unused variables, wrongindentation, etc.) and a real bug in the recovery for drbd8 renameprocedure.
Rename utils.GetUUID to utils.NewUUID.
Import two itertools recipes
The two function 'any' and 'all' are copied as-is from the python 2.4documentation for the itertools module. They are useful (and are alreadybuiltin function in python 2.5).
Add a generic write file function
Currently there are a few version of "write a file safely" in the code.This patch adds a generic function that should be able to replace allthe other versions.
The "take attributes from the target file" option, while useful, is not...
Enhance GetHomeDir to accept either names or UIDs
Currently GetHomeDir accepts UIDs only. Enhance it to accept either auser name or a user id, to allow for nicer usage.
Use the kernel's ability to generate UUIDs.
This removes the dependency on either the uuid module or e2fsprogs' uuidgen.
Add small function to read the homedir of a user
This can be used to replace hardcoded "/root/" paths.
Remove fping as a dependency for Ganeti.
This patch completely gets rid of fping - replace all fping invocations with TcpPing calls - update documentation accordingly. - associated cleanups (use constant for localhost IP, use more sensible defaults for TcpPing and use those)...
Remove the shebang from modules
Since modules are not directly executables, remove the shebang fromthem. This helps with lintian warnings.
Also make the autogenerated _autoconf.py contain two comment lines atthe beginning, like the other modules.
Add function to list files in a directory, excluding those beginning with adot.
Prevent race condition in CreateBackup().
Remove requirement that host names are FQDN
We currently require that hostnames are FQDN not short names(node1.example.com instead of node1). We can allow short names as longas: - we always resolve the names as returned by socket.gethostname() - we rely on having a working resolver...
Change resolved hostname from dict to a class
The current result of utils.LookupHostname() is a dict, but this doesnot allow static checkers to check the correctness of the code. Thispatch introduces a new class names HostInfo and changes LookupHostname...
Cleanup up selective imports for utils and utils unit test.
Reviewed-by: iustinp, imsnah
Added TcpPing to do ping-alike via TCP connect(2) with defined source address.To be used to replace the currently fping(8) based reachability test.
Change the way how locales are overridden:- Set only LC_ALL, it overrides any other variable- Restore environment after unittest
Filters out LANG and LC_* on RunCmd
This filters out the LANG and LC_ variables from the environment of programsexecuted by RunCmd, in order to get consistent output of execute programs likelvs, etc.
Add function to get all local IP addresses, will be used in cmdlib.py.
Switch utils.RunCmd from popen2 to subprocess
This changes the implementation of RunCmd from using the popen2 moduleto using the (new in 2.4) subprocess module.
This is helpful because the subprocess module has more advanced featuresthan popen2, the most important ones being the ability to run code in the...
Style changes for pep-8 and python-3000 compliance.
This changes the raising of exceptions from: raise Exception, valueto raise Exception(value)
as the first form will be removed in python-3000 and the second form ispreferred now.
The changes also involve a few cases of changing from raising standard...
Comment formatting updates.
Initial commit.