Synnefo » snf-ganeti » ganeti-local

Move FieldSet class to utils.py

Since we can use the FieldSet class in cli.py to nicely format disk
sizes and such, we move it to utils.py and also move its associated
unittest. I didn't remove the cmdlib.py unittest file as that's not the
good direction :)...

utils.SetupLogging: Remove previously setup handlers

If a logging function is called before the logging module is setup
using utils.SetupLogging, it calls logging.basicConfig, which adds
a StreamHandler, on its own. If we leave it in, all log output will...

Remove zombie special case from IsProcessAlive

Based on the discussion on the mailing list, we remove the special
casing of zombie processes.

Note we don't use kill -0 since that has a different meaning than "check
process is alive", so this function is still linux-specific....

Implement working directory for utils.RunCmd

In order to replace usage of the shell just for 'cd %s; ...' we need to
be able to specify the working directory for childs.

The patch also changes the default working directory (when not
specified) to "/", as opposed to the current dir; since the daemos run...

Modify utils.RunCmd to write output to file

Currently we launch processes via the shell in a few places only to
redirect standard output and error to a log file ("&> $file"). It is
better to do such redirection from within RunCmd itself.

This patch splits RunCmd in two parts, the setup and the execution part,...

Abstract checking own address into a function

Currently, we check if we have a given ip address (i.e. it's alive on
one of our interfaces) but manually calling TcpPing(source=localhost).
This works, but having it spread all over the code makes it hard to...

Move the pseudo-secret generation to utils.py

The bootstrap code needs a pseudo-secret and this is currently generated
inside the InitGanetiServerSetup function. Since more users will need
this, move it to utils.py

Reviewed-by: ultrotter

utils.FileLock: Implement timeout

The timeout can be used in ganeti-noded to be more robust against
deadlocks.

Reviewed-by: iustinp

utils.SplitTime: Fix rounding of milliseconds

Reported by Iustin.

It used to return this:

utils.SplitTime(1234.999999999999)

(1234, 1000)

while it should've returned this:

utils.SplitTime(1234.999999999999)

(1235, 0)

Reviewed-by: ultrotter

RunCmd: add optional environment overriding

If the user passes an env dict to RunCmd we'll override the environment
passed to the to-be-executed command with the values in the dict. This
allows us to pass arbitrary environment values to commands we run.
...

Add functions to split time into tuple and merge it back

These will be used for job logs.

Reviewed-by: ultrotter

Add FileLock utility class

This class is a wrapper around fcntl.flock and abstracts opening and
closing the lockfile. It'll used for the job queue.

(The patch also removes a duplicate import of tempfile into the unittest)

Reviewed-by: iustinp

Fix pylint-detected issues

This is mostly:
- whitespace fix (space at EOL in some files, not all, broken
indentation, etc)
- variable names overriding others (one is a real bug in there)
- too-long-lines
- cleanup of most unused imports (not all)...

Add a KillProcess function

We cannot depend on all environments to have a start-stop-daemon or
similar tool. We instead implement a KillProcess function that behaves
similar to “start-stop-daemon --retry”.

Note that the attached unittest can hang in foreground if the child...

Fix exception class name in utils.WritePidFile

Reviewed-by: iustinp

Add utils.IsPidFileAlive function

This helper function reads a pid from a file containing it and checks
whether it refers to a live process.

Reviewed-by: iustinp

Move watcher's LockFile function to utils

Reviewed-by: iustinp

Convert utils.py to the logging module

The patch also logs all commands executed from RunCmd when we are at
debug level.

Reviewed-by: imsnah

Add a FirstFree function to utils.py

This function will return the first unused integer based on a list of
used integers (e.g. [0, 1, 3] will return 2).

Reviewed-by: imsnah

Add more parameters to utils.WriteFile

- Make closing file optional: Required by ganeti-watcher to keep
file open after writing it. Changes return value of utils.WriteFile
if "close" parameter evaluates to True.
- Pre- and post-write functions: Can be used to lock files. This...

Move {Set,Remove}EtcHostsEntry wrappers to utils.py

This is required for the split of the cluster initialization code.

Reviewed-by: iustinp, ultrotter

Disable forking in the master daemon

This patch adds a mechanism to disable utils.RunCmd in selected
programs. This is needed in the master daemon unless we confirm
threading doesn't pose any problems.

This makes cluster init fail, but creating new trunk clusters is anyway...

Move the OS search code into an abstract function

Based on the previous OS search code changes, we can now move the OS
search code into a generic look-for-file function in utils.py. This
means that the allocator code can use the same function.

Reviewed-by: ultrotter

Move the daemonize function to utils.py

Currently, in ganeti-noded we have the createDaemon function. Since
we'll need the same in other daemons, we move this function to utils.py

With the move, a few changes were also done:
- change the name to Daemonize()...

Codestyle fixes: adding a few empty lines

Reviewed-by: ultrotter

Allow instance MAC address to be set.

Allow the MAC address of an instance to be specified optionally during
instance creation and later to be changed via instance modify.

Reviewed-by: iustinp

Replace hardcoded lock dir

This patch replaces the hardcoded ‘/var/lock/’ directory with one based on
LOCALSTATEDIR.

Reviewed-by: imsnah

Sort list returned by ListVisibleFiles.

If we didn't sort this list, files would be in arbitrary order.

Reviewed-by: iustinp

Add function to return list with unique elements.

Reviewed-by: ultrotter

Call fsync() after modifying /etc/hosts.

This is a critical file. Breaking it would be very bad. Thus, if the system
crashes before the data is synched to the disk, it should have the complete
file afterwards.

Reviewed-by: iustinp

Rollback commit 356.

Add missing descriptions to {Add,Remove}EtcHostsEntry.

Reviewed-by: schreiberal

Add functions to modify /etc/hosts.

Reviewed-by: schreiberal

Miscellaneous style fixes

This patch fixes some minor pylint warnings (unused variables, wrong
indentation, etc.) and a real bug in the recovery for drbd8 rename
procedure.

Reviewed-by: imsnah

Import two itertools recipes

The two function 'any' and 'all' are copied as-is from the python 2.4
documentation for the itertools module. They are useful (and are already
builtin function in python 2.5).

Reviewed-by: iustinp

Enhance GetHomeDir to accept either names or UIDs

Currently GetHomeDir accepts UIDs only. Enhance it to accept either a
user name or a user id, to allow for nicer usage.

Reviewed-by: imsnah

Add small function to read the homedir of a user

This can be used to replace hardcoded "/root/" paths.

Reviewed-by: imsnah

Remove the shebang from modules

Since modules are not directly executables, remove the shebang from
them. This helps with lintian warnings.

Also make the autogenerated _autoconf.py contain two comment lines at
the beginning, like the other modules.

Reviewed-by: ultrotter

Prevent race condition in CreateBackup().

Reviewed-by: ultrotter

Change resolved hostname from dict to a class

The current result of utils.LookupHostname() is a dict, but this does
not allow static checkers to check the correctness of the code. This
patch introduces a new class names HostInfo and changes LookupHostname...

Added TcpPing to do ping-alike via TCP connect(2) with defined source address.
To be used to replace the currently fping(8) based reachability test.

Reviewed-by: imsnah

Filters out LANG and LC_* on RunCmd

This filters out the LANG and LC_ variables from the environment of programs
executed by RunCmd, in order to get consistent output of execute programs like
lvs, etc.

Reviewed-by: imsnah

Switch utils.RunCmd from popen2 to subprocess

This changes the implementation of RunCmd from using the popen2 module
to using the (new in 2.4) subprocess module.

This is helpful because the subprocess module has more advanced features
than popen2, the most important ones being the ability to run code in the...

Comment formatting updates.

Reviewed-by: iustinp