Synnefo » snf-ganeti » ganeti-local

utils: Move wrapper code around os.makedirs into separate function

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

utils.RunCmd: Test case with reset_env set and setting variables

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

utils: Add function to extract X509 cert validity

X509 uses ASN1 GENERALIZEDTIME or UTCTIME to store certificate validity.
pyOpenSSL 0.7 and above allow us to retrieve both “notBefore” and
“notAfter” as strings. Parsing them turned out to be a challenge since...

Support arguments in utils.RunInSeparateProcess

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

ListVisibleFiles: require normalized path names

This patch changes ListVisibleFiles to raise ProgrammerError if it's
passed a non-absolute/non-normalized path name, and adds unittests for
this behaviour.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Implement utils.RunParts and use it for hooks

This function is a generic pythonic version of runparts. We currently
use it in the backend HooksRunner, but we'll use it for running
different directories as well.

Signed-off-by: Guido Trotter <ultrotter@google.com>...

Move RunInSeparateProcess to ganeti.utils

This function could be useful in other places and this
way we can easily unittest it.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add an UnescapeAndSplit function

In many cases, where we accept (usually from the command line) a list of
parameters, we remove the use of the separator as an component of any of
the elements.

This patch adds a new function that can split strings of the form...

Ignore log messages in unittests

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix utils.MatchNameComponent for full matches

While ‘test1’ matches both ‘test1’ and ‘test1.example’, it has a full,
exact match and we should return it if that is the case.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add utils.FormatTime and a simple unittest

We don't format with subsecond-precision.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Remove utils.CheckDict since it's no longer used

Only the custom handling in rpc.py for call_node_info used this, so now
it can be removed.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add utils.IsNormAbsPath function

Currently most of the time we check for absolute path, but that doesn't
protect us from some invalid paths. In some places we should be more
strict, and this function should help us to.

Signed-off-by: Guido Trotter <ultrotter@google.com>...

Apply the right permissions to /etc/hosts

In the current Ganeti version when modifying /etc/hosts we mistakenly
give it the permissions of the temporary file we create to define its
content, which is by default 0600. This breaks most non-root
applications, and thus must be corrected. This patch forces the mode to...

Add a TailFile function

This patch adds a tail file function, to be used for parsing and returning in
the job log OS installation failures.

Reviewed-by: ultrotter

Add rename function automatically creating directories if needed

Unfortunately, os.makedirs in Python 2.4 is not safe against multiple
processes creating the same directory tree at the same time. This is
only fixed in Python 2.5 and up. Adding more checks in our code doesn't...

Move FieldSet class to utils.py

Since we can use the FieldSet class in cli.py to nicely format disk
sizes and such, we move it to utils.py and also move its associated
unittest. I didn't remove the cmdlib.py unittest file as that's not the
good direction :)...

Add cleanup of processes to utils.KillProcess

In case the process we want to signal is our own process, and the signal
we send is a deadly one, we should also cleanup after the process.

This patch adds a new parameter waitpid to this function that does this,...

Modify utils.RunCmd to write output to file

Currently we launch processes via the shell in a few places only to
redirect standard output and error to a log file ("&> $file"). It is
better to do such redirection from within RunCmd itself.

This patch splits RunCmd in two parts, the setup and the execution part,...

Abstract checking own address into a function

Currently, we check if we have a given ip address (i.e. it's alive on
one of our interfaces) but manually calling TcpPing(source=localhost).
This works, but having it spread all over the code makes it hard to...

utils.SplitTime: Fix rounding of milliseconds

Reported by Iustin.

It used to return this:

utils.SplitTime(1234.999999999999)

(1234, 1000)

while it should've returned this:

utils.SplitTime(1234.999999999999)

(1235, 0)

Reviewed-by: ultrotter

Add FileLock utility class

This class is a wrapper around fcntl.flock and abstracts opening and
closing the lockfile. It'll used for the job queue.

(The patch also removes a duplicate import of tempfile into the unittest)

Reviewed-by: iustinp

Add a KillProcess function

We cannot depend on all environments to have a start-stop-daemon or
similar tool. We instead implement a KillProcess function that behaves
similar to “start-stop-daemon --retry”.

Note that the attached unittest can hang in foreground if the child...

Add utils unittests for new functions

The submitted WritePidFile, RemovePidfile and IsPidFileAlive functions
miss unit tests. Adding a simple one which covers their basic
functionality.

Reviewed-by: iustinp

Remove the old locking functions

This removes (hopefully) all traces of the old locking functions and
uses.

Reviewed-by: imsnah

Fix the zombie process unittest

The failure is because in high load, the parent gets to run before the
child has the chance to os._exit(), and therefore it is still running
when the parent does the check.

The fix removes the chance of this happening by waiting to receive a SIGCHLD...

Modify utils.TcpPing to make source address optional

This patch modifies TcpPing and its callers to make the source address
selection optional. Usually, the kernel will know better what
source address to use, just in some cases we want to enforce a given...

Fix the unittest for locking.

Reviewed-by: imsnah

Add function to return list with unique elements.

Reviewed-by: ultrotter

Use setUp/tearDown for tests using temporary files.

Reviewed-by: schreiberal

Compare file contents directly in unittests.

Reviewed-by: schreiberal, ultrotter

Fix functions for /etc/hosts.

- Combine hostname and aliases on one line
- Fix bug with wrongfully removed newline characters
- Use wrapper for SetEtcHostsEntry in cmdlib

Reviewed-by: iustin

Add functions for known_hosts files.

Reviewed-by: TODO

Rename utils.GetUUID to utils.NewUUID.

Reviewed-by: schreiberal

Remove fping as a dependency for Ganeti.

This patch completely gets rid of fping
- replace all fping invocations with TcpPing calls
- update documentation accordingly.
- associated cleanups (use constant for localhost IP, use more sensible
defaults for TcpPing and use those)...

Cleanup up selective imports for utils and utils unit test.

Reviewed-by: iustinp, imsnah

Fix typo in last commit.