Synnefo

===================================================

Logging mechanism for Synnefo's management commands

===================================================

Abstract

========

Log all stdout and stderr output of every invocation of snf-manage, on unique

filename under a given directory.

Current state and shortcomings

==============================

All Synnefo's management commands are written as custom django-admin commands.

This means that every management command is in fact a class that extends

Django's BaseCommand class.

Django's *BaseCommand* provides the attributes ``self.stdout`` and

``self.stderr`` and Django's documentation encourages the users to use these

attributes if they wish to write to the console. Django doesn't provide an

option to write the output to files and the user has to implement this

explicitly when implementing the ``handle`` method.

We would like to extend the above mechanism to allow every ``snf-manage``

command to log all stdout and stderr output on a unique filename under a given

directory. The implementation should change nothing in the way that users write

management commands (only acceptable change is that the new commands may have

to inherit a new class and not the *BaseCommand* one). This means that

existing management commands should play out of the box and also that the

logging mechanism will globally apply to all of them.

Proposed changes

================

In this section we will try to explain the way that the new logging mechanism

will be implemented as well as the reasons behind these decisions.

As we previously saw, we want the logging mechanism to be global and to work

for all the ``snf-manage`` commands without extra efford. This means that the

management commands will continue to use the ``self.stdout`` and

``self.stderr`` attributes from *BaseCommand* class to provide console

output. Therefor we have to provide our own ``self.stdout`` and ``self.stderr``

objects that will preserve the previous functionality and log to files at the

same time. There are two ways to achieve that:

Patch the Django's *BaseCommand* class and replace ``self.stdout`` and

``self.stderr`` attributes.

    This solution requires the minimum amount of changes to the management

    commands' code as they will use our patched version of *BaseCommand*.

    The downside is that we have to patch a library provided class. We are not

    encouraging these type of patches because it obfuscates the code (the

    programmer is expecting to use Django's *BaseCommand* class, not ours)

    and does not preserve compatiblity with other Django versions (if the

    implementation of Django's *BaseCommand* changes our patch will not

    work).

Create a new class that extends Django's *BaseCommand*.

    The downside of this solution is that we have to change the existing code

    so all management commands will inherit our new class and not Django's

    *BaseCommand*. But we find this solution to be cleaner.

For the above reasons we decided to go with the second option.

Django's ``self.stdout`` and ``self.stderr`` are implemented as

*OutputWrapper* objects. We will create our own class which will handle the

file part of the logging and we will handle the console part to the original

*OutputWrapper* object. We can't have used the ``logger`` libary as we want

to preserve the functionality of Django's *OutputWrapper*  (it uses style

functions to pretty print the messages).

Our new class has to be a **descriptor**. This is because *BaseCommand* doesn't

initialize the ``stdout`` and ``stderr`` attributes at ``__init__`` but sets

them only when it needs to (meaning inside the *execute* method).

The above classes will be written in snf-django-lib package meaning that all

the other packages will have a dependency in snf-django-lib.

We will combine timestamp, command name and PID to form unique names, e.g.:

20140120113432-server-modify-4564, where "4564" was the PID. The timestamp will

be first so that files will be chronologically sorted.

Implementation details

======================

The implementation will follow the folowing steps:

- Change current management commands to use ``self.stdout`` and ``self.stderr``

  to provide console output instead of ``sys.stdout``, ``print`` or anything

  else. This change complies with Django's documentation.

- Write a new class that will replace Django's *OutputWrapper*.

- Write a new class (named **SynnefoBaseCommand**) that will extend Django's

  *BaseCommand* and will replace ``stdout`` and ``stderr`` attributes.

- Change all management commands to inherit this new class

  **SynnefoBaseCommand**.

- Update package dependencies.

- Add a new Synnefo setting to allow the user the change the directory where

  the output will be saved.