root / docs / developers / logging.rst @ 82cc4b8f
History | View | Annotate | Download (4.8 kB)
1 | a582b70e | Stavros Sachtouris | Logging |
---|---|---|---|
2 | a582b70e | Stavros Sachtouris | ======= |
3 | a582b70e | Stavros Sachtouris | |
4 | a582b70e | Stavros Sachtouris | Kamaki uses the standard python logger package to log some of its |
5 | a582b70e | Stavros Sachtouris | functionality. |
6 | a582b70e | Stavros Sachtouris | |
7 | a582b70e | Stavros Sachtouris | All kamaki loggers are named or prefixed after the package they log, e.g. |
8 | a582b70e | Stavros Sachtouris | a logger at `kamaki/cli/argument.__init__.py` should be called |
9 | a582b70e | Stavros Sachtouris | `kamaki.cli.argument` whereas a logger at `kamaki/clients/conf.py` should be |
10 | a582b70e | Stavros Sachtouris | named `kamaki.clients/conf`. In `kamaki/clients/__init__.py` there are two |
11 | a582b70e | Stavros Sachtouris | loggers that use the package name as prefix, and they detailed bellow. |
12 | a582b70e | Stavros Sachtouris | |
13 | a582b70e | Stavros Sachtouris | Monitor requests and responses |
14 | a582b70e | Stavros Sachtouris | ------------------------------ |
15 | a582b70e | Stavros Sachtouris | |
16 | a582b70e | Stavros Sachtouris | The `kamaki.clients` logger contains two subloggers that monitor the HTTP |
17 | a582b70e | Stavros Sachtouris | communication of with the servers:: |
18 | a582b70e | Stavros Sachtouris | |
19 | a582b70e | Stavros Sachtouris | kamaki.clients.send for kamaki requests to the server |
20 | a582b70e | Stavros Sachtouris | kamaki.clients.recv for server responses to kamaki |
21 | a582b70e | Stavros Sachtouris | |
22 | a582b70e | Stavros Sachtouris | These are the only loggers used for purposes other than mere debugging. Both |
23 | a582b70e | Stavros Sachtouris | loggers are defined in the CLI code and are used to (a) log HTTP communication |
24 | a582b70e | Stavros Sachtouris | to the log file as well as to (b) show users the HTTP requests and responses if |
25 | a582b70e | Stavros Sachtouris | kamaki cli is called with options like "verbose" or "debug". |
26 | a582b70e | Stavros Sachtouris | |
27 | a582b70e | Stavros Sachtouris | Logger in external code |
28 | a582b70e | Stavros Sachtouris | ----------------------- |
29 | a582b70e | Stavros Sachtouris | |
30 | a582b70e | Stavros Sachtouris | When a logger is known to be in kamaki code, the script developer may use this |
31 | a582b70e | Stavros Sachtouris | logger to log some needed information. This can be happen either by directly |
32 | a582b70e | Stavros Sachtouris | using the python `logger` package, or the corresponding kamaki wraper |
33 | a582b70e | Stavros Sachtouris | `kamaki.cli.logger` which allows the definition, activation and deactivation |
34 | a582b70e | Stavros Sachtouris | of stream (usually console) or file loggers. |
35 | a582b70e | Stavros Sachtouris | |
36 | a582b70e | Stavros Sachtouris | As an example, we will use |
37 | a582b70e | Stavros Sachtouris | `this script <clients-api.html#register-a-banch-of-pre-uploaded-images>`_ |
38 | a582b70e | Stavros Sachtouris | that loads images from a set of image files already uploaded to Pithos+ |
39 | a582b70e | Stavros Sachtouris | `images` container. |
40 | a582b70e | Stavros Sachtouris | |
41 | a582b70e | Stavros Sachtouris | First, we shall add a logger to keep HTTP communication in `/tmp/my_kamaki.log` |
42 | a582b70e | Stavros Sachtouris | To do this, append the following at the top of the file: |
43 | a582b70e | Stavros Sachtouris | |
44 | a582b70e | Stavros Sachtouris | .. code-block:: python |
45 | a582b70e | Stavros Sachtouris | |
46 | a582b70e | Stavros Sachtouris | from kamaki.cli.logger import add_file_logger |
47 | a582b70e | Stavros Sachtouris | add_file_logger('kamaki', filename='/tmp/my_kamaki.log') |
48 | a582b70e | Stavros Sachtouris | |
49 | a582b70e | Stavros Sachtouris | After a run of the script, a new file will be created at `/tmp/my_kamaki.log` |
50 | a582b70e | Stavros Sachtouris | that will contain logs of the form:: |
51 | a582b70e | Stavros Sachtouris | |
52 | a582b70e | Stavros Sachtouris | > POST https://accounts.okeanos.grnet.gr/identity/v2.0/tokens |
53 | a582b70e | Stavros Sachtouris | > Content-Length: 74 |
54 | a582b70e | Stavros Sachtouris | > Content-Type: application/json |
55 | a582b70e | Stavros Sachtouris | > data size:74 |
56 | a582b70e | Stavros Sachtouris | |
57 | a582b70e | Stavros Sachtouris | < 200 OK |
58 | a582b70e | Stavros Sachtouris | < content-length: 2425 |
59 | a582b70e | Stavros Sachtouris | < content-language: en-us |
60 | a582b70e | Stavros Sachtouris | < expires: Wed, 31 Jul 2013 14:27:47 GMT |
61 | a582b70e | Stavros Sachtouris | < vary: X-Auth-Token,Accept-Language |
62 | a582b70e | Stavros Sachtouris | < server: gunicorn/0.14.5 |
63 | a582b70e | Stavros Sachtouris | < last-modified: Wed, 31 Jul 2013 14:27:47 GMT |
64 | a582b70e | Stavros Sachtouris | < connection: close |
65 | a582b70e | Stavros Sachtouris | < etag: "43af...36" |
66 | a582b70e | Stavros Sachtouris | < cache-control: no-cache, no-store, must-revalidate, max-age=0 |
67 | a582b70e | Stavros Sachtouris | < date: Wed, 31 Jul 2013 14:27:47 GMT |
68 | a582b70e | Stavros Sachtouris | < content-type: application/json; charset=UTF-8 |
69 | a582b70e | Stavros Sachtouris | < data size: 2425 |
70 | a582b70e | Stavros Sachtouris | |
71 | a582b70e | Stavros Sachtouris | .. note:: user token and http body content are not logged by default |
72 | a582b70e | Stavros Sachtouris | |
73 | a582b70e | Stavros Sachtouris | As a second example, let's suppose that we need to see only the http requests |
74 | a582b70e | Stavros Sachtouris | of the `pithos.list_objects()` method. We decide to print these to the console. |
75 | a582b70e | Stavros Sachtouris | To achieve that goal, we should get a stream logger and deactivate it when we |
76 | a582b70e | Stavros Sachtouris | do not need it anymore. |
77 | a582b70e | Stavros Sachtouris | |
78 | a582b70e | Stavros Sachtouris | |
79 | a582b70e | Stavros Sachtouris | .. code-block:: python |
80 | a582b70e | Stavros Sachtouris | |
81 | a582b70e | Stavros Sachtouris | #! /usr/bin/python |
82 | a582b70e | Stavros Sachtouris | |
83 | a582b70e | Stavros Sachtouris | [...] |
84 | a582b70e | Stavros Sachtouris | |
85 | a582b70e | Stavros Sachtouris | from kamaki.cli.logger import add_stream_logger, deactivate |
86 | a582b70e | Stavros Sachtouris | add_stream_logger('kamaki.clients') |
87 | a582b70e | Stavros Sachtouris | |
88 | a582b70e | Stavros Sachtouris | for img in pithos.list_objects(): |
89 | a582b70e | Stavros Sachtouris | deactivate('kamaki.clients') |
90 | a582b70e | Stavros Sachtouris | [...] |
91 | a582b70e | Stavros Sachtouris | |
92 | a582b70e | Stavros Sachtouris | Logger in kamaki code |
93 | a582b70e | Stavros Sachtouris | --------------------- |
94 | a582b70e | Stavros Sachtouris | |
95 | a582b70e | Stavros Sachtouris | When implementing kamaki code, either as part of the main kamaki project or as |
96 | a582b70e | Stavros Sachtouris | an extension, it is often useful to use loggers. The suggested strategy is to |
97 | a582b70e | Stavros Sachtouris | use `kamaki.cli.logger` to create one logger per package, named after the |
98 | a582b70e | Stavros Sachtouris | package itself. Developers may also directly use the python logger module, but |
99 | a582b70e | Stavros Sachtouris | they should respect the naming conventions. |
100 | a582b70e | Stavros Sachtouris | |
101 | a582b70e | Stavros Sachtouris | In this example, we want to log the arguments of the `register` method found in |
102 | a582b70e | Stavros Sachtouris | `kamaki/clients/image/__init__.py`. We will use the python logger module. |
103 | a582b70e | Stavros Sachtouris | |
104 | a582b70e | Stavros Sachtouris | First, we should add a logger initializer at the top of the file. |
105 | a582b70e | Stavros Sachtouris | |
106 | a582b70e | Stavros Sachtouris | .. code-block:: python |
107 | a582b70e | Stavros Sachtouris | |
108 | a582b70e | Stavros Sachtouris | from logging import getLogger |
109 | a582b70e | Stavros Sachtouris | |
110 | a582b70e | Stavros Sachtouris | log = getLogger(__name__) |
111 | a582b70e | Stavros Sachtouris | |
112 | a582b70e | Stavros Sachtouris | Now, we should use the `log` biding to actually log what we need. |
113 | a582b70e | Stavros Sachtouris | |
114 | a582b70e | Stavros Sachtouris | .. code-block:: python |
115 | a582b70e | Stavros Sachtouris | |
116 | a582b70e | Stavros Sachtouris | [...] |
117 | a582b70e | Stavros Sachtouris | |
118 | a582b70e | Stavros Sachtouris | def register(self, name, location, params={}, properties={}): |
119 | a582b70e | Stavros Sachtouris | log.debug('name: %s' % name) |
120 | a582b70e | Stavros Sachtouris | log.debug('location: %s' % location) |
121 | a582b70e | Stavros Sachtouris | log.debug('params: %s' % params) |
122 | a582b70e | Stavros Sachtouris | log.debug('properties: %s' % properties) |
123 | a582b70e | Stavros Sachtouris | [...] |
124 | a582b70e | Stavros Sachtouris | |
125 | a582b70e | Stavros Sachtouris | The logging module will not log anything by itself. It is the caller scripts |
126 | a582b70e | Stavros Sachtouris | responsibility to define the actual logger and set the logging destination. |
127 | a582b70e | Stavros Sachtouris | We are going to use the same script as in the previous examples, but we need |
128 | a582b70e | Stavros Sachtouris | to define logger for `kamaki.clients.image`. |
129 | a582b70e | Stavros Sachtouris | |
130 | a582b70e | Stavros Sachtouris | .. code-block:: python |
131 | a582b70e | Stavros Sachtouris | |
132 | a582b70e | Stavros Sachtouris | #! /usr/bin/python |
133 | a582b70e | Stavros Sachtouris | |
134 | a582b70e | Stavros Sachtouris | from kamaki.cli.logger import add_file_logger |
135 | a582b70e | Stavros Sachtouris | |
136 | a582b70e | Stavros Sachtouris | add_file_logger('kamaki.clients.image', filename='/tmp/kamaki_image.log') |
137 | a582b70e | Stavros Sachtouris | |
138 | a582b70e | Stavros Sachtouris | .. note:: a logger named as `kamaki` will grab everything logged with a name |
139 | a582b70e | Stavros Sachtouris | prefixed as `kamaki`, so if we have two loggers, one named `kamaki` and |
140 | a582b70e | Stavros Sachtouris | another one named `kamaki.clients.image`, they will both grab the |
141 | a582b70e | Stavros Sachtouris | `register` logs. |