root / docs / developers / logging.rst @ fe980c71
History | View | Annotate | Download (4.8 kB)
1 | a582b70e | Stavros Sachtouris | Logging |
---|---|---|---|
2 | a582b70e | Stavros Sachtouris | ======= |
3 | a582b70e | Stavros Sachtouris | |
4 | 16d7b9ff | Stavros Sachtouris | Kamaki uses the standard Python logger package to log some of its functionality. |
5 | a582b70e | Stavros Sachtouris | |
6 | 16d7b9ff | Stavros Sachtouris | All kamaki loggers are named or prefixed after the package they log, e.g., |
7 | 16d7b9ff | Stavros Sachtouris | a logger at `kamaki/cli/argument/__init__.py` should be called |
8 | a582b70e | Stavros Sachtouris | `kamaki.cli.argument` whereas a logger at `kamaki/clients/conf.py` should be |
9 | 16d7b9ff | Stavros Sachtouris | named `kamaki.clients.conf`. In `kamaki/clients/__init__.py` there are two |
10 | a582b70e | Stavros Sachtouris | loggers that use the package name as prefix, and they detailed bellow. |
11 | a582b70e | Stavros Sachtouris | |
12 | a582b70e | Stavros Sachtouris | Monitor requests and responses |
13 | a582b70e | Stavros Sachtouris | ------------------------------ |
14 | a582b70e | Stavros Sachtouris | |
15 | a582b70e | Stavros Sachtouris | The `kamaki.clients` logger contains two subloggers that monitor the HTTP |
16 | 16d7b9ff | Stavros Sachtouris | API calls:: |
17 | a582b70e | Stavros Sachtouris | |
18 | a582b70e | Stavros Sachtouris | kamaki.clients.send for kamaki requests to the server |
19 | a582b70e | Stavros Sachtouris | kamaki.clients.recv for server responses to kamaki |
20 | a582b70e | Stavros Sachtouris | |
21 | a582b70e | Stavros Sachtouris | These are the only loggers used for purposes other than mere debugging. Both |
22 | a582b70e | Stavros Sachtouris | loggers are defined in the CLI code and are used to (a) log HTTP communication |
23 | 16d7b9ff | Stavros Sachtouris | to the log file as well as to (b) show users the HTTP requests and responses in |
24 | 16d7b9ff | Stavros Sachtouris | "verbose" or "debug" modes. |
25 | a582b70e | Stavros Sachtouris | |
26 | a582b70e | Stavros Sachtouris | Logger in external code |
27 | a582b70e | Stavros Sachtouris | ----------------------- |
28 | a582b70e | Stavros Sachtouris | |
29 | a582b70e | Stavros Sachtouris | When a logger is known to be in kamaki code, the script developer may use this |
30 | a582b70e | Stavros Sachtouris | logger to log some needed information. This can be happen either by directly |
31 | abfaa6d9 | Vangelis Koukis | using the Python `logger` package, or the corresponding kamaki wraper |
32 | a582b70e | Stavros Sachtouris | `kamaki.cli.logger` which allows the definition, activation and deactivation |
33 | a582b70e | Stavros Sachtouris | of stream (usually console) or file loggers. |
34 | a582b70e | Stavros Sachtouris | |
35 | a582b70e | Stavros Sachtouris | As an example, we will use |
36 | a582b70e | Stavros Sachtouris | `this script <clients-api.html#register-a-banch-of-pre-uploaded-images>`_ |
37 | a582b70e | Stavros Sachtouris | that loads images from a set of image files already uploaded to Pithos+ |
38 | a582b70e | Stavros Sachtouris | `images` container. |
39 | a582b70e | Stavros Sachtouris | |
40 | a582b70e | Stavros Sachtouris | First, we shall add a logger to keep HTTP communication in `/tmp/my_kamaki.log` |
41 | a582b70e | Stavros Sachtouris | To do this, append the following at the top of the file: |
42 | a582b70e | Stavros Sachtouris | |
43 | a582b70e | Stavros Sachtouris | .. code-block:: python |
44 | a582b70e | Stavros Sachtouris | |
45 | a582b70e | Stavros Sachtouris | from kamaki.cli.logger import add_file_logger |
46 | a582b70e | Stavros Sachtouris | add_file_logger('kamaki', filename='/tmp/my_kamaki.log') |
47 | a582b70e | Stavros Sachtouris | |
48 | a582b70e | Stavros Sachtouris | After a run of the script, a new file will be created at `/tmp/my_kamaki.log` |
49 | a582b70e | Stavros Sachtouris | that will contain logs of the form:: |
50 | a582b70e | Stavros Sachtouris | |
51 | a582b70e | Stavros Sachtouris | > POST https://accounts.okeanos.grnet.gr/identity/v2.0/tokens |
52 | a582b70e | Stavros Sachtouris | > Content-Length: 74 |
53 | a582b70e | Stavros Sachtouris | > Content-Type: application/json |
54 | a582b70e | Stavros Sachtouris | > data size:74 |
55 | a582b70e | Stavros Sachtouris | |
56 | a582b70e | Stavros Sachtouris | < 200 OK |
57 | a582b70e | Stavros Sachtouris | < content-length: 2425 |
58 | a582b70e | Stavros Sachtouris | < content-language: en-us |
59 | a582b70e | Stavros Sachtouris | < expires: Wed, 31 Jul 2013 14:27:47 GMT |
60 | a582b70e | Stavros Sachtouris | < vary: X-Auth-Token,Accept-Language |
61 | a582b70e | Stavros Sachtouris | < server: gunicorn/0.14.5 |
62 | a582b70e | Stavros Sachtouris | < last-modified: Wed, 31 Jul 2013 14:27:47 GMT |
63 | a582b70e | Stavros Sachtouris | < connection: close |
64 | a582b70e | Stavros Sachtouris | < etag: "43af...36" |
65 | a582b70e | Stavros Sachtouris | < cache-control: no-cache, no-store, must-revalidate, max-age=0 |
66 | a582b70e | Stavros Sachtouris | < date: Wed, 31 Jul 2013 14:27:47 GMT |
67 | a582b70e | Stavros Sachtouris | < content-type: application/json; charset=UTF-8 |
68 | a582b70e | Stavros Sachtouris | < data size: 2425 |
69 | a582b70e | Stavros Sachtouris | |
70 | 16d7b9ff | Stavros Sachtouris | .. note:: user token and http body content are not logged by default. This can |
71 | 16d7b9ff | Stavros Sachtouris | be switched on and off by modifing the *kamaki.client.Client.LOG_TOKEN* and |
72 | 16d7b9ff | Stavros Sachtouris | *kamaki.client.Client.LOG_DATA* flags |
73 | a582b70e | Stavros Sachtouris | |
74 | a582b70e | Stavros Sachtouris | As a second example, let's suppose that we need to see only the http requests |
75 | 16d7b9ff | Stavros Sachtouris | of the `pithos.list_objects()` method and print these to stdout. To achieve |
76 | 16d7b9ff | Stavros Sachtouris | that goal, we should get a stream logger and deactivate it when we do not need |
77 | 16d7b9ff | Stavros Sachtouris | it anymore. |
78 | a582b70e | Stavros Sachtouris | |
79 | a582b70e | Stavros Sachtouris | |
80 | a582b70e | Stavros Sachtouris | .. code-block:: python |
81 | a582b70e | Stavros Sachtouris | |
82 | a582b70e | Stavros Sachtouris | #! /usr/bin/python |
83 | a582b70e | Stavros Sachtouris | |
84 | a582b70e | Stavros Sachtouris | [...] |
85 | a582b70e | Stavros Sachtouris | |
86 | a582b70e | Stavros Sachtouris | from kamaki.cli.logger import add_stream_logger, deactivate |
87 | a582b70e | Stavros Sachtouris | add_stream_logger('kamaki.clients') |
88 | a582b70e | Stavros Sachtouris | |
89 | a582b70e | Stavros Sachtouris | for img in pithos.list_objects(): |
90 | a582b70e | Stavros Sachtouris | deactivate('kamaki.clients') |
91 | a582b70e | Stavros Sachtouris | [...] |
92 | a582b70e | Stavros Sachtouris | |
93 | a582b70e | Stavros Sachtouris | Logger in kamaki code |
94 | a582b70e | Stavros Sachtouris | --------------------- |
95 | a582b70e | Stavros Sachtouris | |
96 | a582b70e | Stavros Sachtouris | When implementing kamaki code, either as part of the main kamaki project or as |
97 | a582b70e | Stavros Sachtouris | an extension, it is often useful to use loggers. The suggested strategy is to |
98 | a582b70e | Stavros Sachtouris | use `kamaki.cli.logger` to create one logger per package, named after the |
99 | abfaa6d9 | Vangelis Koukis | package itself. Developers may also directly use the Python logger module, but |
100 | a582b70e | Stavros Sachtouris | they should respect the naming conventions. |
101 | a582b70e | Stavros Sachtouris | |
102 | a582b70e | Stavros Sachtouris | In this example, we want to log the arguments of the `register` method found in |
103 | abfaa6d9 | Vangelis Koukis | `kamaki/clients/image/__init__.py`. We will use the Python logger module. |
104 | a582b70e | Stavros Sachtouris | |
105 | a582b70e | Stavros Sachtouris | First, we should add a logger initializer at the top of the file. |
106 | a582b70e | Stavros Sachtouris | |
107 | a582b70e | Stavros Sachtouris | .. code-block:: python |
108 | a582b70e | Stavros Sachtouris | |
109 | a582b70e | Stavros Sachtouris | from logging import getLogger |
110 | a582b70e | Stavros Sachtouris | |
111 | a582b70e | Stavros Sachtouris | log = getLogger(__name__) |
112 | a582b70e | Stavros Sachtouris | |
113 | a582b70e | Stavros Sachtouris | Now, we should use the `log` biding to actually log what we need. |
114 | a582b70e | Stavros Sachtouris | |
115 | a582b70e | Stavros Sachtouris | .. code-block:: python |
116 | a582b70e | Stavros Sachtouris | |
117 | a582b70e | Stavros Sachtouris | [...] |
118 | a582b70e | Stavros Sachtouris | |
119 | a582b70e | Stavros Sachtouris | def register(self, name, location, params={}, properties={}): |
120 | a582b70e | Stavros Sachtouris | log.debug('name: %s' % name) |
121 | a582b70e | Stavros Sachtouris | log.debug('location: %s' % location) |
122 | a582b70e | Stavros Sachtouris | log.debug('params: %s' % params) |
123 | a582b70e | Stavros Sachtouris | log.debug('properties: %s' % properties) |
124 | a582b70e | Stavros Sachtouris | [...] |
125 | a582b70e | Stavros Sachtouris | |
126 | a582b70e | Stavros Sachtouris | The logging module will not log anything by itself. It is the caller scripts |
127 | a582b70e | Stavros Sachtouris | responsibility to define the actual logger and set the logging destination. |
128 | a582b70e | Stavros Sachtouris | We are going to use the same script as in the previous examples, but we need |
129 | a582b70e | Stavros Sachtouris | to define logger for `kamaki.clients.image`. |
130 | a582b70e | Stavros Sachtouris | |
131 | a582b70e | Stavros Sachtouris | .. code-block:: python |
132 | a582b70e | Stavros Sachtouris | |
133 | a582b70e | Stavros Sachtouris | #! /usr/bin/python |
134 | a582b70e | Stavros Sachtouris | |
135 | a582b70e | Stavros Sachtouris | from kamaki.cli.logger import add_file_logger |
136 | a582b70e | Stavros Sachtouris | |
137 | a582b70e | Stavros Sachtouris | add_file_logger('kamaki.clients.image', filename='/tmp/kamaki_image.log') |
138 | a582b70e | Stavros Sachtouris | |
139 | a582b70e | Stavros Sachtouris | .. note:: a logger named as `kamaki` will grab everything logged with a name |
140 | a582b70e | Stavros Sachtouris | prefixed as `kamaki`, so if we have two loggers, one named `kamaki` and |
141 | a582b70e | Stavros Sachtouris | another one named `kamaki.clients.image`, they will both grab the |
142 | a582b70e | Stavros Sachtouris | `register` logs. |