root / docs / developers / extending-clients-api.rst @ e6ce9ae1
History | View | Annotate | Download (7.9 kB)
1 | 9e4508df | Stavros Sachtouris | Extending kamaki.clients |
---|---|---|---|
2 | 9e4508df | Stavros Sachtouris | ======================== |
3 | 9e4508df | Stavros Sachtouris | |
4 | 16d7b9ff | Stavros Sachtouris | By default, kamaki clients implement REST APIs, therefore they manage HTTP |
5 | 16d7b9ff | Stavros Sachtouris | requests and responses to communicate with services. |
6 | 9e4508df | Stavros Sachtouris | |
7 | 9e4508df | Stavros Sachtouris | How to build a client |
8 | 9e4508df | Stavros Sachtouris | --------------------- |
9 | 9e4508df | Stavros Sachtouris | |
10 | fa382f9e | Stavros Sachtouris | All service clients consist of a subclass of the Client class and implement |
11 | fa382f9e | Stavros Sachtouris | separate client functionalities as member methods. There is also an error class |
12 | fa382f9e | Stavros Sachtouris | to raise exceptions that can be handled by kamaki interfaces. |
13 | 9e4508df | Stavros Sachtouris | |
14 | 9e4508df | Stavros Sachtouris | .. code-block:: python |
15 | 9e4508df | Stavros Sachtouris | |
16 | 9e4508df | Stavros Sachtouris | # ${KAMAKI_PATH}/kamaki/clients/mynewclient.py |
17 | 9e4508df | Stavros Sachtouris | |
18 | 9e4508df | Stavros Sachtouris | from kamaki.clients import Client, ClientError |
19 | 9e4508df | Stavros Sachtouris | |
20 | 16d7b9ff | Stavros Sachtouris | |
21 | 9e4508df | Stavros Sachtouris | class MyNewClient(Client): |
22 | 9e4508df | Stavros Sachtouris | """MyNewClient Description Here""" |
23 | 9e4508df | Stavros Sachtouris | |
24 | 9e4508df | Stavros Sachtouris | def my_first_method(self, **args): |
25 | 9e4508df | Stavros Sachtouris | """Method description""" |
26 | 9e4508df | Stavros Sachtouris | try: |
27 | 9e4508df | Stavros Sachtouris | ... |
28 | 9e4508df | Stavros Sachtouris | method code |
29 | 9e4508df | Stavros Sachtouris | ... |
30 | 9e4508df | Stavros Sachtouris | except SomeKnownException as e1: |
31 | 9e4508df | Stavros Sachtouris | raise ClientError('MyError: %s' % e1) |
32 | 9e4508df | Stavros Sachtouris | except SomeOtherException as e2: |
33 | 9e4508df | Stavros Sachtouris | raise ClientError('MyError: %s' % e2) |
34 | 9e4508df | Stavros Sachtouris | |
35 | 9e4508df | Stavros Sachtouris | def my_second_method(self, **params): |
36 | 9e4508df | Stavros Sachtouris | """Method description""" |
37 | 9e4508df | Stavros Sachtouris | ... |
38 | 9e4508df | Stavros Sachtouris | |
39 | 9e4508df | Stavros Sachtouris | Custom clients can use a set of convenience methods for easy HTTP requests |
40 | 9e4508df | Stavros Sachtouris | |
41 | 9e4508df | Stavros Sachtouris | .. code-block:: python |
42 | 9e4508df | Stavros Sachtouris | |
43 | 9e4508df | Stavros Sachtouris | def get(self, path, **kwargs) |
44 | 9e4508df | Stavros Sachtouris | def head(self, path, **kwargs) |
45 | 9e4508df | Stavros Sachtouris | def post(self, path, **kwargs) |
46 | 9e4508df | Stavros Sachtouris | def put(self, path, **kwargs) |
47 | 9e4508df | Stavros Sachtouris | def delete(self, path, **kwargs) |
48 | 9e4508df | Stavros Sachtouris | def copy(self, path, **kwargs) |
49 | 9e4508df | Stavros Sachtouris | def move(self, path, **kwargs) |
50 | 9e4508df | Stavros Sachtouris | |
51 | 9e4508df | Stavros Sachtouris | How to use your client |
52 | 9e4508df | Stavros Sachtouris | ---------------------- |
53 | 9e4508df | Stavros Sachtouris | |
54 | 9e4508df | Stavros Sachtouris | External applications must instantiate a MyNewClient object. |
55 | 9e4508df | Stavros Sachtouris | |
56 | 9e4508df | Stavros Sachtouris | .. code-block:: python |
57 | 9e4508df | Stavros Sachtouris | |
58 | 9e4508df | Stavros Sachtouris | from kamaki.clients import ClientError |
59 | 9e4508df | Stavros Sachtouris | from kamaki.clients.mynewclient import MyNewClient |
60 | 9e4508df | Stavros Sachtouris | |
61 | 9e4508df | Stavros Sachtouris | ... |
62 | 9e4508df | Stavros Sachtouris | try: |
63 | 9e4508df | Stavros Sachtouris | cl = MyNewClient(args) |
64 | 9e4508df | Stavros Sachtouris | cl.my_first_method(other_args) |
65 | 9e4508df | Stavros Sachtouris | except ClientError as cle: |
66 | 9e4508df | Stavros Sachtouris | print('Client Error: %s' % cle) |
67 | 9e4508df | Stavros Sachtouris | ... |
68 | 9e4508df | Stavros Sachtouris | |
69 | 9e4508df | Stavros Sachtouris | Concurrency control |
70 | 9e4508df | Stavros Sachtouris | ------------------- |
71 | 9e4508df | Stavros Sachtouris | |
72 | cfb87e18 | Stavros Sachtouris | Kamaki clients may handle multiple requests with threads. The easiest way is |
73 | cfb87e18 | Stavros Sachtouris | using the `async_run` method, fed with a list of argument dictionaries (one for |
74 | cfb87e18 | Stavros Sachtouris | each call of the single method). |
75 | 9e4508df | Stavros Sachtouris | |
76 | 9e4508df | Stavros Sachtouris | .. code-block:: python |
77 | 9e4508df | Stavros Sachtouris | |
78 | 9e4508df | Stavros Sachtouris | class MyNewClient(Client): |
79 | 9e4508df | Stavros Sachtouris | ... |
80 | 9e4508df | Stavros Sachtouris | |
81 | 9e4508df | Stavros Sachtouris | def _single_threaded_method(self, **args): |
82 | 9e4508df | Stavros Sachtouris | ... |
83 | 9e4508df | Stavros Sachtouris | |
84 | 9e4508df | Stavros Sachtouris | def multithread_method(self): |
85 | cfb87e18 | Stavros Sachtouris | kwarg_list = [kwarg for each run] |
86 | cfb87e18 | Stavros Sachtouris | self.async_run(self._single_threaded_method, kwarg_list) |
87 | 4054c46d | Stavros Sachtouris | |
88 | 4054c46d | Stavros Sachtouris | Going agile |
89 | 4054c46d | Stavros Sachtouris | ----------- |
90 | 4054c46d | Stavros Sachtouris | |
91 | fa382f9e | Stavros Sachtouris | The kamaki.clients package contains a set of fine-grained unit-tests for all |
92 | 16d7b9ff | Stavros Sachtouris | APIs. |
93 | 4054c46d | Stavros Sachtouris | |
94 | fa382f9e | Stavros Sachtouris | .. note:: unit tests require the optional python-mock package, version 1.X or |
95 | fa382f9e | Stavros Sachtouris | better |
96 | 4054c46d | Stavros Sachtouris | |
97 | 4054c46d | Stavros Sachtouris | Using the tests |
98 | 4054c46d | Stavros Sachtouris | ^^^^^^^^^^^^^^^ |
99 | 4054c46d | Stavros Sachtouris | |
100 | cfb87e18 | Stavros Sachtouris | First, the source code of kamaki must be accessible. If this is not the case, |
101 | cfb87e18 | Stavros Sachtouris | the source code can be obtained from here: |
102 | 4054c46d | Stavros Sachtouris | |
103 | cfb87e18 | Stavros Sachtouris | `https://code.grnet.gr/projects/kamaki/files <https://code.grnet.gr/projects/kamaki/files>`_ |
104 | 4054c46d | Stavros Sachtouris | |
105 | cfb87e18 | Stavros Sachtouris | In each package under kamaki.clients, there is a test module (`test.py`). To |
106 | cfb87e18 | Stavros Sachtouris | run all tests, run the test.py file from kamaki.clients |
107 | 4054c46d | Stavros Sachtouris | |
108 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
109 | 4054c46d | Stavros Sachtouris | |
110 | cfb87e18 | Stavros Sachtouris | $ cd ${KAMAKI_SOURCE_LOCATION}/kamaki/clients |
111 | 4054c46d | Stavros Sachtouris | $ python test.py |
112 | 4054c46d | Stavros Sachtouris | |
113 | 16d7b9ff | Stavros Sachtouris | To test a specific class, add the class name as an argument. e.g., for the |
114 | fa382f9e | Stavros Sachtouris | Client class: |
115 | 4054c46d | Stavros Sachtouris | |
116 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
117 | 4054c46d | Stavros Sachtouris | |
118 | 4054c46d | Stavros Sachtouris | $ python test.py Client |
119 | 4054c46d | Stavros Sachtouris | |
120 | 16d7b9ff | Stavros Sachtouris | To test a specific method in a class, apply an extra argument, e.g., for the |
121 | fa382f9e | Stavros Sachtouris | request method in the Client class: |
122 | 4054c46d | Stavros Sachtouris | |
123 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
124 | 4054c46d | Stavros Sachtouris | |
125 | 4054c46d | Stavros Sachtouris | $ python test.py Client request |
126 | 4054c46d | Stavros Sachtouris | |
127 | fa382f9e | Stavros Sachtouris | Each package contains a test module (test.py) which is also runnable from the |
128 | 16d7b9ff | Stavros Sachtouris | command line. e.g., in the pithos package there is a test module which |
129 | fa382f9e | Stavros Sachtouris | contains, among others, the **download** sub-test: |
130 | 4054c46d | Stavros Sachtouris | |
131 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
132 | 4054c46d | Stavros Sachtouris | |
133 | 4054c46d | Stavros Sachtouris | $ cd pithos |
134 | 4054c46d | Stavros Sachtouris | |
135 | 4054c46d | Stavros Sachtouris | # Run all kamaki.clients.pithos tests |
136 | 4054c46d | Stavros Sachtouris | $ python test.py |
137 | 4054c46d | Stavros Sachtouris | |
138 | 4054c46d | Stavros Sachtouris | # Run all kamaki.clients.pithos.PithoClient tests |
139 | 4054c46d | Stavros Sachtouris | $ python test.py Pithos |
140 | 4054c46d | Stavros Sachtouris | |
141 | 4054c46d | Stavros Sachtouris | # Test kamaki.clients.pithos.PithosClient.download |
142 | 4054c46d | Stavros Sachtouris | $ python test.py Pithos download |
143 | 4054c46d | Stavros Sachtouris | |
144 | 16d7b9ff | Stavros Sachtouris | To fully test a specific package, run test.py from the package location. e.g., |
145 | fa382f9e | Stavros Sachtouris | to test everything in kamaki.clients.pithos package: |
146 | 3c50df2e | Stavros Sachtouris | |
147 | 3c50df2e | Stavros Sachtouris | .. code-block:: console |
148 | 3c50df2e | Stavros Sachtouris | |
149 | 3c50df2e | Stavros Sachtouris | $ cd pithos |
150 | 3c50df2e | Stavros Sachtouris | $ python test.py |
151 | 3c50df2e | Stavros Sachtouris | |
152 | 4054c46d | Stavros Sachtouris | Mechanism |
153 | 4054c46d | Stavros Sachtouris | ^^^^^^^^^ |
154 | 4054c46d | Stavros Sachtouris | |
155 | cfb87e18 | Stavros Sachtouris | Each folder / package contains a test.py file, where its test module lived. All |
156 | cfb87e18 | Stavros Sachtouris | test modules contain a set of classes that extent the TestCase class. They also |
157 | cfb87e18 | Stavros Sachtouris | contain a main method to run the tests. |
158 | 4054c46d | Stavros Sachtouris | |
159 | 16d7b9ff | Stavros Sachtouris | By convention, testing classes have the same name as the class they test. |
160 | 16d7b9ff | Stavros Sachtouris | Methods not grouped in classes are tested by classes named after their |
161 | 16d7b9ff | Stavros Sachtouris | respective module. |
162 | 4054c46d | Stavros Sachtouris | |
163 | 16d7b9ff | Stavros Sachtouris | For example, the *kamaki.clients.pithos.PithosClient* class is tested by the |
164 | 16d7b9ff | Stavros Sachtouris | *kamaki.clients.pithos.test.PithosClient* class, while the methods in |
165 | 16d7b9ff | Stavros Sachtouris | *kamaki.clients.utils* module are tested by *kamaki.clients.utils.test.Utils*. |
166 | 4054c46d | Stavros Sachtouris | |
167 | 4054c46d | Stavros Sachtouris | Adding unit tests |
168 | 4054c46d | Stavros Sachtouris | ^^^^^^^^^^^^^^^^^ |
169 | fa382f9e | Stavros Sachtouris | |
170 | 16d7b9ff | Stavros Sachtouris | After modifying or extending *kamaki.clients* method, classes, modules or |
171 | fa382f9e | Stavros Sachtouris | packages, it is a good practice to also modify or extend the corresponding |
172 | fa382f9e | Stavros Sachtouris | unit tests. What's more, it is recommended to modify or implement the testing |
173 | 16d7b9ff | Stavros Sachtouris | of new behavior before implementing the behavior itself. The goal is to |
174 | 16d7b9ff | Stavros Sachtouris | preserve an 1 to 1 mapping between methods and their tests. |
175 | 4054c46d | Stavros Sachtouris | |
176 | 4054c46d | Stavros Sachtouris | Modifying an existing method |
177 | 4054c46d | Stavros Sachtouris | """""""""""""""""""""""""""" |
178 | 4054c46d | Stavros Sachtouris | |
179 | fa382f9e | Stavros Sachtouris | In case of an existing method modification, the programmer has to modify the |
180 | fa382f9e | Stavros Sachtouris | corresponding test as well. By convention, the test method is located in the |
181 | fa382f9e | Stavros Sachtouris | test module under the same package, in a TestCase subclass that is named with a |
182 | fa382f9e | Stavros Sachtouris | name similar to the package or class that contains the tested method. |
183 | 4054c46d | Stavros Sachtouris | |
184 | 16d7b9ff | Stavros Sachtouris | Example: to modify *kamaki.clients.pithos.PithosRestClient.object_get*, the |
185 | fa382f9e | Stavros Sachtouris | programmer has to also adjust the |
186 | 16d7b9ff | Stavros Sachtouris | *kamaki.clients.pithos.test.PithosRestClient.test.object_get* method. |
187 | 4054c46d | Stavros Sachtouris | |
188 | 4054c46d | Stavros Sachtouris | Adding a new method |
189 | 4054c46d | Stavros Sachtouris | """"""""""""""""""" |
190 | 4054c46d | Stavros Sachtouris | |
191 | 16d7b9ff | Stavros Sachtouris | To implement a new method in an existing class, developers are encouraged to |
192 | 16d7b9ff | Stavros Sachtouris | implement the corresponding unit test first. In order to do that, they should |
193 | cfb87e18 | Stavros Sachtouris | find the testing class that is mapped to the class or module they work on. |
194 | 4054c46d | Stavros Sachtouris | |
195 | 16d7b9ff | Stavros Sachtouris | Example: Adding **list_special** to *kamaki.clients.astakos.AstakosClient*, |
196 | 16d7b9ff | Stavros Sachtouris | requires changes to *kamaki.clients.astakos.test.AstakosClient* too, as shown |
197 | 16d7b9ff | Stavros Sachtouris | bellow: |
198 | 4054c46d | Stavros Sachtouris | |
199 | 4054c46d | Stavros Sachtouris | .. code-block:: python |
200 | 4054c46d | Stavros Sachtouris | |
201 | 3c50df2e | Stavros Sachtouris | # file: ${kamaki}/kamaki/clients/astakos/__init__.py |
202 | 3c50df2e | Stavros Sachtouris | |
203 | 3c50df2e | Stavros Sachtouris | class AstakosClient(TestCase): |
204 | 4054c46d | Stavros Sachtouris | ... |
205 | 4054c46d | Stavros Sachtouris | def test_list_special(self): |
206 | 4054c46d | Stavros Sachtouris | """Test the list_special method""" |
207 | 4054c46d | Stavros Sachtouris | ... |
208 | 4054c46d | Stavros Sachtouris | |
209 | 4054c46d | Stavros Sachtouris | Implementing a new class or module |
210 | 4054c46d | Stavros Sachtouris | """""""""""""""""""""""""""""""""" |
211 | 4054c46d | Stavros Sachtouris | |
212 | fa382f9e | Stavros Sachtouris | Each class or module needs a seperate test sub-module. By convention, each |
213 | 16d7b9ff | Stavros Sachtouris | class or module under *kamaki.clients*, should be located in a separate |
214 | fa382f9e | Stavros Sachtouris | directory. |
215 | 4054c46d | Stavros Sachtouris | |
216 | 16d7b9ff | Stavros Sachtouris | Example 1: To add a NewService class that implements *kamaki.clients.Client*: |
217 | 4054c46d | Stavros Sachtouris | |
218 | 16d7b9ff | Stavros Sachtouris | * create a new_service package and implement the unit tests in |
219 | 16d7b9ff | Stavros Sachtouris | *kamaki.clients.new_service.test*: |
220 | 4054c46d | Stavros Sachtouris | |
221 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
222 | 4054c46d | Stavros Sachtouris | |
223 | 4054c46d | Stavros Sachtouris | $ mkdir new_service && touch new_service/test.py |
224 | 4054c46d | Stavros Sachtouris | |
225 | cfb87e18 | Stavros Sachtouris | * create the package file for the package implementation: |
226 | 4054c46d | Stavros Sachtouris | |
227 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
228 | 4054c46d | Stavros Sachtouris | |
229 | 4054c46d | Stavros Sachtouris | $ touch new_service/__init__.py |
230 | 4054c46d | Stavros Sachtouris | |
231 | 16d7b9ff | Stavros Sachtouris | * Create the test class and methods in *kamaki.clients.new_service.test* |
232 | 4054c46d | Stavros Sachtouris | |
233 | 4054c46d | Stavros Sachtouris | .. code-block:: python |
234 | 4054c46d | Stavros Sachtouris | |
235 | 3c50df2e | Stavros Sachtouris | # file: ${kamaki}/kamaki/clients/new_service/test.py |
236 | 4054c46d | Stavros Sachtouris | from unittest import TestCase |
237 | 4054c46d | Stavros Sachtouris | |
238 | 3c50df2e | Stavros Sachtouris | class NewService(TestCase): |
239 | 4054c46d | Stavros Sachtouris | |
240 | 4054c46d | Stavros Sachtouris | def test_method1(self): |
241 | 4054c46d | Stavros Sachtouris | ... |
242 | 4054c46d | Stavros Sachtouris | |
243 | cfb87e18 | Stavros Sachtouris | * Create the NewService and its actual functionality in |
244 | cfb87e18 | Stavros Sachtouris | kamaki.clients.new_service |
245 | 4054c46d | Stavros Sachtouris | |
246 | 4054c46d | Stavros Sachtouris | .. code-block:: python |
247 | 4054c46d | Stavros Sachtouris | |
248 | 3c50df2e | Stavros Sachtouris | # file: ${kamaki}/kamaki/clients/new_service/__init__.py |
249 | 4054c46d | Stavros Sachtouris | from kamaki.clients import Client |
250 | 4054c46d | Stavros Sachtouris | |
251 | 4054c46d | Stavros Sachtouris | class NewService(Client): |
252 | 4054c46d | Stavros Sachtouris | |
253 | 4054c46d | Stavros Sachtouris | def method1(self, ...): |
254 | 4054c46d | Stavros Sachtouris | ... |
255 | 4054c46d | Stavros Sachtouris | |
256 | 16d7b9ff | Stavros Sachtouris | * Import the test class to *kamaki.clients.test*: |
257 | 4054c46d | Stavros Sachtouris | |
258 | 16d7b9ff | Stavros Sachtouris | .. code-block:: python |
259 | 4054c46d | Stavros Sachtouris | |
260 | 3c50df2e | Stavros Sachtouris | # file: ${kamaki}/kamaki/clients/test.py |
261 | 4054c46d | Stavros Sachtouris | from kamaki.clients.new_service.test import NewService |
262 | 4054c46d | Stavros Sachtouris | |
263 | fa382f9e | Stavros Sachtouris | .. note:: If the new class or module is part of an existing sub-package, it is |
264 | fa382f9e | Stavros Sachtouris | acceptable to append its testing class in the existing test.py file of the |
265 | fa382f9e | Stavros Sachtouris | sub-package it belongs to. For example, the |
266 | fa382f9e | Stavros Sachtouris | kamaki.clients.pithos.PithosClient and |
267 | fa382f9e | Stavros Sachtouris | kamaki.clients.pithos.rest_api.PithosRestClient classes are tested by two |
268 | fa382f9e | Stavros Sachtouris | different classes (PithosClient and PithosRestClient respectively) in the |
269 | fa382f9e | Stavros Sachtouris | same module (kamaki.clients.pithos.test). |