root / docs / developers / extending-clients-api.rst @ fe980c71
History | View | Annotate | Download (8.2 kB)
1 | 9e4508df | Stavros Sachtouris | Extending kamaki.clients |
---|---|---|---|
2 | 9e4508df | Stavros Sachtouris | ======================== |
3 | 9e4508df | Stavros Sachtouris | |
4 | 16d7b9ff | Stavros Sachtouris | By default, kamaki clients implement REST APIs, therefore they manage HTTP |
5 | 16d7b9ff | Stavros Sachtouris | requests and responses to communicate with services. |
6 | 9e4508df | Stavros Sachtouris | |
7 | 9e4508df | Stavros Sachtouris | How to build a client |
8 | 9e4508df | Stavros Sachtouris | --------------------- |
9 | 9e4508df | Stavros Sachtouris | |
10 | fa382f9e | Stavros Sachtouris | All service clients consist of a subclass of the Client class and implement |
11 | fa382f9e | Stavros Sachtouris | separate client functionalities as member methods. There is also an error class |
12 | fa382f9e | Stavros Sachtouris | to raise exceptions that can be handled by kamaki interfaces. |
13 | 9e4508df | Stavros Sachtouris | |
14 | 9e4508df | Stavros Sachtouris | .. code-block:: python |
15 | 9e4508df | Stavros Sachtouris | |
16 | 9e4508df | Stavros Sachtouris | # ${KAMAKI_PATH}/kamaki/clients/mynewclient.py |
17 | 9e4508df | Stavros Sachtouris | |
18 | 9e4508df | Stavros Sachtouris | from kamaki.clients import Client, ClientError |
19 | 9e4508df | Stavros Sachtouris | |
20 | 16d7b9ff | Stavros Sachtouris | |
21 | 9e4508df | Stavros Sachtouris | class MyNewClient(Client): |
22 | 9e4508df | Stavros Sachtouris | """MyNewClient Description Here""" |
23 | 9e4508df | Stavros Sachtouris | |
24 | 9e4508df | Stavros Sachtouris | def my_first_method(self, **args): |
25 | 9e4508df | Stavros Sachtouris | """Method description""" |
26 | 9e4508df | Stavros Sachtouris | try: |
27 | 9e4508df | Stavros Sachtouris | ... |
28 | 9e4508df | Stavros Sachtouris | method code |
29 | 9e4508df | Stavros Sachtouris | ... |
30 | 9e4508df | Stavros Sachtouris | except SomeKnownException as e1: |
31 | 9e4508df | Stavros Sachtouris | raise ClientError('MyError: %s' % e1) |
32 | 9e4508df | Stavros Sachtouris | except SomeOtherException as e2: |
33 | 9e4508df | Stavros Sachtouris | raise ClientError('MyError: %s' % e2) |
34 | 9e4508df | Stavros Sachtouris | |
35 | 9e4508df | Stavros Sachtouris | def my_second_method(self, **params): |
36 | 9e4508df | Stavros Sachtouris | """Method description""" |
37 | 9e4508df | Stavros Sachtouris | ... |
38 | 9e4508df | Stavros Sachtouris | |
39 | 9e4508df | Stavros Sachtouris | Custom clients can use a set of convenience methods for easy HTTP requests |
40 | 9e4508df | Stavros Sachtouris | |
41 | 9e4508df | Stavros Sachtouris | .. code-block:: python |
42 | 9e4508df | Stavros Sachtouris | |
43 | 9e4508df | Stavros Sachtouris | def get(self, path, **kwargs) |
44 | 9e4508df | Stavros Sachtouris | def head(self, path, **kwargs) |
45 | 9e4508df | Stavros Sachtouris | def post(self, path, **kwargs) |
46 | 9e4508df | Stavros Sachtouris | def put(self, path, **kwargs) |
47 | 9e4508df | Stavros Sachtouris | def delete(self, path, **kwargs) |
48 | 9e4508df | Stavros Sachtouris | def copy(self, path, **kwargs) |
49 | 9e4508df | Stavros Sachtouris | def move(self, path, **kwargs) |
50 | 9e4508df | Stavros Sachtouris | |
51 | 9e4508df | Stavros Sachtouris | How to use your client |
52 | 9e4508df | Stavros Sachtouris | ---------------------- |
53 | 9e4508df | Stavros Sachtouris | |
54 | 9e4508df | Stavros Sachtouris | External applications must instantiate a MyNewClient object. |
55 | 9e4508df | Stavros Sachtouris | |
56 | 9e4508df | Stavros Sachtouris | .. code-block:: python |
57 | 9e4508df | Stavros Sachtouris | |
58 | 9e4508df | Stavros Sachtouris | from kamaki.clients import ClientError |
59 | 9e4508df | Stavros Sachtouris | from kamaki.clients.mynewclient import MyNewClient |
60 | 9e4508df | Stavros Sachtouris | |
61 | 9e4508df | Stavros Sachtouris | ... |
62 | 9e4508df | Stavros Sachtouris | try: |
63 | 9e4508df | Stavros Sachtouris | cl = MyNewClient(args) |
64 | 9e4508df | Stavros Sachtouris | cl.my_first_method(other_args) |
65 | 9e4508df | Stavros Sachtouris | except ClientError as cle: |
66 | 9e4508df | Stavros Sachtouris | print('Client Error: %s' % cle) |
67 | 9e4508df | Stavros Sachtouris | ... |
68 | 9e4508df | Stavros Sachtouris | |
69 | 9e4508df | Stavros Sachtouris | Concurrency control |
70 | 9e4508df | Stavros Sachtouris | ------------------- |
71 | 9e4508df | Stavros Sachtouris | |
72 | fa382f9e | Stavros Sachtouris | Kamaki clients may handle multiple requests at once, using threads. In that |
73 | fa382f9e | Stavros Sachtouris | case, users might implement their own thread handling mechanism, use an |
74 | fa382f9e | Stavros Sachtouris | external solution or take advantage of the mechanism featured in kamaki.clients |
75 | 9e4508df | Stavros Sachtouris | |
76 | 9e4508df | Stavros Sachtouris | .. code-block:: python |
77 | 9e4508df | Stavros Sachtouris | |
78 | 9e4508df | Stavros Sachtouris | from threading import enumerate |
79 | 9e4508df | Stavros Sachtouris | from kamaki.clients import SilentEvent |
80 | 9e4508df | Stavros Sachtouris | ... |
81 | 9e4508df | Stavros Sachtouris | |
82 | 9e4508df | Stavros Sachtouris | class MyNewClient(Client): |
83 | 9e4508df | Stavros Sachtouris | ... |
84 | 9e4508df | Stavros Sachtouris | |
85 | 9e4508df | Stavros Sachtouris | def _single_threaded_method(self, **args): |
86 | 9e4508df | Stavros Sachtouris | ... |
87 | 9e4508df | Stavros Sachtouris | request code |
88 | 9e4508df | Stavros Sachtouris | ... |
89 | 9e4508df | Stavros Sachtouris | |
90 | 9e4508df | Stavros Sachtouris | def multithread_method(self): |
91 | 9e4508df | Stavros Sachtouris | thread_list = [] |
92 | 9e4508df | Stavros Sachtouris | self._init_thread_limit() |
93 | 9e4508df | Stavros Sachtouris | while some_condition or thread_list: |
94 | 9e4508df | Stavros Sachtouris | ... |
95 | 9e4508df | Stavros Sachtouris | event = SilentEvent(self._single_threaded_method, **args) |
96 | 9e4508df | Stavros Sachtouris | event.start() |
97 | 9e4508df | Stavros Sachtouris | thread_list.append(event) |
98 | 4054c46d | Stavros Sachtouris | thread_list = self._watch_thread_limit(thread_list) |
99 | 4054c46d | Stavros Sachtouris | |
100 | 4054c46d | Stavros Sachtouris | Going agile |
101 | 4054c46d | Stavros Sachtouris | ----------- |
102 | 4054c46d | Stavros Sachtouris | |
103 | fa382f9e | Stavros Sachtouris | The kamaki.clients package contains a set of fine-grained unit-tests for all |
104 | 16d7b9ff | Stavros Sachtouris | APIs. |
105 | 4054c46d | Stavros Sachtouris | |
106 | fa382f9e | Stavros Sachtouris | .. note:: unit tests require the optional python-mock package, version 1.X or |
107 | fa382f9e | Stavros Sachtouris | better |
108 | 4054c46d | Stavros Sachtouris | |
109 | 4054c46d | Stavros Sachtouris | Using the tests |
110 | 4054c46d | Stavros Sachtouris | ^^^^^^^^^^^^^^^ |
111 | 4054c46d | Stavros Sachtouris | |
112 | 4054c46d | Stavros Sachtouris | To run the tests, the kamaki source code has to be downloaded. |
113 | 4054c46d | Stavros Sachtouris | |
114 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
115 | 4054c46d | Stavros Sachtouris | |
116 | 4054c46d | Stavros Sachtouris | $ git clone https://code.grnet.gr/git/kamaki |
117 | 4054c46d | Stavros Sachtouris | $ cd kamaki/kamaki/clients |
118 | 4054c46d | Stavros Sachtouris | |
119 | fa382f9e | Stavros Sachtouris | In each package under kamaki.clients, there is a test module (test.py) where |
120 | fa382f9e | Stavros Sachtouris | the tests are implemented. To run all tests, run the test.py file from |
121 | fa382f9e | Stavros Sachtouris | kamaki.clients |
122 | 4054c46d | Stavros Sachtouris | |
123 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
124 | 4054c46d | Stavros Sachtouris | |
125 | 4054c46d | Stavros Sachtouris | $ python test.py |
126 | 4054c46d | Stavros Sachtouris | |
127 | 16d7b9ff | Stavros Sachtouris | To test a specific class, add the class name as an argument. e.g., for the |
128 | fa382f9e | Stavros Sachtouris | Client class: |
129 | 4054c46d | Stavros Sachtouris | |
130 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
131 | 4054c46d | Stavros Sachtouris | |
132 | 4054c46d | Stavros Sachtouris | $ python test.py Client |
133 | 4054c46d | Stavros Sachtouris | |
134 | 16d7b9ff | Stavros Sachtouris | To test a specific method in a class, apply an extra argument, e.g., for the |
135 | fa382f9e | Stavros Sachtouris | request method in the Client class: |
136 | 4054c46d | Stavros Sachtouris | |
137 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
138 | 4054c46d | Stavros Sachtouris | |
139 | 4054c46d | Stavros Sachtouris | $ python test.py Client request |
140 | 4054c46d | Stavros Sachtouris | |
141 | fa382f9e | Stavros Sachtouris | Each package contains a test module (test.py) which is also runnable from the |
142 | 16d7b9ff | Stavros Sachtouris | command line. e.g., in the pithos package there is a test module which |
143 | fa382f9e | Stavros Sachtouris | contains, among others, the **download** sub-test: |
144 | 4054c46d | Stavros Sachtouris | |
145 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
146 | 4054c46d | Stavros Sachtouris | |
147 | 4054c46d | Stavros Sachtouris | $ cd pithos |
148 | 4054c46d | Stavros Sachtouris | |
149 | 4054c46d | Stavros Sachtouris | # Run all kamaki.clients.pithos tests |
150 | 4054c46d | Stavros Sachtouris | $ python test.py |
151 | 4054c46d | Stavros Sachtouris | |
152 | 4054c46d | Stavros Sachtouris | # Run all kamaki.clients.pithos.PithoClient tests |
153 | 4054c46d | Stavros Sachtouris | $ python test.py Pithos |
154 | 4054c46d | Stavros Sachtouris | |
155 | 4054c46d | Stavros Sachtouris | # Test kamaki.clients.pithos.PithosClient.download |
156 | 4054c46d | Stavros Sachtouris | $ python test.py Pithos download |
157 | 4054c46d | Stavros Sachtouris | |
158 | 16d7b9ff | Stavros Sachtouris | To fully test a specific package, run test.py from the package location. e.g., |
159 | fa382f9e | Stavros Sachtouris | to test everything in kamaki.clients.pithos package: |
160 | 3c50df2e | Stavros Sachtouris | |
161 | 3c50df2e | Stavros Sachtouris | .. code-block:: console |
162 | 3c50df2e | Stavros Sachtouris | |
163 | 3c50df2e | Stavros Sachtouris | $ cd pithos |
164 | 3c50df2e | Stavros Sachtouris | $ python test.py |
165 | 3c50df2e | Stavros Sachtouris | |
166 | 4054c46d | Stavros Sachtouris | Mechanism |
167 | 4054c46d | Stavros Sachtouris | ^^^^^^^^^ |
168 | 4054c46d | Stavros Sachtouris | |
169 | fa382f9e | Stavros Sachtouris | Each folder / package contains a test.py file, that represents the test module |
170 | fa382f9e | Stavros Sachtouris | of this package. All test modules contain a set of classes that extent the |
171 | fa382f9e | Stavros Sachtouris | TestCase class. They also contain a main method to run the tests. |
172 | 4054c46d | Stavros Sachtouris | |
173 | 16d7b9ff | Stavros Sachtouris | By convention, testing classes have the same name as the class they test. |
174 | 16d7b9ff | Stavros Sachtouris | Methods not grouped in classes are tested by classes named after their |
175 | 16d7b9ff | Stavros Sachtouris | respective module. |
176 | 4054c46d | Stavros Sachtouris | |
177 | 16d7b9ff | Stavros Sachtouris | For example, the *kamaki.clients.pithos.PithosClient* class is tested by the |
178 | 16d7b9ff | Stavros Sachtouris | *kamaki.clients.pithos.test.PithosClient* class, while the methods in |
179 | 16d7b9ff | Stavros Sachtouris | *kamaki.clients.utils* module are tested by *kamaki.clients.utils.test.Utils*. |
180 | 4054c46d | Stavros Sachtouris | |
181 | 4054c46d | Stavros Sachtouris | Adding unit tests |
182 | 4054c46d | Stavros Sachtouris | ^^^^^^^^^^^^^^^^^ |
183 | fa382f9e | Stavros Sachtouris | |
184 | 16d7b9ff | Stavros Sachtouris | After modifying or extending *kamaki.clients* method, classes, modules or |
185 | fa382f9e | Stavros Sachtouris | packages, it is a good practice to also modify or extend the corresponding |
186 | fa382f9e | Stavros Sachtouris | unit tests. What's more, it is recommended to modify or implement the testing |
187 | 16d7b9ff | Stavros Sachtouris | of new behavior before implementing the behavior itself. The goal is to |
188 | 16d7b9ff | Stavros Sachtouris | preserve an 1 to 1 mapping between methods and their tests. |
189 | 4054c46d | Stavros Sachtouris | |
190 | 4054c46d | Stavros Sachtouris | Modifying an existing method |
191 | 4054c46d | Stavros Sachtouris | """""""""""""""""""""""""""" |
192 | 4054c46d | Stavros Sachtouris | |
193 | fa382f9e | Stavros Sachtouris | In case of an existing method modification, the programmer has to modify the |
194 | fa382f9e | Stavros Sachtouris | corresponding test as well. By convention, the test method is located in the |
195 | fa382f9e | Stavros Sachtouris | test module under the same package, in a TestCase subclass that is named with a |
196 | fa382f9e | Stavros Sachtouris | name similar to the package or class that contains the tested method. |
197 | 4054c46d | Stavros Sachtouris | |
198 | 16d7b9ff | Stavros Sachtouris | Example: to modify *kamaki.clients.pithos.PithosRestClient.object_get*, the |
199 | fa382f9e | Stavros Sachtouris | programmer has to also adjust the |
200 | 16d7b9ff | Stavros Sachtouris | *kamaki.clients.pithos.test.PithosRestClient.test.object_get* method. |
201 | 4054c46d | Stavros Sachtouris | |
202 | 4054c46d | Stavros Sachtouris | Adding a new method |
203 | 4054c46d | Stavros Sachtouris | """"""""""""""""""" |
204 | 4054c46d | Stavros Sachtouris | |
205 | 16d7b9ff | Stavros Sachtouris | To implement a new method in an existing class, developers are encouraged to |
206 | 16d7b9ff | Stavros Sachtouris | implement the corresponding unit test first. In order to do that, they should |
207 | 16d7b9ff | Stavros Sachtouris | find the testing class that is mapped to the class or module they working on. |
208 | 4054c46d | Stavros Sachtouris | |
209 | 16d7b9ff | Stavros Sachtouris | Example: Adding **list_special** to *kamaki.clients.astakos.AstakosClient*, |
210 | 16d7b9ff | Stavros Sachtouris | requires changes to *kamaki.clients.astakos.test.AstakosClient* too, as shown |
211 | 16d7b9ff | Stavros Sachtouris | bellow: |
212 | 4054c46d | Stavros Sachtouris | |
213 | 4054c46d | Stavros Sachtouris | .. code-block:: python |
214 | 4054c46d | Stavros Sachtouris | |
215 | 3c50df2e | Stavros Sachtouris | # file: ${kamaki}/kamaki/clients/astakos/__init__.py |
216 | 3c50df2e | Stavros Sachtouris | |
217 | 3c50df2e | Stavros Sachtouris | class AstakosClient(TestCase): |
218 | 4054c46d | Stavros Sachtouris | ... |
219 | 4054c46d | Stavros Sachtouris | def test_list_special(self): |
220 | 4054c46d | Stavros Sachtouris | """Test the list_special method""" |
221 | 4054c46d | Stavros Sachtouris | ... |
222 | 4054c46d | Stavros Sachtouris | |
223 | 4054c46d | Stavros Sachtouris | Implementing a new class or module |
224 | 4054c46d | Stavros Sachtouris | """""""""""""""""""""""""""""""""" |
225 | 4054c46d | Stavros Sachtouris | |
226 | fa382f9e | Stavros Sachtouris | Each class or module needs a seperate test sub-module. By convention, each |
227 | 16d7b9ff | Stavros Sachtouris | class or module under *kamaki.clients*, should be located in a separate |
228 | fa382f9e | Stavros Sachtouris | directory. |
229 | 4054c46d | Stavros Sachtouris | |
230 | 16d7b9ff | Stavros Sachtouris | Example 1: To add a NewService class that implements *kamaki.clients.Client*: |
231 | 4054c46d | Stavros Sachtouris | |
232 | 16d7b9ff | Stavros Sachtouris | * create a new_service package and implement the unit tests in |
233 | 16d7b9ff | Stavros Sachtouris | *kamaki.clients.new_service.test*: |
234 | 4054c46d | Stavros Sachtouris | |
235 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
236 | 4054c46d | Stavros Sachtouris | |
237 | 4054c46d | Stavros Sachtouris | $ mkdir new_service && touch new_service/test.py |
238 | 4054c46d | Stavros Sachtouris | |
239 | 16d7b9ff | Stavros Sachtouris | * create the file to code the package implementation: |
240 | 4054c46d | Stavros Sachtouris | |
241 | 4054c46d | Stavros Sachtouris | .. code-block:: console |
242 | 4054c46d | Stavros Sachtouris | |
243 | 4054c46d | Stavros Sachtouris | $ touch new_service/__init__.py |
244 | 4054c46d | Stavros Sachtouris | |
245 | 16d7b9ff | Stavros Sachtouris | * Create the test class and methods in *kamaki.clients.new_service.test* |
246 | 4054c46d | Stavros Sachtouris | |
247 | 4054c46d | Stavros Sachtouris | .. code-block:: python |
248 | 4054c46d | Stavros Sachtouris | |
249 | 3c50df2e | Stavros Sachtouris | # file: ${kamaki}/kamaki/clients/new_service/test.py |
250 | 4054c46d | Stavros Sachtouris | from unittest import TestCase |
251 | 4054c46d | Stavros Sachtouris | |
252 | 3c50df2e | Stavros Sachtouris | class NewService(TestCase): |
253 | 4054c46d | Stavros Sachtouris | |
254 | 4054c46d | Stavros Sachtouris | def test_method1(self): |
255 | 4054c46d | Stavros Sachtouris | ... |
256 | 4054c46d | Stavros Sachtouris | |
257 | 4054c46d | Stavros Sachtouris | * Create the NewService and its actual functionality in kamaki.clients.new_service |
258 | 4054c46d | Stavros Sachtouris | |
259 | 4054c46d | Stavros Sachtouris | .. code-block:: python |
260 | 4054c46d | Stavros Sachtouris | |
261 | 3c50df2e | Stavros Sachtouris | # file: ${kamaki}/kamaki/clients/new_service/__init__.py |
262 | 4054c46d | Stavros Sachtouris | from kamaki.clients import Client |
263 | 4054c46d | Stavros Sachtouris | |
264 | 4054c46d | Stavros Sachtouris | class NewService(Client): |
265 | 4054c46d | Stavros Sachtouris | |
266 | 4054c46d | Stavros Sachtouris | def method1(self, ...): |
267 | 4054c46d | Stavros Sachtouris | ... |
268 | 4054c46d | Stavros Sachtouris | |
269 | 16d7b9ff | Stavros Sachtouris | * Import the test class to *kamaki.clients.test*: |
270 | 4054c46d | Stavros Sachtouris | |
271 | 16d7b9ff | Stavros Sachtouris | .. code-block:: python |
272 | 4054c46d | Stavros Sachtouris | |
273 | 3c50df2e | Stavros Sachtouris | # file: ${kamaki}/kamaki/clients/test.py |
274 | 4054c46d | Stavros Sachtouris | from kamaki.clients.new_service.test import NewService |
275 | 4054c46d | Stavros Sachtouris | |
276 | fa382f9e | Stavros Sachtouris | .. note:: If the new class or module is part of an existing sub-package, it is |
277 | fa382f9e | Stavros Sachtouris | acceptable to append its testing class in the existing test.py file of the |
278 | fa382f9e | Stavros Sachtouris | sub-package it belongs to. For example, the |
279 | fa382f9e | Stavros Sachtouris | kamaki.clients.pithos.PithosClient and |
280 | fa382f9e | Stavros Sachtouris | kamaki.clients.pithos.rest_api.PithosRestClient classes are tested by two |
281 | fa382f9e | Stavros Sachtouris | different classes (PithosClient and PithosRestClient respectively) in the |
282 | fa382f9e | Stavros Sachtouris | same module (kamaki.clients.pithos.test). |