Statistics
| Branch: | Tag: | Revision:

root / docs / developers / extending-clients-api.rst @ 4054c46d

History | View | Annotate | Download (8.6 kB)

1
Extending kamaki.clients
2
========================
3

    
4
By default, kamaki clients are REST clients (they manage HTTP requests and responses to communicate with services). This is achieved by importing the connection module, which is an httplib wrapper.
5

    
6
Connection
7
----------
8

    
9
The connection module features an error handling and logging system, a lazy response mechanism, a pooling mechanism as well as concurrency control for thread-demanding client functions (e.g. store upload).
10

    
11
How to build a client
12
---------------------
13

    
14
All service clients consist of a subclass of the Client class and implement separate client functionalities as member methods. There is also an error class to raise exceptions that can be handled by kamaki interfaces.
15

    
16
.. code-block:: python
17
    
18
    #  ${KAMAKI_PATH}/kamaki/clients/mynewclient.py
19

    
20
    from kamaki.clients import Client, ClientError
21

    
22
    class MyNewClient(Client):
23
        """MyNewClient Description Here"""
24

    
25
        def my_first_method(self, **args):
26
            """Method description"""
27
            try:
28
                ...
29
                method code
30
                ...
31
            except SomeKnownException as e1:
32
                raise ClientError('MyError: %s' % e1)
33
            except SomeOtherException as e2:
34
                raise ClientError('MyError: %s' % e2)
35

    
36
        def my_second_method(self, **params):
37
            """Method description"""
38
            ...
39

    
40
Custom clients can use a set of convenience methods for easy HTTP requests
41

    
42
.. code-block:: python
43

    
44
    def get(self, path, **kwargs)
45
    def head(self, path, **kwargs)
46
    def post(self, path, **kwargs)
47
    def put(self, path, **kwargs)
48
    def delete(self, path, **kwargs)
49
    def copy(self, path, **kwargs)
50
    def move(self, path, **kwargs)
51

    
52
How to use your client
53
----------------------
54

    
55
External applications must instantiate a MyNewClient object.
56

    
57
.. code-block:: python
58

    
59
    from kamaki.clients import ClientError
60
    from kamaki.clients.mynewclient import MyNewClient
61

    
62
    ...
63
    try:
64
        cl = MyNewClient(args)
65
        cl.my_first_method(other_args)
66
    except ClientError as cle:
67
        print('Client Error: %s' % cle)
68
    ...
69

    
70
Concurrency control
71
-------------------
72

    
73
Kamaki clients may handle multiple requests at once, using threads. In that case, users might implement their own thread handling mechanism, use an external solution or take advantage of the mechanism featured in kamaki.clients
74

    
75
.. code-block:: python
76

    
77
    from threading import enumerate
78
    from kamaki.clients import SilentEvent
79
    ...
80

    
81
    class MyNewClient(Client):
82
        ...
83

    
84
        def _single_threaded_method(self, **args):
85
            ...
86
            request code
87
            ...
88

    
89
        def multithread_method(self):
90
            thread_list = []
91
            self._init_thread_limit()
92
            while some_condition or thread_list:
93
                ...
94
                event = SilentEvent(self._single_threaded_method, **args)
95
                event.start()
96
                thread_list.append(event)
97
                thread_list = self._watch_thread_limit(thread_list)
98

    
99
Going agile
100
-----------
101

    
102
The kamaki.clients package contains a set of fine-grained unit-tests for all its packages. 
103

    
104
.. note:: unit tests require the optional python-mock package, version 1.X or better
105

    
106
Using the tests
107
^^^^^^^^^^^^^^^
108

    
109
To run the tests, the kamaki source code has to be downloaded.
110

    
111
.. code-block:: console
112

    
113
    $ git clone https://code.grnet.gr/git/kamaki
114
    $ cd kamaki/kamaki/clients
115

    
116
In each package under kamaki.clients, there is a test module (test.py) where the tests are implemented. To run all tests, run the test.py file from kamaki.clients
117

    
118
.. code-block:: console
119

    
120
    $ python test.py
121

    
122
To test a specific class, add the class name as an argument. E.g. for the Client class:
123

    
124
.. code-block:: console
125

    
126
    $ python test.py Client
127

    
128
To test a specific method in a class, apply an extra argument, e.g. for the request method in the Client class:
129

    
130
.. code-block:: console
131

    
132
    $ python test.py Client request
133

    
134
Each package contains a test module (test.py) which is also runnable from the command line. E.g. the pithos package contains a Pithos test which contains, among others, the download sub-test:
135

    
136
.. code-block:: console
137

    
138
    $ cd pithos
139

    
140
    # Run all kamaki.clients.pithos tests
141
    $ python test.py
142

    
143
    # Run all kamaki.clients.pithos.PithoClient tests
144
    $ python test.py Pithos
145

    
146
    # Test kamaki.clients.pithos.PithosClient.download
147
    $ python test.py Pithos download
148

    
149
Mechanism
150
^^^^^^^^^
151

    
152
Each folder / package contains a test.py file, that represents the test module of this package. All test modules contain a set of classes that extent the TestCase class. They also contain a main method to run the tests.
153

    
154
All classes in the tested package are tested by similarly-named classes in the test module. Methods that do no belong to a class are tested by testing classes named after the module tested methods belong to.
155

    
156
For example, the kamaki.clients.pithos.PithosClient class is tested by the kamaki.clients.pithos.test.Pithos class, which the method in the kamaki.clients.utils module are tested by the kamaki.clients.utils.test.Utils testing class.
157

    
158

    
159
Adding unit tests
160
^^^^^^^^^^^^^^^^^
161
After modifying or extending kamaki.clients method, classes, modules or packages, it is a good practice to also modify or extend the corresponding unit tests. What's more, it is recommended to modify or implement the tests that test the new behavior, before implementing the behavior itself.
162

    
163
Modifying an existing method
164
""""""""""""""""""""""""""""
165

    
166
In case of an existing method modification, the programmer has to modify the corresponding test as well. By convention, the test method is located in the test module under the same package, in a TestCase subclass that is named with a name similar to the package or class that contains the tested method.
167

    
168
Example 1: to modify the kamaki.clients.utils.filter_in method, the programmer has to also adjust the kamaki.clients.utils.test.Utils.test_filter_in method.
169

    
170
Example 2: to modify the kamaki.clients.pithos.PithosRestClient.object_get, the programmer has to also adjust the kamaki.clients.pithos.test.PithosRest.test_object_get method.
171

    
172
Adding a new method
173
"""""""""""""""""""
174

    
175
Programmers who want to implement a new method in an existing class, are encouraged to implement the corresponding unit test first. In order to do that, they should first find the testing class that is mapped to the class they need to extend.
176

    
177
Example 1: To add a **list_special** method to kamaki.clients.astakos.AstakosClient, extend the kamaki.clients.astakos.test.Astakos test class, as shown bellow:
178

    
179
.. code-block:: python
180

    
181
    class Astakos(TestCase):
182
        ...
183
        def test_list_special(self):
184
            """Test the list_special method"""
185
            ...
186

    
187
Example 2: To add a **get_random_int** method in kamaki.clients.utils module, extend the kamaki.clients.utils.test.Utils test class, as shown bellow:
188

    
189
.. code-block:: python
190

    
191
    class Utils(TestCase):
192
        ...
193
        def test_get_random_int(self):
194
            """Test the get_random_int method"""
195
            ...
196

    
197
Implementing a new class or module
198
""""""""""""""""""""""""""""""""""
199

    
200
Each class or module needs a seperate test sub-module. By convention, each class or module under the kamaki.clients should be located in a separate directory.
201

    
202
Example 1: To add a NewService class, that extents the kamaki.clients.Client interface: 
203

    
204
* create a new_service package and implement the unit tests in the kamaki.clients.new_service.test module:
205

    
206
.. code-block:: console
207

    
208
    $ mkdir new_service && touch new_service/test.py
209

    
210
* create the file that will hold the package code and implement the module there:
211

    
212
.. code-block:: console
213

    
214
    $ touch new_service/__init__.py
215

    
216
* Create the test class and methods in kamaki.clients.new_service.test
217

    
218
.. code-block:: python
219

    
220
    from unittest import TestCase
221

    
222
    class NewServiceTest(TestCase):
223

    
224
        def test_method1(self):
225
            ...
226

    
227
* Create the NewService and its actual functionality in kamaki.clients.new_service
228

    
229
.. code-block:: python
230

    
231
    from kamaki.clients import Client
232

    
233
    class NewService(Client):
234

    
235
        def method1(self, ...):
236
            ...
237

    
238
* Expose the new tests to top test module, by importing the test class to kamaki.clients.test
239

    
240
..code-block:: python
241

    
242
    # kamaki.clients.test module
243
    ...
244
    from kamaki.clients.new_service.test import NewService
245

    
246
.. note:: If the new class or module is part of an existing sub-package, it is best to append its testing class in the existing test.py file of the sub-package it belongs to. For example, the kamaki.clients.pithos.PithosClient and kamaki.clients.pithos.rest_api.PithosRestClient classes are tested by two different classes (Pithos and PithosRest) in the same module (kamaki.clients.pithos.test).
247