Statistics
| Branch: | Tag: | Revision:

root / docs / developers.rst @ 7be7673b

History | View | Annotate | Download (12.6 kB)

1
For Developers
2
==============
3

    
4
Creating applications with kamaki API
5
-------------------------------------
6

    
7
Kamaki features a clients API for building third-party client applications that communicate with OpenStack and / or Synnefo cloud services. The package is called kamaki.clients and contains a number of 
8

    
9
A good example of an application build on kamaki.clients is kamaki.cli, the command line interface of kamaki. 
10

    
11
Since synnefo services are build as OpenStack extensions, an inheritance approach has been chosen for implementing clients for both. In specific, the *compute*, *storage* and *image* modules are clients of the OS compute, OS storage and Glance APIs, respectively. On the contrary, all the other modules are Synnefo extensions (*cyclades* extents *compute*, *pithos* and *pithos_rest_api* extent *storage*) or novel synnefo services (e.g. *astakos*).
12

    
13
Setup a client instance
14
^^^^^^^^^^^^^^^^^^^^^^^
15

    
16
External applications may instantiate one or more kamaki clients.
17

    
18
.. code-block:: python
19
    :emphasize-lines: 1
20

    
21
    Example 1.1: Instantiate a Cyclades client
22

    
23

    
24
    from kamaki.clients.cyclades import CycladesClient
25
    from kamaki.clients.pithos import PithosClient
26

    
27
    my_cyclades_client = CycladesClient(base_url, token)
28
    my_pithos_client = PithosClient(base_url, token, account, container)
29

    
30
.. note:: *cyclades* and *pithos* clients inherit all methods of *compute* and *storage* clients respectively. Separate compute or storage objects should be used only when implementing applications for strict OS Compute or OS Storage services.
31

    
32
Use client methods
33
^^^^^^^^^^^^^^^^^^
34

    
35
Client methods can now be called. Developers are advised to consult :ref:`the-client-api-ref` for details on the available methods and how to use them.
36

    
37
In the following example, the *cyclades* and *pithos* clients of example 1.1 are used to extract some information, that is then printed to the standard output.
38

    
39

    
40
.. code-block:: python
41
    :emphasize-lines: 1,2
42

    
43
    Example 1.2: Print server name and OS for server with server_id
44
                Print objects in container mycont
45

    
46

    
47
    srv = my_cyclades_client.get_server_info(server_id)
48
    print("Server Name: %s (with OS %s" % (srv['name'], srv['os']))
49

    
50
    obj_list = my_pithos_client.list_objects(mycont)
51
    for obj in obj_list:
52
        print('  %s of %s bytes' % (obj['name'], obj['bytes']))
53

    
54
.. code-block:: console
55
    :emphasize-lines: 1
56

    
57
    Run of examples 1.1 + 1.2
58

    
59

    
60
    $ python test_script.py
61
    Server Name: A Debian Server (with OS Debian Base)
62
      lala.txt of 34 bytes
63
      test.txt of 1232 bytes
64
      testDir/ of 0 bytes
65
    $ 
66

    
67
Error handling
68
^^^^^^^^^^^^^^
69

    
70
The kamaki.clients standard error is ClientError. A ClientError is raised for any kind of kamaki.clients errors (errors reported by servers, type errors in arguments, etc.).
71

    
72
A ClientError contains::
73

    
74
    message     The error message.
75
    status      An optional error code, e.g. after a server error.
76
    details     Optional list of messages with error details.
77

    
78
The following example concatenates examples 1.1 and 1.2 plus error handling
79

    
80
.. code-block:: python
81

    
82
    Example 1.3: Error handling
83

    
84

    
85
    from kamaki.clients.cyclades import CycladesClient
86
    from kamaki.clients.pithos import PithosClient
87

    
88
    try:
89
        my_cyclades_client = CycladesClient(base_url, token)
90
    except ClientError:
91
        print('Failed to initialize Cyclades client')
92

    
93
    try:
94
        my_pithos_client = PithosClient(base_url, token, account, container)
95
    except ClientError:
96
        print('Failed to initialize Pithos+ client')
97

    
98
    try:
99
        srv = my_cyclades_client.get_server_info(server_id)
100
        print("Server Name: %s (with OS %s" % (srv['name'], srv['os']))
101

    
102
        obj_list = my_pithos_client.list_objects(mycont)
103
        for obj in obj_list:
104
            print('  %s of %s bytes' % (obj['name'], obj['bytes']))
105
    except ClientError as e:
106
        print('Error: %s' % e)
107
        if e.status:
108
            print('- error code: %s' % e.status)
109
        if e.details:
110
            for detail in e.details:
111
                print('- %s' % detail)
112

    
113
Adding Commands
114
---------------
115

    
116
Architecture
117
^^^^^^^^^^^^
118

    
119
Kamaki commands are implemented as python classes, decorated with a special decorator called *command*. This decorator is a method of kamaki.cli that adds a new command in a CommandTree structure (kamaki.cli.commant_tree). The later is used by interfaces to manage kamaki commands.
120

    
121
The CommandTree structure
122
"""""""""""""""""""""""""
123

    
124
CommandTree manages a command by its path. Each command is stored in multiple nodes on the tree, so that the last term is a leaf and the route from root to that leaf represents the command path. For example the commands *store upload*, *store list* and *store info* are stored together as shown bellow::
125

    
126
    - store
127
    ''''''''|- info
128
            |- list
129
            |- upload
130

    
131
Each command group should be stored on a different CommandTree. For that reason, command specification modules should contain a list of CommandTree objects, named *_commands*
132

    
133
A command group information (name, description) is provided at CommandTree structure initialization::
134

    
135
    _mygrp_commands = CommandTree('mygrp', 'My Group Commands')
136

    
137
The command decorator
138
"""""""""""""""""""""
139

    
140
The *command* decorator mines all the information necessary to build a command specification which is inserted in a CommanTree instance::
141

    
142
    class code  --->  command()  -->  updated CommandTree structure
143

    
144
Kamaki interfaces make use of this CommandTree structure. Optimizations are possible by using special parameters on the command decorator method.
145

    
146
.. code-block:: python
147

    
148
    def command(cmd_tree, prefix='', descedants_depth=None):
149
    """Load a class as a command
150
        @cmd_tree is the CommandTree to be updated with a new command
151
        @prefix of the commands allowed to be inserted ('' for all)
152
        @descedants_depth is the depth of the tree descedants of the
153
            prefix command.
154
    """
155

    
156
Creating a new command specification set
157
""""""""""""""""""""""""""""""""""""""""
158

    
159
A command specification developer should create a new module (python file) with as many classes as the command specifications to be offered. Each class should be decorated with *command*.
160

    
161
A list of CommandTree structures must exist in the module scope, with the name _commands. Different CommandTree objects correspond to different command groups.
162

    
163
Declare run-time argument
164
"""""""""""""""""""""""""
165

    
166
The argument mechanism allows the definition of run-time arguments. Some basic argument types are defined at the `argument module <cli.html#module-kamaki.cli.argument>`_, but it is not uncommon to extent these classes in order to achieve specialized type checking and syntax control (e.g. at `pithos_cli module <cli.html#module-kamaki.cli.commands.pithos_cli>`_).
167

    
168
Putting them all together
169
"""""""""""""""""""""""""
170

    
171
The information that can be mined by *command* for each individual command are presented in the following, for a sample command:
172

    
173
.. code-block:: python
174

    
175
    @command(_mygrp_commands)
176
    class mygrp_cmd1_cmd2(object):
177
        """Command Description"""
178

    
179
        def __init__(self, arguments={}):
180
            arguments['arg1'] = FlagArgument(
181
                'Run with arg1 on',
182
                --arg1',
183
                False)
184

    
185
            self.arguments = arguments
186

    
187
        def main(self, obligatory_param1, obligatory_param2,
188
            optional_param1=None, optional_param2=None):
189
            ...
190
            command code here
191
            ...
192

    
193
This will load the following information on the CommandTree::
194

    
195
    Syntax: mygrp cmd1 cmd2 <objigatory param1> <obligatory param2>
196
        [optional_param1] [optional_param2] [--arg1]
197
    Description: Command Description
198
    Arguments help: --arg1: Run with arg1 on
199

    
200
Letting kamaki know
201
"""""""""""""""""""
202

    
203
Kamaki will load a command specification *only* if it is set as a configurable option. To demonstrate this, let a commands module grps be implemented in the file *grps.py* where there are two command groups defined: mygrp1 mygrp2.
204

    
205
Move grps.py to kamaki/cli/commands, the default place for command specifications
206

    
207
In the configuration file, add:
208

    
209
.. code-block:: console
210

    
211
    [mygrp1]
212
    cli=grps
213

    
214
    [mygrp2]
215
    cli=grps
216

    
217
or alternatively
218

    
219
.. code-block:: console
220

    
221
    $ kamaki config set mygrp1.cli = grps
222
    $ kamaki config set mygrp2.cli = grps
223

    
224
Command modules don't have to live in kamaki/cli/commands, although this is suggested for uniformity. If a command module exist in another path::
225

    
226
    [mygrp]
227
    cli=/another/path/grps.py
228

    
229
Developing a Command Specification Set
230
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
231

    
232
TODO
233

    
234
Extending kamaki.clients
235
------------------------
236

    
237
By default, kamaki clients are REST clients (they manage HTTP requests and responses to communicate with services). This is achieved by importing the connection module, which is an httplib rapper.
238

    
239
Connection
240
^^^^^^^^^^
241

    
242
The connection module features an error handling and logging system, a lazy response mechanism, a pooling mechanism as well as concurrency control for thread-demanding client functions (e.g. store upload).
243

    
244
How to build a client
245
^^^^^^^^^^^^^^^^^^^^^
246

    
247
All service clients consist of a subclass of the Client class and implement separate client functionalities as member methods. There is also an error class to raise exceptions that can be handled by kamaki interfaces.
248

    
249
.. code-block:: python
250
    
251
    #  ${KAMAKI_PATH}/kamaki/clients/mynewclient.py
252

    
253
    from kamaki.clients import Client, ClientError
254

    
255
    class MyNewClient(Client):
256
        """MyNewClient Description Here"""
257

    
258
        def my_first_method(self, **args):
259
            """Method description"""
260
            try:
261
                ...
262
                method code
263
                ...
264
            except SomeKnownException as e1:
265
                raise ClientError('MyError: %s' % e1)
266
            except SomeOtherException as e2:
267
                raise ClientError('MyError: %s' % e2)
268

    
269
        def my_second_method(self, **params):
270
            """Method description"""
271
            ...
272

    
273
Custom clients can use a set of convenience methods for easy HTTP requests
274

    
275
.. code-block:: python
276

    
277
    def get(self, path, **kwargs)
278
    def head(self, path, **kwargs)
279
    def post(self, path, **kwargs)
280
    def put(self, path, **kwargs)
281
    def delete(self, path, **kwargs)
282
    def copy(self, path, **kwargs)
283
    def move(self, path, **kwargs)
284

    
285
How to use your client
286
^^^^^^^^^^^^^^^^^^^^^^
287

    
288
External applications must instantiate a MyNewClient object.
289

    
290
.. code-block:: python
291

    
292
    from kamaki.clients import ClientError
293
    from kamaki.clients.mynewclient import MyNewClient
294

    
295
    ...
296
    try:
297
        cl = MyNewClient(args)
298
        cl.my_first_method(other_args)
299
    except ClientError as cle:
300
        print('Client Error: %s' % cle)
301
    ...
302

    
303
Concurrency control
304
^^^^^^^^^^^^^^^^^^^
305

    
306
Kamaki clients may handle multiple requests at once, using threads. In that case, users might implement their own thread handling mechanism, use an external solution or take advantage of the mechanism featured in kamaki.clients
307

    
308
.. code-block:: python
309

    
310
    from threading import enumerate
311
    from kamaki.clients import SilentEvent
312
    ...
313

    
314
    class MyNewClient(Client):
315
        ...
316

    
317
        def _single_threaded_method(self, **args):
318
            ...
319
            request code
320
            ...
321

    
322
        def multithread_method(self):
323
            thread_list = []
324
            self._init_thread_limit()
325
            while some_condition or thread_list:
326
                ...
327
                event = SilentEvent(self._single_threaded_method, **args)
328
                event.start()
329
                thread_list.append(event)
330
                thread_list = self._watch_thread_limit(thread_list)
331

    
332
The CLI API
333
-----------
334

    
335
.. toctree::
336

    
337
    cli
338

    
339
.. _the-client-api-ref:
340

    
341
The clients API
342
---------------
343

    
344
Imports
345
^^^^^^^
346

    
347
.. toctree::
348
    connection
349

    
350
Modules list
351
^^^^^^^^^^^^
352

    
353
compute
354
^^^^^^^
355

    
356
.. automodule:: kamaki.clients.compute
357
    :members:
358
    :show-inheritance:
359
    :undoc-members:
360

    
361

    
362
cyclades
363
^^^^^^^^
364

    
365
.. automodule:: kamaki.clients.cyclades
366
    :members:
367
    :show-inheritance:
368
    :undoc-members:
369

    
370

    
371
storage
372
^^^^^^^
373

    
374
.. automodule:: kamaki.clients.storage
375
    :members:
376
    :show-inheritance:
377
    :undoc-members:
378

    
379

    
380
pithos
381
^^^^^^
382

    
383
.. automodule:: kamaki.clients.pithos
384
    :members:
385
    :show-inheritance:
386
    :undoc-members:
387

    
388
pithos_rest_api
389
^^^^^^^^^^^^^^^
390

    
391
.. automodule:: kamaki.clients.pithos_rest_api
392
    :members:
393
    :show-inheritance:
394
    :undoc-members:
395

    
396

    
397
image
398
^^^^^
399

    
400
.. automodule:: kamaki.clients.image
401
    :members:
402
    :show-inheritance:
403
    :undoc-members:
404

    
405

    
406
astakos
407
^^^^^^^
408

    
409
.. automodule:: kamaki.clients.astakos
410
    :members:
411
    :show-inheritance:
412
    :undoc-members:
413

    
414

    
415
utils
416
^^^^^
417

    
418
.. automodule:: kamaki.clients.utils
419
    :members:
420
    :show-inheritance:
421
    :undoc-members: