Revision 7be7673b

/dev/null
1
For Developers
2
==============
3

  
4
Creating applications with kamaki API
5
-------------------------------------
6

  
7
Kamaki features a clients API for building third-party client applications that communicate with OpenStack and / or Synnefo cloud services. The package is called kamaki.clients and contains a number of 
8

  
9
A good example of an application build on kamaki.clients is kamaki.cli, the command line interface of kamaki. 
10

  
11
Since synnefo services are build as OpenStack extensions, an inheritance approach has been chosen for implementing clients for both. In specific, the *compute*, *storage* and *image* modules are clients of the OS compute, OS storage and Glance APIs, respectively. On the contrary, all the other modules are Synnefo extensions (*cyclades* extents *compute*, *pithos* and *pithos_rest_api* extent *storage*) or novel synnefo services (e.g. *astakos*).
12

  
13
Setup a client instance
14
^^^^^^^^^^^^^^^^^^^^^^^
15

  
16
External applications may instantiate one or more kamaki clients.
17

  
18
.. code-block:: python
19
    :emphasize-lines: 1
20

  
21
    Example 1.1: Instantiate a Cyclades client
22

  
23

  
24
    from kamaki.clients.cyclades import CycladesClient
25
    from kamaki.clients.pithos import PithosClient
26

  
27
    my_cyclades_client = CycladesClient(base_url, token)
28
    my_pithos_client = PithosClient(base_url, token, account, container)
29

  
30
.. note:: *cyclades* and *pithos* clients inherit all methods of *compute* and *storage* clients respectively. Separate compute or storage objects should be used only when implementing applications for strict OS Compute or OS Storage services.
31

  
32
Use client methods
33
^^^^^^^^^^^^^^^^^^
34

  
35
Client methods can now be called. Developers are advised to consult :ref:`the-client-api-ref` for details on the available methods and how to use them.
36

  
37
In the following example, the *cyclades* and *pithos* clients of example 1.1 are used to extract some information, that is then printed to the standard output.
38

  
39

  
40
.. code-block:: python
41
    :emphasize-lines: 1,2
42

  
43
    Example 1.2: Print server name and OS for server with server_id
44
                Print objects in container mycont
45

  
46

  
47
    srv = my_cyclades_client.get_server_info(server_id)
48
    print("Server Name: %s (with OS %s" % (srv['name'], srv['os']))
49

  
50
    obj_list = my_pithos_client.list_objects(mycont)
51
    for obj in obj_list:
52
        print('  %s of %s bytes' % (obj['name'], obj['bytes']))
53

  
54
.. code-block:: console
55
    :emphasize-lines: 1
56

  
57
    Run of examples 1.1 + 1.2
58

  
59

  
60
    $ python test_script.py
61
    Server Name: A Debian Server (with OS Debian Base)
62
      lala.txt of 34 bytes
63
      test.txt of 1232 bytes
64
      testDir/ of 0 bytes
65
    $ 
66

  
67
Error handling
68
^^^^^^^^^^^^^^
69

  
70
The kamaki.clients standard error is ClientError. A ClientError is raised for any kind of kamaki.clients errors (errors reported by servers, type errors in arguments, etc.).
71

  
72
A ClientError contains::
73

  
74
    message     The error message.
75
    status      An optional error code, e.g. after a server error.
76
    details     Optional list of messages with error details.
77

  
78
The following example concatenates examples 1.1 and 1.2 plus error handling
79

  
80
.. code-block:: python
81

  
82
    Example 1.3: Error handling
83

  
84

  
85
    from kamaki.clients.cyclades import CycladesClient
86
    from kamaki.clients.pithos import PithosClient
87

  
88
    try:
89
        my_cyclades_client = CycladesClient(base_url, token)
90
    except ClientError:
91
        print('Failed to initialize Cyclades client')
92

  
93
    try:
94
        my_pithos_client = PithosClient(base_url, token, account, container)
95
    except ClientError:
96
        print('Failed to initialize Pithos+ client')
97

  
98
    try:
99
        srv = my_cyclades_client.get_server_info(server_id)
100
        print("Server Name: %s (with OS %s" % (srv['name'], srv['os']))
101

  
102
        obj_list = my_pithos_client.list_objects(mycont)
103
        for obj in obj_list:
104
            print('  %s of %s bytes' % (obj['name'], obj['bytes']))
105
    except ClientError as e:
106
        print('Error: %s' % e)
107
        if e.status:
108
            print('- error code: %s' % e.status)
109
        if e.details:
110
            for detail in e.details:
111
                print('- %s' % detail)
112

  
113
Extending kamaki.clients
114
------------------------
115

  
116
By default, kamaki clients are REST clients (they manage HTTP requests and responses to communicate with services). This is achieved by importing the connection module, which is an httplib rapper.
117

  
118
Connection
119
^^^^^^^^^^
120

  
121
The connection module features an error handling and logging system, a lazy response mechanism, a pooling mechanism as well as concurrency control for thread-demanding client functions (e.g. store upload).
122

  
123
How to build a client
124
^^^^^^^^^^^^^^^^^^^^^
125

  
126
All service clients consist of a subclass of the Client class and implement separate client functionalities as member methods. There is also an error class to raise exceptions that can be handled by kamaki interfaces.
127

  
128
.. code-block:: python
129
    
130
    #  ${KAMAKI_PATH}/kamaki/clients/mynewclient.py
131

  
132
    from kamaki.clients import Client, ClientError
133

  
134
    class MyNewClient(Client):
135
        """MyNewClient Description Here"""
136

  
137
        def my_first_method(self, **args):
138
            """Method description"""
139
            try:
140
                ...
141
                method code
142
                ...
143
            except SomeKnownException as e1:
144
                raise ClientError('MyError: %s' % e1)
145
            except SomeOtherException as e2:
146
                raise ClientError('MyError: %s' % e2)
147

  
148
        def my_second_method(self, **params):
149
            """Method description"""
150
            ...
151

  
152
Custom clients can use a set of convenience methods for easy HTTP requests
153

  
154
.. code-block:: python
155

  
156
    def get(self, path, **kwargs)
157
    def head(self, path, **kwargs)
158
    def post(self, path, **kwargs)
159
    def put(self, path, **kwargs)
160
    def delete(self, path, **kwargs)
161
    def copy(self, path, **kwargs)
162
    def move(self, path, **kwargs)
163

  
164
How to use your client
165
^^^^^^^^^^^^^^^^^^^^^^
166

  
167
External applications must instantiate a MyNewClient object.
168

  
169
.. code-block:: python
170

  
171
    from kamaki.clients import ClientError
172
    from kamaki.clients.mynewclient import MyNewClient
173

  
174
    ...
175
    try:
176
        cl = MyNewClient(args)
177
        cl.my_first_method(other_args)
178
    except ClientError as cle:
179
        print('Client Error: %s' % cle)
180
    ...
181

  
182
Concurrency control
183
^^^^^^^^^^^^^^^^^^^
184

  
185
Kamaki clients may handle multiple requests at once, using threads. In that case, users might implement their own thread handling mechanism, use an external solution or take advantage of the mechanism featured in kamaki.clients
186

  
187
.. code-block:: python
188

  
189
    from threading import enumerate
190
    from kamaki.clients import SilentEvent
191
    ...
192

  
193
    class MyNewClient(Client):
194
        ...
195

  
196
        def _single_threaded_method(self, **args):
197
            ...
198
            request code
199
            ...
200

  
201
        def multithread_method(self):
202
            thread_list = []
203
            self._init_thread_limit()
204
            while some_condition or thread_list:
205
                ...
206
                event = SilentEvent(self._single_threaded_method, **args)
207
                event.start()
208
                thread_list.append(event)
209
                thread_list = self._watch_thread_limit(thread_list)
210

  
211
The CLI API
212
-----------
213

  
214
.. toctree::
215

  
216
    cli
217

  
218
.. _the-client-api-ref:
219

  
220
The clients API
221
---------------
222

  
223
Imports
224
^^^^^^^
225

  
226
.. toctree::
227
    connection
228

  
229
Modules list
230
^^^^^^^^^^^^
231

  
232
compute
233
^^^^^^^
234

  
235
.. automodule:: kamaki.clients.compute
236
    :members:
237
    :show-inheritance:
238
    :undoc-members:
239

  
240

  
241
cyclades
242
^^^^^^^^
243

  
244
.. automodule:: kamaki.clients.cyclades
245
    :members:
246
    :show-inheritance:
247
    :undoc-members:
248

  
249

  
250
storage
251
^^^^^^^
252

  
253
.. automodule:: kamaki.clients.storage
254
    :members:
255
    :show-inheritance:
256
    :undoc-members:
257

  
258

  
259
pithos
260
^^^^^^
261

  
262
.. automodule:: kamaki.clients.pithos
263
    :members:
264
    :show-inheritance:
265
    :undoc-members:
266

  
267
pithos_rest_api
268
^^^^^^^^^^^^^^^
269

  
270
.. automodule:: kamaki.clients.pithos_rest_api
271
    :members:
272
    :show-inheritance:
273
    :undoc-members:
274

  
275

  
276
image
277
^^^^^
278

  
279
.. automodule:: kamaki.clients.image
280
    :members:
281
    :show-inheritance:
282
    :undoc-members:
283

  
284

  
285
astakos
286
^^^^^^^
287

  
288
.. automodule:: kamaki.clients.astakos
289
    :members:
290
    :show-inheritance:
291
    :undoc-members:
292

  
293

  
294
utils
295
^^^^^
296

  
297
.. automodule:: kamaki.clients.utils
298
    :members:
299
    :show-inheritance:
300
    :undoc-members:
b/docs/developers.rst
1
For Developers
2
==============
3

  
4
Creating applications with kamaki API
5
-------------------------------------
6

  
7
Kamaki features a clients API for building third-party client applications that communicate with OpenStack and / or Synnefo cloud services. The package is called kamaki.clients and contains a number of 
8

  
9
A good example of an application build on kamaki.clients is kamaki.cli, the command line interface of kamaki. 
10

  
11
Since synnefo services are build as OpenStack extensions, an inheritance approach has been chosen for implementing clients for both. In specific, the *compute*, *storage* and *image* modules are clients of the OS compute, OS storage and Glance APIs, respectively. On the contrary, all the other modules are Synnefo extensions (*cyclades* extents *compute*, *pithos* and *pithos_rest_api* extent *storage*) or novel synnefo services (e.g. *astakos*).
12

  
13
Setup a client instance
14
^^^^^^^^^^^^^^^^^^^^^^^
15

  
16
External applications may instantiate one or more kamaki clients.
17

  
18
.. code-block:: python
19
    :emphasize-lines: 1
20

  
21
    Example 1.1: Instantiate a Cyclades client
22

  
23

  
24
    from kamaki.clients.cyclades import CycladesClient
25
    from kamaki.clients.pithos import PithosClient
26

  
27
    my_cyclades_client = CycladesClient(base_url, token)
28
    my_pithos_client = PithosClient(base_url, token, account, container)
29

  
30
.. note:: *cyclades* and *pithos* clients inherit all methods of *compute* and *storage* clients respectively. Separate compute or storage objects should be used only when implementing applications for strict OS Compute or OS Storage services.
31

  
32
Use client methods
33
^^^^^^^^^^^^^^^^^^
34

  
35
Client methods can now be called. Developers are advised to consult :ref:`the-client-api-ref` for details on the available methods and how to use them.
36

  
37
In the following example, the *cyclades* and *pithos* clients of example 1.1 are used to extract some information, that is then printed to the standard output.
38

  
39

  
40
.. code-block:: python
41
    :emphasize-lines: 1,2
42

  
43
    Example 1.2: Print server name and OS for server with server_id
44
                Print objects in container mycont
45

  
46

  
47
    srv = my_cyclades_client.get_server_info(server_id)
48
    print("Server Name: %s (with OS %s" % (srv['name'], srv['os']))
49

  
50
    obj_list = my_pithos_client.list_objects(mycont)
51
    for obj in obj_list:
52
        print('  %s of %s bytes' % (obj['name'], obj['bytes']))
53

  
54
.. code-block:: console
55
    :emphasize-lines: 1
56

  
57
    Run of examples 1.1 + 1.2
58

  
59

  
60
    $ python test_script.py
61
    Server Name: A Debian Server (with OS Debian Base)
62
      lala.txt of 34 bytes
63
      test.txt of 1232 bytes
64
      testDir/ of 0 bytes
65
    $ 
66

  
67
Error handling
68
^^^^^^^^^^^^^^
69

  
70
The kamaki.clients standard error is ClientError. A ClientError is raised for any kind of kamaki.clients errors (errors reported by servers, type errors in arguments, etc.).
71

  
72
A ClientError contains::
73

  
74
    message     The error message.
75
    status      An optional error code, e.g. after a server error.
76
    details     Optional list of messages with error details.
77

  
78
The following example concatenates examples 1.1 and 1.2 plus error handling
79

  
80
.. code-block:: python
81

  
82
    Example 1.3: Error handling
83

  
84

  
85
    from kamaki.clients.cyclades import CycladesClient
86
    from kamaki.clients.pithos import PithosClient
87

  
88
    try:
89
        my_cyclades_client = CycladesClient(base_url, token)
90
    except ClientError:
91
        print('Failed to initialize Cyclades client')
92

  
93
    try:
94
        my_pithos_client = PithosClient(base_url, token, account, container)
95
    except ClientError:
96
        print('Failed to initialize Pithos+ client')
97

  
98
    try:
99
        srv = my_cyclades_client.get_server_info(server_id)
100
        print("Server Name: %s (with OS %s" % (srv['name'], srv['os']))
101

  
102
        obj_list = my_pithos_client.list_objects(mycont)
103
        for obj in obj_list:
104
            print('  %s of %s bytes' % (obj['name'], obj['bytes']))
105
    except ClientError as e:
106
        print('Error: %s' % e)
107
        if e.status:
108
            print('- error code: %s' % e.status)
109
        if e.details:
110
            for detail in e.details:
111
                print('- %s' % detail)
112

  
113
Adding Commands
114
---------------
115

  
116
Architecture
117
^^^^^^^^^^^^
118

  
119
Kamaki commands are implemented as python classes, decorated with a special decorator called *command*. This decorator is a method of kamaki.cli that adds a new command in a CommandTree structure (kamaki.cli.commant_tree). The later is used by interfaces to manage kamaki commands.
120

  
121
The CommandTree structure
122
"""""""""""""""""""""""""
123

  
124
CommandTree manages a command by its path. Each command is stored in multiple nodes on the tree, so that the last term is a leaf and the route from root to that leaf represents the command path. For example the commands *store upload*, *store list* and *store info* are stored together as shown bellow::
125

  
126
    - store
127
    ''''''''|- info
128
            |- list
129
            |- upload
130

  
131
Each command group should be stored on a different CommandTree. For that reason, command specification modules should contain a list of CommandTree objects, named *_commands*
132

  
133
A command group information (name, description) is provided at CommandTree structure initialization::
134

  
135
    _mygrp_commands = CommandTree('mygrp', 'My Group Commands')
136

  
137
The command decorator
138
"""""""""""""""""""""
139

  
140
The *command* decorator mines all the information necessary to build a command specification which is inserted in a CommanTree instance::
141

  
142
    class code  --->  command()  -->  updated CommandTree structure
143

  
144
Kamaki interfaces make use of this CommandTree structure. Optimizations are possible by using special parameters on the command decorator method.
145

  
146
.. code-block:: python
147

  
148
    def command(cmd_tree, prefix='', descedants_depth=None):
149
    """Load a class as a command
150
        @cmd_tree is the CommandTree to be updated with a new command
151
        @prefix of the commands allowed to be inserted ('' for all)
152
        @descedants_depth is the depth of the tree descedants of the
153
            prefix command.
154
    """
155

  
156
Creating a new command specification set
157
""""""""""""""""""""""""""""""""""""""""
158

  
159
A command specification developer should create a new module (python file) with as many classes as the command specifications to be offered. Each class should be decorated with *command*.
160

  
161
A list of CommandTree structures must exist in the module scope, with the name _commands. Different CommandTree objects correspond to different command groups.
162

  
163
Declare run-time argument
164
"""""""""""""""""""""""""
165

  
166
The argument mechanism allows the definition of run-time arguments. Some basic argument types are defined at the `argument module <cli.html#module-kamaki.cli.argument>`_, but it is not uncommon to extent these classes in order to achieve specialized type checking and syntax control (e.g. at `pithos_cli module <cli.html#module-kamaki.cli.commands.pithos_cli>`_).
167

  
168
Putting them all together
169
"""""""""""""""""""""""""
170

  
171
The information that can be mined by *command* for each individual command are presented in the following, for a sample command:
172

  
173
.. code-block:: python
174

  
175
    @command(_mygrp_commands)
176
    class mygrp_cmd1_cmd2(object):
177
        """Command Description"""
178

  
179
        def __init__(self, arguments={}):
180
            arguments['arg1'] = FlagArgument(
181
                'Run with arg1 on',
182
                --arg1',
183
                False)
184

  
185
            self.arguments = arguments
186

  
187
        def main(self, obligatory_param1, obligatory_param2,
188
            optional_param1=None, optional_param2=None):
189
            ...
190
            command code here
191
            ...
192

  
193
This will load the following information on the CommandTree::
194

  
195
    Syntax: mygrp cmd1 cmd2 <objigatory param1> <obligatory param2>
196
        [optional_param1] [optional_param2] [--arg1]
197
    Description: Command Description
198
    Arguments help: --arg1: Run with arg1 on
199

  
200
Letting kamaki know
201
"""""""""""""""""""
202

  
203
Kamaki will load a command specification *only* if it is set as a configurable option. To demonstrate this, let a commands module grps be implemented in the file *grps.py* where there are two command groups defined: mygrp1 mygrp2.
204

  
205
Move grps.py to kamaki/cli/commands, the default place for command specifications
206

  
207
In the configuration file, add:
208

  
209
.. code-block:: console
210

  
211
    [mygrp1]
212
    cli=grps
213

  
214
    [mygrp2]
215
    cli=grps
216

  
217
or alternatively
218

  
219
.. code-block:: console
220

  
221
    $ kamaki config set mygrp1.cli = grps
222
    $ kamaki config set mygrp2.cli = grps
223

  
224
Command modules don't have to live in kamaki/cli/commands, although this is suggested for uniformity. If a command module exist in another path::
225

  
226
    [mygrp]
227
    cli=/another/path/grps.py
228

  
229
Developing a Command Specification Set
230
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
231

  
232
TODO
233

  
234
Extending kamaki.clients
235
------------------------
236

  
237
By default, kamaki clients are REST clients (they manage HTTP requests and responses to communicate with services). This is achieved by importing the connection module, which is an httplib rapper.
238

  
239
Connection
240
^^^^^^^^^^
241

  
242
The connection module features an error handling and logging system, a lazy response mechanism, a pooling mechanism as well as concurrency control for thread-demanding client functions (e.g. store upload).
243

  
244
How to build a client
245
^^^^^^^^^^^^^^^^^^^^^
246

  
247
All service clients consist of a subclass of the Client class and implement separate client functionalities as member methods. There is also an error class to raise exceptions that can be handled by kamaki interfaces.
248

  
249
.. code-block:: python
250
    
251
    #  ${KAMAKI_PATH}/kamaki/clients/mynewclient.py
252

  
253
    from kamaki.clients import Client, ClientError
254

  
255
    class MyNewClient(Client):
256
        """MyNewClient Description Here"""
257

  
258
        def my_first_method(self, **args):
259
            """Method description"""
260
            try:
261
                ...
262
                method code
263
                ...
264
            except SomeKnownException as e1:
265
                raise ClientError('MyError: %s' % e1)
266
            except SomeOtherException as e2:
267
                raise ClientError('MyError: %s' % e2)
268

  
269
        def my_second_method(self, **params):
270
            """Method description"""
271
            ...
272

  
273
Custom clients can use a set of convenience methods for easy HTTP requests
274

  
275
.. code-block:: python
276

  
277
    def get(self, path, **kwargs)
278
    def head(self, path, **kwargs)
279
    def post(self, path, **kwargs)
280
    def put(self, path, **kwargs)
281
    def delete(self, path, **kwargs)
282
    def copy(self, path, **kwargs)
283
    def move(self, path, **kwargs)
284

  
285
How to use your client
286
^^^^^^^^^^^^^^^^^^^^^^
287

  
288
External applications must instantiate a MyNewClient object.
289

  
290
.. code-block:: python
291

  
292
    from kamaki.clients import ClientError
293
    from kamaki.clients.mynewclient import MyNewClient
294

  
295
    ...
296
    try:
297
        cl = MyNewClient(args)
298
        cl.my_first_method(other_args)
299
    except ClientError as cle:
300
        print('Client Error: %s' % cle)
301
    ...
302

  
303
Concurrency control
304
^^^^^^^^^^^^^^^^^^^
305

  
306
Kamaki clients may handle multiple requests at once, using threads. In that case, users might implement their own thread handling mechanism, use an external solution or take advantage of the mechanism featured in kamaki.clients
307

  
308
.. code-block:: python
309

  
310
    from threading import enumerate
311
    from kamaki.clients import SilentEvent
312
    ...
313

  
314
    class MyNewClient(Client):
315
        ...
316

  
317
        def _single_threaded_method(self, **args):
318
            ...
319
            request code
320
            ...
321

  
322
        def multithread_method(self):
323
            thread_list = []
324
            self._init_thread_limit()
325
            while some_condition or thread_list:
326
                ...
327
                event = SilentEvent(self._single_threaded_method, **args)
328
                event.start()
329
                thread_list.append(event)
330
                thread_list = self._watch_thread_limit(thread_list)
331

  
332
The CLI API
333
-----------
334

  
335
.. toctree::
336

  
337
    cli
338

  
339
.. _the-client-api-ref:
340

  
341
The clients API
342
---------------
343

  
344
Imports
345
^^^^^^^
346

  
347
.. toctree::
348
    connection
349

  
350
Modules list
351
^^^^^^^^^^^^
352

  
353
compute
354
^^^^^^^
355

  
356
.. automodule:: kamaki.clients.compute
357
    :members:
358
    :show-inheritance:
359
    :undoc-members:
360

  
361

  
362
cyclades
363
^^^^^^^^
364

  
365
.. automodule:: kamaki.clients.cyclades
366
    :members:
367
    :show-inheritance:
368
    :undoc-members:
369

  
370

  
371
storage
372
^^^^^^^
373

  
374
.. automodule:: kamaki.clients.storage
375
    :members:
376
    :show-inheritance:
377
    :undoc-members:
378

  
379

  
380
pithos
381
^^^^^^
382

  
383
.. automodule:: kamaki.clients.pithos
384
    :members:
385
    :show-inheritance:
386
    :undoc-members:
387

  
388
pithos_rest_api
389
^^^^^^^^^^^^^^^
390

  
391
.. automodule:: kamaki.clients.pithos_rest_api
392
    :members:
393
    :show-inheritance:
394
    :undoc-members:
395

  
396

  
397
image
398
^^^^^
399

  
400
.. automodule:: kamaki.clients.image
401
    :members:
402
    :show-inheritance:
403
    :undoc-members:
404

  
405

  
406
astakos
407
^^^^^^^
408

  
409
.. automodule:: kamaki.clients.astakos
410
    :members:
411
    :show-inheritance:
412
    :undoc-members:
413

  
414

  
415
utils
416
^^^^^
417

  
418
.. automodule:: kamaki.clients.utils
419
    :members:
420
    :show-inheritance:
421
    :undoc-members:
b/docs/index.rst
28 28
   setup
29 29
   usage
30 30
   commands
31
   clients
31
   developers
32 32

  
33 33

  
34 34
Indices and tables

Also available in: Unified diff