commit 961e60407e66be1e9db8df1fb36509268bcff888
Author: Stavros Sachtouris <saxtouri@admin.grnet.gr>
Date:   Wed Nov 28 16:54:26 2012 +0200

    Rearrange sections in docs, add developer corner

diff --git a/docs/cli.rst b/docs/cli.rst
index 303354e..288f3e2 100644
--- a/docs/cli.rst
+++ b/docs/cli.rst
@@ -1,4 +1,52 @@
-Command Line Interfaces
+Command Specifications
+======================
+
+astakos
+-------
+
+.. automodule:: kamaki.cli.commands.astakos_cli
+    :members:
+    :undoc-members:
+
+cyclades (server, flavor, network)
+----------------------------------
+
+.. automodule:: kamaki.cli.commands.cyclades_cli
+    :members:
+    :undoc-members:
+
+pithos (store)
+--------------
+
+.. automodule:: kamaki.cli.commands.pithos_cli
+    :members:
+    :undoc-members:
+
+image
+-----
+
+.. automodule:: kamaki.cli.commands.image_cli
+    :members:
+    :undoc-members:
+
+Kamaki commands
+---------------
+
+config
+^^^^^^
+
+.. automodule:: kamaki.cli.commands.config_cli
+    :members:
+    :undoc-members:
+
+history
+^^^^^^^
+
+.. automodule:: kamaki.cli.commands.history_cli
+    :members:
+    :undoc-members:
+
+Command Line Interface Code
 ===========================
 
 argument
diff --git a/docs/clients.rst b/docs/clients.rst
index a76de5a..45ee6df 100644
--- a/docs/clients.rst
+++ b/docs/clients.rst
@@ -1,20 +1,233 @@
-Clients lib
-===========
+For Developers
+==============
 
-Creating applications with the clients API
-------------------------------------------
+Creating applications with kamaki API
+-------------------------------------
 
-Please do!
+Kamaki features a clients API for building third-party client applications that communicate with OpenStack and / or Synnefo cloud services. The package is called kamaki.clients and contains a number of 
+
+A good example of an application build on kamaki.clients is kamaki.cli, the command line interface of kamaki. 
+
+Since synnefo services are build as OpenStack extensions, an inheritance approach has been chosen for implementing clients for both. In specific, the *compute*, *storage* and *image* modules are clients of the OS compute, OS storage and Glance APIs, respectively. On the contrary, all the other modules are Synnefo extensions (*cyclades* extents *compute*, *pithos* and *pithos_rest_api* extent *storage*) or novel synnefo services (e.g. *astakos*).
+
+Setup a client instance
+^^^^^^^^^^^^^^^^^^^^^^^
+
+External applications may instantiate one or more kamaki clients.
+
+.. code-block:: python
+    :emphasize-lines: 1
+
+    Example 1.1: Instantiate a Cyclades client
+
+
+    from kamaki.clients.cyclades import CycladesClient
+    from kamaki.clients.pithos import PithosClient
+
+    my_cyclades_client = CycladesClient(base_url, token)
+    my_pithos_client = PithosClient(base_url, token, account, container)
+
+.. note:: *cyclades* and *pithos* clients inherit all methods of *compute* and *storage* clients respectively. Separate compute or storage objects should be used only when implementing applications for strict OS Compute or OS Storage services.
+
+Use client methods
+^^^^^^^^^^^^^^^^^^
+
+Client methods can now be called. Developers are advised to consult :ref:`the-client-api-ref` for details on the available methods and how to use them.
+
+In the following example, the *cyclades* and *pithos* clients of example 1.1 are used to extract some information, that is then printed to the standard output.
+
+
+.. code-block:: python
+    :emphasize-lines: 1,2
+
+    Example 1.2: Print server name and OS for server with server_id
+                Print objects in container mycont
+
+
+    srv = my_cyclades_client.get_server_info(server_id)
+    print("Server Name: %s (with OS %s" % (srv['name'], srv['os']))
+
+    obj_list = my_pithos_client.list_objects(mycont)
+    for obj in obj_list:
+        print('  %s of %s bytes' % (obj['name'], obj['bytes']))
+
+.. code-block:: console
+    :emphasize-lines: 1
+
+    Run of examples 1.1 + 1.2
+
+
+    $ python test_script.py
+    Server Name: A Debian Server (with OS Debian Base)
+      lala.txt of 34 bytes
+      test.txt of 1232 bytes
+      testDir/ of 0 bytes
+    $ 
+
+Error handling
+^^^^^^^^^^^^^^
+
+The kamaki.clients standard error is ClientError. A ClientError is raised for any kind of kamaki.clients errors (errors reported by servers, type errors in arguments, etc.).
+
+A ClientError contains::
+
+    message     The error message.
+    status      An optional error code, e.g. after a server error.
+    details     Optional list of messages with error details.
+
+The following example concatenates examples 1.1 and 1.2 plus error handling
+
+.. code-block:: python
+
+    Example 1.3: Error handling
+
+
+    from kamaki.clients.cyclades import CycladesClient
+    from kamaki.clients.pithos import PithosClient
+
+    try:
+        my_cyclades_client = CycladesClient(base_url, token)
+    except ClientError:
+        print('Failed to initialize Cyclades client')
+
+    try:
+        my_pithos_client = PithosClient(base_url, token, account, container)
+    except ClientError:
+        print('Failed to initialize Pithos+ client')
+
+    try:
+        srv = my_cyclades_client.get_server_info(server_id)
+        print("Server Name: %s (with OS %s" % (srv['name'], srv['os']))
+
+        obj_list = my_pithos_client.list_objects(mycont)
+        for obj in obj_list:
+            print('  %s of %s bytes' % (obj['name'], obj['bytes']))
+    except ClientError as e:
+        print('Error: %s' % e)
+        if e.status:
+            print('- error code: %s' % e.status)
+        if e.details:
+            for detail in e.details:
+                print('- %s' % detail)
+
+Extending kamaki.clients
+------------------------
+
+By default, kamaki clients are REST clients (they manage HTTP requests and responses to communicate with services). This is achieved by importing the connection module, which is an httplib rapper.
+
+Connection
+^^^^^^^^^^
+
+The connection module features an error handling and logging system, a lazy response mechanism, a pooling mechanism as well as concurrency control for thread-demanding client functions (e.g. store upload).
+
+How to build a client
+^^^^^^^^^^^^^^^^^^^^^
+
+All service clients consist of a subclass of the Client class and implement separate client functionalities as member methods. There is also an error class to raise exceptions that can be handled by kamaki interfaces.
+
+.. code-block:: python
+    
+    #  ${KAMAKI_PATH}/kamaki/clients/mynewclient.py
+
+    from kamaki.clients import Client, ClientError
+
+    class MyNewClient(Client):
+        """MyNewClient Description Here"""
+
+        def my_first_method(self, **args):
+            """Method description"""
+            try:
+                ...
+                method code
+                ...
+            except SomeKnownException as e1:
+                raise ClientError('MyError: %s' % e1)
+            except SomeOtherException as e2:
+                raise ClientError('MyError: %s' % e2)
+
+        def my_second_method(self, **params):
+            """Method description"""
+            ...
+
+Custom clients can use a set of convenience methods for easy HTTP requests
+
+.. code-block:: python
+
+    def get(self, path, **kwargs)
+    def head(self, path, **kwargs)
+    def post(self, path, **kwargs)
+    def put(self, path, **kwargs)
+    def delete(self, path, **kwargs)
+    def copy(self, path, **kwargs)
+    def move(self, path, **kwargs)
+
+How to use your client
+^^^^^^^^^^^^^^^^^^^^^^
+
+External applications must instantiate a MyNewClient object.
+
+.. code-block:: python
+
+    from kamaki.clients import ClientError
+    from kamaki.clients.mynewclient import MyNewClient
+
+    ...
+    try:
+        cl = MyNewClient(args)
+        cl.my_first_method(other_args)
+    except ClientError as cle:
+        print('Client Error: %s' % cle)
+    ...
+
+Concurrency control
+^^^^^^^^^^^^^^^^^^^
+
+Kamaki clients may handle multiple requests at once, using threads. In that case, users might implement their own thread handling mechanism, use an external solution or take advantage of the mechanism featured in kamaki.clients
+
+.. code-block:: python
+
+    from threading import enumerate
+    from kamaki.clients import SilentEvent
+    ...
+
+    class MyNewClient(Client):
+        ...
+
+        def _single_threaded_method(self, **args):
+            ...
+            request code
+            ...
+
+        def multithread_method(self):
+            thread_list = []
+            self._init_thread_limit()
+            while some_condition or thread_list:
+                ...
+                event = SilentEvent(self._single_threaded_method, **args)
+                event.start()
+                thread_list.append(event)
+                thread_list = self._watch_thread_limit(thread_list)
+
+The CLI API
+-----------
+
+.. toctree::
+
+    cli
+
+.. _the-client-api-ref:
 
 The clients API
 ---------------
 
-The clients API is based on:
+Imports
+^^^^^^^
 
 .. toctree::
     connection
 
-It contains the following modules:
+Modules list
+^^^^^^^^^^^^
 
 compute
 ^^^^^^^
diff --git a/docs/commands.rst b/docs/commands.rst
index 54982fb..5b4ba2f 100644
--- a/docs/commands.rst
+++ b/docs/commands.rst
@@ -1,19 +1,20 @@
-Commands
-========
-
 List of commands
-----------------
+================
+
+The commands described bellow are grouped by service. The examples showcase a sample set of group commands. The kamaki interactive shell (check `Usage section <usage.html#interactive-shell>`_ for details) is chosen as the execution environment.
 
-The commands described bellow are grouped by service. The examples showcase a sample set of group commands. The kamaki interactive shell (check `Usage section <usage.html#interactive-shell>`_ for details) is chosen as the execution environment:
 
 astakos (Identity Manager)
-^^^^^^^^^^^^^^^^^^^^^^^^^^
+--------------------------
 
 .. code-block:: text
 
     authenticate:  Authenticate a user
 
-Showcase: get user information, provided the token was set
+Showcase: get user information
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In the following, the token has been set in a previous step (see `setup section <setup.html>`_ or the `quick setup guide <usage.html#quick-setup>`_)
 
 .. code-block:: console
     :emphasize-lines: 1,4
@@ -34,7 +35,7 @@ Showcase: get user information, provided the token was set
     username          :  4215th3b357num9323v32
 
 flavor (Compute/Cyclades)
-^^^^^^^^^^^^^^^^^^^^^^^^^
+-------------------------
 
 .. code-block:: text
 
@@ -42,6 +43,7 @@ flavor (Compute/Cyclades)
     list:  List flavors
 
 Showcase: show details for flavor with id 43
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 .. code-block:: console
     :emphasize-lines: 1,4
@@ -59,7 +61,7 @@ Showcase: show details for flavor with id 43
     ram              :  2048
 
 image (Compute/Cyclades + Glance)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+---------------------------------
 
 .. code-block:: text
 
@@ -80,6 +82,7 @@ image (Compute/Cyclades + Glance)
     shared     :  List shared images
 
 Showcase: Pick an image and list the properties
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 .. code-block:: console
     :emphasize-lines: 1,4,18
@@ -113,7 +116,7 @@ Showcase: Pick an image and list the properties
     users         :  root
 
 server (Compute/Cyclades)
-^^^^^^^^^^^^^^^^^^^^^^^^^
+-------------------------
 
 .. code-block:: text
 
@@ -135,7 +138,8 @@ server (Compute/Cyclades)
     stats   :  Get server statistics
     wait    :  Wait for server to finish
 
-Showcase: Create a server.
+Showcase: Create a server
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 .. code-block:: console
     :emphasize-lines: 1,4,21,35,44,62
@@ -209,7 +213,7 @@ Showcase: Create a server.
 .. Note:: In kamaki shell, / is used to access top-level command groups while working in command group contexts
 
 network (Compute/Cyclades)
-^^^^^^^^^^^^^^^^^^^^^^^^^^
+--------------------------
 
 .. code-block:: text
 
@@ -222,6 +226,7 @@ network (Compute/Cyclades)
     rename    :  Update network name
 
 Showcase: Connect a network to a VM
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 .. code-block:: console
     :emphasize-lines: 1,4,9,24,27,44
@@ -286,7 +291,7 @@ Showcase: Connect a network to a VM
 .. Note:: In kamaki shell, / is used to access top-level command groups while working in command group contexts
 
 store (Storage/Pithos+)
-^^^^^^^^^^^^^^^^^^^^^^^
+-----------------------
 
 .. code-block:: text
 
@@ -324,7 +329,8 @@ store (Storage/Pithos+)
     versioning    :  Get  versioning for account [or container ]
     versions      :  Get the version list of an object
 
-Showcase: Upload and download a file.
+Showcase: Upload and download a file
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 .. code-block:: console
     :emphasize-lines: 1,7,11,16,21,29,33,37,41,44,51,55,60,64
@@ -399,53 +405,3 @@ Showcase: Upload and download a file.
     [store]:!diff rndm_local.file rndm_remote.file
 
 .. Note:: In kamaki shell, ! is used to execute OS shell commands (bash in the above)
-
-
-Command specifications
-----------------------
-
-cyclades_cli
-^^^^^^^^^^^^
-
-.. automodule:: kamaki.cli.commands.cyclades_cli
-    :members:
-    :undoc-members:
-
-
-pithos_cli
-^^^^^^^^^^
-
-.. automodule:: kamaki.cli.commands.pithos_cli
-    :members:
-    :show-inheritance:
-    :undoc-members:
-
-image_cli
-^^^^^^^^^
-
-.. automodule:: kamaki.cli.commands.image_cli
-    :members:
-    :show-inheritance:
-    :undoc-members:
-
-
-astakos_cli
-^^^^^^^^^^^
-
-.. automodule:: kamaki.cli.commands.astakos_cli
-    :members:
-    :undoc-members:
-
-config_cli
-^^^^^^^^^^
-
-.. automodule:: kamaki.cli.commands.config_cli
-    :members:
-    :undoc-members:
-
-history_cli
-^^^^^^^^^^^
-
-.. automodule:: kamaki.cli.commands.history_cli
-    :members:
-    :undoc-members:
diff --git a/docs/index.rst b/docs/index.rst
index c5ec3e6..672f0ce 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -21,7 +21,7 @@ Contents:
 *********
 
 .. toctree::
-   :maxdepth: 3
+   :maxdepth: 2
    
    overview
    installation
diff --git a/docs/installation.rst b/docs/installation.rst
index 2be955f..09cb261 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -16,6 +16,18 @@ Linux and Unix-like enviroments
 Installing from source (git repos.)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
+Requirements
+""""""""""""
+
+Essential:
+
+ * Python 2.6 or better [http://www.python.org]
+ * Python setuptools [http://pypi.python.org/pypi/setuptools]
+
+Optional:
+
+ * VirtualEnv (python-virtualenv) [http://www.virtualenv.org]
+
 1. Setup a virtual enviroment (optional)
 """"""""""""""""""""""""""""""""""""""""
 
@@ -27,12 +39,12 @@ With virtualenv users can setup kamaki and synnefo services in a sandbox environ
 
     $ source kamaki-env/bin/activate
 
-.. hint:: More about virtualenv: `<http://www.virtualenv.org>`_
+A more detailed example of using virtual env can be found at the `snf-image-creator setup guide <http://docs.dev.grnet.gr/snf-image-creator/latest/install.html#python-virtual-environment>`_
 
-2. Install snf-common from synnefo project (required since v0.6.0)
-""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+2. Install snf-common
+"""""""""""""""""""""
 
-Package snf-common is required since kamaki 0.6.0.
+Package snf-common is part of the synnefo project and is a kamaki dependency since version 0.6.0.
 
 .. code-block:: console
 
diff --git a/docs/setup.rst b/docs/setup.rst
index 81d3032..ff2e95e 100644
--- a/docs/setup.rst
+++ b/docs/setup.rst
@@ -6,23 +6,30 @@ Kamaki is easy to install from source or as a package. Some ui features are opti
 Requirements
 ------------
 
-* python 2.6 or better
+Essential:
+
+ * Python 2.6 or better [http://www.python.org/]
+ * snf-common 0.10 or better [https://code.grnet.gr/projects/synnefo]
+
+Optional:
+
+ * ANSI colors for Python [http://pypi.python.org/pypi/ansicolors]
+ * progress [http://pypi.python.org/pypi/progress]
 
-* snf-common 0.10 or better
 
 The snf-common package is part of the Synnefo project of the Greek Research and Development Network and is available from the same official sources as Kamaki (e.g. http://apt.dev.grnet.gr ).
 
 Optional features
 -----------------
 
+For installing any all of the following, consult the `kamaki installation guide <installation.html#install-progress-and-or-ansicolors-optional>`_
+
 * ansicolors
     * Make command line / console user interface responses prettier with text formating (colors, bold, etc.)
     * Can be switched on/off in kamaki configuration file: colors=on/off
-    * Installation: pip install ansicolors
 
 * progress 
     * Attach progress bars to various kamaki commands (e.g. kamaki store upload)
-    * Installation: pip install progressbar
     * Since version 0.6.1 kamaki "requires" progress version 1.0.2 or better
 
 Any of the above features can be installed at any time before or after kamaki installation.
diff --git a/docs/usage.rst b/docs/usage.rst
index 356c7bb..47c2c42 100644
--- a/docs/usage.rst
+++ b/docs/usage.rst
@@ -5,20 +5,17 @@ Kamaki offers command line interfaces that implement specific command specificat
 
 What's more, kamaki offers a clients API that allows the development of external applications for synnefo. The clients API is listed in the `Clients lib <clients.html>`_ section. The recommended method of utilizing this API is explained in the present.
 
-Setup
------
+Quick Setup
+-----------
 
 Kamaki interfaces rely on a list of configuration options. In the initial state, kamaki is configured to communicate with the Okeanos IaaS. A detailed guide for setting up kamaki can be found in the `Setup <setup.html>`_ section.
 
-Quick guide
-^^^^^^^^^^^
-
-It is essential for users to get a configuration token (to get in Okeanos.grnet.gr log `here <https://accounts.okeanos.grnet.gr/im/>`_) and provide it to kamaki:
+It is essential for users to get a configuration token (okeanos.grnet.gr users go `here <https://accounts.okeanos.grnet.gr/im/>`_) and provide it to kamaki:
 
 .. code-block:: console
     :emphasize-lines: 1
 
-    Example 1.1.1: Set user token to myt0k3n==
+    Example 1.1: Set user token to myt0k3n==
 
     $ kamaki set token myt0k3n==
 
@@ -27,7 +24,7 @@ To use the storage service, a user should also provide the corresponding user-na
 .. code-block:: console
     :emphasize-lines: 1
 
-    Example 1.1.2: Set user name to user@domain.com
+    Example 1.2: Set user name to user@domain.com
 
     $ kamaki set account user@domain.com
 
diff --git a/kamaki/clients/pithos.py b/kamaki/clients/pithos.py
index d2b6d8a..75823fc 100644
--- a/kamaki/clients/pithos.py
+++ b/kamaki/clients/pithos.py
@@ -206,7 +206,7 @@ class PithosClient(PithosRestAPI):
         assert offset == size
 
     def _upload_missing_blocks(self, missing, hmap, fileobj, upload_cb=None):
-        """upload missing blocks asynchronously. 
+        """upload missing blocks asynchronously.
         """
         if upload_cb:
             upload_gen = upload_cb(len(missing))
