Synnefo » ./kamaki

The commands described bellow are grouped by service. The examples showcase a

sample set of group commands. The kamaki interactive shell (check

`Usage section <usage.html#interactive-shell>`_ for details) is chosen as the

execution environment.

In the following, the token has been set in a previous step (see

`setup section <setup.html>`_ or the

`quick setup guide <usage.html#quick-setup>`_)

    ...

    user:

        name:  My Real Name

        uuid:  ab1cde23-45fg-6h7i-8j9k-10l1m11no2pq

.. note:: actual call returns a full list of service endpoints accessible to

    the user with a specific token

    cpu :  4

    disk:  10

    id  :  43

    name:  C4R2048D10

    ram :  2048

    list      :  List images accessible by user

    meta      :  Get image metadata

    register  :  (Re)Register an image

    unregister:  Unregister an image (does not delete the image file)

    shared    :  List shared images

    compute   :  Compute Image API commands

        list      :  List images

        delete    :  Delete image

        info      :  Get image details

        properties:  Manage properties related to OS installation in an image

    members   :  Manage members (users who can modify an image)

        add   :  Add a member to an image

        delete:  Remove a member from an image

        list  :  List members of an image

        set   :  Set the members of an image

    926ab1c5-2d85-49d4-aebe-0fce712789b9 Windows Server 2008

    78262ee7-949e-4d70-af3a-85360c3de57a Windows Server 2012

    5ed5a29b-292c-4fe0-b32c-2e2b65628635 ubuntu

    1f8454f0-8e3e-4b6c-ab8e-5236b728dffe Debian_Wheezy_Base

            [--personality PERSONALITY] [-h] [--config CONFIG] [--cloud CLOUD]

      --cloud CLOUD         Chose a cloud to connect to

               os   :  debian

               users:  root

      cidr    :  192.168.1.0/24

      cidr6   :  None

      created :  2012-11-23T17:17:20.560098+00:00

      dhcp    :  True

      gateway :  None

      gateway6:  None

      id      :  1409

      name    :  my network

      public  :  False

      status  :  ACTIVE

      type    :  MAC_FILTERED

      updated :  2012-11-23T17:18:25.095225+00:00

.. Note:: In kamaki shell, ! is used to execute OS shell commands (e.g. bash)

.. warning:: The container:object/path syntax does not function if the

    container and / or the object path contain one or more : characters. To use

    containers and objects with : use the --container and --dst-container

    arguments, e.g. to copy test.py object from grnet:dev container to

    grnet:deploy ::

Kamaki commands are implemented as python classes, decorated with a special

decorator called *command*. This decorator is a method of kamaki.cli that adds

a new command in a CommandTree structure (kamaki.cli.commant_tree). The later

is used by interfaces to manage kamaki commands.

There are two command sets to implement, namely mygrp1 and mygrp2. The first

will contain two commands, namely list-all and list-details. The second one

will also contain two commands, list-all and info. To avoid ambiguities,

command names should rather be prefixed with the group they belong to, e.g.

mygrp1-list-all and mygrp2-list-all.

The first command has the simplest possible syntax: no parameters, no runtime

arguments. The second accepts an optional runtime argument with a value. The

third features an optional argument and an optional runtime flag argument. The

last is an example of a command with an obligatory and an optional argument.

The above example will be used throughout the present guide.

CommandTree manages a command by its path. Each command is stored in multiple

nodes on the tree, so that the last term is a leaf and the route from root to

that leaf represents the command path. For example the commands *file upload*,

*file list* and *file info* are stored together as shown bellow::

Each command group should be stored on a different CommandTree. For that

reason, command specification modules should contain a list of CommandTree

objects, named *_commands*

A command group information (name, description) is provided at CommandTree

structure initialization:

The *command* decorator mines all the information necessary to build a command

specification which is then inserted in a CommanTree instance::

Kamaki interfaces make use of this CommandTree structure. Optimizations are

possible by using special parameters on the command decorator method.

A list of CommandTree structures must exist in the module scope, with the name

_commands, as shown above. Different CommandTree objects correspond to

different command groups.

The description of each command is the first line of the class commend. The

following declaration of *mygrp2-info* command has a "*get information for

subject with id*" description.

The argument mechanism allows the definition of run-time arguments. Some basic

argument types are defined at the

`argument module <code.html#module-kamaki.cli.argument>`_, but it is not

uncommon to extent these classes in order to achieve specialized type checking

and syntax control (e.g. at

`pithos cli module <code.html#module-kamaki.cli.commands.pithos>`_).

To declare a run-time argument on a specific command, the object class should

initialize a dict called *arguments* , where Argument objects are stored. Each

argument object is a possible run-time argument. Syntax checking happens at

client level, while the type checking is implemented in the Argument code

(thus, many different Argument types might be needed).`

                ('-m', '--match'))

or more usually and elegantly:

.. code-block:: python

    from kamaki.cli.argument import ValueArgument

    @command(_mygrp1_commands)

    class mygrp1_list_details():

    """List of details"""

        arguments = dict(

            match=ValueArgument(

            'Filter output to match string', ('-m', --match'))

        )

Accessing run-time arguments

----------------------------

To access run-time arguments, users can use the _command_init interface, which

implements __item__ accessors to handle run-time argument values. In specific,

an instance of _command_init can use brackets to set or read <argument>.value .

.. code-block:: python

    from kamaki.cli.argument import ValueArgument

    from kamaki.cli.commands import _command_init

    @command(_mygrp1_commands)

    class mygrp1_list_details(_command_init):

        """List of details"""

        arguments = dict(

            match=ValueArgument(

                'Filter output to match string', ('-m', --match'))

        )

        def check_runtime_arguments(self):

            ...

            assert self['match'] == self.arguments['match'].value

            ...

The command behavior for each command / class is coded in *main*. The

parameters of *main* method defines the command parameters part of the syntax.

In specific::

The information that can be mined by *command* for each individual command is

presented in the following:

    class mygrp2_list_all():

        arguments = dict(FlagArgument('detailed list', '-l'))

Kamaki will load a command specification *only* if it is set as a configurable

option. To demonstrate this, let the command specifications coded above be

stored in a file named *grps.py*.

The developer should move file *grps.py* to kamaki/cli/commands, the default

place for command specifications, although running a command specification from

a different path is also a kamaki feature.

    [global]

    mygrp1_cli = grps

    mygrp2_cli = grps

or equivalently:

    $ kamaki config set mygrp1_cli grps

    $ kamaki config set mygrp2_cli grps

Command specification modules don't need to live in kamaki/cli/commands,

although this is suggested for uniformity. If a command module exist in another

path::

    [global]

    mygrp_cli = /another/path/grps.py

    from kamaki.cli.commands import _command_init

    class mygrp1_list_all(_command_init):

    class mygrp1_list_details(_command_init):

        arguments = dict(

            match=ValueArgument(

                'Filter output to match string', ('-m', --match'))

        )

            match_value = self['match']

    class mygrp2_list_all(_command_init):

        arguments = dict(

            list=FlagArgument('detailed listing', '-l')

        )

            detail_flag = self['list']

    class mygrp2_info(_command_init):

Kamaki features a clients API for building third-party client applications that

communicate with OpenStack and / or Synnefo cloud services. The package is

called kamaki.clients and servers as a lib.

A showcase of an application built on kamaki.clients is kamaki.cli, the command

line interface of kamaki.

Since synnefo services are build as OpenStack extensions, an inheritance

approach has been chosen for implementing clients for both. In specific,

the *compute*, *storage* and *image* modules are clients of the OS compute, OS

object-store, respectively. On the contrary, all the other modules are Synnefo

extensions (*cyclades* extents *compute*, *pithos* and *pithos_rest_api*

extent *storage*) or novel synnefo services (e.g. *astakos* for IM, *image*

for *plankton*).

.. note:: *cyclades* and *pithos* clients inherit all methods of *compute*

    and *storage* clients respectively. Separate compute or storage objects

    should be used only when implementing applications for strict OS Compute or

    OS Storage services.

Client methods can now be called. Developers are advised to

consult :ref:`the-client-api-ref` for details on the available methods and how

to use them.

In the following example, the *cyclades* and *pithos* clients of example 1.1

are used to extract some information, that is then printed to the standard

output.

The kamaki.clients standard error is ClientError. A ClientError is raised for

any kind of kamaki.clients errors (errors reported by servers, type errors in

arguments, etc.).

By default, kamaki clients are REST clients (they manage HTTP requests and

responses to communicate with services).

All service clients consist of a subclass of the Client class and implement

separate client functionalities as member methods. There is also an error class

to raise exceptions that can be handled by kamaki interfaces.

Kamaki clients may handle multiple requests at once, using threads. In that

case, users might implement their own thread handling mechanism, use an

external solution or take advantage of the mechanism featured in kamaki.clients

The kamaki.clients package contains a set of fine-grained unit-tests for all

its packages. 

.. note:: unit tests require the optional python-mock package, version 1.X or

    better

In each package under kamaki.clients, there is a test module (test.py) where

the tests are implemented. To run all tests, run the test.py file from

kamaki.clients

To test a specific class, add the class name as an argument. E.g. for the

Client class:

To test a specific method in a class, apply an extra argument, e.g. for the

request method in the Client class:

Each package contains a test module (test.py) which is also runnable from the

command line. E.g. in the pithos package there is a test module which

contains, among others, the **download** sub-test:

To fully test a specific package, run test.py from the package location. E.g.

to test everything in kamaki.clients.pithos package:

Each folder / package contains a test.py file, that represents the test module

of this package. All test modules contain a set of classes that extent the

TestCase class. They also contain a main method to run the tests.

By convention, testing classes are named as <Tested Class> where <Test Class>

is the name of the tested class or module. Methods not grouped in classes are

tested by classes named after their respective module.

For example, the kamaki.clients.pithos.PithosClient class is tested by the

kamaki.clients.pithos.test.PithosClient class, while the methods in

kamaki.clients.utils module are tested by the kamaki.clients.utils.test.Utils

testing class.

After modifying or extending kamaki.clients method, classes, modules or

packages, it is a good practice to also modify or extend the corresponding

unit tests. What's more, it is recommended to modify or implement the testing

of new behavior before implementing the behavior itself. The aim for

kamaki.clients package is an 1 to 1 mapping between methods and their tests.

In case of an existing method modification, the programmer has to modify the

corresponding test as well. By convention, the test method is located in the

test module under the same package, in a TestCase subclass that is named with a

name similar to the package or class that contains the tested method.

Example 1: to modify the kamaki.clients.utils.filter_in method, the programmer

has to also adjust the kamaki.clients.utils.test.Utils.test_filter_in method.

Example 2: to modify the kamaki.clients.pithos.PithosRestClient.object_get, the

programmer has to also adjust the

kamaki.clients.pithos.test.PithosRestClient.test_object_get method.

Programmers who want to implement a new method in an existing class, are

encouraged to implement the corresponding unit test first. In order to do that,

they should find the testing class that is mapped to the class or module they

need to extend.

Example 1: To add a **list_special** method to

kamaki.clients.astakos.AstakosClient, extend the

kamaki.clients.astakos.test.AstakosClient class, as shown bellow:

Example 2: To add a **get_random_int** method in kamaki.clients.utils module,

extend the kamaki.clients.utils.test.Utils test class, as shown bellow:

Each class or module needs a seperate test sub-module. By convention, each

class or module under the kamaki.clients should be located in a separate

directory.

Example 1: To add a NewService class that implements the kamaki.clients.Client

interface: 

.. note:: If the new class or module is part of an existing sub-package, it is

    acceptable to append its testing class in the existing test.py file of the

    sub-package it belongs to. For example, the

    kamaki.clients.pithos.PithosClient and

    kamaki.clients.pithos.rest_api.PithosRestClient classes are tested by two

    different classes (PithosClient and PithosRestClient respectively) in the

    same module (kamaki.clients.pithos.test).

As a development library, it implements the Synnefo API (

`Synnefo IaaS <http://www.synnefo.org/>`_ cloud management software) which

extends OpenStack, therefore is also compatible with the OpenStack API.

Linux and Unix-like environments

--------------------------------

* append         Append local file to remote file

Synnefo is an IaaS system which implements and extents OpenStack. Synnefo has

been deployed in many environments to cover multiple needs. The most notable

deployment is probably the GRNET's

Kamaki has to be configured for a specific Synnefo deployment, with an

authentication url and user token pair. Users should also pick an alias to name

the cloud configuration. This can be any single word, e.g. "default", "mycloud"

or whatever suits kamaki users.

    $ kamaki config set cloud.<cloud alias>.url <cloud-authentication-URL>

    $ kamaki config set cloud.<cloud alias>.token myt0k3n==

If only one cloud is configured, kamaki automatically picks it as the default.

Otherwise, a default cloud should be specified:

.. code-block:: console

    $ kamaki config set default_cloud <cloud alias>

Since Synnefo version 0.14, a synnefo cloud UI offers a single authentication

URL, which should be set as the cloud URL for kamaki. All service-specific URLs

are retrieved and handled automatically by kamaki, through this URL. Users of

synnefo clouds >=0.14 are advised against using any service-specific URLs.

This section refers to running installations of kamaki version <= 0.8.X To

check the current kamaki version:

Existing kamaki users should convert their configuration files to v9. To do

that, kamaki 0.9 can inspect the configuration file and suggests a list of

config file transformations, which are performed automatically (after users'

permission). This mechanism is invoked when an API-related kamaki command is

fired. On example 2.1 we suggest using the `user authenticate` command to fire

the kamaki config file conversion mechanism.

    Config file format version >= 9.0 is required

    Configuration file: "/home/exampleuser/.kamakirc"

    ... rescue global.token => cloud.default.token

    Kamaki is ready to convert the config file to version 9.0

    In this example, the user accidentally misspelled the term `log_data`

Multiple clouds

---------------

frequently be called as a **cloud**.

Kamaki supports accessing multiple clouds from the same kamaki setup. Before

kamaki 0.9, this was possible only by using multiple config files. Since 0.9,

kamaki supports multiple clouds in the same configuration.

Each cloud corresponds to a Synnefo (or Open Stack) cloud deployment.

Once a user has retrieved one URL/token pair per cloud, it is time to assign a

name to each cloud and let kamaki know about them.

For example, let the user have access to two clouds with the following authentication information ::

    $ kamaki config set cloud.devel.url https://devel.example.com/astakos/identity/v2.0/

    $ kamaki config set cloud.devel.token myd3v3170k3n==

    $ kamaki config set cloud.testing.url https://testing.example.com/astakos/identity/v2.0/

    $ kamaki config set cloud.testing.token my73571ng70k3n==

To check if all settings are loaded, a user may list all clouds, as shown

    $ kamaki config getcloud 

     cloud.default.url = https://example.com/astakos.identity/v2.0/

     cloud.default.url = myd3f4u1770k3n==

     cloud.devel.url = https://devel.example.com/astakos/identity/v2.0/

     cloud.devel.token = myd3v3170k3n==

     cloud.testing.url = https://testing.example.com/astakos/identity/v2.0/

     cloud.testing.token = my73571ng70k3n==

or query kamaki for a specific cloud:

    $ kamaki config get cloud.devel

     cloud.devel.url = https://devel.example.com/astakos/identity/v2.0/

     cloud.devel.token = myd3v3170k3n==

Now kamaki can use any of these clouds, with the **- - cloud** attribute. If

the **- - cloud** option is ommited, kamaki will query the `default` cloud.

In interactive cell, the cloud is picked when invoking the shell, with

    * Add colors to command line / console output

    * Can be switched on/off in kamaki configuration file: `colors = on/off`

    * Allow unit tests to run on kamaki.clients package

Any of the above features can be installed at any time before or after kamaki

installation.

* cloud-related

    information needed to connect and use one or more clouds. There are some

    mandatory options (URL, token) and some advanced / optional (e.g.

    service-specific URL overrides or versions)

provided by the user. Kamaki-related options can also be modified.

explicitly load configuration files with the **- - config** (or **- c**) option

.. note:: For accessing multiple clouds, users do NOT need to create multiple

    configuration files. Instead, we suggest using a single configuration file

    with multiple cloud setups. More details can be found at the

    `multiple clouds guide <#multiple-clouds>`_.

    *global*.

The commands above can also be used for **clouds** handling, using the `cloud.`

prefix. The cloud handling cases are similar but with slightly different

semantics:

* kamaki config get cloud[.<cloud alias>[.option]]

    * cloud 

        list all clouds and their settings

    * cloud.<cloud alias>

        list settings of the cloud aliased as <cloud alias>. If no

        special is configured, use the term `cloud.default`

    * cloud.<cloud alias>.<option>

        configured, use `cloud.default.<option>`

* kamaki config set cloud.<cloud alias>.<option> <value>

    If the cloud alias <cloud alias> does not exist, create it. Then, create

    (or update) the option <option> of this cloud, by setting its value

* kamaki config delete cloud.<cloud alias>[.<option>]

    * cloud.<cloud alias>

    * cloud.<cloud alias>.<option>

        delete the <option> and its value from the cloud cloud aliased as

.. note:: To see if a default cloud is configured, get the default_cloud value

    .. code-block:: console

        $ kamaki config get default_cloud

    [cloud "default"]

The [cloud "default"] section is special and is used to configure the default

cloud cloud. Kamaki will not be able to run without setting the url and token

More clouds can be created  on the side of the default cloud, e.g using the

examples at the `multiple clouds guide <#multiple-clouds>`_ ::

    [cloud "devel"]

    [cloud "testing"]

In most tests, livetest will run as long as the default cloud is configured

    * livetest.testcloud = <the cloud alias this test will run against>

    Example 1.1: Set authentication URL, user token and cloud alias "default"

    $ kamaki config set cloud.default.url <authentication URL>

    $ kamaki config set cloud.default.token myt0k3n==

.. note:: The term *default* can be replaced by any arbitary term chosen by

    the user. This term will serve as a cloud alias for kamaki users, and can

    be easily modified.

* tab completion for commands (if supported by host command line shell)

* session history with ↑ or ↓ keys (if supported by host command line shell)

    $ kamaki -c myconfig.file

      --cloud CLOUD         Chose a cloud to connect to

    config :  Kamaki configurations

    file   :  Pithos+/Storage API commands

    flavor :  Cyclades/Compute API flavor commands

    history:  Kamaki command history

    image  :  Cyclades/Plankton API image commands

    network:  Cyclades/Compute API network commands

    server :  Cyclades/Compute API server commands

    user   :  Astakos API commands

      --cloud CLOUD         Chose a cloud to connect to

    * to set authentication token: /config set cloud.default.token <token>

    --cloud CLOUD         Chose a cloud to connect to

    $ kamaki file list -o cloud.default.pithos_container=trash| grep c1_

    [file]: /config get cloud.default.token

    [file]: /config set cloud.default.token myfr35ht0k3n==

    [file]: /config get cloud.default

    cloud.default.url = https://astakos.example.com/astakos/identity/2.0/

    cloud.default.token = myfr35ht0k3n==

    Actual kamaki error messages are more helpful and descriptive.

The configuration mechanism of kamaki is detailed in the

    Example 4.4.1: Set default storage container (cloud: default)

    [file]: /config set cloud.default.pithos_container mycontainer

    Example 4.4.2: Delete a setting option (cloud: default)

    [file]: /config delete cloud.default.pithos_container

        print('Attempting to fix this:')

        print('Kamaki is ready to convert the config file')

                    'Please, update config file',

                        print('%s.%s = %s' % (prefix, k, v))