Synnefo » ./kamaki

.. hint:: To activate functional tests in kamaki. enable the preconfigured

    *livetest* command group:

    .. code-block:: console

        $ kamaki config set livetest_cli livetest

Install astakosclient (optional)

""""""""""""""""""""""""""""""""

A seperate project called

`astakosclient <https://pypi.python.org/pypi/astakosclient>`_ can be used for

advanced user and service management.

.. code-block:: console

    $ apt-get install python-astakosclient

.. hint:: To activate astakosclient commands in kamaki, enable the

    preconfigured *astakos* command group:

    .. code-block:: console

        $ kamaki config set astakos_cli astakos

Install ansicolors

""""""""""""""""""

.. hint:: To activate functional tests in kamaki. enable the preconfigured

    *livetest* command group:

    .. code-block:: console

        $ kamaki config set livetest_cli livetest

Install astakosclient

"""""""""""""""""""""

A seperate project called

`astakosclient <https://pypi.python.org/pypi/astakosclient>`_ can be used for

advanced user and service management.

.. code-block:: console

    $ pip install astakosclient

.. hint:: To activate astakosclient commands in kamaki, enable the

    preconfigured *astakos* command group:

    .. code-block:: console

        $ kamaki config set astakos_cli astakos

Kamaki has to be configured to use a specific Synnefo deployment url and a user

token.

    $ kamaki config set remote.default.url <cloud-authentication-URL>

Since Synnefo version 0.14, a synnefo cloud remote UI offers a single

authentication URL, which can to be set as the default URL for kamaki. All

service-specific URLs are now retrieved and handled automatically. Users of

synnefo clouds >= 0.14 are advised against using any service-specific URLs.

This mechanism is invoked when the first API-related kamaki command is fired.

We suggest the `user authenticate` command, as shown on example 2.1.

    Example 2.1: Convert config file while authenticating user "exampleuser"

be understood by the new kamaki.

* `global.account` and `user.account` are not used anymore.

    The same is true for the synonyms `store.account` and `pithos.account`.

    These options were used to explicitly set a user account or uuid to a

    pithos call. In the latest Synnefo version (>= 0.14), these features are

    meaningless and therefore omitted.

* `global.data_log` option has never been a valid kamaki config option.

    In this example, the user accidentally misspelled the terms `log_data`

    (which is a valid kamaki config option) as `data_log`. To fix this, the

    user should set the correct option after the conversion is complete

    (Example 2.2).

In order to convert more files, users may run kamaki with the -c option, which

runs kamaki with a different configuration file (Example 2.3) and apply the

steps described above.

Multiple cloud remotes

----------------------

The following refers to users of multiple Synnefo and/or Open Stack

deployments. In the following, a Synnefo or Open Stack cloud deployment will

frequently be called as a **remote** or a **cloud remote**.

Kamaki supports accessing multiple cloud remotes from the same kamaki setup.

Bofore kamaki 0.9, this was possible only by using multiple config files. Since

0.9, kamaki supports multiple cloud remotes in the same configuration.

Each cloud remote corresponds to a Synnefo (or Open Stack) cloud deployment.

Since Synnefo version 0.14, each deployment offers a single point of

authentication, as an **authentication URL** and **token** pair. Users can

retrieve this information through the cloud UI.

Once a user has retrieved one URL/token pair per cloud remote, it is time to

assign a name to each cloud and let kamaki know about them.

For example, let the user have access to two remote clouds with the following authentication information ::

    cloud alias: devel

    authentication URL: https://devel.example.com/astakos/identity/v2.0/

    authentication token: myd3v3170k3n==

    cloud alias: testing

    autentication URL: https://testing.example.com/astakos/identity/v2.0/

    authentication token: my73571ng70k3n==

.. note:: the cloud alias is arbitrary and decided by the user. It is just a

    name to call a cloud setup in the kamaki context.

The user should let kamaki know about these setups:

.. code-block:: console

    $ kamaki config set remote.devel.url https://devel.example.com/astakos/identity/v2.0/

    $ kamaki config set remote.devel.token myd3v3170k3n==

    $

    $ kamaki config set remote.testing.url https://testing.example.com/astakos/identity/v2.0/

    $ kamaki config set remote.testing.token my73571ng70k3n==

    $

To check if all settings are loaded, a user may list all remotes, as shown

bellow:

.. code-block:: console

    $ kamaki config get remote

     remote.default.url = https://example.com/astakos.identity/v2.0/

     remote.default.url = myd3f4u1770k3n==

     remote.devel.url = https://devel.example.com/astakos/identity/v2.0/

     remote.devel.token = myd3v3170k3n==

     remote.testing.url = https://testing.example.com/astakos/identity/v2.0/

     remote.testing.token = my73571ng70k3n==

    $

or query kamaki for a specific cloud remote:

.. code-block:: console

    $ kamaki config get remote.devel

     remote.devel.url = https://devel.example.com/astakos/identity/v2.0/

     remote.devel.token = myd3v3170k3n==

    $

Now kamaki can use any of there remotes, with the **- - cloud** attribute. If

the **- - cloud** option is ommited, kamaki will query the `default` cloud remote.

One way to test this, is the `user athenticate` command:

.. code-block:: console

    $ kamaki --cloud=devel user authenticate

     ...

     user          : 

        id         :  725d5de4-1bab-45ac-9e98-38a60a8c543c

        name       :  Devel User

    $

    $ kamaki --cloud=testing user authenticate

     ...

     user          : 

        id         :  4ed5d527-bab1-ca54-89e9-c345c8a06a83

        name       :  Testing User

    $

    $ kamaki --cloud=default user authenticate

     ...

     user          : 

        id         :  4d3f4u17-u53r-4u7h-451n-4u7h3n7ic473

        name       :  Default User

    $

    $ kamaki user authenticate

     ...

     user          : 

        id         :  4d3f4u17-u53r-4u7h-451n-4u7h3n7ic473

        name       :  Default User

    $

In interactive cell, the cloud remote is picked when invoking the shell, with

the **- - cloud** option.

* astakosclient

    * For advanced users mostly

    * Allows the use of a full astakos command line client

There are two kinds of configuration options:

* kamaki-related (global)

    interface settings and constants of the kamaki internal mechanism, e.g.

    colors in the output, maximum threads per connection, custom logging or

    history files, etc.

* cloud-related (remote.XXX)

    information needed to connect and use one or more remote clouds. There are

    some mandatory options (url, token) and some advanced / optional (e.g.

    service-specific url overrides or versions)

Kamaki comes with preset default values to all kamaki-releated configuration

options. Cloud-related information is not included in presets and should be

provided. Kamaki-related options can also be modified.

There are two ways of managing configuration options: edit the config file or

use the kamaki config command.

Using multiple configuration files

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Kamaki setups are stored in configuration files. By default, a Kamaki

installation stores options in *.kamakirc* file located at the user home

directory.

If a user needs to switch between different kamaki-related setups, Kamaki can

explicitly load configuration files with the **- - config** (or **- c**)

option:

.. note:: For access to multiple cloud remotes, users do NOT need to create

    multiple configuration files. Instead, we suggest using a single

    configuration file with multiple remote setups. More details can be found

    at the `multiple remotes guide <#multiple-cloud-remotes>`_.

    $ kamaki file list -o global.pithos_container=anothercontainer

will invoke *kamaki file list* with the specified options, but the initial

global.pithos_container values will not be modified.

    show the value of a specific configuration option. Options must be of the

    form *group.option*. The term *option* is equivalent to *global.option*

    set the group.option to value. If no group is given, the defaults to

    *global*

    delete a configuration option. If no group is given, the defaults to

    *global*

The commands above can also be used for **cloud remotes** handling, using the

`remote.` prefix. The remote handling cases are similar but with slightly

different semantics:

* kamaki config get remote[.<cloud alias>[.option]]

    * remote

        list all cloud remotes (including `default`) and their settings

    * remote.<cloud alias>

        list settings of the cloud remote aliased as <cloud alias>. If no

        special is configured, use the term `remote.default`

    * remote.<cloud alias>.<option>

        show the value of the specified option. If no special alias is

        configured, use `remote.default.<option>`

* kamaki config set remote.<cloud alias>.<option> <value>

    If the remote alias <cloud alias> does not exist, create it. Then, create

    (or update) the option <option> of this cloud remote, by setting its value

    to <value>.

* kamaki config delete remote.<cloud alias>[.<option>]

    * remote.<cloud alias>

        delete the cloud alias <cloud alias> and all its options

    * remote.<cloud alias>.<option>

        delete the <option> and its value from the cloud remote aliased as

        <cloud alias>

.. note:: users of kamaki < 0.9 can use kamaki 0.9.X to automatically convert

    their old configuration files to the new config file version (>= 3.0). To

    do this, follow `these instructions <#migrating-from-kamaki-0-8-x-to-0-9>`_

A simple way to create the configuration file is to set a configuration option

using the kamaki config command. For example:

    $ kamaki config set global.log_file /home/exampleuser/logs/kamaki.log

In the above example, if the kamaki configuration file does not exist, it will

be created with all the default values plus the *global.log_file* option set to

`/home/exampleuser/logs/kamaki.log`

    [global]

    log_file = /home/exampleuser/logs/kamaki.log

    max_threads = 7

    colors = off

    [remote "default"]

    url =

    token =

A bunch of configuration options are created and set to their default options,

except the log_file option which is set to whatever the specified value.

The [remote "default"] section is special and is used to configure the default

cloud remote. Kamaki will not be able to run without setting the url and token

values to that section.

More cloud remotes can be created  on the side of the default remote, e.g

using the examples at the `multiple remotes guide <#multiple-cloud-remotes>`_::

    [remote "devel"]

    url = https://devel.example.com/astakos/identity/v2.0/

    token = myd3v3170k3n==

    [remote "testing"]

    url = https://testing.example.com/astakos/identity/v2.0/

    token = my73571ng70k3n==

The [global] group is treated by kamaki as a generic group for kamaki-related

settings, namely command cli specifications, the thread limit, console colors,

history and log files, log detail options and pithos-specific options.

    allow kamaki to log http data (by default, it logs only method, URL and

    headers)

* global.file_cli <UI command specifications for file>

    a special package that is used to load storage commands to kamaki UIs.

    Don't touch this unless if you know what you are doing.

* global.cyclades_cli <UI command specifications for cyclades>

    a special package that is used to load cyclades commands to kamaki UIs.

    Don't touch this unless you know what you are doing.

* global.flavor_cli <UI command specifications for VM flavors>

    a special package that is used to load cyclades VM flavor commands to

    kamaki UIs. Don't touch this unless you know what you are doing.

* global.network_cli <UI command specifications for virtual networks>

    a special package that is used to load cyclades virtual network commands to

    kamaki UIs. Don't touch this unless you know what you are doing.

* global.image_cli <UI command specs for Plankton or Compute image service>

* global.user_cli <UI command specs for Astakos authentication service>

    a special package that is used to load astakos-related commands to kamaki

    UIs. Don't touch this unless you know what you are doing.

* global.history_file <history file path>

    the path of a simple file for inter-session kamaki history. Make sure

    kamaki is executed in a context where this file is accessible for reading

    and writing. Kamaki automatically creates the file if it doesn't exist

Kamaki contains a live test suite for the kamaki.clients API, where "live"

means that the tests are performed against active services that up and running.

The live test package is named "livetest", it is accessible as kamaki.clients.

livetest and it is designed to check the actual relation between kamaki and

synnefo services.

    [global]

    livetest_cli=livetest

or with this kamaki command::

    kamaki config set livetest_cli livetest

In most tests, livetest will run as long as the default remote is configured

correctly. Some commands, though, need some extra settings related to the cloud

the test is performed against, or the example files used in kamaki.

Here is a list of settings needed:

* for all tests::

    * livetest.testremote = <the cloud alias this test will run against>

* for astakos client::

    * livetest.astakos_details = <A file with an authentication output>

        To create this file, pipeline the output of an authentication command

        with the -j option for raw jason output

        .. code-block:: console

            $ kamaki user authenticate -j > astakos.details

    * livetest.astakos_name = <The exact "real" name of testing user>

    * livetest.astakos_id = <The valid unique user id of the testing user>

* for image client:

    * livetest.image_details = <A file with the image's metadata>

        To create this file, pipeline the output of an image metadata command

        with the -j option for raw jason output

        .. code-block:: console

            $ kamaki image meta <img id> -j > img.details

    * livetest.image_id = <A valid image id used for testing>

    * livetest.image_local_path = <The local path of the testing image>

* for flavors (part of the compute client):

    * livetest.flavor_details = <A file with the flavor details>

        To create this file, pipeline the output of a flavor info command

        with the -j option for raw jason output

        .. code-block:: console

            $ kamaki flavor info <flavor id> -j > flavor.details

Kamaki container a set of finegrained unit tests for the kamaki.clients

package. This set is not used when kamaki is running. Instead, it is aimed to

developers who debug or extent the kamaki clients library. For more

information, check the

`Going Agile <developers/extending-clients-api.html#going-agile>`_ entry at the

`developers section <developers/extending-clients-api.html>`_.

Kamaki offers command line interfaces that implement specific command

specifications. A detailed list of the command specifications can be found in

`Commands <commands.html>`_ section. This guide covers the generic usage of

both interfaces.

What's more, kamaki offers a clients API that allows the development of

external applications for synnefo. The clients API is listed in the

`Clients lib <developers/code.html#the-clients-api>`_ section.

Kamaki interfaces rely on a list of configuration options. A detailed guide for

setting up kamaki can be found in the `Setup <setup.html>`_ section.

As rule of the thump, it is enough to set the authentication URL and user token

for the cloud kamaki should communicate with by default:

    Example 1.1: Set default authentication user and token

    $ kamaki config set remote.default.url <authentication URL>

    $ kamaki config set remote.default.token myt0k3n==

Kamaki users can access synnefo services through either the interactive shell

or the one-command behaviors. In practice, both systems rely on the same

command set implementations and API clients, with identical responses and error

messages. Still, there are some differences.

* tab completion for commands (if supported by host OS)

* session history with ↑ or ↓ keys

Kamaki help is used to list available commands with description, syntax and

corresponding optional arguments.

To see the command groups, use -h or --help like in example 1.3.1. In the same

way, help information for command groups and commands is printed. In the

following examples, the help messages of kamaki, of a command group (server)

and of a command in that group (list) are shown.

        [-v] [-s] [-V] [-d] [-i] [-c CONFIG] [-o OPTIONS] [--cloud CLOUD] [-h]

      -c CONFIG, --config CONFIG

                            Path to configuration file

      --cloud CLOUD         Chose a remote cloud to connect to

    config  :  Kamaki configurations

    file    :  Pithos+/Storage API commands

    flavor  :  Cyclades/Compute API flavor commands

    history :  Kamaki command history

    image   :  Cyclades/Plankton API image commands

    image compute:  Cyclades/Compute API image commands

    network :  Cyclades/Compute API network commands

    server  :  Cyclades/Compute API server commands

    user    :  Astakos API commands

    :emphasize-lines: 1,2

    Example 3.1.2: Cyclades help contains all first-level commands of Cyclades

    command group

    usage: kamaki server <...> [-v] [-s] [-V] [-d] [-i] [-c CONFIG]

                               [-o OPTIONS] [--cloud CLOUD] [-h]

      -c CONFIG, --config CONFIG

                            Path to configuration file

      --cloud CLOUD         Chose a remote cloud to connect to

    addr    :  List the addresses of all network interfaces on a server (VM)

    console :  Get a VNC console to access an existing server (VM)

    create  :  Create a server (aka Virtual Machine)

    delete  :  Delete a server (VM)

    firewall:  Manage server (VM) firewall profiles for public networks

    info    :  Detailed information on a Virtual Machine

    list    :  List Virtual Machines accessible by user

    metadata:  Manage Server metadata (key:value pairs of server attributes)

    reboot  :  Reboot a server (VM)

    rename  :  Set/update a server (VM) name

    shutdown:  Shutdown an active server (VM)

    start   :  Start an existing server (VM)

    stats   :  Get server (VM) statistics

    :emphasize-lines: 1,2

    Example 3.1.3: Help for command "server list" with syntax, description and

    available user options

    usage: kamaki server list [-v] [-s] [-V] [-d] [-i] [-c CONFIG] [-o OPTIONS]

                              [--cloud CLOUD] [-h] [--since SINCE] [--enumerate]

                              [-l] [--more] [-n LIMIT] [-j]

    List Virtual Machines accessible by user

    User Authentication:    

    * to check authentication: /user authenticate    

    * to set authentication token: /config set remote.default.token <token>

    -v, --verbose         More info at response

    -s, --silent          Do not output anything

    -V, --version         Print current version

    -d, --debug           Include debug output

    -i, --include         Include raw connection data in the output

    -c CONFIG, --config CONFIG

                          Path to configuration file

    -o OPTIONS, --options OPTIONS

                          Override a config value

    --cloud CLOUD         Chose a remote cloud to connect to

    -h, --help            Show help message

    --since SINCE         show only items since date (' d/m/Y H:M:S ')

    --enumerate           Enumerate results

    -l, --details         show detailed output

    --more                output results in pages (-n to set items per page,

                          default 10)

    -n LIMIT, --number LIMIT

                          limit number of listed VMs

    -j, --json            show headers in json

Debug and logging

^^^^^^^^^^^^^^^^^

"""""

In case of errors, kamaki in debug mode shows useful debug information, like

the stack trace. Kamaki in debug mode cancels suppression of warning messages.

To run kamaki in debug mode use the -d or --debug option (can be combined with

any other parameters or options)

Logging

"""""""

Kamaki keeps its logs in a file specified as global.log_file and its value

defaults to ${HOME}/.kamaki.log . This value may change with a config setting::

    kamaki config set log_file /new/log/file/path

Kamaki logs mostly the http connection requests and responses, including http

methods, urls, parameters and headers. There is some special handling in two

cases:

* HTTP bodies are not logged by default

    to enable logging the full http bodies, set log_data to `on`::

        kamaki config set log_data on

    to disable it, set it to `off`::

        kamaki config set log_data off

    or delete it::

        kamaki config delete log_data

* X-Auth-Token header is not logged by default

    to enable logging the X-Auth-Token header, set log_token to `on`::

        kamaki config set log_token on

    to disable it, set it to `off`::

        kamaki config set log_token off

    or delete it::

        kamaki config delete log_token

Most kamaki commands are translated into http requests. Kamaki clients API

translated the semantics to REST and handles the response. Users who need to

have access to these commands can use the verbose mode that presents the HTTP

Request details as well as the full server response.

Be default, kamaki in verbose mode prints down only the headers and the address

information, thus hiding the data body of the request or response. To see the

data body, the -i option can be used.

    Example 3.4.1: List the trash container contents, containing c1_

    $ kamaki file list -o global.pithos_container=trash| grep c1_

    c1_1370859409.0 20KB

    c1_1370859414.0 9MB

    c1_1370859409.1 110B

The -o argument can be used to temporarily override various (set or unset)

options. In one command, all -o option sets are forgotten just after the

command has been completed, and the previous settings are restored (the

configuration file is not modified).

The file-list command in example 3.4.1 runs with an explicitly provided

pithos_account, which temporarily overrides the one probably provided in the

configuration file (it works even if the user has not set the optional

pithos_account config option).

The kamaki interactive shell implements the notion of command contexts. Each

command group is also a context where the users can **enter** by typing the

group name. If the context switch is successful, the kamaki shell prompt

changes to present the new context ("file" in example 4.1.1).

Type **exit** (alternatively **ctrl-D** in (X)nix systems or **ctrl-Z** in

Windows) to exit a context and return to the context of origin. If already at

the top context (kamaki), an exit is equivalent to exiting the program.

Users have the option to avoid switching between contexts: all commands can run

from the **top context**. As a result, examples 4.1.3 and 4.1.4 are equivalent.

**Context-level help** lists the available commands in a context and can also

offer a short description for each command.

The context-level help results may change from context to context

    Example 4.2.1: Get available commands and then get help in a context

In context-level, there is a distinction between kamaki-commands and

interactive shell commands. The former are available in one-command mode and

are related to the cloud client setup and use, while the later are

context-shell functions.

**Command-level help** prints the syntax, arguments and description of a

specific (terminal) command

Command-level help mechanism is exactly the same as the one used in

one-command mode. For example, it is invoked by using the -h or --help

parameter at any point.

When working in a context, it is often useful to access other contexts or

top-level commands. Kamaki offers access to top-level commands by using the

`/` prefix, as shown bellow::

An example (4.3.1) that showcases how top-level access improves user experience

is the creation of a VM. A VM is created with the command server-create. This

command is called with three parameters:

It is often the case that a user who works in the context command, needs to

create a new VM, but hasn't selected a flavor or an image id, or cannot recall

the id of that flavor or image. Therefore, it is necessary to list all

available flavors (flavor-list) or images (image-compute-list). Both commands

belong to different contexts.

An other example (4.3.2) showcases how to acquire and modify configuration

settings from a different context. In this scenario, the user token expires at

server side while the user is working. When that happens, the system responds

with an *(401) UNAUTHORIZED* message. The user can acquire a new token (valid

for the Astakos identity manager of preference) which has to be set to kamaki.

    [file]: /config get remote.default.token

    [file]: /config set remote.default.token myfr35ht0k3n==

    [file]: /config get remote.default

    remote.default.url = https://astakos.example.com/astakos/identity/2.0/

    remote.default.token = myfr35ht0k3n==

.. note:: The error messages on this example where shortened for clarity.

Actual kamaki error messages are more helpful and descriptive.

The following example compares some equivalent calls that run

*user-authenticate* after a *file-list* 401 failure.

    Example 4.3.3: Equivalent user-authenticate calls after a file-list 401

The configuration mechanism of kamaki is detailed at the

`setup section <setup.html>`_ and it is common for both interaction modes. In

specific, the configuration mechanism is implemented as a command group, namely

`config`. Using the config commands is as straightforward as any other kamaki

commands.

It is often useful to set, delete or update a value. This can be managed either

inside the config context or from any command context by using the / prefix.

.. Note:: config updates in kamaki shell persist even after the session is over

All setting changes affect the physical kamaki config file. The config file is

created automatically at callers' home firectory the first time a config option

is set, and lives there as *.kamakirc* . It can be edited with any text editor

or managed with kamaki config commands.

In example 4.4.1 the user is going to work with only one storage container. The

file commands use the container:path syntax, but if the user sets a container

name as default, the container name can be omitted. This is possible by setting

a *file.container* setting.

    [file]: /config set pithos_container mycontainer

After a while, the user needs to work with multiple containers, therefore a

default container is no longer needed. The *pithos_container* setting can be

deleted, as shown in example 4.4.2

    [file]: /config delete pithos_container

There are two history modes: session and permanent. Session history keeps

record of all actions in a kamaki shell session, while permanent history

appends all commands to an accessible history file.

Session history is only available in interactive shell mode. Users can iterate

through past commands in the same session with the ↑ and ↓ keys. Session

history is not stored, although syntactically correct commands are recorded

through the permanent history mechanism.

Permanent history is implemented as a command group and is common to both the

one-command and shell interfaces. In specific, every syntactically correct

command is appended in a history file (configured as `history_file` in

settings, see `setup section <setup.html>`_ for details). Commands executed in

one-command mode are mixed with the ones run in kamaki shell (also

see :ref:`using-history-ref` section on this guide).

The history-run feature allows the sequential run of previous command

executions in kamaki shell.

The following sequence copies and downloads a file from *mycontainer1* ,

uploads it to *mycontainer2* , then undo the proccess and repeats it with

history-run

For powerfull scripting, users are advised to take advantage of their os shell

scripting capabilities and combine them with kamaki one-command. Still, the

history-run functionality might prove handy in many occasions.

Kamaki shell features tab completion for the first level of command terms of

the current context. Tab completion pool changes dynamically when the context

is switched. Currently, tab completion is not supported when / is used

(see :ref:`accessing-top-level-commands-ref` ).

Kamaki shell features the ability to execute OS-shell commands from any

context. This can be achieved by typing *!* or *shell*::

Kamaki shell commits command strings to the outside shell and prints the

results, without interacting with it. After a command is finished, kamaki shell

returns to its initial state, which involves the current directory, as show in

example 4.8.2

        stdout.write('Create (overwrite) file %s ? [y/N] ' % cnf.path)