Archipelago » qemu

= How to use the QAPI code generator =

* Note: as of this writing, QMP does not use QAPI. Eventually QMP

commands will be converted to use QAPI internally. The following

information describes QMP/QAPI as it will exist after the

conversion.

QAPI is a native C API within QEMU which provides management-level

functionality to internal/external users. For external

users/processes, this interface is made available by a JSON-based

QEMU Monitor protocol that is provided by the QMP server.

To map QMP-defined interfaces to the native C QAPI implementations,

a JSON-based schema is used to define types and function

signatures, and a set of scripts is used to generate types/signatures,

and marshaling/dispatch code. The QEMU Guest Agent also uses these

scripts, paired with a separate schema, to generate

marshaling/dispatch code for the guest agent server running in the

guest.

This document will describe how the schemas, scripts, and resulting

code is used.

== QMP/Guest agent schema ==

This file defines the types, commands, and events used by QMP.  It should

fully describe the interface used by QMP.

This file is designed to be loosely based on JSON although it's technically

executable Python.  While dictionaries are used, they are parsed as

OrderedDicts so that ordering is preserved.

There are two basic syntaxes used, type definitions and command definitions.

The first syntax defines a type and is represented by a dictionary.  There are

three kinds of user-defined types that are supported: complex types,

enumeration types and union types.

Generally speaking, types definitions should always use CamelCase for the type

names. Command names should be all lower case with words separated by a hyphen.

=== Complex types ===

A complex type is a dictionary containing a single key whose value is a

dictionary.  This corresponds to a struct in C or an Object in JSON.  An

example of a complex type is:

 { 'type': 'MyType',

   'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }

The use of '*' as a prefix to the name means the member is optional.  Optional

members should always be added to the end of the dictionary to preserve

backwards compatibility.

=== Enumeration types ===

An enumeration type is a dictionary containing a single key whose value is a

list of strings.  An example enumeration is:

 { 'enum': 'MyEnum', 'data': [ 'value1', 'value2', 'value3' ] }

=== Union types ===

Union types are used to let the user choose between several different data

types.  A union type is defined using a dictionary as explained in the

following paragraphs.

A simple union type defines a mapping from discriminator values to data types

like in this example:

 { 'type': 'FileOptions', 'data': { 'filename': 'str' } }

 { 'type': 'Qcow2Options',

   'data': { 'backing-file': 'str', 'lazy-refcounts': 'bool' } }

 { 'union': 'BlockdevOptions',

   'data': { 'file': 'FileOptions',

             'qcow2': 'Qcow2Options' } }

In the QMP wire format, a simple union is represented by a dictionary that

contains the 'type' field as a discriminator, and a 'data' field that is of the

specified data type corresponding to the discriminator value:

 { "type": "qcow2", "data" : { "backing-file": "/some/place/my-image",

                               "lazy-refcounts": true } }

A union definition can specify a complex type as its base. In this case, the

fields of the complex type are included as top-level fields of the union

dictionary in the QMP wire format. An example definition is:

 { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }

 { 'union': 'BlockdevOptions',

   'base': 'BlockdevCommonOptions',

   'data': { 'raw': 'RawOptions',

             'qcow2': 'Qcow2Options' } }

And it looks like this on the wire:

 { "type": "qcow2",

   "readonly": false,

   "data" : { "backing-file": "/some/place/my-image",

              "lazy-refcounts": true } }

Flat union types avoid the nesting on the wire. They are used whenever a

specific field of the base type is declared as the discriminator ('type' is

then no longer generated). The discriminator must always be a string field.

The above example can then be modified as follows:

 { 'type': 'BlockdevCommonOptions',

   'data': { 'driver': 'str', 'readonly': 'bool' } }

 { 'union': 'BlockdevOptions',

   'base': 'BlockdevCommonOptions',

   'discriminator': 'driver',

   'data': { 'raw': 'RawOptions',

             'qcow2': 'Qcow2Options' } }

Resulting in this JSON object:

 { "driver": "qcow2",

   "readonly": false,

   "backing-file": "/some/place/my-image",

   "lazy-refcounts": true }

A special type of unions are anonymous unions. They don't form a dictionary in

the wire format but allow the direct use of different types in their place. As

they aren't structured, they don't have any explicit discriminator but use

the (QObject) data type of their value as an implicit discriminator. This means

that they are restricted to using only one discriminator value per QObject

type. For example, you cannot have two different complex types in an anonymous

union, or two different integer types.

Anonymous unions are declared using an empty dictionary as their discriminator.

The discriminator values never appear on the wire, they are only used in the

generated C code. Anonymous unions cannot have a base type.

 { 'union': 'BlockRef',

   'discriminator': {},

   'data': { 'definition': 'BlockdevOptions',

             'reference': 'str' } }

This example allows using both of the following example objects:

 { "file": "my_existing_block_device_id" }

 { "file": { "driver": "file",

             "readonly": false,

             'filename': "/tmp/mydisk.qcow2" } }

=== Commands ===

Commands are defined by using a list containing three members.  The first

member is the command name, the second member is a dictionary containing

arguments, and the third member is the return type.

An example command is:

 { 'command': 'my-command',

   'data': { 'arg1': 'str', '*arg2': 'str' },

   'returns': 'str' }

== Code generation ==

Schemas are fed into 3 scripts to generate all the code/files that, paired

with the core QAPI libraries, comprise everything required to take JSON

commands read in by a QMP/guest agent server, unmarshal the arguments into

the underlying C types, call into the corresponding C function, and map the

response back to a QMP/guest agent response to be returned to the user.

As an example, we'll use the following schema, which describes a single

complex user-defined type (which will produce a C struct, along with a list

node structure that can be used to chain together a list of such types in

case we want to accept/return a list of this type with a command), and a

command which takes that type as a parameter and returns the same type:

    mdroth@illuin:~/w/qemu2.git$ cat example-schema.json

    { 'type': 'UserDefOne',

      'data': { 'integer': 'int', 'string': 'str' } }

    { 'command': 'my-command',

      'data':    {'arg1': 'UserDefOne'},

      'returns': 'UserDefOne' }

    mdroth@illuin:~/w/qemu2.git$

=== scripts/qapi-types.py ===

Used to generate the C types defined by a schema. The following files are

created:

$(prefix)qapi-types.h - C types corresponding to types defined in

                        the schema you pass in

$(prefix)qapi-types.c - Cleanup functions for the above C types

The $(prefix) is an optional parameter used as a namespace to keep the

generated code from one schema/code-generation separated from others so code

can be generated/used from multiple schemas without clobbering previously

created code.

Example:

    mdroth@illuin:~/w/qemu2.git$ python scripts/qapi-types.py \

      --output-dir="qapi-generated" --prefix="example-" < example-schema.json

    mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qapi-types.c

    /* AUTOMATICALLY GENERATED, DO NOT MODIFY */

    #include "qapi/qapi-dealloc-visitor.h"

    #include "example-qapi-types.h"

    #include "example-qapi-visit.h"

    void qapi_free_UserDefOne(UserDefOne * obj)

    {

        QapiDeallocVisitor *md;

        Visitor *v;

        if (!obj) {

            return;

        }

        md = qapi_dealloc_visitor_new();

        v = qapi_dealloc_get_visitor(md);

        visit_type_UserDefOne(v, &obj, NULL, NULL);

        qapi_dealloc_visitor_cleanup(md);

    }

    mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qapi-types.h

    /* AUTOMATICALLY GENERATED, DO NOT MODIFY */

    #ifndef QAPI_GENERATED_EXAMPLE_QAPI_TYPES

    #define QAPI_GENERATED_EXAMPLE_QAPI_TYPES

    #include "qapi/qapi-types-core.h"

    typedef struct UserDefOne UserDefOne;

    typedef struct UserDefOneList

    {

        UserDefOne *value;

        struct UserDefOneList *next;

    } UserDefOneList;

    struct UserDefOne

    {

        int64_t integer;

        char * string;

    };

    void qapi_free_UserDefOne(UserDefOne * obj);

    #endif

=== scripts/qapi-visit.py ===

Used to generate the visitor functions used to walk through and convert

a QObject (as provided by QMP) to a native C data structure and

vice-versa, as well as the visitor function used to dealloc a complex

schema-defined C type.

The following files are generated:

$(prefix)qapi-visit.c: visitor function for a particular C type, used

                       to automagically convert QObjects into the

                       corresponding C type and vice-versa, as well

                       as for deallocating memory for an existing C

                       type

$(prefix)qapi-visit.h: declarations for previously mentioned visitor

                       functions

Example:

    mdroth@illuin:~/w/qemu2.git$ python scripts/qapi-visit.py \

        --output-dir="qapi-generated" --prefix="example-" < example-schema.json

    mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qapi-visit.c

    /* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */

    #include "example-qapi-visit.h"

    void visit_type_UserDefOne(Visitor *m, UserDefOne ** obj, const char *name, Error **errp)

    {

        visit_start_struct(m, (void **)obj, "UserDefOne", name, sizeof(UserDefOne), errp);

        visit_type_int(m, (obj && *obj) ? &(*obj)->integer : NULL, "integer", errp);

        visit_type_str(m, (obj && *obj) ? &(*obj)->string : NULL, "string", errp);

        visit_end_struct(m, errp);

    }

    void visit_type_UserDefOneList(Visitor *m, UserDefOneList ** obj, const char *name, Error **errp)

    {

        GenericList *i, **prev = (GenericList **)obj;

        visit_start_list(m, name, errp);

        for (; (i = visit_next_list(m, prev, errp)) != NULL; prev = &i) {

            UserDefOneList *native_i = (UserDefOneList *)i;

            visit_type_UserDefOne(m, &native_i->value, NULL, errp);

        }

        visit_end_list(m, errp);

    }

    mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qapi-visit.h

    /* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */

    #ifndef QAPI_GENERATED_EXAMPLE_QAPI_VISIT

    #define QAPI_GENERATED_EXAMPLE_QAPI_VISIT

    #include "qapi/qapi-visit-core.h"

    #include "example-qapi-types.h"

    void visit_type_UserDefOne(Visitor *m, UserDefOne ** obj, const char *name, Error **errp);

    void visit_type_UserDefOneList(Visitor *m, UserDefOneList ** obj, const char *name, Error **errp);

    #endif

    mdroth@illuin:~/w/qemu2.git$

(The actual structure of the visit_type_* functions is a bit more complex

in order to propagate errors correctly and avoid leaking memory).

=== scripts/qapi-commands.py ===

Used to generate the marshaling/dispatch functions for the commands defined

in the schema. The following files are generated:

$(prefix)qmp-marshal.c: command marshal/dispatch functions for each

                        QMP command defined in the schema. Functions

                        generated by qapi-visit.py are used to

                        convert QObjects received from the wire into

                        function parameters, and uses the same

                        visitor functions to convert native C return

                        values to QObjects from transmission back

                        over the wire.

$(prefix)qmp-commands.h: Function prototypes for the QMP commands

                         specified in the schema.

Example:

    mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qmp-marshal.c

    /* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */

    #include "qemu-objects.h"

    #include "qapi/qmp-core.h"

    #include "qapi/qapi-visit-core.h"

    #include "qapi/qmp-output-visitor.h"

    #include "qapi/qmp-input-visitor.h"

    #include "qapi/qapi-dealloc-visitor.h"

    #include "example-qapi-types.h"

    #include "example-qapi-visit.h"

    #include "example-qmp-commands.h"

    static void qmp_marshal_output_my_command(UserDefOne * ret_in, QObject **ret_out, Error **errp)

    {

        QapiDeallocVisitor *md = qapi_dealloc_visitor_new();

        QmpOutputVisitor *mo = qmp_output_visitor_new();

        Visitor *v;

        v = qmp_output_get_visitor(mo);

        visit_type_UserDefOne(v, &ret_in, "unused", errp);

        v = qapi_dealloc_get_visitor(md);

        visit_type_UserDefOne(v, &ret_in, "unused", errp);

        qapi_dealloc_visitor_cleanup(md);

        *ret_out = qmp_output_get_qobject(mo);

    }

    static void qmp_marshal_input_my_command(QmpState *qmp__sess, QDict *args, QObject **ret, Error **errp)

    {

        UserDefOne * retval = NULL;

        QmpInputVisitor *mi;

        QapiDeallocVisitor *md;

        Visitor *v;

        UserDefOne * arg1 = NULL;

        mi = qmp_input_visitor_new(QOBJECT(args));

        v = qmp_input_get_visitor(mi);

        visit_type_UserDefOne(v, &arg1, "arg1", errp);

        if (error_is_set(errp)) {

            goto out;

        }

        retval = qmp_my_command(arg1, errp);

        qmp_marshal_output_my_command(retval, ret, errp);

    out:

        md = qapi_dealloc_visitor_new();

        v = qapi_dealloc_get_visitor(md);

        visit_type_UserDefOne(v, &arg1, "arg1", errp);

        qapi_dealloc_visitor_cleanup(md);

        return;

    }

    static void qmp_init_marshal(void)

    {

        qmp_register_command("my-command", qmp_marshal_input_my_command);

    }

    qapi_init(qmp_init_marshal);

    mdroth@illuin:~/w/qemu2.git$ cat qapi-generated/example-qmp-commands.h

    /* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */

    #ifndef QAPI_GENERATED_EXAMPLE_QMP_COMMANDS

    #define QAPI_GENERATED_EXAMPLE_QMP_COMMANDS

    #include "example-qapi-types.h"

    #include "error.h"

    UserDefOne * qmp_my_command(UserDefOne * arg1, Error **errp);

    #endif

    mdroth@illuin:~/w/qemu2.git$