Archipelago » qemu

All the files in this directory and subdirectories are released under

a BSD like license (see header in each file). No other license is

accepted.

Tiny Code Generator - Fabrice Bellard.

1) Introduction

TCG (Tiny Code Generator) began as a generic backend for a C

compiler. It was simplified to be used in QEMU. It also has its roots

in the QOP code generator written by Paul Brook. 

2) Definitions

The TCG "target" is the architecture for which we generate the

code. It is of course not the same as the "target" of QEMU which is

the emulated architecture. As TCG started as a generic C backend used

for cross compiling, it is assumed that the TCG target is different

from the host, although it is never the case for QEMU.

A TCG "function" corresponds to a QEMU Translated Block (TB).

A TCG "temporary" is a variable only live in a given

function. Temporaries are allocated explicitely in each function.

A TCG "global" is a variable which is live in all the functions. They

are defined before the functions defined. A TCG global can be a memory

location (e.g. a QEMU CPU register), a fixed host register (e.g. the

QEMU CPU state pointer) or a memory location which is stored in a

register outside QEMU TBs (not implemented yet).

A TCG "basic block" corresponds to a list of instructions terminated

by a branch instruction. 

3) Intermediate representation

3.1) Introduction

TCG instructions operate on variables which are temporaries or

globals. TCG instructions and variables are strongly typed. Two types

are supported: 32 bit integers and 64 bit integers. Pointers are

defined as an alias to 32 bit or 64 bit integers depending on the TCG

target word size.

Each instruction has a fixed number of output variable operands, input

variable operands and always constant operands.

The notable exception is the call instruction which has a variable

number of outputs and inputs.

In the textual form, output operands come first, followed by input

operands, followed by constant operands. The output type is included

in the instruction name. Constants are prefixed with a '$'.

add_i32 t0, t1, t2  (t0 <- t1 + t2)

sub_i64 t2, t3, $4  (t2 <- t3 - 4)

3.2) Assumptions

* Basic blocks

- Basic blocks end after branches (e.g. brcond_i32 instruction),

  goto_tb and exit_tb instructions.

- Basic blocks end before legacy dyngen operations.

- Basic blocks start after the end of a previous basic block, at a

  set_label instruction or after a legacy dyngen operation.

After the end of a basic block, temporaries at destroyed and globals

are stored at their initial storage (register or memory place

depending on their declarations).

* Floating point types are not supported yet

* Pointers: depending on the TCG target, pointer size is 32 bit or 64

  bit. The type TCG_TYPE_PTR is an alias to TCG_TYPE_I32 or

  TCG_TYPE_I64.

* Helpers:

Using the tcg_gen_helper_x_y it is possible to call any function

taking i32, i64 or pointer types types. Before calling an helper, all

globals are stored at their canonical location and it is assumed that

the function can modify them. In the future, function modifiers will

be allowed to tell that the helper does not read or write some globals.

On some TCG targets (e.g. x86), several calling conventions are

supported.

* Branches:

Use the instruction 'br' to jump to a label. Use 'jmp' to jump to an

explicit address. Conditional branches can only jump to labels.

3.3) Code Optimizations

When generating instructions, you can count on at least the following

optimizations:

- Single instructions are simplified, e.g.

   and_i32 t0, t0, $0xffffffff

  is suppressed.

- A liveness analysis is done at the basic block level. The

  information is used to suppress moves from a dead temporary to

  another one. It is also used to remove instructions which compute

  dead results. The later is especially useful for condition code

  optimisation in QEMU.

  In the following example:

  add_i32 t0, t1, t2

  add_i32 t0, t0, $1

  mov_i32 t0, $1

  only the last instruction is kept.

- A macro system is supported (may get closer to function inlining

  some day). It is useful if the liveness analysis is likely to prove

  that some results of a computation are indeed not useful. With the

  macro system, the user can provide several alternative

  implementations which are used depending on the used results. It is

  especially useful for condition code optimisation in QEMU.

  Here is an example:

  macro_2 t0, t1, $1

  mov_i32 t0, $0x1234

  The macro identified by the ID "$1" normally returns the values t0

  and t1. Suppose its implementation is:

  macro_start

  brcond_i32  t2, $0, $TCG_COND_EQ, $1

  mov_i32 t0, $2

  br $2

  set_label $1

  mov_i32 t0, $3

  set_label $2

  add_i32 t1, t3, t4

  macro_end

  If t0 is not used after the macro, the user can provide a simpler

  implementation:

  macro_start

  add_i32 t1, t2, t4

  macro_end

  TCG automatically chooses the right implementation depending on

  which macro outputs are used after it.

  Note that if TCG did more expensive optimizations, macros would be

  less useful. In the previous example a macro is useful because the

  liveness analysis is done on each basic block separately. Hence TCG

  cannot remove the code computing 't0' even if it is not used after

  the first macro implementation.

3.4) Instruction Reference

********* Function call

* call <ret> <params> ptr

call function 'ptr' (pointer type)

<ret> optional 32 bit or 64 bit return value

<params> optional 32 bit or 64 bit parameters

********* Jumps/Labels

* jmp t0

Absolute jump to address t0 (pointer type).

* set_label $label

Define label 'label' at the current program point.

* br $label

Jump to label.

* brcond_i32/i64 cond, t0, t1, label

Conditional jump if t0 cond t1 is true. cond can be:

    TCG_COND_EQ

    TCG_COND_NE

    TCG_COND_LT /* signed */

    TCG_COND_GE /* signed */

    TCG_COND_LE /* signed */

    TCG_COND_GT /* signed */

    TCG_COND_LTU /* unsigned */

    TCG_COND_GEU /* unsigned */

    TCG_COND_LEU /* unsigned */

    TCG_COND_GTU /* unsigned */

********* Arithmetic

* add_i32/i64 t0, t1, t2

t0=t1+t2

* sub_i32/i64 t0, t1, t2

t0=t1-t2

* mul_i32/i64 t0, t1, t2

t0=t1*t2

* div_i32/i64 t0, t1, t2

t0=t1/t2 (signed). Undefined behavior if division by zero or overflow.

* divu_i32/i64 t0, t1, t2

t0=t1/t2 (unsigned). Undefined behavior if division by zero.

* rem_i32/i64 t0, t1, t2

t0=t1%t2 (signed). Undefined behavior if division by zero or overflow.

* remu_i32/i64 t0, t1, t2

t0=t1%t2 (unsigned). Undefined behavior if division by zero.

* and_i32/i64 t0, t1, t2

********* Logical

t0=t1&t2

* or_i32/i64 t0, t1, t2

t0=t1|t2

* xor_i32/i64 t0, t1, t2

t0=t1^t2

* shl_i32/i64 t0, t1, t2

********* Shifts

* shl_i32/i64 t0, t1, t2

t0=t1 << t2. Undefined behavior if t2 < 0 or t2 >= 32 (resp 64)

* shr_i32/i64 t0, t1, t2

t0=t1 >> t2 (unsigned). Undefined behavior if t2 < 0 or t2 >= 32 (resp 64)

* sar_i32/i64 t0, t1, t2

t0=t1 >> t2 (signed). Undefined behavior if t2 < 0 or t2 >= 32 (resp 64)

********* Misc

* mov_i32/i64 t0, t1

t0 = t1

Move t1 to t0 (both operands must have the same type).

* ext8s_i32/i64 t0, t1

ext16s_i32/i64 t0, t1

ext32s_i64 t0, t1

8, 16 or 32 bit sign extension (both operands must have the same type)

* bswap16_i32 t0, t1

16 bit byte swap on a 32 bit value. The two high order bytes must be set

to zero.

* bswap_i32 t0, t1

32 bit byte swap

* bswap_i64 t0, t1

64 bit byte swap

********* Type conversions

* ext_i32_i64 t0, t1

Convert t1 (32 bit) to t0 (64 bit) and does sign extension

* extu_i32_i64 t0, t1

Convert t1 (32 bit) to t0 (64 bit) and does zero extension

* trunc_i64_i32 t0, t1

Truncate t1 (64 bit) to t0 (32 bit)

********* Load/Store

* ld_i32/i64 t0, t1, offset

ld8s_i32/i64 t0, t1, offset

ld8u_i32/i64 t0, t1, offset

ld16s_i32/i64 t0, t1, offset

ld16u_i32/i64 t0, t1, offset

ld32s_i64 t0, t1, offset

ld32u_i64 t0, t1, offset

t0 = read(t1 + offset)

Load 8, 16, 32 or 64 bits with or without sign extension from host memory. 

offset must be a constant.

* st_i32/i64 t0, t1, offset

st8_i32/i64 t0, t1, offset

st16_i32/i64 t0, t1, offset

st32_i64 t0, t1, offset

write(t0, t1 + offset)

Write 8, 16, 32 or 64 bits to host memory.

********* QEMU specific operations

* tb_exit t0

Exit the current TB and return the value t0 (word type).

* goto_tb index

Exit the current TB and jump to the TB index 'index' (constant) if the

current TB was linked to this TB. Otherwise execute the next

instructions.

* qemu_ld_i32/i64 t0, t1, flags

qemu_ld8u_i32/i64 t0, t1, flags

qemu_ld8s_i32/i64 t0, t1, flags

qemu_ld16u_i32/i64 t0, t1, flags

qemu_ld16s_i32/i64 t0, t1, flags

qemu_ld32u_i64 t0, t1, flags

qemu_ld32s_i64 t0, t1, flags

Load data at the QEMU CPU address t1 into t0. t1 has the QEMU CPU

address type. 'flags' contains the QEMU memory index (selects user or

kernel access) for example.

* qemu_st_i32/i64 t0, t1, flags

qemu_st8_i32/i64 t0, t1, flags

qemu_st16_i32/i64 t0, t1, flags

qemu_st32_i64 t0, t1, flags

Store the data t0 at the QEMU CPU Address t1. t1 has the QEMU CPU

address type. 'flags' contains the QEMU memory index (selects user or

kernel access) for example.

Note 1: Some shortcuts are defined when the last operand is known to be

a constant (e.g. addi for add, movi for mov).

Note 2: When using TCG, the opcodes must never be generated directly

as some of them may not be available as "real" opcodes. Always use the

function tcg_gen_xxx(args).

4) Backend

tcg-target.h contains the target specific definitions. tcg-target.c

contains the target specific code.

4.1) Assumptions

The target word size (TCG_TARGET_REG_BITS) is expected to be 32 bit or

64 bit. It is expected that the pointer has the same size as the word.

On a 32 bit target, all 64 bit operations are converted to 32 bits. A

few specific operations must be implemented to allow it (see add2_i32,

sub2_i32, brcond2_i32).

Floating point operations are not supported in this version. A

previous incarnation of the code generator had full support of them,

but it is better to concentrate on integer operations first.

On a 64 bit target, no assumption is made in TCG about the storage of

the 32 bit values in 64 bit registers.

4.2) Constraints

GCC like constraints are used to define the constraints of every

instruction. Memory constraints are not supported in this

version. Aliases are specified in the input operands as for GCC.

A target can define specific register or constant constraints. If an

operation uses a constant input constraint which does not allow all

constants, it must also accept registers in order to have a fallback.

The movi_i32 and movi_i64 operations must accept any constants.

The mov_i32 and mov_i64 operations must accept any registers of the

same type.

The ld/st instructions must accept signed 32 bit constant offsets. It

can be implemented by reserving a specific register to compute the

address if the offset is too big.

The ld/st instructions must accept any destination (ld) or source (st)

register.

4.3) Function call assumptions

- The only supported types for parameters and return value are: 32 and

  64 bit integers and pointer.

- The stack grows downwards.

- The first N parameters are passed in registers.

- The next parameters are passed on the stack by storing them as words.

- Some registers are clobbered during the call. 

- The function can return 0 or 1 value in registers. On a 32 bit

  target, functions must be able to return 2 values in registers for

  64 bit return type.

5) Migration from dyngen to TCG

TCG is backward compatible with QEMU "dyngen" operations. It means

that TCG instructions can be freely mixed with dyngen operations. It

is expected that QEMU targets will be progressively fully converted to

TCG. Once a target is fully converted to dyngen, it will be possible

to apply more optimizations because more registers will be free for

the generated code.

The exception model is the same as the dyngen one.

- test macro system

- test conditional jumps

- test mul, div, ext8s, ext16s, bswap

- generate a global TB prologue and epilogue to save/restore registers

  to/from the CPU state and to reserve a stack frame to optimize

  helper calls. Modify cpu-exec.c so that it does not use global

  register variables (except maybe for 'env').

- fully convert the x86 target. The minimal amount of work includes:

  - add cc_src, cc_dst and cc_op as globals

  - disable its eflags optimization (the liveness analysis should

    suffice)

  - move complicated operations to helpers (in particular FPU, SSE, MMX).

- optimize the x86 target:

  - move some or all the registers as globals

  - use the TB prologue and epilogue to have QEMU target registers in

    pre assigned host registers.

Ideas:

- Move the slow part of the qemu_ld/st ops after the end of the TB.

- Experiment: change instruction storage to simplify macro handling

  and to handle dynamic allocation and see if the translation speed is

  OK.

- change exception syntax to get closer to QOP system (exception

  parameters given with a specific instruction).

/*

 * Tiny Code Generator for QEMU

 *

 * Copyright (c) 2008 Fabrice Bellard

 *

 * Permission is hereby granted, free of charge, to any person obtaining a copy

 * of this software and associated documentation files (the "Software"), to deal

 * in the Software without restriction, including without limitation the rights

 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

 * copies of the Software, and to permit persons to whom the Software is

 * furnished to do so, subject to the following conditions:

 *

 * The above copyright notice and this permission notice shall be included in

 * all copies or substantial portions of the Software.

 *

 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN

 * THE SOFTWARE.

 */

const char *tcg_target_reg_names[TCG_TARGET_NB_REGS] = {

    "%eax",

    "%ecx",

    "%edx",

    "%ebx",

    "%esp",

    "%ebp",

    "%esi",

    "%edi",

};

int tcg_target_reg_alloc_order[TCG_TARGET_NB_REGS] = {

    TCG_REG_EAX,

    TCG_REG_EDX,

    TCG_REG_ECX,

    TCG_REG_EBX,

    TCG_REG_ESI,

    TCG_REG_EDI,

    TCG_REG_EBP,

    TCG_REG_ESP,

};

const int tcg_target_call_iarg_regs[3] = { TCG_REG_EAX, TCG_REG_EDX, TCG_REG_ECX };

const int tcg_target_call_oarg_regs[2] = { TCG_REG_EAX, TCG_REG_EDX };

static void patch_reloc(uint8_t *code_ptr, int type, 

                        tcg_target_long value)

{

    switch(type) {

    case R_386_32:

        *(uint32_t *)code_ptr = value;

        break;

    case R_386_PC32:

        *(uint32_t *)code_ptr = value - (long)code_ptr;

        break;

    default:

        tcg_abort();

    }

}

/* maximum number of register used for input function arguments */

static inline int tcg_target_get_call_iarg_regs_count(int flags)

{

    flags &= TCG_CALL_TYPE_MASK;

    switch(flags) {

    case TCG_CALL_TYPE_STD:

        return 0;

    case TCG_CALL_TYPE_REGPARM_1:

    case TCG_CALL_TYPE_REGPARM_2:

    case TCG_CALL_TYPE_REGPARM:

        return flags - TCG_CALL_TYPE_REGPARM_1 + 1;

    default:

        tcg_abort();

    }

}

/* parse target specific constraints */

int target_parse_constraint(TCGArgConstraint *ct, const char **pct_str)

{

    const char *ct_str;

    ct_str = *pct_str;

    switch(ct_str[0]) {

    case 'a':

        ct->ct |= TCG_CT_REG;

        tcg_regset_set_reg(ct->u.regs, TCG_REG_EAX);

        break;

    case 'b':

        ct->ct |= TCG_CT_REG;

        tcg_regset_set_reg(ct->u.regs, TCG_REG_EBX);

        break;

    case 'c':

        ct->ct |= TCG_CT_REG;

        tcg_regset_set_reg(ct->u.regs, TCG_REG_ECX);

        break;

    case 'd':

        ct->ct |= TCG_CT_REG;

        tcg_regset_set_reg(ct->u.regs, TCG_REG_EDX);

        break;

    case 'S':

        ct->ct |= TCG_CT_REG;

        tcg_regset_set_reg(ct->u.regs, TCG_REG_ESI);

        break;

    case 'D':

        ct->ct |= TCG_CT_REG;

        tcg_regset_set_reg(ct->u.regs, TCG_REG_EDI);

        break;

    case 'q':

        ct->ct |= TCG_CT_REG;

        tcg_regset_set32(ct->u.regs, 0, 0xf);

        break;

    case 'r':

        ct->ct |= TCG_CT_REG;

        tcg_regset_set32(ct->u.regs, 0, 0xff);

        break;

        /* qemu_ld/st address constraint */

    case 'L':

        ct->ct |= TCG_CT_REG;

        tcg_regset_set32(ct->u.regs, 0, 0xff);

        tcg_regset_reset_reg(ct->u.regs, TCG_REG_EAX);

        tcg_regset_reset_reg(ct->u.regs, TCG_REG_EDX);

        break;

    default:

        return -1;

    }

    ct_str++;

    *pct_str = ct_str;

    return 0;

}

/* test if a constant matches the constraint */

static inline int tcg_target_const_match(tcg_target_long val,

                                         const TCGArgConstraint *arg_ct)

{

    int ct;

    ct = arg_ct->ct;

    if (ct & TCG_CT_CONST)

        return 1;

    else

        return 0;

}

#define ARITH_ADD 0

#define ARITH_OR  1

#define ARITH_ADC 2

#define ARITH_SBB 3

#define ARITH_AND 4

#define ARITH_SUB 5

#define ARITH_XOR 6

#define ARITH_CMP 7

#define SHIFT_SHL 4

#define SHIFT_SHR 5

#define SHIFT_SAR 7

#define JCC_JMP (-1)

#define JCC_JO  0x0

#define JCC_JNO 0x1

#define JCC_JB  0x2

#define JCC_JAE 0x3

#define JCC_JE  0x4

#define JCC_JNE 0x5

#define JCC_JBE 0x6

#define JCC_JA  0x7

#define JCC_JS  0x8

#define JCC_JNS 0x9

#define JCC_JP  0xa

#define JCC_JNP 0xb

#define JCC_JL  0xc

#define JCC_JGE 0xd

#define JCC_JLE 0xe

#define JCC_JG  0xf

#define P_EXT   0x100 /* 0x0f opcode prefix */

static const uint8_t tcg_cond_to_jcc[10] = {

    [TCG_COND_EQ] = JCC_JE,

    [TCG_COND_NE] = JCC_JNE,

    [TCG_COND_LT] = JCC_JL,

    [TCG_COND_GE] = JCC_JGE,

    [TCG_COND_LE] = JCC_JLE,

    [TCG_COND_GT] = JCC_JG,

    [TCG_COND_LTU] = JCC_JB,

    [TCG_COND_GEU] = JCC_JAE,

    [TCG_COND_LEU] = JCC_JBE,

    [TCG_COND_GTU] = JCC_JA,

};

static inline void tcg_out_opc(TCGContext *s, int opc)

{

    if (opc & P_EXT)

        tcg_out8(s, 0x0f);

    tcg_out8(s, opc);

}

static inline void tcg_out_modrm(TCGContext *s, int opc, int r, int rm)

{

    tcg_out_opc(s, opc);

    tcg_out8(s, 0xc0 | (r << 3) | rm);

}

/* rm == -1 means no register index */

static inline void tcg_out_modrm_offset(TCGContext *s, int opc, int r, int rm, 

                                        int32_t offset)

{

    tcg_out_opc(s, opc);

    if (rm == -1) {

        tcg_out8(s, 0x05 | (r << 3));

        tcg_out32(s, offset);

    } else if (offset == 0 && rm != TCG_REG_EBP) {

        if (rm == TCG_REG_ESP) {

            tcg_out8(s, 0x04 | (r << 3));

            tcg_out8(s, 0x24);

        } else {

            tcg_out8(s, 0x00 | (r << 3) | rm);

        }

    } else if ((int8_t)offset == offset) {

        if (rm == TCG_REG_ESP) {

            tcg_out8(s, 0x44 | (r << 3));

            tcg_out8(s, 0x24);

        } else {

            tcg_out8(s, 0x40 | (r << 3) | rm);

        }

        tcg_out8(s, offset);

    } else {

        if (rm == TCG_REG_ESP) {

            tcg_out8(s, 0x84 | (r << 3));

            tcg_out8(s, 0x24);

        } else {

            tcg_out8(s, 0x80 | (r << 3) | rm);

        }

        tcg_out32(s, offset);

    }

}

static inline void tcg_out_mov(TCGContext *s, int ret, int arg)

{

    if (arg != ret)

        tcg_out_modrm(s, 0x8b, ret, arg);

}

static inline void tcg_out_movi(TCGContext *s, TCGType type,

                                int ret, int32_t arg)

{

    if (arg == 0) {

        /* xor r0,r0 */

        tcg_out_modrm(s, 0x01 | (ARITH_XOR << 3), ret, ret);

    } else {

        tcg_out8(s, 0xb8 + ret);

        tcg_out32(s, arg);

    }

}

static inline void tcg_out_ld(TCGContext *s, int ret, 

                              int arg1, int32_t arg2)

{

    /* movl */

    tcg_out_modrm_offset(s, 0x8b, ret, arg1, arg2);

}

static inline void tcg_out_st(TCGContext *s, int arg, 

                              int arg1, int32_t arg2)

{

    /* movl */

    tcg_out_modrm_offset(s, 0x89, arg, arg1, arg2);

}

static inline void tgen_arithi(TCGContext *s, int c, int r0, int32_t val)

{

    if (val == (int8_t)val) {

        tcg_out_modrm(s, 0x83, c, r0);

        tcg_out8(s, val);

    } else {

        tcg_out_modrm(s, 0x81, c, r0);

        tcg_out32(s, val);

    }

}

void tcg_out_addi(TCGContext *s, int reg, tcg_target_long val)

{

    if (val != 0)

        tgen_arithi(s, ARITH_ADD, reg, val);

}

static void tcg_out_jxx(TCGContext *s, int opc, int label_index)

{

    int32_t val, val1;

    TCGLabel *l = &s->labels[label_index];

    if (l->has_value) {

        val = l->u.value - (tcg_target_long)s->code_ptr;

        val1 = val - 2;

        if ((int8_t)val1 == val1) {

            if (opc == -1)

                tcg_out8(s, 0xeb);

            else

                tcg_out8(s, 0x70 + opc);

            tcg_out8(s, val1);

        } else {

            if (opc == -1) {

                tcg_out8(s, 0xe9);

                tcg_out32(s, val - 5);

            } else {

                tcg_out8(s, 0x0f);

                tcg_out8(s, 0x80 + opc);

                tcg_out32(s, val - 6);

            }

        }

    } else {

        if (opc == -1) {

            tcg_out8(s, 0xe9);

        } else {

            tcg_out8(s, 0x0f);

            tcg_out8(s, 0x80 + opc);

        }

        tcg_out_reloc(s, s->code_ptr, R_386_PC32, label_index, -4);

        tcg_out32(s, -4);

    }

}

static void tcg_out_brcond(TCGContext *s, int cond, 

                           TCGArg arg1, TCGArg arg2, int const_arg2,

                           int label_index)

{

    int c;

    if (const_arg2) {

        if (arg2 == 0) {

            /* use test */

            switch(cond) {

            case TCG_COND_EQ:

                c = JCC_JNE;

                break;

            case TCG_COND_NE:

                c = JCC_JNE;

                break;

            case TCG_COND_LT:

                c = JCC_JS;

                break;

            case TCG_COND_GE:

                c = JCC_JNS;

                break;

            default:

                goto do_cmpi;

            }

            /* test r, r */

            tcg_out_modrm(s, 0x85, arg1, arg1);

            tcg_out_jxx(s, c, label_index);

        } else {

        do_cmpi:

            tgen_arithi(s, ARITH_CMP, arg1, arg2);

            tcg_out_jxx(s, tcg_cond_to_jcc[cond], label_index);

        }

    } else {

        tcg_out_modrm(s, 0x01 | (ARITH_CMP << 3), arg1, arg2);

        tcg_out_jxx(s, tcg_cond_to_jcc[cond], label_index);

    }

}

/* XXX: we implement it at the target level to avoid having to

   handle cross basic blocks temporaries */

static void tcg_out_brcond2(TCGContext *s,

                            const TCGArg *args, const int *const_args)

{

    int label_next;

    label_next = gen_new_label();

    switch(args[4]) {

    case TCG_COND_EQ:

        tcg_out_brcond(s, TCG_COND_NE, args[0], args[2], const_args[2], label_next);

        tcg_out_brcond(s, TCG_COND_EQ, args[1], args[3], const_args[3], args[5]);

        break;

    case TCG_COND_NE:

        tcg_out_brcond(s, TCG_COND_NE, args[0], args[2], const_args[2], args[5]);

        tcg_out_brcond(s, TCG_COND_EQ, args[1], args[3], const_args[3], label_next);

        break;

    case TCG_COND_LT:

        tcg_out_brcond(s, TCG_COND_LT, args[1], args[3], const_args[3], args[5]);

        tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);

        tcg_out_brcond(s, TCG_COND_LT, args[0], args[2], const_args[2], args[5]);

        break;

    case TCG_COND_LE:

        tcg_out_brcond(s, TCG_COND_LT, args[1], args[3], const_args[3], args[5]);

        tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);

        tcg_out_brcond(s, TCG_COND_LE, args[0], args[2], const_args[2], args[5]);

        break;

    case TCG_COND_GT:

        tcg_out_brcond(s, TCG_COND_GT, args[1], args[3], const_args[3], args[5]);

        tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);

        tcg_out_brcond(s, TCG_COND_GT, args[0], args[2], const_args[2], args[5]);

        break;

    case TCG_COND_GE:

        tcg_out_brcond(s, TCG_COND_GT, args[1], args[3], const_args[3], args[5]);

        tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);

        tcg_out_brcond(s, TCG_COND_GE, args[0], args[2], const_args[2], args[5]);

        break;

    case TCG_COND_LTU:

        tcg_out_brcond(s, TCG_COND_LTU, args[1], args[3], const_args[3], args[5]);

        tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);

        tcg_out_brcond(s, TCG_COND_LTU, args[0], args[2], const_args[2], args[5]);

        break;

    case TCG_COND_LEU:

        tcg_out_brcond(s, TCG_COND_LTU, args[1], args[3], const_args[3], args[5]);

        tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);

        tcg_out_brcond(s, TCG_COND_LEU, args[0], args[2], const_args[2], args[5]);

        break;

    case TCG_COND_GTU:

        tcg_out_brcond(s, TCG_COND_GTU, args[1], args[3], const_args[3], args[5]);

        tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);

        tcg_out_brcond(s, TCG_COND_GTU, args[0], args[2], const_args[2], args[5]);

        break;

    case TCG_COND_GEU:

        tcg_out_brcond(s, TCG_COND_GTU, args[1], args[3], const_args[3], args[5]);

        tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);

        tcg_out_brcond(s, TCG_COND_GEU, args[0], args[2], const_args[2], args[5]);

        break;

    default:

        tcg_abort();

    }

    tcg_out_label(s, label_next, (tcg_target_long)s->code_ptr);

}

#if defined(CONFIG_SOFTMMU)

extern void __ldb_mmu(void);

extern void __ldw_mmu(void);

extern void __ldl_mmu(void);

extern void __ldq_mmu(void);

extern void __stb_mmu(void);

extern void __stw_mmu(void);

extern void __stl_mmu(void);

extern void __stq_mmu(void);

static void *qemu_ld_helpers[4] = {

    __ldb_mmu,

    __ldw_mmu,

    __ldl_mmu,

    __ldq_mmu,

};

static void *qemu_st_helpers[4] = {

    __stb_mmu,

    __stw_mmu,

    __stl_mmu,

    __stq_mmu,

};

#endif

/* XXX: qemu_ld and qemu_st could be modified to clobber only EDX and

   EAX. It will be useful once fixed registers globals are less

   common. */

static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args,

                            int opc)

{

    int addr_reg, data_reg, data_reg2, r0, r1, mem_index, s_bits, bswap;

#if defined(CONFIG_SOFTMMU)

    uint8_t *label1_ptr, *label2_ptr;

#endif

#if TARGET_LONG_BITS == 64

#if defined(CONFIG_SOFTMMU)

    uint8_t *label3_ptr;

#endif

    int addr_reg2;

#endif

    data_reg = *args++;

    if (opc == 3)

        data_reg2 = *args++;

    else

        data_reg2 = 0;

    addr_reg = *args++;

#if TARGET_LONG_BITS == 64

    addr_reg2 = *args++;

#endif

    mem_index = *args;

    s_bits = opc & 3;

    r0 = TCG_REG_EAX;

    r1 = TCG_REG_EDX;

#if defined(CONFIG_SOFTMMU)

    tcg_out_mov(s, r1, addr_reg); 

    tcg_out_mov(s, r0, addr_reg); 

    tcg_out_modrm(s, 0xc1, 5, r1); /* shr $x, r1 */

    tcg_out8(s, TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); 

    tcg_out_modrm(s, 0x81, 4, r0); /* andl $x, r0 */

    tcg_out32(s, TARGET_PAGE_MASK | ((1 << s_bits) - 1));

    tcg_out_modrm(s, 0x81, 4, r1); /* andl $x, r1 */

    tcg_out32(s, (CPU_TLB_SIZE - 1) << CPU_TLB_ENTRY_BITS);

    tcg_out_opc(s, 0x8d); /* lea offset(r1, %ebp), r1 */

    tcg_out8(s, 0x80 | (r1 << 3) | 0x04);

    tcg_out8(s, (5 << 3) | r1);

    tcg_out32(s, offsetof(CPUState, tlb_table[mem_index][0].addr_read));

    /* cmp 0(r1), r0 */

    tcg_out_modrm_offset(s, 0x3b, r0, r1, 0);

    tcg_out_mov(s, r0, addr_reg);

#if TARGET_LONG_BITS == 32

    /* je label1 */

    tcg_out8(s, 0x70 + JCC_JE);

    label1_ptr = s->code_ptr;

    s->code_ptr++;

#else

    /* jne label3 */

    tcg_out8(s, 0x70 + JCC_JNE);

    label3_ptr = s->code_ptr;

    s->code_ptr++;

    /* cmp 4(r1), addr_reg2 */

    tcg_out_modrm_offset(s, 0x3b, addr_reg2, r1, 4);

    /* je label1 */

    tcg_out8(s, 0x70 + JCC_JE);

    label1_ptr = s->code_ptr;

    s->code_ptr++;

    /* label3: */

    *label3_ptr = s->code_ptr - label3_ptr - 1;

#endif

    /* XXX: move that code at the end of the TB */

#if TARGET_LONG_BITS == 32

    tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_EDX, mem_index);

#else

    tcg_out_mov(s, TCG_REG_EDX, addr_reg2);

    tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_ECX, mem_index);

#endif

    tcg_out8(s, 0xe8);

    tcg_out32(s, (tcg_target_long)qemu_ld_helpers[s_bits] - 

              (tcg_target_long)s->code_ptr - 4);

    switch(opc) {

    case 0 | 4:

        /* movsbl */

        tcg_out_modrm(s, 0xbe | P_EXT, data_reg, TCG_REG_EAX);

        break;

    case 1 | 4:

        /* movswl */

        tcg_out_modrm(s, 0xbf | P_EXT, data_reg, TCG_REG_EAX);

        break;

    case 0:

    case 1:

    case 2:

    default:

        tcg_out_mov(s, data_reg, TCG_REG_EAX);

        break;

    case 3:

        if (data_reg == TCG_REG_EDX) {

            tcg_out_opc(s, 0x90 + TCG_REG_EDX); /* xchg %edx, %eax */

            tcg_out_mov(s, data_reg2, TCG_REG_EAX);

        } else {

            tcg_out_mov(s, data_reg, TCG_REG_EAX);

            tcg_out_mov(s, data_reg2, TCG_REG_EDX);

        }

        break;

    }

    /* jmp label2 */

    tcg_out8(s, 0xeb);

    label2_ptr = s->code_ptr;

    s->code_ptr++;

    /* label1: */

    *label1_ptr = s->code_ptr - label1_ptr - 1;

    /* add x(r1), r0 */

    tcg_out_modrm_offset(s, 0x03, r0, r1, offsetof(CPUTLBEntry, addend) - 

                         offsetof(CPUTLBEntry, addr_read));

#else

    r0 = addr_reg;

#endif

#ifdef TARGET_WORDS_BIGENDIAN

    bswap = 1;

#else

    bswap = 0;

#endif

    switch(opc) {

    case 0:

        /* movzbl */

        tcg_out_modrm_offset(s, 0xb6 | P_EXT, data_reg, r0, 0);

        break;

    case 0 | 4:

        /* movsbl */

        tcg_out_modrm_offset(s, 0xbe | P_EXT, data_reg, r0, 0);

        break;

    case 1:

        /* movzwl */

        tcg_out_modrm_offset(s, 0xb7 | P_EXT, data_reg, r0, 0);

        if (bswap) {

            /* rolw $8, data_reg */

            tcg_out8(s, 0x66); 

            tcg_out_modrm(s, 0xc1, 0, data_reg);

            tcg_out8(s, 8);

        }

        break;

    case 1 | 4:

        /* movswl */

        tcg_out_modrm_offset(s, 0xbf | P_EXT, data_reg, r0, 0);

        if (bswap) {

            /* rolw $8, data_reg */

            tcg_out8(s, 0x66); 

            tcg_out_modrm(s, 0xc1, 0, data_reg);

            tcg_out8(s, 8);

            /* movswl data_reg, data_reg */

            tcg_out_modrm(s, 0xbf | P_EXT, data_reg, data_reg);

        }

        break;

    case 2:

        /* movl (r0), data_reg */

        tcg_out_modrm_offset(s, 0x8b, data_reg, r0, 0);

        if (bswap) {

            /* bswap */

            tcg_out_opc(s, (0xc8 + data_reg) | P_EXT);

        }

        break;

    case 3:

        /* XXX: could be nicer */

        if (r0 == data_reg) {

            r1 = TCG_REG_EDX;

            if (r1 == data_reg)

                r1 = TCG_REG_EAX;

            tcg_out_mov(s, r1, r0);

            r0 = r1;

        }

        if (!bswap) {

            tcg_out_modrm_offset(s, 0x8b, data_reg, r0, 0);

            tcg_out_modrm_offset(s, 0x8b, data_reg2, r0, 4);

        } else {

            tcg_out_modrm_offset(s, 0x8b, data_reg, r0, 4);

            tcg_out_opc(s, (0xc8 + data_reg) | P_EXT);

            tcg_out_modrm_offset(s, 0x8b, data_reg2, r0, 0);

            /* bswap */

            tcg_out_opc(s, (0xc8 + data_reg2) | P_EXT);

        }

        break;

    default:

        tcg_abort();

    }

#if defined(CONFIG_SOFTMMU)

    /* label2: */

    *label2_ptr = s->code_ptr - label2_ptr - 1;

#endif

}

static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args,

                            int opc)

{

    int addr_reg, data_reg, data_reg2, r0, r1, mem_index, s_bits, bswap;

#if defined(CONFIG_SOFTMMU)

    uint8_t *label1_ptr, *label2_ptr;

#endif

#if TARGET_LONG_BITS == 64

#if defined(CONFIG_SOFTMMU)

    uint8_t *label3_ptr;

#endif

    int addr_reg2;

#endif

    data_reg = *args++;

    if (opc == 3)

        data_reg2 = *args++;

    else

        data_reg2 = 0;

    addr_reg = *args++;

#if TARGET_LONG_BITS == 64

    addr_reg2 = *args++;

#endif

    mem_index = *args;

    s_bits = opc;

    r0 = TCG_REG_EAX;

    r1 = TCG_REG_EDX;

#if defined(CONFIG_SOFTMMU)

    tcg_out_mov(s, r1, addr_reg); 

    tcg_out_mov(s, r0, addr_reg); 

    tcg_out_modrm(s, 0xc1, 5, r1); /* shr $x, r1 */

    tcg_out8(s, TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); 

    tcg_out_modrm(s, 0x81, 4, r0); /* andl $x, r0 */

    tcg_out32(s, TARGET_PAGE_MASK | ((1 << s_bits) - 1));

    tcg_out_modrm(s, 0x81, 4, r1); /* andl $x, r1 */

    tcg_out32(s, (CPU_TLB_SIZE - 1) << CPU_TLB_ENTRY_BITS);

    tcg_out_opc(s, 0x8d); /* lea offset(r1, %ebp), r1 */

    tcg_out8(s, 0x80 | (r1 << 3) | 0x04);

    tcg_out8(s, (5 << 3) | r1);

    tcg_out32(s, offsetof(CPUState, tlb_table[mem_index][0].addr_write));

    /* cmp 0(r1), r0 */

    tcg_out_modrm_offset(s, 0x3b, r0, r1, 0);

    tcg_out_mov(s, r0, addr_reg);

#if TARGET_LONG_BITS == 32

    /* je label1 */

    tcg_out8(s, 0x70 + JCC_JE);

    label1_ptr = s->code_ptr;

    s->code_ptr++;

#else

    /* jne label3 */

    tcg_out8(s, 0x70 + JCC_JNE);

    label3_ptr = s->code_ptr;

    s->code_ptr++;

    /* cmp 4(r1), addr_reg2 */

    tcg_out_modrm_offset(s, 0x3b, addr_reg2, r1, 4);

    /* je label1 */

    tcg_out8(s, 0x70 + JCC_JE);

    label1_ptr = s->code_ptr;

    s->code_ptr++;

    /* label3: */

    *label3_ptr = s->code_ptr - label3_ptr - 1;

#endif

    /* XXX: move that code at the end of the TB */

#if TARGET_LONG_BITS == 32

    if (opc == 3) {

        tcg_out_mov(s, TCG_REG_EDX, data_reg);

        tcg_out_mov(s, TCG_REG_ECX, data_reg2);

        tcg_out8(s, 0x6a); /* push Ib */

        tcg_out8(s, mem_index);

        tcg_out8(s, 0xe8);

        tcg_out32(s, (tcg_target_long)qemu_st_helpers[s_bits] - 

                  (tcg_target_long)s->code_ptr - 4);

        tcg_out_addi(s, TCG_REG_ESP, 4);

    } else {

        switch(opc) {

        case 0:

            /* movzbl */

            tcg_out_modrm(s, 0xb6 | P_EXT, TCG_REG_EDX, data_reg);

            break;

        case 1:

            /* movzwl */

            tcg_out_modrm(s, 0xb7 | P_EXT, TCG_REG_EDX, data_reg);

            break;

        case 2:

            tcg_out_mov(s, TCG_REG_EDX, data_reg);

            break;

        }

        tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_ECX, mem_index);

        tcg_out8(s, 0xe8);

        tcg_out32(s, (tcg_target_long)qemu_st_helpers[s_bits] - 

                  (tcg_target_long)s->code_ptr - 4);

    }

#else

    if (opc == 3) {

        tcg_out_mov(s, TCG_REG_EDX, addr_reg2);

        tcg_out8(s, 0x6a); /* push Ib */

        tcg_out8(s, mem_index);

        tcg_out_opc(s, 0x50 + data_reg2); /* push */

        tcg_out_opc(s, 0x50 + data_reg); /* push */

        tcg_out8(s, 0xe8);

        tcg_out32(s, (tcg_target_long)qemu_st_helpers[s_bits] - 

                  (tcg_target_long)s->code_ptr - 4);

        tcg_out_addi(s, TCG_REG_ESP, 12);

    } else {

        tcg_out_mov(s, TCG_REG_EDX, addr_reg2);

        switch(opc) {

        case 0:

            /* movzbl */

            tcg_out_modrm(s, 0xb6 | P_EXT, TCG_REG_ECX, data_reg);

            break;

        case 1:

            /* movzwl */

            tcg_out_modrm(s, 0xb7 | P_EXT, TCG_REG_ECX, data_reg);

            break;

        case 2:

            tcg_out_mov(s, TCG_REG_ECX, data_reg);

            break;

        }

        tcg_out8(s, 0x6a); /* push Ib */

        tcg_out8(s, mem_index);

        tcg_out8(s, 0xe8);

        tcg_out32(s, (tcg_target_long)qemu_st_helpers[s_bits] - 

                  (tcg_target_long)s->code_ptr - 4);

        tcg_out_addi(s, TCG_REG_ESP, 4);

    }

#endif

    /* jmp label2 */

    tcg_out8(s, 0xeb);

    label2_ptr = s->code_ptr;

    s->code_ptr++;

    /* label1: */

    *label1_ptr = s->code_ptr - label1_ptr - 1;

    /* add x(r1), r0 */

    tcg_out_modrm_offset(s, 0x03, r0, r1, offsetof(CPUTLBEntry, addend) - 

                         offsetof(CPUTLBEntry, addr_write));

#else

    r0 = addr_reg;

#endif

#ifdef TARGET_WORDS_BIGENDIAN

    bswap = 1;

#else

    bswap = 0;

#endif

    switch(opc) {

    case 0:

        /* movb */

        tcg_out_modrm_offset(s, 0x88, data_reg, r0, 0);

        break;

    case 1:

        if (bswap) {

            tcg_out_mov(s, r1, data_reg);

            tcg_out8(s, 0x66); /* rolw $8, %ecx */

            tcg_out_modrm(s, 0xc1, 0, r1);

            tcg_out8(s, 8);

            data_reg = r1;

        }

        /* movw */

        tcg_out8(s, 0x66);

        tcg_out_modrm_offset(s, 0x89, data_reg, r0, 0);

        break;

    case 2:

        if (bswap) {

            tcg_out_mov(s, r1, data_reg);

            /* bswap data_reg */

            tcg_out_opc(s, (0xc8 + r1) | P_EXT);

            data_reg = r1;

        }

        /* movl */

        tcg_out_modrm_offset(s, 0x89, data_reg, r0, 0);

        break;

    case 3:

        if (bswap) {

            tcg_out_mov(s, r1, data_reg2);

            /* bswap data_reg */

            tcg_out_opc(s, (0xc8 + r1) | P_EXT);

            tcg_out_modrm_offset(s, 0x89, r1, r0, 0);

            tcg_out_mov(s, r1, data_reg);

            /* bswap data_reg */

            tcg_out_opc(s, (0xc8 + r1) | P_EXT);

            tcg_out_modrm_offset(s, 0x89, r1, r0, 4);

        } else {

            tcg_out_modrm_offset(s, 0x89, data_reg, r0, 0);

            tcg_out_modrm_offset(s, 0x89, data_reg2, r0, 4);

        }

        break;

    default:

        tcg_abort();

    }

#if defined(CONFIG_SOFTMMU)

    /* label2: */

    *label2_ptr = s->code_ptr - label2_ptr - 1;

#endif

}

static inline void tcg_out_op(TCGContext *s, int opc, 

                              const TCGArg *args, const int *const_args)

{

    int c;

    switch(opc) {

    case INDEX_op_exit_tb:

        tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_EAX, args[0]);

        tcg_out8(s, 0xc3); /* ret */

        break;

    case INDEX_op_goto_tb:

        if (s->tb_jmp_offset) {

            /* direct jump method */

            tcg_out8(s, 0xe9); /* jmp im */

            s->tb_jmp_offset[args[0]] = s->code_ptr - s->code_buf;

            tcg_out32(s, 0);

        } else {

            /* indirect jump method */

            /* jmp Ev */

            tcg_out_modrm_offset(s, 0xff, 4, -1, 

                                 (tcg_target_long)(s->tb_next + args[0]));

        }

        s->tb_next_offset[args[0]] = s->code_ptr - s->code_buf;

        break;

    case INDEX_op_call:

        if (const_args[0]) {

            tcg_out8(s, 0xe8);

            tcg_out32(s, args[0] - (tcg_target_long)s->code_ptr - 4);

        } else {

            tcg_out_modrm(s, 0xff, 2, args[0]);

        }

        break;

    case INDEX_op_jmp:

        if (const_args[0]) {

            tcg_out8(s, 0xe9);

            tcg_out32(s, args[0] - (tcg_target_long)s->code_ptr - 4);

        } else {

            tcg_out_modrm(s, 0xff, 4, args[0]);

        }

        break;

    case INDEX_op_br:

        tcg_out_jxx(s, JCC_JMP, args[0]);

        break;

    case INDEX_op_movi_i32:

        tcg_out_movi(s, TCG_TYPE_I32, args[0], args[1]);

        break;

    case INDEX_op_ld8u_i32:

        /* movzbl */

        tcg_out_modrm_offset(s, 0xb6 | P_EXT, args[0], args[1], args[2]);

        break;

    case INDEX_op_ld8s_i32:

        /* movsbl */

        tcg_out_modrm_offset(s, 0xbe | P_EXT, args[0], args[1], args[2]);

        break;

    case INDEX_op_ld16u_i32:

        /* movzwl */

        tcg_out_modrm_offset(s, 0xb7 | P_EXT, args[0], args[1], args[2]);

        break;

    case INDEX_op_ld16s_i32:

        /* movswl */

        tcg_out_modrm_offset(s, 0xbf | P_EXT, args[0], args[1], args[2]);

        break;

    case INDEX_op_ld_i32:

        /* movl */

        tcg_out_modrm_offset(s, 0x8b, args[0], args[1], args[2]);

        break;

    case INDEX_op_st8_i32:

        /* movb */

        tcg_out_modrm_offset(s, 0x88, args[0], args[1], args[2]);

        break;

    case INDEX_op_st16_i32:

        /* movw */

        tcg_out8(s, 0x66);

        tcg_out_modrm_offset(s, 0x89, args[0], args[1], args[2]);

        break;

    case INDEX_op_st_i32:

        /* movl */

        tcg_out_modrm_offset(s, 0x89, args[0], args[1], args[2]);

        break;

    case INDEX_op_sub_i32:

        c = ARITH_SUB;

        goto gen_arith;

    case INDEX_op_and_i32:

        c = ARITH_AND;

        goto gen_arith;

    case INDEX_op_or_i32:

        c = ARITH_OR;

        goto gen_arith;

    case INDEX_op_xor_i32:

        c = ARITH_XOR;

        goto gen_arith;

    case INDEX_op_add_i32:

        c = ARITH_ADD;

    gen_arith:

        if (const_args[2]) {

            tgen_arithi(s, c, args[0], args[2]);

        } else {

            tcg_out_modrm(s, 0x01 | (c << 3), args[2], args[0]);

        }

        break;

    case INDEX_op_mul_i32:

        if (const_args[2]) {

            int32_t val;

            val = args[2];

            if (val == (int8_t)val) {

                tcg_out_modrm(s, 0x6b, args[0], args[0]);

                tcg_out8(s, val);

            } else {

                tcg_out_modrm(s, 0x69, args[0], args[0]);

                tcg_out32(s, val);

            }

        } else {

            tcg_out_modrm(s, 0xaf | P_EXT, args[0], args[2]);

        }

        break;

    case INDEX_op_mulu2_i32:

        tcg_out_modrm(s, 0xf7, 4, args[3]);

        break;

    case INDEX_op_div2_i32:

        tcg_out_modrm(s, 0xf7, 7, args[4]);

        break;

    case INDEX_op_divu2_i32:

        tcg_out_modrm(s, 0xf7, 6, args[4]);

        break;

    case INDEX_op_shl_i32:

        c = SHIFT_SHL;

    gen_shift32:

        if (const_args[2]) {

            if (args[2] == 1) {

                tcg_out_modrm(s, 0xd1, c, args[0]);

            } else {

                tcg_out_modrm(s, 0xc1, c, args[0]);

                tcg_out8(s, args[2]);

            }

        } else {

            tcg_out_modrm(s, 0xd3, c, args[0]);

        }

        break;

    case INDEX_op_shr_i32:

        c = SHIFT_SHR;

        goto gen_shift32;