Archipelago » qemu

@settitle QEMU CPU Emulator Reference Documentation

@center @titlefont{QEMU CPU Emulator Reference Documentation}

QEMU is a FAST! processor emulator. By using dynamic translation it

achieves a reasonnable speed while being easy to port on new host

CPUs.

User mode emulation. In this mode, QEMU can launch Linux processes

compiled for one CPU on another CPU. Linux system calls are converted

because of endianness and 32/64 bit mismatches. The Wine Windows API

emulator (@url{http://www.winehq.org}) and the DOSEMU DOS emulator

(@url{http://www.dosemu.org}) are the main targets for QEMU.

Full system emulation. In this mode, QEMU emulates a full

system, including a processor and various peripherials. Currently, it

is only used to launch an x86 Linux kernel on an x86 Linux system. It

enables easier testing and debugging of system code. It can also be

used to provide virtual hosting of several virtual PCs on a single

server.

As QEMU requires no host kernel patches to run, it is very safe and

QEMU generic features:

@itemize 

@item User space only or full system emulation.

@item Using dynamic translation to native code for reasonnable speed.

@item Working on x86 and PowerPC hosts. Being tested on ARM, Sparc32, Alpha and S390.

@item Self-modifying code support.

@item Precise exceptions support.

@item The virtual CPU is a library (@code{libqemu}) which can be used 

in other projects.

@end itemize

QEMU user mode emulation features:

@itemize 

@item Generic Linux system call converter, including most ioctls.

@item clone() emulation using native CPU clone() to use Linux scheduler for threads.

@item Accurate signal handling by remapping host signals to target signals. 

@end itemize

@end itemize

QEMU full system emulation features:

@itemize 

@item QEMU can either use a full software MMU for maximum portability or use the host system call mmap() to simulate the target MMU.

@end itemize

@section x86 emulation

QEMU x86 target features:

@itemize 

@item The virtual x86 CPU supports 16 bit and 32 bit addressing with segmentation. 

LDT/GDT and IDT are emulated. VM86 mode is also supported to run DOSEMU.

@item Support of host page sizes bigger than 4KB in user mode emulation.

@item QEMU can emulate itself on x86.

@item An extensive Linux x86 CPU test program is included @file{tests/test-i386}. 

It can be used to test other x86 virtual CPUs.

@end itemize

Current QEMU limitations:

@itemize 

@item No SSE/MMX support (yet).

@item No x86-64 support.

@item IPC syscalls are missing.

@item The x86 segment limits and access rights are not tested at every 

memory access.

@item On non x86 host CPUs, @code{double}s are used instead of the non standard 

10 byte @code{long double}s of x86 for floating point emulation to get

maximum performances.

@item Some priviledged instructions or behaviors are missing, especially for segment protection testing (yet). 

@end itemize

@section ARM emulation

@itemize

@item ARM emulation can currently launch small programs while using the

generic dynamic code generation architecture of QEMU.

@item No FPU support (yet).

@item No automatic regression testing (yet).

@end itemize

@section SPARC emulation

The SPARC emulation is currently in development.

@chapter QEMU User space emulator invocation

@section Quick Start

In order to launch a Linux process, QEMU needs the process executable

itself and all the target (x86) dynamic libraries used by it. 

@item On x86, you can just try to launch any process by using the native

libraries:

@example 

qemu-i386 -L / /bin/ls

@end example

@code{-L /} tells that the x86 dynamic linker must be searched with a

@file{/} prefix.

@item Since QEMU is also a linux process, you can launch qemu with qemu (NOTE: you can only do that if you compiled QEMU from the sources):

@example 

qemu-i386 -L / qemu-i386 -L / /bin/ls

@end example

@item On non x86 CPUs, you need first to download at least an x86 glibc

(@file{qemu-runtime-i386-XXX-.tar.gz} on the QEMU web page). Ensure that

@code{LD_LIBRARY_PATH} is not set:

@example

unset LD_LIBRARY_PATH 

@end example

Then you can launch the precompiled @file{ls} x86 executable:

@example

qemu-i386 tests/i386/ls

@end example

You can look at @file{qemu-binfmt-conf.sh} so that

QEMU is automatically launched by the Linux kernel when you try to

launch x86 executables. It requires the @code{binfmt_misc} module in the

Linux kernel.

@item The x86 version of QEMU is also included. You can try weird things such as:

@example

qemu-i386 /usr/local/qemu-i386/bin/qemu-i386 /usr/local/qemu-i386/bin/ls-i386

@end example

@section Wine launch

@item Ensure that you have a working QEMU with the x86 glibc

distribution (see previous section). In order to verify it, you must be

able to do:

qemu-i386 /usr/local/qemu-i386/bin/ls-i386

@item Download the binary x86 Wine install

(@file{qemu-XXX-i386-wine.tar.gz} on the QEMU web page). 

@item Configure Wine on your account. Look at the provided script

@file{/usr/local/qemu-i386/bin/wine-conf.sh}. Your previous

@code{$@{HOME@}/.wine} directory is saved to @code{$@{HOME@}/.wine.org}.

@item Then you can try the example @file{putty.exe}:

@example

qemu-i386 /usr/local/qemu-i386/wine/bin/wine /usr/local/qemu-i386/wine/c/Program\ Files/putty.exe

@end example

@section Command line options

@example

usage: qemu-i386 [-h] [-d] [-L path] [-s size] program [arguments...]

@end example

@table @option

@item -h

Print the help

@item -L path   

Set the x86 elf interpreter prefix (default=/usr/local/qemu-i386)

@item -s size

Set the x86 stack size in bytes (default=524288)

@end table

Debug options:

@table @option

@item -d

Activate log (logfile=/tmp/qemu.log)

@item -p pagesize

Act as if the host page size was 'pagesize' bytes

@end table

The QEMU System emulator simulates a complete PC. It can either boot

directly a Linux kernel (without any BIOS or boot loader) or boot like a

real PC with the included BIOS.

NE2000 network adapter (port=0x300, irq=9)

@item

PIC (interrupt controler)

@item

PIT (timers)

@item 

CMOS memory

@section Direct Linux Boot and Network emulation

This section explains how to launch a Linux kernel inside QEMU without

having to make a full bootable image. It is very useful for fast Linux

kernel testing. The QEMU network configuration is also explained.

@enumerate

@item

Download the archive @file{linux-test-xxx.tar.gz} containing a Linux

kernel and a disk image. 

@item Optional: If you want network support (for example to launch X11 examples), you

must copy the script @file{qemu-ifup} in @file{/etc} and configure

properly @code{sudo} so that the command @code{ifconfig} contained in

@file{qemu-ifup} can be executed as root. You must verify that your host

kernel supports the TUN/TAP network interfaces: the device

@file{/dev/net/tun} must be present.

When network is enabled, there is a virtual network connection between

the host kernel and the emulated kernel. The emulated kernel is seen

from the host kernel at IP address 172.20.0.2 and the host kernel is

seen from the emulated kernel at IP address 172.20.0.1.

@item Launch @code{qemu.sh}. You should have the following output:

@example

> ./qemu.sh 

Connected to host network interface: tun0

Linux version 2.4.21 (bellard@voyager.localdomain) (gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)) #5 Tue Nov 11 18:18:53 CET 2003

BIOS-provided physical RAM map:

 BIOS-e801: 0000000000000000 - 000000000009f000 (usable)

 BIOS-e801: 0000000000100000 - 0000000002000000 (usable)

32MB LOWMEM available.

On node 0 totalpages: 8192

zone(0): 4096 pages.

zone(1): 4096 pages.

zone(2): 0 pages.

Kernel command line: root=/dev/hda sb=0x220,5,1,5 ide2=noprobe ide3=noprobe ide4=noprobe ide5=noprobe console=ttyS0

ide_setup: ide2=noprobe

ide_setup: ide3=noprobe

ide_setup: ide4=noprobe

ide_setup: ide5=noprobe

Initializing CPU#0

Detected 2399.621 MHz processor.

Console: colour EGA 80x25

Calibrating delay loop... 4744.80 BogoMIPS

Memory: 28872k/32768k available (1210k kernel code, 3508k reserved, 266k data, 64k init, 0k highmem)

Dentry cache hash table entries: 4096 (order: 3, 32768 bytes)

Inode cache hash table entries: 2048 (order: 2, 16384 bytes)

Mount cache hash table entries: 512 (order: 0, 4096 bytes)

Buffer-cache hash table entries: 1024 (order: 0, 4096 bytes)

Page-cache hash table entries: 8192 (order: 3, 32768 bytes)

CPU: Intel Pentium Pro stepping 03

Checking 'hlt' instruction... OK.

POSIX conformance testing by UNIFIX

Linux NET4.0 for Linux 2.4

Based upon Swansea University Computer Society NET3.039

Initializing RT netlink socket

apm: BIOS not found.

Starting kswapd

Journalled Block Device driver loaded

Detected PS/2 Mouse Port.

pty: 256 Unix98 ptys configured

Serial driver version 5.05c (2001-07-08) with no serial options enabled

ttyS00 at 0x03f8 (irq = 4) is a 16450

ne.c:v1.10 9/23/94 Donald Becker (becker@scyld.com)

Last modified Nov 1, 2000 by Paul Gortmaker

NE*000 ethercard probe at 0x300: 52 54 00 12 34 56

eth0: NE2000 found at 0x300, using IRQ 9.

RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize

Uniform Multi-Platform E-IDE driver Revision: 7.00beta4-2.4

ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx

hda: QEMU HARDDISK, ATA DISK drive

ide0 at 0x1f0-0x1f7,0x3f6 on irq 14

hda: attached ide-disk driver.

hda: 20480 sectors (10 MB) w/256KiB Cache, CHS=20/16/63

Partition check:

 hda:

Soundblaster audio driver Copyright (C) by Hannu Savolainen 1993-1996

NET4: Linux TCP/IP 1.0 for NET4.0

IP Protocols: ICMP, UDP, TCP, IGMP

IP: routing cache hash table of 512 buckets, 4Kbytes

TCP: Hash tables configured (established 2048 bind 4096)

NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.

EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended

VFS: Mounted root (ext2 filesystem).

Freeing unused kernel memory: 64k freed

Linux version 2.4.21 (bellard@voyager.localdomain) (gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)) #5 Tue Nov 11 18:18:53 CET 2003

QEMU Linux test distribution (based on Redhat 9)

Type 'exit' to halt the system

sh-2.05b# 

@end example

@item

Then you can play with the kernel inside the virtual serial console. You

can launch @code{ls} for example. Type @key{Ctrl-a h} to have an help

about the keys you can type inside the virtual serial console. In

particular, use @key{Ctrl-a x} to exit QEMU and use @key{Ctrl-a b} as

the Magic SysRq key.

@item 

If the network is enabled, launch the script @file{/etc/linuxrc} in the

emulator (don't forget the leading dot):

@example

. /etc/linuxrc

@end example

Then enable X11 connections on your PC from the emulated Linux: 

@example

xhost +172.20.0.2

@end example

You can now launch @file{xterm} or @file{xlogo} and verify that you have

a real Virtual Linux system !

@end enumerate

NOTES:

@enumerate

@item 

A 2.5.74 kernel is also included in the archive. Just

replace the bzImage in qemu.sh to try it.

@item 

qemu creates a temporary file in @var{$QEMU_TMPDIR} (@file{/tmp} is the

default) containing all the simulated PC memory. If possible, try to use

a temporary directory using the tmpfs filesystem to avoid too many

unnecessary disk accesses.

@item 

In order to exit cleanly from qemu, you can do a @emph{shutdown} inside

qemu. qemu will automatically exit when the Linux shutdown is done.

@item 

You can boot slightly faster by disabling the probe of non present IDE

interfaces. To do so, add the following options on the kernel command

line:

@example

ide1=noprobe ide2=noprobe ide3=noprobe ide4=noprobe ide5=noprobe

@end example

@item 

The example disk image is a modified version of the one made by Kevin

Lawton for the plex86 Project (@url{www.plex86.org}).

@end enumerate

@item -boot [a|b|c|d]

Boot on floppy (a, b), hard disk (c) or CD-ROM (d). Hard disk boot is

@item -n script      

Set network init script [default=/etc/qemu-ifup]. This script is

launched to configure the host network interface (usually tun0)

corresponding to the virtual NE2000 card.

@item -tun-fd fd      

Assumes @var{fd} talks to tap/tun and use it. Read

@url{http://bellard.org/qemu/tetrinet.html} to have an example of its

use.

Linux boot specific (does not require a full PC boot with a BIOS):

During emulation, use @key{C-a h} to get terminal commands:

Send break (magic sysrq)

@c man begin SEEALSO

The HTML documentation of QEMU for more precise information and Linux

user mode emulator invocation.

@c man end

@c man begin AUTHOR

Fabrice Bellard

@c man end

@end ignore

@end ignore

@code{qemu-fast} to get maximum performances, you should make the

following changes to the Linux kernel (only 2.4.x and 2.5.x were

tested):

If you are not using a 2.5 kernel as host kernel but if you use a target

2.5 kernel, you must also ensure that the 'HZ' define is set to 100

frequencies greater than 100 Hz on host Linux systems < 2.5. In

@chapter QEMU Internals

@section QEMU compared to other emulators

Like bochs [3], QEMU emulates an x86 CPU. But QEMU is much faster than

bochs as it uses dynamic compilation and because it uses the host MMU to

simulate the x86 MMU. The downside is that currently the emulation is

not as accurate as bochs (for example, you cannot currently run Windows

inside QEMU).

Like Valgrind [2], QEMU does user space emulation and dynamic

translation. Valgrind is mainly a memory debugger while QEMU has no

support for it (QEMU could be used to detect out of bound memory

accesses as Valgrind, but it has no support to track uninitialised data

as Valgrind does). The Valgrind dynamic translator generates better code

than QEMU (in particular it does register allocation) but it is closely

tied to an x86 host and target and has no support for precise exceptions

and system emulation.

EM86 [4] is the closest project to user space QEMU (and QEMU still uses

some of its code, in particular the ELF file loader). EM86 was limited

to an alpha host and used a proprietary and slow interpreter (the

interpreter part of the FX!32 Digital Win32 code translator [5]).

TWIN [6] is a Windows API emulator like Wine. It is less accurate than

Wine but includes a protected mode x86 interpreter to launch x86 Windows

executables. Such an approach as greater potential because most of the

Windows API is executed natively but it is far more difficult to develop

because all the data structures and function parameters exchanged

between the API and the x86 code must be converted.

User mode Linux [7] was the only solution before QEMU to launch a Linux

kernel as a process while not needing any host kernel patches. However,

user mode Linux requires heavy kernel patches while QEMU accepts

unpatched Linux kernels. It would be interesting to compare the

performance of the two approaches.

The new Plex86 [8] PC virtualizer is done in the same spirit as the QEMU

system emulator. It requires a patched Linux kernel to work (you cannot

launch the same kernel on your PC), but the patches are really small. As

it is a PC virtualizer (no emulation is done except for some priveledged

instructions), it has the potential of being faster than QEMU. The

downside is that a complicated (and potentially unsafe) host kernel

patch is needed.

@section Portable dynamic translation

QEMU is a dynamic translator. When it first encounters a piece of code,

it converts it to the host instruction set. Usually dynamic translators

are very complicated and highly CPU dependent. QEMU uses some tricks

which make it relatively easily portable and simple while achieving good

performances.

The basic idea is to split every x86 instruction into fewer simpler

instructions. Each simple instruction is implemented by a piece of C

code (see @file{op-i386.c}). Then a compile time tool (@file{dyngen})

takes the corresponding object file (@file{op-i386.o}) to generate a

dynamic code generator which concatenates the simple instructions to

build a function (see @file{op-i386.h:dyngen_code()}).

In essence, the process is similar to [1], but more work is done at

compile time. 

A key idea to get optimal performances is that constant parameters can

be passed to the simple operations. For that purpose, dummy ELF

relocations are generated with gcc for each constant parameter. Then,

the tool (@file{dyngen}) can locate the relocations and generate the

appriopriate C code to resolve them when building the dynamic code.

That way, QEMU is no more difficult to port than a dynamic linker.

To go even faster, GCC static register variables are used to keep the

state of the virtual CPU.

@section Register allocation

Since QEMU uses fixed simple instructions, no efficient register

allocation can be done. However, because RISC CPUs have a lot of

register, most of the virtual CPU state can be put in registers without

doing complicated register allocation.

@section Condition code optimisations

Good CPU condition codes emulation (@code{EFLAGS} register on x86) is a

critical point to get good performances. QEMU uses lazy condition code

evaluation: instead of computing the condition codes after each x86

instruction, it just stores one operand (called @code{CC_SRC}), the

result (called @code{CC_DST}) and the type of operation (called

@code{CC_OP}).

@code{CC_OP} is almost never explicitely set in the generated code

because it is known at translation time.

In order to increase performances, a backward pass is performed on the

generated simple instructions (see

@code{translate-i386.c:optimize_flags()}). When it can be proved that

the condition codes are not needed by the next instructions, no

condition codes are computed at all.

@section CPU state optimisations

The x86 CPU has many internal states which change the way it evaluates

instructions. In order to achieve a good speed, the translation phase

considers that some state information of the virtual x86 CPU cannot

change in it. For example, if the SS, DS and ES segments have a zero

base, then the translator does not even generate an addition for the

segment base.

[The FPU stack pointer register is not handled that way yet].

@section Translation cache

A 2MByte cache holds the most recently used translations. For

simplicity, it is completely flushed when it is full. A translation unit

contains just a single basic block (a block of x86 instructions

terminated by a jump or by a virtual CPU state change which the

translator cannot deduce statically).

@section Direct block chaining

After each translated basic block is executed, QEMU uses the simulated

Program Counter (PC) and other cpu state informations (such as the CS

segment base value) to find the next basic block.

In order to accelerate the most common cases where the new simulated PC

is known, QEMU can patch a basic block so that it jumps directly to the

next one.

The most portable code uses an indirect jump. An indirect jump makes it

easier to make the jump target modification atomic. On some

architectures (such as PowerPC), the @code{JUMP} opcode is directly

patched so that the block chaining has no overhead.

@section Self-modifying code and translated code invalidation

Self-modifying code is a special challenge in x86 emulation because no

instruction cache invalidation is signaled by the application when code

is modified.

When translated code is generated for a basic block, the corresponding

host page is write protected if it is not already read-only (with the

system call @code{mprotect()}). Then, if a write access is done to the

page, Linux raises a SEGV signal. QEMU then invalidates all the

translated code in the page and enables write accesses to the page.

Correct translated code invalidation is done efficiently by maintaining

a linked list of every translated block contained in a given page. Other

linked lists are also maintained to undo direct block chaining. 

Although the overhead of doing @code{mprotect()} calls is important,

most MSDOS programs can be emulated at reasonnable speed with QEMU and

DOSEMU.

Note that QEMU also invalidates pages of translated code when it detects

that memory mappings are modified with @code{mmap()} or @code{munmap()}.

@section Exception support

longjmp() is used when an exception such as division by zero is

encountered. 

The host SIGSEGV and SIGBUS signal handlers are used to get invalid

memory accesses. The exact CPU state can be retrieved because all the

x86 registers are stored in fixed host registers. The simulated program

counter is found by retranslating the corresponding basic block and by

looking where the host program counter was at the exception point.

The virtual CPU cannot retrieve the exact @code{EFLAGS} register because

in some cases it is not computed because of condition code

optimisations. It is not a big concern because the emulated code can

still be restarted in any cases.

@section Linux system call translation

QEMU includes a generic system call translator for Linux. It means that

the parameters of the system calls can be converted to fix the

endianness and 32/64 bit issues. The IOCTLs are converted with a generic

type description system (see @file{ioctls.h} and @file{thunk.c}).

QEMU supports host CPUs which have pages bigger than 4KB. It records all

the mappings the process does and try to emulated the @code{mmap()}

system calls in cases where the host @code{mmap()} call would fail

because of bad page alignment.

@section Linux signals

Normal and real-time signals are queued along with their information

(@code{siginfo_t}) as it is done in the Linux kernel. Then an interrupt

request is done to the virtual CPU. When it is interrupted, one queued

signal is handled by generating a stack frame in the virtual CPU as the

Linux kernel does. The @code{sigreturn()} system call is emulated to return

from the virtual signal handler.

Some signals (such as SIGALRM) directly come from the host. Other

signals are synthetized from the virtual CPU exceptions such as SIGFPE

when a division by zero is done (see @code{main.c:cpu_loop()}).

The blocked signal mask is still handled by the host Linux kernel so

that most signal system calls can be redirected directly to the host

Linux kernel. Only the @code{sigaction()} and @code{sigreturn()} system

calls need to be fully emulated (see @file{signal.c}).

@section clone() system call and threads

The Linux clone() system call is usually used to create a thread. QEMU

uses the host clone() system call so that real host threads are created

for each emulated thread. One virtual CPU instance is created for each

thread.

The virtual x86 CPU atomic operations are emulated with a global lock so

that their semantic is preserved.

Note that currently there are still some locking issues in QEMU. In

particular, the translated cache flush is not protected yet against

reentrancy.

@section Self-virtualization

QEMU was conceived so that ultimately it can emulate itself. Although

it is not very useful, it is an important test to show the power of the

emulator.

Achieving self-virtualization is not easy because there may be address

space conflicts. QEMU solves this problem by being an executable ELF

shared object as the ld-linux.so ELF interpreter. That way, it can be

relocated at load time.

@section MMU emulation

For system emulation, QEMU uses the mmap() system call to emulate the

target CPU MMU. It works as long the emulated OS does not use an area

reserved by the host OS (such as the area above 0xc0000000 on x86

Linux).

It is planned to add a slower but more precise MMU emulation

with a software MMU.

@section Bibliography

@table @asis

@item [1] 

@url{http://citeseer.nj.nec.com/piumarta98optimizing.html}, Optimizing

direct threaded code by selective inlining (1998) by Ian Piumarta, Fabio

Riccardi.

@item [2]

@url{http://developer.kde.org/~sewardj/}, Valgrind, an open-source

memory debugger for x86-GNU/Linux, by Julian Seward.

@item [3]

@url{http://bochs.sourceforge.net/}, the Bochs IA-32 Emulator Project,

by Kevin Lawton et al.

@item [4]

@url{http://www.cs.rose-hulman.edu/~donaldlf/em86/index.html}, the EM86

x86 emulator on Alpha-Linux.

@item [5]

@url{http://www.usenix.org/publications/library/proceedings/usenix-nt97/full_papers/chernoff/chernoff.pdf},

DIGITAL FX!32: Running 32-Bit x86 Applications on Alpha NT, by Anton

Chernoff and Ray Hookway.

@item [6]

@url{http://www.willows.com/}, Windows API library emulation from

Willows Software.

@item [7]

@url{http://user-mode-linux.sourceforge.net/}, 

The User-mode Linux Kernel.

@item [8]

@url{http://www.plex86.org/}, 

The new Plex86 project.

@chapter Regression Tests

In the directory @file{tests/}, various interesting testing programs

are available. There are used for regression testing.

@section @file{test-i386}

This program executes most of the 16 bit and 32 bit x86 instructions and

generates a text output. It can be compared with the output obtained with

a real CPU or another emulator. The target @code{make test} runs this

program and a @code{diff} on the generated output.

The Linux system call @code{modify_ldt()} is used to create x86 selectors

to test some 16 bit addressing and 32 bit with segmentation cases.

The Linux system call @code{vm86()} is used to test vm86 emulation.

Various exceptions are raised to test most of the x86 user space

exception reporting.

@section @file{linux-test}

This program tests various Linux system calls. It is used to verify

that the system call parameters are correctly converted between target

and host CPUs.

@section @file{hello-i386}

Very simple statically linked x86 program, just to test QEMU during a

port to a new host CPU.

@section @file{hello-arm}

Very simple statically linked ARM program, just to test QEMU during a

port to a new host CPU.

@section @file{sha1}

It is a simple benchmark. Care must be taken to interpret the results

because it mostly tests the ability of the virtual CPU to optimize the

@code{rol} x86 instruction and the condition code computations.