root / qemu-tech.texi @ 3f20e1dd
History | View | Annotate | Download (18 kB)
1 |
\input texinfo @c -*- texinfo -*- |
---|---|
2 |
|
3 |
@iftex |
4 |
@settitle QEMU Internals |
5 |
@titlepage |
6 |
@sp 7 |
7 |
@center @titlefont{QEMU Internals} |
8 |
@sp 3 |
9 |
@end titlepage |
10 |
@end iftex |
11 |
|
12 |
@chapter Introduction |
13 |
|
14 |
@section Features |
15 |
|
16 |
QEMU is a FAST! processor emulator using a portable dynamic |
17 |
translator. |
18 |
|
19 |
QEMU has two operating modes: |
20 |
|
21 |
@itemize @minus |
22 |
|
23 |
@item |
24 |
Full system emulation. In this mode, QEMU emulates a full system |
25 |
(usually a PC), including a processor and various peripherals. It can |
26 |
be used to launch an different Operating System without rebooting the |
27 |
PC or to debug system code. |
28 |
|
29 |
@item |
30 |
User mode emulation (Linux host only). In this mode, QEMU can launch |
31 |
Linux processes compiled for one CPU on another CPU. It can be used to |
32 |
launch the Wine Windows API emulator (@url{http://www.winehq.org}) or |
33 |
to ease cross-compilation and cross-debugging. |
34 |
|
35 |
@end itemize |
36 |
|
37 |
As QEMU requires no host kernel driver to run, it is very safe and |
38 |
easy to use. |
39 |
|
40 |
QEMU generic features: |
41 |
|
42 |
@itemize |
43 |
|
44 |
@item User space only or full system emulation. |
45 |
|
46 |
@item Using dynamic translation to native code for reasonnable speed. |
47 |
|
48 |
@item Working on x86 and PowerPC hosts. Being tested on ARM, Sparc32, Alpha and S390. |
49 |
|
50 |
@item Self-modifying code support. |
51 |
|
52 |
@item Precise exceptions support. |
53 |
|
54 |
@item The virtual CPU is a library (@code{libqemu}) which can be used |
55 |
in other projects (look at @file{qemu/tests/qruncom.c} to have an |
56 |
example of user mode @code{libqemu} usage). |
57 |
|
58 |
@end itemize |
59 |
|
60 |
QEMU user mode emulation features: |
61 |
@itemize |
62 |
@item Generic Linux system call converter, including most ioctls. |
63 |
|
64 |
@item clone() emulation using native CPU clone() to use Linux scheduler for threads. |
65 |
|
66 |
@item Accurate signal handling by remapping host signals to target signals. |
67 |
@end itemize |
68 |
@end itemize |
69 |
|
70 |
QEMU full system emulation features: |
71 |
@itemize |
72 |
@item QEMU can either use a full software MMU for maximum portability or use the host system call mmap() to simulate the target MMU. |
73 |
@end itemize |
74 |
|
75 |
@section x86 emulation |
76 |
|
77 |
QEMU x86 target features: |
78 |
|
79 |
@itemize |
80 |
|
81 |
@item The virtual x86 CPU supports 16 bit and 32 bit addressing with segmentation. |
82 |
LDT/GDT and IDT are emulated. VM86 mode is also supported to run DOSEMU. |
83 |
|
84 |
@item Support of host page sizes bigger than 4KB in user mode emulation. |
85 |
|
86 |
@item QEMU can emulate itself on x86. |
87 |
|
88 |
@item An extensive Linux x86 CPU test program is included @file{tests/test-i386}. |
89 |
It can be used to test other x86 virtual CPUs. |
90 |
|
91 |
@end itemize |
92 |
|
93 |
Current QEMU limitations: |
94 |
|
95 |
@itemize |
96 |
|
97 |
@item No SSE/MMX support (yet). |
98 |
|
99 |
@item No x86-64 support. |
100 |
|
101 |
@item IPC syscalls are missing. |
102 |
|
103 |
@item The x86 segment limits and access rights are not tested at every |
104 |
memory access (yet). Hopefully, very few OSes seem to rely on that for |
105 |
normal use. |
106 |
|
107 |
@item On non x86 host CPUs, @code{double}s are used instead of the non standard |
108 |
10 byte @code{long double}s of x86 for floating point emulation to get |
109 |
maximum performances. |
110 |
|
111 |
@end itemize |
112 |
|
113 |
@section ARM emulation |
114 |
|
115 |
@itemize |
116 |
|
117 |
@item Full ARM 7 user emulation. |
118 |
|
119 |
@item NWFPE FPU support included in user Linux emulation. |
120 |
|
121 |
@item Can run most ARM Linux binaries. |
122 |
|
123 |
@end itemize |
124 |
|
125 |
@section PowerPC emulation |
126 |
|
127 |
@itemize |
128 |
|
129 |
@item Full PowerPC 32 bit emulation, including privileged instructions, |
130 |
FPU and MMU. |
131 |
|
132 |
@item Can run most PowerPC Linux binaries. |
133 |
|
134 |
@end itemize |
135 |
|
136 |
@section SPARC emulation |
137 |
|
138 |
@itemize |
139 |
|
140 |
@item Somewhat complete SPARC V8 emulation, including privileged |
141 |
instructions, FPU and MMU. SPARC V9 emulation includes most privileged |
142 |
instructions, FPU and I/D MMU, but misses VIS instructions. |
143 |
|
144 |
@item Can run some 32-bit SPARC Linux binaries. |
145 |
|
146 |
@end itemize |
147 |
|
148 |
Current QEMU limitations: |
149 |
|
150 |
@itemize |
151 |
|
152 |
@item Tagged add/subtract instructions are not supported, but they are |
153 |
probably not used. |
154 |
|
155 |
@item IPC syscalls are missing. |
156 |
|
157 |
@item 128-bit floating point operations are not supported, though none of the |
158 |
real CPUs implement them either. FCMPE[SD] are not correctly |
159 |
implemented. Floating point exception support is untested. |
160 |
|
161 |
@item Alignment is not enforced at all. |
162 |
|
163 |
@item Atomic instructions are not correctly implemented. |
164 |
|
165 |
@item Sparc64 emulators are not usable for anything yet. |
166 |
|
167 |
@end itemize |
168 |
|
169 |
@chapter QEMU Internals |
170 |
|
171 |
@section QEMU compared to other emulators |
172 |
|
173 |
Like bochs [3], QEMU emulates an x86 CPU. But QEMU is much faster than |
174 |
bochs as it uses dynamic compilation. Bochs is closely tied to x86 PC |
175 |
emulation while QEMU can emulate several processors. |
176 |
|
177 |
Like Valgrind [2], QEMU does user space emulation and dynamic |
178 |
translation. Valgrind is mainly a memory debugger while QEMU has no |
179 |
support for it (QEMU could be used to detect out of bound memory |
180 |
accesses as Valgrind, but it has no support to track uninitialised data |
181 |
as Valgrind does). The Valgrind dynamic translator generates better code |
182 |
than QEMU (in particular it does register allocation) but it is closely |
183 |
tied to an x86 host and target and has no support for precise exceptions |
184 |
and system emulation. |
185 |
|
186 |
EM86 [4] is the closest project to user space QEMU (and QEMU still uses |
187 |
some of its code, in particular the ELF file loader). EM86 was limited |
188 |
to an alpha host and used a proprietary and slow interpreter (the |
189 |
interpreter part of the FX!32 Digital Win32 code translator [5]). |
190 |
|
191 |
TWIN [6] is a Windows API emulator like Wine. It is less accurate than |
192 |
Wine but includes a protected mode x86 interpreter to launch x86 Windows |
193 |
executables. Such an approach has greater potential because most of the |
194 |
Windows API is executed natively but it is far more difficult to develop |
195 |
because all the data structures and function parameters exchanged |
196 |
between the API and the x86 code must be converted. |
197 |
|
198 |
User mode Linux [7] was the only solution before QEMU to launch a |
199 |
Linux kernel as a process while not needing any host kernel |
200 |
patches. However, user mode Linux requires heavy kernel patches while |
201 |
QEMU accepts unpatched Linux kernels. The price to pay is that QEMU is |
202 |
slower. |
203 |
|
204 |
The new Plex86 [8] PC virtualizer is done in the same spirit as the |
205 |
qemu-fast system emulator. It requires a patched Linux kernel to work |
206 |
(you cannot launch the same kernel on your PC), but the patches are |
207 |
really small. As it is a PC virtualizer (no emulation is done except |
208 |
for some priveledged instructions), it has the potential of being |
209 |
faster than QEMU. The downside is that a complicated (and potentially |
210 |
unsafe) host kernel patch is needed. |
211 |
|
212 |
The commercial PC Virtualizers (VMWare [9], VirtualPC [10], TwoOStwo |
213 |
[11]) are faster than QEMU, but they all need specific, proprietary |
214 |
and potentially unsafe host drivers. Moreover, they are unable to |
215 |
provide cycle exact simulation as an emulator can. |
216 |
|
217 |
@section Portable dynamic translation |
218 |
|
219 |
QEMU is a dynamic translator. When it first encounters a piece of code, |
220 |
it converts it to the host instruction set. Usually dynamic translators |
221 |
are very complicated and highly CPU dependent. QEMU uses some tricks |
222 |
which make it relatively easily portable and simple while achieving good |
223 |
performances. |
224 |
|
225 |
The basic idea is to split every x86 instruction into fewer simpler |
226 |
instructions. Each simple instruction is implemented by a piece of C |
227 |
code (see @file{target-i386/op.c}). Then a compile time tool |
228 |
(@file{dyngen}) takes the corresponding object file (@file{op.o}) |
229 |
to generate a dynamic code generator which concatenates the simple |
230 |
instructions to build a function (see @file{op.h:dyngen_code()}). |
231 |
|
232 |
In essence, the process is similar to [1], but more work is done at |
233 |
compile time. |
234 |
|
235 |
A key idea to get optimal performances is that constant parameters can |
236 |
be passed to the simple operations. For that purpose, dummy ELF |
237 |
relocations are generated with gcc for each constant parameter. Then, |
238 |
the tool (@file{dyngen}) can locate the relocations and generate the |
239 |
appriopriate C code to resolve them when building the dynamic code. |
240 |
|
241 |
That way, QEMU is no more difficult to port than a dynamic linker. |
242 |
|
243 |
To go even faster, GCC static register variables are used to keep the |
244 |
state of the virtual CPU. |
245 |
|
246 |
@section Register allocation |
247 |
|
248 |
Since QEMU uses fixed simple instructions, no efficient register |
249 |
allocation can be done. However, because RISC CPUs have a lot of |
250 |
register, most of the virtual CPU state can be put in registers without |
251 |
doing complicated register allocation. |
252 |
|
253 |
@section Condition code optimisations |
254 |
|
255 |
Good CPU condition codes emulation (@code{EFLAGS} register on x86) is a |
256 |
critical point to get good performances. QEMU uses lazy condition code |
257 |
evaluation: instead of computing the condition codes after each x86 |
258 |
instruction, it just stores one operand (called @code{CC_SRC}), the |
259 |
result (called @code{CC_DST}) and the type of operation (called |
260 |
@code{CC_OP}). |
261 |
|
262 |
@code{CC_OP} is almost never explicitely set in the generated code |
263 |
because it is known at translation time. |
264 |
|
265 |
In order to increase performances, a backward pass is performed on the |
266 |
generated simple instructions (see |
267 |
@code{target-i386/translate.c:optimize_flags()}). When it can be proved that |
268 |
the condition codes are not needed by the next instructions, no |
269 |
condition codes are computed at all. |
270 |
|
271 |
@section CPU state optimisations |
272 |
|
273 |
The x86 CPU has many internal states which change the way it evaluates |
274 |
instructions. In order to achieve a good speed, the translation phase |
275 |
considers that some state information of the virtual x86 CPU cannot |
276 |
change in it. For example, if the SS, DS and ES segments have a zero |
277 |
base, then the translator does not even generate an addition for the |
278 |
segment base. |
279 |
|
280 |
[The FPU stack pointer register is not handled that way yet]. |
281 |
|
282 |
@section Translation cache |
283 |
|
284 |
A 16 MByte cache holds the most recently used translations. For |
285 |
simplicity, it is completely flushed when it is full. A translation unit |
286 |
contains just a single basic block (a block of x86 instructions |
287 |
terminated by a jump or by a virtual CPU state change which the |
288 |
translator cannot deduce statically). |
289 |
|
290 |
@section Direct block chaining |
291 |
|
292 |
After each translated basic block is executed, QEMU uses the simulated |
293 |
Program Counter (PC) and other cpu state informations (such as the CS |
294 |
segment base value) to find the next basic block. |
295 |
|
296 |
In order to accelerate the most common cases where the new simulated PC |
297 |
is known, QEMU can patch a basic block so that it jumps directly to the |
298 |
next one. |
299 |
|
300 |
The most portable code uses an indirect jump. An indirect jump makes |
301 |
it easier to make the jump target modification atomic. On some host |
302 |
architectures (such as x86 or PowerPC), the @code{JUMP} opcode is |
303 |
directly patched so that the block chaining has no overhead. |
304 |
|
305 |
@section Self-modifying code and translated code invalidation |
306 |
|
307 |
Self-modifying code is a special challenge in x86 emulation because no |
308 |
instruction cache invalidation is signaled by the application when code |
309 |
is modified. |
310 |
|
311 |
When translated code is generated for a basic block, the corresponding |
312 |
host page is write protected if it is not already read-only (with the |
313 |
system call @code{mprotect()}). Then, if a write access is done to the |
314 |
page, Linux raises a SEGV signal. QEMU then invalidates all the |
315 |
translated code in the page and enables write accesses to the page. |
316 |
|
317 |
Correct translated code invalidation is done efficiently by maintaining |
318 |
a linked list of every translated block contained in a given page. Other |
319 |
linked lists are also maintained to undo direct block chaining. |
320 |
|
321 |
Although the overhead of doing @code{mprotect()} calls is important, |
322 |
most MSDOS programs can be emulated at reasonnable speed with QEMU and |
323 |
DOSEMU. |
324 |
|
325 |
Note that QEMU also invalidates pages of translated code when it detects |
326 |
that memory mappings are modified with @code{mmap()} or @code{munmap()}. |
327 |
|
328 |
When using a software MMU, the code invalidation is more efficient: if |
329 |
a given code page is invalidated too often because of write accesses, |
330 |
then a bitmap representing all the code inside the page is |
331 |
built. Every store into that page checks the bitmap to see if the code |
332 |
really needs to be invalidated. It avoids invalidating the code when |
333 |
only data is modified in the page. |
334 |
|
335 |
@section Exception support |
336 |
|
337 |
longjmp() is used when an exception such as division by zero is |
338 |
encountered. |
339 |
|
340 |
The host SIGSEGV and SIGBUS signal handlers are used to get invalid |
341 |
memory accesses. The exact CPU state can be retrieved because all the |
342 |
x86 registers are stored in fixed host registers. The simulated program |
343 |
counter is found by retranslating the corresponding basic block and by |
344 |
looking where the host program counter was at the exception point. |
345 |
|
346 |
The virtual CPU cannot retrieve the exact @code{EFLAGS} register because |
347 |
in some cases it is not computed because of condition code |
348 |
optimisations. It is not a big concern because the emulated code can |
349 |
still be restarted in any cases. |
350 |
|
351 |
@section MMU emulation |
352 |
|
353 |
For system emulation, QEMU uses the mmap() system call to emulate the |
354 |
target CPU MMU. It works as long the emulated OS does not use an area |
355 |
reserved by the host OS (such as the area above 0xc0000000 on x86 |
356 |
Linux). |
357 |
|
358 |
In order to be able to launch any OS, QEMU also supports a soft |
359 |
MMU. In that mode, the MMU virtual to physical address translation is |
360 |
done at every memory access. QEMU uses an address translation cache to |
361 |
speed up the translation. |
362 |
|
363 |
In order to avoid flushing the translated code each time the MMU |
364 |
mappings change, QEMU uses a physically indexed translation cache. It |
365 |
means that each basic block is indexed with its physical address. |
366 |
|
367 |
When MMU mappings change, only the chaining of the basic blocks is |
368 |
reset (i.e. a basic block can no longer jump directly to another one). |
369 |
|
370 |
@section Hardware interrupts |
371 |
|
372 |
In order to be faster, QEMU does not check at every basic block if an |
373 |
hardware interrupt is pending. Instead, the user must asynchrously |
374 |
call a specific function to tell that an interrupt is pending. This |
375 |
function resets the chaining of the currently executing basic |
376 |
block. It ensures that the execution will return soon in the main loop |
377 |
of the CPU emulator. Then the main loop can test if the interrupt is |
378 |
pending and handle it. |
379 |
|
380 |
@section User emulation specific details |
381 |
|
382 |
@subsection Linux system call translation |
383 |
|
384 |
QEMU includes a generic system call translator for Linux. It means that |
385 |
the parameters of the system calls can be converted to fix the |
386 |
endianness and 32/64 bit issues. The IOCTLs are converted with a generic |
387 |
type description system (see @file{ioctls.h} and @file{thunk.c}). |
388 |
|
389 |
QEMU supports host CPUs which have pages bigger than 4KB. It records all |
390 |
the mappings the process does and try to emulated the @code{mmap()} |
391 |
system calls in cases where the host @code{mmap()} call would fail |
392 |
because of bad page alignment. |
393 |
|
394 |
@subsection Linux signals |
395 |
|
396 |
Normal and real-time signals are queued along with their information |
397 |
(@code{siginfo_t}) as it is done in the Linux kernel. Then an interrupt |
398 |
request is done to the virtual CPU. When it is interrupted, one queued |
399 |
signal is handled by generating a stack frame in the virtual CPU as the |
400 |
Linux kernel does. The @code{sigreturn()} system call is emulated to return |
401 |
from the virtual signal handler. |
402 |
|
403 |
Some signals (such as SIGALRM) directly come from the host. Other |
404 |
signals are synthetized from the virtual CPU exceptions such as SIGFPE |
405 |
when a division by zero is done (see @code{main.c:cpu_loop()}). |
406 |
|
407 |
The blocked signal mask is still handled by the host Linux kernel so |
408 |
that most signal system calls can be redirected directly to the host |
409 |
Linux kernel. Only the @code{sigaction()} and @code{sigreturn()} system |
410 |
calls need to be fully emulated (see @file{signal.c}). |
411 |
|
412 |
@subsection clone() system call and threads |
413 |
|
414 |
The Linux clone() system call is usually used to create a thread. QEMU |
415 |
uses the host clone() system call so that real host threads are created |
416 |
for each emulated thread. One virtual CPU instance is created for each |
417 |
thread. |
418 |
|
419 |
The virtual x86 CPU atomic operations are emulated with a global lock so |
420 |
that their semantic is preserved. |
421 |
|
422 |
Note that currently there are still some locking issues in QEMU. In |
423 |
particular, the translated cache flush is not protected yet against |
424 |
reentrancy. |
425 |
|
426 |
@subsection Self-virtualization |
427 |
|
428 |
QEMU was conceived so that ultimately it can emulate itself. Although |
429 |
it is not very useful, it is an important test to show the power of the |
430 |
emulator. |
431 |
|
432 |
Achieving self-virtualization is not easy because there may be address |
433 |
space conflicts. QEMU solves this problem by being an executable ELF |
434 |
shared object as the ld-linux.so ELF interpreter. That way, it can be |
435 |
relocated at load time. |
436 |
|
437 |
@section Bibliography |
438 |
|
439 |
@table @asis |
440 |
|
441 |
@item [1] |
442 |
@url{http://citeseer.nj.nec.com/piumarta98optimizing.html}, Optimizing |
443 |
direct threaded code by selective inlining (1998) by Ian Piumarta, Fabio |
444 |
Riccardi. |
445 |
|
446 |
@item [2] |
447 |
@url{http://developer.kde.org/~sewardj/}, Valgrind, an open-source |
448 |
memory debugger for x86-GNU/Linux, by Julian Seward. |
449 |
|
450 |
@item [3] |
451 |
@url{http://bochs.sourceforge.net/}, the Bochs IA-32 Emulator Project, |
452 |
by Kevin Lawton et al. |
453 |
|
454 |
@item [4] |
455 |
@url{http://www.cs.rose-hulman.edu/~donaldlf/em86/index.html}, the EM86 |
456 |
x86 emulator on Alpha-Linux. |
457 |
|
458 |
@item [5] |
459 |
@url{http://www.usenix.org/publications/library/proceedings/usenix-nt97/full_papers/chernoff/chernoff.pdf}, |
460 |
DIGITAL FX!32: Running 32-Bit x86 Applications on Alpha NT, by Anton |
461 |
Chernoff and Ray Hookway. |
462 |
|
463 |
@item [6] |
464 |
@url{http://www.willows.com/}, Windows API library emulation from |
465 |
Willows Software. |
466 |
|
467 |
@item [7] |
468 |
@url{http://user-mode-linux.sourceforge.net/}, |
469 |
The User-mode Linux Kernel. |
470 |
|
471 |
@item [8] |
472 |
@url{http://www.plex86.org/}, |
473 |
The new Plex86 project. |
474 |
|
475 |
@item [9] |
476 |
@url{http://www.vmware.com/}, |
477 |
The VMWare PC virtualizer. |
478 |
|
479 |
@item [10] |
480 |
@url{http://www.microsoft.com/windowsxp/virtualpc/}, |
481 |
The VirtualPC PC virtualizer. |
482 |
|
483 |
@item [11] |
484 |
@url{http://www.twoostwo.org/}, |
485 |
The TwoOStwo PC virtualizer. |
486 |
|
487 |
@end table |
488 |
|
489 |
@chapter Regression Tests |
490 |
|
491 |
In the directory @file{tests/}, various interesting testing programs |
492 |
are available. There are used for regression testing. |
493 |
|
494 |
@section @file{test-i386} |
495 |
|
496 |
This program executes most of the 16 bit and 32 bit x86 instructions and |
497 |
generates a text output. It can be compared with the output obtained with |
498 |
a real CPU or another emulator. The target @code{make test} runs this |
499 |
program and a @code{diff} on the generated output. |
500 |
|
501 |
The Linux system call @code{modify_ldt()} is used to create x86 selectors |
502 |
to test some 16 bit addressing and 32 bit with segmentation cases. |
503 |
|
504 |
The Linux system call @code{vm86()} is used to test vm86 emulation. |
505 |
|
506 |
Various exceptions are raised to test most of the x86 user space |
507 |
exception reporting. |
508 |
|
509 |
@section @file{linux-test} |
510 |
|
511 |
This program tests various Linux system calls. It is used to verify |
512 |
that the system call parameters are correctly converted between target |
513 |
and host CPUs. |
514 |
|
515 |
@section @file{qruncom.c} |
516 |
|
517 |
Example of usage of @code{libqemu} to emulate a user mode i386 CPU. |