5 This document describes the tracing infrastructure in QEMU and how to use it
6 for debugging, profiling, and observing execution.
10 1. Build with the 'simple' trace backend:
12 ./configure --enable-trace-backends=simple
15 2. Create a file with the events you want to trace:
17 echo bdrv_aio_readv > /tmp/events
18 echo bdrv_aio_writev >> /tmp/events
20 3. Run the virtual machine to produce a trace file:
22 qemu -trace events=/tmp/events ... # your normal QEMU invocation
24 4. Pretty-print the binary trace file:
26 ./scripts/simpletrace.py trace-events trace-* # Override * with QEMU <pid>
30 There is a set of static trace events declared in the "trace-events" source
31 file. Each trace event declaration names the event, its arguments, and the
32 format string which can be used for pretty-printing:
34 qemu_vmalloc(size_t size, void *ptr) "size %zu ptr %p"
35 qemu_vfree(void *ptr) "ptr %p"
37 The "trace-events" file is processed by the "tracetool" script during build to
38 generate code for the trace events. Trace events are invoked directly from
39 source code like this:
41 #include "trace.h" /* needed for trace event prototype */
43 void *qemu_vmalloc(size_t size)
46 size_t align = QEMU_VMALLOC_ALIGN;
49 align = getpagesize();
51 ptr = qemu_memalign(align, size);
52 trace_qemu_vmalloc(size, ptr);
56 === Declaring trace events ===
58 The "tracetool" script produces the trace.h header file which is included by
59 every source file that uses trace events. Since many source files include
60 trace.h, it uses a minimum of types and other header files included to keep the
61 namespace clean and compile times and dependencies down.
63 Trace events should use types as follows:
65 * Use stdint.h types for fixed-size types. Most offsets and guest memory
66 addresses are best represented with uint32_t or uint64_t. Use fixed-size
67 types over primitive types whose size may change depending on the host
68 (32-bit versus 64-bit) so trace events don't truncate values or break
71 * Use void * for pointers to structs or for arrays. The trace.h header
72 cannot include all user-defined struct declarations and it is therefore
73 necessary to use void * for pointers to structs.
75 * For everything else, use primitive scalar types (char, int, long) with the
76 appropriate signedness.
78 Format strings should reflect the types defined in the trace event. Take
79 special care to use PRId64 and PRIu64 for int64_t and uint64_t types,
80 respectively. This ensures portability between 32- and 64-bit platforms.
82 === Hints for adding new trace events ===
84 1. Trace state changes in the code. Interesting points in the code usually
85 involve a state change like starting, stopping, allocating, freeing. State
86 changes are good trace events because they can be used to understand the
87 execution of the system.
89 2. Trace guest operations. Guest I/O accesses like reading device registers
90 are good trace events because they can be used to understand guest
93 3. Use correlator fields so the context of an individual line of trace output
94 can be understood. For example, trace the pointer returned by malloc and
95 used as an argument to free. This way mallocs and frees can be matched up.
96 Trace events with no context are not very useful.
98 4. Name trace events after their function. If there are multiple trace events
99 in one function, append a unique distinguisher at the end of the name.
101 == Generic interface and monitor commands ==
103 You can programmatically query and control the state of trace events through a
104 backend-agnostic interface provided by the header "trace/control.h".
106 Note that some of the backends do not provide an implementation for some parts
107 of this interface, in which case QEMU will just print a warning (please refer to
108 header "trace/control.h" to see which routines are backend-dependent).
110 The state of events can also be queried and modified through monitor commands:
113 View available trace events and their state. State 1 means enabled, state 0
116 * trace-event NAME on|off
117 Enable/disable a given trace event or a group of events (using wildcards).
119 The "-trace events=<file>" command line argument can be used to enable the
120 events listed in <file> from the very beginning of the program. This file must
121 contain one event name per line.
123 If a line in the "-trace events=<file>" file begins with a '-', the trace event
124 will be disabled instead of enabled. This is useful when a wildcard was used
125 to enable an entire family of events but one noisy event needs to be disabled.
127 Wildcard matching is supported in both the monitor command "trace-event" and the
128 events list file. That means you can enable/disable the events having a common
129 prefix in a batch. For example, virtio-blk trace events could be enabled using
130 the following monitor command:
132 trace-event virtio_blk_* on
136 The "tracetool" script automates tedious trace event code generation and also
137 keeps the trace event declarations independent of the trace backend. The trace
138 events are not tightly coupled to a specific trace backend, such as LTTng or
139 SystemTap. Support for trace backends can be added by extending the "tracetool"
142 The trace backends are chosen at configure time:
144 ./configure --enable-trace-backends=simple
146 For a list of supported trace backends, try ./configure --help or see below.
147 If multiple backends are enabled, the trace is sent to them all.
149 The following subsections describe the supported trace backends.
153 The "nop" backend generates empty trace event functions so that the compiler
154 can optimize out trace events completely. This is the default and imposes no
157 Note that regardless of the selected trace backend, events with the "disable"
158 property will be generated with the "nop" backend.
162 The "stderr" backend sends trace events directly to standard error. This
163 effectively turns trace events into debug printfs.
165 This is the simplest backend and can be used together with existing code that
170 The "simple" backend supports common use cases and comes as part of the QEMU
171 source tree. It may not be as powerful as platform-specific or third-party
172 trace backends but it is portable. This is the recommended trace backend
173 unless you have specific needs for more advanced backends.
177 The "ftrace" backend writes trace data to ftrace marker. This effectively
178 sends trace events to ftrace ring buffer, and you can compare qemu trace
179 data and kernel(especially kvm.ko when using KVM) trace data.
181 if you use KVM, enable kvm events in ftrace:
183 # echo 1 > /sys/kernel/debug/tracing/events/kvm/enable
185 After running qemu by root user, you can get the trace:
187 # cat /sys/kernel/debug/tracing/trace
189 Restriction: "ftrace" backend is restricted to Linux only.
191 ==== Monitor commands ====
193 * trace-file on|off|flush|set <path>
194 Enable/disable/flush the trace file or set the trace file name.
196 ==== Analyzing trace files ====
198 The "simple" backend produces binary trace files that can be formatted with the
199 simpletrace.py script. The script takes the "trace-events" file and the binary
202 ./scripts/simpletrace.py trace-events trace-12345
204 You must ensure that the same "trace-events" file was used to build QEMU,
205 otherwise trace event declarations may have changed and output will not be
208 === LTTng Userspace Tracer ===
210 The "ust" backend uses the LTTng Userspace Tracer library. There are no
211 monitor commands built into QEMU, instead UST utilities should be used to list,
212 enable/disable, and dump traces.
214 Package lttng-tools is required for userspace tracing. You must ensure that the
215 current user belongs to the "tracing" group, or manually launch the
216 lttng-sessiond daemon for the current user prior to running any instance of
219 While running an instrumented QEMU, LTTng should be able to list all available
224 Create tracing session:
226 lttng create mysession
230 lttng enable-event qemu:g_malloc -u
232 Where the events can either be a comma-separated list of events, or "-a" to
233 enable all tracepoint events. Start and stop tracing as needed:
242 Destroy tracing session:
246 Babeltrace can be used at any later time to view the trace:
248 babeltrace $HOME/lttng-traces/mysession-<date>-<time>
252 The "dtrace" backend uses DTrace sdt probes but has only been tested with
253 SystemTap. When SystemTap support is detected a .stp file with wrapper probes
254 is generated to make use in scripts more convenient. This step can also be
255 performed manually after a build in order to change the binary name in the .stp
258 scripts/tracetool.py --backends=dtrace --format=stap \
259 --binary path/to/qemu-binary \
260 --target-type system \
261 --target-name x86_64 \
262 <trace-events >qemu.stp
264 == Trace event properties ==
266 Each event in the "trace-events" file can be prefixed with a space-separated
267 list of zero or more of the following event properties.
271 If a specific trace event is going to be invoked a huge number of times, this
272 might have a noticeable performance impact even when the event is
273 programmatically disabled.
275 In this case you should declare such event with the "disable" property. This
276 will effectively disable the event at compile time (by using the "nop" backend),
277 thus having no performance impact at all on regular builds (i.e., unless you
278 edit the "trace-events" file).
280 In addition, there might be cases where relatively complex computations must be
281 performed to generate values that are only used as arguments for a trace
282 function. In these cases you can use the macro 'TRACE_${EVENT_NAME}_ENABLED' to
283 guard such computations and avoid its compilation when the event is disabled:
285 #include "trace.h" /* needed for trace event prototype */
287 void *qemu_vmalloc(size_t size)
290 size_t align = QEMU_VMALLOC_ALIGN;
293 align = getpagesize();
295 ptr = qemu_memalign(align, size);
296 if (TRACE_QEMU_VMALLOC_ENABLED) { /* preprocessor macro */
298 /* some complex computations to produce the 'complex' value */
299 trace_qemu_vmalloc(size, ptr, complex);
304 You can check both if the event has been disabled and is dynamically enabled at
305 the same time using the 'trace_event_get_state' routine (see header
306 "trace/control.h" for more information).
310 Guest code generated by TCG can be traced by defining an event with the "tcg"
311 event property. Internally, this property generates two events:
312 "<eventname>_trans" to trace the event at translation time, and
313 "<eventname>_exec" to trace the event at execution time.
315 Instead of using these two events, you should instead use the function
316 "trace_<eventname>_tcg" during translation (TCG code generation). This function
317 will automatically call "trace_<eventname>_trans", and will generate the
318 necessary TCG code to call "trace_<eventname>_exec" during guest code execution.
320 Events with the "tcg" property can be declared in the "trace-events" file with a
321 mix of native and TCG types, and "trace_<eventname>_tcg" will gracefully forward
322 them to the "<eventname>_trans" and "<eventname>_exec" events. Since TCG values
323 are not known at translation time, these are ignored by the "<eventname>_trans"
324 event. Because of this, the entry in the "trace-events" file needs two printing
325 formats (separated by a comma):
327 tcg foo(uint8_t a1, TCGv_i32 a2) "a1=%d", "a1=%d a2=%d"
331 #include "trace-tcg.h"
333 void some_disassembly_func (...)
337 trace_foo_tcg(a1, a2);
340 This will immediately call:
342 void trace_foo_trans(uint8_t a1);
344 and will generate the TCG code to call:
346 void trace_foo(uint8_t a1, uint32_t a2);
350 Identifies events that trace vCPU-specific information. It implicitly adds a
351 "CPUState*" argument, and extends the tracing print format to show the vCPU
352 information. If used together with the "tcg" property, it adds a second
353 "TCGv_env" argument that must point to the per-target global TCG register that
354 points to the vCPU when guest code is executed (usually the "cpu_env" variable).
356 The following example events:
358 foo(uint32_t a) "a=%x"
359 vcpu bar(uint32_t a) "a=%x"
360 tcg vcpu baz(uint32_t a) "a=%x", "a=%x"
364 #include "trace-tcg.h"
369 void some_disassembly_func(...)
371 /* trace emitted at this point */
373 /* trace emitted at this point */
374 trace_bar(ENV_GET_CPU(env), 0xd2);
375 /* trace emitted at this point (env) and when guest code is executed (cpu_env) */
376 trace_baz_tcg(ENV_GET_CPU(env), cpu_env, 0xd3);
379 If the translating vCPU has address 0xc1 and code is later executed by vCPU
380 0xc2, this would be an example output:
382 // at guest code translation
385 baz_trans cpu=0xc1 a=0xd3
386 // at guest code execution
387 baz_exec cpu=0xc2 a=0xd3