5 This document describes the virtual-device fuzzing infrastructure in QEMU and
6 how to use it to implement additional fuzzers.
11 Fuzzing operates by passing inputs to an entry point/target function. The
12 fuzzer tracks the code coverage triggered by the input. Based on these
13 findings, the fuzzer mutates the input and repeats the fuzzing.
15 To fuzz QEMU, we rely on libfuzzer. Unlike other fuzzers such as AFL, libfuzzer
16 is an *in-process* fuzzer. For the developer, this means that it is their
17 responsibility to ensure that state is reset between fuzzing-runs.
22 *NOTE*: If possible, build a 32-bit binary. When forking, the 32-bit fuzzer is
23 much faster, since the page-map has a smaller size. This is due to the fact that
24 AddressSanitizer maps ~20TB of memory, as part of its detection. This results
25 in a large page-map, and a much slower ``fork()``.
27 To build the fuzzers, install a recent version of clang:
28 Configure with (substitute the clang binaries with the version you installed).
29 Here, enable-sanitizers, is optional but it allows us to reliably detect bugs
30 such as out-of-bounds accesses, use-after-frees, double-frees etc.::
32 CC=clang-8 CXX=clang++-8 /path/to/configure --enable-fuzzing \
35 Fuzz targets are built similarly to system targets::
39 This builds ``./qemu-fuzz-i386``
41 The first option to this command is: ``--fuzz-target=FUZZ_NAME``
42 To list all of the available fuzzers run ``qemu-fuzz-i386`` with no arguments.
46 ./qemu-fuzz-i386 --fuzz-target=virtio-scsi-fuzz
48 Internally, libfuzzer parses all arguments that do not begin with ``"--"``.
49 Information about these is available by passing ``-help=1``
51 Now the only thing left to do is wait for the fuzzer to trigger potential
54 Useful libFuzzer flags
55 ----------------------
57 As mentioned above, libFuzzer accepts some arguments. Passing ``-help=1`` will
58 list the available arguments. In particular, these arguments might be helpful:
60 * ``CORPUS_DIR/`` : Specify a directory as the last argument to libFuzzer.
61 libFuzzer stores each "interesting" input in this corpus directory. The next
62 time you run libFuzzer, it will read all of the inputs from the corpus, and
63 continue fuzzing from there. You can also specify multiple directories.
64 libFuzzer loads existing inputs from all specified directories, but will only
65 write new ones to the first one specified.
67 * ``-max_len=4096`` : specify the maximum byte-length of the inputs libFuzzer
70 * ``-close_fd_mask={1,2,3}`` : close, stderr, or both. Useful for targets that
71 trigger many debug/error messages, or create output on the serial console.
73 * ``-jobs=4 -workers=4`` : These arguments configure libFuzzer to run 4 fuzzers in
74 parallel (4 fuzzing jobs in 4 worker processes). Alternatively, with only
75 ``-jobs=N``, libFuzzer automatically spawns a number of workers less than or equal
76 to half the available CPU cores. Replace 4 with a number appropriate for your
77 machine. Make sure to specify a ``CORPUS_DIR``, which will allow the parallel
78 fuzzers to share information about the interesting inputs they find.
80 * ``-use_value_profile=1`` : For each comparison operation, libFuzzer computes
81 ``(caller_pc&4095) | (popcnt(Arg1 ^ Arg2) << 12)`` and places this in the
82 coverage table. Useful for targets with "magic" constants. If Arg1 came from
83 the fuzzer's input and Arg2 is a magic constant, then each time the Hamming
84 distance between Arg1 and Arg2 decreases, libFuzzer adds the input to the
87 * ``-shrink=1`` : Tries to make elements of the corpus "smaller". Might lead to
88 better coverage performance, depending on the target.
90 Note that libFuzzer's exact behavior will depend on the version of
91 clang and libFuzzer used to build the device fuzzers.
93 Generating Coverage Reports
94 ---------------------------
96 Code coverage is a crucial metric for evaluating a fuzzer's performance.
97 libFuzzer's output provides a "cov: " column that provides a total number of
98 unique blocks/edges covered. To examine coverage on a line-by-line basis we
99 can use Clang coverage:
101 1. Configure libFuzzer to store a corpus of all interesting inputs (see
103 2. ``./configure`` the QEMU build with ::
106 --extra-cflags="-fprofile-instr-generate -fcoverage-mapping"
108 3. Re-run the fuzzer. Specify $CORPUS_DIR/* as an argument, telling libfuzzer
109 to execute all of the inputs in $CORPUS_DIR and exit. Once the process
110 exits, you should find a file, "default.profraw" in the working directory.
111 4. Execute these commands to generate a detailed HTML coverage-report::
113 llvm-profdata merge -output=default.profdata default.profraw
114 llvm-cov show ./path/to/qemu-fuzz-i386 -instr-profile=default.profdata \
115 --format html -output-dir=/path/to/output/report
120 Coverage over virtual devices can be improved by adding additional fuzzers.
121 Fuzzers are kept in ``tests/qtest/fuzz/`` and should be added to
122 ``tests/qtest/fuzz/Makefile.include``
124 Fuzzers can rely on both qtest and libqos to communicate with virtual devices.
126 1. Create a new source file. For example ``tests/qtest/fuzz/foo-device-fuzz.c``.
128 2. Write the fuzzing code using the libqtest/libqos API. See existing fuzzers
131 3. Register the fuzzer in ``tests/fuzz/Makefile.include`` by appending the
132 corresponding object to fuzz-obj-y
134 Fuzzers can be more-or-less thought of as special qtest programs which can
135 modify the qtest commands and/or qtest command arguments based on inputs
136 provided by libfuzzer. Libfuzzer passes a byte array and length. Commonly the
137 fuzzer loops over the byte-array interpreting it as a list of qtest commands,
138 addresses, or values.
143 Writing a fuzz target can be a lot of effort (especially if a device driver has
144 not be built-out within libqos). Many devices can be fuzzed to some degree,
145 without any device-specific code, using the generic-fuzz target.
147 The generic-fuzz target is capable of fuzzing devices over their PIO, MMIO,
148 and DMA input-spaces. To apply the generic-fuzz to a device, we need to define
149 two env-variables, at minimum:
151 * ``QEMU_FUZZ_ARGS=`` is the set of QEMU arguments used to configure a machine, with
152 the device attached. For example, if we want to fuzz the virtio-net device
153 attached to a pc-i440fx machine, we can specify::
155 QEMU_FUZZ_ARGS="-M pc -nodefaults -netdev user,id=user0 \
156 -device virtio-net,netdev=user0"
158 * ``QEMU_FUZZ_OBJECTS=`` is a set of space-delimited strings used to identify
159 the MemoryRegions that will be fuzzed. These strings are compared against
160 MemoryRegion names and MemoryRegion owner names, to decide whether each
161 MemoryRegion should be fuzzed. These strings support globbing. For the
162 virtio-net example, we could use one of ::
164 QEMU_FUZZ_OBJECTS='virtio-net'
165 QEMU_FUZZ_OBJECTS='virtio*'
166 QEMU_FUZZ_OBJECTS='virtio* pcspk' # Fuzz the virtio devices and the speaker
167 QEMU_FUZZ_OBJECTS='*' # Fuzz the whole machine``
169 The ``"info mtree"`` and ``"info qom-tree"`` monitor commands can be especially
170 useful for identifying the ``MemoryRegion`` and ``Object`` names used for
173 As a generic rule-of-thumb, the more ``MemoryRegions``/Devices we match, the
174 greater the input-space, and the smaller the probability of finding crashing
175 inputs for individual devices. As such, it is usually a good idea to limit the
176 fuzzer to only a few ``MemoryRegions``.
178 To ensure that these env variables have been configured correctly, we can use::
180 ./qemu-fuzz-i386 --fuzz-target=generic-fuzz -runs=0
182 The output should contain a complete list of matched MemoryRegions.
184 Implementation Details / Fuzzer Lifecycle
185 -----------------------------------------
187 The fuzzer has two entrypoints that libfuzzer calls. libfuzzer provides it's
188 own ``main()``, which performs some setup, and calls the entrypoints:
190 ``LLVMFuzzerInitialize``: called prior to fuzzing. Used to initialize all of the
193 ``LLVMFuzzerTestOneInput``: called for each fuzzing run. Processes the input and
194 resets the state at the end of each run.
198 ``LLVMFuzzerInitialize`` parses the arguments to the fuzzer (must start with two
199 dashes, so they are ignored by libfuzzer ``main()``). Currently, the arguments
200 select the fuzz target. Then, the qtest client is initialized. If the target
201 requires qos, qgraph is set up and the QOM/LIBQOS modules are initialized.
202 Then the QGraph is walked and the QEMU cmd_line is determined and saved.
204 After this, the ``vl.c:qemu_main`` is called to set up the guest. There are
205 target-specific hooks that can be called before and after qemu_main, for
206 additional setup(e.g. PCI setup, or VM snapshotting).
208 ``LLVMFuzzerTestOneInput``: Uses qtest/qos functions to act based on the fuzz
209 input. It is also responsible for manually calling ``main_loop_wait`` to ensure
210 that bottom halves are executed and any cleanup required before the next input.
212 Since the same process is reused for many fuzzing runs, QEMU state needs to
213 be reset at the end of each run. There are currently two implemented
214 options for resetting state:
216 - Reboot the guest between runs.
217 - *Pros*: Straightforward and fast for simple fuzz targets.
219 - *Cons*: Depending on the device, does not reset all device state. If the
220 device requires some initialization prior to being ready for fuzzing (common
221 for QOS-based targets), this initialization needs to be done after each
224 - *Example target*: ``i440fx-qtest-reboot-fuzz``
226 - Run each test case in a separate forked process and copy the coverage
227 information back to the parent. This is fairly similar to AFL's "deferred"
230 - *Pros*: Relatively fast. Devices only need to be initialized once. No need to
231 do slow reboots or vmloads.
233 - *Cons*: Not officially supported by libfuzzer. Does not work well for
234 devices that rely on dedicated threads.
236 - *Example target*: ``virtio-net-fork-fuzz``