1 ==================================
2 The QEMU build system architecture
3 ==================================
5 This document aims to help developers understand the architecture of the
6 QEMU build system. As with projects using GNU autotools, the QEMU build
7 system has two stages; first the developer runs the "configure" script
8 to determine the local build environment characteristics, then they run
9 "make" to build the project. This is about where the similarities with
10 GNU autotools end, so try to forget what you know about them.
12 The two general ways to perform a build are as follows:
14 - build artifacts outside of QEMU source tree entirely::
22 - build artifacts in a subdir of QEMU source tree::
29 Most of the actual build process uses Meson under the hood, therefore
30 build artifacts cannot be placed in the source tree itself.
36 The configure script has five tasks:
38 - detect the host architecture
40 - list the targets for which to build emulators; the list of
41 targets also affects which firmware binaries and tests to build
43 - find the compilers (native and cross) used to build executables,
44 firmware and tests. The results are written as either Makefile
45 fragments (``config-host.mak``) or a Meson machine file
46 (``config-meson.cross``)
48 - create a virtual environment in which all Python code runs during
49 the build, and possibly install packages into it from PyPI
51 - invoke Meson in the virtual environment, to perform the actual
52 configuration step for the emulator build
54 The configure script automatically recognizes command line options for
55 which a same-named Meson option exists; dashes in the command line are
56 replaced with underscores.
58 Almost all QEMU developers that need to modify the build system will
59 only be concerned with Meson, and therefore can skip the rest of this
63 Modifying ``configure``
64 -----------------------
66 ``configure`` is a shell script; it uses ``#!/bin/sh`` and therefore
67 should be compatible with any POSIX shell. It is important to avoid
68 using bash-isms to avoid breaking development platforms where bash is
71 The configure script provides a variety of functions to help writing
72 portable shell code and providing consistent behavior across architectures
73 and operating systems:
75 ``error_exit $MESSAGE $MORE...``
76 Print $MESSAGE to stderr, followed by $MORE... and then exit from the
77 configure script with non-zero status.
80 Determine if $COMMAND exists in the current environment, either as a
81 shell builtin, or executable binary, returning 0 on success. The
82 replacement in Meson is ``find_program()``.
84 ``probe_target_compiler $TARGET``
85 Detect a cross compiler and cross tools for the QEMU target $TARGET (e.g.,
86 ``$CPU-softmmu``, ``$CPU-linux-user``, ``$CPU-bsd-user``). If a working
87 compiler is present, return success and set variables ``$target_cc``,
88 ``$target_ar``, etc. to non-empty values.
90 ``write_target_makefile``
91 Write a Makefile fragment to stdout, exposing the result of the most
92 ``probe_target_compiler`` call as the usual Make variables (``CC``,
93 ``AR``, ``LD``, etc.).
96 Configure does not generally perform tests for compiler options beyond
97 basic checks to detect the host platform and ensure the compiler is
98 functioning. These are performed using a few more helper functions:
100 ``compile_object $CFLAGS``
101 Attempt to compile a test program with the system C compiler using
102 $CFLAGS. The test program must have been previously written to a file
105 ``compile_prog $CFLAGS $LDFLAGS``
106 Attempt to compile a test program with the system C compiler using
107 $CFLAGS and link it with the system linker using $LDFLAGS. The test
108 program must have been previously written to a file called $TMPC.
110 ``check_define $NAME``
111 Determine if the macro $NAME is defined by the system C compiler.
113 ``do_compiler $CC $ARGS...``
114 Attempt to run the C compiler $CC, passing it $ARGS... This function
115 does not use flags passed via options such as ``--extra-cflags``, and
116 therefore can be used to check for cross compilers. However, most
117 such checks are done at ``make`` time instead (see for example the
118 ``cc-option`` macro in ``pc-bios/option-rom/Makefile``).
121 Write a minimal C program main() function to the temporary file
125 Python virtual environments and the build process
126 -------------------------------------------------
128 An important step in ``configure`` is to create a Python virtual
129 environment (venv) during the configuration phase. The Python interpreter
130 comes from the ``--python`` command line option, the ``$PYTHON`` variable
131 from the environment, or the system PATH, in this order. The venv resides
132 in the ``pyvenv`` directory in the build tree, and provides consistency
133 in how the build process runs Python code.
135 At this stage, ``configure`` also queries the chosen Python interpreter
136 about QEMU's build dependencies. Note that the build process does *not*
137 look for ``meson``, ``sphinx-build`` or ``avocado`` binaries in the PATH;
138 likewise, there are no options such as ``--meson`` or ``--sphinx-build``.
139 This avoids a potential mismatch, where Meson and Sphinx binaries on the
140 PATH might operate in a different Python environment than the one chosen
141 by the user during the build process. On the other hand, it introduces
142 a potential source of confusion where the user installs a dependency but
143 ``configure`` is not able to find it. When this happens, the dependency
144 was installed in the ``site-packages`` directory of another interpreter,
145 or with the wrong ``pip`` program.
147 If a package is available for the chosen interpreter, ``configure``
148 prepares a small script that invokes it from the venv itself[#distlib]_.
149 If not, ``configure`` can also optionally install dependencies in the
150 virtual environment with ``pip``, either from wheels in ``python/wheels``
151 or by downloading the package with PyPI. Downloading can be disabled with
152 ``--disable-download``; and anyway, it only happens when a ``configure``
153 option (currently, only ``--enable-docs``) is explicitly enabled but
154 the dependencies are not present[#pip]_.
156 .. [#distlib] The scripts are created based on the package's metadata,
157 specifically the ``console_script`` entry points. This is the
158 same mechanism that ``pip`` uses when installing a package.
159 Currently, in all cases it would be possible to use ``python -m``
160 instead of an entry point script, which makes this approach a
161 bit overkill. On the other hand, creating the scripts is
162 future proof and it makes the contents of the ``pyvenv/bin``
163 directory more informative. Portability is also not an issue,
164 because the Python Packaging Authority provides a package
165 ``distlib.scripts`` to perform this task.
167 .. [#pip] ``pip`` might also be used when running ``make check-avocado``
168 if downloading is enabled, to ensure that Avocado is
171 The required versions of the packages are stored in a configuration file
172 ``pythondeps.toml``. The format is custom to QEMU, but it is documented
173 at the top of the file itself and it should be easy to understand. The
174 requirements should make it possible to use the version that is packaged
175 that is provided by supported distros.
177 When dependencies are downloaded, instead, ``configure`` uses a "known
178 good" version that is also listed in ``pythondeps.toml``. In this
179 scenario, ``pythondeps.toml`` behaves like the "lock file" used by
180 ``cargo``, ``poetry`` or other dependency management systems.
183 Bundled Python packages
184 -----------------------
186 Python packages that are **mandatory** dependencies to build QEMU,
187 but are not available in all supported distros, are bundled with the
188 QEMU sources. Currently this includes Meson (outdated in CentOS 8
189 and derivatives, Ubuntu 20.04 and 22.04, and openSUSE Leap) and tomli
190 (absent in Ubuntu 20.04).
192 If you need to update these, please do so by modifying and rerunning
193 ``python/scripts/vendor.py``. This script embeds the sha256 hash of
194 package sources and checks it. The pypi.org web site provides an easy
195 way to retrieve the sha256 hash of the sources.
201 The Meson build system describes the build and install process for:
203 1) executables, which include:
205 - Tools - ``qemu-img``, ``qemu-nbd``, ``qemu-ga`` (guest agent), etc
207 - System emulators - ``qemu-system-$ARCH``
209 - Userspace emulators - ``qemu-$ARCH``
215 3) ROMs, whether provided as binary blobs in the QEMU distributions
216 or cross compiled under the direction of the configure script
218 4) other data files, such as icons or desktop files
220 All executables are built by default, except for some ``contrib/``
221 binaries that are known to fail to build on some platforms (for example
222 32-bit or big-endian platforms). Tests are also built by default,
223 though that might change in the future.
225 The source code is highly modularized, split across many files to
226 facilitate building of all of these components with as little duplicated
227 compilation as possible. Using the Meson "sourceset" functionality,
228 ``meson.build`` files group the source files in rules that are
229 enabled according to the available system libraries and to various
230 configuration symbols. Sourcesets belong to one of four groups:
232 Subsystem sourcesets:
233 Various subsystems that are common to both tools and emulators have
234 their own sourceset, for example ``block_ss`` for the block device subsystem,
235 ``chardev_ss`` for the character device subsystem, etc. These sourcesets
236 are then turned into static libraries as follows::
238 libchardev = static_library('chardev', chardev_ss.sources(),
240 build_by_default: false)
242 chardev = declare_dependency(link_whole: libchardev)
244 As of Meson 0.55.1, the special ``.fa`` suffix should be used for everything
245 that is used with ``link_whole``, to ensure that the link flags are placed
246 correctly in the command line.
248 Target-independent emulator sourcesets:
249 Various general purpose helper code is compiled only once and
250 the .o files are linked into all output binaries that need it.
251 This includes error handling infrastructure, standard data structures,
252 platform portability wrapper functions, etc.
254 Target-independent code lives in the ``common_ss``, ``system_ss`` and
255 ``user_ss`` sourcesets. ``common_ss`` is linked into all emulators,
256 ``system_ss`` only in system emulators, ``user_ss`` only in user-mode
259 Target-dependent emulator sourcesets:
260 In the target-dependent set lives CPU emulation, some device emulation and
261 much glue code. This sometimes also has to be compiled multiple times,
262 once for each target being built. Target-dependent files are included
263 in the ``specific_ss`` sourceset.
265 Each emulator also includes sources for files in the ``hw/`` and ``target/``
266 subdirectories. The subdirectory used for each emulator comes
267 from the target's definition of ``TARGET_BASE_ARCH`` or (if missing)
268 ``TARGET_ARCH``, as found in ``default-configs/targets/*.mak``.
270 Each subdirectory in ``hw/`` adds one sourceset to the ``hw_arch`` dictionary,
273 arm_ss = ss.source_set()
274 arm_ss.add(files('boot.c'), fdt)
276 hw_arch += {'arm': arm_ss}
278 The sourceset is only used for system emulators.
280 Each subdirectory in ``target/`` instead should add one sourceset to each
281 of the ``target_arch`` and ``target_system_arch``, which are used respectively
282 for all emulators and for system emulators only. For example::
284 arm_ss = ss.source_set()
285 arm_system_ss = ss.source_set()
287 target_arch += {'arm': arm_ss}
288 target_system_arch += {'arm': arm_system_ss}
291 There are two dictionaries for modules: ``modules`` is used for
292 target-independent modules and ``target_modules`` is used for
293 target-dependent modules. When modules are disabled the ``module``
294 source sets are added to ``system_ss`` and the ``target_modules``
295 source sets are added to ``specific_ss``.
297 Both dictionaries are nested. One dictionary is created per
298 subdirectory, and these per-subdirectory dictionaries are added to
299 the toplevel dictionaries. For example::
301 hw_display_modules = {}
302 qxl_ss = ss.source_set()
304 hw_display_modules += { 'qxl': qxl_ss }
305 modules += { 'hw-display': hw_display_modules }
308 All binaries link with a static library ``libqemuutil.a``. This library
309 is built from several sourcesets; most of them however host generated
310 code, and the only two of general interest are ``util_ss`` and ``stub_ss``.
312 The separation between these two is purely for documentation purposes.
313 ``util_ss`` contains generic utility files. Even though this code is only
314 linked in some binaries, sometimes it requires hooks only in some of
315 these and depend on other functions that are not fully implemented by
316 all QEMU binaries. ``stub_ss`` links dummy stubs that will only be linked
317 into the binary if the real implementation is not present. In a way,
318 the stubs can be thought of as a portable implementation of the weak
322 The following files concur in the definition of which files are linked
325 ``default-configs/devices/*.mak``
326 The files under ``default-configs/devices/`` control the boards and devices
327 that are built into each QEMU system emulation targets. They merely contain
328 a list of config variable definitions such as::
330 include arm-softmmu.mak
331 CONFIG_XLNX_ZYNQMP_ARM=y
335 These files are processed together with ``default-configs/devices/*.mak`` and
336 describe the dependencies between various features, subsystems and
337 device models. They are described in :ref:`kconfig`
339 ``default-configs/targets/*.mak``
340 These files mostly define symbols that appear in the ``*-config-target.h``
341 file for each emulator [#cfgtarget]_. However, the ``TARGET_ARCH``
342 and ``TARGET_BASE_ARCH`` will also be used to select the ``hw/`` and
343 ``target/`` subdirectories that are compiled into each target.
345 .. [#cfgtarget] This header is included by ``qemu/osdep.h`` when
346 compiling files from the target-specific sourcesets.
348 These files rarely need changing unless you are adding a completely
349 new target, or enabling new devices or hardware for a particular
350 system/userspace emulation target
356 Compiler checks can be as simple as the following::
358 config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h'))
360 A more complex task such as adding a new dependency usually
361 comprises the following tasks:
363 - Add a Meson build option to meson_options.txt.
365 - Add code to perform the actual feature check.
367 - Add code to include the feature status in ``config-host.h``
369 - Add code to print out the feature status in the configure summary
372 Taking the probe for SDL2_Image as an example, we have the following
373 in ``meson_options.txt``::
375 option('sdl_image', type : 'feature', value : 'auto',
376 description: 'SDL Image support for icons')
378 Unless the option was given a non-``auto`` value (on the configure
379 command line), the detection code must be performed only if the
380 dependency will be used::
382 sdl_image = not_found
383 if not get_option('sdl_image').auto() or have_system
384 sdl_image = dependency('SDL2_image', required: get_option('sdl_image'),
385 method: 'pkg-config')
388 This avoids warnings on static builds of user-mode emulators, for example.
389 Most of the libraries used by system-mode emulators are not available for
392 The other supporting code is generally simple::
394 # Create config-host.h (if applicable)
395 config_host_data.set('CONFIG_SDL_IMAGE', sdl_image.found())
398 summary_info += {'SDL image support': sdl_image.found()}
400 For the configure script to parse the new option, the
401 ``scripts/meson-buildoptions.sh`` file must be up-to-date; ``make
402 update-buildoptions`` (or just ``make``) will take care of updating it.
408 Meson has a special convention for invoking Python scripts: if their
409 first line is ``#! /usr/bin/env python3`` and the file is *not* executable,
410 find_program() arranges to invoke the script under the same Python
411 interpreter that was used to invoke Meson. This is the most common
412 and preferred way to invoke support scripts from Meson build files,
413 because it automatically uses the value of configure's --python= option.
415 In case the script is not written in Python, use a ``#! /usr/bin/env ...``
416 line and make the script executable.
418 Scripts written in Python, where it is desirable to make the script
419 executable (for example for test scripts that developers may want to
420 invoke from the command line, such as tests/qapi-schema/test-qapi.py),
421 should be invoked through the ``python`` variable in meson.build. For
424 test('QAPI schema regression tests', python,
425 args: files('test-qapi.py'),
426 env: test_env, suite: ['qapi-schema', 'qapi-frontend'])
428 This is needed to obey the --python= option passed to the configure
429 script, which may point to something other than the first python3
432 By the time Meson runs, Python dependencies are available in the virtual
433 environment and should be invoked through the scripts that ``configure``
434 places under ``pyvenv``. One way to do so is as follows, using Meson's
435 ``find_program`` function::
437 sphinx_build = find_program(
438 fs.parent(python.full_path()) / 'sphinx-build',
439 required: get_option('docs'))
445 The next step in building QEMU is to invoke make. GNU Make is required
446 to build QEMU, and may be installed as ``gmake`` on some hosts.
448 The output of Meson is a ``build.ninja`` file, which is used with the
449 Ninja build tool. However, QEMU's build comprises other components than
450 just the emulators (namely firmware and the tests in ``tests/tcg``) which
451 need different cross compilers. The QEMU Makefile wraps both Ninja and
452 the smaller build systems for firmware and tests; it also takes care of
453 running ``configure`` again when the script changes. Apart from invoking
454 these sub-Makefiles, the resulting build is largely non-recursive.
456 Tests, whether defined in ``meson.build`` or not, are also ran by the
457 Makefile with the traditional ``make check`` phony target, while benchmarks
458 are run with ``make bench``. Meson test suites such as ``unit`` can be ran
459 with ``make check-unit``, and ``make check-tcg`` builds and runs "non-Meson"
460 tests for all targets.
462 If desired, it is also possible to use ``ninja`` and ``meson test``,
463 respectively to build emulators and run tests defined in meson.build.
464 The main difference is that ``make`` needs the ``-jN`` flag in order to
465 enable parallel builds or tests.
471 Print a help message for the most common build targets.
474 Print the value of the variable VAR. Useful for debugging the build
478 Important files for the build system
479 ====================================
481 Statically defined files
482 ------------------------
484 The following key files are statically defined in the source tree, with
485 the rules needed to build QEMU. Their behaviour is influenced by a
486 number of dynamically created files listed later.
489 The main entry point used when invoking make to build all the components
490 of QEMU. The default 'all' target will naturally result in the build of
494 The meson.build file in the root directory is the main entry point for the
495 Meson build system, and it coordinates the configuration and build of all
496 executables. Build rules for various subdirectories are included in
497 other meson.build files spread throughout the QEMU source tree.
499 ``python/scripts/mkvenv.py``
500 A wrapper for the Python ``venv`` and ``distlib.scripts`` packages.
501 It handles creating the virtual environment, creating scripts in
502 ``pyvenv/bin``, and calling ``pip`` to install dependencies.
504 ``tests/Makefile.include``
505 Rules for external test harnesses. These include the TCG tests
506 and the Avocado-based integration tests.
508 ``tests/docker/Makefile.include``
509 Rules for Docker tests. Like ``tests/Makefile.include``, this file is
510 included directly by the top level Makefile, anything defined in this
511 file will influence the entire build system.
513 ``tests/vm/Makefile.include``
514 Rules for VM-based tests. Like ``tests/Makefile.include``, this file is
515 included directly by the top level Makefile, anything defined in this
516 file will influence the entire build system.
518 Dynamically created files
519 -------------------------
521 The following files are generated at run-time in order to control the
522 behaviour of the Makefiles. This avoids the need for QEMU makefiles to
523 go through any pre-processing as seen with autotools, where configure
524 generates ``Makefile`` from ``Makefile.in``.
529 When configure has determined the characteristics of the build host it
530 will write the paths to various tools to this file, for use in ``Makefile``
531 and to a smaller extent ``meson.build``.
533 ``config-host.mak`` is also used as a dependency checking mechanism. If make
534 sees that the modification timestamp on configure is newer than that on
535 ``config-host.mak``, then configure will be re-run.
537 ``config-meson.cross``
539 A Meson "cross file" (or native file) used to communicate the paths to
540 the toolchain and other configuration options.
544 A small shell script that will invoke configure again with the same
545 environment variables that were set during the first run. It's used to
546 rerun configure after changes to the source code, but it can also be
547 inspected manually to check the contents of the environment.
551 A set of Makefile dependencies that order the build and execution of
552 firmware and tests after the container images and emulators that they
555 ``pc-bios/*/config.mak``, ``tests/tcg/config-host.mak``, ``tests/tcg/*/config-target.mak``
557 Configuration variables used to build the firmware and TCG tests,
558 including paths to cross compilation toolchains.
562 A Python virtual environment that is used for all Python code running
563 during the build. Using a virtual environment ensures that even code
564 that is run via ``sphinx-build``, ``meson`` etc. uses the same interpreter
570 Used by C code to determine the properties of the build environment
571 and the set of enabled features for the entire build.
573 ``${TARGET-NAME}-config-devices.mak``
574 TARGET-NAME is the name of a system emulator. The file is
575 generated by Meson using files under ``configs/devices`` as input.
577 ``${TARGET-NAME}-config-target.mak``
578 TARGET-NAME is the name of a system or usermode emulator. The file is
579 generated by Meson using files under ``configs/targets`` as input.
581 ``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h``
582 Used by C code to determine the properties and enabled
583 features for each target. enabled. They are generated from
584 the contents of the corresponding ``*.mak`` files using Meson's
585 ``configure_file()`` function; each target can include them using
586 the ``CONFIG_TARGET`` and ``CONFIG_DEVICES`` macro respectively.
595 A Makefile include that bridges to ninja for the actual build. The
596 Makefile is mostly a list of targets that Meson included in build.ninja.
599 The Makefile definitions that let "make check" run tests defined in
600 meson.build. The rules are produced from Meson's JSON description of
601 tests (obtained with "meson introspect --tests") through the script
602 scripts/mtest2make.py.