mirror of https://github.com/xemu-project/xemu.git
docs/devel: update build system docs
configure is only doing compiler and host setup now, so adjust the relevant documentation. It is also possible to build emulators with ninja directly if one is so inclined, so mention that as well. The Python virtual environment set up is a new major task of configure as well. Mention it in the list of produced files, while leaving it for a future patch to document how it works and how ``mkvenv ensure`` is used. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This commit is contained in:
parent
d71ccfa1b8
commit
fe3ab4eb2d
|
@ -4,30 +4,14 @@ The QEMU build system architecture
|
||||||
|
|
||||||
This document aims to help developers understand the architecture of the
|
This document aims to help developers understand the architecture of the
|
||||||
QEMU build system. As with projects using GNU autotools, the QEMU build
|
QEMU build system. As with projects using GNU autotools, the QEMU build
|
||||||
system has two stages, first the developer runs the "configure" script
|
system has two stages; first the developer runs the "configure" script
|
||||||
to determine the local build environment characteristics, then they run
|
to determine the local build environment characteristics, then they run
|
||||||
"make" to build the project. There is about where the similarities with
|
"make" to build the project. This is about where the similarities with
|
||||||
GNU autotools end, so try to forget what you know about them.
|
GNU autotools end, so try to forget what you know about them.
|
||||||
|
|
||||||
|
The two general ways to perform a build are as follows:
|
||||||
|
|
||||||
Stage 1: configure
|
- build artifacts outside of QEMU source tree entirely::
|
||||||
==================
|
|
||||||
|
|
||||||
The QEMU configure script is written directly in shell, and should be
|
|
||||||
compatible with any POSIX shell, hence it uses #!/bin/sh. An important
|
|
||||||
implication of this is that it is important to avoid using bash-isms on
|
|
||||||
development platforms where bash is the primary host.
|
|
||||||
|
|
||||||
In contrast to autoconf scripts, QEMU's configure is expected to be
|
|
||||||
silent while it is checking for features. It will only display output
|
|
||||||
when an error occurs, or to show the final feature enablement summary
|
|
||||||
on completion.
|
|
||||||
|
|
||||||
Because QEMU uses the Meson build system under the hood, only VPATH
|
|
||||||
builds are supported. There are two general ways to invoke configure &
|
|
||||||
perform a build:
|
|
||||||
|
|
||||||
- VPATH, build artifacts outside of QEMU source tree entirely::
|
|
||||||
|
|
||||||
cd ../
|
cd ../
|
||||||
mkdir build
|
mkdir build
|
||||||
|
@ -35,80 +19,122 @@ perform a build:
|
||||||
../qemu/configure
|
../qemu/configure
|
||||||
make
|
make
|
||||||
|
|
||||||
- VPATH, build artifacts in a subdir of QEMU source tree::
|
- build artifacts in a subdir of QEMU source tree::
|
||||||
|
|
||||||
mkdir build
|
mkdir build
|
||||||
cd build
|
cd build
|
||||||
../configure
|
../configure
|
||||||
make
|
make
|
||||||
|
|
||||||
The configure script automatically recognizes
|
Most of the actual build process uses Meson under the hood, therefore
|
||||||
command line options for which a same-named Meson option exists;
|
build artifacts cannot be placed in the source tree itself.
|
||||||
dashes in the command line are replaced with underscores.
|
|
||||||
|
|
||||||
Many checks on the compilation environment are still found in configure
|
|
||||||
rather than ``meson.build``, but new checks should be added directly to
|
|
||||||
``meson.build``.
|
|
||||||
|
|
||||||
Patches are also welcome to move existing checks from the configure
|
Stage 1: configure
|
||||||
phase to ``meson.build``. When doing so, ensure that ``meson.build`` does
|
==================
|
||||||
not use anymore the keys that you have removed from ``config-host.mak``.
|
|
||||||
Typically these will be replaced in ``meson.build`` by boolean variables,
|
|
||||||
``get_option('optname')`` invocations, or ``dep.found()`` expressions.
|
|
||||||
In general, the remaining checks have little or no interdependencies,
|
|
||||||
so they can be moved one by one.
|
|
||||||
|
|
||||||
Helper functions
|
The configure script has five tasks:
|
||||||
----------------
|
|
||||||
|
|
||||||
The configure script provides a variety of helper functions to assist
|
- detect the host architecture
|
||||||
developers in checking for system features:
|
|
||||||
|
|
||||||
``do_cc $ARGS...``
|
- list the targets for which to build emulators; the list of
|
||||||
Attempt to run the system C compiler passing it $ARGS...
|
targets also affects which firmware binaries and tests to build
|
||||||
|
|
||||||
``do_cxx $ARGS...``
|
- find the compilers (native and cross) used to build executables,
|
||||||
Attempt to run the system C++ compiler passing it $ARGS...
|
firmware and tests. The results are written as either Makefile
|
||||||
|
fragments (``config-host.mak``) or a Meson machine file
|
||||||
|
(``config-meson.cross``)
|
||||||
|
|
||||||
``compile_object $CFLAGS``
|
- create a virtual environment in which all Python code runs during
|
||||||
Attempt to compile a test program with the system C compiler using
|
the build, and possibly install packages into it from PyPI
|
||||||
$CFLAGS. The test program must have been previously written to a file
|
|
||||||
called $TMPC. The replacement in Meson is the compiler object ``cc``,
|
|
||||||
which has methods such as ``cc.compiles()``,
|
|
||||||
``cc.check_header()``, ``cc.has_function()``.
|
|
||||||
|
|
||||||
``compile_prog $CFLAGS $LDFLAGS``
|
- invoke Meson in the virtual environment, to perform the actual
|
||||||
Attempt to compile a test program with the system C compiler using
|
configuration step for the emulator build
|
||||||
$CFLAGS and link it with the system linker using $LDFLAGS. The test
|
|
||||||
program must have been previously written to a file called $TMPC.
|
The configure script automatically recognizes command line options for
|
||||||
The replacement in Meson is ``cc.find_library()`` and ``cc.links()``.
|
which a same-named Meson option exists; dashes in the command line are
|
||||||
|
replaced with underscores.
|
||||||
|
|
||||||
|
Almost all QEMU developers that need to modify the build system will
|
||||||
|
only be concerned with Meson, and therefore can skip the rest of this
|
||||||
|
section.
|
||||||
|
|
||||||
|
|
||||||
|
Modifying ``configure``
|
||||||
|
-----------------------
|
||||||
|
|
||||||
|
``configure`` is a shell script; it uses ``#!/bin/sh`` and therefore
|
||||||
|
should be compatible with any POSIX shell. It is important to avoid
|
||||||
|
using bash-isms to avoid breaking development platforms where bash is
|
||||||
|
the primary host.
|
||||||
|
|
||||||
|
The configure script provides a variety of functions to help writing
|
||||||
|
portable shell code and providing consistent behavior across architectures
|
||||||
|
and operating systems:
|
||||||
|
|
||||||
|
``error_exit $MESSAGE $MORE...``
|
||||||
|
Print $MESSAGE to stderr, followed by $MORE... and then exit from the
|
||||||
|
configure script with non-zero status.
|
||||||
|
|
||||||
``has $COMMAND``
|
``has $COMMAND``
|
||||||
Determine if $COMMAND exists in the current environment, either as a
|
Determine if $COMMAND exists in the current environment, either as a
|
||||||
shell builtin, or executable binary, returning 0 on success. The
|
shell builtin, or executable binary, returning 0 on success. The
|
||||||
replacement in Meson is ``find_program()``.
|
replacement in Meson is ``find_program()``.
|
||||||
|
|
||||||
|
``probe_target_compiler $TARGET``
|
||||||
|
Detect a cross compiler and cross tools for the QEMU target $TARGET (e.g.,
|
||||||
|
``$CPU-softmmu``, ``$CPU-linux-user``, ``$CPU-bsd-user``). If a working
|
||||||
|
compiler is present, return success and set variables ``$target_cc``,
|
||||||
|
``$target_ar``, etc. to non-empty values.
|
||||||
|
|
||||||
|
``write_target_makefile``
|
||||||
|
Write a Makefile fragment to stdout, exposing the result of the most
|
||||||
|
``probe_target_compiler`` call as the usual Make variables (``CC``,
|
||||||
|
``AR``, ``LD``, etc.).
|
||||||
|
|
||||||
|
|
||||||
|
Configure does not generally perform tests for compiler options beyond
|
||||||
|
basic checks to detect the host platform and ensure the compiler is
|
||||||
|
functioning. These are performed using a few more helper functions:
|
||||||
|
|
||||||
|
``compile_object $CFLAGS``
|
||||||
|
Attempt to compile a test program with the system C compiler using
|
||||||
|
$CFLAGS. The test program must have been previously written to a file
|
||||||
|
called $TMPC.
|
||||||
|
|
||||||
|
``compile_prog $CFLAGS $LDFLAGS``
|
||||||
|
Attempt to compile a test program with the system C compiler using
|
||||||
|
$CFLAGS and link it with the system linker using $LDFLAGS. The test
|
||||||
|
program must have been previously written to a file called $TMPC.
|
||||||
|
|
||||||
``check_define $NAME``
|
``check_define $NAME``
|
||||||
Determine if the macro $NAME is defined by the system C compiler
|
Determine if the macro $NAME is defined by the system C compiler.
|
||||||
|
|
||||||
|
``do_compiler $CC $ARGS...``
|
||||||
|
Attempt to run the C compiler $CC, passing it $ARGS... This function
|
||||||
|
does not use flags passed via options such as ``--extra-cflags``, and
|
||||||
|
therefore can be used to check for cross compilers. However, most
|
||||||
|
such checks are done at ``make`` time instead (see for example the
|
||||||
|
``cc-option`` macro in ``pc-bios/option-rom/Makefile``).
|
||||||
|
|
||||||
``write_c_skeleton``
|
``write_c_skeleton``
|
||||||
Write a minimal C program main() function to the temporary file
|
Write a minimal C program main() function to the temporary file
|
||||||
indicated by $TMPC
|
indicated by $TMPC.
|
||||||
|
|
||||||
``error_exit $MESSAGE $MORE...``
|
|
||||||
Print $MESSAGE to stderr, followed by $MORE... and then exit from the
|
|
||||||
configure script with non-zero status
|
|
||||||
|
|
||||||
|
Python virtual environments and the QEMU build system
|
||||||
|
-----------------------------------------------------
|
||||||
|
|
||||||
|
TBD
|
||||||
|
|
||||||
Stage 2: Meson
|
Stage 2: Meson
|
||||||
==============
|
==============
|
||||||
|
|
||||||
The Meson build system is currently used to describe the build
|
The Meson build system describes the build and install process for:
|
||||||
process for:
|
|
||||||
|
|
||||||
1) executables, which include:
|
1) executables, which include:
|
||||||
|
|
||||||
- Tools - ``qemu-img``, ``qemu-nbd``, ``qga`` (guest agent), etc
|
- Tools - ``qemu-img``, ``qemu-nbd``, ``qemu-ga`` (guest agent), etc
|
||||||
|
|
||||||
- System emulators - ``qemu-system-$ARCH``
|
- System emulators - ``qemu-system-$ARCH``
|
||||||
|
|
||||||
|
@ -118,7 +144,8 @@ process for:
|
||||||
|
|
||||||
2) documentation
|
2) documentation
|
||||||
|
|
||||||
3) ROMs, which can be either installed as binary blobs or compiled
|
3) ROMs, whether provided as binary blobs in the QEMU distributions
|
||||||
|
or cross compiled under the direction of the configure script
|
||||||
|
|
||||||
4) other data files, such as icons or desktop files
|
4) other data files, such as icons or desktop files
|
||||||
|
|
||||||
|
@ -273,8 +300,7 @@ system/userspace emulation target
|
||||||
Adding checks
|
Adding checks
|
||||||
-------------
|
-------------
|
||||||
|
|
||||||
New checks should be added to Meson. Compiler checks can be as simple as
|
Compiler checks can be as simple as the following::
|
||||||
the following::
|
|
||||||
|
|
||||||
config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h'))
|
config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h'))
|
||||||
|
|
||||||
|
@ -351,22 +377,30 @@ script, which may point to something other than the first python3
|
||||||
binary on the path.
|
binary on the path.
|
||||||
|
|
||||||
|
|
||||||
Stage 3: makefiles
|
Stage 3: Make
|
||||||
==================
|
=============
|
||||||
|
|
||||||
The use of GNU make is required with the QEMU build system.
|
The next step in building QEMU is to invoke make. GNU Make is required
|
||||||
|
to build QEMU, and may be installed as ``gmake`` on some hosts.
|
||||||
|
|
||||||
The output of Meson is a build.ninja file, which is used with the Ninja
|
The output of Meson is a ``build.ninja`` file, which is used with the
|
||||||
build system. QEMU uses a different approach, where Makefile rules are
|
Ninja build tool. However, QEMU's build comprises other components than
|
||||||
synthesized from the build.ninja file. The main Makefile includes these
|
just the emulators (namely firmware and the tests in ``tests/tcg``) which
|
||||||
rules and wraps them so that e.g. submodules are built before QEMU.
|
need different cross compilers. The QEMU Makefile wraps both Ninja and
|
||||||
The resulting build system is largely non-recursive in nature, in
|
the smaller build systems for firmware and tests; it also takes care of
|
||||||
contrast to common practices seen with automake.
|
running ``configure`` again when the script changes. Apart from invoking
|
||||||
|
these sub-Makefiles, the resulting build is largely non-recursive.
|
||||||
|
|
||||||
Tests are also ran by the Makefile with the traditional ``make check``
|
Tests, whether defined in ``meson.build`` or not, are also ran by the
|
||||||
phony target, while benchmarks are run with ``make bench``. Meson test
|
Makefile with the traditional ``make check`` phony target, while benchmarks
|
||||||
suites such as ``unit`` can be ran with ``make check-unit`` too. It is also
|
are run with ``make bench``. Meson test suites such as ``unit`` can be ran
|
||||||
possible to run tests defined in meson.build with ``meson test``.
|
with ``make check-unit``, and ``make check-tcg`` builds and runs "non-Meson"
|
||||||
|
tests for all targets.
|
||||||
|
|
||||||
|
If desired, it is also possible to use ``ninja`` and ``meson test``,
|
||||||
|
respectively to build emulators and run tests defined in meson.build.
|
||||||
|
The main difference is that ``make`` needs the ``-jN`` flag in order to
|
||||||
|
enable parallel builds or tests.
|
||||||
|
|
||||||
Useful make targets
|
Useful make targets
|
||||||
-------------------
|
-------------------
|
||||||
|
@ -378,6 +412,7 @@ Useful make targets
|
||||||
Print the value of the variable VAR. Useful for debugging the build
|
Print the value of the variable VAR. Useful for debugging the build
|
||||||
system.
|
system.
|
||||||
|
|
||||||
|
|
||||||
Important files for the build system
|
Important files for the build system
|
||||||
====================================
|
====================================
|
||||||
|
|
||||||
|
@ -391,8 +426,7 @@ number of dynamically created files listed later.
|
||||||
``Makefile``
|
``Makefile``
|
||||||
The main entry point used when invoking make to build all the components
|
The main entry point used when invoking make to build all the components
|
||||||
of QEMU. The default 'all' target will naturally result in the build of
|
of QEMU. The default 'all' target will naturally result in the build of
|
||||||
every component. Makefile takes care of recursively building submodules
|
every component.
|
||||||
directly via a non-recursive set of rules.
|
|
||||||
|
|
||||||
``*/meson.build``
|
``*/meson.build``
|
||||||
The meson.build file in the root directory is the main entry point for the
|
The meson.build file in the root directory is the main entry point for the
|
||||||
|
@ -401,59 +435,92 @@ number of dynamically created files listed later.
|
||||||
other meson.build files spread throughout the QEMU source tree.
|
other meson.build files spread throughout the QEMU source tree.
|
||||||
|
|
||||||
``tests/Makefile.include``
|
``tests/Makefile.include``
|
||||||
Rules for external test harnesses. These include the TCG tests,
|
Rules for external test harnesses. These include the TCG tests
|
||||||
``qemu-iotests`` and the Avocado-based integration tests.
|
and the Avocado-based integration tests.
|
||||||
|
|
||||||
``tests/docker/Makefile.include``
|
``tests/docker/Makefile.include``
|
||||||
Rules for Docker tests. Like tests/Makefile, this file is included
|
Rules for Docker tests. Like ``tests/Makefile.include``, this file is
|
||||||
directly by the top level Makefile, anything defined in this file will
|
included directly by the top level Makefile, anything defined in this
|
||||||
influence the entire build system.
|
file will influence the entire build system.
|
||||||
|
|
||||||
``tests/vm/Makefile.include``
|
``tests/vm/Makefile.include``
|
||||||
Rules for VM-based tests. Like tests/Makefile, this file is included
|
Rules for VM-based tests. Like ``tests/Makefile.include``, this file is
|
||||||
directly by the top level Makefile, anything defined in this file will
|
included directly by the top level Makefile, anything defined in this
|
||||||
influence the entire build system.
|
file will influence the entire build system.
|
||||||
|
|
||||||
Dynamically created files
|
Dynamically created files
|
||||||
-------------------------
|
-------------------------
|
||||||
|
|
||||||
The following files are generated dynamically by configure in order to
|
The following files are generated at run-time in order to control the
|
||||||
control the behaviour of the statically defined makefiles. This avoids
|
behaviour of the Makefiles. This avoids the need for QEMU makefiles to
|
||||||
the need for QEMU makefiles to go through any pre-processing as seen
|
go through any pre-processing as seen with autotools, where configure
|
||||||
with autotools, where Makefile.am generates Makefile.in which generates
|
generates ``Makefile`` from ``Makefile.in``.
|
||||||
Makefile.
|
|
||||||
|
|
||||||
Built by configure:
|
Built by configure:
|
||||||
|
|
||||||
``config-host.mak``
|
``config-host.mak``
|
||||||
When configure has determined the characteristics of the build host it
|
When configure has determined the characteristics of the build host it
|
||||||
will write a long list of variables to config-host.mak file. This
|
will write them to this file for use in ``Makefile`` and to a smaller
|
||||||
provides the various install directories, compiler / linker flags and a
|
extent ``meson.build``. These include the paths to various tools and a
|
||||||
variety of ``CONFIG_*`` variables related to optionally enabled features.
|
variety of ``CONFIG_*`` variables related to optionally enabled features.
|
||||||
This is imported by the top level Makefile and meson.build in order to
|
|
||||||
tailor the build output.
|
|
||||||
|
|
||||||
config-host.mak is also used as a dependency checking mechanism. If make
|
``config-host.mak`` is also used as a dependency checking mechanism. If make
|
||||||
sees that the modification timestamp on configure is newer than that on
|
sees that the modification timestamp on configure is newer than that on
|
||||||
config-host.mak, then configure will be re-run.
|
``config-host.mak``, then configure will be re-run.
|
||||||
|
|
||||||
The variables defined here are those which are applicable to all QEMU
|
The variables defined here apply to all QEMU
|
||||||
build outputs. Variables which are potentially different for each
|
build outputs.
|
||||||
emulator target are defined by the next file...
|
|
||||||
|
|
||||||
|
``config-meson.cross``
|
||||||
|
|
||||||
|
A Meson "cross file" (or native file) used to communicate the paths to
|
||||||
|
the toolchain and other configuration options.
|
||||||
|
|
||||||
|
``config.status``
|
||||||
|
|
||||||
|
A small shell script that will invoke configure again with the same
|
||||||
|
environment variables that were set during the first run. It's used to
|
||||||
|
rerun configure after changes to the source code, but it can also be
|
||||||
|
inspected manually to check the contents of the environment.
|
||||||
|
|
||||||
|
``Makefile.prereqs``
|
||||||
|
|
||||||
|
A set of Makefile dependencies that order the build and execution of
|
||||||
|
firmware and tests after the container images and emulators that they
|
||||||
|
need.
|
||||||
|
|
||||||
|
``pc-bios/*/config.mak``, ``tests/tcg/config-host.mak``, ``tests/tcg/*/config-target.mak``
|
||||||
|
|
||||||
|
Configuration variables used to build the firmware and TCG tests,
|
||||||
|
including paths to cross compilation toolchains.
|
||||||
|
|
||||||
|
``pyvenv``
|
||||||
|
|
||||||
|
A Python virtual environment that is used for all Python code running
|
||||||
|
during the build. Using a virtual environment ensures that even code
|
||||||
|
that is run via ``sphinx-build``, ``meson`` etc. uses the same interpreter
|
||||||
|
and packages.
|
||||||
|
|
||||||
Built by Meson:
|
Built by Meson:
|
||||||
|
|
||||||
``${TARGET-NAME}-config-devices.mak``
|
``config-host.h``
|
||||||
TARGET-NAME is again the name of a system or userspace emulator. The
|
Used by C code to determine the properties of the build environment
|
||||||
config-devices.mak file is automatically generated by make using the
|
and the set of enabled features for the entire build.
|
||||||
scripts/make_device_config.sh program, feeding it the
|
|
||||||
default-configs/$TARGET-NAME file as input.
|
|
||||||
|
|
||||||
``config-host.h``, ``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h``
|
``${TARGET-NAME}-config-devices.mak``
|
||||||
These files are used by source code to determine what features are
|
TARGET-NAME is the name of a system emulator. The file is
|
||||||
enabled. They are generated from the contents of the corresponding
|
generated by Meson using files under ``configs/devices`` as input.
|
||||||
``*.mak`` files using Meson's ``configure_file()`` function.
|
|
||||||
|
``${TARGET-NAME}-config-target.mak``
|
||||||
|
TARGET-NAME is the name of a system or usermode emulator. The file is
|
||||||
|
generated by Meson using files under ``configs/targets`` as input.
|
||||||
|
|
||||||
|
``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h``
|
||||||
|
Used by C code to determine the properties and enabled
|
||||||
|
features for each target. enabled. They are generated from
|
||||||
|
the contents of the corresponding ``*.mak`` files using Meson's
|
||||||
|
``configure_file()`` function; each target can include them using
|
||||||
|
the ``CONFIG_TARGET`` and ``CONFIG_DEVICES`` macro respectively.
|
||||||
|
|
||||||
``build.ninja``
|
``build.ninja``
|
||||||
The build rules.
|
The build rules.
|
||||||
|
|
Loading…
Reference in New Issue