From 0defa86ca372efd4a60259c970181a66d3b24304 Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:44 +0000 Subject: [PATCH 01/12] docs/cpu-hotplug.rst: Fix rST markup issues MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit sphinx-build complains: docs/cpu-hotplug.rst:67: ERROR: Unexpected indentation. docs/cpu-hotplug.rst:69: ERROR: Unexpected indentation. docs/cpu-hotplug.rst:74: WARNING: Block quote ends without a blank line; unexpected unindent. docs/cpu-hotplug.rst:75: WARNING: Block quote ends without a blank line; unexpected unindent. docs/cpu-hotplug.rst:76: SEVERE: Unexpected section title. } { docs/cpu-hotplug.rst:78: WARNING: Block quote ends without a blank line; unexpected unindent. These are the result of not indicating one of the literal blocks by finishing the preceding paragraph with the "::" marker. Signed-off-by: Peter Maydell Reviewed-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Acked-by: Aleksandar Markovic Reviewed-by: Richard Henderson Reviewed-by: Cleber Rosa Message-id: 20190305172139.32662-2-peter.maydell@linaro.org Message-id: 20190228145624.24885-2-peter.maydell@linaro.org --- docs/cpu-hotplug.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cpu-hotplug.rst b/docs/cpu-hotplug.rst index cfeb79f571..d0b06403f1 100644 --- a/docs/cpu-hotplug.rst +++ b/docs/cpu-hotplug.rst @@ -60,7 +60,7 @@ vCPU hotplug hot-plugged (no "qom-path" member). From its output in step (3), we can see that ``IvyBridge-IBRS-x86_64-cpu`` is present in socket 0, while hot-plugging a CPU into socket 1 requires passing the listed - properties to QMP ``device_add``: + properties to QMP ``device_add``:: (QEMU) device_add id=cpu-2 driver=IvyBridge-IBRS-x86_64-cpu socket-id=1 core-id=0 thread-id=0 { From 859cdc01a0f9d914fa74892270d40516398d089a Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:44 +0000 Subject: [PATCH 02/12] docs: Convert memory.txt to rst format MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Convert the memory API documentation from plain text to restructured text format. This is a very minimal conversion: all I had to change was to mark up the ASCII art parts as Sphinx expects for 'literal blocks', and fix up the bulleted lists (Sphinx expects no leading space before the bullet, and wants a blank line before after any list). Signed-off-by: Peter Maydell Reviewed-by: Alex Bennée Acked-by: Aleksandar Markovic Reviewed-by: Richard Henderson Reviewed-by: Cleber Rosa Message-id: 20190305172139.32662-3-peter.maydell@linaro.org Message-id: 20190228145624.24885-3-peter.maydell@linaro.org --- docs/devel/{memory.txt => memory.rst} | 128 ++++++++++++++------------ 1 file changed, 70 insertions(+), 58 deletions(-) rename docs/devel/{memory.txt => memory.rst} (85%) diff --git a/docs/devel/memory.txt b/docs/devel/memory.rst similarity index 85% rename from docs/devel/memory.txt rename to docs/devel/memory.rst index 42577e1d86..b6a4c37ea5 100644 --- a/docs/devel/memory.txt +++ b/docs/devel/memory.rst @@ -1,19 +1,20 @@ +============== The memory API ============== The memory API models the memory and I/O buses and controllers of a QEMU machine. It attempts to allow modelling of: - - ordinary RAM - - memory-mapped I/O (MMIO) - - memory controllers that can dynamically reroute physical memory regions - to different destinations +- ordinary RAM +- memory-mapped I/O (MMIO) +- memory controllers that can dynamically reroute physical memory regions + to different destinations The memory model provides support for - - tracking RAM changes by the guest - - setting up coalesced memory for kvm - - setting up ioeventfd regions for kvm +- tracking RAM changes by the guest +- setting up coalesced memory for kvm +- setting up ioeventfd regions for kvm Memory is modelled as an acyclic graph of MemoryRegion objects. Sinks (leaves) are RAM and MMIO regions, while other nodes represent @@ -98,25 +99,30 @@ ROM device memory region types), this host memory needs to be copied to the destination on migration. These APIs which allocate the host memory for you will also register the memory so it is migrated: - - memory_region_init_ram() - - memory_region_init_rom() - - memory_region_init_rom_device() + +- memory_region_init_ram() +- memory_region_init_rom() +- memory_region_init_rom_device() For most devices and boards this is the correct thing. If you have a special case where you need to manage the migration of the backing memory yourself, you can call the functions: - - memory_region_init_ram_nomigrate() - - memory_region_init_rom_nomigrate() - - memory_region_init_rom_device_nomigrate() + +- memory_region_init_ram_nomigrate() +- memory_region_init_rom_nomigrate() +- memory_region_init_rom_device_nomigrate() + which only initialize the MemoryRegion and leave handling migration to the caller. The functions: - - memory_region_init_resizeable_ram() - - memory_region_init_ram_from_file() - - memory_region_init_ram_from_fd() - - memory_region_init_ram_ptr() - - memory_region_init_ram_device_ptr() + +- memory_region_init_resizeable_ram() +- memory_region_init_ram_from_file() +- memory_region_init_ram_from_fd() +- memory_region_init_ram_ptr() +- memory_region_init_ram_device_ptr() + are for special cases only, and so they do not automatically register the backing memory for migration; the caller must manage migration if necessary. @@ -218,7 +224,7 @@ For example, suppose we have a container A of size 0x8000 with two subregions B and C. B is a container mapped at 0x2000, size 0x4000, priority 2; C is an MMIO region mapped at 0x0, size 0x6000, priority 1. B currently has two of its own subregions: D of size 0x1000 at offset 0 and E of size 0x1000 at -offset 0x2000. As a diagram: +offset 0x2000. As a diagram:: 0 1000 2000 3000 4000 5000 6000 7000 8000 |------|------|------|------|------|------|------|------| @@ -228,8 +234,9 @@ offset 0x2000. As a diagram: D: [DDDDD] E: [EEEEE] -The regions that will be seen within this address range then are: - [CCCCCCCCCCCC][DDDDD][CCCCC][EEEEE][CCCCC] +The regions that will be seen within this address range then are:: + + [CCCCCCCCCCCC][DDDDD][CCCCC][EEEEE][CCCCC] Since B has higher priority than C, its subregions appear in the flat map even where they overlap with C. In ranges where B has not mapped anything @@ -237,8 +244,9 @@ C's region appears. If B had provided its own MMIO operations (ie it was not a pure container) then these would be used for any addresses in its range not handled by -D or E, and the result would be: - [CCCCCCCCCCCC][DDDDD][BBBBB][EEEEE][BBBBB] +D or E, and the result would be:: + + [CCCCCCCCCCCC][DDDDD][BBBBB][EEEEE][BBBBB] Priority values are local to a container, because the priorities of two regions are only compared when they are both children of the same container. @@ -257,6 +265,7 @@ guest accesses an address: - all direct subregions of the root region are matched against the address, in descending priority order + - if the address lies outside the region offset/size, the subregion is discarded - if the subregion is a leaf (RAM or MMIO), the search terminates, returning @@ -270,36 +279,39 @@ guest accesses an address: address range), then if this is a container with its own MMIO or RAM backing the search terminates, returning the container itself. Otherwise we continue with the next subregion in priority order + - if none of the subregions match the address then the search terminates with no match found Example memory map ------------------ -system_memory: container@0-2^48-1 - | - +---- lomem: alias@0-0xdfffffff ---> #ram (0-0xdfffffff) - | - +---- himem: alias@0x100000000-0x11fffffff ---> #ram (0xe0000000-0xffffffff) - | - +---- vga-window: alias@0xa0000-0xbffff ---> #pci (0xa0000-0xbffff) - | (prio 1) - | - +---- pci-hole: alias@0xe0000000-0xffffffff ---> #pci (0xe0000000-0xffffffff) +:: -pci (0-2^32-1) - | - +--- vga-area: container@0xa0000-0xbffff - | | - | +--- alias@0x00000-0x7fff ---> #vram (0x010000-0x017fff) - | | - | +--- alias@0x08000-0xffff ---> #vram (0x020000-0x027fff) - | - +---- vram: ram@0xe1000000-0xe1ffffff - | - +---- vga-mmio: mmio@0xe2000000-0xe200ffff + system_memory: container@0-2^48-1 + | + +---- lomem: alias@0-0xdfffffff ---> #ram (0-0xdfffffff) + | + +---- himem: alias@0x100000000-0x11fffffff ---> #ram (0xe0000000-0xffffffff) + | + +---- vga-window: alias@0xa0000-0xbffff ---> #pci (0xa0000-0xbffff) + | (prio 1) + | + +---- pci-hole: alias@0xe0000000-0xffffffff ---> #pci (0xe0000000-0xffffffff) -ram: ram@0x00000000-0xffffffff + pci (0-2^32-1) + | + +--- vga-area: container@0xa0000-0xbffff + | | + | +--- alias@0x00000-0x7fff ---> #vram (0x010000-0x017fff) + | | + | +--- alias@0x08000-0xffff ---> #vram (0x020000-0x027fff) + | + +---- vram: ram@0xe1000000-0xe1ffffff + | + +---- vga-mmio: mmio@0xe2000000-0xe200ffff + + ram: ram@0x00000000-0xffffffff This is a (simplified) PC memory map. The 4GB RAM block is mapped into the system address space via two aliases: "lomem" is a 1:1 mapping of the first @@ -336,16 +348,16 @@ rather than completing successfully; those devices can use the In addition various constraints can be supplied to control how these callbacks are called: - - .valid.min_access_size, .valid.max_access_size define the access sizes - (in bytes) which the device accepts; accesses outside this range will - have device and bus specific behaviour (ignored, or machine check) - - .valid.unaligned specifies that the *device being modelled* supports - unaligned accesses; if false, unaligned accesses will invoke the - appropriate bus or CPU specific behaviour. - - .impl.min_access_size, .impl.max_access_size define the access sizes - (in bytes) supported by the *implementation*; other access sizes will be - emulated using the ones available. For example a 4-byte write will be - emulated using four 1-byte writes, if .impl.max_access_size = 1. - - .impl.unaligned specifies that the *implementation* supports unaligned - accesses; if false, unaligned accesses will be emulated by two aligned - accesses. +- .valid.min_access_size, .valid.max_access_size define the access sizes + (in bytes) which the device accepts; accesses outside this range will + have device and bus specific behaviour (ignored, or machine check) +- .valid.unaligned specifies that the *device being modelled* supports + unaligned accesses; if false, unaligned accesses will invoke the + appropriate bus or CPU specific behaviour. +- .impl.min_access_size, .impl.max_access_size define the access sizes + (in bytes) supported by the *implementation*; other access sizes will be + emulated using the ones available. For example a 4-byte write will be + emulated using four 1-byte writes, if .impl.max_access_size = 1. +- .impl.unaligned specifies that the *implementation* supports unaligned + accesses; if false, unaligned accesses will be emulated by two aligned + accesses. From 5329da6a4f65b394a3131e00af142ad5294345c1 Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:45 +0000 Subject: [PATCH 03/12] docs: Commit initial files from sphinx-quickstart Commit the initial Sphinx conf.py and skeleton index.rst as generated with sphinx-quickstart. We'll update these to add QEMU-specific tweaks in subsequent commits. Signed-off-by: Peter Maydell Acked-by: Aleksandar Markovic Acked-by: Richard Henderson Message-id: 20190305172139.32662-4-peter.maydell@linaro.org Message-id: 20190228145624.24885-4-peter.maydell@linaro.org --- docs/conf.py | 168 +++++++++++++++++++++++++++++++++++++++++++++++++ docs/index.rst | 20 ++++++ 2 files changed, 188 insertions(+) create mode 100644 docs/conf.py create mode 100644 docs/index.rst diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 0000000000..53a1750661 --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,168 @@ +# -*- coding: utf-8 -*- +# +# QEMU documentation build configuration file, created by +# sphinx-quickstart on Thu Jan 31 16:40:14 2019. +# +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) + + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +# +# needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# +# source_suffix = ['.rst', '.md'] +source_suffix = '.rst' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = u'QEMU' +copyright = u'2019, The QEMU Project Developers' +author = u'The QEMU Project Developers' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = u'4.0' +# The full version, including alpha/beta/rc tags. +release = u'4.0' + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# +# This is also used if you do content translation via gettext catalogs. +# Usually you set "language" from the command line for these cases. +language = None + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This patterns also effect to html_static_path and html_extra_path +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = False + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'alabaster' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +# +# html_theme_options = {} + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# Custom sidebar templates, must be a dictionary that maps document names +# to template names. +# +# This is required for the alabaster theme +# refs: http://alabaster.readthedocs.io/en/latest/installation.html#sidebars +html_sidebars = { + '**': [ + 'relations.html', # needs 'show_related': True theme option to display + 'searchbox.html', + ] +} + + +# -- Options for HTMLHelp output ------------------------------------------ + +# Output file base name for HTML help builder. +htmlhelp_basename = 'QEMUdoc' + + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + # + # 'papersize': 'letterpaper', + + # The font size ('10pt', '11pt' or '12pt'). + # + # 'pointsize': '10pt', + + # Additional stuff for the LaTeX preamble. + # + # 'preamble': '', + + # Latex figure (float) alignment + # + # 'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, 'QEMU.tex', u'QEMU Documentation', + u'The QEMU Project Developers', 'manual'), +] + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (master_doc, 'qemu', u'QEMU Documentation', + [author], 1) +] + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + (master_doc, 'QEMU', u'QEMU Documentation', + author, 'QEMU', 'One line description of project.', + 'Miscellaneous'), +] + + + diff --git a/docs/index.rst b/docs/index.rst new file mode 100644 index 0000000000..93f8222831 --- /dev/null +++ b/docs/index.rst @@ -0,0 +1,20 @@ +.. QEMU documentation master file, created by + sphinx-quickstart on Thu Jan 31 16:40:14 2019. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +Welcome to QEMU's documentation! +================================ + +.. toctree:: + :maxdepth: 2 + :caption: Contents: + + + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` From 07fd6563ae435cc78ec64d245a59d13373c2c686 Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:45 +0000 Subject: [PATCH 04/12] docs/conf.py: Disable unused _static directory MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We don't yet have any custom static files, so disable this config file setting to avoid a warning from sphinx about not being able to find the directory. Signed-off-by: Peter Maydell Reviewed-by: Alex Bennée Acked-by: Aleksandar Markovic Reviewed-by: Richard Henderson Message-id: 20190305172139.32662-5-peter.maydell@linaro.org Message-id: 20190228145624.24885-5-peter.maydell@linaro.org --- docs/conf.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/conf.py b/docs/conf.py index 53a1750661..e1d08a34a6 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -93,7 +93,11 @@ html_theme = 'alabaster' # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ['_static'] +# QEMU doesn't yet have any static files, so comment this out so we don't +# get a warning about a missing directory. +# If we do ever add this then it would probably be better to call the +# subdirectory sphinx_static, as the Linux kernel does. +# html_static_path = ['_static'] # Custom sidebar templates, must be a dictionary that maps document names # to template names. From 4fad3864a88170fd16f59b73b0d6a3b3c29692dc Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:45 +0000 Subject: [PATCH 05/12] docs/conf.py: Configure the 'alabaster' theme MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add the 'navigation' bar to the sidebar, which for some reason is not enabled by default. Remove 'relations', which is effectively disabled anyway and isn't useful for us. This requires that we mandate having at least Sphinx 1.3, where the theme was added. Signed-off-by: Peter Maydell Reviewed-by: Alex Bennée Acked-by: Aleksandar Markovic Message-id: 20190305172139.32662-6-peter.maydell@linaro.org Message-id: 20190228145624.24885-6-peter.maydell@linaro.org --- docs/conf.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index e1d08a34a6..d118757cbd 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -25,7 +25,8 @@ # If your documentation needs a minimal Sphinx version, state it here. # -# needs_sphinx = '1.0' +# 1.3 is where the 'alabaster' theme was shipped with Sphinx. +needs_sphinx = '1.3' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom @@ -106,7 +107,8 @@ html_theme = 'alabaster' # refs: http://alabaster.readthedocs.io/en/latest/installation.html#sidebars html_sidebars = { '**': [ - 'relations.html', # needs 'show_related': True theme option to display + 'about.html', + 'navigation.html', 'searchbox.html', ] } From 479fb8a5114cc87fdf96bb21e4cf0b5798286444 Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:46 +0000 Subject: [PATCH 06/12] docs/conf.py: Don't include rST sources in HTML build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sphinx defaults to including all the rST source files in the HTML build and making each HTML page link to the source file. Disable that. Signed-off-by: Peter Maydell Reviewed-by: Alex Bennée Acked-by: Aleksandar Markovic Reviewed-by: Richard Henderson Message-id: 20190305172139.32662-7-peter.maydell@linaro.org Message-id: 20190228145624.24885-7-peter.maydell@linaro.org --- docs/conf.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/conf.py b/docs/conf.py index d118757cbd..d3018bc539 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -113,6 +113,9 @@ html_sidebars = { ] } +# Don't copy the rST source files to the HTML output directory, +# and don't put links to the sources into the output HTML. +html_copy_source = False # -- Options for HTMLHelp output ------------------------------------------ From e250e867dc0172203b30aa25032efe29f1102b21 Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:46 +0000 Subject: [PATCH 07/12] docs/conf.py: Disable option warnings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit sphinx-build complains about using :option: to mark up option flags that it doesn't know about (because they were not defined using the "option::" directive): docs/pr-manager.rst:68: WARNING: unknown option: -d Suppress these warnings. This way we get the semantic markup of the option flag but no cross-referencing hyperlink. Signed-off-by: Peter Maydell Reviewed-by: Alex Bennée Acked-by: Aleksandar Markovic Reviewed-by: Richard Henderson Message-id: 20190305172139.32662-8-peter.maydell@linaro.org Message-id: 20190228145624.24885-8-peter.maydell@linaro.org --- docs/conf.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/conf.py b/docs/conf.py index d3018bc539..56a74e0fcb 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -77,6 +77,9 @@ pygments_style = 'sphinx' # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = False +# Sphinx defaults to warning about use of :option: for options not defined +# with "option::" in the document being processed. Turn that off. +suppress_warnings = ["ref.option"] # -- Options for HTML output ---------------------------------------------- From f8cf7147f10a3cee64396614218b08e3c9161d4a Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:46 +0000 Subject: [PATCH 08/12] docs: Provide separate conf.py for each manual we want MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit By default Sphinx wants to build a single manual at once. For QEMU, this doesn't suit us, because we want to have separate manuals for "Developer's Guide", "User Manual", and so on, and we don't want to ship the Developer's Guide to end-users. However, we don't want to completely duplicate conf.py for each manual, and we'd like to continue to support "build all docs in one run" for third-party sites like readthedocs.org. Make the top-level conf.py support two usage forms: (1) as a common config file which is included by the conf.py for each of QEMU's manuals: in this case sphinx-build is run multiple times, once per subdirectory. (2) as a top level conf file which will result in building all the manuals into a single document: in this case sphinx-build is run once, on the top-level docs directory. Provide per-manual conf.py files and top level pages for our first two manuals: * QEMU Developer's Guide (docs/devel) * QEMU System Emulation Management and Interoperability Guide (docs/interop) Reviewed-by: Alex Bennée Signed-off-by: Peter Maydell Acked-by: Aleksandar Markovic Message-id: 20190305172139.32662-9-peter.maydell@linaro.org Message-id: 20190228145624.24885-9-peter.maydell@linaro.org --- docs/conf.py | 37 +++++++++++++++++++++++++++++++------ docs/devel/conf.py | 15 +++++++++++++++ docs/devel/index.rst | 21 +++++++++++++++++++++ docs/index.rst | 9 ++------- docs/interop/conf.py | 15 +++++++++++++++ docs/interop/index.rst | 18 ++++++++++++++++++ 6 files changed, 102 insertions(+), 13 deletions(-) create mode 100644 docs/devel/conf.py create mode 100644 docs/devel/index.rst create mode 100644 docs/interop/conf.py create mode 100644 docs/interop/index.rst diff --git a/docs/conf.py b/docs/conf.py index 56a74e0fcb..f452e424cf 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -3,6 +3,20 @@ # QEMU documentation build configuration file, created by # sphinx-quickstart on Thu Jan 31 16:40:14 2019. # +# This config file can be used in one of two ways: +# (1) as a common config file which is included by the conf.py +# for each of QEMU's manuals: in this case sphinx-build is run multiple +# times, once per subdirectory. +# (2) as a top level conf file which will result in building all +# the manuals into a single document: in this case sphinx-build is +# run once, on the top-level docs directory. +# +# QEMU's makefiles take option (1), which allows us to install +# only the ones the user cares about (in particular we don't want +# to ship the 'devel' manual to end-users). +# Third-party sites such as readthedocs.org will take option (2). +# +# # This file is execfile()d with the current directory set to its # containing dir. # @@ -12,13 +26,22 @@ # All configuration values have a default; values that are commented out # serve to show the default. +import os +import sys + +# The per-manual conf.py will set qemu_docdir for a single-manual build; +# otherwise set it here if this is an entire-manual-set build. +# This is always the absolute path of the docs/ directory in the source tree. +try: + qemu_docdir +except NameError: + qemu_docdir = os.path.abspath(".") + # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the -# documentation root, use os.path.abspath to make it absolute, like shown here. +# documentation root, use an absolute path starting from qemu_docdir. # -# import os -# import sys -# sys.path.insert(0, os.path.abspath('.')) +# sys.path.insert(0, os.path.join(qemu_docdir, "my_subdir")) # -- General configuration ------------------------------------------------ @@ -91,8 +114,10 @@ html_theme = 'alabaster' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. -# -# html_theme_options = {} +# We initialize this to empty here, so the per-manual conf.py can just +# add individual key/value entries. +html_theme_options = { +} # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, diff --git a/docs/devel/conf.py b/docs/devel/conf.py new file mode 100644 index 0000000000..7441f87e7f --- /dev/null +++ b/docs/devel/conf.py @@ -0,0 +1,15 @@ +# -*- coding: utf-8 -*- +# +# QEMU documentation build configuration file for the 'devel' manual. +# +# This includes the top level conf file and then makes any necessary tweaks. +import sys +import os + +qemu_docdir = os.path.abspath("..") +parent_config = os.path.join(qemu_docdir, "conf.py") +exec(compile(open(parent_config, "rb").read(), parent_config, 'exec')) + +# This slightly misuses the 'description', but is the best way to get +# the manual title to appear in the sidebar. +html_theme_options['description'] = u'Developer''s Guide' diff --git a/docs/devel/index.rst b/docs/devel/index.rst new file mode 100644 index 0000000000..cd0fa6c9ba --- /dev/null +++ b/docs/devel/index.rst @@ -0,0 +1,21 @@ +.. This is the top level page for the 'devel' manual. + + +QEMU Developer's Guide +====================== + +This manual documents various parts of the internals of QEMU. +You only need to read it if you are interested in reading or +modifying QEMU's source code. + +Contents: + +.. toctree:: + :maxdepth: 2 + + loads-stores + memory + migration + stable-process + testing + diff --git a/docs/index.rst b/docs/index.rst index 93f8222831..3690955dd1 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -10,11 +10,6 @@ Welcome to QEMU's documentation! :maxdepth: 2 :caption: Contents: + interop/index + devel/index - -Indices and tables -================== - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` diff --git a/docs/interop/conf.py b/docs/interop/conf.py new file mode 100644 index 0000000000..cf3c69d4a7 --- /dev/null +++ b/docs/interop/conf.py @@ -0,0 +1,15 @@ +# -*- coding: utf-8 -*- +# +# QEMU documentation build configuration file for the 'interop' manual. +# +# This includes the top level conf file and then makes any necessary tweaks. +import sys +import os + +qemu_docdir = os.path.abspath("..") +parent_config = os.path.join(qemu_docdir, "conf.py") +exec(compile(open(parent_config, "rb").read(), parent_config, 'exec')) + +# This slightly misuses the 'description', but is the best way to get +# the manual title to appear in the sidebar. +html_theme_options['description'] = u'System Emulation Management and Interoperability Guide' diff --git a/docs/interop/index.rst b/docs/interop/index.rst new file mode 100644 index 0000000000..2df977dd52 --- /dev/null +++ b/docs/interop/index.rst @@ -0,0 +1,18 @@ +.. This is the top level page for the 'interop' manual. + + +QEMU System Emulation Management and Interoperability Guide +=========================================================== + +This manual contains documents and specifications that are useful +for making QEMU interoperate with other software. + +Contents: + +.. toctree:: + :maxdepth: 2 + + bitmaps + live-block-operations + pr-helper + From 5f71eac06e15b9a3fa1134d446f0ca2d52e45f0a Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:46 +0000 Subject: [PATCH 09/12] Makefile, configure: Support building rST documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add support to our configure and makefile machinery for building our rST docs into HTML files. Building the documentation now requires that sphinx-build is available; this seems better than allowing half the docs to be built if it is not present but having half of them missing. (In particular it means that assuming that distros configured with --enable-docs they'll get a helpful error from configure telling them the new build dependency.) Signed-off-by: Peter Maydell Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Acked-by: Aleksandar Markovic Reviewed-by: Richard Henderson Message-id: 20190305172139.32662-10-peter.maydell@linaro.org Message-id: 20190228145624.24885-10-peter.maydell@linaro.org --- .gitignore | 1 + Makefile | 45 ++++++++++++++++++++++++++++++++++++++++++--- configure | 15 +++++++++++++-- 3 files changed, 56 insertions(+), 5 deletions(-) diff --git a/.gitignore b/.gitignore index b66b772551..77522561b8 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,4 @@ +/.doctrees /config-devices.* /config-all-devices.* /config-all-disas.* diff --git a/Makefile b/Makefile index 2208bde419..add22cf294 100644 --- a/Makefile +++ b/Makefile @@ -388,7 +388,7 @@ dummy := $(call unnest-vars,, \ include $(SRC_PATH)/tests/Makefile.include -all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all modules +all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) $(HELPERS-y) recurse-all modules qemu-version.h: FORCE $(call quiet-command, \ @@ -637,6 +637,14 @@ dist: qemu-$(VERSION).tar.bz2 qemu-%.tar.bz2: $(SRC_PATH)/scripts/make-release "$(SRC_PATH)" "$(patsubst qemu-%.tar.bz2,%,$@)" +# Note that these commands assume that there are no HTML files in +# the docs subdir in the source tree! If there are then this will +# blow them away for an in-source-tree 'make clean'. +define clean-manual = +rm -rf docs/$1/_static +rm -f docs/$1/objects.inv docs/$1/searchindex.js docs/$1/*.html +endef + distclean: clean rm -f config-host.mak config-host.h* config-host.ld $(DOCS) qemu-options.texi qemu-img-cmds.texi qemu-monitor.texi qemu-monitor-info.texi rm -f config-all-devices.mak config-all-disas.mak config.status @@ -657,6 +665,9 @@ distclean: clean rm -f docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html rm -f docs/qemu-block-drivers.7 rm -f docs/qemu-cpu-models.7 + rm -f .doctrees + $(call clean-manual,devel) + $(call clean-manual,interop) for d in $(TARGET_DIRS); do \ rm -rf $$d || exit 1 ; \ done @@ -690,7 +701,18 @@ else BLOBS= endif -install-doc: $(DOCS) +define install-manual = +for d in $$(cd docs && find $1 -type d); do $(INSTALL_DIR) "$(DESTDIR)$(qemu_docdir)/$$d"; done +for f in $$(cd docs && find $1 -type f); do $(INSTALL_DATA) "docs/$$f" "$(DESTDIR)$(qemu_docdir)/$$f"; done +endef + +# Note that we deliberately do not install the "devel" manual: it is +# for QEMU developers, and not interesting to our users. +.PHONY: install-sphinxdocs +install-sphinxdocs: sphinxdocs + $(call install-manual,interop) + +install-doc: $(DOCS) install-sphinxdocs $(INSTALL_DIR) "$(DESTDIR)$(qemu_docdir)" $(INSTALL_DATA) qemu-doc.html "$(DESTDIR)$(qemu_docdir)" $(INSTALL_DATA) qemu-doc.txt "$(DESTDIR)$(qemu_docdir)" @@ -841,6 +863,23 @@ docs/version.texi: $(SRC_PATH)/VERSION %.pdf: %.texi docs/version.texi $(call quiet-command,texi2pdf $(TEXI2PDFFLAGS) $< -o $@,"GEN","$@") +# Sphinx builds all its documentation at once in one invocation +# and handles "don't rebuild things unless necessary" itself. +# The '.doctrees' files are cached information to speed this up. +.PHONY: sphinxdocs +sphinxdocs: docs/devel/index.html docs/interop/index.html + +# Canned command to build a single manual +build-manual = $(call quiet-command,sphinx-build $(if $(V),,-q) -b html -d .doctrees/$1 $(SRC_PATH)/docs/$1 docs/$1 ,"SPHINX","docs/$1") +# We assume all RST files in the manual's directory are used in it +manual-deps = $(wildcard $(SRC_PATH)/docs/$1/*.rst) $(SRC_PATH)/docs/$1/conf.py $(SRC_PATH)/docs/conf.py + +docs/devel/index.html: $(call manual-deps,devel) + $(call build-manual,devel) + +docs/interop/index.html: $(call manual-deps,interop) + $(call build-manual,interop) + qemu-options.texi: $(SRC_PATH)/qemu-options.hx $(SRC_PATH)/scripts/hxtool $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"GEN","$@") @@ -869,7 +908,7 @@ docs/qemu-block-drivers.7: docs/qemu-block-drivers.texi docs/qemu-cpu-models.7: docs/qemu-cpu-models.texi scripts/qemu-trace-stap.1: scripts/qemu-trace-stap.texi -html: qemu-doc.html docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html +html: qemu-doc.html docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html sphinxdocs info: qemu-doc.info docs/interop/qemu-qmp-ref.info docs/interop/qemu-ga-ref.info pdf: qemu-doc.pdf docs/interop/qemu-qmp-ref.pdf docs/interop/qemu-ga-ref.pdf txt: qemu-doc.txt docs/interop/qemu-qmp-ref.txt docs/interop/qemu-ga-ref.txt diff --git a/configure b/configure index cefeb8fcce..47bf617fcc 100755 --- a/configure +++ b/configure @@ -4589,13 +4589,24 @@ if compile_prog "" "" ; then syncfs=yes fi +# Check we have a new enough version of sphinx-build +has_sphinx_build() { + # This is a bit awkward but works: create a trivial document and + # try to run it with our configuration file (which enforces a + # version requirement). This will fail if either + # sphinx-build doesn't exist at all or if it is too old. + mkdir -p "$TMPDIR1/sphinx" + touch "$TMPDIR1/sphinx/index.rst" + sphinx-build -c "$source_path/docs" -b html "$TMPDIR1/sphinx" "$TMPDIR1/sphinx/out" >/dev/null 2>&1 +} + # Check if tools are available to build documentation. if test "$docs" != "no" ; then - if has makeinfo && has pod2man; then + if has makeinfo && has pod2man && has_sphinx_build; then docs=yes else if test "$docs" = "yes" ; then - feature_not_found "docs" "Install texinfo and Perl/perl-podlators" + feature_not_found "docs" "Install texinfo, Perl/perl-podlators and python-sphinx" fi docs=no fi From 57b49737ae467fd95504de7b2d755a994e7cec4b Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:47 +0000 Subject: [PATCH 10/12] Makefile: Abstract out "identify the pkgversion" code MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Abstract out the "identify the pkgversion" code from the rule for creating qemu-version.h, so it sets makefile variables for QEMU_PKGVERSION and QEMU_FULL_VERSION. (We will want to use these when building the Sphinx docs.) NB: As we abstract this out, we use -e to check for .git rather than -d, since in some situations .git may be a file rather than a directory. Signed-off-by: Peter Maydell Reviewed-by: Alex Bennée Acked-by: Aleksandar Markovic Reviewed-by: Richard Henderson Message-id: 20190305172139.32662-11-peter.maydell@linaro.org Message-id: 20190228145624.24885-11-peter.maydell@linaro.org --- Makefile | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/Makefile b/Makefile index add22cf294..b746d112a7 100644 --- a/Makefile +++ b/Makefile @@ -87,6 +87,20 @@ endif include $(SRC_PATH)/rules.mak +# Create QEMU_PKGVERSION and FULL_VERSION strings +# If PKGVERSION is set, use that; otherwise get version and -dirty status from git +QEMU_PKGVERSION := $(if $(PKGVERSION),$(PKGVERSION),$(shell \ + cd $(SRC_PATH); \ + if test -e .git; then \ + git describe --match 'v*' 2>/dev/null | tr -d '\n'; \ + if ! git diff-index --quiet HEAD &>/dev/null; then \ + echo "-dirty"; \ + fi; \ + fi)) + +# Either "version (pkgversion)", or just "version" if pkgversion not set +FULL_VERSION := $(if $(QEMU_PKGVERSION),$(VERSION) ($(QEMU_PKGVERSION)),$(VERSION)) + GENERATED_FILES = qemu-version.h config-host.h qemu-options.def GENERATED_QAPI_FILES = qapi/qapi-builtin-types.h qapi/qapi-builtin-types.c @@ -392,23 +406,8 @@ all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) $(HELPERS-y) recurse-all mo qemu-version.h: FORCE $(call quiet-command, \ - (cd $(SRC_PATH); \ - if test -n "$(PKGVERSION)"; then \ - pkgvers="$(PKGVERSION)"; \ - else \ - if test -d .git; then \ - pkgvers=$$(git describe --match 'v*' 2>/dev/null | tr -d '\n');\ - if ! git diff-index --quiet HEAD &>/dev/null; then \ - pkgvers="$${pkgvers}-dirty"; \ - fi; \ - fi; \ - fi; \ - printf "#define QEMU_PKGVERSION \"$${pkgvers}\"\n"; \ - if test -n "$${pkgvers}"; then \ - printf '#define QEMU_FULL_VERSION QEMU_VERSION " (" QEMU_PKGVERSION ")"\n'; \ - else \ - printf '#define QEMU_FULL_VERSION QEMU_VERSION\n'; \ - fi; \ + (printf '#define QEMU_PKGVERSION "$(QEMU_PKGVERSION)"\n'; \ + printf '#define QEMU_FULL_VERSION "$(FULL_VERSION)"\n'; \ ) > $@.tmp) $(call quiet-command, if ! cmp -s $@ $@.tmp; then \ mv $@.tmp $@; \ From 6038f5fca588d8f44c403a8b8742163d9ab4adba Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:47 +0000 Subject: [PATCH 11/12] docs/conf.py: Don't hard-code QEMU version MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Don't hard-code the QEMU version number into conf.py. Instead we either pass it to sphinx-build on the command line, or (if doing a standalone Sphinx run in a readthedocs.org setup) extract it from the VERSION file. Signed-off-by: Peter Maydell Reviewed-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Acked-by: Aleksandar Markovic Reviewed-by: Richard Henderson Message-id: 20190305172139.32662-12-peter.maydell@linaro.org Message-id: 20190228145624.24885-12-peter.maydell@linaro.org --- Makefile | 2 +- docs/conf.py | 21 ++++++++++++++++----- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index b746d112a7..cad585b4d6 100644 --- a/Makefile +++ b/Makefile @@ -869,7 +869,7 @@ docs/version.texi: $(SRC_PATH)/VERSION sphinxdocs: docs/devel/index.html docs/interop/index.html # Canned command to build a single manual -build-manual = $(call quiet-command,sphinx-build $(if $(V),,-q) -b html -d .doctrees/$1 $(SRC_PATH)/docs/$1 docs/$1 ,"SPHINX","docs/$1") +build-manual = $(call quiet-command,sphinx-build $(if $(V),,-q) -b html -D version=$(VERSION) -D release="$(FULL_VERSION)" -d .doctrees/$1 $(SRC_PATH)/docs/$1 docs/$1 ,"SPHINX","docs/$1") # We assume all RST files in the manual's directory are used in it manual-deps = $(wildcard $(SRC_PATH)/docs/$1/*.rst) $(SRC_PATH)/docs/$1/conf.py $(SRC_PATH)/docs/conf.py diff --git a/docs/conf.py b/docs/conf.py index f452e424cf..befbcc6c3e 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -76,11 +76,22 @@ author = u'The QEMU Project Developers' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. -# -# The short X.Y version. -version = u'4.0' -# The full version, including alpha/beta/rc tags. -release = u'4.0' + +# Extract this information from the VERSION file, for the benefit of +# standalone Sphinx runs as used by readthedocs.org. Builds run from +# the Makefile will pass version and release on the sphinx-build +# command line, which override this. +try: + extracted_version = None + with open(os.path.join(qemu_docdir, '../VERSION')) as f: + extracted_version = f.readline().strip() +except: + pass +finally: + if extracted_version: + version = release = extracted_version + else: + version = release = "unknown version" # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. From c10e01b996df09f6cb4eceb2b7a9754bece927c9 Mon Sep 17 00:00:00 2001 From: Peter Maydell Date: Thu, 7 Mar 2019 14:26:47 +0000 Subject: [PATCH 12/12] MAINTAINERS: Add entry for Sphinx documentation infrastructure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a MAINTAINERS entry for Sphinx documentation infrastructure: this doesn't cover actual content, only the machinery we use to build the docs. Signed-off-by: Peter Maydell Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Richard Henderson Message-id: 20190305172139.32662-13-peter.maydell@linaro.org --- MAINTAINERS | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 5040d9dfb1..074ad46d47 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2567,3 +2567,9 @@ GIT submodules M: Daniel P. Berrange S: Odd Fixes F: scripts/git-submodule.sh + +Sphinx documentation configuration and build machinery +M: Peter Maydell +S: Maintained +F: docs/conf.py +F: docs/*/conf.py