mirror of https://github.com/xemu-project/xemu.git
tests/functional: Convert the acpi-bits test into a standalone test
Mostly a straight-forward conversion. Looks like we can simply drop the avocado datadrainer stuff when not using the avocado framework anymore. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-ID: <20240830133841.142644-30-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
This commit is contained in:
Thomas Huth
parent
576fffbc8e
commit
05caa06242
|
|
@ -2074,8 +2074,8 @@ ACPI/AVOCADO/BIOSBITS
|
|||
M: Ani Sinha <anisinha@redhat.com>
|
||||
M: Michael S. Tsirkin <mst@redhat.com>
|
||||
S: Supported
|
||||
F: tests/avocado/acpi-bits/*
|
||||
F: tests/avocado/acpi-bits.py
|
||||
F: tests/functional/acpi-bits/*
|
||||
F: tests/functional/test_acpi_bits.py
|
||||
F: docs/devel/acpi-bits.rst
|
||||
|
||||
ACPI/HEST/GHES
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
=============================================================================
|
||||
ACPI/SMBIOS avocado tests using biosbits
|
||||
=============================================================================
|
||||
==================================
|
||||
ACPI/SMBIOS testing using biosbits
|
||||
==================================
|
||||
************
|
||||
Introduction
|
||||
************
|
||||
|
|
@ -35,7 +35,7 @@ for developing biosbits and its real life uses can be found in [#a]_ and [#b]_.
|
|||
For QEMU, we maintain a fork of bios bits in gitlab along with all the
|
||||
dependent submodules `here <https://gitlab.com/qemu-project/biosbits-bits>`__.
|
||||
This fork contains numerous fixes, a newer acpica and changes specific to
|
||||
running this avocado QEMU tests using bits. The author of this document
|
||||
running these functional QEMU tests using bits. The author of this document
|
||||
is the sole maintainer of the QEMU fork of bios bits repository. For more
|
||||
information, please see author's `FOSDEM talk on this bios-bits based test
|
||||
framework <https://fosdem.org/2024/schedule/event/fosdem-2024-2262-exercising-qemu-generated-acpi-smbios-tables-using-biosbits-from-within-a-guest-vm-/>`__.
|
||||
|
|
@ -44,12 +44,12 @@ framework <https://fosdem.org/2024/schedule/event/fosdem-2024-2262-exercising-qe
|
|||
Description of the test framework
|
||||
*********************************
|
||||
|
||||
Under the directory ``tests/avocado/``, ``acpi-bits.py`` is a QEMU avocado
|
||||
test that drives all this.
|
||||
Under the directory ``tests/functional/``, ``test_acpi_bits.py`` is a QEMU
|
||||
functional test that drives all this.
|
||||
|
||||
A brief description of the various test files follows.
|
||||
|
||||
Under ``tests/avocado/`` as the root we have:
|
||||
Under ``tests/functional/`` as the root we have:
|
||||
|
||||
::
|
||||
|
||||
|
|
@ -60,12 +60,12 @@ Under ``tests/avocado/`` as the root we have:
|
|||
│ ├── smbios.py2
|
||||
│ ├── testacpi.py2
|
||||
│ └── testcpuid.py2
|
||||
├── acpi-bits.py
|
||||
├── test_acpi_bits.py
|
||||
|
||||
* ``tests/avocado``:
|
||||
* ``tests/functional``:
|
||||
|
||||
``acpi-bits.py``:
|
||||
This is the main python avocado test script that generates a
|
||||
``test_acpi_bits.py``:
|
||||
This is the main python functional test script that generates a
|
||||
biosbits iso. It then spawns a QEMU VM with it, collects the log and reports
|
||||
test failures. This is the script one would be interested in if they wanted
|
||||
to add or change some component of the log parsing, add a new command line
|
||||
|
|
@ -79,35 +79,22 @@ Under ``tests/avocado/`` as the root we have:
|
|||
you to inspect and run the specific commands manually.
|
||||
|
||||
In order to run this test, please perform the following steps from the QEMU
|
||||
build directory:
|
||||
build directory (assuming that the sources are in ".."):
|
||||
::
|
||||
|
||||
$ make check-venv (needed only the first time to create the venv)
|
||||
$ ./pyvenv/bin/avocado run -t acpi tests/avocado
|
||||
$ export PYTHONPATH=../python:../tests/functional
|
||||
$ export QEMU_TEST_QEMU_BINARY=$PWD/qemu-system-x86_64
|
||||
$ python3 ../tests/functional/test_acpi_bits.py
|
||||
|
||||
The above will run all acpi avocado tests including this one.
|
||||
In order to run the individual tests, perform the following:
|
||||
::
|
||||
The above will run all acpi-bits functional tests (producing output in
|
||||
tap format).
|
||||
|
||||
$ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py --tap -
|
||||
You can inspect the log files in tests/functional/x86_64/test_acpi_bits.*/
|
||||
for more information about the run or in order to diagnoze issues.
|
||||
If you pass V=1 in the environment, more diagnostic logs will be put into
|
||||
the test log.
|
||||
|
||||
The above will produce output in tap format. You can omit "--tap -" in the
|
||||
end and it will produce output like the following:
|
||||
::
|
||||
|
||||
$ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py
|
||||
Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
|
||||
JOB ID : eab225724da7b64c012c65705dc2fa14ab1defef
|
||||
JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log
|
||||
(1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (33.09 s)
|
||||
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
|
||||
JOB TIME : 39.22 s
|
||||
|
||||
You can inspect the log file for more information about the run or in order
|
||||
to diagnoze issues. If you pass V=1 in the environment, more diagnostic logs
|
||||
would be found in the test log.
|
||||
|
||||
* ``tests/avocado/acpi-bits/bits-config``:
|
||||
* ``tests/functional/acpi-bits/bits-config``:
|
||||
|
||||
This location contains biosbits configuration files that determine how the
|
||||
software runs the tests.
|
||||
|
|
@ -117,7 +104,7 @@ Under ``tests/avocado/`` as the root we have:
|
|||
or actions are performed by bits. The description of the config options are
|
||||
provided in the file itself.
|
||||
|
||||
* ``tests/avocado/acpi-bits/bits-tests``:
|
||||
* ``tests/functional/acpi-bits/bits-tests``:
|
||||
|
||||
This directory contains biosbits python based tests that are run from within
|
||||
the biosbits environment in the spawned VM. New additions of test cases can
|
||||
|
|
@ -155,7 +142,8 @@ Under ``tests/avocado/`` as the root we have:
|
|||
(a) They are python2.7 based scripts and not python 3 scripts.
|
||||
(b) They are run from within the bios bits VM and is not subjected to QEMU
|
||||
build/test python script maintenance and dependency resolutions.
|
||||
(c) They need not be loaded by avocado framework when running tests.
|
||||
(c) They need not be loaded by the test framework by accident when running
|
||||
tests.
|
||||
|
||||
|
||||
Author: Ani Sinha <anisinha@redhat.com>
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ endif
|
|||
|
||||
# Timeouts for individual tests that can be slow e.g. with debugging enabled
|
||||
test_timeouts = {
|
||||
'acpi_bits' : 240,
|
||||
'netdev_ethtool' : 180,
|
||||
'ppc_40p' : 240,
|
||||
'ppc64_hv' : 1000,
|
||||
|
|
@ -96,6 +97,7 @@ tests_x86_64_system_quick = [
|
|||
]
|
||||
|
||||
tests_x86_64_system_thorough = [
|
||||
'acpi_bits',
|
||||
'netdev_ethtool',
|
||||
'virtio_gpu',
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
# group: rw quick
|
||||
#
|
||||
# Exercise QEMU generated ACPI/SMBIOS tables using biosbits,
|
||||
# https://biosbits.org/
|
||||
#
|
||||
|
|
@ -24,7 +24,7 @@
|
|||
# pylint: disable=consider-using-f-string
|
||||
|
||||
"""
|
||||
This is QEMU ACPI/SMBIOS avocado tests using biosbits.
|
||||
This is QEMU ACPI/SMBIOS functional tests using biosbits.
|
||||
Biosbits is available originally at https://biosbits.org/.
|
||||
This test uses a fork of the upstream bits and has numerous fixes
|
||||
including an upgraded acpica. The fork is located here:
|
||||
|
|
@ -41,15 +41,16 @@ import tarfile
|
|||
import tempfile
|
||||
import time
|
||||
import zipfile
|
||||
|
||||
from pathlib import Path
|
||||
from typing import (
|
||||
List,
|
||||
Optional,
|
||||
Sequence,
|
||||
)
|
||||
from qemu.machine import QEMUMachine
|
||||
from avocado import skipIf
|
||||
from avocado.utils import datadrainer as drainer
|
||||
from avocado_qemu import QemuBaseTest
|
||||
from unittest import skipIf
|
||||
from qemu_test import QemuBaseTest, Asset
|
||||
|
||||
deps = ["xorriso", "mformat"] # dependent tools needed in the test setup/box.
|
||||
supported_platforms = ['x86_64'] # supported test platforms.
|
||||
|
|
@ -129,34 +130,32 @@ class QEMUBitsMachine(QEMUMachine): # pylint: disable=too-few-public-methods
|
|||
class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
||||
"""
|
||||
ACPI and SMBIOS tests using biosbits.
|
||||
|
||||
:avocado: tags=arch:x86_64
|
||||
:avocado: tags=acpi
|
||||
|
||||
"""
|
||||
# in slower systems the test can take as long as 3 minutes to complete.
|
||||
timeout = BITS_TIMEOUT
|
||||
|
||||
# following are some standard configuration constants
|
||||
# gitlab CI does shallow clones of depth 20
|
||||
BITS_INTERNAL_VER = 2020
|
||||
# commit hash must match the artifact tag below
|
||||
BITS_COMMIT_HASH = 'c7920d2b'
|
||||
# this is the latest bits release as of today.
|
||||
BITS_TAG = "qemu-bits-10262023"
|
||||
|
||||
ASSET_BITS = Asset(("https://gitlab.com/qemu-project/"
|
||||
"biosbits-bits/-/jobs/artifacts/%s/"
|
||||
"download?job=qemu-bits-build" % BITS_TAG),
|
||||
'1b8dd612c6831a6b491716a77acc486666aaa867051cdc34f7ce169c2e25f487')
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self._vm = None
|
||||
self._workDir = None
|
||||
self._baseDir = None
|
||||
|
||||
# following are some standard configuration constants
|
||||
self._bitsInternalVer = 2020 # gitlab CI does shallow clones of depth 20
|
||||
self._bitsCommitHash = 'c7920d2b' # commit hash must match
|
||||
# the artifact tag below
|
||||
self._bitsTag = "qemu-bits-10262023" # this is the latest bits
|
||||
# release as of today.
|
||||
self._bitsArtSHA1Hash = 'b22cdfcfc7453875297d06d626f5474ee36a343f'
|
||||
self._bitsArtURL = ("https://gitlab.com/qemu-project/"
|
||||
"biosbits-bits/-/jobs/artifacts/%s/"
|
||||
"download?job=qemu-bits-build" %self._bitsTag)
|
||||
self._debugcon_addr = '0x403'
|
||||
self._debugcon_log = 'debugcon-log.txt'
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
self.logger = logging.getLogger('acpi-bits')
|
||||
self.logger = self.log
|
||||
|
||||
def _print_log(self, log):
|
||||
self.logger.info('\nlogs from biosbits follows:')
|
||||
|
|
@ -171,7 +170,7 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
|||
bits_config_dir = os.path.join(self._baseDir, 'acpi-bits',
|
||||
'bits-config')
|
||||
target_config_dir = os.path.join(self._workDir,
|
||||
'bits-%d' %self._bitsInternalVer,
|
||||
'bits-%d' %self.BITS_INTERNAL_VER,
|
||||
'boot')
|
||||
self.assertTrue(os.path.exists(bits_config_dir))
|
||||
self.assertTrue(os.path.exists(target_config_dir))
|
||||
|
|
@ -188,7 +187,7 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
|||
bits_test_dir = os.path.join(self._baseDir, 'acpi-bits',
|
||||
'bits-tests')
|
||||
target_test_dir = os.path.join(self._workDir,
|
||||
'bits-%d' %self._bitsInternalVer,
|
||||
'bits-%d' %self.BITS_INTERNAL_VER,
|
||||
'boot', 'python')
|
||||
|
||||
self.assertTrue(os.path.exists(bits_test_dir))
|
||||
|
|
@ -248,9 +247,9 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
|||
test scripts
|
||||
"""
|
||||
bits_dir = os.path.join(self._workDir,
|
||||
'bits-%d' %self._bitsInternalVer)
|
||||
'bits-%d' %self.BITS_INTERNAL_VER)
|
||||
iso_file = os.path.join(self._workDir,
|
||||
'bits-%d.iso' %self._bitsInternalVer)
|
||||
'bits-%d.iso' %self.BITS_INTERNAL_VER)
|
||||
mkrescue_script = os.path.join(self._workDir,
|
||||
'grub-inst-x86_64-efi', 'bin',
|
||||
'grub-mkrescue')
|
||||
|
|
@ -264,8 +263,12 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
|||
|
||||
try:
|
||||
if os.getenv('V') or os.getenv('BITS_DEBUG'):
|
||||
subprocess.check_call([mkrescue_script, '-o', iso_file,
|
||||
bits_dir], stderr=subprocess.STDOUT)
|
||||
proc = subprocess.run([mkrescue_script, '-o', iso_file,
|
||||
bits_dir],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
check=True)
|
||||
self.logger.info("grub-mkrescue output %s" % proc.stdout)
|
||||
else:
|
||||
subprocess.check_call([mkrescue_script, '-o',
|
||||
iso_file, bits_dir],
|
||||
|
|
@ -282,8 +285,9 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
|||
|
||||
def setUp(self): # pylint: disable=arguments-differ
|
||||
super().setUp('qemu-system-')
|
||||
self.logger = self.log
|
||||
|
||||
self._baseDir = os.getenv('AVOCADO_TEST_BASEDIR')
|
||||
self._baseDir = Path(__file__).parent
|
||||
|
||||
# workdir could also be avocado's own workdir in self.workdir.
|
||||
# At present, I prefer to maintain my own temporary working
|
||||
|
|
@ -300,15 +304,14 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
|||
os.mkdir(prebuiltDir, mode=0o775)
|
||||
|
||||
bits_zip_file = os.path.join(prebuiltDir, 'bits-%d-%s.zip'
|
||||
%(self._bitsInternalVer,
|
||||
self._bitsCommitHash))
|
||||
%(self.BITS_INTERNAL_VER,
|
||||
self.BITS_COMMIT_HASH))
|
||||
grub_tar_file = os.path.join(prebuiltDir,
|
||||
'bits-%d-%s-grub.tar.gz'
|
||||
%(self._bitsInternalVer,
|
||||
self._bitsCommitHash))
|
||||
%(self.BITS_INTERNAL_VER,
|
||||
self.BITS_COMMIT_HASH))
|
||||
|
||||
bitsLocalArtLoc = self.fetch_asset(self._bitsArtURL,
|
||||
asset_hash=self._bitsArtSHA1Hash)
|
||||
bitsLocalArtLoc = self.ASSET_BITS.fetch()
|
||||
self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLoc)
|
||||
|
||||
# extract the bits artifact in the temp working directory
|
||||
|
|
@ -369,7 +372,7 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
|||
"""The main test case implementation."""
|
||||
|
||||
iso_file = os.path.join(self._workDir,
|
||||
'bits-%d.iso' %self._bitsInternalVer)
|
||||
'bits-%d.iso' %self.BITS_INTERNAL_VER)
|
||||
|
||||
self.assertTrue(os.access(iso_file, os.R_OK))
|
||||
|
||||
|
|
@ -393,12 +396,6 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
|||
self._vm.set_console()
|
||||
self._vm.launch()
|
||||
|
||||
self.logger.debug("Console output from bits VM follows ...")
|
||||
c_drainer = drainer.LineLogger(self._vm.console_socket.fileno(),
|
||||
logger=self.logger.getChild("console"),
|
||||
stop_check=(lambda :
|
||||
not self._vm.is_running()))
|
||||
c_drainer.start()
|
||||
|
||||
# biosbits has been configured to run all the specified test suites
|
||||
# in batch mode and then automatically initiate a vm shutdown.
|
||||
|
|
@ -406,4 +403,8 @@ class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
|
|||
# with the avocado test timeout.
|
||||
self._vm.event_wait('SHUTDOWN', timeout=BITS_TIMEOUT)
|
||||
self._vm.wait(timeout=None)
|
||||
self.logger.debug("Checking console output ...")
|
||||
self.parse_log()
|
||||
|
||||
if __name__ == '__main__':
|
||||
QemuBaseTest.main()
|
||||
Loading…
Reference in New Issue