Commit Graph

8 Commits

Author SHA1 Message Date
Philippe Mathieu-Daudé 432f936ae1 tests/avocado: Re-enable MIPS Malta tests (GitLab issue #1884 fixed)
Commit 18a536f1f8 ("accel/tcg: Always require can_do_io") fixed
the GitLab issue #1884: we can now re-enable those tests.

This reverts commit f959c3d87c.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231003063808.66564-1-philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-10-04 11:03:54 -07:00
Philippe Mathieu-Daudé f959c3d87c tests/avocado: Disable MIPS Malta tests due to GitLab issue #1884
Commit 0d58c66068 ("softmmu: Use async_run_on_cpu in tcg_commit")
introduced a regression which is only triggered by the MIPS Malta
machine. Since those tests are gatting and disturb the CI workflow,
disable them until https://gitlab.com/qemu-project/qemu/-/issues/1866
is fixed.

  $ make check-avocado \
      AVOCADO_TAGS='arch:mipsel arch:mips64el' \
      AVOCADO_ALLOW_UNTRUSTED_CODE=1 \
      AVOCADO_TIMEOUT_EXPECTED=1
    AVOCADO tests/avocado
   (04/24) tests/avocado/boot_linux_console.py:BootLinuxConsole.test_mips_malta32el_nanomips_4k: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n... (90.39 s)
   (05/24) tests/avocado/boot_linux_console.py:BootLinuxConsole.test_mips_malta32el_nanomips_16k_up: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n... (90.29 s)
   (06/24) tests/avocado/boot_linux_console.py:BootLinuxConsole.test_mips_malta32el_nanomips_64k_dbg: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n... (92.53 s)
   (11/24) tests/avocado/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n... (25.78 s)
  RESULTS    : PASS 8 | ERROR 0 | FAIL 0 | SKIP 7 | WARN 2 | INTERRUPT 5 | CANCEL 2
  JOB TIME   : 525.60 s                                      ^^^^^^^^^^^

Reported-by: Thomas Huth <thuth@redhat.com>
Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230913135339.9128-1-philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-10-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Thomas Huth 12cae32fe1 tests/avocado/tuxrun_baselines: Fix ppc64 tests for binaries without slirp
The ppc64 tuxrun tests are currently failing if "slirp" has been
disabled in the binary since they are using "-netdev user" now.
We have to skip the test if this network backend is missing.

Fixes: 6ee3624236 ("improve code coverage for ppc64")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230606192802.666000-1-thuth@redhat.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2023-06-10 10:19:24 -03:00
Kautuk Consul 6ee3624236 tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64
Commit c0c8687ef0 disabled the
boot_linux.py test-case due to which the code coverage for ppc
decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/87sfdpqcy4.fsf@linaro.org/ it
was mentioned that the baseline test for ppc64 could be modified
to make up this 2% code coverage. This patch attempts to achieve
this 2% code coverage by adding various device command line
arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py
test-case.

The code coverage report with boot_linux.py, without it and finally
with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
  lines......: 13.8% (58006 of 420997 lines)
  functions..: 20.7% (7675 of 36993 functions)
  branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
  lines......: 11.9% (50174 of 420997 lines)
  functions..: 18.8% (6947 of 36993 functions)
  branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
  lines......: 13.8% (58287 of 420997 lines)
  functions..: 20.7% (7640 of 36993 functions)
  branches...: 8.4% (20223 of 240611 branches)

Rebased on Alex Benee's testing/next branch:
https://gitlab.com/stsquad/qemu/-/tree/testing/next

Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com>
Reported-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424041830.1275636-1-kconsul@linux.vnet.ibm.com>
Message-Id: <20230424092249.58552-13-alex.bennee@linaro.org>
2023-04-27 14:58:23 +01:00
Alex Bennée fa6ecc9bc0 tests/avocado: use the new snapshots for testing
The tuxboot images now have a stable snapshot URL so we can enable the
checksums and remove the avocado warnings. We will have to update as
old snapshots retire but that won't be too frequent.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230424092249.58552-3-alex.bennee@linaro.org>
2023-04-27 14:57:40 +01:00
Philippe Mathieu-Daudé 9d403d27bc tests/avocado: Enable TuxRun/mips64 (big-endian) test
Now that the previous commit ("hw/mips/gt64xxx_pci: Don't
endian-swap GT_PCI0_CFGADDR") fixed the issue accessing
the GT64120 PCI config-address register on big-endian
targets, we can enable this TuxRun test.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230223220404.63630-1-philmd@linaro.org>
2023-03-30 15:03:36 +02:00
Alex Bennée fb3af2d182 tests/avocado: don't use tags to define drive
We are abusing the avocado tags which are intended to provide test
selection metadata to provide parameters to our test. This works OK up
until the point you need to have ,'s in the field as this is the tag
separator character which is the case for a number of the drive
parameters. Fix this by making drive a parameter to the common helper
function.

Fixes: 267fe57c23 (tests: add tuxrun baseline test to avocado)
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230315174331.2959-11-alex.bennee@linaro.org>
2023-03-22 15:06:57 +00:00
Alex Bennée 267fe57c23 tests: add tuxrun baseline test to avocado
The TuxRun project (www.tuxrun.org) uses QEMU to run tests on a wide
variety of kernel configurations on wide range of our emulated
platforms. They publish a known good set of images at:

  https://storage.tuxboot.com/

to help with bisecting regressions in either the kernel, firmware or
QEMU itself. The tests are pretty lightweight as they contain just a
kernel with a minimal rootfs which boots a lot faster than most of the
distros. In time they might be persuaded to version their known good
baselines and we can then enable proper checksums.

For a couple of tests we currently skip:

  - mips64, a regression against previous stable release
  - sh4, very unstable with intermittent oops

Total run time: 340s (default) -> 890s (debug)

Overall coverage rate (tested targets + disabled tests):
  lines......: 16.1% (126894 of 789848 lines)
  functions..: 20.6% (15954 of 77489 functions)
  branches...: 9.3% (40727 of 439365 branches)

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Anders Roxell <anders.roxell@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230228190653.1602033-11-alex.bennee@linaro.org>
2023-03-01 10:31:52 +00:00