Commit Graph

57 Commits

Author SHA1 Message Date
Thomas Huth 380f7268b7 tests/functional: Convert the OrangePi tests to the functional framework
Move the OrangePi tests from tests/avocado/boot_linux_console.py into
a new file dedicated for OrangePi tests in the functional framework
and update the hash sums of the assets to sha256 along the way.
For the buildroot image and the Armbian image, we've got to switch to
a newer version since the old images have been removed from the server,
and the NetBSD image has been moved to the archive, so we need to update
this URL as well.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241029092440.25021-3-thuth@redhat.com>
2024-11-04 14:16:12 +01:00
Thomas Huth f7d6b77220 tests/functional: Convert BananaPi tests to the functional framework
Move the BananaPi tests from tests/avocado/boot_linux_console.py into
a new file dedicated for Banana Pi tests in the functional framework.
Update the hash sums of the assets to sha256 along the way and fix the
broken link for the buildroot image from storage.kernelci.org.

(Note: The test_arm_bpim2u_openwrt_22_03_3 test is currently broken
due to a regression in commit 4c2c047469 ("target/arm: Fix usage of MMU
indexes when EL3 is AArch32") - it works if that commit gets reverted)

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241029092440.25021-2-thuth@redhat.com>
2024-11-04 14:16:12 +01:00
Thomas Huth 4f37abff1c tests/functional: Convert the powernv tests from boot_linux_console.py
Move the tests into the already existing test_ppc64_powernv.py
file.

Message-ID: <20240920150319.81723-1-thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-24 13:45:06 +02:00
Thomas Huth a94bfe1b18 tests/functional: Convert the r2d sh4 Avocado test
This is the last test that is using the do_test_advcal_2018()
function, so we can now remove that function from boot_linux_console.py,
too.

Message-ID: <20240919185749.71222-8-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-24 13:40:32 +02:00
Thomas Huth 12c0b40798 tests/functional: Convert the mac ppc Avocado tests
The g3beige and mac99 tests use the same asset, so put them together
in a new test_ppc_mac.py file.

Message-ID: <20240919185749.71222-7-thuth@redhat.com>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-24 13:40:23 +02:00
Thomas Huth 53a62fdeb2 tests/functional: Convert the e500 ppc64 Avocado test
Use the new launch_kernel function to convert this test in a simple way.

Message-ID: <20240919185749.71222-6-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-24 13:39:12 +02:00
Thomas Huth f90527d3d1 tests/functional: Convert the SPARCStation Avocado test
Use the new launch_kernel function to convert this test in a simple way.

Message-ID: <20240919185749.71222-5-thuth@redhat.com>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-24 13:38:59 +02:00
Thomas Huth d1939097c7 tests/functional: Convert the xtensa lx60 Avocado test
Use the new launch_kernel function to convert this test in a simple way.

Message-ID: <20240919185749.71222-4-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-24 13:38:53 +02:00
Thomas Huth c3cff7279a tests/functional: Convert the vexpressa9 Avocado test
Use the new launch_kernel function to convert this test in a simple way.

Message-ID: <20240919185749.71222-3-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-24 13:38:40 +02:00
Thomas Huth 3a07875fd3 tests/functional: Convert the or1k-sim Avocado test
We've got to do_test_advcal_2018() here now that the test resides
in a separate file. Also switch back to the original URL (since
the site did not vanish as originally expected) and update the
hashsum to use SHA256.

Message-ID: <20240906180549.792832-15-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Thomas Huth 9f989d6557 tests/functional: Convert the m68k MCF5208EVB Avocado test
We've got to do_test_advcal_2018() here now that the test resides
in a separate file. Also switch back to the original URL (since
the site did not vanish as originally expected) and update the
hashsum to use SHA256.

Message-ID: <20240906180549.792832-14-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Thomas Huth 9a76bc0433 tests/functional: Convert the Alpha Clipper Avocado test
Straight forward conversion, just switch to SHA256 hashsum now.

Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240906180549.792832-13-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé 547738be48 tests/functional: Convert Aarch64 Raspi4 avocado tests
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826221058.75126-5-philmd@linaro.org>
[thuth: Use LinuxKernelTest class]
Message-ID: <20240906180549.792832-12-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé 471f7ff715 tests/functional: Convert Aarch64 Raspi3 avocado tests
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826221058.75126-4-philmd@linaro.org>
[thuth: Use the LinuxKernelTest class]
Message-ID: <20240906180549.792832-11-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé 165ab27447 tests/functional: Convert ARM Raspi2 avocado tests
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826221058.75126-3-philmd@linaro.org>
[thuth: Use LinuxKernelTest class]
Message-ID: <20240906180549.792832-10-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé 79cb4a14cb tests/functional: Convert mips32eb 4Kc Malta avocado tests
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-8-philmd@linaro.org>
[thuth: Use LinuxKernelTest class]
Message-ID: <20240906180549.792832-9-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé 0e85442408 tests/functional: Convert nanomips Malta avocado tests
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.

  $ QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 \
        make check-functional-mipsel
  ...
  ▶ 4/4 test_mipsel_malta.MaltaMachineConsole.test_mips_malta32el_nanomips_16k_up   OK
  ▶ 4/4 test_mipsel_malta.MaltaMachineConsole.test_mips_malta32el_nanomips_4k       OK
  ▶ 4/4 test_mipsel_malta.MaltaMachineConsole.test_mips_malta32el_nanomips_64k_dbg  OK
  ▶ 4/4 test_mipsel_malta.MaltaMachineYAMON.test_mipsel_malta_yamon                 OK
  4/4 qemu:func-thorough+func-mipsel-thorough+thorough / func-mipsel-mipsel_malta   OK   9.95s   4 subtests passed

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-7-philmd@linaro.org>
[thuth: Use LinuxKernelTest]
Message-ID: <20240906180549.792832-8-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé 0e165dfc52 tests/functional: Convert mips64el 5KEc Malta avocado tests
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-5-philmd@linaro.org>
Message-ID: <20240906180549.792832-6-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé d2fce37597 tests/functional: Convert mips64el Fuloong2e avocado test (2/2)
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-3-philmd@linaro.org>
[thuth: Use LinuxKernelTest class for this test]
Message-ID: <20240906180549.792832-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Thomas Huth c7f3663c60 tests/functional: Convert the m68k Q800 Avocado test into a functional test
Just had to update the asset checksum to use SHA256 instead of SHA1,
but apart from that it is a pretty much straightforward conversion.

Message-ID: <20240906180549.792832-3-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Thomas Huth b5347978a9 tests/avocado/boot_linux_console: Remove the s390x subtest
We've got a much more sophisticated, Fedora-based test for s390x
("test_s390x_fedora" in another file) already, so the test in
boot_linux_console.py seems to be rather a waste of precious test
cycles. Thus move the command line check and delete the s390x
test in boot_linux_console.py.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240830133841.142644-7-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 10:52:29 +02:00
Cleber Rosa 3185e5a6f7 tests/avocado: apply proper skipUnless decorator
Commit 9b45cc993 added many cases of skipUnless for the sake of
organizing flaky tests.  But, Python decorators *must* follow what
they decorate, so the newlines added should *not* exist there.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-ID: <20240806173119.582857-3-crosa@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-08-16 09:16:03 +02:00
Cleber Rosa 4ec88f1640 tests/avocado: test_arm_emcraft_sf2: handle RW requirements for asset
The asset used in the mentioned test gets truncated before it's used
in the test.  This means that the file gets modified, and thus the
asset's expected hash doesn't match anymore.  This causes cache misses
and re-downloads every time the test is re-run.

Let's make a copy of the asset so that the one in the cache is
preserved and the cache sees a hit on re-runs.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240726134438.14720-9-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2024-07-31 00:14:22 +02:00
Cleber Rosa 085ea2b868 tests/avocado: mips: add hint for fetchasset plugin
Avocado's fetchasset plugin runs before the actual Avocado job (and
any test).  It analyses the test's code looking for occurrences of
"self.fetch_asset()" in the either the actual test or setUp() method.
It's not able to fully analyze all code, though.

The way these tests are written, make the fetchasset plugin blind to
the assets.  This adds some more code duplication, true, but it will
aid the fetchasset plugin to download or verify the existence of these
assets in advance.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240726134438.14720-3-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2024-07-31 00:14:22 +02:00
Cleber Rosa b8f5932dda tests/avocado: mips: fallback to HTTP given certificate expiration
The SSL certificate installed at mipsdistros.mips.com has expired:

 0 s:CN = mipsdistros.mips.com
 i:C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
 a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
 v:NotBefore: Dec 23 00:00:00 2019 GMT; NotAfter: Jan 23 12:00:00 2021 GMT

Because this project has no control over that certificate and host,
this falls back to plain HTTP instead.  The integrity of the
downloaded files can be guaranteed by the existing hashes for those
files (which are not modified here).

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240726134438.14720-2-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2024-07-31 00:14:22 +02:00
Peter Maydell dcc5c018c7 tests/avocado: update sunxi kernel from armbian to 6.6.16
The Linux kernel 5.10.16 binary for sunxi has been removed from
apt.armbian.com. This means that the avocado tests for these machines
will be skipped (status CANCEL) if the old binary isn't present in
the avocado cache.

Update to 6.6.16, in the same way we did in commit e384db41d8
when we moved to 5.10.16 in 2021.

Cc: qemu-stable@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2284
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Strahinja Jankovic <strahinja.p.jankovic@gmail.com>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Message-id: 20240415151845.1564201-1-peter.maydell@linaro.org
2024-04-30 15:06:26 +01:00
Philippe Mathieu-Daudé 6c3014858c target/nios2: Remove the deprecated Nios II target
The Nios II target is deprecated since v8.2 in commit 9997771bc1
("target/nios2: Deprecate the Nios II architecture").

Remove:
- Buildsys / CI infra
- User emulation
- System emulation (10m50-ghrd & nios2-generic-nommu machines)
- Tests

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Marek Vasut <marex@denx.de>
Message-Id: <20240327144806.11319-3-philmd@linaro.org>
2024-04-24 16:03:38 +02:00
Peter Maydell 2f79d2ed64 tests/avocado/boot_linux_console.py: Add Rpi4b boot tests
Signed-off-by: Sergey Kambalin <sergey.kambalin@auriga.com>
Message-id: 20240226000259.2752893-31-sergey.kambalin@auriga.com
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
[PMM: Comment out use of USB, which depends on PCI]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-02-27 13:01:43 +00:00
Nicholas Piggin 234aa6d62a tests/avocado: ppc add powernv10 boot_linux_console test
Add test for POWER10.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:34 +10:00
Nicholas Piggin 458a6aa3b3 tests/avocado: improve flaky ppc/pnv boot_linux_console.py test
The expected MTD partition detection output does not always appear on
the console, despite the test reaching the boot loader and the string
appearing in dmesg. Possibly due to an init script that quietens the
console output. Using an earlier log message improves reliability.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:34 +10:00
Alex Bennée 5d25fcb702 gitlab: add optional job to run flaky avocado tests
One problem with flaky tests is they often only fail under CI
conditions which makes it hard to debug. We add an optional allow_fail
job so developers can trigger the only the flaky tests in the CI
environment if they are debugging.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-8-alex.bennee@linaro.org>
2023-12-01 17:47:20 +00:00
Alex Bennée 9b45cc9931 docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS
It doesn't make sense to have two classes of flaky tests. While it may
take the constrained environment of CI to trigger failures easily it
doesn't mean they don't occasionally happen on developer machines. As
CI is the gating factor to passing there is no point developers
running the tests locally anyway unless they are trying to fix things.

While we are at it update the language in the docs to discourage the
QEMU_TEST_FLAKY_TESTS becoming a permanent solution.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-3-alex.bennee@linaro.org>
2023-12-01 14:11:55 +00:00
Philippe Mathieu-Daudé 432f936ae1 tests/avocado: Re-enable MIPS Malta tests (GitLab issue #1884 fixed)
Commit 18a536f1f8 ("accel/tcg: Always require can_do_io") fixed
the GitLab issue #1884: we can now re-enable those tests.

This reverts commit f959c3d87c.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231003063808.66564-1-philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-10-04 11:03:54 -07:00
Philippe Mathieu-Daudé f959c3d87c tests/avocado: Disable MIPS Malta tests due to GitLab issue #1884
Commit 0d58c66068 ("softmmu: Use async_run_on_cpu in tcg_commit")
introduced a regression which is only triggered by the MIPS Malta
machine. Since those tests are gatting and disturb the CI workflow,
disable them until https://gitlab.com/qemu-project/qemu/-/issues/1866
is fixed.

  $ make check-avocado \
      AVOCADO_TAGS='arch:mipsel arch:mips64el' \
      AVOCADO_ALLOW_UNTRUSTED_CODE=1 \
      AVOCADO_TIMEOUT_EXPECTED=1
    AVOCADO tests/avocado
   (04/24) tests/avocado/boot_linux_console.py:BootLinuxConsole.test_mips_malta32el_nanomips_4k: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n... (90.39 s)
   (05/24) tests/avocado/boot_linux_console.py:BootLinuxConsole.test_mips_malta32el_nanomips_16k_up: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n... (90.29 s)
   (06/24) tests/avocado/boot_linux_console.py:BootLinuxConsole.test_mips_malta32el_nanomips_64k_dbg: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n... (92.53 s)
   (11/24) tests/avocado/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n... (25.78 s)
  RESULTS    : PASS 8 | ERROR 0 | FAIL 0 | SKIP 7 | WARN 2 | INTERRUPT 5 | CANCEL 2
  JOB TIME   : 525.60 s                                      ^^^^^^^^^^^

Reported-by: Thomas Huth <thuth@redhat.com>
Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230913135339.9128-1-philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-10-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Thomas Huth 7ea47af390 tests/avocado: Make the test_arm_bpim2u_gmac test more reliable
The test_arm_bpim2u_gmac test sometimes fails (ca. 1 out of 20 runs
here) since the disk shows up as /dev/mmcblk1 instead of /dev/mmcblk0
in some runs. No matter of the name in /dev, the major:minor encoding
seems always to be the same, so we can fix this issue by using the
correct major:minor hex number in the "root=" parameter instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230630161604.446394-1-thuth@redhat.com>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-18-alex.bennee@linaro.org>
2023-07-03 12:51:58 +01:00
qianfan Zhao 6c4f229a2e tests: avocado: boot_linux_console: Add test case for bpim2u
Add test case for booting from initrd and sd card.

Signed-off-by: qianfan Zhao <qianfanguijin@163.com>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2023-06-06 10:19:34 +01:00
Strahinja Jankovic d784c5fbba tests/avocado: Add reboot tests to Cubieboard
Cubieboard tests end with comment "reboot not functioning; omit test".
Fix this so reboot is done at the end of each test.

Signed-off-by: Strahinja Jankovic <strahinja.p.jankovic@gmail.com>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Message-id: 20230326202256.22980-5-strahinja.p.jankovic@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2023-04-20 10:21:14 +01:00
Fabiano Rosas 9bb9a3f3c8 tests/avocado: Tag TCG tests with accel:tcg
This allows the test to be skipped when TCG is not present in the QEMU
binary.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2023-02-16 16:26:22 +00:00
Joel Stanley 9b983dc78b avocado/boot_linux_console.py: Update ast2600 test
Update the test_arm_ast2600_debian test to

 - the latest Debian kernel
 - use the Rainier machine instead of Tacoma

Both of which contains support for more hardware and thus exercises more
of the hardware Qemu models.

Signed-off-by: Joel Stanley <joel@jms.id.au>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220607011938.1676459-1-joel@jms.id.au>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
2023-02-07 09:02:04 +01:00
Philippe Mathieu-Daudé a7f16aed39 tests/avocado: Truncate M2S-FG484 SOM SPI flash to 16MiB
The M2S-FG484 SOM uses a 16 MiB SPI flash (Spansion
S25FL128SDPBHICO).  Since the test asset is bigger,
truncate it to the correct size to avoid when running
the test_arm_emcraft_sf2 test:

  qemu-system-arm: device requires 16777216 bytes, block backend provides 67108864 bytes

Add comment regarding the M2S-FG484 SOM hardware in
hw/arm/msf2-som.c.

Reported-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Tested-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
2023-02-07 09:02:04 +01:00
Philippe Mathieu-Daudé ffd0cac708 tests/avocado: Introduce file_truncate()
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Link: https://lore.kernel.org/r/20230120134314.81956-2-philmd@linaro.org
[ clg: remove image_pow2ceil_expand() factoring ]
Signed-off-by: Cédric Le Goater <clg@kaod.org>
2023-02-07 09:02:04 +01:00
Strahinja Jankovic 22bd244a1c tests/avocado: Add SD boot test to Cubieboard
Cubieboard now can boot directly from SD card, without the need to pass
`-kernel` parameter. Update Avocado tests to cover this functionality.

Signed-off-by: Strahinja Jankovic <strahinja.p.jankovic@gmail.com>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Message-id: 20221226220303.14420-8-strahinja.p.jankovic@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2023-01-12 16:50:19 +00:00
Alex Bennée 4189af72dd tests/avocado: use new rootfs for orangepi test
The old URL wasn't stable. I suspect the current URL will only be
stable for a few months so maybe we need another strategy for hosting
rootfs snapshots?

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221118113309.1057790-1-alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-11-23 10:58:48 +01:00
Thomas Huth 72cf57b074 tests/avocado: Update the URLs of the advent calendar images
The qemu-advent-calendar.org server will be decommissioned soon.
I've mirrored the images that we use for the QEMU CI to gitlab,
so update their URLs to point to the new location.

Message-Id: <20221121102436.78635-1-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-11-23 10:58:48 +01:00
Alex Bennée 500f73b1aa tests/avocado: disable sh4 rd2 tests on Gitlab
Running repeated invocations on a number of test boxes show a fairly
high error rate:

  $ retry.py -n 100 -c -- ./tests/venv/bin/avocado run tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d
  retry.py called with ['./tests/venv/bin/avocado', 'run', 'tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d']
  Results:
  Run, Ret, Pass/Fail, Time, Total Pass, Total Run
  ...
  Results summary:
  0: 94 times (94.00%), avg time 2.254 (0.00 varience/0.04 deviation)
  1: 3 times (3.00%), avg time 1.837 (0.02 varience/0.14 deviation)
  8: 3 times (3.00%), avg time 91.288 (0.02 varience/0.15 deviation)

Examining the logs they fall into various categories of un-handled
unaligned access by user space and unexpected FPU usage by the kernel
which ultimately lead to the failure to reach the login prompt. This
could be bugs in the translator that only get hit occasionally or just
a flaky kernel - its hard to tell. To avoid these failures gating CI
lets skip on GitLab.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Magnus Damm <magnus.damm@gmail.com>
Message-Id: <20221027183637.2772968-21-alex.bennee@linaro.org>
2022-10-31 20:37:59 +00:00
Peter Maydell cc45d25c65 tests/avocado: raspi2_initrd: Wait for guest shutdown message before stopping
The avocado test
 tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_initrd
finishes with

    exec_command(self, 'halt')
    # Wait for VM to shut down gracefully
    self.vm.wait()

In theory this should be fine. In practice it runs into two bugs:

 * when the test calls self.vm.wait() Avocado closes the socket
   connection to the guest serial console immediately, so the
   avocado logs don't have the last part of the guest output:
   https://gitlab.com/qemu-project/qemu/-/issues/1265
 * when the socket is closed, a bug in the QEMU socket chardev
   means that it loses any data that the guest UART has not
   yet consumed. This means that the guest doesn't always read
   the full 'halt' command string, so the test intermittently
   fails with a timeout:
   https://gitlab.com/qemu-project/qemu/-/issues/1264

Work around both of these by waiting for the guest to print the
string that means it has completed the shutdown process.  This fixes
a very long standing intermittent failure in this test.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/636
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20221020102012.3015662-1-peter.maydell@linaro.org>
Message-Id: <20221027183637.2772968-20-alex.bennee@linaro.org>
2022-10-31 20:37:59 +00:00
Peter Maydell 0793fe014c tests/avocado: Add missing require_netdev('user') checks
Some avocado tests fail if QEMU was built without libslirp. Add
require_netdev('user') checks where necessary:

These tests try to ping 10.0.2.2 and expect it to succeed:
  boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2
  boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
  ppc_bamboo.py:BambooMachine.test_ppc_bamboo

These tests run a commandline that includes '-net user':
  machine_aspeed.py:AST2x00Machine.test_arm_ast2500_evb_builroot
  (and others that use the do_test_arm_aspeed_buidroot_start()
  or do_test_arm_aspeed_sdk_start() helper functions)

These changes seem to be sufficient for 'make check-avocado'
to not fail on a --disable-slirp build.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20221001195224.2453581-1-peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-10-11 17:25:50 +02:00
Thomas Huth 7fcc3f7595 tests/avocado/boot_linux_console: Fix the test_aarch64_xlnx_versal_virt test
The assets that this test tries to download have been removed from the
server. Update to a newer version to get it working again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220829080940.110831-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220914155950.804707-3-alex.bennee@linaro.org>
2022-09-20 15:14:59 +01:00
Cédric Le Goater 341e21fa13 test/avocado/machine_aspeed.py: Move OpenBMC tests
It's easier to run. Keep test_arm_ast2600_debian() under the
boot_linux_console.py file because it requires the extract_from_deb()
helper. We could remove it when we have tests for the AST2600.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
2022-06-22 09:49:33 +02:00
Alex Bennée 11593544df tests/avocado: update aarch64_virt test to exercise -cpu max
The Fedora 29 kernel is quite old and importantly fails when running
in LPA2 scenarios. As it's not really exercising much of the CPU space
replace it with a custom 5.16.12 kernel with all the architecture
options turned on. There is a minimal buildroot initramfs included in
the kernel which has a few tools for stress testing the memory
subsystem. The userspace also targets the Neoverse N1 processor so
would fail with a v8.0 cpu like cortex-a53.

While we are at it move the test into its own file so it can have an
assigned maintainer.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220419091020.3008144-2-alex.bennee@linaro.org>
2022-04-20 16:04:16 +01:00