Преглед на файлове

tests/avocado: mark boot_linux.py long runtime instead of flaky

The ppc64 and s390x tests were first marked skipIf GITLAB_CI by commit
c0c8687ef0f ("tests/avocado: disable BootLinuxPPC64 test in CI"), and
commit 0f26d94ec9e ("tests/acceptance: skip s390x_ccw_vrtio_tcg on
GitLab") due to being very heavy-weight for gitlab CI.

Commit 9b45cc99318 ("docs/devel: rationalise unstable gitlab tests under
FLAKY_TESTS") changed this to being flaky but it isn't really, it just
had a long runtime.

So take the SPEED=slow variable from qtests and introduce it to avocado,
and make these tests require it.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Nicholas Piggin преди 1 година
родител
ревизия
e5f1500144
променени са 2 файла, в които са добавени 13 реда и са изтрити 6 реда
  1. 11 0
      docs/devel/testing.rst
  2. 2 6
      tests/avocado/boot_linux.py

+ 11 - 0
docs/devel/testing.rst

@@ -1346,6 +1346,17 @@ the environment.
 The definition of *large* is a bit arbitrary here, but it usually means an
 The definition of *large* is a bit arbitrary here, but it usually means an
 asset which occupies at least 1GB of size on disk when uncompressed.
 asset which occupies at least 1GB of size on disk when uncompressed.
 
 
+SPEED
+^^^^^
+Tests which have a long runtime will not be run unless ``SPEED=slow`` is
+exported on the environment.
+
+The definition of *long* is a bit arbitrary here, and it depends on the
+usefulness of the test too. A unique test is worth spending more time on,
+small variations on existing tests perhaps less so. As a rough guide,
+a test or set of similar tests which take more than 100 seconds to
+complete.
+
 AVOCADO_ALLOW_UNTRUSTED_CODE
 AVOCADO_ALLOW_UNTRUSTED_CODE
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 There are tests which will boot a kernel image or firmware that can be
 There are tests which will boot a kernel image or firmware that can be

+ 2 - 6
tests/avocado/boot_linux.py

@@ -93,13 +93,11 @@ class BootLinuxPPC64(LinuxTest):
 
 
     timeout = 360
     timeout = 360
 
 
-    @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
-
+    @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
     def test_pseries_tcg(self):
     def test_pseries_tcg(self):
         """
         """
         :avocado: tags=machine:pseries
         :avocado: tags=machine:pseries
         :avocado: tags=accel:tcg
         :avocado: tags=accel:tcg
-        :avocado: tags=flaky
         """
         """
         self.require_accelerator("tcg")
         self.require_accelerator("tcg")
         self.vm.add_args("-accel", "tcg")
         self.vm.add_args("-accel", "tcg")
@@ -113,13 +111,11 @@ class BootLinuxS390X(LinuxTest):
 
 
     timeout = 240
     timeout = 240
 
 
-    @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
-
+    @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
     def test_s390_ccw_virtio_tcg(self):
     def test_s390_ccw_virtio_tcg(self):
         """
         """
         :avocado: tags=machine:s390-ccw-virtio
         :avocado: tags=machine:s390-ccw-virtio
         :avocado: tags=accel:tcg
         :avocado: tags=accel:tcg
-        :avocado: tags=flaky
         """
         """
         self.require_accelerator("tcg")
         self.require_accelerator("tcg")
         self.vm.add_args("-accel", "tcg")
         self.vm.add_args("-accel", "tcg")