This scenario has been failing randomly but in a different way, it usually fails on the first run and passes on the following re-runs but it can fail on the reruns as well, see the following build for instance:

• https://ci.moodle.org/job/S402.04.01%20-%20Behat%20-%20Firefox%20+%20Postgres%20(HEADED)/51/

Even though it doesn't appear on the list of failures, this random has been happening a lot and causing unnecessary Behat re-runs.

This failure doesn't seem to be browser/database relevant and it happens on all versions.

Console output

002 Scenario: An automatic forum lists all subscribers # /var/www/html/mod/forum/tests/behat/forum_subscriptions_availability.feature:92

      Then I should see "Student 1"                    # /var/www/html/mod/forum/tests/behat/forum_subscriptions_availability.feature:95

        "Student 1" text was not found in the page (Behat\Mink\Exception\ExpectationException)

Root Cause

The forum subscriptions class caches forum and subscription information in a static php cache on the class. When multiple tests are run in the same process the cache was not being reset properly.

This only happens in cases where the subscription data is set in the Behat process because the actual run is a separate set of processes for each run. When we create test data we do so in the same behat process all along, which can lead to cache bleed where caches are not reset between tests.