Requirements Ability to run the testing instructions with and without the patch installed. Execution of commands via command line. With the patch not applied (or reverted) Init behat for 2 parallel runs php admin/tool/behat/cli/init.php --parallel=2 Let's run (with run.php, for parallel) some choice features having some name (no selenium is needed) php admin/tool/behat/cli/run.php --tags='@mod_choice&&~@javascript' --name='a choice' Verify that 2 parallel runs happen and both pass (10 scenarios right now). From the test output , after the Running 2 parallel behat sites , copy the [behatrun1] vendor... and [behatrun2] vendor... . We'll need them later. Edit mod/choice/tests/behat/change_response.feature and change any of the I should see ... expectations to force a failure (with some text not matching, like, yay! . Let's run again (with run.php, for parallel) some choice features having some name (no selenium is needed) php admin/tool/behat/cli/run.php --tags='@mod_choice&&~@javascript' --name='a choice' Verify that 2 parallel runs happen and one of them fails now. Let's call FAILED to the one failing (1 or 2). Let's rerun the failed run using run.php , replacing FAILED in the command below by the one that failed: php admin/tool/behat/cli/run.php --tags='@mod_choice&&~@javascript' --name='a choice' --fromrun=FAILED --torun=FAILED --rerun Verify that the run continues failing as expected and that only one scenario is executed (and failed). Now let's rerun the failed one using vendor/bin/behat , replacing CONFIGPATH in the command below with the path to the behat.yml corresponding to the run FAILED that you copied in step #4 above. Note that there is a whitespace between --config and CONFIGPATH! Note that you may need to create a link manually in your moodle root directory, depending of your webserver configuration if the behat wwwroot is not accesible: ln -s . behatrunX . vendor/bin/behat --config CONFIGPATH --tags='@mod_choice&&~@javascript' --name='a choice' --rerun Verify that the run continues failing as expected and that only one scenario is executed (and failed). Now let's rerun the failed one, exactly the same than the previous step but, instead of using a space between --config and CONFIGPATH, we use an equal symbol: vendor/bin/behat --config=CONFIGPATH --tags='@mod_choice&&~@javascript' --name='a choice' --rerun Verify that the run is not executed now and instead this error is shown: Your behat test site is outdated, please run   php admin/tool/behat/cli/init.php   from your moodle dirroot to drop and install the behat test site again. (this is the bug that we are fixing here, using "=" with --config is broken) With the patch applied Repeat steps 1-12 from the previous section. Verify that the execution of step 12, using the equal symbol instead of space, does not lead to the failure in step 13. Verify that, instead, the run is executed and the scenario fails as expected. Undo the changes that were performed to mod/choice/tests/behat/change_response.feature (in step 5 from the previous section) so it will be a valid test again. Run again the command in step 12 from the previous section. Verify that this time the run ends ok and the scenario passes. That's all, now we can use both space and equal with the --config parameter, and the other rerun methods do continue working, thanks for testing! Notes If at step 13 you instead see the following, you will need to add symlinks to your dirroot: 2031 im:master> ./vendor/bin/behat --config /Users/nicols/Sites/moodles/im/moodledata_behat/behatrun1/behat/behat.yml --tags='@mod_choice&&~@javascript' --name='a choice' --rerun Behat requirement not satisfied: http://127.0.0.1/im/behatrun1 is not available, ensure you specified correct url and that the server is set up and started. More info in https://docs.moodle.org/dev/Running_acceptance_test You need to add the following symlinks: ln -s . behatrun1 ln -s . behatrun2