Quoting a dev chat discussion (slightly abbreviated):
Once again, I have got a question to all Behat gurus out there.
I am currently working on a Behat test for a plugin of ours. The plugin's main feature is packaged in a scheduled task, thus I have to create a scenario, run the scheduled task and then check what has been changed by the task.
For running a task, Moodle provides this Behat step:
And I run the scheduled task "local_sandbox\task\restore_courses"
With this step, I was able to create a successful Behat test.
However, for debugging of the Behat test, I needed to have a look at the output of the scheduled task and replaced the abovementioned line with this:
And the following config values are set as admin:
config value pathtophp /usr/local/bin/php
And I navigate to "Server > Tasks > Scheduled tasks" in site administration
And I click on "Run now" "link" in the "Restore sandbox courses" "table_row"
And I click on "Run now" "button"
I was able to see the output of the task.
But confusingly, the test did fail.
Upon further investigation, I saw that when the scheduled task in run in this way, at least $DB and $SITE do not point to the Behat site but to the standard Moodle site. And that's why my test fails.
Could anyone explain me why?
"Could anyone explain me why?" Because when you manually run a Moodle task from the web interface, it will not actually run within the interface. Instead, it will open a shell in the background, starting the task via the command that you set in the instructions ("/usr/local/bin/php"). This will start a new PHP process that runs the task, and the new process will be completely oblivious to the fact that it was started from within Behat.
wow this is really dangerous, I think we might need to disable setting pathtophp in behat environment
Exactly, and it cannot know that it should do otherwise. I think Marina is right. Tests that try and set (and use) this setting are probably always incorrect
The word "dangerous" above was indeed used in a general meaning, as there is a potential that someone would run their Behat tests in a production environment.
In behat environments, the "Run now" feature of scheduled tasks should not be possible.
This can be achieved by forbidding the use of $CFG->pathtophp in behat environments.