Quoting a dev chat discussion (slightly abbreviated):

abias:

Once again, I have got a question to all Behat gurus out there.

I am currently working on a Behat test for a plugin of ours. The plugin's main feature is packaged in a scheduled task, thus I have to create a scenario, run the scheduled task and then check what has been changed by the task.

For running a task, Moodle provides this Behat step:
And I run the scheduled task "local_sandbox\task\restore_courses"

With this step, I was able to create a successful Behat test.

However, for debugging of the Behat test, I needed to have a look at the output of the scheduled task and replaced the abovementioned line with this:

And the following config values are set as admin:

config	value
pathtophp	/usr/local/bin/php

And I navigate to "Server > Tasks > Scheduled tasks" in site administration
And I click on "Run now" "link" in the "Restore sandbox courses" "table_row"
And I click on "Run now" "button"

I was able to see the output of the task.
But confusingly, the test did fail.

Upon further investigation, I saw that when the scheduled task in run in this way, at least $DB and $SITE do not point to the Behat site but to the standard Moodle site. And that's why my test fails.

Could anyone explain me why?

jan.dagefoerde:

"Could anyone explain me why?" Because when you manually run a Moodle task from the web interface, it will not actually run within the interface. Instead, it will open a shell in the background, starting the task via the command that you set in the instructions ("/usr/local/bin/php"). This will start a new PHP process that runs the task, and the new process will be completely oblivious to the fact that it was started from within Behat.

marina:

wow this is really dangerous, I think we might need to disable setting pathtophp in behat environment

jan.dagefoerde:

Exactly, and it cannot know that it should do otherwise. I think Marina is right. Tests that try and set (and use) this setting are probably always incorrect

mudrd8mz:

The word "dangerous" above was indeed used in a general meaning, as there is a potential that someone would run their Behat tests in a production environment.

Conclusion:
In behat environments, the "Run now" feature of scheduled tasks should not be possible.
This can be achieved by forbidding the use of $CFG->pathtophp in behat environments.