Preparations

This is about to inspect apache logs for complete behat runs and fix as many notices / problems are shown there. Aggregating information from all supported themes.

Used this regexp against goutte and chrome webserver logs, for reference:

ag -v 'POST|GET|OPTIONS|\] (checking plan|(saving|calculating|loading|setting) controller|(applying|loading|instantiating) (backup|restore|plan)|(activity|section) skipped|setting file inclusion|processing file aliases|Unknown setting: (questionbank|overwrite_conf|course_shortname|course_fullname|course_startdate))|Failed Login: ' aggregated.txt > results.txt

(where aggregated.txt is the union of goutte and chrome web server logs of all core themes)

Trick: When it's difficult to find which exact behat test is causing the problem, it's useful to run the tests with the --pretty command, getting the date prepended, that way later can match them with the dates in apache error logs. To do so just add the following to your behat run command:

vendor/bin/behat ......  --format pretty --no-colors | awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; fflush(); }'

You can see the actual results @ results.txt

Trick: Always look to previous release issue to verify if the error already was detected there and considered ok.

Summary:

Problem A: Invalid get_string() identifier '' or component 'core'. It happens 662 times.

MDL-69553

Happens with each scenario adding an activity, when the chooser is displayed. For example "'Add a quiz"

Can be also manually reproduced by going to a couse, enable edition, then purge caches and, immediately after they are purged, click on "Add and activity or resource". Debugging to web server logs will happen.

391541:[Fri Oct 16 19:48:07.760852 2020] [php7:notice] [pid 1407] [client 172.28.0.13:54790] Debugging: Invalid get_string() identifier: '' or component 'core'. Perhaps you are missing $string[''] = ''; in lang/en/core.php? in

• line 353 of /lib/classes/string_manager_standard.php: call to debugging()
• line 206 of /lib/classes/output/mustache_template_source_loader.php: call to core_string_manager_standard->get_string()
• line 181 of /lib/classes/output/mustache_template_source_loader.php: call to core\\output\\mustache_template_source_loader->core\\output\\\{closure()
• line 209 of /lib/classes/output/mustache_template_source_loader.php: call to core\\output\\mustache_template_source_loader->core\\output\\\{closure}()
• line 231 of /lib/classes/output/mustache_template_source_loader.php: call to core\\output
  mustache_template_source_loader->load_with_dependencies()
• line 181 of /lib/classes/output/mustache_template_source_loader.php: call to core\\output\\mustache_template_source_loader->core\\output\\\{closure}()
• line 253 of /lib/classes/output/mustache_template_source_loader.php: call to core\\output\\mustache_template_source_loader->core\\output\\\{closure}()
• line 152 of /lib/classes/output/external.php: call to core\\output
  mustache_template_source_loader->load_with_dependencies()
• line 261 of /lib/externallib.php: call to core\\output
  external::load_template_with_dependencies()
• line 81 of /lib/ajax/service.php: call to external_api::call_external_function()
• line 33 of /lib/ajax/service-nologin.php: call to require_once()\n, referer: http://run443860bff9a85b27/behatrun3/course/view.php?id=101000&notifyeditingon=1

Problem B: Cannot find session record. It happens 2 times.

Happens in scenario "As a student, request deletion of account and data"

77079:[Sat Oct 17 07:55:07.133032 2020] [php7:notice] [pid 25076] [client 172.21.0.3:45938] Cannot find session record da0d1f0c0bf2f6f4864e37319d156999 for user 106000, creating new session., referer: http://runb21888ffe3c5386b/behatrun1/user/profile.php?id=106000

Confirmed that it's testing exactly that the session becomes has expired (see prev. release). So 100% ignorable.

Problem C: User id 'xxxxxx' already has a grade. It happens 2 times.

MDL-68961

Happens with scenario "Delete and restore a quiz with user data.

160073:[Sat Oct 17 09:47:05.520710 2020] [php7:notice] [pid 9209] [client 172.21.0.5:37174] User id '103001' already has a grade entry for grade item id '165000', referer: http://runb21888ffe3c5386b/behatrun3/admin/tool/recyclebin/index.php?contextid=104002

Confirmed that it's trying to recreate already existing course grade (see MDL-68961). So 100% ignorable.

Problem D: Can't find data record in database table course. It happens 1 times.

MDL-69993

Happens with scenario "To an attempts and check on course log report". Execution ends ok, but with the error in the logs.

366422:[Fri Oct 16 19:23:54.586462 2020] [php7:notice] [pid 25663] [client 172.28.0.13:51906] Default exception handler: Coding error detected, it must be fixed by a programmer: Error: Can't find data record in database table course. (SELECT * FROM {course} WHERE id = ?
[array (
0 => '0',
)]) Debug: \nError code: codingerror

• line 120 of /lib/ajax/getnavbranch.php: coding_exception thrown,
  referer: http://run443860bff9a85b27/behatrun3/report/log/user.php?id=104001&course=101000&mode=today

Problem E: Coding error detected, it must be fixed by a programmer. It happens 2 times (classic).

MDL-70099

Happens with scenarios: "Edit a self-enrolled user's enrolment from the course participants page" and "Unenrol a self-enrolled student from the course participants page"

[Sun Nov 01 01:10:17.961587 2020] [php7:notice] [pid 24759] [client 192.168.48.3:56376] Default exception handler: Coding error detected, it must be fixed by a programmer: No further information available for this branch Debug:
Error code: codingerror

• line 129 of /lib/ajax/getnavbranch.php: coding_exception thrown,
  referer: http://run6e9ed569225277ac/behatrun1/course/view.php?id=103000