PHPUnit memory usage almost doubled recently (3.10dev)

We have started to get some out-of-memory errors in some branches having a good number of unit tests. But it was hard to trace.

So we tried some "phpunit leak detectors" (a plugin, a listerner...) and, while they reported a good number of tests leaking memory, it was not easy to go checking each:

https://gist.github.com/stronk7/3632139e3a52d4137a66338c5b005719

But the final surprise came when it was discovered that 39_STABLE (and earlier) was using "only" ~800 MB to perform complete runs. And MOODLE_310_STABLE (current) and up are the ones requiring ~1800 MB to perform the very same complete runs.

Initially my thoughts were that, maybe, the upgrade to PHPUnit 8.5 (MDL-67673) could be the culprit, but did run a test before that issue and the memory problem was already there (~1700 MB to run).

Note that all the runs are with PHPUNIT_LONGTEST set.

So, at the end, a bisect has been performed between:

• good: 500c131 - Moodle 3.9.0 release : 886 MB
• bad: 5903054 - The issue BEFORE PHPUnit 8.5, already wrong: 1700 MB

And the bisect has ended with:

0deb19468d0e61c0f3c41fb8d350cd2611fad47e is the first bad commit

commit 0deb19468d0e61c0f3c41fb8d350cd2611fad47e

Author: sam marshall <s.marshall@open.ac.uk>

Date:   Wed May 13 12:11:21 2020 +0100

    MDL-68690 Search: Allow Solr to add documents in batches

    Adding documents in batches instead of one at a time can make

    indexing using Solr significantly faster.

    This adds new API functions for search engines, including

    add_document_batch() to add a batch of documents,

    supports_add_document_batch(), get_batch_max_documents() and

    get_batch_max_content().

 search/classes/engine.php                |  97 +++++++++++++++++++--

 search/classes/manager.php               |  36 ++++++--

 search/engine/solr/classes/engine.php    | 100 +++++++++++++++++++++-

 search/engine/solr/tests/engine_test.php | 142 +++++++++++++++++++++++++++++++

 search/upgrade.txt                       |   8 ++

 5 files changed, 367 insertions(+), 16 deletions(-)

To confirm it, I've run PHPUnit tests @ CI:

• just before MDL-68690 applied (22f4eab): (link) => 900 MB
• with MDL-68690 applied (9e65bc8): (link) => 1700 MB

So, double confirmation about the issue causing the double memory regression.

Ciao