[MDL-74096] Various errors and failures in mlbackend phpunit tests - Moodle Tracker

XML

Word

Printable

Details

Type: Bug
Status: Closed
Priority: Critical
Resolution: Fixed
Affects Version/s: 3.11.6, 4.0
Fix Version/s: 3.9.14, 3.10.11, 3.11.7
Component/s: Analytics, Unit tests
Labels:

Affected Branches:
MOODLE_311_STABLE, MOODLE_400_STABLE
Fixed Branches:
MOODLE_310_STABLE, MOODLE_311_STABLE, MOODLE_39_STABLE
Pull from Repository:
https://github.com/ilyatregubov/moodle.git
Pull 3.9 Branch:
~~MDL-74096~~-39
Pull 3.9 Diff URL:
https://github.com/ilyatregubov/moodle/compare/db6f627bad...MDL-74096-39
Pull 3.11 Branch:
~~MDL-74096~~-311
Pull 3.11 Diff URL:
https://github.com/ilyatregubov/moodle/compare/3ab09c775d...MDL-74096-311
Pull Master Branch:
~~MDL-74096~~-master
Pull Master Diff URL:
https://github.com/ilyatregubov/moodle/compare/7ce003b666...MDL-74096-master
Documentation link:
https://docs.moodle.org/311/en/Analytics_settings#Python_predictions_processor

Testing Instructions:

Hide

Setup

Run Moodle ML backend via docker:
- For master
  docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:3.0.4-python3.7.5
- For 311 and below: (when testing on other mlbackend container, you may need to stop the container running the 3.0.4 version. "docker stop [container_name]")
  docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:2.6.5-python3.7.5

Setup the needed constants in config.php:

define('TEST_MLBACKEND_PYTHON_HOST', 'localhost');

define('TEST_MLBACKEND_PYTHON_PORT', 5000);

define('TEST_MLBACKEND_PYTHON_USERNAME', 'default');

define('TEST_MLBACKEND_PYTHON_PASSWORD', 'sshhhh');

define('PHPUNIT_LONGTEST', true);

Phpunit

Run analytics unit tests

vendor/bin/phpunit -v --testsuite core_analytics_testsuite

Ensure tests are passing

Behat

mdk behat -r -n "Create a model"
mdk behat -r -n "Evaluate a model"
Ensure tests are passing

Show

Setup Run Moodle ML backend via docker: For master docker run -d -p 5000 : 5000 moodlehq/moodle-mlbackend-python: 3.0 . 4 -python3. 7.5 For 311 and below: (when testing on other mlbackend container, you may need to stop the container running the 3.0.4 version. " docker stop [container_name] ") docker run -d -p 5000 : 5000 moodlehq/moodle-mlbackend-python: 2.6 . 5 -python3. 7.5 Setup the needed constants in config.php: define( 'TEST_MLBACKEND_PYTHON_HOST' , 'localhost' ); define( 'TEST_MLBACKEND_PYTHON_PORT' , 5000 ); define( 'TEST_MLBACKEND_PYTHON_USERNAME' , 'default' ); define( 'TEST_MLBACKEND_PYTHON_PASSWORD' , 'sshhhh' ); define( 'PHPUNIT_LONGTEST' , true ); Phpunit Run analytics unit tests vendor/bin/phpunit -v --testsuite core_analytics_testsuite Ensure tests are passing Behat mdk behat -r -n "Create a model" mdk behat -r -n "Evaluate a model" Ensure tests are passing

Sprint:
Moppies Kanban

Description

It seems that unit tests are not passing against the python mlbackend.

To reproduce

Run the mlbackend docker image:

docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:3.0.2-python3.7.5

Setup the needed constants in config.php:

define('TEST_MLBACKEND_PYTHON_HOST', 'localhost');

define('TEST_MLBACKEND_PYTHON_PORT', 5000);

define('TEST_MLBACKEND_PYTHON_USERNAME', 'default');

define('TEST_MLBACKEND_PYTHON_PASSWORD', 'sshhhh');

Init PHPUnit and run the core_analytics_prediction_testcase testcase:

vendor/bin/phpunit -v --filter core_analytics_prediction_testcase

Expected

The tests pass ok.

Current status

There are errors (2) and failures (2)

Moodle 4.0beta (Build: 20220303), 87791a13a413498116fc5f081f5e48256f6e492f

Php: 8.0.16, mysqli: 8.0.27, OS: Darwin 20.6.0 x86_64

PHPUnit 9.5.13 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.0.16

Configuration: /Users/stronk7/git_moodle/testing/phpunit.xml

.....F.E.........F.....E..                                        26 / 26 (100%)

Time: 02:49.630, Memory: 291.00 MB

There were 2 errors:

1) core_analytics_prediction_testcase::test_ml_export_import with data set "case-\mlbackend_python\processor-server" ('\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))

moodle_exception: Server error HTTP status code 500: 1

/Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:642

/Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:401

/Users/stronk7/git_moodle/testing/analytics/classes/model_config.php:91

/Users/stronk7/git_moodle/testing/analytics/classes/model.php:1605

/Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:392

/Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80

/private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98

2) core_analytics_prediction_testcase::test_ml_evaluation_trained_model with data set "case-\mlbackend_python\processor-server" ('\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))

moodle_exception: Server error HTTP status code 500: 1

/Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:642

/Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:401

/Users/stronk7/git_moodle/testing/analytics/classes/model_config.php:91

/Users/stronk7/git_moodle/testing/analytics/classes/model.php:1605

/Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:684

/Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80

/private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98

--

There were 2 failures:

1) core_analytics_prediction_testcase::test_ml_training_and_prediction with data set "quarters-\mlbackend_python\processor-server" ('\core\analytics\time_splittin...arters', 3, 4, '\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))

Failed asserting that '0' matches expected 1.

/Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:254

/Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80

/private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98

2) core_analytics_prediction_testcase::test_ml_multi_classifier with data set "notimesplitting-\mlbackend_python\processor-server" ('\core\analytics\time_splittin...itting', '\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))

Failed asserting that '1' matches expected 2.

/Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:578

/Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80

/private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98

ERRORS!

Tests: 26, Assertions: 213, Errors: 2, Failures: 2.

This has not been tested using local server mlbackend instalation. Only using the docker one. Also, it may affect older versions.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

MDL-74096_master_behat_1.png
122 kB
04/Apr/22 7:44 PM
MDL-74096_master_behat_2.png
121 kB
04/Apr/22 7:44 PM
MDL-74096_master_phpunit.png
51 kB
04/Apr/22 7:44 PM
MDL-74096_v310_behat_1.png
130 kB
04/Apr/22 7:44 PM
MDL-74096_v310_behat_2.png
131 kB
04/Apr/22 7:44 PM
MDL-74096_v310_phpunit.png
52 kB
04/Apr/22 7:44 PM
MDL-74096_v311_behat_1.png
123 kB
04/Apr/22 7:44 PM
MDL-74096_v311_behat_2.png
124 kB
04/Apr/22 7:44 PM
MDL-74096_v311_phpunit.png
51 kB
04/Apr/22 7:44 PM
MDL-74096_v39_behat_1.png
121 kB
04/Apr/22 7:44 PM
MDL-74096_v39_behat_2.png
121 kB
04/Apr/22 7:44 PM
MDL-74096_v39_phpunit.png
56 kB
04/Apr/22 7:44 PM

Issue Links

blocks

MDLQA-16576 CLONE - Admins can change analytics settings and setup different machine learning backends

Passed

MDLQA-16623 CLONE - An admin can set up the Python machine learning backend on a separate server

Passed

Discovered while testing

MDLQA-16576 CLONE - Admins can change analytics settings and setup different machine learning backends

Passed

Activity

People

Assignee:: Ilya Tregubov

Reporter:: Eloy Lafuente (stronk7)

Peer reviewer:: Amaia Anabitarte

Integrator:: Jun Pataleta

Tester:: Angelia Dela Cruz

Participants:: Amaia Anabitarte, Angelia Dela Cruz, CiBoT, Eloy Lafuente (stronk7), Ilya Tregubov, Jun Pataleta

Component watchers:: David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 06/Mar/22 6:12 AM

Updated:: 05/Apr/22 7:56 PM

Resolved:: 05/Apr/22 7:56 PM

Fix Release Date:: 9/May/22

Time Tracking

Estimated:

Remaining:

Logged:

3d 1h 20m