[MDL-67622] The evaluation metrics are not ready for multiclass in the python ML backend - Moodle Tracker

XML

Word

Printable

Details

Type: Bug
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: 3.8, 3.9
Fix Version/s: 3.8.2
Component/s: Analytics
Labels:
- ci
- triaged

Affected Branches:
MOODLE_38_STABLE, MOODLE_39_STABLE
Fixed Branches:
MOODLE_38_STABLE
Pull from Repository:
git://github.com/dmonllao/moodle.git
Pull Master Branch:
~~MDL-67622~~_master
Pull Master Diff URL:
https://github.com/dmonllao/moodle/compare/MDL-67622_master~...MDL-67622_master
Testing Instructions:
Hide

Prerequisites:

Need python3

Need pip3 (apt-get install python3-pip)

Testing

Install the new version of the python package

git clone git://github.com/dmonllao/moodle-mlbackend-python.git

cd moodle-mlbackend-python

git checkout MDL-67622_master

pip3 install setuptools wheel

pip3 install -e .

Confirm that you did not see any error during the installation process

Check the version:

python3 -m moodlemlbackend.version

Confirm that the version reported was 2.3.1

Set $CFG->pathtopython = '/full/path/to/python3'; in your config.php (to just specify 'python3' is probably enough)

Unit tests including PHPUNIT_LONGTEST should pass and no analytics tests should be skipped. You can do it running the command below:

vendor/bin/phpunit analytics/tests/prediction_test.php
Show
Prerequisites: Need python3 Need pip3 (apt-get install python3-pip) Testing Install the new version of the python package git clone git://github.com/dmonllao/moodle-mlbackend-python.git cd moodle-mlbackend-python git checkout MDL-67622_master pip3 install setuptools wheel pip3 install -e . Confirm that you did not see any error during the installation process Check the version: python3 -m moodlemlbackend.version Confirm that the version reported was 2.3.1 Set $CFG->pathtopython = '/full/path/to/python3'; in your config.php (to just specify 'python3' is probably enough) Unit tests including PHPUNIT_LONGTEST should pass and no analytics tests should be skipped. You can do it running the command below: vendor/bin/phpunit analytics/tests/prediction_test.php

Description

In ~~MDL-58992~~ we added multi-class capabilities to the ML backends in moodle core. We didn't update calculate_metrics function in the python ML backend. This function is responsible of returning the accuracy metrics that serve as a measure of the quality of the model. Also, the f1 score should be calculated using a weighted average like we do for the PHP ML backend.

Attachments

Issue Links

blocks

MDL-67040 Improve the performance of the Analytics ML layer when using small datasets

Closed

caused a regression

MDL-58992 Add multi-class capabilities to prediction processors

Closed

Activity

People

Assignee:: David Monllaó

Reporter:: David Monllaó

Peer reviewer:: Víctor Déniz Falcón

Integrator:: Eloy Lafuente (stronk7)

Tester:: Eloy Lafuente (stronk7)

Participants:: CiBoT, David Monllaó, Eloy Lafuente (stronk7), Víctor Déniz Falcón

Component watchers:: Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Ilya Tregubov, Laurent David, Sara Arjona (@sarjona)

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 06/Jan/20 12:43 PM

Updated:: 17/Jan/20 1:56 AM

Resolved:: 17/Jan/20 1:56 AM

Fix Release Date:: 9/Mar/20

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

2d 50m