Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-67622

The evaluation metrics are not ready for multiclass in the python ML backend

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 3.8, 3.9
    • 3.8.2
    • Analytics
    • MOODLE_38_STABLE, MOODLE_39_STABLE
    • MOODLE_38_STABLE
    • MDL-67622_master
    • Hide

      Prerequisites:

      1. Need python3
      2. Need pip3 (apt-get install python3-pip)

      Testing

      1. Install the new version of the python package

        git clone git://github.com/dmonllao/moodle-mlbackend-python.git
        cd moodle-mlbackend-python
        git checkout MDL-67622_master
        pip3 install setuptools wheel
        pip3 install -e .
        

        1. Confirm that you did not see any error during the installation process
      2. Check the version:

        python3 -m moodlemlbackend.version
        

        1. Confirm that the version reported was 2.3.1
      3. Set $CFG->pathtopython = '/full/path/to/python3'; in your config.php (to just specify 'python3' is probably enough)
      4. Unit tests including PHPUNIT_LONGTEST should pass and no analytics tests should be skipped. You can do it running the command below:

        vendor/bin/phpunit analytics/tests/prediction_test.php
        

      Show
      Prerequisites: Need python3 Need pip3 (apt-get install python3-pip) Testing Install the new version of the python package git clone git://github.com/dmonllao/moodle-mlbackend-python.git cd moodle-mlbackend-python git checkout MDL-67622_master pip3 install setuptools wheel pip3 install -e . Confirm that you did not see any error during the installation process Check the version: python3 -m moodlemlbackend.version Confirm that the version reported was 2.3.1 Set $CFG->pathtopython = '/full/path/to/python3'; in your config.php (to just specify 'python3' is probably enough) Unit tests including PHPUNIT_LONGTEST should pass and no analytics tests should be skipped. You can do it running the command below: vendor/bin/phpunit analytics/tests/prediction_test.php

    Description

      In MDL-58992 we added multi-class capabilities to the ML backends in moodle core. We didn't update calculate_metrics function in the python ML backend. This function is responsible of returning the accuracy metrics that serve as a measure of the quality of the model. Also, the f1 score should be calculated using a weighted average like we do for the PHP ML backend.

      Attachments

        Issue Links

          Activity

            People

              dmonllao David Monllaó
              dmonllao David Monllaó
              Victor Déniz Falcón Victor Déniz Falcón
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                9/Mar/20

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 days, 50 minutes
                  2d 50m