Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-67622

The evaluation metrics are not ready for multiclass in the python ML backend

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • 3.8.2
    • 3.8, 3.9
    • Analytics
    • MOODLE_38_STABLE, MOODLE_39_STABLE
    • MOODLE_38_STABLE
    • MDL-67622_master
    • Hide

      Prerequisites:

      1. Need python3
      2. Need pip3 (apt-get install python3-pip)

      Testing

      1. Install the new version of the python package 

        git clone git://github.com/dmonllao/moodle-mlbackend-python.git
        		 
        cd moodle-mlbackend-python
        		 
        git checkout MDL-67622_master
        		 
        pip3 install setuptools wheel
        		 
        pip3 install -e .

        1. Confirm that you did not see any error during the installation process
      2. Check the version: 

        python3 -m moodlemlbackend.version

        1. Confirm that the version reported was 2.3.1
      3. Set $CFG->pathtopython = '/full/path/to/python3'; in your config.php (to just specify 'python3' is probably enough)
      4. Unit tests including PHPUNIT_LONGTEST should pass and no analytics tests should be skipped. You can do it running the command below: 

        vendor/bin/phpunit analytics/tests/prediction_test.php

      Show
      Prerequisites: Need python3 Need pip3 (apt-get install python3-pip) Testing Install the new version of the python package git clone git://github.com/dmonllao/moodle-mlbackend-python.git cd moodle-mlbackend-python git checkout MDL-67622_master pip3 install setuptools wheel pip3 install -e . Confirm that you did not see any error during the installation process Check the version: python3 -m moodlemlbackend.version Confirm that the version reported was 2.3.1 Set $CFG->pathtopython = '/full/path/to/python3'; in your config.php (to just specify 'python3' is probably enough) Unit tests including PHPUNIT_LONGTEST should pass and no analytics tests should be skipped. You can do it running the command below: vendor/bin/phpunit analytics/tests/prediction_test.php

      In MDL-58992 we added multi-class capabilities to the ML backends in moodle core. We didn't update calculate_metrics function in the python ML backend. This function is responsible of returning the accuracy metrics that serve as a measure of the quality of the model. Also, the f1 score should be calculated using a weighted average like we do for the PHP ML backend.

            dmonllao David Monllaó
            dmonllao David Monllaó
            Victor Déniz Falcón Victor Déniz Falcón
            Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
            Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 days, 50 minutes
                2d 50m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.