Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-60667

Only linear functions in mlbackend_python

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.4
    • Fix Version/s: 3.4
    • Component/s: Analytics
    • Labels:
    • Testing Instructions:
      Hide

      Test that there are no regressions:

      1. Uninstall moodlemlbackend if it was already installed
        1. sudo pip uninstall moodlemlbackend
      2. Install the new version
        • During peer review: (no new release available yet) git clone git://github.com/dmonllao/https://github.com/dmonllao/moodle-mlbackend-python ; cd moodle-mlbackend-python ; sudo pip install . -e
        • During integration review and testing: (new versoin released) sudo pip install moodlemlbackend
      3. Run vendor/bin/phpunit -c analytics, they should all pass with no skipped tests

      Test the accuracy improvement:

      1. Download the attached MDL-60667.dataset.csv
      2. Execute python -m moodlemlbackend.evaluation 'whatever' '/tmp/dataset-mdl-60667' '/YOUR/PATH/TO/MDL-60667.dataset.csv' '0.7' '0.05' '10' | python -m json.tool
      3. Copy these results to a .txt file to later compare them with the previous version
      4. Change your moodlemlbackend version to the previous one:
        • If you cloned the repo you will also need to rm -rf /PATH/TO/YOUR/CLONE/moodlemlbackend.egg-info
        • sudo pip install moodlemlbackend==0.0.4
      5. Check that you got version 0.0.4 by executing python -m moodlemlbackend.version
      6. Execute again python -m moodlemlbackend.evaluation 'whatever' '/tmp/dataset-mdl-60667' '/YOUR/PATH/TO/MDL-60667.dataset.csv' '0.7' '0.05' '10' | python -m json.tool
      7. The score value should be lower than the previous execution value
      Show
      Test that there are no regressions: Uninstall moodlemlbackend if it was already installed sudo pip uninstall moodlemlbackend Install the new version During peer review: (no new release available yet) git clone git://github.com/dmonllao/ https://github.com/dmonllao/moodle-mlbackend-python ; cd moodle-mlbackend-python ; sudo pip install . -e During integration review and testing: (new versoin released) sudo pip install moodlemlbackend Run vendor/bin/phpunit -c analytics , they should all pass with no skipped tests Test the accuracy improvement: Download the attached MDL-60667 .dataset.csv Execute python -m moodlemlbackend.evaluation 'whatever' '/tmp/dataset-mdl-60667' '/YOUR/PATH/TO/ MDL-60667 .dataset.csv' '0.7' '0.05' '10' | python -m json.tool Copy these results to a .txt file to later compare them with the previous version Change your moodlemlbackend version to the previous one: If you cloned the repo you will also need to rm -rf /PATH/TO/YOUR/CLONE/moodlemlbackend.egg-info sudo pip install moodlemlbackend==0.0.4 Check that you got version 0.0.4 by executing python -m moodlemlbackend.version Execute again python -m moodlemlbackend.evaluation 'whatever' '/tmp/dataset-mdl-60667' '/YOUR/PATH/TO/ MDL-60667 .dataset.csv' '0.7' '0.05' '10' | python -m json.tool The score value should be lower than the previous execution value
    • Affected Branches:
      MOODLE_34_STABLE
    • Fixed Branches:
      MOODLE_34_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-60667_master

      Description

      Predictions accuracy is significantly lower because the tensorflow classifier (mlbackend_python) only splits the hyperplane linearly. I thought I changed this months ago but I didn't; changes were part of a WIP branch I worked on during a project week and never had time to complete.

      The only required changes in moodle code are a bump to the next moodlemlbackend package release.

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Fix Release Date:
                13/Nov/17