Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-60944

Allow models to be created, deleted, imported and exported

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      Prerequisites

      1. Install https://github.com/dmonllao/moodle-local_testanalytics
      2. Install the python ML backend as specified in https://docs.moodle.org/36/en/Analytics#Predictions_processor
        • If you already got it installed you will need to update it to the latest version (pip install -I moodlemlbackend==0.37.0)
      3. Enable analytics and make sure everything is working, refer to the docs on how to set it up if you don't have it already.
      4. Go to site admin > analytics > analytics settings. Disable 'onlycli' option and ensure that 'Default predictions processor' is set to 'PHP machine learning backend'

      Export

      1. Go to site admin > analytics > analytics models
      2. Run "Get predictions" for "Inactive users" model
      3. Return to the models list page and expand "Inactive users" model actions' menu
        1. There should not be a "Export configuration" option
        2. There should be a "Delete" option
      4. Run "Get predictions" for "Binary" model
      5. Return to the models list page and expand "Binary" model actions' menu
      6. There should be an "Export configuration" option. Click on it, a ZIP file should be downloaded
      7. Confirm that the ZIP contains a JSON file and a mlbackend directory with a model.ser file inside
      8. Open the JSON file in it and verify that it contains something like this (ignore the numbers under dependencies>xxx)

        {"target":"\\local_testanalytics\\analytics\\target\\binary_example","timesplitting":"\\core\\analytics\\time_splitting\\single_range","indicators":["\\local_testanalytics\\analytics\\indicator\\set_setting"],"dependencies":{"local_testanalytics":2017051501,"core":2019011800}}
        

      9. Return to the models list page and select "Edit" from "Binary" model actions' menu
      10. Set "Predictions processor" to "Python machine learning backend" and save changes
      11. Run "Get predictions" for "Binary" model
      12. Run "Export configuration" for "Binary" model
      13. Confirm that the ZIP contains a JSON file and a mlbackend directory with a model.json file inside. Open it and verify that it contains a serialised object (mostly float numbers)
      14. Return to the models list page and select "Edit" from "Binary" model actions' menu
      15. Untick "Enabled", set "Time-splitting method" to empty and save changes
      16. Return to the models list page and expand "Binary" model actions' menu, there should be no "Export configuration" option
      17. Run "Actions > Edit" for "Discrete" model, change the "Predictions processor" option from "Default..." to "PHP machine learning backend" and save changes.
      18. Run "Actions > Export configuration" for the following models: discrete, users without pic and linear

      Delete

      1. Just delete all your models by clicking on "Actions > Delete" and confirm the deletion

      Import

      1. Go to "site admin > analytics > analytics models" and press the "Import model" button
      2. Make sure the file area allows only .zip files.
      3. Import all five configuration files you previously exported (including the two Binary models you first exported).
        1. You should not get any error, and you should see a nice and green "The model has been successfully imported." message
        2. For each of them click on "Actions > Edit", confirm that "Enabled" is not ticked, that a time splitting method is selected, that there is at least one indicator and that the predictions processor is different than "Default..."
      4. Open the model-config.json files inside one of the zip files you downloaded and change one of the versions under "dependencies"
      5. Go to "site admin > analytics > analytics models > import model" and upload the modified file
      6. "Ignore version mismatches" should be disabled by default
      7. You should see an error message stating something like this: "The version of the following components differ from the version installed in this site: local_testanalytics (20170515031). You can use "Ignore version mismatches" option to ignore these differences."
      8. Upload the same file and now tick "Ignore version mismatches"
      9. The model should now be successfully imported (green message for the win)

      Create

      1. Go to "site admin > analytics > analytics models > create model"
      2. "No teaching" target should not appear as an option in "Target" menu
      3. Try to save changes, you shouldn't be able to do it
      4. Select "Students at risk of dropping out" as a "Target", "Read actions amount" as "Indicators" and save changes
      5. Return to the models list page and select "Actions > Edit" for "Students at risk of dropping out" model.
      6. Verify that there is no option to modify the "Target" field
      Show
      Prerequisites Install https://github.com/dmonllao/moodle-local_testanalytics Install the python ML backend as specified in https://docs.moodle.org/36/en/Analytics#Predictions_processor If you already got it installed you will need to update it to the latest version (pip install -I moodlemlbackend==0.37.0) Enable analytics and make sure everything is working, refer to the docs on how to set it up if you don't have it already. Go to site admin > analytics > analytics settings. Disable 'onlycli' option and ensure that 'Default predictions processor' is set to 'PHP machine learning backend' Export Go to site admin > analytics > analytics models Run "Get predictions" for "Inactive users" model Return to the models list page and expand "Inactive users" model actions' menu There should not be a "Export configuration" option There should be a "Delete" option Run "Get predictions" for "Binary" model Return to the models list page and expand "Binary" model actions' menu There should be an "Export configuration" option. Click on it, a ZIP file should be downloaded Confirm that the ZIP contains a JSON file and a mlbackend directory with a model.ser file inside Open the JSON file in it and verify that it contains something like this (ignore the numbers under dependencies>xxx) {"target":"\\local_testanalytics\\analytics\\target\\binary_example","timesplitting":"\\core\\analytics\\time_splitting\\single_range","indicators":["\\local_testanalytics\\analytics\\indicator\\set_setting"],"dependencies":{"local_testanalytics":2017051501,"core":2019011800}} Return to the models list page and select "Edit" from "Binary" model actions' menu Set "Predictions processor" to "Python machine learning backend" and save changes Run "Get predictions" for "Binary" model Run "Export configuration" for "Binary" model Confirm that the ZIP contains a JSON file and a mlbackend directory with a model.json file inside. Open it and verify that it contains a serialised object (mostly float numbers) Return to the models list page and select "Edit" from "Binary" model actions' menu Untick "Enabled", set "Time-splitting method" to empty and save changes Return to the models list page and expand "Binary" model actions' menu, there should be no "Export configuration" option Run "Actions > Edit" for "Discrete" model, change the "Predictions processor" option from "Default..." to "PHP machine learning backend" and save changes. Run "Actions > Export configuration" for the following models: discrete, users without pic and linear Delete Just delete all your models by clicking on "Actions > Delete" and confirm the deletion Import Go to "site admin > analytics > analytics models" and press the "Import model" button Make sure the file area allows only .zip files. Import all five configuration files you previously exported (including the two Binary models you first exported). You should not get any error, and you should see a nice and green "The model has been successfully imported." message For each of them click on "Actions > Edit", confirm that "Enabled" is not ticked, that a time splitting method is selected, that there is at least one indicator and that the predictions processor is different than "Default..." Open the model-config.json files inside one of the zip files you downloaded and change one of the versions under "dependencies" Go to "site admin > analytics > analytics models > import model" and upload the modified file "Ignore version mismatches" should be disabled by default You should see an error message stating something like this: "The version of the following components differ from the version installed in this site: local_testanalytics (20170515031). You can use "Ignore version mismatches" option to ignore these differences." Upload the same file and now tick "Ignore version mismatches" The model should now be successfully imported (green message for the win) Create Go to "site admin > analytics > analytics models > create model" "No teaching" target should not appear as an option in "Target" menu Try to save changes, you shouldn't be able to do it Select "Students at risk of dropping out" as a "Target", "Read actions amount" as "Indicators" and save changes Return to the models list page and select "Actions > Edit" for "Students at risk of dropping out" model. Verify that there is no option to modify the "Target" field
    • Affected Branches:
      MOODLE_35_STABLE
    • Fixed Branches:
      MOODLE_37_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-60944_master

      Description

      This issue belongs to MDL-64775 epic.

      This idea came from a discussion in Moodle HQ analytics chat room.

      Admins and researchers should be able to share the models they found to be useful. I am not talking about the training data nor the trained models weights, but about the model configuration.

      Models are composed by:

      • A target
      • A set of indicators
      • A time-splitting method

      I would suggest a basic export to .json file which would contain something like:

      {
          "target": "mod\awesomeness\analytics\target\potential_mentors",
          "timesplitting": "core\analytics\time_splitting\single_range",
          "indicators": [
              "\mod\awesomeness\analytics\indicator\gentle",
              "\mod\awesomeness\analytics\indicator\formal",
              "\mod\awesomeness\analytics\indicator\skilled",
              "\mod\awesomeness\analytics\indicator\gentle",
              "\mod\assign\analytics\indicator\submits_before_deadline",
          ],
          
          "moodleversion": "2017010100"
      }
      

      The system would obviously notify users if:

      • Error: Any of the specified classes is not available on the system.
      • Warning: The Moodle major version the model was created for differs from the user site major version

        Attachments

        1. export-13.png
          export-13.png
          22 kB
        2. import-7.png
          import-7.png
          32 kB
        3. import-7-edit.png
          import-7-edit.png
          29 kB
        4. import-9.png
          import-9.png
          31 kB
        5. Selection_540.png
          Selection_540.png
          70 kB

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  20/May/19

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 5 hours, 5 minutes
                  5h 5m