Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-58835

Ability to hide predictions (flag them as not useful or acknowledged / fixed)

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major Major
    • 3.4
    • 3.4
    • Analytics, Reports
    • MOODLE_34_STABLE
    • MOODLE_34_STABLE
    • MDL-58835_master
    • Hide
      1. Download and install git clone git://github.com/dmonllao/moodle-local_testanalytics.git local/testanalytics (best plugin ever)
      2. Create 2 new categories without courses
      3. Go to admin/tool/analytics and execute Actions > Get predictions in "Binary" model
      4. Select "All predictions" from insights drop down menu (Binary model)
      5. Refresh the page a few times, checking that the order of the items remains the same
      6. Expand the first item Actions menu and click "Acknowledged / fixed"
      7. The item SHOULD be removed from the page
      8. Refresh the page, the remove item SHOULD NOT be there
      9. Expand the new first item Actions menu and click "Not useful"
      10. The item SHOULD be removed from the page
      11. Expand the new first item Actions menu and click "View prediction details"
      12. Select * from mdl_analytics_prediction_actions, you should see 3 different records, each of them with a different actionname value (you can find more records here if you played a bit with the feature)
      13. Go to report/log/index.php?id=0 and get those logs, you should see some "Prediction process started" logs and some "Insights viewed" logs
      14. Go to admin/tool/analytics and execute Actions > Get predictions in "Useless categories" model
      15. Back in admin/tool/analytics/index.php select one of the categories listed in Insights column, "Useless categories"
      16. Flag the only item that appears there as "Not useful"
      17. You SHOULD be redirected to admin/tool/analytics/index.php
      18. The category you flagged as not useful should not appear in the Insights column menu anymore, if there are no remaining categories you will see "No insights reported" message instead of the dropdown
      Show
      Download and install git clone git://github.com/dmonllao/moodle-local_testanalytics.git local/testanalytics (best plugin ever) Create 2 new categories without courses Go to admin/tool/analytics and execute Actions > Get predictions in "Binary" model Select "All predictions" from insights drop down menu (Binary model) Refresh the page a few times, checking that the order of the items remains the same Expand the first item Actions menu and click "Acknowledged / fixed" The item SHOULD be removed from the page Refresh the page, the remove item SHOULD NOT be there Expand the new first item Actions menu and click "Not useful" The item SHOULD be removed from the page Expand the new first item Actions menu and click "View prediction details" Select * from mdl_analytics_prediction_actions, you should see 3 different records, each of them with a different actionname value (you can find more records here if you played a bit with the feature) Go to report/log/index.php?id=0 and get those logs, you should see some "Prediction process started" logs and some "Insights viewed" logs Go to admin/tool/analytics and execute Actions > Get predictions in "Useless categories" model Back in admin/tool/analytics/index.php select one of the categories listed in Insights column, "Useless categories" Flag the only item that appears there as "Not useful" You SHOULD be redirected to admin/tool/analytics/index.php The category you flagged as not useful should not appear in the Insights column menu anymore, if there are no remaining categories you will see "No insights reported" message instead of the dropdown

      The insights page lists predictions.Targets define suggested actions teachers can perform over these predictions. A new action to flag the prediction as not useful would help us refine the model predictions.

      This would require a new default action for targets using insights and a new field in tool_inspire_predictions to store the teacher action.

      We can start by skipping these predictions from being used for training but not useful predictions will keep coming up as the model is trained with these samples. We can go further than this and force future predictions (that come from the machine learning backends) pass through another model (trained with the prediction calculations + teachers input) and filter out predictions that match the pattern of previous predictions that teachers marked as not useful.

      This would require some changes in the predictions processor API.

      We agreed to work on refining predictions with teachers input during latest analytics meeting (May 2017)

        1. prediction-actions.png
          prediction-actions.png
          43 kB

            dmonllao David Monllaó
            dmonllao David Monllaó
            Damyon Wiese Damyon Wiese
            Dan Poltawski Dan Poltawski
            Adrian Greeve Adrian Greeve
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.