Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-58835

Ability to hide predictions (flag them as not useful or acknowledged / fixed)

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.4
    • Fix Version/s: 3.4
    • Component/s: Analytics, Reports
    • Labels:
    • Testing Instructions:
      Hide
      1. Download and install git clone git://github.com/dmonllao/moodle-local_testanalytics.git local/testanalytics (best plugin ever)
      2. Create 2 new categories without courses
      3. Go to admin/tool/analytics and execute Actions > Get predictions in "Binary" model
      4. Select "All predictions" from insights drop down menu (Binary model)
      5. Refresh the page a few times, checking that the order of the items remains the same
      6. Expand the first item Actions menu and click "Acknowledged / fixed"
      7. The item SHOULD be removed from the page
      8. Refresh the page, the remove item SHOULD NOT be there
      9. Expand the new first item Actions menu and click "Not useful"
      10. The item SHOULD be removed from the page
      11. Expand the new first item Actions menu and click "View prediction details"
      12. Select * from mdl_analytics_prediction_actions, you should see 3 different records, each of them with a different actionname value (you can find more records here if you played a bit with the feature)
      13. Go to report/log/index.php?id=0 and get those logs, you should see some "Prediction process started" logs and some "Insights viewed" logs
      14. Go to admin/tool/analytics and execute Actions > Get predictions in "Useless categories" model
      15. Back in admin/tool/analytics/index.php select one of the categories listed in Insights column, "Useless categories"
      16. Flag the only item that appears there as "Not useful"
      17. You SHOULD be redirected to admin/tool/analytics/index.php
      18. The category you flagged as not useful should not appear in the Insights column menu anymore, if there are no remaining categories you will see "No insights reported" message instead of the dropdown
      Show
      Download and install git clone git://github.com/dmonllao/moodle-local_testanalytics.git local/testanalytics (best plugin ever) Create 2 new categories without courses Go to admin/tool/analytics and execute Actions > Get predictions in "Binary" model Select "All predictions" from insights drop down menu (Binary model) Refresh the page a few times, checking that the order of the items remains the same Expand the first item Actions menu and click "Acknowledged / fixed" The item SHOULD be removed from the page Refresh the page, the remove item SHOULD NOT be there Expand the new first item Actions menu and click "Not useful" The item SHOULD be removed from the page Expand the new first item Actions menu and click "View prediction details" Select * from mdl_analytics_prediction_actions, you should see 3 different records, each of them with a different actionname value (you can find more records here if you played a bit with the feature) Go to report/log/index.php?id=0 and get those logs, you should see some "Prediction process started" logs and some "Insights viewed" logs Go to admin/tool/analytics and execute Actions > Get predictions in "Useless categories" model Back in admin/tool/analytics/index.php select one of the categories listed in Insights column, "Useless categories" Flag the only item that appears there as "Not useful" You SHOULD be redirected to admin/tool/analytics/index.php The category you flagged as not useful should not appear in the Insights column menu anymore, if there are no remaining categories you will see "No insights reported" message instead of the dropdown
    • Affected Branches:
      MOODLE_34_STABLE
    • Fixed Branches:
      MOODLE_34_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-58835_master

      Description

      The insights page lists predictions.Targets define suggested actions teachers can perform over these predictions. A new action to flag the prediction as not useful would help us refine the model predictions.

      This would require a new default action for targets using insights and a new field in tool_inspire_predictions to store the teacher action.

      We can start by skipping these predictions from being used for training but not useful predictions will keep coming up as the model is trained with these samples. We can go further than this and force future predictions (that come from the machine learning backends) pass through another model (trained with the prediction calculations + teachers input) and filter out predictions that match the pattern of previous predictions that teachers marked as not useful.

      This would require some changes in the predictions processor API.

      We agreed to work on refining predictions with teachers input during latest analytics meeting (May 2017)

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              dmonllao David Monllaó
              Reporter:
              dmonllao David Monllaó
              Peer reviewer:
              Damyon Wiese
              Integrator:
              Dan Poltawski
              Tester:
              Adrian Greeve
              Participants:
              Component watchers:
              Elizabeth Dalton, Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Sara Arjona (@sarjona), Víctor Déniz Falcón, Adrian Greeve, Jake Dallimore, Mathew May, Mihail Geshoski, Peter Dias
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                13/Nov/17