Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-30031

Adaptive mode should ignore invalid answers without penalty

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.2, 2.2, 2.3
    • Fix Version/s: 2.1.5, 2.2.2
    • Component/s: Questions
    • Labels:
    • Testing Instructions:
      Hide

      Preparation:

      • Create a quiz with the following setting:
        • Title: "Adaptive Test"
        • How questions behave: Adaptive mode
      • Create a question with the following settings, and add it to the quiz:
        • Type: Numerical
        • Question text: "What is 2+2?"
        • Answer: 4 (Grade 100%)
      • Duplicate the quiz, and change settings in the copy as follows:
        • Title: "AdaptiveNoPenalty Test"
        • How questions behave: Adaptive (no penalties)

      Test script 1:

      • Preview the quiz "Adaptive test"
      • As the answer, enter "four", and click "Check".

      VERIFY: The following two messages are displayed:
      "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

      • As the answer, enter "4", and click "Check"

      VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

      • Click "Next", then "Submit all and finish".

      VERIFY: Total marks given for the quiz attempt are 1.00/1.00.

      Test script 2:

      • Preview the quiz "AdaptiveNoPenalties test"
      • As the answer, enter "four", and click "Check".

      VERIFY: The following message is displayed (but none beyond this):
      "You must enter a valid number"

      • As the answer, enter "4", and click "Check"

      VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

      Test script 3:

      Run all the unit tests in path "question".

      Show
      Preparation: Create a quiz with the following setting: Title: "Adaptive Test" How questions behave: Adaptive mode Create a question with the following settings, and add it to the quiz: Type: Numerical Question text: "What is 2+2?" Answer: 4 (Grade 100%) Duplicate the quiz, and change settings in the copy as follows: Title: "AdaptiveNoPenalty Test" How questions behave: Adaptive (no penalties) Test script 1: Preview the quiz "Adaptive test" As the answer, enter "four", and click "Check". VERIFY: The following two messages are displayed: "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty" As the answer, enter "4", and click "Check" VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00." Click "Next", then "Submit all and finish". VERIFY: Total marks given for the quiz attempt are 1.00/1.00. Test script 2: Preview the quiz "AdaptiveNoPenalties test" As the answer, enter "four", and click "Check". VERIFY: The following message is displayed (but none beyond this): "You must enter a valid number" As the answer, enter "4", and click "Check" VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00." Test script 3: Run all the unit tests in path "question".
    • Affected Branches:
      MOODLE_21_STABLE, MOODLE_22_STABLE, MOODLE_23_STABLE
    • Fixed Branches:
      MOODLE_21_STABLE, MOODLE_22_STABLE
    • Pull from Repository:
    • Pull Master Branch:

      Description

      Quizzes in adaptive mode should ignore invalid answers to a question (e.g., non-numerical answers to numerical questions) without applying a penalty.

      This currently works with numerical questions in interactive mode, but fails with numerical questions in adaptive mode. (Other combinations not yet tested.)

      See http://moodle.org/mod/forum/discuss.php?d=189276

        Gliffy Diagrams

          Activity

          bostelm Henning Bostelmann created issue -
          bostelm Henning Bostelmann made changes -
          Field Original Value New Value
          Assignee Tim Hunt [ timhunt ] Henning Bostelmann [ bostelm ]
          Hide
          timhunt Tim Hunt added a comment -

          Henning, thanks very much for taking this on. Can I ask that when you assign a quiz/question bug to yourself, can you add me as a watcher if I am not otherwise linked to the bug. Thanks. Don't worry if you forget, I will probably notice and add myself.

          Show
          timhunt Tim Hunt added a comment - Henning, thanks very much for taking this on. Can I ask that when you assign a quiz/question bug to yourself, can you add me as a watcher if I am not otherwise linked to the bug. Thanks. Don't worry if you forget, I will probably notice and add myself.
          timhunt Tim Hunt made changes -
          Fix Version/s STABLE backlog [ 10463 ]
          Labels triaged
          bostelm Henning Bostelmann made changes -
          Pull Master Diff URL https://github.com/bostelm/moodle/compare/master...MDL-30031
          Pull Master Branch MDL-30031
          Testing Instructions * Create a quiz with the following setting:
          ** How questions behave: Adaptive mode
          * Create a question with the following settings, and add it to the quiz:
          ** Type: Numerical
          ** Question text: "What is 2+2?"
          ** Answer: 4 (Grade 100%)
          * Preview the quiz
          * As the answer, enter "four", and click "Check".

          VERIFY: The following two messages are displayed:
          "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

          * As the answer, enter "4", and click "Check"

          VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

          * Click "Next", then "Submit all and finish".

          VERIFY: Total marks given for the quiz attempt are 1.00/1.00.
          Pull from Repository git://github.com/bostelm/moodle
          timhunt Tim Hunt made changes -
          Status Open [ 1 ] Peer review in progress [ 10013 ]
          Peer reviewer timhunt
          Hide
          timhunt Tim Hunt added a comment -

          This looks very good, and I was about to submit it for integration when I spotted a small problem.

          1. Preview your example question using 'Adaptive (no penalty)' behaviour.
          2. Submit the answer 'four'.

          Actual result: You are told your invalid submission did not attract a penalty.

          Expected behaviour: Well, I am not sure what the ideal behaviour is, but the word penalty should no be there. We need to change the no penalties renderer slightly, and add at least one no-penalties unit test.

          Will you be able to tackle that? If you do, could you also make branches for 21_ and 22_STABLE? Thanks.

          Show
          timhunt Tim Hunt added a comment - This looks very good, and I was about to submit it for integration when I spotted a small problem. 1. Preview your example question using 'Adaptive (no penalty)' behaviour. 2. Submit the answer 'four'. Actual result: You are told your invalid submission did not attract a penalty. Expected behaviour: Well, I am not sure what the ideal behaviour is, but the word penalty should no be there. We need to change the no penalties renderer slightly, and add at least one no-penalties unit test. Will you be able to tackle that? If you do, could you also make branches for 21_ and 22_STABLE? Thanks.
          timhunt Tim Hunt made changes -
          Status Peer review in progress [ 10013 ] Development in progress [ 3 ]
          timhunt Tim Hunt made changes -
          Testing Instructions * Create a quiz with the following setting:
          ** How questions behave: Adaptive mode
          * Create a question with the following settings, and add it to the quiz:
          ** Type: Numerical
          ** Question text: "What is 2+2?"
          ** Answer: 4 (Grade 100%)
          * Preview the quiz
          * As the answer, enter "four", and click "Check".

          VERIFY: The following two messages are displayed:
          "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

          * As the answer, enter "4", and click "Check"

          VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

          * Click "Next", then "Submit all and finish".

          VERIFY: Total marks given for the quiz attempt are 1.00/1.00.
          * Create a quiz with the following setting:
          ** How questions behave: Adaptive mode
          * Create a question with the following settings, and add it to the quiz:
          ** Type: Numerical
          ** Question text: "What is 2+2?"
          ** Answer: 4 (Grade 100%)
          * Preview the quiz
          * As the answer, enter "four", and click "Check".

          VERIFY: The following two messages are displayed:
          "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

          * As the answer, enter "4", and click "Check"

          VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

          * Click "Next", then "Submit all and finish".

          VERIFY: Total marks given for the quiz attempt are 1.00/1.00.


          ... some similar test using Adaptive (no penalty) ... behaviour


          Also, run all the unit tests in question.
          Hide
          bostelm Henning Bostelmann added a comment -

          Good point. I think it will be best to suppress the message entirely in "Adaptive No Penalties" mode. I have changed the code accordingly. Unit test added.

          Show
          bostelm Henning Bostelmann added a comment - Good point. I think it will be best to suppress the message entirely in "Adaptive No Penalties" mode. I have changed the code accordingly. Unit test added.
          bostelm Henning Bostelmann made changes -
          Testing Instructions * Create a quiz with the following setting:
          ** How questions behave: Adaptive mode
          * Create a question with the following settings, and add it to the quiz:
          ** Type: Numerical
          ** Question text: "What is 2+2?"
          ** Answer: 4 (Grade 100%)
          * Preview the quiz
          * As the answer, enter "four", and click "Check".

          VERIFY: The following two messages are displayed:
          "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

          * As the answer, enter "4", and click "Check"

          VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

          * Click "Next", then "Submit all and finish".

          VERIFY: Total marks given for the quiz attempt are 1.00/1.00.


          ... some similar test using Adaptive (no penalty) ... behaviour


          Also, run all the unit tests in question.
          Preparation:

          * Create a quiz with the following setting:
          ** Title: "Adaptive Test"
          ** How questions behave: Adaptive mode
          * Create a question with the following settings, and add it to the quiz:
          ** Type: Numerical
          ** Question text: "What is 2+2?"
          ** Answer: 4 (Grade 100%)
          * Duplicate the quiz, and change settings in the copy as follows:
          ** Title: "AdaptiveNoPenalty Test"
          ** How questions behave: Adaptive (no penalties)

          Test script 1:

          * Preview the quiz "Adaptive test"
          * As the answer, enter "four", and click "Check".

          VERIFY: The following two messages are displayed:
          "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

          * As the answer, enter "4", and click "Check"

          VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

          * Click "Next", then "Submit all and finish".

          VERIFY: Total marks given for the quiz attempt are 1.00/1.00.

          Test script 2:

          * Preview the quiz "AdaptiveNoPenalties test"
          * As the answer, enter "four", and click "Check".

          VERIFY: The following message is displayed (but none beyond this):
          "You must enter a valid number"

          * As the answer, enter "4", and click "Check"

          VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

          Test script 3:

          Run all the unit tests in path "question".
          Pull 2.2 Diff URL https://github.com/bostelm/moodle/compare/MOODLE_22_STABLE...MDL-30031-22
          Pull 2.1 Branch MDL-30031-21
          Pull 2.2 Branch MDL-30031-22
          Pull 2.1 Diff URL https://github.com/bostelm/moodle/compare/MOODLE_21_STABLE...MDL-30031-21
          bostelm Henning Bostelmann made changes -
          Status Development in progress [ 3 ] Waiting for peer review [ 10012 ]
          Hide
          timhunt Tim Hunt added a comment -

          Great, thanks Henning! I'm submitting this for integration now.

          Show
          timhunt Tim Hunt added a comment - Great, thanks Henning! I'm submitting this for integration now.
          timhunt Tim Hunt made changes -
          Status Waiting for peer review [ 10012 ] Waiting for integration review [ 10010 ]
          Fix Version/s 2.1.4 [ 11452 ]
          Fix Version/s 2.2.1 [ 11456 ]
          Fix Version/s 2.3 [ 10657 ]
          Fix Version/s STABLE backlog [ 10463 ]
          stronk7 Eloy Lafuente (stronk7) made changes -
          Currently in integration Yes [ 10041 ]
          stronk7 Eloy Lafuente (stronk7) made changes -
          Status Waiting for integration review [ 10010 ] Integration review in progress [ 10004 ]
          Integrator stronk7
          stronk7 Eloy Lafuente (stronk7) made changes -
          Status Integration review in progress [ 10004 ] Waiting for integration review [ 10010 ]
          Hide
          stronk7 Eloy Lafuente (stronk7) added a comment -

          The integration of this issue has been delayed to next week because the integration period is over (Monday, Tuesday) and testing must happen on Wednesday.

          This change to a more rigid timeframe on each integration/testing cycle aims to produce a better and clear separation and organization of tasks for everybody.

          This is a bulk-automated message, so if you want to blame somebody/thing/where, don't do it here (use git instead) :-D :-P

          Apologizes for the inconvenient, this will be integrated next week. Thanks for your collaboration & ciao

          Show
          stronk7 Eloy Lafuente (stronk7) added a comment - The integration of this issue has been delayed to next week because the integration period is over (Monday, Tuesday) and testing must happen on Wednesday. This change to a more rigid timeframe on each integration/testing cycle aims to produce a better and clear separation and organization of tasks for everybody. This is a bulk-automated message, so if you want to blame somebody/thing/where, don't do it here (use git instead) :-D :-P Apologizes for the inconvenient, this will be integrated next week. Thanks for your collaboration & ciao
          stronk7 Eloy Lafuente (stronk7) made changes -
          Currently in integration Yes [ 10041 ]
          Hide
          stronk7 Eloy Lafuente (stronk7) added a comment -

          The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.

          TIA and ciao

          Show
          stronk7 Eloy Lafuente (stronk7) added a comment - The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week. TIA and ciao
          stronk7 Eloy Lafuente (stronk7) made changes -
          Fix Version/s 2.2.2 [ 11552 ]
          Fix Version/s 2.2.1 [ 11456 ]
          stronk7 Eloy Lafuente (stronk7) made changes -
          Fix Version/s 2.1.5 [ 11553 ]
          Fix Version/s 2.1.4 [ 11452 ]
          samhemelryk Sam Hemelryk made changes -
          Currently in integration Yes [ 10041 ]
          stronk7 Eloy Lafuente (stronk7) made changes -
          Status Waiting for integration review [ 10010 ] Integration review in progress [ 10004 ]
          Hide
          stronk7 Eloy Lafuente (stronk7) added a comment -

          Integrated, thanks! I adore issues with tests, yay!

          Show
          stronk7 Eloy Lafuente (stronk7) added a comment - Integrated, thanks! I adore issues with tests, yay!
          stronk7 Eloy Lafuente (stronk7) made changes -
          Status Integration review in progress [ 10004 ] Waiting for testing [ 10005 ]
          Affects Version/s 2.3 [ 10657 ]
          Fix Version/s 2.3 [ 10657 ]
          rajeshtaneja Rajesh Taneja made changes -
          Status Waiting for testing [ 10005 ] Testing in progress [ 10011 ]
          Tester rajeshtaneja
          Hide
          rajeshtaneja Rajesh Taneja added a comment -

          Works Great
          Thanks for fixing this, Henning.

          Show
          rajeshtaneja Rajesh Taneja added a comment - Works Great Thanks for fixing this, Henning.
          rajeshtaneja Rajesh Taneja made changes -
          Status Testing in progress [ 10011 ] Tested [ 10006 ]
          Hide
          stronk7 Eloy Lafuente (stronk7) added a comment -

          This virus has been spread upstream, everybody will be infected soon. Congrats, you did it!

          Closing, ciao

          Show
          stronk7 Eloy Lafuente (stronk7) added a comment - This virus has been spread upstream, everybody will be infected soon. Congrats, you did it! Closing, ciao
          stronk7 Eloy Lafuente (stronk7) made changes -
          Status Tested [ 10006 ] Closed [ 6 ]
          Resolution Fixed [ 1 ]
          Currently in integration Yes [ 10041 ]
          Integration date 12/Jan/12

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Fix Release Date:
                12/Mar/12