Moodle
  1. Moodle
  2. MDL-30031

Adaptive mode should ignore invalid answers without penalty

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.2, 2.2, 2.3
    • Fix Version/s: 2.1.5, 2.2.2
    • Component/s: Questions
    • Labels:
    • Testing Instructions:
      Hide

      Preparation:

      • Create a quiz with the following setting:
        • Title: "Adaptive Test"
        • How questions behave: Adaptive mode
      • Create a question with the following settings, and add it to the quiz:
        • Type: Numerical
        • Question text: "What is 2+2?"
        • Answer: 4 (Grade 100%)
      • Duplicate the quiz, and change settings in the copy as follows:
        • Title: "AdaptiveNoPenalty Test"
        • How questions behave: Adaptive (no penalties)

      Test script 1:

      • Preview the quiz "Adaptive test"
      • As the answer, enter "four", and click "Check".

      VERIFY: The following two messages are displayed:
      "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

      • As the answer, enter "4", and click "Check"

      VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

      • Click "Next", then "Submit all and finish".

      VERIFY: Total marks given for the quiz attempt are 1.00/1.00.

      Test script 2:

      • Preview the quiz "AdaptiveNoPenalties test"
      • As the answer, enter "four", and click "Check".

      VERIFY: The following message is displayed (but none beyond this):
      "You must enter a valid number"

      • As the answer, enter "4", and click "Check"

      VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

      Test script 3:

      Run all the unit tests in path "question".

      Show
      Preparation: Create a quiz with the following setting: Title: "Adaptive Test" How questions behave: Adaptive mode Create a question with the following settings, and add it to the quiz: Type: Numerical Question text: "What is 2+2?" Answer: 4 (Grade 100%) Duplicate the quiz, and change settings in the copy as follows: Title: "AdaptiveNoPenalty Test" How questions behave: Adaptive (no penalties) Test script 1: Preview the quiz "Adaptive test" As the answer, enter "four", and click "Check". VERIFY: The following two messages are displayed: "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty" As the answer, enter "4", and click "Check" VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00." Click "Next", then "Submit all and finish". VERIFY: Total marks given for the quiz attempt are 1.00/1.00. Test script 2: Preview the quiz "AdaptiveNoPenalties test" As the answer, enter "four", and click "Check". VERIFY: The following message is displayed (but none beyond this): "You must enter a valid number" As the answer, enter "4", and click "Check" VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00." Test script 3: Run all the unit tests in path "question".
    • Affected Branches:
      MOODLE_21_STABLE, MOODLE_22_STABLE, MOODLE_23_STABLE
    • Fixed Branches:
      MOODLE_21_STABLE, MOODLE_22_STABLE
    • Pull from Repository:
    • Pull Master Branch:
    • Rank:
      19573

      Description

      Quizzes in adaptive mode should ignore invalid answers to a question (e.g., non-numerical answers to numerical questions) without applying a penalty.

      This currently works with numerical questions in interactive mode, but fails with numerical questions in adaptive mode. (Other combinations not yet tested.)

      See http://moodle.org/mod/forum/discuss.php?d=189276

        Activity

        Henning Bostelmann created issue -
        Henning Bostelmann made changes -
        Field Original Value New Value
        Assignee Tim Hunt [ timhunt ] Henning Bostelmann [ bostelm ]
        Hide
        Tim Hunt added a comment -

        Henning, thanks very much for taking this on. Can I ask that when you assign a quiz/question bug to yourself, can you add me as a watcher if I am not otherwise linked to the bug. Thanks. Don't worry if you forget, I will probably notice and add myself.

        Show
        Tim Hunt added a comment - Henning, thanks very much for taking this on. Can I ask that when you assign a quiz/question bug to yourself, can you add me as a watcher if I am not otherwise linked to the bug. Thanks. Don't worry if you forget, I will probably notice and add myself.
        Tim Hunt made changes -
        Fix Version/s STABLE backlog [ 10463 ]
        Labels triaged
        Henning Bostelmann made changes -
        Pull Master Diff URL https://github.com/bostelm/moodle/compare/master...MDL-30031
        Pull Master Branch MDL-30031
        Testing Instructions * Create a quiz with the following setting:
        ** How questions behave: Adaptive mode
        * Create a question with the following settings, and add it to the quiz:
        ** Type: Numerical
        ** Question text: "What is 2+2?"
        ** Answer: 4 (Grade 100%)
        * Preview the quiz
        * As the answer, enter "four", and click "Check".

        VERIFY: The following two messages are displayed:
        "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

        * As the answer, enter "4", and click "Check"

        VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

        * Click "Next", then "Submit all and finish".

        VERIFY: Total marks given for the quiz attempt are 1.00/1.00.
        Pull from Repository git://github.com/bostelm/moodle
        Tim Hunt made changes -
        Status Open [ 1 ] Peer review in progress [ 10013 ]
        Peer reviewer timhunt
        Hide
        Tim Hunt added a comment -

        This looks very good, and I was about to submit it for integration when I spotted a small problem.

        1. Preview your example question using 'Adaptive (no penalty)' behaviour.
        2. Submit the answer 'four'.

        Actual result: You are told your invalid submission did not attract a penalty.

        Expected behaviour: Well, I am not sure what the ideal behaviour is, but the word penalty should no be there. We need to change the no penalties renderer slightly, and add at least one no-penalties unit test.

        Will you be able to tackle that? If you do, could you also make branches for 21_ and 22_STABLE? Thanks.

        Show
        Tim Hunt added a comment - This looks very good, and I was about to submit it for integration when I spotted a small problem. 1. Preview your example question using 'Adaptive (no penalty)' behaviour. 2. Submit the answer 'four'. Actual result: You are told your invalid submission did not attract a penalty. Expected behaviour: Well, I am not sure what the ideal behaviour is, but the word penalty should no be there. We need to change the no penalties renderer slightly, and add at least one no-penalties unit test. Will you be able to tackle that? If you do, could you also make branches for 21_ and 22_STABLE? Thanks.
        Tim Hunt made changes -
        Status Peer review in progress [ 10013 ] Development in progress [ 3 ]
        Tim Hunt made changes -
        Testing Instructions * Create a quiz with the following setting:
        ** How questions behave: Adaptive mode
        * Create a question with the following settings, and add it to the quiz:
        ** Type: Numerical
        ** Question text: "What is 2+2?"
        ** Answer: 4 (Grade 100%)
        * Preview the quiz
        * As the answer, enter "four", and click "Check".

        VERIFY: The following two messages are displayed:
        "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

        * As the answer, enter "4", and click "Check"

        VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

        * Click "Next", then "Submit all and finish".

        VERIFY: Total marks given for the quiz attempt are 1.00/1.00.
        * Create a quiz with the following setting:
        ** How questions behave: Adaptive mode
        * Create a question with the following settings, and add it to the quiz:
        ** Type: Numerical
        ** Question text: "What is 2+2?"
        ** Answer: 4 (Grade 100%)
        * Preview the quiz
        * As the answer, enter "four", and click "Check".

        VERIFY: The following two messages are displayed:
        "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

        * As the answer, enter "4", and click "Check"

        VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

        * Click "Next", then "Submit all and finish".

        VERIFY: Total marks given for the quiz attempt are 1.00/1.00.


        ... some similar test using Adaptive (no penalty) ... behaviour


        Also, run all the unit tests in question.
        Hide
        Henning Bostelmann added a comment -

        Good point. I think it will be best to suppress the message entirely in "Adaptive No Penalties" mode. I have changed the code accordingly. Unit test added.

        Show
        Henning Bostelmann added a comment - Good point. I think it will be best to suppress the message entirely in "Adaptive No Penalties" mode. I have changed the code accordingly. Unit test added.
        Henning Bostelmann made changes -
        Testing Instructions * Create a quiz with the following setting:
        ** How questions behave: Adaptive mode
        * Create a question with the following settings, and add it to the quiz:
        ** Type: Numerical
        ** Question text: "What is 2+2?"
        ** Answer: 4 (Grade 100%)
        * Preview the quiz
        * As the answer, enter "four", and click "Check".

        VERIFY: The following two messages are displayed:
        "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

        * As the answer, enter "4", and click "Check"

        VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

        * Click "Next", then "Submit all and finish".

        VERIFY: Total marks given for the quiz attempt are 1.00/1.00.


        ... some similar test using Adaptive (no penalty) ... behaviour


        Also, run all the unit tests in question.
        Preparation:

        * Create a quiz with the following setting:
        ** Title: "Adaptive Test"
        ** How questions behave: Adaptive mode
        * Create a question with the following settings, and add it to the quiz:
        ** Type: Numerical
        ** Question text: "What is 2+2?"
        ** Answer: 4 (Grade 100%)
        * Duplicate the quiz, and change settings in the copy as follows:
        ** Title: "AdaptiveNoPenalty Test"
        ** How questions behave: Adaptive (no penalties)

        Test script 1:

        * Preview the quiz "Adaptive test"
        * As the answer, enter "four", and click "Check".

        VERIFY: The following two messages are displayed:
        "You must enter a valid number"; "The submission was invalid, and has been disregarded without penalty"

        * As the answer, enter "4", and click "Check"

        VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

        * Click "Next", then "Submit all and finish".

        VERIFY: Total marks given for the quiz attempt are 1.00/1.00.

        Test script 2:

        * Preview the quiz "AdaptiveNoPenalties test"
        * As the answer, enter "four", and click "Check".

        VERIFY: The following message is displayed (but none beyond this):
        "You must enter a valid number"

        * As the answer, enter "4", and click "Check"

        VERIFY: The following message is displayed: "Correct. Marks for this submission: 1.00/1.00."

        Test script 3:

        Run all the unit tests in path "question".
        Pull 2.2 Diff URL https://github.com/bostelm/moodle/compare/MOODLE_22_STABLE...MDL-30031-22
        Pull 2.1 Branch MDL-30031-21
        Pull 2.2 Branch MDL-30031-22
        Pull 2.1 Diff URL https://github.com/bostelm/moodle/compare/MOODLE_21_STABLE...MDL-30031-21
        Henning Bostelmann made changes -
        Status Development in progress [ 3 ] Waiting for peer review [ 10012 ]
        Hide
        Tim Hunt added a comment -

        Great, thanks Henning! I'm submitting this for integration now.

        Show
        Tim Hunt added a comment - Great, thanks Henning! I'm submitting this for integration now.
        Tim Hunt made changes -
        Status Waiting for peer review [ 10012 ] Waiting for integration review [ 10010 ]
        Fix Version/s 2.1.4 [ 11452 ]
        Fix Version/s 2.2.1 [ 11456 ]
        Fix Version/s 2.3 [ 10657 ]
        Fix Version/s STABLE backlog [ 10463 ]
        Eloy Lafuente (stronk7) made changes -
        Currently in integration Yes [ 10041 ]
        Eloy Lafuente (stronk7) made changes -
        Status Waiting for integration review [ 10010 ] Integration review in progress [ 10004 ]
        Integrator stronk7
        Eloy Lafuente (stronk7) made changes -
        Status Integration review in progress [ 10004 ] Waiting for integration review [ 10010 ]
        Hide
        Eloy Lafuente (stronk7) added a comment -

        The integration of this issue has been delayed to next week because the integration period is over (Monday, Tuesday) and testing must happen on Wednesday.

        This change to a more rigid timeframe on each integration/testing cycle aims to produce a better and clear separation and organization of tasks for everybody.

        This is a bulk-automated message, so if you want to blame somebody/thing/where, don't do it here (use git instead) :-D :-P

        Apologizes for the inconvenient, this will be integrated next week. Thanks for your collaboration & ciao

        Show
        Eloy Lafuente (stronk7) added a comment - The integration of this issue has been delayed to next week because the integration period is over (Monday, Tuesday) and testing must happen on Wednesday. This change to a more rigid timeframe on each integration/testing cycle aims to produce a better and clear separation and organization of tasks for everybody. This is a bulk-automated message, so if you want to blame somebody/thing/where, don't do it here (use git instead) :-D :-P Apologizes for the inconvenient, this will be integrated next week. Thanks for your collaboration & ciao
        Eloy Lafuente (stronk7) made changes -
        Currently in integration Yes [ 10041 ]
        Hide
        Eloy Lafuente (stronk7) added a comment -

        The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.

        TIA and ciao

        Show
        Eloy Lafuente (stronk7) added a comment - The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week. TIA and ciao
        Eloy Lafuente (stronk7) made changes -
        Fix Version/s 2.2.2 [ 11552 ]
        Fix Version/s 2.2.1 [ 11456 ]
        Eloy Lafuente (stronk7) made changes -
        Fix Version/s 2.1.5 [ 11553 ]
        Fix Version/s 2.1.4 [ 11452 ]
        Sam Hemelryk made changes -
        Currently in integration Yes [ 10041 ]
        Eloy Lafuente (stronk7) made changes -
        Status Waiting for integration review [ 10010 ] Integration review in progress [ 10004 ]
        Hide
        Eloy Lafuente (stronk7) added a comment -

        Integrated, thanks! I adore issues with tests, yay!

        Show
        Eloy Lafuente (stronk7) added a comment - Integrated, thanks! I adore issues with tests, yay!
        Eloy Lafuente (stronk7) made changes -
        Status Integration review in progress [ 10004 ] Waiting for testing [ 10005 ]
        Affects Version/s 2.3 [ 10657 ]
        Fix Version/s 2.3 [ 10657 ]
        Rajesh Taneja made changes -
        Status Waiting for testing [ 10005 ] Testing in progress [ 10011 ]
        Tester rajeshtaneja
        Hide
        Rajesh Taneja added a comment -

        Works Great
        Thanks for fixing this, Henning.

        Show
        Rajesh Taneja added a comment - Works Great Thanks for fixing this, Henning.
        Rajesh Taneja made changes -
        Status Testing in progress [ 10011 ] Tested [ 10006 ]
        Hide
        Eloy Lafuente (stronk7) added a comment -

        This virus has been spread upstream, everybody will be infected soon. Congrats, you did it!

        Closing, ciao

        Show
        Eloy Lafuente (stronk7) added a comment - This virus has been spread upstream, everybody will be infected soon. Congrats, you did it! Closing, ciao
        Eloy Lafuente (stronk7) made changes -
        Status Tested [ 10006 ] Closed [ 6 ]
        Resolution Fixed [ 1 ]
        Currently in integration Yes [ 10041 ]
        Integration date 12/Jan/12

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: