Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-35325

Question engine provides feedback for last saved answer, not last submitted/graded

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.1.2, 2.3.2, 2.4.3, 2.5.4, 2.6.1
    • Fix Version/s: STABLE backlog
    • Component/s: Questions, Quiz
    • Labels:
    • Testing Instructions:
      Hide

      1. Create a new quiz, and select the "adaptive" question behavior. This bug technically affects any of the multiple-try question types; but only question types like adaptive demonstrate the vulnerability.
      2. Enable feedback. To see all aspects of the quiz at once, enable the display of general feedback, specific feedback, and marks during the attempt.
      3. Add at least one question with responses (e.g. short answer, multianswer, numeric), and provide per-answer feedback for each response. I recommend trying the mutlianswer/cloze question type, as it has the largest vulnerability.
      4. Start a quiz attempt.
      5. Enter an incorrect answer to the question, and submit it for grading.
      6. Change your answer to one of the other responses you provided, but do not submit it.
      7. Perform any action that causes the question to be saved- e.g. submit a different question, hit the next button at the bottom of the page, etc.
      8. Navigate back to the question. Verify the question's feedback does not change to reflect the changed response.

      Show
      1. Create a new quiz, and select the "adaptive" question behavior. This bug technically affects any of the multiple-try question types; but only question types like adaptive demonstrate the vulnerability. 2. Enable feedback. To see all aspects of the quiz at once, enable the display of general feedback, specific feedback, and marks during the attempt. 3. Add at least one question with responses (e.g. short answer, multianswer, numeric), and provide per-answer feedback for each response. I recommend trying the mutlianswer/cloze question type, as it has the largest vulnerability. 4. Start a quiz attempt. 5. Enter an incorrect answer to the question, and submit it for grading. 6. Change your answer to one of the other responses you provided, but do not submit it . 7. Perform any action that causes the question to be saved- e.g. submit a different question, hit the next button at the bottom of the page, etc. 8. Navigate back to the question. Verify the question's feedback does not change to reflect the changed response.
    • Difficulty:
      Moderate
    • Affected Branches:
      MOODLE_21_STABLE, MOODLE_23_STABLE, MOODLE_24_STABLE, MOODLE_25_STABLE, MOODLE_26_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-35325-feedback

      Description

      The newer Moodle question engine currently provides feedback based on the last saved answer, rather than the last graded or submitted answer. In many cases, this could allow an opportunistic student to receive feedback (and, in some cases, an indication of correctness) without submitting a question for grading- and without the risk of receiving penalties!

      In the case of the multi-answer question, the per-question marks are also based on the last saved answer.

      This bug is best seen for oneself:

      1. Create a new quiz, and select the "adaptive" question behavior. This bug technically affects any of the multiple-try question types; but only question types like adaptive demonstrate the vulnerability.
      2. Enable feedback. To see all aspects of the quiz at once, enable the display of general feedback, specific feedback, and marks during the attempt.
      3. Add at least one question with responses (e.g. short answer, multianswer, numeric), and provide per-answer feedback for each response. I recommend trying the mutlianswer/cloze question type, as it has the largest vulnerability.
      4. Start a quiz attempt.
      5. Enter an incorrect answer to the question, and submit it for grading.
      6. Change your answer to one of the other responses you provided, but do not submit it.
      7. Perform any action that causes the question to be saved- e.g. submit a different question, hit the next button at the bottom of the page, etc.
      8. Navigate back to the question. Its feedback will have changed to reflect the feedback the user would have received, had they submitted the question!

      This strikes me as very undesirable behavior- as it allows users to circumvent submission penalties, and potentially create misleading submission histories. I plan on uploading a backup of an example quiz momentarily, and I plan on starting work on a fix tonight.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Votes:
                  6 Vote for this issue
                  Watchers:
                  9 Start watching this issue

                  Dates

                  • Created:
                    Updated: