Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-61077

Quiz statistics calculation code is too fragile

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.2.6, 3.3.3, 3.4
    • Fix Version/s: 3.3.5, 3.4.2
    • Component/s: Questions, Quiz
    • Labels:
    • Testing Instructions:
      Hide

      The main thing to test is that this does not cause any regressions for quiz statistics reprots that currently work:

      1. In whatever Moodle site you have available, find the quiz with the most different question types and attempts.
      2. Run the Statistics report.
      3. Acutally, if possible, run the report using last week's Moodle code, and do Download complete report as HTML, then upgrade to the fixed code, and download the complete report, and verify that all the numbers are identical.

      The other thing to test is that we have fixed the problem. The way to do this is basically to reproduce MDL-51695, and now verify that instead of a fatal error, instead you get a report, but with some develope debug output to hilight the problem:

      1. Ensure that Debugging is set to DEVELOPER level.
      2. In a test course, create a quiz, add a multiple-choice question with three choices, A, B and C. (A as the right answer.)
      3. Attempt the quiz as a student, select choice 'A', and submit the attempt.
      4. Attempt the quiz as a second student, select choice 'C' and submit the attempt
      5. Edit the question, to remove choice 'C' from the mulitple choice question.
      6. Run the statistics report.
      7. (For this particular problem, no debug output is generated at this stage, but in other cases it might be.)
      8. Verify that, despite the problem, the report displays, with reasonsable data (e.g. it shows there were 2 attempts, the facility index for the question is 0.5.
      9. Click the question name link.
      10. Verify that there is some debug output after the resonse analysis table.
      11. Verify that, in the response analysis it shows that A was selected 1 time, and the actual response C is shown, even though there is no corresponding model response.
      Show
      The main thing to test is that this does not cause any regressions for quiz statistics reprots that currently work: In whatever Moodle site you have available, find the quiz with the most different question types and attempts. Run the Statistics report. Acutally, if possible, run the report using last week's Moodle code, and do Download complete report as HTML, then upgrade to the fixed code, and download the complete report, and verify that all the numbers are identical. The other thing to test is that we have fixed the problem. The way to do this is basically to reproduce MDL-51695 , and now verify that instead of a fatal error, instead you get a report, but with some develope debug output to hilight the problem: Ensure that Debugging is set to DEVELOPER level. In a test course, create a quiz, add a multiple-choice question with three choices, A, B and C. (A as the right answer.) Attempt the quiz as a student, select choice 'A', and submit the attempt. Attempt the quiz as a second student, select choice 'C' and submit the attempt Edit the question, to remove choice 'C' from the mulitple choice question. Run the statistics report. (For this particular problem, no debug output is generated at this stage, but in other cases it might be.) Verify that, despite the problem, the report displays, with reasonsable data (e.g. it shows there were 2 attempts, the facility index for the question is 0.5. Click the question name link. Verify that there is some debug output after the resonse analysis table. Verify that, in the response analysis it shows that A was selected 1 time, and the actual response C is shown, even though there is no corresponding model response.
    • Affected Branches:
      MOODLE_32_STABLE, MOODLE_33_STABLE, MOODLE_34_STABLE
    • Fixed Branches:
      MOODLE_33_STABLE, MOODLE_34_STABLE
    • Pull from Repository:
    • Pull Master Branch:

      Description

      There have been several bugs (e.g. MDL-51147, MDL-51695) where slightly buggy code in one question type has caused the quiz statistics report to completely fail to display.

      This is due to the programming idiom

      $somearray[$key]->some_method($data);

      If $key is not present in the array, then you get a fatal error call to member function on non-object.

      Since this relates to data coming from arbitrary plugins, this is not a good thing.

        Attachments

          Activity

            People

            • Assignee:
              timhunt Tim Hunt
              Reporter:
              timhunt Tim Hunt
              Peer reviewer:
              Shamim Rezaie
              Integrator:
              Eloy Lafuente (stronk7)
              Tester:
              Adrian Greeve
              Participants:
              Component watchers:
              Tim Hunt, Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze, Tim Hunt, Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Fix Release Date:
                19/Mar/18