Moodle
  1. Moodle
  2. MDL-3357

Random questions in mulit-page quizzes

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 1.5
    • Fix Version/s: None
    • Component/s: Quiz
    • Labels:
      None
    • Environment:
      Windows XP
    • Database:
      MySQL
    • Affected Branches:
      MOODLE_15_STABLE

      Description

      20-question quiz, items selected randomly from among 82 items in 4 item banks, timed (10 minutes). If given as one page, no problems found. If divide into two pages, 3 errors:

      a. time resets to full time whenever page changed (so have unlimited time)

      b. same item may appear on different pages (but is never repeated on same page)

      c. score reflects either 1 page only or no pages – as near as I can recall tried following:

      (1) take quize normally, submit at end of page 2, score reflects only page 2 items correct.

      (2) take quiz, save after page 1, continue to page 2, save after page 2, return to page 1 and submit, score shows no items correct.

      (3) take quiz, save after page 1, continue to page 2, submit, socre reflects only page 1 items correct

        Gliffy Diagrams

          Activity

          Hide
          Martin Dougiamas added a comment -

          From Jean_Michel Vedrine (jean-michel.vedrine at wanadoo.fr) Sunday, 29 May 2005, 12:13 AM:

          This bug is reproductible : I made a test containing 6 random questions on 3 pages from a category of 10 T/F questions and I was able to get a mark as low as 0 with all corrects answers just by navigating between pages during the test.

          The problem with the timer restarting at each page is also reproductible.

          From Gustav Delius (gwd2 at york.ac.uk) Monday, 30 May 2005, 10:10 AM:

          I tested this in the past, so this must have been broken by some of my more recent 'bug fixes'. Thanks for reporting this, I have upgraded it to critical.

          From Gustav Delius (gwd2 at york.ac.uk) Wednesday, 1 June 2005, 02:54 PM:

          Problem a) was due to a typo that I introduced recently and is now fixed.

          Problem b) is a bigger bug due to a faulty implementation of the random questions. I am copying in Julian because he adapted the random question type to the new model recently. I don't think the current implementation can be rescued very easily. I think we should now go over to the new model that I proposed in MDL-3281.

          Is problem c) related to problem b), i.e., does it only occur due to the random questions?

          From James Robertson (jimrprovo at gmail.com) Thursday, 2 June 2005, 02:16 AM:

          I created a 3-page quiz using 30 questions selected from 4 item banks. The items were selected normally (not as random items), though random administration and choice order were both selected. I logged in as a student and tried out a few different paging sequences, but found no errors in scoring. By contrast, in the randomly selected version of the test I was unable to find any paging sequence that produced a correct score. So it appears that item (c), scoring errors, is related to randomly selected items with multiple pages only.

          One possible quick fix for immediate release (while developing as described under MDL-3281) might be to disable multiple pages for tests with randomly selected items. Cheap & dirty, but perhaps it would allow the effort to be put where it's needed with no harm done in the meantime: the new page feature just isn't yet available with randomly selected items.

          At least from my perspective as a user, I would prefer you put your effort towards improving/correcting the underlying structure for a long-term solution rather than spending time on doing a midway fix to support a new feature in an old problemmatic structure that needs to be replaced.

          From Gustav Delius (gwd2 at york.ac.uk) Thursday, 2 June 2005, 02:43 PM:

          The proposal in MDL-3281 has its own problems so here comes an alternative solution for this bug. The current problem is that when a random question session is started an actual question is chosen without taking into accout whether that question is not already in use on other pages of the quiz. It would be inefficient to try to do that in the current model because it is so difficult to find out what questions are actually used in the random questions on other pages.

          Let's introduce a new table quiz_random_attempts with fields

          id, quiz, question, attempt, actualquestion. This table will allow a random question to find out which actual question is being used in a particular attempt. The quiz index is in there for convenience because it will allow an easy way to find all the questions used in a particular quiz.

          When create_session_and_responses() for the random question finds that for that particular random question in that particular attempt there is not yet an entry in this table then it creates entries for ALL random questions in that quiz for that attempt, Making sure that there is no duplication of questions.

          From James Robertson (jimrprovo at gmail.com) Friday, 3 June 2005, 03:33 AM:

          That sounds like it should work. If I understand correctly, you are proposing an items-delivered table, which would be utilized for all quizzes as well: i.e., for all quiz types, the new table is the definitive list of what was actually delivered, while the existing quiz table serves as the prescription for what will be delivered. In addition, this would also work in the future if computer-adaptive format is added (using current response history to select the next item using difficulty and discrimination parameters, stop at max num or at statistical criterion) – which I expect will happen sooner or later. An items-delivered table should support any format, with normal sequential format a special case.

          Would the ghost items still be required? It seems like they should be removed – though maybe they already have been moved to a separate ghost item database. In any case, thanks for removing them visually from the user interface and using the actual items rather than the ghost items in the item statistics.

          From Julian Sedding (jps502 at york.ac.uk) Friday, 3 June 2005, 04:22 PM:

          At firs I was a bit surprised that the questionsused field used in the create_session_and_responses of the random questiontype does not do it's job properly, however the reason is quite clearly that the questionsused field is not persistant.

          It gets either $attempts->layout or $quiz->questions as its initial value. Instead of the $attempts->layout the information should come from the new table instead.

          Therefore I think that introducing such a table should work fine. I would reconsider how to name the table, because it sounds like it has nothing to do with random in particular. It is rather concerned with wrapping questions that get substituted during runtime and can't therefore make use of the parent field in the quiz_questions table. Maybe quiz_question_substitutes would be better?

          Currently there are only random and randomsamatch, where it makes sense to use this table. It can of course also be used for all other questions for consistency as I think jimrprovo implies.

          From Julian Sedding (jps502 at york.ac.uk) Friday, 3 June 2005, 09:00 PM:

          Fixed this without any fancy stuff. The attempt.php script now initialises (and saves!) all questions at once. This way when we navigate the random questions are restored, not created, which works.

          I'll leave this open as a reminder for the solution with the quiz_question_substitutes table.

          Show
          Martin Dougiamas added a comment - From Jean_Michel Vedrine (jean-michel.vedrine at wanadoo.fr) Sunday, 29 May 2005, 12:13 AM: This bug is reproductible : I made a test containing 6 random questions on 3 pages from a category of 10 T/F questions and I was able to get a mark as low as 0 with all corrects answers just by navigating between pages during the test. The problem with the timer restarting at each page is also reproductible. From Gustav Delius (gwd2 at york.ac.uk) Monday, 30 May 2005, 10:10 AM: I tested this in the past, so this must have been broken by some of my more recent 'bug fixes'. Thanks for reporting this, I have upgraded it to critical. From Gustav Delius (gwd2 at york.ac.uk) Wednesday, 1 June 2005, 02:54 PM: Problem a) was due to a typo that I introduced recently and is now fixed. Problem b) is a bigger bug due to a faulty implementation of the random questions. I am copying in Julian because he adapted the random question type to the new model recently. I don't think the current implementation can be rescued very easily. I think we should now go over to the new model that I proposed in MDL-3281 . Is problem c) related to problem b), i.e., does it only occur due to the random questions? From James Robertson (jimrprovo at gmail.com) Thursday, 2 June 2005, 02:16 AM: I created a 3-page quiz using 30 questions selected from 4 item banks. The items were selected normally (not as random items), though random administration and choice order were both selected. I logged in as a student and tried out a few different paging sequences, but found no errors in scoring. By contrast, in the randomly selected version of the test I was unable to find any paging sequence that produced a correct score. So it appears that item (c), scoring errors, is related to randomly selected items with multiple pages only. One possible quick fix for immediate release (while developing as described under MDL-3281 ) might be to disable multiple pages for tests with randomly selected items. Cheap & dirty, but perhaps it would allow the effort to be put where it's needed with no harm done in the meantime: the new page feature just isn't yet available with randomly selected items. At least from my perspective as a user, I would prefer you put your effort towards improving/correcting the underlying structure for a long-term solution rather than spending time on doing a midway fix to support a new feature in an old problemmatic structure that needs to be replaced. From Gustav Delius (gwd2 at york.ac.uk) Thursday, 2 June 2005, 02:43 PM: The proposal in MDL-3281 has its own problems so here comes an alternative solution for this bug. The current problem is that when a random question session is started an actual question is chosen without taking into accout whether that question is not already in use on other pages of the quiz. It would be inefficient to try to do that in the current model because it is so difficult to find out what questions are actually used in the random questions on other pages. Let's introduce a new table quiz_random_attempts with fields id, quiz, question, attempt, actualquestion. This table will allow a random question to find out which actual question is being used in a particular attempt. The quiz index is in there for convenience because it will allow an easy way to find all the questions used in a particular quiz. When create_session_and_responses() for the random question finds that for that particular random question in that particular attempt there is not yet an entry in this table then it creates entries for ALL random questions in that quiz for that attempt, Making sure that there is no duplication of questions. From James Robertson (jimrprovo at gmail.com) Friday, 3 June 2005, 03:33 AM: That sounds like it should work. If I understand correctly, you are proposing an items-delivered table, which would be utilized for all quizzes as well: i.e., for all quiz types, the new table is the definitive list of what was actually delivered, while the existing quiz table serves as the prescription for what will be delivered. In addition, this would also work in the future if computer-adaptive format is added (using current response history to select the next item using difficulty and discrimination parameters, stop at max num or at statistical criterion) – which I expect will happen sooner or later. An items-delivered table should support any format, with normal sequential format a special case. Would the ghost items still be required? It seems like they should be removed – though maybe they already have been moved to a separate ghost item database. In any case, thanks for removing them visually from the user interface and using the actual items rather than the ghost items in the item statistics. From Julian Sedding (jps502 at york.ac.uk) Friday, 3 June 2005, 04:22 PM: At firs I was a bit surprised that the questionsused field used in the create_session_and_responses of the random questiontype does not do it's job properly, however the reason is quite clearly that the questionsused field is not persistant. It gets either $attempts->layout or $quiz->questions as its initial value. Instead of the $attempts->layout the information should come from the new table instead. Therefore I think that introducing such a table should work fine. I would reconsider how to name the table, because it sounds like it has nothing to do with random in particular. It is rather concerned with wrapping questions that get substituted during runtime and can't therefore make use of the parent field in the quiz_questions table. Maybe quiz_question_substitutes would be better? Currently there are only random and randomsamatch, where it makes sense to use this table. It can of course also be used for all other questions for consistency as I think jimrprovo implies . From Julian Sedding (jps502 at york.ac.uk) Friday, 3 June 2005, 09:00 PM: Fixed this without any fancy stuff. The attempt.php script now initialises (and saves!) all questions at once. This way when we navigate the random questions are restored, not created, which works. I'll leave this open as a reminder for the solution with the quiz_question_substitutes table.
          Hide
          Michael Blake added a comment -

          assign to a valid user

          Show
          Michael Blake added a comment - assign to a valid user

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: