Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-63905

Multianswer and XML question import formats can create corrupt cloze questions

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.4.5, 3.5.2, 3.6
    • Fix Version/s: 3.5.6, 3.6.4
    • Component/s: Questions
    • Labels:
    • Testing Instructions:
      Hide

      This issue try to fix the same problem in the 2 qformat plugins that can import multianswer (cloze) questions (other core qformat plugins can't import multianswer questions). The problem is that no validation is done after the question's text is decoded, so broken questions can be saved into the database and lead to big problems after that (particularly if the course containing these questions are backuped and restored).

      Verify that validation when creating multianswer questions using the UI is still working

      As this issue modify the code used to validate multianswer questions when using the Moodle UI it is necessary to verify that this is still working

      • Login as a teacher
      • Go to the question bank
      • Create a new "Embedded answers (Cloze)" question
      • Type something for the question name
      • Type some text without any embedded subquestion for the question text.
      • Click on the Save changes button and verify that you get the "The question text must include at least one embedded answer. " error message under the question text
      • Include a multichoice subquestion with only one choice, (for instance {1:MULTICHOICE:=Apple#Correct}) in the question text.
      • Click on Save choices button and verify that you get the "This type of question requires at least 2 choices" error message under the subquestion answer
      • Include a numerical subquestion with an answer that is not a valid number (for instance {3:NUMERICAL:=two})
      • Click on the Save changes button and verify that you get the "The answer must be a number, for example -1.234 or 3e8, or '*'." error message under the subquestion answer
      • Now enter a correct multianswer question text (you can find some sample in the documentation https://docs.moodle.org/36/en/Embedded_Answers_(Cloze)_question_type#Examples) and verify that it is correctly saved when you click on the Save changes button
      • I think that this is enough to verify that validation is working but feel free to try any other incorrect cloze syntax and report if you find anything incorrectly validated

      Verify that questions are validated when imported using the Embedded answers (Cloze) file format

      This step is somewhat tedious as this import format is only able to import one question at a time !

      • Login as a teacher
      • Go to the question bank import page
      • Choose the "Embedded answers (Cloze)" File format
      • Open the General subsection and set "Stop on error" to "No" (This is optional, you can also test to import the same file leaving this setting to the default value.
      • Choose the "broken_multianswer_1.txt" file attached to this issue
      • Click on the import button
      • Verify that the error message "Error importing question Invalid embedded answers (Cloze) question (This type of question requires at least 2 choices)." is displayed and that you also get "There are no questions in the import file ". this is normal because here the import format can only contains one question so if this question is invalid the file is considered empty.
      • Redo the same process with the broken_multianswer_2.txt and broken_multianswer_3.txt files and verify that you get the "Error importing question Invalid embedded answers (Cloze) question (One of the answers should have a score of 100% so it is possible to get full marks for this question.)." and "Error importing question Invalid embedded answers (Cloze) question (The answer must be a number, for example -1.234 or 3e8, or ''.).*" errors messages.
      • Redo the same process with the "correct_multianswer.txt" file and verify that the question is correctly imported without any error message
      • Try any other sample of file containing a bad multianswer question that you can think of and report if you find any buggy validation.

      Verify that questions are validated when imported using the Moodle XML file format

      Fortunately here when can import many questions in the same file so this is a lot faster.

      • Login as a teacher
      • Go to the question bank import
      • Choose "Moodle XML format" File format
      • Open the general subsection and set "Stop on error" to "No"
      • Choose the "broken_cloze_questions.xml" file attached to this issue
      • Click on the Import button
      • Verify that you get 5 errors messages and that no questions are imported from the file
      • Redo the same process with the "moodlexml_with_error.xml" file uploaded by Koen Roggemans and verify that you get one "Error importing question Invalid embedded answers (Cloze) question (The question text must include at least one embedded answer.)." error message and that 15 good cloze questions are imported. Verify that a correct question name is created for each 15 questions during the import process (there was none in the xml file).
      • Try any other sample of file containing a mix of bad and good multianswer questions that you can think of and report if you find any buggy validation.
      Show
      This issue try to fix the same problem in the 2 qformat plugins that can import multianswer (cloze) questions (other core qformat plugins can't import multianswer questions). The problem is that no validation is done after the question's text is decoded, so broken questions can be saved into the database and lead to big problems after that (particularly if the course containing these questions are backuped and restored). Verify that validation when creating multianswer questions using the UI is still working As this issue modify the code used to validate multianswer questions when using the Moodle UI it is necessary to verify that this is still working Login as a teacher Go to the question bank Create a new "Embedded answers (Cloze)" question Type something for the question name Type some text without any embedded subquestion for the question text. Click on the Save changes button and verify that you get the "The question text must include at least one embedded answer. " error message under the question text Include a multichoice subquestion with only one choice, (for instance {1:MULTICHOICE:=Apple#Correct}) in the question text. Click on Save choices button and verify that you get the "This type of question requires at least 2 choices" error message under the subquestion answer Include a numerical subquestion with an answer that is not a valid number (for instance {3:NUMERICAL:=two}) Click on the Save changes button and verify that you get the "The answer must be a number, for example -1.234 or 3e8, or '*'." error message under the subquestion answer Now enter a correct multianswer question text (you can find some sample in the documentation https://docs.moodle.org/36/en/Embedded_Answers_(Cloze)_question_type#Examples ) and verify that it is correctly saved when you click on the Save changes button I think that this is enough to verify that validation is working but feel free to try any other incorrect cloze syntax and report if you find anything incorrectly validated Verify that questions are validated when imported using the Embedded answers (Cloze) file format This step is somewhat tedious as this import format is only able to import one question at a time ! Login as a teacher Go to the question bank import page Choose the "Embedded answers (Cloze)" File format Open the General subsection and set "Stop on error" to "No" (This is optional, you can also test to import the same file leaving this setting to the default value. Choose the "broken_multianswer_1.txt" file attached to this issue Click on the import button Verify that the error message " Error importing question Invalid embedded answers (Cloze) question (This type of question requires at least 2 choices). " is displayed and that you also get "There are no questions in the import file ". this is normal because here the import format can only contains one question so if this question is invalid the file is considered empty. Redo the same process with the broken_multianswer_2.txt and broken_multianswer_3.txt files and verify that you get the " Error importing question Invalid embedded answers (Cloze) question (One of the answers should have a score of 100% so it is possible to get full marks for this question.). " and " Error importing question Invalid embedded answers (Cloze) question (The answer must be a number, for example -1.234 or 3e8, or ' '.).*" errors messages. Redo the same process with the "correct_multianswer.txt" file and verify that the question is correctly imported without any error message Try any other sample of file containing a bad multianswer question that you can think of and report if you find any buggy validation. Verify that questions are validated when imported using the Moodle XML file format Fortunately here when can import many questions in the same file so this is a lot faster. Login as a teacher Go to the question bank import Choose "Moodle XML format" File format Open the general subsection and set "Stop on error" to "No" Choose the "broken_cloze_questions.xml" file attached to this issue Click on the Import button Verify that you get 5 errors messages and that no questions are imported from the file Redo the same process with the "moodlexml_with_error.xml" file uploaded by Koen Roggemans and verify that you get one " Error importing question Invalid embedded answers (Cloze) question (The question text must include at least one embedded answer.). " error message and that 15 good cloze questions are imported. Verify that a correct question name is created for each 15 questions during the import process (there was none in the xml file). Try any other sample of file containing a mix of bad and good multianswer questions that you can think of and report if you find any buggy validation.
    • Affected Branches:
      MOODLE_34_STABLE, MOODLE_35_STABLE, MOODLE_36_STABLE
    • Fixed Branches:
      MOODLE_35_STABLE, MOODLE_36_STABLE
    • Pull 3.5 Branch:
    • Pull 3.6 Branch:
    • Pull Master Branch:

      Description

      Currently no validation is done in question/format/multianswer/format.php in the readquestions function of the qformat_multianswer class after calling qtype_multianswer_extract_question. As a consequence it is possible to create broken multianswer questions in the database.

      The same problem also exists in question/format/xml/format.php in the import_multianswer function after the call to qtype_multianswer_extract_question.

      I discovered this while working on MDL-60897 but here the problem is more severe as there is no validation at all while in MDL-60897 only the number of multichoice subquestion's answers was not validated properly.

        Attachments

        1. moodlexml_with_error.xml
          3 kB
        2. correct_multianswer.txt
          0.5 kB
        3. broken_multianswer_3.txt
          0.0 kB
        4. broken_multianswer_2.txt
          0.1 kB
        5. broken_multianswer_1.txt
          0.1 kB
        6. broken_cloze_questions.xml
          2 kB

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Fix Release Date:
                13/May/19

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 20 minutes
                20m