Moodle
  1. Moodle
  2. MDL-6231

Better syntax error messages during GIFT import

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Not a bug
    • Affects Version/s: 1.6.1
    • Fix Version/s: None
    • Component/s: Quiz
    • Labels:
      None
    • Environment:
      Linux
    • Rank:
      11642

      Description

      Creating or updating Quiz activity, importing quizzes from GIFT text file (~250 questions, mostly multiple choice);

      GIFT file has occasional typos;

      import chokes without explanation, just Cannot insert question

      I have several problems with this:

      First, it makes no sense that the ONLY place the question import function is available is inside the process for creating a specific quiz: the question bank is for all quizzes.

      Second, it is immensely frustrating that when the import fails as described, I am hurled out of the course back into the moodle lobby. To continue, I am forced to reselect the course category, reselect the course, and then reselect the quiz for editing (but dammit! I am NOT editing the quiz - I'm trying to import questions!!!)

      Third, (and worst) I have no idea why the import has choked in the first place, and so I'm reduced to editing the file, deleting that question, and trying again.

      p.s. sorry if this comes off as griping - I really do think the quiz module is wonderful ... and I can't use it unless I can import questions from file.

        Activity

        Hide
        Martin Dougiamas added a comment -

        From Howard Miller (howard.miller at udcf.gla.ac.uk) Thursday, 10 August 2006, 06:45 PM:

        Please supply the file that you are having problems with.

        You should not be thrown out of quiz - this may be some other bug.

        I'm not sure what you mean by the placing of the import function. It is NOT attached to any particular quiz instance - it is part of the question bank editing screens.

        From Brian Linard (blinard at cs.ucr.edu) Friday, 11 August 2006, 08:51 AM:

        Hi Howard

        I have attached two versions of the gift-formatted questions file:

        cs12_quiz.txt, which contains a whole bunch of non-UTF-8 characters, which cause import to choke in one way; and cs12_quiz_revised.txt, which is UTF-8 valid (I think), but causes import to choke in a different way (error message: Cannot insert question), and then dumps me out into the moodle lobby.

        My apologies for the reference to the placement of the import function itself: I hadn't realized that the Questions entry in the Admin block was for quiz imports ... I was accessing it by creating a new quiz - very dumb!!

        Also, is there any way of setting the question category in the GIFT format? I have category data embedded in the file, but it's not being recognized by the import.

        Thanks!!!

        From Howard Miller (howard.miller at udcf.gla.ac.uk) Friday, 11 August 2006, 05:10 PM:

        I am tempted to say RTFM... but I won't

        A couple of things - you can't have blank lines in the middle of a question. GIFT uses blank lines as a question delimiter. I don't think that this is clear in the instructions, so I will ammend that. If you need a blank line use n.

        Comment markers (//) must be in the first two spaces of the line.

        The encoding format must be utf-8. There was another complaint that bogus characters completely broke the import, but I have been unable to reproduce this. I suspect that string handling somewhere else has been fixed that has sorted this. If possible can you try the latest build of 1.6.1+ to see if things have improved.

        There is no provision for specifying the category in the GIFT file. This has been asked for before, but never implemented - I recall that we couldn't think of an intuitive way to do it.

        However, I do take the point that in some cases the error reporting could be better, but it is difficult when the imput has lots of syntax errors. I will change this to a feature suggestion and will (seriously) consider making some improvements were possible.

        Thank you for your report - no, not griping at all :-D

        From Howard Miller (howard.miller at udcf.gla.ac.uk) Friday, 11 August 2006, 05:11 PM:

        bug tracker ate my characters ... blank line \n

        From Brian Linard (blinard at cs.ucr.edu) Saturday, 12 August 2006, 09:24 AM:

        Hi Howard

        thanks for not giving in to temptation!!!

        And thanks for the clarifications - I should be able to clean it all up properly now.

        From Brian Linard (blinard at cs.ucr.edu) Tuesday, 15 August 2006, 12:39 AM:

        ok, I understand the process better now, so I hope I can make a couple of sensible suggestions:

        • the ungraceful behavior of the import function when it chokes really is a bug (when you hit Continue, it dumps you back in the moodle lobby rather than returning you to Edit question)
        • it would be very nice to have at least a clue as to the reason for choking
        • the GIFT format really needs to handle categories ... it may be hard, but it must be doable (you can handle names, why not categories???)
        • there is no way to export the entire question bank - you can only export one category at a time; likewise, the import imposes the current category on all imported questions.

        Thanks for the \n newline tip! Now I can set up multi-line answers (which I use a lot) as imports. Unfortunately, there does not seem to be any way of doing this in the question create interface ... why not?

        Finally, one major drawback:

        • under some conditions, the import has a limit of 250 characters in the question (I had to throw out about 100 of my best questions to get the import to work!!). This does not seem to be affected or to affect the answer list.

        The behavior is bizarre: if the question includes a name (::question name: it will be imported correctly, but if there is no name field, then the import chokes if the question xceeds 250 characters.

        From Brian Linard (blinard at cs.ucr.edu) Tuesday, 15 August 2006, 12:42 AM:

        stupid filter - it turned my perfectly good colon-close paren into a smiley ... that should have been ::question name:then:then)

        From Howard Miller (howard.miller at udcf.gla.ac.uk) Tuesday, 15 August 2006, 01:32 AM:

        You get used to this bug tracker Luckily it's on its way out....

        Some of this stuff is not entirely down to the import/export functionality, so I have added Tim Hunt to the CC list for any comments he may have.

        The whole unlocking import/export from categories thing requires a complete interface change. Worse.. I can't change all the import/export plugins. As you seem closer to this, can you explain how you would prefer it to look. I am aware that others may like it as it is. I am sympathetic though, and there have been some patches and suggestions made in the past. So maybe!

        I have been unable to reproduce your error behaviour - are you using the latest build? I take it you are seeing one of the 'red' error boxes? That should be exceptional.

        From Brian Linard (blinard at cs.ucr.edu) Tuesday, 15 August 2006, 04:00 AM:

        Hi Howard

        (that one was intentional!)

        Yes, I'm getting the red error box - and I've just realized that the ungracious behavior (getting dumped out into the moodle lobby, instead of being returned to your course) is a general property of the red error boxes and is not specific to the quiz imports.

        I don't know if that is its intended behavior, but I find it rather rude ... It would also be nice to provide slightly less cryptic error messages than Cannot insert question or cannot create new attempt (another error I just got trying to test a quiz). These just leave you scratching your head, without a cluse as to what action to take ...

        Regarding the 250 character limit:

        I am using v1.6.1 (not +), with postgreSQL

        I am attaching a file with a single long question in it, starting with the name ::some name::

        You'll see that it imports happily...

        Now remove the name field completely, and you get the error described.

        If you then cut the question down to <= 250 characters, it will import correctly even without the name field.

        BTW, the newline \n doesn't actually work for me - but <br> does, as does \n<br>, independently of whether I specify [html]

        Regarding export all categories:

        The ability to publish categories, making them available to all courses on the server, partly takes care of this concern: however, it would still be helpful to be able to back up my entire question bank in one hit. I was think of having an option added to the list of categories: All categories for this course.

        Of course, that would make it all the more useful to capture category in the GIFT format!

        From Tim Hunt (T.J.Hunt at open.ac.uk) Tuesday, 15 August 2006, 05:52 AM:

        To 'back up' an entire question bank - well, what about the course 'backup' facility.

        Actually, due to another MDL-6244, to do this, create a dummy quiz with at least one question from each category in the course, then backup the course, include that quiz in the backup on the backup options screen (and if you really only want the questions, turn off everything else.

        As for the 250 character thing. I think what is going on is that when the GIFT file does not supply a question name, we are using the questiontext for the name as well.

        Questiontext is unlimited, but question name is limited to 255 chars.

        So when making up a question name, we should run it through shorten_text from moodlelib.php.

        What with the number of gripes I have seen about UTF-8 in the forums, I think we probably need a function we call before opening a text file, which scans the file to see if it is valid UTF-8, and if not fails with an error message giving the exact location of the first invalid character, and some general advice about how to create a UTF-8 file. I'll post about this in the general developer forum. Or maybe the languages forum. It does not belong in this bug, which is now about better error reporting in GIFT import. I've changed the subject to reflect this.

        From Brian Linard (blinard at cs.ucr.edu) Tuesday, 15 August 2006, 09:26 AM:

        Thank you!

        I love rational explanations for bizarre phenomena!

        A name extractor utility would be excellent, but in the meantime I now know what to do with my GIFT files, so my original problem is fully solved.

        Regarding the quiz backup: I know there are workarounds, but they are always inelegant ... full question bank backup is really not terribly important, but I guess I just thought it was a no-brainer (add one more option ...) that would be nice to have - but I can certainly live without iy.

        The UTF-8 validator would be very helpful, even as a separate utility. Unfortunately, even though I found lots of UTF-8 enabled text editors, I can't find anything that does what you describe: checks a file & locates any non-valid characters. Do you know of any? (sorry if this question is now a bit off-topic!).

        From Howard Miller (howard.miller at udcf.gla.ac.uk) Tuesday, 15 August 2006, 01:30 PM:

        Thanks Tim - your character limit analysis was spot on - I'll fix that later on.

        I'll have a look around for utf-8 checkers. It can't be too hard to implement, surely? - I'm going to regret saying that I know

        From Howard Miller (howard.miller at udcf.gla.ac.uk) Tuesday, 15 August 2006, 10:02 PM:

        A quick dredge on the internet, found this regular expresssion:

        $field =~

        m/^(

        [

        Show
        Martin Dougiamas added a comment - From Howard Miller (howard.miller at udcf.gla.ac.uk) Thursday, 10 August 2006, 06:45 PM: Please supply the file that you are having problems with. You should not be thrown out of quiz - this may be some other bug. I'm not sure what you mean by the placing of the import function. It is NOT attached to any particular quiz instance - it is part of the question bank editing screens. From Brian Linard (blinard at cs.ucr.edu) Friday, 11 August 2006, 08:51 AM: Hi Howard I have attached two versions of the gift-formatted questions file: cs12_quiz.txt, which contains a whole bunch of non-UTF-8 characters, which cause import to choke in one way; and cs12_quiz_revised.txt, which is UTF-8 valid (I think), but causes import to choke in a different way (error message: Cannot insert question), and then dumps me out into the moodle lobby. My apologies for the reference to the placement of the import function itself: I hadn't realized that the Questions entry in the Admin block was for quiz imports ... I was accessing it by creating a new quiz - very dumb!! Also, is there any way of setting the question category in the GIFT format? I have category data embedded in the file, but it's not being recognized by the import. Thanks!!! From Howard Miller (howard.miller at udcf.gla.ac.uk) Friday, 11 August 2006, 05:10 PM: I am tempted to say RTFM... but I won't A couple of things - you can't have blank lines in the middle of a question. GIFT uses blank lines as a question delimiter. I don't think that this is clear in the instructions, so I will ammend that. If you need a blank line use n. Comment markers (//) must be in the first two spaces of the line. The encoding format must be utf-8. There was another complaint that bogus characters completely broke the import, but I have been unable to reproduce this. I suspect that string handling somewhere else has been fixed that has sorted this. If possible can you try the latest build of 1.6.1+ to see if things have improved. There is no provision for specifying the category in the GIFT file. This has been asked for before, but never implemented - I recall that we couldn't think of an intuitive way to do it. However, I do take the point that in some cases the error reporting could be better, but it is difficult when the imput has lots of syntax errors. I will change this to a feature suggestion and will (seriously) consider making some improvements were possible. Thank you for your report - no, not griping at all :-D From Howard Miller (howard.miller at udcf.gla.ac.uk) Friday, 11 August 2006, 05:11 PM: bug tracker ate my characters ... blank line \n From Brian Linard (blinard at cs.ucr.edu) Saturday, 12 August 2006, 09:24 AM: Hi Howard thanks for not giving in to temptation!!! And thanks for the clarifications - I should be able to clean it all up properly now. From Brian Linard (blinard at cs.ucr.edu) Tuesday, 15 August 2006, 12:39 AM: ok, I understand the process better now, so I hope I can make a couple of sensible suggestions: the ungraceful behavior of the import function when it chokes really is a bug (when you hit Continue, it dumps you back in the moodle lobby rather than returning you to Edit question) it would be very nice to have at least a clue as to the reason for choking the GIFT format really needs to handle categories ... it may be hard, but it must be doable (you can handle names, why not categories???) there is no way to export the entire question bank - you can only export one category at a time; likewise, the import imposes the current category on all imported questions. Thanks for the \n newline tip! Now I can set up multi-line answers (which I use a lot ) as imports. Unfortunately, there does not seem to be any way of doing this in the question create interface ... why not? Finally, one major drawback: under some conditions, the import has a limit of 250 characters in the question (I had to throw out about 100 of my best questions to get the import to work!!). This does not seem to be affected or to affect the answer list. The behavior is bizarre: if the question includes a name (::question name: it will be imported correctly, but if there is no name field, then the import chokes if the question xceeds 250 characters. From Brian Linard (blinard at cs.ucr.edu) Tuesday, 15 August 2006, 12:42 AM: stupid filter - it turned my perfectly good colon-close paren into a smiley ... that should have been ::question name:then:then) From Howard Miller (howard.miller at udcf.gla.ac.uk) Tuesday, 15 August 2006, 01:32 AM: You get used to this bug tracker Luckily it's on its way out.... Some of this stuff is not entirely down to the import/export functionality, so I have added Tim Hunt to the CC list for any comments he may have. The whole unlocking import/export from categories thing requires a complete interface change. Worse.. I can't change all the import/export plugins. As you seem closer to this, can you explain how you would prefer it to look. I am aware that others may like it as it is. I am sympathetic though, and there have been some patches and suggestions made in the past. So maybe! I have been unable to reproduce your error behaviour - are you using the latest build? I take it you are seeing one of the 'red' error boxes? That should be exceptional. From Brian Linard (blinard at cs.ucr.edu) Tuesday, 15 August 2006, 04:00 AM: Hi Howard (that one was intentional!) Yes, I'm getting the red error box - and I've just realized that the ungracious behavior (getting dumped out into the moodle lobby, instead of being returned to your course) is a general property of the red error boxes and is not specific to the quiz imports. I don't know if that is its intended behavior, but I find it rather rude ... It would also be nice to provide slightly less cryptic error messages than Cannot insert question or cannot create new attempt (another error I just got trying to test a quiz). These just leave you scratching your head, without a cluse as to what action to take ... Regarding the 250 character limit: I am using v1.6.1 (not +), with postgreSQL I am attaching a file with a single long question in it, starting with the name ::some name:: You'll see that it imports happily... Now remove the name field completely, and you get the error described. If you then cut the question down to <= 250 characters, it will import correctly even without the name field. BTW, the newline \n doesn't actually work for me - but <br> does, as does \n<br>, independently of whether I specify [html] Regarding export all categories: The ability to publish categories, making them available to all courses on the server, partly takes care of this concern: however, it would still be helpful to be able to back up my entire question bank in one hit. I was think of having an option added to the list of categories: All categories for this course. Of course, that would make it all the more useful to capture category in the GIFT format! From Tim Hunt (T.J.Hunt at open.ac.uk) Tuesday, 15 August 2006, 05:52 AM: To 'back up' an entire question bank - well, what about the course 'backup' facility. Actually, due to another MDL-6244 , to do this, create a dummy quiz with at least one question from each category in the course, then backup the course, include that quiz in the backup on the backup options screen (and if you really only want the questions, turn off everything else. As for the 250 character thing. I think what is going on is that when the GIFT file does not supply a question name, we are using the questiontext for the name as well. Questiontext is unlimited, but question name is limited to 255 chars. So when making up a question name, we should run it through shorten_text from moodlelib.php. What with the number of gripes I have seen about UTF-8 in the forums, I think we probably need a function we call before opening a text file, which scans the file to see if it is valid UTF-8, and if not fails with an error message giving the exact location of the first invalid character, and some general advice about how to create a UTF-8 file. I'll post about this in the general developer forum. Or maybe the languages forum. It does not belong in this bug, which is now about better error reporting in GIFT import. I've changed the subject to reflect this. From Brian Linard (blinard at cs.ucr.edu) Tuesday, 15 August 2006, 09:26 AM: Thank you! I love rational explanations for bizarre phenomena! A name extractor utility would be excellent, but in the meantime I now know what to do with my GIFT files, so my original problem is fully solved. Regarding the quiz backup: I know there are workarounds, but they are always inelegant ... full question bank backup is really not terribly important, but I guess I just thought it was a no-brainer (add one more option ...) that would be nice to have - but I can certainly live without iy. The UTF-8 validator would be very helpful, even as a separate utility. Unfortunately, even though I found lots of UTF-8 enabled text editors, I can't find anything that does what you describe: checks a file & locates any non-valid characters. Do you know of any? (sorry if this question is now a bit off-topic!). From Howard Miller (howard.miller at udcf.gla.ac.uk) Tuesday, 15 August 2006, 01:30 PM: Thanks Tim - your character limit analysis was spot on - I'll fix that later on. I'll have a look around for utf-8 checkers. It can't be too hard to implement, surely? - I'm going to regret saying that I know From Howard Miller (howard.miller at udcf.gla.ac.uk) Tuesday, 15 August 2006, 10:02 PM: A quick dredge on the internet, found this regular expresssion: $field =~ m/^( [
        Hide
        Timothy Takemoto added a comment -

        My bug / feature request for GIFT import (and output) of categories is here
        http://tracker.moodle.org/browse/MDL-4163

        Show
        Timothy Takemoto added a comment - My bug / feature request for GIFT import (and output) of categories is here http://tracker.moodle.org/browse/MDL-4163
        Hide
        Howard Miller added a comment -

        I'm going to close this as the multiple problems that were discussed in this report either were not bugs, have been reported elsewhere or have been fixed. If this is not the case for any items may I request that a new, specific, report be created.

        Thanks!

        Show
        Howard Miller added a comment - I'm going to close this as the multiple problems that were discussed in this report either were not bugs, have been reported elsewhere or have been fixed. If this is not the case for any items may I request that a new, specific, report be created. Thanks!

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: