Non-core contributed modules

When importing HotPotatoes quizzes, problem with non-ascii characters in response.

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 1.9.11
  • Fix Version/s: None
  • Component/s: Module: HotPot
  • Labels:
    None
  • Affected Branches:
    MOODLE_19_STABLE

Description

When importing a question in the HotPotatoes format, non-ascii characters in the response part will not get converted, but passed on as html entities. Correct answer "prøve" will be "prøve" in the resulting quiz question.

This only applies to import of questionsto the question bank in the HotPotatoes format (.xml), and not to HotPotatoes exercises added as .htm files.

  1. test_unicode.jcl
    29/Jan/08 5:14 PM
    3 kB
    Tormod Aagaard
  1. editquestion_unicode.png
    1.17 MB
    30/Jan/08 9:27 PM
  2. test_unicode_ correct.png
    18 kB
    29/Jan/08 10:48 PM
  3. test_unicode_ screen.png
    6 kB
    29/Jan/08 10:47 PM
  4. unicode.jcl.jpg
    40 kB
    29/Jan/08 10:10 PM

Activity

Hide
Gordon Bateson added a comment -

Please could you upload the XML file of your HotPot quiz, so that I can investigate further.

If you don't want to make the quiz public, please send it to me via email:
gordon at-sign kanazawa hyphen kanazawa dot ac dot jp

thanks

Show
Gordon Bateson added a comment - Please could you upload the XML file of your HotPot quiz, so that I can investigate further. If you don't want to make the quiz public, please send it to me via email: gordon at-sign kanazawa hyphen kanazawa dot ac dot jp thanks
Hide
Tormod Aagaard added a comment -

A test question in cloze format.

Show
Tormod Aagaard added a comment - A test question in cloze format.
Hide
Gordon Bateson added a comment -

Please see the attached screen shot (unicode.jcl.jpg) of how your quiz appears after it has been imported into the Quiz module on my Moodle 1.9 site.

Everything looks OK to me, so perhaps I have not understood the problem.

Please could you show me a screen shot of the problem you are seeing.

thanks
Gordon

Show
Gordon Bateson added a comment - Please see the attached screen shot (unicode.jcl.jpg) of how your quiz appears after it has been imported into the Quiz module on my Moodle 1.9 site. Everything looks OK to me, so perhaps I have not understood the problem. Please could you show me a screen shot of the problem you are seeing. thanks Gordon
Hide
Tormod Aagaard added a comment -

How did you enter the answer? cut and paste from the web? Maybe you actually entered the html-entity? When I enter the correct answer, "Tromsø" with my norwegian keyboard, it is not accepted as the right answer.

See attached screen shots of how it appears in my system. The first image shows "Tromsø" as wrong answer, the second shows the right answer according to the system( "enter the correct answer" button).

Thank you for helping.

Show
Tormod Aagaard added a comment - How did you enter the answer? cut and paste from the web? Maybe you actually entered the html-entity? When I enter the correct answer, "Tromsø" with my norwegian keyboard, it is not accepted as the right answer. See attached screen shots of how it appears in my system. The first image shows "Tromsø" as wrong answer, the second shows the right answer according to the system( "enter the correct answer" button). Thank you for helping.
Hide
Tormod Aagaard added a comment -

Shows the right answer according to the system.

Show
Tormod Aagaard added a comment - Shows the right answer according to the system.
Hide
Gordon Bateson added a comment -

> How did you enter the answer? cut and paste ... ?

Yes. I cut and paste the "Tromsø" prompt on the quiz page.
Unfortunately, I don't know how to enter Norwegian on my keyboard

> When I enter the correct answer, "Tromsø" with my norwegian keyboard,
> it is not accepted as the right answer.

I don't know why that would be.

I further investigated the HotPot import script

I added the following line at line 198:
print htmlspecialchars($wrapped->questiontext);

This produced the following output on the screen:

{1:SHORTANSWER:=Tromsø}

Therefore, I am pretty confident that the HotPot import script is doing what it is supposed to do. i.e. it is passing back the "ø" as a unicode character not an HTML entity.

I will try to find out how to enter Norwegian so I can replicate what you did, but at the moment I do not think this problem is caused by the HotPot import script.

Please could you give details of the OS and browser you are using on your client machine.

thanks
Gordon

Show
Gordon Bateson added a comment - > How did you enter the answer? cut and paste ... ? Yes. I cut and paste the "Tromsø" prompt on the quiz page. Unfortunately, I don't know how to enter Norwegian on my keyboard > When I enter the correct answer, "Tromsø" with my norwegian keyboard, > it is not accepted as the right answer. I don't know why that would be. I further investigated the HotPot import script I added the following line at line 198: print htmlspecialchars($wrapped->questiontext); This produced the following output on the screen: {1:SHORTANSWER:=Tromsø} Therefore, I am pretty confident that the HotPot import script is doing what it is supposed to do. i.e. it is passing back the "ø" as a unicode character not an HTML entity. I will try to find out how to enter Norwegian so I can replicate what you did, but at the moment I do not think this problem is caused by the HotPot import script. Please could you give details of the OS and browser you are using on your client machine. thanks Gordon
Hide
Gordon Bateson added a comment -

I found a page telling me how to install the Norwegian input method on Windows XP
http://www.norwegianlanguage.info/resource/keyboard.html

Having installed Norwegian keyboard, the above page says I can input a "ø" by pressing the ";" key on my English keyboard.

I installed the Norwegian keyboard and then retried the imported version of your JQuiz on my Moodle 1.9 site. I entered "ø" usingthe ";" key on my keyboard, and I confirmed that the quiz was judged to be correct by Moodle.

In your initial report you said:

> non-ascii characters in the response part will not get converted, but passed on as html entities.
> Correct answer "prøve" will be "prøve" in the resulting quiz question.

Please can you tell me where you saw the HTML entitiy: prøve

thanks
Gordon

Show
Gordon Bateson added a comment - I found a page telling me how to install the Norwegian input method on Windows XP http://www.norwegianlanguage.info/resource/keyboard.html Having installed Norwegian keyboard, the above page says I can input a "ø" by pressing the ";" key on my English keyboard. I installed the Norwegian keyboard and then retried the imported version of your JQuiz on my Moodle 1.9 site. I entered "ø" usingthe ";" key on my keyboard, and I confirmed that the quiz was judged to be correct by Moodle. In your initial report you said: > non-ascii characters in the response part will not get converted, but passed on as html entities. > Correct answer "prøve" will be "prøve" in the resulting quiz question. Please can you tell me where you saw the HTML entitiy: prøve thanks Gordon
Hide
Tormod Aagaard added a comment -

Hi,

You are really working hard to solve this. Thank you!

I've tried both importing and entering "Tromsø" with opera, firefox and IE on Windows XP and firefox on linux.

I'va also tried importing the question to other installations.

It worked correctly on my test installation. 1.9. beta 3 on Windows XP. My site, however, is on Linux.

Could it be a server problem?

Importing to 1.8.2 ( on linux) gives me this message: "Warning: cannot yet handle MBCS in html_entity_decode()! in /home/2/n/nuie/www/question/format/hotpot/format.php(488) : regexp code on line 1"

I'm attaching a screen shot of the "edit question" window of the imported question.

Thank you.
Tormod

Show
Tormod Aagaard added a comment - Hi, You are really working hard to solve this. Thank you! I've tried both importing and entering "Tromsø" with opera, firefox and IE on Windows XP and firefox on linux. I'va also tried importing the question to other installations. It worked correctly on my test installation. 1.9. beta 3 on Windows XP. My site, however, is on Linux. Could it be a server problem? Importing to 1.8.2 ( on linux) gives me this message: "Warning: cannot yet handle MBCS in html_entity_decode()! in /home/2/n/nuie/www/question/format/hotpot/format.php(488) : regexp code on line 1" I'm attaching a screen shot of the "edit question" window of the imported question. Thank you. Tormod
Hide
Tormod Aagaard added a comment -

This is how the imported question looks when I open it for editing.

Show
Tormod Aagaard added a comment - This is how the imported question looks when I open it for editing.
Hide
Gordon Bateson added a comment -

> Could it be a server problem?

Yes, I think you have hit the nail on the head there.

a Google search for the error your reported, brings up the following page:
http://bugs.php.net/bug.php?id=25670

According to the above page this issue applies to PHP 4. It seems that in certain versions of PHP4, the "html_entity_decode" function cannot decode to UTF-8. This means when Moodle is running on certain PHP4 sites, the HTML entities will not be converted to UTF-8.

I am guessing your server uses PHP4 - would that be right?

Looking elsewhere in the Moodle codebase I see that there is already a function which seems to handle HTML entity to UTF-8 conversion. The function is in the following Moodle script:
lib/tcpdf/html_entity_decode_php4.php

Some time next week, I will change the HotPot import script so that it uses the above library and I will let you know when I have done this so that you can verify whether the fix works for you or not.

Show
Gordon Bateson added a comment - > Could it be a server problem? Yes, I think you have hit the nail on the head there. a Google search for the error your reported, brings up the following page: http://bugs.php.net/bug.php?id=25670 According to the above page this issue applies to PHP 4. It seems that in certain versions of PHP4, the "html_entity_decode" function cannot decode to UTF-8. This means when Moodle is running on certain PHP4 sites, the HTML entities will not be converted to UTF-8. I am guessing your server uses PHP4 - would that be right? Looking elsewhere in the Moodle codebase I see that there is already a function which seems to handle HTML entity to UTF-8 conversion. The function is in the following Moodle script: lib/tcpdf/html_entity_decode_php4.php Some time next week, I will change the HotPot import script so that it uses the above library and I will let you know when I have done this so that you can verify whether the fix works for you or not.
Hide
Gordon Bateson added a comment -

I have modified the HotPot import script in Moodle 1.6 -> 2.0, so that it does not use the html_entity_decode() function, which appears to be broken on PHP4.

Instead the HTML entities will be converted using hotpot_charcode_to_utf8() which is based on the lib/tcpdf/html_entity_decode_php4.php which is available from Moodle 1.7 onwards.

I have confirmed that the amended HotPot import script will correctly import your JCloze activity. Please could you check that the new script works on your Moodle site too, and let me know what happens.

regards
Gordon

Show
Gordon Bateson added a comment - I have modified the HotPot import script in Moodle 1.6 -> 2.0, so that it does not use the html_entity_decode() function, which appears to be broken on PHP4. Instead the HTML entities will be converted using hotpot_charcode_to_utf8() which is based on the lib/tcpdf/html_entity_decode_php4.php which is available from Moodle 1.7 onwards. I have confirmed that the amended HotPot import script will correctly import your JCloze activity. Please could you check that the new script works on your Moodle site too, and let me know what happens. regards Gordon
Hide
Tormod Aagaard added a comment -

Great work!

I'll tell our systems administrator to update the system, and then I'll let you know if the fix is working.

Show
Tormod Aagaard added a comment - Great work! I'll tell our systems administrator to update the system, and then I'll let you know if the fix is working.
Hide
Tormod Aagaard added a comment - - edited

It works!!!

Thank you so much!

Tormod

Show
Tormod Aagaard added a comment - - edited It works!!! Thank you so much! Tormod
Hide
Gordon Bateson added a comment -

øøøøøøøøøøøø
Great news!
øøøøøøøøøøøø

Thanks again for reporting the issue and making the test quiz and screen shots available. I will now close this issue.

all the best
Gordon

Show
Gordon Bateson added a comment - øøøøøøøøøøøø Great news! øøøøøøøøøøøø Thanks again for reporting the issue and making the test quiz and screen shots available. I will now close this issue. all the best Gordon
Hide
Gordon Bateson added a comment -

Fix is confirmed for Moodle 1.6 -> 2.0, so I am closing this issue.

Show
Gordon Bateson added a comment - Fix is confirmed for Moodle 1.6 -> 2.0, so I am closing this issue.
Hide
Martin Dougiamas added a comment -

Just fixing the versions

Show
Martin Dougiamas added a comment - Just fixing the versions

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: