Issue Details (XML | Word | Printable)

Key: MDL-6336
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Yu Zhang
Reporter: Jussi Hannunen
Votes: 0
Watchers: 3
Operations

Add/Edit UI Mockup to this issue
If you were logged in you would be able to see more operations.
Moodle

After utf8-migration unserializing of old essay fails

Created: 23/Aug/06 11:49 PM   Updated: 06/Oct/06 11:09 PM
Component/s: Lesson, Unicode
Affects Version/s: 1.6.1
Fix Version/s: 1.6.3

Environment: MySQL 5.0.22, PHP 4.3.10, Apache 1.3.33, Debian Linux
Issue Links:
Blockers
 

Database: MySQL
Participants: Eloy Lafuente (stronk7), Jussi Hannunen, Mark Nielsen and Yu Zhang
Security Level: None
Resolved date: 06/Oct/06
Affected Branches: MOODLE_16_STABLE
Fixed Branches: MOODLE_16_STABLE


 Description  « Hide
After upgrade from 1.5.2+ to 1.6.1 and utf8-migration, essays writen before upgrade and therefore migrated do not get show to users (inc. admin). Turning on debug show a message:
Notice: unserialize(): Error at offset 909 of 1110 bytes in /var/www/moodle/mod/lesson/view.php on line 1672.

All pre-migration essays throw that error. New essays seem to work. In the db the attempts look to be fine. Not truncated or anything obvious. Essays are writen in Finnish, so there are a lot of non-ascii characters in them.

Seem fairly obvious that unserialize can handle "useranswer" strings that have gone through utf8-migration. However, the error must be rather subtle, because the "useranswer" strings do seem identical.

Here's the "useranswer" field of lesson_attempt table for the same attempt at the error message above.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
O:8:"stdClass":5:{s:4:"sent";i:1;s:6:"graded";i:1;s:5:"score";i:0;s:6:"answer";s:823:"Ensimmäinen askel kohti sähköistä arkistoa on viitearkisto. Siinä on sähköisessä muodossa vain viitetiedot ja varsinaiset dokumentit ovat paperilla. Tämä soveltuu parhaiten, jos kaikkien tietojen siirto sähköiseen muotoon vaatisi liikaa resursseja. Esim. kirjastojen hakukoneet ovat viitearkistoja. Sähköinen tekstiarkisto sisältää dokumenttien tekstit sähköisessä muodossa. Kuvat ovat tallennettu erikseen kuvatietokannaksi. tämä sopii silloin, jos arkistoitavasta tiedosta suurin osa on tekstiä. esim. kirjanpidon aineistot ovat näitä tekstiarkistoja. Sähköinen dokumenttiaineisto perustu kokonaisten dokumenttien käsittelyjärjestelmään. Dokumenttiarkistojen tieto tallennetaan yleensä kuvamuodossa. Tiedonhaku perustuu näytöltä luettavassa muodossa olevaan hakemistoon, joka sisältää jäsentelyn arkiston tiedoista.";s:8:"response";s:130:"Ok, hyvä vastaus! Lisäksi olisit voinut vielä pohtia kunkin arkistotason soveltumista taloushallinnon aineistojen säilyttämiseen.";}
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------



 All   Comments   Change History   Version Control      Sort Order: Ascending order - Click to sort in descending order
Jussi Hannunen added a comment - 24/Aug/06 12:07 AM
Doh! Always forgetting details.

That specific issue/error is from https://moodle.tpu.fi/mod/lesson/view.php?id=21198&action=essaygrade&attemptid=83, a page which doesn't have a name, but is the "single essay view", accessible from Grade Essays tab (https://moodle.tpu.fi/mod/lesson/view.php?id=21198&action=essayview). Every essay is unserialized when viewing the Grade Essays tab itself and a couple of dozen error messages are show like this:
Notice: unserialize(): Error at offset 769 of 832 bytes in /var/www/moodle/mod/lesson/view.php on line 1603

In moodle/mod/lesson/view.php both the line 1603 and the line 1672 are:
$essayinfo = unserialize($essay->useranswer);

Default charset for db and table mdl_lesson_attempt is utf8.


Mark Nielsen added a comment - 26/Aug/06 01:48 AM
I'm not at all familiar with the migration process

I'm guessing that some sort of procedure was used directly on the useranswer field for the LESSON_ESSAY question type before unserializing it. This would then break the serialized string if any of the values changed in length.

More info:

The useranswer field for LESSON_ESSAY is a serialized object with the following set:

sent = 1 or 0
graded = 1 or 0
score = a number
answer = text
response = text

I looked in the migrate2utf8 scripts in mod/lesson/db and found the only reference to useranswer was in migrate2utf8.xml at line 8, but I have no clue what it is doing

If someone could point me to where I would need to modify the migration2utf8 script to fix this problem, I'll be happy to tackle it. But right now I feel like I would have to dig around for hours just to figure out how it works.

Cheers,
Mark


Mark Nielsen added a comment - 24/Sep/06 02:04 PM
Added Eloy as a Watcher so hopefully he will be emailed. Sorry Eloy if you do not handle utf8 migration bugs, but I'm sure you will know who would.

I think the problem with this one lies in the migrate2utf8.xml file near the top. It needs to call a php function for lesson_attempts table, but I have no clue how to do this. We need this function because the useranswer field for essay questions has a serialized object and needs to be unserialized, converted and then serialized for storage. Least this is what I think is going wrong

Thanks for any help in resolving this issue.


Mark Nielsen added a comment - 27/Sep/06 02:17 PM
Added Petr as a watcher. Just trying to get some help

Eloy Lafuente (stronk7) added a comment - 27/Sep/06 03:02 PM
Aha!

1- So, the lesson_attempts->useranswer CAN contain:

  • One normal string
  • One serialised object

2- Oki. And when it contains one serialised object the structure of such object is ALWAYS:

  • sent (number)
  • graded (number)
  • score (number)
  • answer (text)
  • response (text)

3- And the answer is written by the student.

4- And the response is writen by the teacher.

5- And this is the only one field in the lesson module using serialised storage.

Please, check and conffirm/fix all the statements above. Thats really important in order to be able to follow the behaviour below.

The basic idea is that we have to change current lesson_attempts->useranswer conversion from the current simple SQL definition (done, because we assumed it was always a simple text field) to one spesialised function to be stored in mod/lesson/migrate2utf8.php called migrate2utf8_lesson_attempts_useranswer()

This function should:

  • Detect the nature of the useranswer field for the recordid passed
  • If it's one plain string, just convert and return the converted field
  • If it's one serialised object, unserialise, convert attributes (answer, response), serialise and return the converted field

(I've added Yu here too) Ciao


Mark Nielsen added a comment - 27/Sep/06 03:13 PM
Hi Eloy,

Thanks for your comment!

All 1 through 4 are correct. As for other serialized objects, there is one in the lesson and lesson_default tables. The field in both tables is called conditions. Its object contains three settings: timespent, completed, and gradebetterthan. All three are number values.

Everything in your post sounds great as well. Let me know if I can help further.


Yu Zhang added a comment - 06/Oct/06 04:37 PM
Hi,

I made some changes in 1.6 and HEAD regarding this issue. Basically checking if unserialization succeeds or not, and perform the right action accordingly. Please let me know if there are still problems with this, as I do not have a large number of data to play with.

Are the other serialized objects giving you problems?

Yu


Eloy Lafuente (stronk7) added a comment - 06/Oct/06 10:48 PM
Just testing MDL-6854 I got this:

Processsing db table lesson_branch...

Success

Success

Success

Parse error: syntax error, unexpected T_OBJECT_OPERATOR in /mod/lesson/db/migrate2utf8.php on line 44

So, perhaps, it's broken now...reopening...


Eloy Lafuente (stronk7) added a comment - 06/Oct/06 11:09 PM
resolving it again. It was one simple typo, preventing migration to continue.