Details
Bug
Major
Description
After upgrade from 1.5.2+ to 1.6.1 and utf8-migration, essays writen before upgrade and therefore migrated do not get show to users (inc. admin). Turning on debug show a message:
Notice: unserialize(): Error at offset 909 of 1110 bytes in /var/www/moodle/mod/lesson/view.php on line 1672.
All pre-migration essays throw that error. New essays seem to work. In the db the attempts look to be fine. Not truncated or anything obvious. Essays are writen in Finnish, so there are a lot of non-ascii characters in them.
Seem fairly obvious that unserialize can handle "useranswer" strings that have gone through utf8-migration. However, the error must be rather subtle, because the "useranswer" strings do seem identical.
Here's the "useranswer" field of lesson_attempt table for the same attempt at the error message above.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
O:8:"stdClass":5:{s:4:"sent";i:1;s:6:"graded";i:1;s:5:"score";i:0;s:6:"answer";s:823:"Ensimmäinen askel kohti sähköistä arkistoa on viitearkisto. Siinä on sähköisessä muodossa vain viitetiedot ja varsinaiset dokumentit ovat paperilla. Tämä soveltuu parhaiten, jos kaikkien tietojen siirto sähköiseen muotoon vaatisi liikaa resursseja. Esim. kirjastojen hakukoneet ovat viitearkistoja.
Sähköinen tekstiarkisto sisältää dokumenttien tekstit sähköisessä muodossa. Kuvat ovat tallennettu erikseen kuvatietokannaksi. tämä sopii silloin, jos arkistoitavasta tiedosta suurin osa on tekstiä. esim. kirjanpidon aineistot ovat näitä tekstiarkistoja.
Sähköinen dokumenttiaineisto perustu kokonaisten dokumenttien käsittelyjärjestelmään. Dokumenttiarkistojen tieto tallennetaan yleensä kuvamuodossa. Tiedonhaku perustuu näytöltä luettavassa muodossa olevaan hakemistoon, joka sisältää jäsentelyn arkiston tiedoista.";s:8:"response";s:130:"Ok, hyvä vastaus!
Lisäksi olisit voinut vielä pohtia kunkin arkistotason soveltumista taloushallinnon aineistojen säilyttämiseen.";}
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Issue Links
| This issue blocks: | ||||
| MDL-6854 | Review UTF migration to check everything works properly... |
|
|
|
Activity
- All
- Comments
- History
- Activity
- Source
- Test Sessions
I'm not at all familiar with the migration process 
I'm guessing that some sort of procedure was used directly on the useranswer field for the LESSON_ESSAY question type before unserializing it. This would then break the serialized string if any of the values changed in length.
More info:
The useranswer field for LESSON_ESSAY is a serialized object with the following set:
sent = 1 or 0
graded = 1 or 0
score = a number
answer = text
response = text
I looked in the migrate2utf8 scripts in mod/lesson/db and found the only reference to useranswer was in migrate2utf8.xml at line 8, but I have no clue what it is doing
If someone could point me to where I would need to modify the migration2utf8 script to fix this problem, I'll be happy to tackle it. But right now I feel like I would have to dig around for hours just to figure out how it works.
Cheers,
Mark
I'm guessing that some sort of procedure was used directly on the useranswer field for the LESSON_ESSAY question type before unserializing it. This would then break the serialized string if any of the values changed in length.
More info:
The useranswer field for LESSON_ESSAY is a serialized object with the following set:
sent = 1 or 0
graded = 1 or 0
score = a number
answer = text
response = text
I looked in the migrate2utf8 scripts in mod/lesson/db and found the only reference to useranswer was in migrate2utf8.xml at line 8, but I have no clue what it is doing
If someone could point me to where I would need to modify the migration2utf8 script to fix this problem, I'll be happy to tackle it. But right now I feel like I would have to dig around for hours just to figure out how it works.
Cheers,
Mark Added Eloy as a Watcher so hopefully he will be emailed. Sorry Eloy if you do not handle utf8 migration bugs, but I'm sure you will know who would.
I think the problem with this one lies in the migrate2utf8.xml file near the top. It needs to call a php function for lesson_attempts table, but I have no clue how to do this. We need this function because the useranswer field for essay questions has a serialized object and needs to be unserialized, converted and then serialized for storage. Least this is what I think is going wrong 
Thanks for any help in resolving this issue.
Thanks for any help in resolving this issue. Aha!
1- So, the lesson_attempts->useranswer CAN contain:
- One normal string
- One serialised object
2- Oki. And when it contains one serialised object the structure of such object is ALWAYS:
- sent (number)
- graded (number)
- score (number)
- answer (text)
- response (text)
3- And the answer is written by the student.
4- And the response is writen by the teacher.
5- And this is the only one field in the lesson module using serialised storage.
Please, check and conffirm/fix all the statements above. Thats really important in order to be able to follow the behaviour below.
The basic idea is that we have to change current lesson_attempts->useranswer conversion from the current simple SQL definition (done, because we assumed it was always a simple text field) to one spesialised function to be stored in mod/lesson/migrate2utf8.php called migrate2utf8_lesson_attempts_useranswer()
This function should:
- Detect the nature of the useranswer field for the recordid passed
- If it's one plain string, just convert and return the converted field
- If it's one serialised object, unserialise, convert attributes (answer, response), serialise and return the converted field
(I've added Yu here too) Ciao 
- One normal string
- One serialised object
- sent (number)
- graded (number)
- score (number)
- answer (text)
- response (text)
Please, check and conffirm/fix all the statements above. Thats really important in order to be able to follow the behaviour below.
The basic idea is that we have to change current lesson_attempts->useranswer conversion from the current simple SQL definition (done, because we assumed it was always a simple text field) to one spesialised function to be stored in mod/lesson/migrate2utf8.php called migrate2utf8_lesson_attempts_useranswer()
This function should:
- Detect the nature of the useranswer field for the recordid passed
- If it's one plain string, just convert and return the converted field
- If it's one serialised object, unserialise, convert attributes (answer, response), serialise and return the converted field
Doh! Always forgetting details.
That specific issue/error is from https://moodle.tpu.fi/mod/lesson/view.php?id=21198&action=essaygrade&attemptid=83, a page which doesn't have a name, but is the "single essay view", accessible from Grade Essays tab (https://moodle.tpu.fi/mod/lesson/view.php?id=21198&action=essayview). Every essay is unserialized when viewing the Grade Essays tab itself and a couple of dozen error messages are show like this:
Notice: unserialize(): Error at offset 769 of 832 bytes in /var/www/moodle/mod/lesson/view.php on line 1603
In moodle/mod/lesson/view.php both the line 1603 and the line 1672 are:
$essayinfo = unserialize($essay->useranswer);
Default charset for db and table mdl_lesson_attempt is utf8.