Issue Details (XML | Word | Printable)

Key: MDL-12439
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Critical Critical
Assignee: Mathieu Petit-Clair
Reporter: Helen Foster
Votes: 0
Watchers: 4
Operations

Add/Edit UI Mockup to this issue
If you were logged in you would be able to see more operations.
Moodle

Templates not imported in database preset import

Created: 05/Dec/07 05:41 AM   Updated: 12/Apr/08 12:29 AM
Return to search
Component/s: Database activity module
Affects Version/s: 1.9
Fix Version/s: 1.9.1

File Attachments: 1. Text File MDL-12439_preset-import-fix.patch (0.6 kB)

Image Attachments:

1. import_error1.png
(244 kB)
Issue Links:
Relates
 

Database: Any
Participants: Eloy Lafuente (stronk7), Helen Foster, Martin Dougiamas, Mathieu Petit-Clair, Robert Allerstorfer, Stephen Bourget, Urs Hunkler and Yu Zhang
Security Level: None
QA Assignee: Eloy Lafuente (stronk7)
Resolved date: 12/Mar/08
Affected Branches: MOODLE_19_STABLE
Fixed Branches: MOODLE_19_STABLE


 Description  « Hide
See http://code.google.com/p/google-highly-open-participation-moodle/issues/detail?id=54 for a description of the issue plus database preset attachment.

 All   Comments   Change History   Version Control      Sort Order: Ascending order - Click to sort in descending order
Martin Dougiamas added a comment - 05/Dec/07 10:42 AM
Yu, can you take a look at this next? The whole preset code is dodgy, so any cleaning up and testing you can do in here would be great (and also help the GHOP students too)

Martin Dougiamas added a comment - 05/Dec/07 10:43 AM

Yu Zhang added a comment - 05/Dec/07 02:15 PM
Hi Helen, I put a patch in 1.9, there are a few bugs. Could you please test this on the latest 1.9? It works for me and if this works for you I will post on the GHOP page. Cheers, Yu

Helen Foster added a comment - 05/Dec/07 08:43 PM
Hi Yu,

Thanks for your patch. The database preset import works perfectly for me now

Does this need testing in Moodle 1.8 too before resolving the issue?


Yu Zhang added a comment - 06/Dec/07 08:49 AM
Hi Helen, I just patched up 1.8 as well, please test that too because I think it was broken too, as the code does not retain unmodified templates.

Cheers,

Yu


Urs Hunkler added a comment - 25/Feb/08 11:23 PM
I am sorry, but this issue needs to be reopend.

In the latest Moodle 1.8.4 the templates are not imported. I tried the same preset zip in 1.8.4 and in 1.9. In 1.9 all is ok, 1.8.4 does not import the templates, only the fields.


Mathieu Petit-Clair added a comment - 12/Mar/08 03:40 PM
This might have been fixed since the report. I can sucessfully import a preset, like the one given on http://moodle.org/mod/forum/discuss.php?d=84050#p371801 by Martin.

If this one works but another one doesn't, can you send it to me?


Robert Allerstorfer added a comment - 05/Apr/08 09:02 PM
Confirmed that importing (as well as exporting) a preview.zip still does NOT work in the latest Moodle 1.8.4+, which uses moodle/mod/data/preset.php v1.20.2.4 from Dec 6, 2007.

The content of all templates will be ignored, in both importing and exporting! Only the fields will be taken care of. you can try for example with the "official" preset for Moodle Buzz, posted by Martin at http://moodle.org/mod/forum/discuss.php?d=84050&parent=371801


Helen Foster added a comment - 05/Apr/08 09:16 PM
Reopening following Robert's comment.

Robert Allerstorfer added a comment - 05/Apr/08 09:54 PM
To make it more clear what works and what doesn't on 1.8.4+:

If you create a fresh new DB and then immediately import the Buzz preset.zip, all works fine so far. So testing it up to here is not enough.

Because: after you edit content from the "Add template" sub page of the "Templates" tab, save it (by clicking on "Save template") and then try to use "Save as preset" and "Use a preset" from the "Presets" tab, the export/import will start to ignore the modified and saved templates.


Robert Allerstorfer added a comment - 06/Apr/08 09:55 PM
All the problems with preset.zip observed in Moodle 1.8.4+ are gone with 1.9+, so a solution would be to just go with the 1.9 Moodle. It's probably not worth to struggle around with the database module's preset functionality in Moodle 1.8 too much, since it works fine in 1.9.

Robert Allerstorfer added a comment - 07/Apr/08 05:26 AM
I have now intensively tested the "preset.zip" import and export ability with Moodle 1.9+. Things that did not even work in 1.8.4+ work in 1.9, but unfortunately, there are still many bugs:

After creating the fields within the web interface, exporting the resulting "preset.zip" works fine. However, re-importing it does not work properly when the "preset.xml" file contains certain characters within the tags, such as a single-quote (' char). For example:

Does NOT work:
<field>
<type>text</type>
<name>Title</name>
<description>Item's title</description>
</field>

Does work:
<field>
<type>text</type>
<name>Title</name>
<description>Title of the item</description>
</field>

Next problem:
All option lines within a menu field will be merged into one single line at importing the "preset.xml" within the "preset.zip". For example:

<field>
<type>menu</type>
<name>Format</name>
<description>What format is this resource?</description>
<param1>html
doc
pdf
image
audio/video</param1>
</field>

will be converted to
<field>
<type>menu</type>
<name>Format</name>
<description>What format is this resource?</description>
<param1>htmldocpdfimageaudio/video</param1>
</field>


Eloy Lafuente (stronk7) added a comment - 07/Apr/08 07:09 AM
Tip for first issue: Quotes read from XML and importing failing = Missing addslashes() somewhere.

Tip for 2nd one: IMO, separing choices with linefeed is 100% incorrect. So I'd propose to:

a) Make code able to read linefeeds for backwards compatibility when importing menus
b) Export and import choices properly separated or (better) XML encapsulated within something like:
<field>
<type>menu</type>
<name>Format</name>
<description>What format is this resource?</description>
<param1>
<menu>
<value>html</value>
<value>doc</value>
<value>pdf</value>
<value>image</value>
<value>audio/video</value>
</menu>
</param1>
</field>


Robert Allerstorfer added a comment - 07/Apr/08 04:39 PM
Thanks for the tips, Eloy.Unfortunately, I need a quick fix for that to work now...

My workaround for the first bug was to rename all description text so that they no more contain any characters ' or ´

I still need a workaround for the second bug. I totally agree on your proposal - the current way of separating more options is anything but not XML In which source file should I look for a quick bugfix?


Robert Allerstorfer added a comment - 08/Apr/08 02:39 AM
I have now found and fixed the bug which prevented to import a "preset.xml" containing menu or multimenu field types. The problem was related to "moodle/ib/xmlize.php" which incorrectly documents that xmlize($data, $WHITE, $encoding) defaults to $WHITE=0 if not specified. In fact, it defaults to 1.

"moodle/mod/data/lib.php" uses just xmlize($data), so $WHITE should be set to zero, but in fact gets one, which strips the newlines.

My fix explicitly calls xmlize() with setting $WHITE to 0, which keeps the newlines


Eloy Lafuente (stronk7) added a comment - 08/Apr/08 09:20 AM
Wow, $WHITE=1 sets XML_OPTION_SKIP_WHITE parser option. And that option AFAIK only is used to ignore XML nodes which contents are, COMPLETELY whitespace chars (space, linefeeds, tabs).

But it should affect to the <param1> tag above at all. It isn't formed exclusively by whitespace chars. Has other contents. So the XML_OPTION_SKIP_WHITE won't make any difference.

I've done this quick test with the contents above, using both possibilities... and in both cases, the menu elements are linefeed-separated. Differences are only in whitespace-only contents.

<?php

include_once('config.php');
include_once('lib/xmlize.php');

$data='<field>
<type>menu</type>
<name>Format</name>
<description>What format is this resource?</description>
<param1>html
doc
pdf
image
audio/video</param1>
</field>';

traverse_xmlize(xmlize($data,0));
print_object($GLOBALS['traverse_array']);
$GLOBALS['traverse_array']="";
traverse_xmlize(xmlize($data,1));
print_object($GLOBALS['traverse_array']);
$GLOBALS['traverse_array']="";

?>

Uhm... sure it's an explanation for this (different linefeed/return chars?), but not sure if the $WHITE value is the correct one... although you say it worked for you... uhm... we need proper XML ASAP (export/import) although fixing linefeeeds and maintaining them for backwards compatibility (import only).

Ciao


Robert Allerstorfer added a comment - 08/Apr/08 04:24 PM
Yes this seems strange but nevertheless it works I spent hours in an in-deep code look -through to find out the step where the newlines got stripped. And this was $parsedxml = xmlize($presetxml) within "moodle/mod/data/lib.php". So I took a closer look at the xmlize() function which is defined in "moodle/ib/xmlize.php" and first found that is has two optional parameters - $WHITE and $encoding., but the documentation of the default $WHITE value is WRONG. So I just set the parameter to the value it should be according to the documentation, tried a preset.zip import once again and - bingo - worked for the first time with menu fields.

You could easily try it yourself within a latest 1.9+ Moodle installation: Create a new database activity within a course, then immediately go to the Presets tab and import "buzz-db-preset.zip" (provided by Martin at the link given above). After the import has been successfully, go to the Fields tab, click on the Field type icon next to the Field name "Format" and you will see only one option - "htmldocpdfimageaudio/video".

Then to the same again after patching "moodle/mod/data/lib.php". The result will then be correct - having several options
html
doc
pdf
image
audio/video

I think this little but effective patch does not harm, it just calls xmlize() with its defaults according to its documentation. It's a workaround that works now. So I don't see a reason to not commit the patch.


Stephen Bourget added a comment - 11/Apr/08 04:19 AM
I'm running into the same problem the Urs encountered where the database preset is not importing templates, but it is importing the fields. if I try to import the buzz preset referenced above (with debugging set to all) I get the following error:

Notice: Array to string conversion in C:\Program Files\Apache Group\Apache2\htdocs\Moodle\moodle19\lib\datalib.php on line 28
Incorrect integer value: '' for column 'notification' at row 1

It seems that the notification field requires an integer value and will not accept a Null value (like the one being passed). This causes the import to fail (but it actually displays a message saying that the preset was applied successfully)

I've attached a screenshot of the error.
I'm seeing this on Moodle 1.9+ (Build 20080410)

Environment:
Windows Server 2003
Apache 2.0.59
PHP 5.2.5
MySQL 5.0.51a


Stephen Bourget added a comment - 11/Apr/08 04:20 AM
Screenshot of Import error

Eloy Lafuente (stronk7) added a comment - 11/Apr/08 10:46 AM
Please, can we use different bugs to track all these errors. Sumarizing... we have:

1) MDL-14137: Fixed! Encoding problems, causing entities, ampersands and quotes to fail on import (due to a incorrect export).

2) MDL-9534: Not fixed! Problems importing menu options, needs apparently $WHITE=1 ASAP, and for 2.0 we should save those values with proper XML.

3) MDL-12439 (this). Two problems, AFAIK: The quotes one (that should be fixed now by MDL-14137), and the menu options that is being worked in MDL-9534. So I think we can safely close this as duplicate of the other ones.

4) MDL-12439 (this too). The problem reported by Sthepen. Could you please, create that as a new bug in the tracker? That way we'll be able to work on it without mixing different problems together. That would be great!

And that's all... am I missing anything? Do you agree on 1-4 ?

TIA and ciao


Robert Allerstorfer added a comment - 11/Apr/08 05:42 PM
Eloy, agreed in all 1-4. This bug can be closed and we should continue discussing the others at the appropriate places.

Eloy Lafuente (stronk7) added a comment - 12/Apr/08 12:29 AM
Definitively closing this:

Thanks everybody! Ciao