Moodle

Dongsheng said he would probably be able to do this, so reassigning it with many thanks.

Things to do:

• fixed the location of exported file, it should be in user context
• fixed the import/export process page to display the pictures in question text
• fixed moodle xml format to support file api fully

That todo list looks just right.

Tim, I fixed the exported file link today: http://github.com/dongsheng/moodle/commit/f7be3a82960fdb5e1b10f04012c547242eae15ed

Before finishing moodle xml format, can we got it into cvs, so users can continue export questions.

Some thoughts on moodle xml format, I am thinking we should embed <image> <image_base64> tags in <quesitontext> <answer> <generalfeedback> <incorrectfeedback> <partiallycorrectfeedback> <correctfeedback> tags, the <image> tag in <question> will be used only for back compatible.

if we do, the new question output will be:

<question type="shortanswer">
  <name><text>who are you?</text></name>
  <questiontext format="moodle_auto_format">
    <text>describe yourself</text>
    <image>question.jpg</image>
    <image_base64>FFFFFFFF</image_base64>
    <image>question2.jpg</image>
    <image_base64>EEEEEEE</image_base64>
    <image>question3.jpg</image>
    <image_base64>DDDDDD</image_base64>
  </questiontext>
  <answer>
    <feedback>
      <text></text>
      <image></image>
    </feedback>
  </answer>
  <incorrectfeedback>
      <text>incorrectfeedback</text>
      <image>lal.jpg</image>
   </incorrectfeedback>
</question>

Does this make sense to you?

Just to make the example clearer, presumably it will be more like this:

<question type="shortanswer">
  <name><text>Compare images</text></name>
  <questiontext format="moodle_auto_format">
    <text>What are the biggest differences between these images <img src="@@PLUGINFILE@@/photo1.jpg" />  <img src="@@PLUGINFILE@@/photo2.jpg" /></text>
    <image>photo1.jpg</image>
    <image_base64>FFFFFFFF</image_base64>
    <image>photo2.jpg</image>
    <image_base64>EEEEEEE</image_base64>
  </questiontext>
  <answer>
    <text>*colour*</text>
    <feedback>
      <text>That's right, the biggest difference is that one of the images is mostly blue, and the other is mostly orange. They are complementary colours on the colour wheel: <img src="@@PLUGINFILE@@/colourwheel.jpg" /></text>
      <image>colourwheel.jpg</image>
      <image_base64>DDDDDD</image_base64>
    </feedback>
  </answer>
</question>

Also, it is possible that the files may not all be images. Would it be better to change the tags to <image> and <image_base64>. Ah! I suppose you have chose <image> to be backwards compatible.

Hmm. My feeling is that this change is not backwards compatible anyway. When an XML file contains old-style <image> and <image_base64> tags outside <questiontext> we need to continue to import files like that, but for the new file tags I think we should use:

<question type="shortanswer">
  <name><text>Compare images</text></name>
  <questiontext format="moodle_auto_format">
    <text>What are the biggest differences between these images <img src="@@PLUGINFILE@@/photo1.jpg" />  <img src="@@PLUGINFILE@@/photo2.jpg" /></text>
    <file name="photo1.jpg">FFFFFFFF</file>
    <file name="photo2.jpg">EEEEEEE</file>
  </questiontext>
  <answer>
    <text>*colour*</text>
    <feedback>
      <text>That's right, the biggest difference is that one of the images is mostly blue, and the other is mostly orange. They are complementary colours on the colour wheel: <img src="@@PLUGINFILE@@/colourwheel.jpg" /></text>
      <file name="colourwheel.jpg">DDDDDD</file>
    </feedback>
  </answer>
</question>

You could even make it

<file name="colourwheel.jpg" encoding="base64">DDDDDD</file>

if you thought it was better to be more self-documenting. (That would give us the option to support more efficient encoding schemes in the future.)

A few more points:

• I just stared a thread in the quiz forum http://moodle.org/mod/forum/discuss.php?d=155747 to alert more people to this change.

• I asked Petr to review the patch. In principle I am happy for that to go into CVS as soon as it is ready, and separate from the rest of the changes. However, I think Petr has some comments on the patch.

• Before we finally decide the XML format, we should have a look at what the XML for a file Moodle backup looks like. It is not really the same, because a backup is a zip file, with the files in the zip, and we need the file contents inline, base-64 encoded, but still, it would be good if the different XML formats were as similar as possible.

I do not like the use of draffiles for export at all, it is not intended for this.

The solution is to create a virtual filearea and instead use standard pluginfile.php, in the UI you would just create a link export link to pluginfile.php, the actual content would be created on the fly in the xxx_pluginfile() function.
The benefits are:

• that you may also expose the export link in the file_browser if you want to.
• no need to do a two phase export
• no need for temporary storage, if you have the data in string you may server it easily
• it is much easier to implement proper access control

One feature I would love to see in the MoodleXML format is for it to preserve categories that are empty. Currently the scheme to list category path names, followed by questions within that category:

<question type="category">
    <category>
        <text>$course$/ParentCategory/SubCategory/CategoryNumberOne</text>
    </category>
</question>

<question type="cloze">
    <name>
        <text>QuestionName</text>
    </name>
    <questiontext>
        <text>QuestionTextHere {1:MULTICHOICE:choiceA~choiceB~=choiceC~choiceD} moreTextHere.</text>
    </questiontext>
    <generalfeedback>
        <text></text>
    </generalfeedback>
    <shuffleanswers>0</shuffleanswers>
</question>

An inefficiency is that the categories are not always continuous, so that the first entry above (that gives the path for "CategoryNumberOne" may show up several times throughout the XML file. A more significant problem is that empty categories are not written at all. I have in the past created empty categories with the intent of "next term I'll add a unit on X, but I don't have time to write the questions now" only to find the categories missing from the newly-imported question bank the next term. It seems a more sensible way to write questions within categories would be like this:

<category name="ParentCategory">
    <category name="SubCategory">
        <category name="CategoryNumberOne">
            <question> ...questionOne...  </question>
            <question> ...questionTwo...  </question>
        </category>
    </category>
</category>

That should easily allow empty categories to be created, and should also help address the issue of categories being re-ordered upon import (see here http://moodle.org/mod/forum/discuss.php?d=147961). Apologies if this suggestion is naive, as I know next to nothing about coding PHP generally, nor the specific PHP used in Moodle's question import/export. But it just makes intuitive sense to me to utilize the hierarchical nature of XML to mirror the hierarchical nature of the question bank.

Backing up categories is a completely separate issue. Please do not pollute this bug report. It would be better to discuss this in the quiz forum in the first instance, and then, if necessary, file a separate tracker issue.

Petr,

I am not sure if I understand your comment, are you saying we should create a link like:
http://moodle.local/pluginfile.php/1/question/export/11/exportedfile.xml

and we call the exportprocess function inside question_pluginfile()?

if we do so, how we can see the preview of the question export?

Dongsheng: Yes, that is the export URL I was proposing. I do not understand why you need to preview raw xml, if it means you just want some confirmation of what gets exported then I do not see any problem with that.

The preview is not of the raw XML. The point is that as the XML is generated, which can take a long time if it is a big file, then we display something so that teachers can see what is going on.

I propose the following solution for exported question files:
1. Put them in a new area "Exported question" in side the "My personal files" area (user context) for the current user.
2. (Possibly, if users demand it) also put a copy inside the course context, also called "Exported questions".

I think that is simple to implement now, and quite easy for most users to work with.

Petr, how do you think of Tim's comment?

I do not agree with Tim, because it looks more like portfolio. I do not think we should create yet another export framework when we already have the portfolio api which is supposed to do exactly this.
IMO the pluginfile.php solution I described is good enough for most simple cases such as glossary export, book export and even gradebook.

I may not understand question specific problems, but it seems like the question size is generally a few kilobytes only no matter what format you produce - why should it take a long time?

We are not creating yet another export format. Question export has been a feature for years, and it has always worked this way.

I agree that, in the future, it would make sense to change this from being a separate feature, to being part of the portfolio API. However, I do not think it is sensible to do that refactoring before the 2.0 release.

As I proposed above, I think we should do something simple to make this work in Moodle 2.0, and then file a separate issue (assigned to me) to rewrite this in the future.

Yes, I agree it is too late to significantly refactor anything in 2.0.

Another patch to store file in user/private area.

Petr, are you happy with storing files in user context for now, and we improve the export process later on?

you might have problems with overriding in user area, how are you going to solve that? How are you going to localize the export directory name? Oh, you are storing it directly into private files root, hmm, not nice.

Developer chat about this issue: http://moodle.org/mod/cvsadmin/view.php?conversationid=5503#c219597

Hi, Helen, thanks for the link

What other file areas are there in the private files area? (Sorry, I don't know how it works.) Can't we make in Exported questions file area in there?

There are no "subareas" it is just one "Private files" area for each user that should be fully controlled by user. If you start creating folders there you have to deal with file name collisions, renames, translation issues, etc.

Donsheng just reported during the meeting that there will be XML created on the fly, embedding attached images, too. Can we please prepare a general wrapper for this functionality so that it can be used for exporting in other modules, too? I will need something very similar for workshop soon as the assessment form definition (containing images) should be exportable. Glossary would benefit from it, too.

Hi David
sorry for delay.

Can you please provide more specific info about what you need? Because question xml format may different from workshop's xml format.

Tim, can you please review the patch

Hi Dongsheng,
Your last patch include calculated question types where the fomat code need some review.
This was not done as the export code was not working.
At first sight, things seem OK.
However where can I find your new code so that I can test it on "real calculated" questions?

For calculatedmulti which is a multichoice type the {x..} wild cards are included in formula like {={x..}+.._} so the result is what appears to the user.
As for regular multichoice images can exist in the answer.

Hi, Pierre

The new export/import code assumes all files are stored in moodle 2.0 file pool, images in html will be <IMG src="%%PLUGINFILE%%" alt="image030" align="bottom" border=0>, we don't need to search question/feedback text to find them.
About the legacy file url, should we fix all links in question text in upgrade.php, this will make everything easier.

Tim,
This question is for you
"About the legacy file url, should we fix all links in question text in upgrade.php, this will make everything easier."

Dongsheng, sorry I am having a bad week. I have not managed to review your patch yet. I will try again tomorrow.

Will you have any time to work on the upgrade code? Thanks

sorry, I attached wrong file here, please see my github branch: http://github.com/dongsheng/moodle/tree/questionimport

Hi, Tim

I am not sure how question files worked in 1.9, when upgrading 2.0, I already moved file to questiontext area, and appended correct file url to question text.
in lib/db/upgrade.php line 5011, I don't think the other text have files.

Petr can you review this so we can get it in?

1/ The messaging related changes are not supposed to be part of this, right?
2/ question_pluginfile() should do full access control checks, I can see only require_login() there - it might be hidden somewhere at lower level, but I think that would not be optimal anyway
3/ + $importfile = "{$CFG->dataroot}/{$COURSE->id}/{$realfilename}"; still in question import - abuses COURSE and all temporary files must go to /temp anyway

Hi, Petr

1. what messaging related changes are you talking about? The patch only affects files under question/ and lib/questionlib.php
2. We talked about access control checks at http://tracker.moodle.org/browse/MDL-16094?focusedCommentId=89708&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_89708
is there any missing?
3. fixed in new patch

Cloze question code need fixing to be able to work with 1,9 export files.
I will do it monday .

closing this issue, the remaining export problems will be fixed in MDL-25086, thanks for Pierre's help.

While I am grateful to you for doing this Dongsheng, , I am currently a bit annoyed because I have had to fix a lot of minor bugs in the code today