Moodle
  1. Moodle
  2. MDL-29256

WYSIWYG editor disappeared after restored courses from 1.9.1 to 2.1

    Details

    • Database:
      Any
    • Testing Instructions:
      Hide

      Testing difficulty level: Easy, requires Moodle 1.9 available.

      1. In Moodle 1.9, create a course with the following resources having both Summary and Full text filled.
      1.1 Web page (via "Compose a web page")
      1.2 Text page (via "Compose a text page") with the format field set to "Moodle auto-format"
      1.2 Text page with the format field set to "HTML format"
      1.3 Text page with the format field set to "Plain text format"
      1.4 Text page with the format field set to "Markdown format"
      2. Make a ZIP backup of the course
      3. Restore the backup in 2.1 or 2.2 or 2.3dev
      4. Make sure you have TinyMCE enabled in your profile
      5. TEST: Go and try to re-edit all four resources. Make sure that
      4.1 TinyMCE is used to edit the page content of the legacy Web page resource
      4.2 TinyMCE is used to edit the page content of the legacy Text page resource with the format set to "HTML format"
      4.3 Good old textarea is used to edit the content of all other pages, with the format field displaying the correct value.

      Show
      Testing difficulty level: Easy, requires Moodle 1.9 available. 1. In Moodle 1.9, create a course with the following resources having both Summary and Full text filled. 1.1 Web page (via "Compose a web page") 1.2 Text page (via "Compose a text page") with the format field set to "Moodle auto-format" 1.2 Text page with the format field set to "HTML format" 1.3 Text page with the format field set to "Plain text format" 1.4 Text page with the format field set to "Markdown format" 2. Make a ZIP backup of the course 3. Restore the backup in 2.1 or 2.2 or 2.3dev 4. Make sure you have TinyMCE enabled in your profile 5. TEST: Go and try to re-edit all four resources. Make sure that 4.1 TinyMCE is used to edit the page content of the legacy Web page resource 4.2 TinyMCE is used to edit the page content of the legacy Text page resource with the format set to "HTML format" 4.3 Good old textarea is used to edit the content of all other pages, with the format field displaying the correct value.
    • Workaround:
      Hide

      Change the format option below the editor to HTML format. After saving, the HTML editor should appear.

      Show
      Change the format option below the editor to HTML format. After saving, the HTML editor should appear.
    • Affected Branches:
      MOODLE_21_STABLE, MOODLE_22_STABLE
    • Fixed Branches:
      MOODLE_21_STABLE, MOODLE_22_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-29256-resource-format
    • Rank:
      18798

      Description

      The WYSIWYG editor disappeared after restored the courses from 1.9 + to 2.1

        Issue Links

          Activity

          Hide
          Michael de Raadt added a comment -

          Hi, George.

          Could you please be more specific about the circumstances surrounding this issue?

          Perhaps you try to restore the backup on demo.moodle.net to see if the problem is with the system or with the backup itself.

          Show
          Michael de Raadt added a comment - Hi, George. Could you please be more specific about the circumstances surrounding this issue? Perhaps you try to restore the backup on demo.moodle.net to see if the problem is with the system or with the backup itself.
          Hide
          George Chen added a comment -

          Hello Michael,
          How to restore the backup on the demo.moodle.net site? I have registered as username with georchen. Please allow me to restore the course.

          Show
          George Chen added a comment - Hello Michael, How to restore the backup on the demo.moodle.net site? I have registered as username with georchen. Please allow me to restore the course.
          Hide
          George Chen added a comment -

          Michael,
          Please ignore the previous message. I can login as a demo teacher. I will let you know once the course is restored.

          Show
          George Chen added a comment - Michael, Please ignore the previous message. I can login as a demo teacher. I will let you know once the course is restored.
          Hide
          George Chen added a comment -

          Hi Michael,
          My client has upload the course at http://demo.moodle.net/course/view.php?id=625
          Please look into the WYSIWYG editor of this course and let me know, thanks.

          Show
          George Chen added a comment - Hi Michael, My client has upload the course at http://demo.moodle.net/course/view.php?id=625 Please look into the WYSIWYG editor of this course and let me know, thanks.
          Hide
          Michael de Raadt added a comment -

          Hi, George.

          Unfortunately this site resets every hour. Perhaps you could capture screenshots. We really need more to work with here.

          Show
          Michael de Raadt added a comment - Hi, George. Unfortunately this site resets every hour. Perhaps you could capture screenshots. We really need more to work with here.
          Hide
          George Chen added a comment -

          Hi Michael,
          I have get my client approved and created an account for your at
          http://sandbox.cteched.net/course/modedit.php?update=48&return=0
          here is your login information
          username:moodletest
          password:moodletest
          my client complained about the WYSIWYG editor not working on this.Please let me know if you have any questions

          Show
          George Chen added a comment - Hi Michael, I have get my client approved and created an account for your at http://sandbox.cteched.net/course/modedit.php?update=48&return=0 here is your login information username:moodletest password:moodletest my client complained about the WYSIWYG editor not working on this.Please let me know if you have any questions
          Hide
          George Chen added a comment -

          Hi Michael,
          Any update?

          Show
          George Chen added a comment - Hi Michael, Any update?
          Hide
          Michael de Raadt added a comment -

          Hi, George.

          The Moodle auto-format isn't supported in 2.x.

          To get the editor working, change the format option below the editor to HTML format, save the configuration, then edit the configuration again.

          Show
          Michael de Raadt added a comment - Hi, George. The Moodle auto-format isn't supported in 2.x. To get the editor working, change the format option below the editor to HTML format, save the configuration, then edit the configuration again.
          Hide
          Michael de Raadt added a comment -

          Hi, Eloy.

          Can we change the "Moodle auto-format" to "HTML format" in an upgrade/restore?

          Show
          Michael de Raadt added a comment - Hi, Eloy. Can we change the "Moodle auto-format" to "HTML format" in an upgrade/restore?
          Hide
          George Chen added a comment -

          That resolved the issues. Thank you for your helps, Michael.

          Show
          George Chen added a comment - That resolved the issues. Thank you for your helps, Michael.
          Hide
          Eloy Lafuente (stronk7) added a comment -

          I think that the cause for this is that, in 1.9 (and its backups) we didn't have all those xxxformat fields that are everywhere in 2.0.

          So, surely, in the conversion we are defaulting to FORMAT_MOODLE or to "nothing" that also leads to FORMAT_MOODLE.

          Perhaps we could try some sort of format detection in the conversion process, I've to check that with David.

          Surely the detectors won't be 100% accurate but I think that looking for some html tags we could get a good number of FORMAT_HTML detected. The ones not detected will need to be solved by hand (with the workaround commented above).

          Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - I think that the cause for this is that, in 1.9 (and its backups) we didn't have all those xxxformat fields that are everywhere in 2.0. So, surely, in the conversion we are defaulting to FORMAT_MOODLE or to "nothing" that also leads to FORMAT_MOODLE. Perhaps we could try some sort of format detection in the conversion process, I've to check that with David. Surely the detectors won't be 100% accurate but I think that looking for some html tags we could get a good number of FORMAT_HTML detected. The ones not detected will need to be solved by hand (with the workaround commented above). Ciao
          Hide
          David Mudrak added a comment -

          Firstly, regarding "The Moodle auto-format isn't supported in 2.x." - that is not true. Of course the "Moodle auto-format" is supported in 2.x, as well as "Plain text" and "Markdown" are. They just do not have wysiwyg intentionally as their users prefer plain text areas.

          Eloy is right, this sounds like a result of the format detection during the backup conversion. The 1.9->2.1 backup converter follows the same pattern as the upgrade code does: usually using something like

          if ($CFG->texteditors !== 'textarea') {
              $data['intro']       = text_to_html($data['intro'], false, false, true);
              $data['introformat'] = FORMAT_HTML;
          }
          

          So most of the places from 1.9 are supposed to be migrated into FORMAT_HTML if the TinyMCE was enabled during the migration / upgrade.

          George: is it that the HTML editor disappeared from all fields in the restored course? Or just from some of them?

          Show
          David Mudrak added a comment - Firstly, regarding "The Moodle auto-format isn't supported in 2.x." - that is not true. Of course the "Moodle auto-format" is supported in 2.x, as well as "Plain text" and "Markdown" are. They just do not have wysiwyg intentionally as their users prefer plain text areas. Eloy is right, this sounds like a result of the format detection during the backup conversion. The 1.9->2.1 backup converter follows the same pattern as the upgrade code does: usually using something like if ($CFG->texteditors !== 'textarea') { $data['intro'] = text_to_html($data['intro'], false , false , true ); $data['introformat'] = FORMAT_HTML; } So most of the places from 1.9 are supposed to be migrated into FORMAT_HTML if the TinyMCE was enabled during the migration / upgrade. George: is it that the HTML editor disappeared from all fields in the restored course? Or just from some of them?
          Hide
          Michael de Raadt added a comment -

          Thanks, David and Eloy.

          Show
          Michael de Raadt added a comment - Thanks, David and Eloy.
          Hide
          George Chen added a comment -

          David,According to my client's response, it happened from all fields in the resorted course.

          Show
          George Chen added a comment - David,According to my client's response, it happened from all fields in the resorted course.
          Hide
          David Mudrak added a comment -

          George, can you please send me that ZIP backup to david@moodle.com so that I can try to reproduce the problem and analyze the contents of the backup file. Of course I won't share the backup nor will provide it to 3rd parties.

          Show
          David Mudrak added a comment - George, can you please send me that ZIP backup to david@moodle.com so that I can try to reproduce the problem and analyze the contents of the backup file. Of course I won't share the backup nor will provide it to 3rd parties.
          Hide
          Craig Drayton added a comment -

          Hi all, any progress on this issue?

          Is there a way to change all "Moodle Auto-Format" to "HTML Format"? We have migrated hundreds of M1.9 courses and wish to avoid tutors having to manually change every page to HTML format.

          Show
          Craig Drayton added a comment - Hi all, any progress on this issue? Is there a way to change all "Moodle Auto-Format" to "HTML Format"? We have migrated hundreds of M1.9 courses and wish to avoid tutors having to manually change every page to HTML format.
          Hide
          Justin Litalien added a comment -

          We are in the process of migrating instructors over to Moodle 2.2 and this issue still persists (restoring from 1.9). I've found that this is regularly happening with Pages (like Craig mentioned). With hundreds of courses being moved over, we are going to have some big trouble with our faculty if they have to use this work around to get this squared away.

          David or Eloy, is there anything I can do to help move this forward? I'm not a developer but I can sure help with testing and providing more details on this.

          Thanks!

          Show
          Justin Litalien added a comment - We are in the process of migrating instructors over to Moodle 2.2 and this issue still persists (restoring from 1.9). I've found that this is regularly happening with Pages (like Craig mentioned). With hundreds of courses being moved over, we are going to have some big trouble with our faculty if they have to use this work around to get this squared away. David or Eloy, is there anything I can do to help move this forward? I'm not a developer but I can sure help with testing and providing more details on this. Thanks!
          Hide
          David Mudrak added a comment -

          I just tried to look at http://sandbox.cteched.net/course/modedit.php?update=48&return=0 (the link that George Chen added above on 09/Sep/11 4:47 PM) and I can see the TinyMCE editor displayed there normally.

          Can you send me some 1.9 course backup where this can be reproduced? If you don't want or can't share the course, send it to my email david@moodle.com (I won't provide it to the 3rd parties). When sending the source, please append the instructions to reproduce - ie what module (Page etc) does not work after the restore for you.

          Show
          David Mudrak added a comment - I just tried to look at http://sandbox.cteched.net/course/modedit.php?update=48&return=0 (the link that George Chen added above on 09/Sep/11 4:47 PM) and I can see the TinyMCE editor displayed there normally. Can you send me some 1.9 course backup where this can be reproduced? If you don't want or can't share the course, send it to my email david@moodle.com (I won't provide it to the 3rd parties). When sending the source, please append the instructions to reproduce - ie what module (Page etc) does not work after the restore for you.
          Hide
          George Chen added a comment -

          Hi David,

          I was not able to get the 1.9 course backup from my client.

          Show
          George Chen added a comment - Hi David, I was not able to get the 1.9 course backup from my client.
          Hide
          David Mudrak added a comment -

          Anybody else experiencing this problem who could provide a backup to reproduce?

          Show
          David Mudrak added a comment - Anybody else experiencing this problem who could provide a backup to reproduce?
          Hide
          Justin Litalien added a comment -

          David, I just sent an email your way...

          Show
          Justin Litalien added a comment - David, I just sent an email your way...
          Hide
          David Mudrak added a comment -

          Many thanks to Justin Litalien for the excellent steps to reproduce provided in email. I was finally able to reproduce the problem in restored 1.9 Resources converted to 2.x Page modules. And also I was able to track the problem pretty easily.

          It turns out that this is the buggy code in moodle1_mod_page_handler::process_legacy_resource() method:

          $page['contentformat'] = (int)$data['reference'];
          

          This code was copied from the page_20_migrate() function but applied incorrectly for all resource types. But it must be applied only for 1.9 resources of the type "Plain text page" (that stores the used format in the "reference" field). The resources of type "Web page" must force FORMAT_HTML but the casting code above assigns the format to a value 0, that is FORMAT_MOODLE.

          I'm working on the patch for this bug in resources. I'm wondering how this could be missed during the testing as it is pretty obvious problem...

          Please note I was not (and still am not) sure if this report was originally meant for resources only or for HTML editors everywhere. I believe the most of places that are trying to calculate the format field are correct. So if you experience missing HTML editors EVERYWHERE in the restored course, surely the problem will be elsewhere.

          Increasing severenity to Critical as this bug leads to data loss with no easy way of auto-repair.

          Show
          David Mudrak added a comment - Many thanks to Justin Litalien for the excellent steps to reproduce provided in email. I was finally able to reproduce the problem in restored 1.9 Resources converted to 2.x Page modules. And also I was able to track the problem pretty easily. It turns out that this is the buggy code in moodle1_mod_page_handler::process_legacy_resource() method: $page['contentformat'] = ( int )$data['reference']; This code was copied from the page_20_migrate() function but applied incorrectly for all resource types. But it must be applied only for 1.9 resources of the type "Plain text page" (that stores the used format in the "reference" field). The resources of type "Web page" must force FORMAT_HTML but the casting code above assigns the format to a value 0, that is FORMAT_MOODLE. I'm working on the patch for this bug in resources. I'm wondering how this could be missed during the testing as it is pretty obvious problem... Please note I was not (and still am not) sure if this report was originally meant for resources only or for HTML editors everywhere. I believe the most of places that are trying to calculate the format field are correct. So if you experience missing HTML editors EVERYWHERE in the restored course, surely the problem will be elsewhere. Increasing severenity to Critical as this bug leads to data loss with no easy way of auto-repair.
          Hide
          David Mudrak added a comment -

          I just submitted patches for 2.1, 2.2 and 2.3dev to be reviewed and hopefully integrated into this weekly builds.

          Unfortunately there is no way how we could fix already restored resources in a safe way via the upgrade step. However, those who have hundreds of resources with this issue, may want to consider the following SQL:

          UPDATE mdl_page SET contentformat = 1 WHERE content LIKE '%<%>%' AND contentformat = 1;
          

          Beware! You should use a set of SELECT statements to explore the data to be modified first. This SQL statements sets the format of all Page modules to HTML if the page content field contains characters used for HTML tags. If there is a page written in different format (like Plain text or Markdown), this statement will ruin its formatting. Even worse, it can introduce a security problem (imagine there was a Plain text page explaining how JavaScript and HTML can be used in some nasty way - by setting the format field to 1 you effectively execute the examples in the page visitors' browsers). There is no reliable way how to get to the original type and text format of the resource... Good luck.

          Show
          David Mudrak added a comment - I just submitted patches for 2.1, 2.2 and 2.3dev to be reviewed and hopefully integrated into this weekly builds. Unfortunately there is no way how we could fix already restored resources in a safe way via the upgrade step. However, those who have hundreds of resources with this issue, may want to consider the following SQL: UPDATE mdl_page SET contentformat = 1 WHERE content LIKE '%<%>%' AND contentformat = 1; Beware! You should use a set of SELECT statements to explore the data to be modified first. This SQL statements sets the format of all Page modules to HTML if the page content field contains characters used for HTML tags. If there is a page written in different format (like Plain text or Markdown), this statement will ruin its formatting. Even worse, it can introduce a security problem (imagine there was a Plain text page explaining how JavaScript and HTML can be used in some nasty way - by setting the format field to 1 you effectively execute the examples in the page visitors' browsers). There is no reliable way how to get to the original type and text format of the resource... Good luck.
          Hide
          David Mudrak added a comment -

          BTW I think I know how this was missed during the testing now. Testers just made sure that the restored pages are displayed correctly (and they are as "Moodle auto-format" displays HTML correctly) but they did not try to re-edit them.

          Show
          David Mudrak added a comment - BTW I think I know how this was missed during the testing now. Testers just made sure that the restored pages are displayed correctly (and they are as "Moodle auto-format" displays HTML correctly) but they did not try to re-edit them.
          Hide
          Justin Litalien added a comment -

          Great work David, thanks for solving this one! I look forward to seeing this go live. Our campus tips its collective hat your way...

          Show
          Justin Litalien added a comment - Great work David, thanks for solving this one! I look forward to seeing this go live. Our campus tips its collective hat your way...
          Hide
          Eloy Lafuente (stronk7) added a comment -

          The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.

          TIA and ciao

          Show
          Eloy Lafuente (stronk7) added a comment - The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week. TIA and ciao
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Integrated, thanks! (21, 22 & master)

          Show
          Eloy Lafuente (stronk7) added a comment - Integrated, thanks! (21, 22 & master)
          Hide
          Michael de Raadt added a comment -

          Test result: Success. Great work. It was good to see this one come to a resolution.

          Show
          Michael de Raadt added a comment - Test result: Success. Great work. It was good to see this one come to a resolution.
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Well,

          I wish I said it every time
          you do the things you do.
          You always lend a helping hand,
          and I'm filled with gratitude.

          You are strong and generous
          for each and everyone one of us.
          I am eternally grateful,
          I cannot say thanks enough.

          Sorry for the (un)cool bit above, lol. Closing this as fixed. Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - Well, I wish I said it every time you do the things you do. You always lend a helping hand, and I'm filled with gratitude. You are strong and generous for each and everyone one of us. I am eternally grateful, I cannot say thanks enough. Sorry for the (un)cool bit above, lol. Closing this as fixed. Ciao
          Hide
          Susan Mangan added a comment -

          We are just in the process of backing up 1.9 courses and restoring into 2.2.2 and have come across this problem in the Summary section of Assignments. Can this be reopened to address this?

          Show
          Susan Mangan added a comment - We are just in the process of backing up 1.9 courses and restoring into 2.2.2 and have come across this problem in the Summary section of Assignments. Can this be reopened to address this?
          Hide
          Helen Foster added a comment -

          Hi Susan,

          Rather than reopening this issue, please create a new issue for the summary section of assignments. If you could provide a sample backup file, this would help developers to reproduce the problem easily.

          Show
          Helen Foster added a comment - Hi Susan, Rather than reopening this issue, please create a new issue for the summary section of assignments. If you could provide a sample backup file, this would help developers to reproduce the problem easily.
          Hide
          Susan Mangan added a comment -

          OK, great will do. Thanks!

          Show
          Susan Mangan added a comment - OK, great will do. Thanks!

            People

            • Votes:
              7 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: