Moodle

Just deleted some old scripts / tables and disabled restore / import.

Eloy, how's restore coming along? We just need a plan, and we can help finish the rest. Would a restore meeting help?

Eloy, can you please add all the other restore tasks here?

I cannot see quiz or groups referenced here as a sub-task. neither of those are backuping up at the moment.

nevermind about quiz, i found it in MDL-23479
thanks

This is a summary of the status of the whole thing:

• By activity: All them are implemented and tested but these two:
  • data activity: there are some problems in the module (file related mainly) being the cause for its backup being halted. Anyway, once everything is in place, it's a matter of a 4-5 hours to implement and test both backup & restore. Part of MDL-22151 and MDL-23479
  • hotpot activity: as far as it has been moved to contrib (lack of 2.0 compliant code), cannot be done.
  • quizz activity: this is the worst one. And needs deep analysis. Main problem is that current way to select candidate question categories can lead to lost categories (causing quizzes to be inconsistent) under some circumstances. Also, the backup format will have to change in a noticeable way from 1.9.x one. I think it's a matter of, at least 2 weeks of work to understand/implement the whole thing. Part of MDL-22151, MDL-23479 and MDL-22138 (the complex part).
  • workshop activity: little mention to it because there are a few things to add to it. MDL-19936. Not problematic at all. David will complete them without problems.

• By missing functionalities: There are a bunch of missing functionalities, some of them not critical at all like blogs, course logs, messages, course formats, local plugins... Each one has one assigned subtask in the meta bug. MDL-21432. Apart from those, some of them are more important and require implementation asap:
  • cohorts and new enrollments: MDL-22148. Right now, Petr did the enrollments backup and I did, exclusively, the "manual" enrolment restore. I think Petr is on it.
  • Restore security: MDL-23437 - I've it near finished and will commit in 2-3 days (after some testing). It is the part of restore controlling/enabling what the user can restore from a backup file and affects directly to the allowed selections in the UI. 100% important to have it implemented ASAP. Right now one teacher can restore practically everything.
  • activity completion/availability: Everything is included in backup (for each activity), but right now we haven't the corresponding implementation finished. I've some code here and will be thinking a bit on it next 14 days.
  • activity logs: they are included in backup, but not restored yet. Note logs is one of the heaviest operations on restore (because the need to recode zillions of ids here and there). I've something similar to the implementation performed for decoding links, but still need some love. Actively improving it here.
  • course completion: I know nothing about this. I tried some months ago when implementing backup, found some bad errors, reported them and haven't returned to it since then. It is MDL-22254 and, it is open if somebody wants to try it.
  • Scheduled backups: MDL-22184. Not implemented. Basically it's about to change current iterator to use new backup code, although there are various problems about to have that task in moodle's cron, to decide if we have another cron for them or no. In any case, it is open if someone wants to try it.
  • Export/import UI: we need that done. It's one "reduced" version of the backup UI and chaining one backup and one restore in the same request. Discussed up to some point with Sam, I've requested him about how to continue with that in MDL-22142
  • Gradebook: While activity grade items are already restored as part of each activity, we are missing the restore of gradebook categories/calculations when possible. MDL-23362 is about it and I'd suggest Andrew to work on it as far as he did backup.

Finally, there are another two subtasks, both related with output that need some discussion and I'd define them as "very important" although not blockers at all. They are MDL-22143 and MDL-22144 both related with outputting/logging information about the backup & restore progress. Right now the "progress" stages both in backup and restore do nothing visible. And that is a no-no situation, as far as browser timeouts can happen leaving the process in an inconsistent status, so we need to push info continuously to the browser. Easy implementable with the new task/steps approach, just require some output/js magic. Both to output and to logging subsystem. Somebody could take a look to them a make some proposal about what/how to output.

And that's all right now. All the enumerated subtasks in this summary must be considered critical (but the ones explicitly defined with lower priorities) so they should have one assignee ASAP in order to advance in parallel in as many as possible.

I'll be back on August 21th. I will try to read/reply to any question related with backup/restore, in the tracker or by email at least on daily basis. Feel free to ping me by email if necessary.

Ciao

Thanks for the update! This really helps me see where things are at in terms of 2.0.

Unfortunately, this also tells me that we will not be able to use Moodle 2.0 this year for our clients. School starts in a couple weeks and we need to have everything ready prior to that in order for the teachers to have time to get their courses ready. We've been working hard all summer to get our custom changes merged into the 2.0 code in hopes that we would begin the year with the new version.

We HEAVILY rely on backup/restore as a way to move courses from one instance of Moodle to another. Here are the things that we have to have working before we can roll 2.0 out to our clients:

• 1.9 -> 2.0 restore. We don't plan to upgrade all of our clients at once, so being able to move a course from a 1.9 install to a 2.0 install will be crucial.
• Reliable 2.0 backup/restore. Again, moving courses is a HUGE part of what we do.

With Eloy out for two weeks and, according to the issues laid out above, another few weeks worth of work left to do, it doesn't look like basic 2.0 backup/restore will be solid until mid-September at the absolute earliest. I'm guessing this is a blocker for a 2.0 release candidate, so you might want to revise your roadmap to show September or October as a possible release date. It might help other Moodle administrators like myself that have been hoping/praying that Moodle 2.0 would get done before the school year starts.

I'm truly not complaining here, I understand how big of an undertaking Moodle 2.0 is. I just wanted to let you know how important backup/restore is to us and probably a lot of other Moodle administrators.

I guess I'm also looking for a little validation that my decision to use 1.9 this year makes sense. If it appeared that everything would be stable-ish by mid September I might take the chance and go with 2.0, but that really doesn't appear to be the case.

Thanks for all your hard work!

Bryce Thornton
Data Services Director
eSchool Consultants

Thank you Eloy for the detailed update on the backup restore functionality for Moodle 2.0. It is so helpfully for us in the field who are closely following developments and planning testing deployment timelines.

Bryce. I total agree with your assessment. Basic 2.0 backup/restore solid by mid-September at the absolute earliest.

When trying to restore a course from MoodleCommons.org I got the error "error/missing_moodle_backup_xml_file".
From /var/log/httpd/error_log:

[Thu Aug 19 20:44:31 2010] [error] [client 172.16.249.152] Default exception handler: error/missing_moodle_backup_xml_file Debug: \n* line 119 of /backup/util/helper/backup_general_helper.class.php: backup_helper_exception thrown\n* line 172 of /backup/util/ui/restore_ui_stage.class.php: call to backup_general_helper::get_backup_information()\n* line 58 of /backup/restore.php: call to restore_ui_stage_confirm->display()\n, referer: http://moodle.ioc.mercyships.org/moodle/backup/restorefile.php?contextid=26

However, the upload did extract correctly into /usr/moodle_data/moodle/temp/backup/f81014f68bbbd0e5620269f98d6f08f1/ and shows
moodle.xml together with the sub-directories course_files and moddata.

Hope this is helpful!

Thanks everyone for your comments.

Regarding backup and restore from 1.9 to 2.0, as far as I know this has not yet been implemented. A workaround is to upgrade the complete 1.9 site to 2.0.

Hi Eloy,

In 1.9 there used to be an option called "Backup role assignments for these roles" with a tick box for each role. In 2.0 there is only one blanket option to "include user role assignments". So is there a way to just back up with Administrators and Teachers and not Students? This might be a loss of function from 1.9 to 2.0.

Ashley

Hi Ashey,

yes, Moodle 2.0 backup only has the option of backup all/none role_assignments. Not individual picking anymore. The rationale beyond this is that, under Moodle 2.0, they new enrollment plugins are highly coupled with the roles (assignments) stuff and, in order to keep consistency, the all/none approach was the decided one. Under restore you'll find the same all/none setting but also the list of roles to map is showed when some of them cannot be matched (like in 1.9).

So I'd say that, for sure, Moodle 2 won't have any option to pick roles in backup anymore. At least until we get all the enrollments dependencies addressed (specially when using "external" enrollments (ldap...) and once everything is under control in that area, we could improve a bit the restore process in order allow picking of roles there, but for sake of consistency I don't think it is a good option for backup, only for restore.

Ciao

When I reported in MDL-24855 that I was not able to restore a 1.9 course to a new 2.0 installation, I was told that it duplicated a previously reported bug.

When I asked for clarification, I was told to search for a bug before filling a report, but this bug was given as reference.

The only thing that suggests that it is attached to this bug, is a comment by Helen Foster above.

Can we attach my bug report as a blocker? Certainly 2.0 should not go into beta testing until we can have a way of testing authentic 1.9 courses in a 2.0 environment with real students. Given another bug that does not allow mySQL backup databases to be converted to 2.0 because of "Illegal mix of collations" and that the problem will not be fixed, there does not seem any sensible way to compare a 1.9 course with a 2.0 course for most users.

I would like to be able to help test 2.0 from a teacher/student perspective with real classes, but there seems no way for me to do that without going into hacker mode.

Gary - Sorry to hear you are having troubles. The recommended way to get a 1.9 site into 2.0 is to upgrade the site. In your case, that would mean duplicating the 1.9 site (database and file storage) and then upgrading the site to 2.0. You mention another issue about mySQL backup database having some illegal mix of collations. I am not sure what issue that is but I would be happy to work you through it should you need to go into "hacker" mode. As I understand it, the ability to go from 1.9 backup to 2.0 will be worked on for 2.1. Let me know how I can be of help to you so that you can begin comparing things. Peace - Anthony

Gary, you'll notice I've changed the linked issue on MDL-24855 to MDL-22414 rather than this one. Apologies for the mix-up.

(Sorry, if this isn't the right place for posting this issue. I was redirected here from http://moodle.org/mod/forum/discuss.php?d=156305 )

I have successfully made a backup of the Moodle Features Demo course (so the backup & restore should cover all Moodle features) from http://qa.moodle.net to transfer it to my local development machine.

Restoring went fine until the very last step (see screenshot).

Hi Frank,

the "trying to restore user xxxx will cause conflict" message happens when:

1) Your target site has one xxxx user. (xxxx being the username)
2) The backup being restored also has one xxxx user
3) After various comparisons, Moodle has detected that the xxxx user in 1) and the xxxx user in 2) aren't the same user.

If the 1) , 2) and 3) above happens, restore stops before performing any action as far as restoring that user will end associating "things" (create activities, posts, attempts... whatever...) to the xxxx user in the target site, when those "things" weren't performed by the same user at all.

Those checks and behavior were introduced in Moodle 1.9.x and continue being valid under 2.0. Usually the xxxx user in the messages uses to be the "admin" user (that exists in practically all Moodle installations).

Perhaps that behavior will start encouraging people to STOP doing "things" as admins in courses, so that user won't be included in backups at all.

Finally note that there are two possible workarounds to make both xxxx users in 1) and 2) to match (and avoid the conflict):

a) Modify the backup's users.xml file and make his email or firstaccess fields to match with the ones in target site.
b) Modify the target site and set the user email or firstaccess fields to match with the ones in backup's user.xml file.

That way - personally I prefer a), edit the users.xml file - the restore process will match both xxxx users and every "thing" in the backup file belonging to xxxx will be associated to the already existing xxxx user in the target site.

Ciao

PS: Please, create new issues for everything, keeping this META just to talk about the global evolution of the backup & restore subsystem... tia!

Hi Eloy,

Thanks for the explanation (and sorry for posting in the META queue). I'm only glad that it wasn't me who created that bad admin user on the demo site in the first place

adding Eloy as a watcher...

oops - wrong bug - sorry!

Eloy, any chance you could prioritise/triage any remainining subtasks for sprints please?

Hi All,

Just wonder when Groups backup / restore will be added to this list?

Regards,
Tim

Should backup destinations be added to this list?