Non-core contributed modules

Jerome - Thanks for creating this issue for improving the WebCTImport tool. I have contacted Dan Stowell who origianally contributed the code to see if he is willing to maintain it. If so, I will setup things so that these issues get assigned to him. I appreciate your patience in the process. Of course, if you have more patches or suggestions please make them here in the tracker. Peace - Anthony

Jerome - I posted on the Moodle.org an invitation for someone to step forward and maintain the webctimport tool. I do not know if we will find any takers. I did hear back from Dan (my apologies for not commenting here sooner) and he is unable to continue maintaining the code. To follow the discussioni in the forums see: http://moodle.org/mod/forum/discuss.php?d=100879

Ashley - This issue is rightfully yours. I'm not sure why I did not assign it to you when we set you up with CVS access. Thanks for you help in keeping up with the WebCTImport tool. Peace - Anthony

moving to the Tool: WebCTImport component

Hi all,

I've been working a lot with the import tool in Moodle 1.9 and have made many bug fixes but haven't yet had the time to share the code. I've zipped up the latest code that I've been working with and attached it here. The code desperately needs to be cleaned up, and the import log is way too verbose, but at least this should be a working conversion tool for 1.9.

There's many bug fixes/enhancements - heres the ones I can think of:

• included patches to fix quiz creation, utf8 support, etc
• Added support for WebCT question sets (the previous code used to add all questions from the set, rather than using random questions). Note that questions in Moodle cannot exist within multiple categories, so the questions actually get duplicated into each category.
• Fixed up question images
• Added proper support for non-integer scores in webct quizzes (Moodle only supports integer scores, so if a question in webct has a 0.5 score, the script will multiply all scores by powers of 10 to make them integers)
• fixed scoring of multiple short answer (cloze) question types
• added some scripts for importing via command line rather than through the browser. also added some scripts for batch importing of multiple webct backups.

I need to properly document how to use the tool and also do a big code cleanup but for now I thought I'd share what I'm working with if anyone is intersted.

Feedback would be appreciated!

Thanks

Hi Ashley, with debug on there seems to be a few errors with easy fixes. I'm using a semi-recent version of 1.9.2stable- these errrors are on the index.php page, I've just copied the webctimport directory into my moodle webroot and opened it in the browser (haven't "followed" any instructions)

print_header() was sent a string as 3rd (WebCT import) parameter. This is deprecated in favour of an array built by build_navigation(). Please upgrade your code.

• line 45 of webctimport/index.php: call to print_header()

Warning: opendir(/home/dan/moodledata/private/incoming/) [function.opendir]: failed to open dir: No such file or directory in /home/dan/www/private/moodle-r2/webctimport/index.php on line 119

Warning: readdir(): supplied argument is not a valid Directory resource in /home/dan/www/private/moodle-r2/webctimport/index.php on line 121

Warning: closedir(): supplied argument is not a valid Directory resource in /home/dan/www/private/moodle-r2/webctimport/index.php on line 132

there also seems to be a bit of french in the interface:
Sélectionnez le type d'importation désirée pour ce cours
Régulière
Fichiers seulement
Tout dans le premier bloc

Translate.google.com
Select the desired type of import for the course
Regular
Files only
Everything in the first bloc

Sorry for the Full dump! - when trying an import - few errors - a lot of these appear multiple times - trying to just include them once though.

Notice: Undefined variable: echotrue in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 628
Notice: Undefined index: identifierref in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 630
Notice: Undefined index: isvisible in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 634
Notice: Undefined variable: echotrue in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 616
Notice: Undefined index: identifierref in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 804
Notice: Undefined index: identifierref in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 806
Notice: Undefined index: href in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 285
Notice: Undefined index: refobjtypes in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 735
Notice: Undefined index: reftranslations in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 736
Notice: Undefined index: DIS_9775739_R in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 735
Notice: Undefined index: DIS_9775739_R in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 736
Notice: Undefined index: SPG_9775779_R in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 806
Notice: Undefined property: stdClass::$feedbackid in /home/dan/www/private/moodle-r2/webctimport/wct_qti_lib.php on line 980
Notice: Undefined property: stdClass::$feedback in /home/dan/www/private/moodle-r2/webctimport/wct_qti_lib.php on line 1439
Notice: Undefined property: stdClass::$ffeedback in /home/dan/www/private/moodle-r2/webctimport/wct_qti_lib.php on line 1440

ERROR: null value in column "intro" violates not-null constraint
INSERT INTO mdl_forum ( ID, COURSE, TYPE, NAME, ASSESSED, ASSESSTIMESTART, ASSESSTIMEFINISH, FORCESUBSCRIBE, TIMEMODIFIED ) VALUES ( 4, 3, 'general', 'General', 0, 0, 0, 0, 1222228724 )

• line 1554 of lib/dmllib.php: call to debugging()
• line 53 of mod/forum/lib.php: call to insert_record()
• line 399 of webctimport/lib.php: call to forum_add_instance()
• line 330 of webctimport/parse-webct-xml.php: call to convertWebctDiscussion()
• line ? of unknownfile: call to wctparser_start_element()
• line 241 of webctimport/parse-webct-xml.php: call to xml_parse()
• line 132 of webctimport/parse-webct-xml.php: call to parseWctXmlFromFile()
• line 99 of webctimport/parse-webct-xml.php: call to loadWctFile()
• line 77 of webctimport/index.php: call to loadWctZipFile()

ERROR: null value in column "alltext" violates not-null constraint
INSERT INTO mdl_resource ( ID, COURSE, NAME, TYPE, REFERENCE, SUMMARY, POPUP, OPTIONS, TIMEMODIFIED ) VALUES ( 2, 3, '101reviews12008.pdf', 'file', '2008 Lectures/101reviews12008.pdf', '', 'resizable=1,scrollbars=1,directories=1,location=1,menubar=1,toolbar=1,status=1,width=620,height=450', 'frame', 1222228725 )

• line 1554 of lib/dmllib.php: call to debugging()
• line 412 of webctimport/parse-webct-xml.php: call to insert_record()
• line ? of unknownfile: call to wctparser_start_element()
• line 241 of webctimport/parse-webct-xml.php: call to xml_parse()
• line 132 of webctimport/parse-webct-xml.php: call to parseWctXmlFromFile()
• line 99 of webctimport/parse-webct-xml.php: call to loadWctFile()
• line 77 of webctimport/index.php: call to loadWctZipFile()

ERROR: null value in column "correctfeedback" violates not-null constraint
INSERT INTO mdl_question_multichoice ( ID, QUESTION, ANSWERS, SINGLE, SHUFFLEANSWERS ) VALUES ( 1, 1, '1,2,3,4,5', 1, 1 )

• line 1554 of lib/dmllib.php: call to debugging()
• line 1512 of webctimport/wct_qti_lib.php: call to insert_record()
• line 127 of webctimport/wct_qti_lib.php: call to addQuestion()
• line 548 of webctimport/parse-webct-xml.php: call to processWebctQDB()
• line ? of unknownfile: call to wctparser_start_element()
• line 241 of webctimport/parse-webct-xml.php: call to xml_parse()
• line 132 of webctimport/parse-webct-xml.php: call to parseWctXmlFromFile()
• line 99 of webctimport/parse-webct-xml.php: call to loadWctFile(

ERROR: invalid byte sequence for encoding "UTF8": 0xa0 HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".
INSERT INTO mdl_question ( ID, CATEGORY, PARENT, NAME, QUESTIONTEXT, QUESTIONTEXTFORMAT, IMAGE, GENERALFEEDBACK, PENALTY, QTYPE, LENGTH, STAMP ) VALUES ( 1284, 25, 0, 'moderate - ch11extra #3', 'Mexico and the members of OPEC produce crude oil. Realizing that it would be in their best interests to form an agreement on production goals, a meeting is arranged and an informal, verbal agreement is reached. If both Mexico and OPEC stick to the agreement OPEC will earn profits of $200 million and Mexico will earn profits of $100 million. If both Mexico and OPEC cheat then OPEC will earn $175 million and Mexico will earn $80 million. If only OPEC cheats, then OPEC earns $185 million and Mexico $60 million. If only Mexico cheats, then Mexico earns $110 million and OPEC $150 million.<br><br><br>?<IMG style="vertical-align: middle" SRC="Test-Questions/ch11extra/image040.png" ALT="Picture">?<br>?<br>To OPEC, the payoff to not cheating is either', 1, '', '', 0, 'multichoice', 1, 'localhost+080924035905+IiIBQg' )

• line 1554 of lib/dmllib.php: call to debugging()
• line 1472 of webctimport/wct_qti_lib.php: call to insert_record()
• line 127 of webctimport/wct_qti_lib.php: call to addQuestion()
• line 548 of webctimport/parse-webct-xml.php: call to processWebctQDB()
• line ? of unknownfile: call to wctparser_start_element()
• line 241 of webctimport/parse-webct-xml.php: call to xml_parse()
• line 132 of webctimport/parse-webct-xml.php: call to parseWctXmlFromFile()
• line 99 of webctimport/parse-webct-xml.php: call to loadWctFile()
• line 77 of webctimport/index.php: call to loadWctZipFile()

Notice: Undefined property: stdClass::$item in /home/dan/www/private/moodle-r2/webctimport/lib.php on line 1611
placeInstanceOntoCourse() debug info: Item is of type "resource", $instance=, $section=11. - This is the webct_add_section_item_recursive() function
ERROR: invalid input syntax for integer: ""
SELECT * FROM mdl_course_modules WHERE module = '15' AND instance = ''

• line 686 of lib/dmllib.php: call to debugging()
• line 481 of lib/dmllib.php: call to get_recordset_sql()
• line 421 of lib/dmllib.php: call to get_record_sql()
• line 696 of webctimport/lib.php: call to get_record()
• line 1684 of webctimport/lib.php: call to placeInstanceOntoCourse()
• line 933 of webctimport/lib.php: call to webct_add_section_item_recursive()
• line 767 of webctimport/parse-webct-xml.php: call to moveAllItemsToSections()
• line ? of unknownfile: call to wctparser_stop_element()

Would like to add resource[] $kid=SPG_9786220_R
Notice: Undefined index: SPG_9786220_R in /home/dan/www/private/moodle-r2/webctimport/lib.php on line 901
placeInstanceOntoCourse() debug info: Item is of type "resource", $instance=, $section=6. - This is the loop through Org Pages in moveAllItemsToSections. Currently org #41
ERROR: invalid input syntax for integer: ""
SELECT * FROM mdl_course_modules WHERE module = '15' AND instance = ''

• line 686 of lib/dmllib.php: call to debugging()
• line 481 of lib/dmllib.php: call to get_recordset_sql()
• line 421 of lib/dmllib.php: call to get_record_sql()
• line 696 of webctimport/lib.php: call to get_record()
• line 902 of webctimport/lib.php: call to placeInstanceOntoCourse()
• line 767 of webctimport/parse-webct-xml.php: call to moveAllItemsToSections()
• line ? of unknownfile: call to wctparser_stop_element()
• line 241 of webctimport/parse-webct-xml.php: call to xml_parse()
• line 132 of webctimport/parse-webct-xml.php: call to parseWctXmlFromFile()
• line 99 of webctimport/parse-webct-xml.php: call to loadWctFile()
• line 77 of webctimport/index.php: call to loadWctZipFile()

• line 241 of webctimport/parse-webct-xml.php: call to xml_parse()
• line 132 of webctimport/parse-webct-xml.php: call to parseWctXmlFromFile()
• line 99 of webctimport/parse-webct-xml.php: call to loadWctFile()
• line 77 of webctimport/index.php: call to loadWctZipFile()

stacks of these errors:
Warning: mkdir() [function.mkdir]: File exists in /home/dan/www/private/moodle-r2/webctimport/parse-webct-xml.php on line 189

Hi Dan,

Thanks for the feedback. I always run the tool with debug and display_errors turned on so I'm used to seeing those PHP warning notices and have become somewhat desensitised to them . I've just been focusing on getting the imported courses working. That would be great if you want to fix some of those warnings and clean up the UI to remove the french etc (the tool has had a lot of work done on it by some french canadians). I've been focusing mostly on the results of the import rather than UI issues, and I think that has also been the case with the other developers of the tool, so the interface needs a lot of work.

The SQL errors you're getting look to me like you're using postgres? Or possibly some other non-MySQL database? I tried the tool with postgres recently and had the same issues with not-null constraints. There needs to be some work done to fix that, however with MySQL you shouldn't get those errors.

I recommend not using the tool to import into a production database as it has potential to corrupt things. What I've been doing is having a separate Moodle instance, running on MySQL, which I import to. I then backup/restore the course onto a production server which can be running on postgres. The backup/restore process cleanses the course since it gets created via the proper Moodle functions, rather than having records inserted directly into the database.

Hi Ashley,

This page should also do some form of security checking too. - you don't even have to be logged in to use the page! - If you didn't want to create your own capabilities, I'd suggest only allowing people with the capability "moodle/course:create" to be able to use the page.

If we could get your latest code into contrib CVS, I might even be able to spend some time tidying some of it up myself.

thanks,

Dan

hah - posted that last one before seeing your post. - yeah, I'm using postgres, yeah - our client wants to use the tool in a test environment then use Moodle backup/restore to transfer from their test to prod environments1

I'm attaching the ReadMe.txt file from an older version - in hopes it can be edited and included or in case someone needs it.

Best,
calrk

Hi Ashley,

I've just committed your patch to contrib, and also committed another patch to tidy up a few things - seems to "work" in my install now, although there are still a lot of errors.

Dan

I'm flagging this as resolved - changes are now in contrib 1.9Stable and HEAD - thanks to the Netspot team for the major patch.

Now I'm confused, is the latest official version attached above, or should I grab it from moodle.org (http://download.moodle.org/download.php/tools/webctimport.zip) ?

Perhaps if the most up to date version is on moodle.org - can the version attached above be deleted?

A couple notes:

1) when I try to use the latest version from Moodle.org with 1.8, it just gives me a white screen. So it works with 1.9, but not 1.8. Isn't it supposed to be retroactive, too?

2) when I use it with 1.9, and I try to browse and upload the file from the browser, rather than FTPing the file, it says "zipfile is
/home/public_html/moodle/uploaddata/incoming/file : /home/public_html/moodle/uploaddata/incoming/
newcoursecode : test01
initialcrscode :
type : reguliere

File not found: file not found /home/public_html/moodle/uploaddata/incoming/

So one must upload via FTP first.

3) the ZIP file for webctimport does not contain nor set up the "incoming" directory within it, like the old versions did. Could this be related to #2?

Thanks,
clark

Hi Clark - just noticed your comments... There are a range of branches in CVS - the 19_STABLE and HEAD versions should be used for 1.9 and 2.0 respectively

if you want a version that works with 1.8 -use the 18_STABLE version - (although this version is 2 years old, and I haven't ported any of the recent changes back)

the link you've posted above is a download for the HEAD branch that is not backwards compatible with 1.8 (it works with 1.9 though as there aren't any differences between the 1.9 and HEAD branches in CVS.....yet......) - this will likely change if someone starts working on getting it work with all the newer changes in HEAD. (btw - that download is auto-generated from CVS every 24hours)

The file upload thing is something I noticed recently too - I fixed that a week or so ago and pushed the changes into CVS

I also made a few more changes today - we found issues with the courses not being assigned to a course category correctly, so now it creates it's own "webct imported courses" course category, and all courses get placed in that category. I've also added a link to the bottom of the page after import that allows you to go straight into the course.

I've started to move a lot of the debug output to be reliant on debug developer being enabled and display errors - a lot of the output is pretty useless for general teachers just importing their own courses - it still needs a lot more cleaning/tidying