Moodle

http://moodle.org/mod/forum/discuss.php?d=27333

As you can see from the above nearly three-year discussion, there is tremendous interest in bulk course creation. I was pursuaded of its importance by a presentation by A.T. Wyatt at the Oklahoma City Moodle Moot in June 2008. At Wyatt's university, a course is created for each teacher training student to serve as a preservice teaching porfolio. FYI Wyatt uses a script written by Mark Pearson and posted in http://moodle.org/mod/forum/discuss.php?d=30126. This script is no longer being maintained.

The Modules and Plugins entry http://moodle.org/mod/data/view.php?d=13&rid=395 ("Bulk Course and Category Creation Tool (Upload CSV)") which seems to be most popular is also no longer actively maintained by its author, Rory Alford, and zip files have once again begun to appear in the forum. Rory's tool can create categories (and subcategories) as well as courses.

Is there any way that this feature can be made standard? For example, Site administration -> Courses -> Upload courses.

I have to change the attribution to Jeff Church.
http://moodle.org/mod/forum/discuss.php?d=96622

Mark Pearson and I did work on this, and Mark has some good documentation on it. But Mark did not write the script.

But I think this is sorely needed in the standard instance. There are a lot of people who have the need to do bulk course creation (and deletions). Mike Churchward had some good ideas on this also; I visited with him about it last September.

I will 'third' this as an important issue to address and would support this being bumped from minor to major priority. Besides it's inherent value, having a bulk course upload script and procedure formalized and incorporated into the Moodle core would also be a first step towards formalizing integration with institutional ERP systems. I would like to see some discussion about whether the data format should be simplified and some definition of what the expected behaviour of the script should be (eg not duplicating courses already present). It seems to me that this process should not be an onerous task since the script is already there and working (though I have not tested it with 1.9).

The bulk course creation tool still works in 1.9. I was not aware that the script is no longer maintained. This is bad news for us as we require it's use. I really hope that this makes it into the default Moodle install. Why not? All the work is basicaly done, it just now needs to be maintained for future Moodle releases. This has been a major frustration for us in the Moodle project. Fairly complex features like Moodle Networking are developed, but simple things like batch course create/delete are not. This seems like such a basic feature....

I have tried to review the discussion in the forum from time to time on the bulk course add and delete admin scripts as there has been a good deal of confusion from time to time as to what's what.

As I understand the situation now, there is an upload utility in M&P that came via Jeff in M&P and a delete utility that was also posted by Jeff and then massaged by Chad posted to the forum, but not added to M&P, and both add and delete utilities now work in 1.8 and 1.9. As I understand it, these utilities have been functioning appropriately and I don;t think I have seen anything in the way of problems lately.

I created a tracker issue to add the delete utility as a new M&P page or as part of an add/delete utility M&P page pending addition of the utility to Moodle core.

The discussion still seems to get convoluted and I would like to see perhaps the following steps taken:

1) clearly document the source of the bulk add and delete course utilities that were offered by Jeff and then tweaked for 1.9

2) clearly identify both of the bulk course utilities in M&P, preferably packaging them together and offering them from the same M&P page

3) review same to determine what needs to be done to incorporate these utilities into core

4) incoporate them into core.

Marc - OK, I think I am ready to start making some progress on this. I think this is where we stand.

1) The course for the bulk course upload done by Jeff is available at http://moodle.org/file.php/5/moddata/forum/33/427997/bulk_course_tools.zip and the M&P entry which should have its download link updated is http://moodle.org/mod/data/view.php?d=13&rid=1536&filter=1. As for backup and delete, Moodlerooms has a course management block that has these functions available at http://development.moodlerooms.com/file.php/1/Moodlerooms_Releases/Course_Management_Block_for_1.9/coursemanagementblock19.zip and one provided by Robert Russo in CONTRIB-605 is available at http://download.moodle.org/download.php/plugins/blocks/backadel.zip. I've asked for Justin Hardin, Robert Russo and Ashley Gooding to help me understand the relationship (if any) between these two similar blocks. In speaking with Dan Marsden, I think the goal we are looking for is to have it as consistent as possible with other areas of import and functionality in Moodle so it would be integrated under Administration rather than a block.

2) Once we have identified the preferred backup and delete tool we can package them together into a single patch.

3) Once we have a patch we can then move forward on incorporating these tools into core.

Peace - Anthony

Should we change the affects version to 1.9? I do not anticipate doing this for 1.8 but perhaps we can backport it once we have it working.

Hi, yesterday (26-4-2009) we have just release a free tool for realizing bulk actions on moodle. This tool allows you realize bacth process for deleting courses, creating courses, assaign teachers to courses and deleting users.

We develop this applicarion as a result of our work Universidad Adolfo Ibañez. They needed a tool the perform the change of semester (delete oldo courses, create new courses, assaign teacher to the new courses and delete old users) and this application is the result. We tested duringa the past year deleting more than 700 courses each semester and creating over 800 courses each semester without problems.

This is our first release of this applicarion and we hope to continue improving it in time. Please take a look at it at www.moodleadminpro.com

Thanks
Sebastián Espinosa
MoodleAdmin Team

Here is a minor fix of uploadcourse.php that restores the ability to add multiple teachers to a course

http://moodle.org/mod/forum/discuss.php?d=91850#p572909

It seems that this has been dormant for a short time. I would really like to see the Bulk Course Upload tool for 1.9 (one version with all of the features of Rory Alford & Jeff Church's contributions) added to and maintained in the Modules and Plugins page. I am not sure if this capability is being added to 2.0 or not but nevertheless it needs to be maintained for users of 1.9 who are not going to make the jump to 2.0 for some time.

I have incorporated the following fixes into the tool and am willing to volunteer to maintain it in the Modules and Plugins area for 1.9 as best I can until it is added to the 1.9 core. I am not a true programmer but am capable of making adjustments as people on the forums contribute as well as at making small additions myself and am willing to do the legwork to keep this updated for 1.9.

Updates to the tool I originally obtained from Jeff Church here (http://moodle.org/mod/forum/discuss.php?d=91850)

1. Small changes to the text strings to make the explanation of the fields easier to understand and/or to update information as field possibilities have changed.

2. Added the ability to use the 'enrol' field in the CSV file to set the enrollment setting for courses.

3. Made adjustments explained here (http://moodle.org/mod/forum/discuss.php?d=91850#p572025) to enable the tool to enroll multiple teachers.

And will also update the code to remove the bug explained by Jeff Sherk here (http://moodle.org/mod/forum/discuss.php?d=91850#p405748) as well as some of the field 'expansion' fixes explained by Fernando here (http://moodle.org/mod/forum/discuss.php?d=27333#p405593).

Is there any way that we can move forward a little on this and get one entry in the Modules and Plugins area that points to a file instead of to a forum? Again, I'm willing to help out by maintaining this in the M & P area.

Jason

This version includes the fix to allow for multiple teacher enrollment and also the addition of the enrol field so that the course enrolment plugin can be set via the tool.

*Bump... Any news on if this functionality will be added/avaialble for Moodle 2.0? Currently evaluating Moodle 2.0 RC1 and sadly there appears no way to bulk create courses.

Greg,

Sorry, I'm not sure. I haven't experimented with 2.0 enough yet to even know that the capability doesn't exist. I'm a bit surprised that it wasn't added though....!

I'll try and work with it a little and see how far off this tool is in terms of being usable for 2.0. If it requires serious modification I'm afraid it will be beyond me but I'll see what I can do.

Anyone else know what the plan is for bulk creation of courses in 2.0 onward?

Jason

I'm adding 1.9 and 2.0 affected branches so that this does not fall through the cracks. I'm also adding the Fix Version of DEV backlog to hopefully get work on this moving forward. Peace - Anthony

This is going to be a key component in getting my organization from switching from BB to Moodle. I know that there was a patch in 1.9 backwards but there is nothing yet for 2.0 (which is what they would want to switch to). Unless someone can fix the old file to work in 2.0.

Anthony: any progress?
I notice there is some code in this thread: http://moodle.org/mod/forum/discuss.php?d=91850&parent=744931

-Derek

Oh how I wish......

I had a quick look at the script in the forums and unfortunately there's no way it can make core in the current condition. It's not even remotely following our coding guidelines or existing Moodle code. Just some quick examples:

direct access to MySQL at the top (not using our DB layer).
doesn't use core API to do the work
no string translations
no phpdocs

Is there some kind of spec on exactly what this function should do? Can somebody start one?

Just a quick note to mention this would be very useful for myself as well. If I can help in any way that would be great!

Elevating the priority on this so it is more likely to get some love post-2.1.

As someone who's moved over from being a Blackboard admin I'm missing this functionality badly.

Well thanks for the vote Peter, that makes 99.
-Derek

As Martin mentions above, some spec of the functionality would be nice to have before we start work on this. Let me suggest to follow "do one thing and do it well" approach instead of trying to combine course creation with enrolments etc via one CVS.

We seem to be stalled here on something that
1) a lot of users agree is an essential item of functionality
2) we had a running version of for 1.9
3) is fundamental to upscaling Moodle to large organisations

Martin's question was "Is there some kind of spec on exactly what this function should do? Can somebody start one?". Clearly the currently available fixes are not meeting the tech spec that is required for 'growed up' Moodle - however these can act as functional models of what is required. Hence the 'what it should do' is addressed by existing fixes and patches.

For my money I simply need

• to create (or draw down from our local registry) a list of courses
• to manipulate the list to add/edit shortname, course code etc.
• to be able to set core settings for courses (format, number of topics, start and end date etc.)
• to be able to add at least one editing teacher via the list
• to upload to the moodle site and cross check for duplicates

Can we flesh this out towards a fuller spec ?

Here's an example of a 'quick'n dirty .csv upload that I was using for 1.9 using one of the patches

category,enrol,enrollable,expirynotify,format,fullname,groupmode,groupmodeforce,guest,metacourse,newsitems,notifystudents,numsections,password,shortname,showgrades,showreports
2011.1 PG Courses,manual,0,0,topics,011.1 719 Cost Price and Revenue Management (PG),0,0,0,0,0,0,5,locked,011.1 719 Cost ,0,0

In my view the current urgency is to establish the bulk course creation, for those about to join Moodle, and for those of us trying to start a new teaching year on 2.0/2.1 - bulk course removal isn't going to be an issue for some time.....

Do you need a CSV file?
We're doing this for years in XML with an IMS Enterprise file.

Maybe this can help to someone: http://www.stylusstudio.com/csv_to_xml.html or http://csv2xml.sourceforge.net/
you can convert the CSV file to IMS file

If this gets implemented could we please first create real CSV import/export library that supports different separators and standard quoting?

"If this gets implemented could we please first create real CSV import/export library that supports different separators and standard quoting?" Petr, please translate.

"Current implementation for CVS upload to create accounts is inadequate"
"Old methods need to be fixed to new methods"
"If we fix some things now it will improve things and make other CSV importe easy"
"We now use only one separator and non standard quotes"
"This means a further delay since 2008 when this was first proposed"
"What is currently used is not a real import/export/library" ????????????????

I'm not a PHP coder. But it just seems that this should be not a big deal. (is it?) What saddens me is that people are creating tons of hacks (which uses time) but no-one is putting in the effort to co-ordinate and focus efforst to get this into standard Moodle.

What is involved in doing what you ask Petr? A day of coding? Does MoodleHQ want it or see the need for it?
</ramble> </rant>
-Derek

I've written a small proof-of-concept patch for 2.1 which borrows very liberally from the bulk user upload page. This patch is deliberately incomplete. It's missing most forms of error-checking and validation. It doesn't display results unless something blows up. It does, however, use a standard interface and standard internal functions, which I believe addresses a number of the basic concerns raised by Martin and others.

Is it worth taking this patch further? If not, I don't mind the wasted morning.

Patch here on github: https://github.com/mackensen/moodle/compare/mdl-13114.

As far as I have followed this question, I realize that course creation and user enrolment depend on very singular situations in the different schools and universities. That is why I started to program my own import script with a GUI AND an unattended mode version. See here: http://moodle.org/mod/forum/discuss.php?d=128317

With Anthnoy Borrow I considered to modify my script so as to:

• import/create categories (adding the function into the GUI, like upload users)
• import/create courses (adding the function into teh GUI, like upload users)
• import/create/enrol users (as already in the GUI)
• delete courses (adding the function into the GUI, like upload users)
• all-in-one script/function to do all this with one and the same .csv File and script (adding this function to the GUI, like upload users)
• all-in-one script that can be executed in unattended mode, during night, through a cron-job (already working for me and my needs. Our Workflow:

• courses maintained in our school administration system are exported nightly to a windows-share in .csv UTF-8 format. The csv file contains all the columns you can see in my template in the above thread, short everything needed to create new courses and enroll the given users in their respective roles

• the Moodle cron-job executes my uploadusersandcourses.php script every night writing back to the share a text file with error-codes like "course xyz already exists", or "category xyz does not exist", or "course xyz created", etc. so that we know exactly what the script did during night)

My script only lacks deletion of courses as at our university we have to consider also archiving of old courses. So we could define what archiving means: copying only the backup-zip-File to a certain backup-server oder backup-archive-directory on the same server, and so on. Archiving could also mean MOVING the whole course to an archive category in the same Moodle server. Actually I move the courses through the GUI to an archive or toDelete category where I leave them hidden and without teachers for another 6 months before deleting them definitively or letting them in the archive category forever.

But my script has also some more features (realized and planned):

• course copy/creation from a given template (backup-zip-file) of an existing course (working fine)
• rename course from old name to new name (because we have courses that change only their name before starting over for a next semester or academic year, not programmed yet, but feasible, scheduled for next release)
• archive course (has to be defined, not programmed yet)

So we just had to join our efforts to get existing code of different programmers into the GUI and core of Moodle. I hope to find enough time before winter to realize the above listed functions separatedly and in an all-in-one script.

And this gui already programmed a course upload script for Moodle 2.x: http://moodle.org/mod/forum/discuss.php?d=173705

Rosario

Hi Rosario,

as I wrote at some comments above, we've been doing all this kind of stuff since 2005 with XML (not CSV) with an "IMS Enterprise" file that is built-in in moodle (you don't need to maintain it) and it's an standard (we didn't want to reinvent the wheel and yes, we know some other people feels more comfortable with csv).

The "IMS Enterprise" lacks the course creation from a given template (but you can rename a course, move it from one category to an other...), so I'll study the way to do this with the "IMS Enterprise" file, this will save lot of work to our staff.

Ciao.

Eduardo, I think the main reason behind csv is that we usually take the courses list from another system. Some of these may be made with old languages and can be a PITA to export data to xml. Ie: Fox and Clipper. It's easy to get a .dbf and convert it to csv and when the original programmer or source code from that system is not available it's a problem.

To transform the csv to xml is a one-step procedure in excell using the concat() function. This is what I do when some teachers want me to create 50 courses, they give me a csv file and I add the last column (this is a short example and nobody needs a programmer to do this, but if you have one, you can automate this step in a couple of hours):

and so on.

What I want to said is that lot of people have been waiting for an utility that is already made. I know that the people is busy and maybe they don't have time to read the IMS Enterprise specification, what I'm telling is that the solution to their needs has been there for years and that someone, like me, can tell them where it is.

Alejandro, Eduardo, I DID consider IMS Enterprise Import before I decided to program my own uploadusersandcourses.php and I made some posts at that time in the moodle forums. The problem is exactly as Alejandro says, other systems used for course and user management are all able to export into simple text or csv files, but not to IMS Enterprise format. So even if we/I could automate a csv into IMS transformation, this would be an extra step to control and maintain over years. And my intention was to offer the same GUI interface Moodle offers in upload users which uses csv files and to separately combine all the code into one all-in-one script you can schedule during night.

108 votes - how many does it take?
:-~

Awesome thanks for the tip Eduardo, legend.

Eduardo, I found your guidance on using the IMS Enterprise import to be very helpful, so I'd also like to add my thanks.
However I think we still need a csv upload/delete.

I'm just starting Moodle 2.x , and trying to visualise how it might be scaled to Enterprise level. I seem to be stuck at these issues:

IMS Enterprise does not delete courses
IMS Enterprise requires a unique course before creating a category (At the start I only want an empty structure)
IMS Enterprise does not create sub-categories
IMS Enterprise does not support course properties (number of sections etc.)

It seems to be a fundamental restriction that IMS only knows about generic IMS things, whereas a CSV import could be targetted at Moodle concepts.

Thanx to you two too,

I missed some functionality to IMS Enterprise too (groups and grouping for instance) but IMS can be extended in any way you need. We'd extended the use to support WebDAV and SSO from our intranet.

I'll post a suggestion to the developers to do the same job to IMS and CSV import. This makes no sense to have different functionality between two import systems from a file (csv, ims, ods...)

Hi -

This may or may not be of any use to people on this thread, but I create a basic course uploader for 2.x modelled on the user uploader, as I needed something for NZ schools - it's available here https://gitorious.org/moodle-tool_uploadcourse/moodle-tool_uploadcourse for the curious.

Cheers,
Piers Harding.

Piers, I'm trying to use your course upload tool with Moodle 2.1.2, but I am unable to get Moodle to recognise it. Does it work with 2.1.x? I put the files in the relevant place, but nothing I do kicks off an installation so it never shows up on the menu.

Sean K