Moodle

Export records of a database activity as a CSV, XLS, or ODS file

Details

  • Type: Improvement Improvement
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 1.9.1
  • Fix Version/s: 1.9.2
  • Component/s: Database activity module
  • Labels:
    None
  • Database:
    Any
  • Affected Branches:
    MOODLE_19_STABLE
  • Fixed Branches:
    MOODLE_19_STABLE

Description

Though this module has been handed off it still needs attention. The database activity module would greatly be improved if the teacher/admin could export the database entries into a csv or better yet, provide a preformatted GIFT database entry to speed quiz/text creation.

Attachments

  1. Text File
    db_export.patch
    18/Apr/08 3:31 AM
    16 kB
    Dan Poltawski
  2. Text File
    MDL-8407_20080518.patch
    19/May/08 12:14 AM
    15 kB
    Robert Allerstorfer
  3. Text File
    MDL-8407-20080419.patch
    20/Apr/08 7:49 AM
    18 kB
    Robert Allerstorfer
  4. Text File
    MDL-8407-capability-fix.patch
    20/Apr/08 10:09 PM
    0.7 kB
    Robert Allerstorfer
  5. Text File
    MDL-8407-capability-fix2.patch
    20/Apr/08 10:24 PM
    0.3 kB
    Robert Allerstorfer
  6. Text File
    MDL-8407-capability-fix2a.patch
    21/Apr/08 12:55 AM
    0.6 kB
    Robert Allerstorfer
  7. Text File
    MDL-8407-date-fix.patch
    13/May/08 8:33 AM
    0.5 kB
    Robert Allerstorfer
  8. Text File
    MDL-8407-encdelim-fix.patch
    19/Apr/08 9:23 AM
    0.3 kB
    Robert Allerstorfer
  9. Text File
    MDL-8407-latlong-fix.patch
    19/Apr/08 11:28 AM
    0.9 kB
    Robert Allerstorfer
  10. Text File
    MDL-8407-MOODLE_19_STABLE.patch
    19/Apr/08 7:53 AM
    18 kB
    Robert Allerstorfer
  11. Text File
    MDL-8407-url-fix.patch
    20/Apr/08 9:32 PM
    0.3 kB
    Robert Allerstorfer

Issue Links

This issue is duplicated by:
MDL-10530 Database Module does not allow data export Major Closed
MDL-8488 export for the database modul Minor Closed
This issue has a non-specific relationship to:
MDL-14442 Improvements importing into database activity when using CSV files produced by export Major Closed
MDL-8386 Database Module - Add Ability to Perform Summary Analysis of Fields or Export data to Excel Minor Open

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Beatriz Beltran added a comment -

This is a very much needed feature for the database. Several of my users are asking how to export the databases they create with Moodle and we have many.

Show
Beatriz Beltran added a comment - This is a very much needed feature for the database. Several of my users are asking how to export the databases they create with Moodle and we have many.
Hide
Permalink
Anthony Borrow added a comment -

I have made this the central tracker for requests to add the capability to export data from the database module to either CSV, Excel, ODS, XML, etc. I think that the database module should follow and use similar code as the feedback module, gradebook, etc. I don't think GIFT would be all that usable (IMHO) but I suspect it was related to the issue of using the database module to have students write quiz questions; however, this should be addressed in the question code. Since one of our teachers recently requested this feature I suspect it will be a popular request. I'll post something in the forums to encourage folks to vote for this issue. Peace - Anthony

Show
Anthony Borrow added a comment - I have made this the central tracker for requests to add the capability to export data from the database module to either CSV, Excel, ODS, XML, etc. I think that the database module should follow and use similar code as the feedback module, gradebook, etc. I don't think GIFT would be all that usable (IMHO) but I suspect it was related to the issue of using the database module to have students write quiz questions; however, this should be addressed in the question code. Since one of our teachers recently requested this feature I suspect it will be a popular request. I'll post something in the forums to encourage folks to vote for this issue. Peace - Anthony
Hide
Permalink
Aaron Russell added a comment -

The option of exporting the data should be an option just like exporting templates.

Show
Aaron Russell added a comment - The option of exporting the data should be an option just like exporting templates.
Hide
Permalink
Jason Hollowell added a comment -

I agree that this would be a very welcome and useful fix/improvement to the database module. With the addition of the advanced search option in 1.9 it would be an even more valuable capability to be able to export search results to a file.

Show
Jason Hollowell added a comment - I agree that this would be a very welcome and useful fix/improvement to the database module. With the addition of the advanced search option in 1.9 it would be an even more valuable capability to be able to export search results to a file.
Hide
Permalink
Anthony Borrow added a comment -

MDL-10530 (essentially a duplicate) mentions:

currently the very cool Database activity does not allow data export. Would be very useful a new "Export template" allowing at least teacher to:

1) Able to select the desired fields.
2) Able to use the format in the template itself or some other alternative (XML, ODS and CVS)
3) Or any of them

Show
Anthony Borrow added a comment - MDL-10530 (essentially a duplicate) mentions: currently the very cool Database activity does not allow data export. Would be very useful a new "Export template" allowing at least teacher to: 1) Able to select the desired fields. 2) Able to use the format in the template itself or some other alternative (XML, ODS and CVS) 3) Or any of them
Hide
Permalink
Jay Melton added a comment -

I would like to put in a request for the export/download feature as well. The ability to export and download CSV, Excel, ODS, and XML files would greatly enhance this useful module's capabilities.

Show
Jay Melton added a comment - I would like to put in a request for the export/download feature as well. The ability to export and download CSV, Excel, ODS, and XML files would greatly enhance this useful module's capabilities.
Hide
Permalink
Dan Poltawski added a comment -

I have a patch incoming for this

Show
Dan Poltawski added a comment - I have a patch incoming for this
Hide
Permalink
Robert Allerstorfer added a comment -

Good news, Dan, who is working on it and when do you expect it to be available?

Show
Robert Allerstorfer added a comment - Good news, Dan, who is working on it and when do you expect it to be available?
Hide
Permalink
Dan Poltawski added a comment -

I am working on this, I have a patch available today for testing, but was planning on tidying it a bit more in spare time this week

Show
Dan Poltawski added a comment - I am working on this, I have a patch available today for testing, but was planning on tidying it a bit more in spare time this week
Hide
Permalink
Dan Poltawski added a comment -

Here is first go of patch

Todo: preventdata leaking (groups mode etc), add new export capabilities?

Also I considered making the export things more 'pluggable' then I just wondered if there should be a system wide version of that for things like gradebook, users download etc etc etc for everything which re-implements this wheel

Show
Dan Poltawski added a comment - Here is first go of patch Todo: preventdata leaking (groups mode etc), add new export capabilities? Also I considered making the export things more 'pluggable' then I just wondered if there should be a system wide version of that for things like gradebook, users download etc etc etc for everything which re-implements this wheel
Hide
Permalink
Robert Allerstorfer added a comment -

Thank you so much, Dan, your work is much appreciated - really looks great!!

I have attached your patch in a reformatted format, so that it can be applied against MOODLE_19_STABLE by executing "patch -p0 < MDL-8407-MOODLE_19_STABLE.patch" in the moodle root web directory:

[root@anet moodle]# patch -p0 < MDL-8407-MOODLE_19_STABLE.patch
patching file lang/en_utf8/data.php
patching file mod/data/field/checkbox/field.class.php
patching file mod/data/field/date/field.class.php
patching file mod/data/field/latlong/field.class.php
patching file mod/data/field/menu/field.class.php
patching file mod/data/field/multimenu/field.class.php
patching file mod/data/field/number/field.class.php
patching file mod/data/field/radiobutton/field.class.php
patching file mod/data/field/text/field.class.php
patching file mod/data/field/textarea/field.class.php
patching file mod/data/field/url/field.class.php
patching file mod/data/lib.php
patching file mod/data/tabs.php
patching file mod/data/export.php
patching file mod/data/export_form.php

What is missing for me is an option to specify the CSV separating character, similar than in the CSV import. Currently, a comma will be used which is bad IMO, since it prevents you to have commas in the data.

Show
Robert Allerstorfer added a comment - Thank you so much, Dan, your work is much appreciated - really looks great!! I have attached your patch in a reformatted format, so that it can be applied against MOODLE_19_STABLE by executing "patch -p0 < MDL-8407-MOODLE_19_STABLE.patch" in the moodle root web directory: [root@anet moodle]# patch -p0 < MDL-8407-MOODLE_19_STABLE.patch patching file lang/en_utf8/data.php patching file mod/data/field/checkbox/field.class.php patching file mod/data/field/date/field.class.php patching file mod/data/field/latlong/field.class.php patching file mod/data/field/menu/field.class.php patching file mod/data/field/multimenu/field.class.php patching file mod/data/field/number/field.class.php patching file mod/data/field/radiobutton/field.class.php patching file mod/data/field/text/field.class.php patching file mod/data/field/textarea/field.class.php patching file mod/data/field/url/field.class.php patching file mod/data/lib.php patching file mod/data/tabs.php patching file mod/data/export.php patching file mod/data/export_form.php What is missing for me is an option to specify the CSV separating character, similar than in the CSV import. Currently, a comma will be used which is bad IMO, since it prevents you to have commas in the data.
Hide
Permalink
Robert Allerstorfer added a comment -

I noticed that the delimiter character (,) within records will be encoded, so commas in it should work. However, the encoding was incomplete. Apply the attached patch (after MDL-8407-MOODLE_19_STABLE.patch has already been applied) to fix it.

Show
Robert Allerstorfer added a comment - I noticed that the delimiter character (,) within records will be encoded, so commas in it should work. However, the encoding was incomplete. Apply the attached patch (after MDL-8407-MOODLE_19_STABLE.patch has already been applied) to fix it.
Hide
Permalink
Robert Allerstorfer added a comment -

Here is another fix ("MDL-8407-latlong-fix.patch") to store the values of a latlong field as (for example) "-31.9545 115.8770" instead of "31.9545°S 115.8770°E". This allows easy re-importing of the exported CSV file.

Show
Robert Allerstorfer added a comment - Here is another fix ("MDL-8407-latlong-fix.patch") to store the values of a latlong field as (for example) "-31.9545 115.8770" instead of "31.9545°S 115.8770°E". This allows easy re-importing of the exported CSV file.
Hide
Permalink
Dan Poltawski added a comment -

Thanks Robert!

I have to admit i'd forgotten to check the import formats via csv. Is latlong the only one which conflicts? How do multiselect imports work?

Show
Dan Poltawski added a comment - Thanks Robert! I have to admit i'd forgotten to check the import formats via csv. Is latlong the only one which conflicts? How do multiselect imports work?
Hide
Permalink
Robert Allerstorfer added a comment -

Dan,

latlong was the only problem at the export side. I will submit a patch within the next few hours for the import side that allows to import the csv created with these export patches. Currently, the import does not even work on multiselect fields.

Show
Robert Allerstorfer added a comment - Dan, latlong was the only problem at the export side. I will submit a patch within the next few hours for the import side that allows to import the csv created with these export patches. Currently, the import does not even work on multiselect fields.
Hide
Permalink
Robert Allerstorfer added a comment -

To make things easier, I have attached an all-in-one patch against the latest 1.9+ Moodle from today's CVS ("MDL-8407-20080419.patch"). This is all you need and should be applied to use my patch I will submit now at an own tracker ID which allows to import everything again which has been exported by this as CSV file.

Show
Robert Allerstorfer added a comment - To make things easier, I have attached an all-in-one patch against the latest 1.9+ Moodle from today's CVS ("MDL-8407-20080419.patch"). This is all you need and should be applied to use my patch I will submit now at an own tracker ID which allows to import everything again which has been exported by this as CSV file.
Hide
Permalink
Robert Allerstorfer added a comment -

Dan, the url field's content2 (an optional name) should also be exported. The attached patch ("MDL-8407-url-fix.patch") fixes this. Should be applied after applying "MDL-8407-20080419.patch".

These two patches, together with my new "MDL-14442-20080420.patch" available at MDL-14442, allow the complete export and import of all supported fields

I have tested it successfully on all field types except Checkbox, Date, Number and Radio buttons.

Show
Robert Allerstorfer added a comment - Dan, the url field's content2 (an optional name) should also be exported. The attached patch ("MDL-8407-url-fix.patch") fixes this. Should be applied after applying "MDL-8407-20080419.patch". These two patches, together with my new "MDL-14442-20080420.patch" available at MDL-14442, allow the complete export and import of all supported fields I have tested it successfully on all field types except Checkbox, Date, Number and Radio buttons.
Hide
Permalink
Robert Allerstorfer added a comment -

Next patch and (I hope) the last one from me for a while now:

Fixes the capability. Allow the export only to those people who are also allowed to import (capability 'mod/data:managetemplates').

Finally, after patching code from Moodle's database activity during the last two weeks' days and nights, and the great work of the Moodle core developers from the last two weeks, most important things seem to work fine now...

Show
Robert Allerstorfer added a comment - Next patch and (I hope) the last one from me for a while now: Fixes the capability. Allow the export only to those people who are also allowed to import (capability 'mod/data:managetemplates'). Finally, after patching code from Moodle's database activity during the last two weeks' days and nights, and the great work of the Moodle core developers from the last two weeks, most important things seem to work fine now...
Hide
Permalink
Robert Allerstorfer added a comment -

This one was missing....

Show
Robert Allerstorfer added a comment - This one was missing....
Hide
Permalink
Robert Allerstorfer added a comment -

the last patch should have better been this one...

Show
Robert Allerstorfer added a comment - the last patch should have better been this one...
Hide
Permalink
Robert Allerstorfer added a comment -

The attached patch "MDL-8407-date-fix.patch" (to be applied after "MDL-8407-20080419.patch") exports date fields in "raw" format, so they can easily be re-imported by the importer from MDL-14442.

I have tested importing successfully now also with the field types "Date" and "Number", so the only yet untested field types are "Checkbox" and "Radio buttons".

Show
Robert Allerstorfer added a comment - The attached patch "MDL-8407-date-fix.patch" (to be applied after "MDL-8407-20080419.patch") exports date fields in "raw" format, so they can easily be re-imported by the importer from MDL-14442. I have tested importing successfully now also with the field types "Date" and "Number", so the only yet untested field types are "Checkbox" and "Radio buttons".
Hide
Permalink
Robert Allerstorfer added a comment -

Just wanted to remark that I have now also tested successfully the "Checkbox" and "Radio buttons" field types, so all possible fields exported via this series of patches import fine using the new import code

Show
Robert Allerstorfer added a comment - Just wanted to remark that I have now also tested successfully the "Checkbox" and "Radio buttons" field types, so all possible fields exported via this series of patches import fine using the new import code
Hide
Permalink
Robert Allerstorfer added a comment -

Here is a new all-in-one patch (MDL-8407_20080518.patch) against Moodle 1.9.1 and the current MOODLE_19_STABLE CVS branch with further improvements. I think it is ready to commit.

Show
Robert Allerstorfer added a comment - Here is a new all-in-one patch (MDL-8407_20080518.patch) against Moodle 1.9.1 and the current MOODLE_19_STABLE CVS branch with further improvements. I think it is ready to commit.
Hide
Permalink
Robert Allerstorfer added a comment -

Dan,

could you please let me know if you are still planning to commit something of this solution into MOODLE_19_STABLE in the nearer future (before the release of 1.9.2)? If not, should I reassign this issue to me, so I could commit parts of it stepwise, as a counterpart to MDL-14442?

Thanks,
rob.

Show
Robert Allerstorfer added a comment - Dan, could you please let me know if you are still planning to commit something of this solution into MOODLE_19_STABLE in the nearer future (before the release of 1.9.2)? If not, should I reassign this issue to me, so I could commit parts of it stepwise, as a counterpart to MDL-14442? Thanks, rob.
Hide
Permalink
Dan Poltawski added a comment -

Hi Robert,

Sorry i've been on holiday for a few weeks so haven't been able to look at the updates, assigning to you

Show
Dan Poltawski added a comment - Hi Robert, Sorry i've been on holiday for a few weeks so haven't been able to look at the updates, assigning to you
Hide
Permalink
Robert Allerstorfer added a comment -

Hi Dan,

OK, I'll be glad to take care over this issue. The first commits are now finished - to prepare the required language strings. The next step will be to commit the four modified mod/data/field/*/field.class.php files - but not before tomorrow.

cheers,
rob.

Show
Robert Allerstorfer added a comment - Hi Dan, OK, I'll be glad to take care over this issue. The first commits are now finished - to prepare the required language strings. The next step will be to commit the four modified mod/data/field/*/field.class.php files - but not before tomorrow. cheers, rob.
Hide
Permalink
Robert Allerstorfer added a comment -

Everything is committed now - it works fine for me Now to the QA...

Show
Robert Allerstorfer added a comment - Everything is committed now - it works fine for me Now to the QA...
Hide
Permalink
Robert Allerstorfer added a comment -

When exporting as CSV, the semicolon should not be offered as delimiter, since its encrypted form also contains a semicolon. I will solve this problem soon.

Show
Robert Allerstorfer added a comment - When exporting as CSV, the semicolon should not be offered as delimiter, since its encrypted form also contains a semicolon. I will solve this problem soon.
Hide
Permalink
Robert Allerstorfer added a comment -

Fixed in CSV - re-resolving this issue.

Show
Robert Allerstorfer added a comment - Fixed in CSV - re-resolving this issue.
Hide
Permalink
Robert Allerstorfer added a comment -

Hello Nicolas Martignoni,

do you think you will find time to make the QA (as assigned above)?

Thanks,
rob.

Show
Robert Allerstorfer added a comment - Hello Nicolas Martignoni, do you think you will find time to make the QA (as assigned above)? Thanks, rob.
Hide
Permalink
Nicolas Martignoni added a comment -

Yes Rob. Will do it soon (this WE). Thanks for the heads up.

Show
Nicolas Martignoni added a comment - Yes Rob. Will do it soon (this WE). Thanks for the heads up.
Hide
Permalink
Petr Škoda (skodak) added a comment -

oh, please revert the changes in mod/data/lib.php HEAD and do proper merging

Show
Petr Škoda (skodak) added a comment - oh, please revert the changes in mod/data/lib.php HEAD and do proper merging
Hide
Permalink
Petr Škoda (skodak) added a comment -

the HEAD commit reverted and merged flag moved back...

Show
Petr Škoda (skodak) added a comment - the HEAD commit reverted and merged flag moved back...
Hide
Permalink
Petr Škoda (skodak) added a comment -

Thanks

Show
Petr Škoda (skodak) added a comment - Thanks
Hide
Permalink
Robert Allerstorfer added a comment -

Re-resolving because no known problem so far - so QA can be done.

Show
Robert Allerstorfer added a comment - Re-resolving because no known problem so far - so QA can be done.
Hide
Permalink
Nicolas Martignoni added a comment -

Works as described. Commas in field names and data are correctly encoded when CSV with sep comma choosen No problem found. Thanks for this cool new feature.

Suggestion: make "image" and "file" field types exportable in a XML export/import format would really be very cool.

Show
Nicolas Martignoni added a comment - Works as described. Commas in field names and data are correctly encoded when CSV with sep comma choosen No problem found. Thanks for this cool new feature. Suggestion: make "image" and "file" field types exportable in a XML export/import format would really be very cool.
Hide
Permalink
Nate Baxley added a comment -

We upgraded from 1.8 this weekend and found that the files in this patch conflicted with some customizations we had made earlier to create our own database export tool. I'd be happy to use this new tool, but can't figure out how to access it. Can some one let me know how to use this tool or point me somewhere that can show me?

Thanks much!

Show
Nate Baxley added a comment - We upgraded from 1.8 this weekend and found that the files in this patch conflicted with some customizations we had made earlier to create our own database export tool. I'd be happy to use this new tool, but can't figure out how to access it. Can some one let me know how to use this tool or point me somewhere that can show me? Thanks much!
Hide
Permalink
Robert Allerstorfer added a comment -

Nate,

login as a user that has assigned the (default) role "Non-editing teacher" or higher ("Teacher" or "Administrator") at the course the database activity is in.

Then, just click the "Export" tab.

Show
Robert Allerstorfer added a comment - Nate, login as a user that has assigned the (default) role "Non-editing teacher" or higher ("Teacher" or "Administrator") at the course the database activity is in. Then, just click the "Export" tab.
Hide
Permalink
Nate Baxley added a comment -

Robert,
Thanks. I didn't realize that this only "baked in" after build 20080601. It is working as you described in the latest build.
Thanks.

Show
Nate Baxley added a comment - Robert, Thanks. I didn't realize that this only "baked in" after build 20080601. It is working as you described in the latest build. Thanks.

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Moodle. Try JIRA - bug tracking software for your team.