Moodle

csv will not import into database activity

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Not a bug
  • Affects Version/s: 1.9.2
  • Fix Version/s: None
  • Component/s: Database activity module
  • Labels:
    None
  • Environment:
    Moodle 1.9.2+ (Build: 20081001), Debian, My SQL 5.0.45, PHP 5.2.2
  • Affected Branches:
    MOODLE_19_STABLE

Description

I've added a database activity and when trying to import entries from a csv file, depending on which of the numerous formats and settings I've tried, I get one of two error messages: either " get_records_csv failed to read data from the uploaded file. Please check file for field name typos and formatting errors" or " could not update record ID xx "

When I get the could not update record ID xx message, a new record is added, but it is blank, and the database doe indicate it has x number of entries.

I've used excel and notepad to create the csv file. I've exported the table as csv from the server where it resides to my local computer, then directly imported into Moodle. I've used every combination of delimiter, etc. I can think of and entered (and not entered) those in the Moodle import dialog window. I've imported through IE and Mozilla (Windows XP), and I've tried to import lots of records and just two records. I've included the field names on the top line and I've excluded them (not that I expected that to work, but you never know). I have held my breath and held my tongue that certain way that sometimes works, to no avail. I've looked at various patches related to database import/export, but it seems I'm current on those (though they would be easy to miss, as I'm cross-eyed now). I can manually make entries with no problems what so ever

Any help would be most appreciated. I need this database, but if have have to manually enter the 600-800 records one by one, I may just change careers.

Attaching most recent sample csv file

Thanks so much!

Attachments

  1. File
    test.csv
    19/Oct/08 11:38 AM
    0.1 kB
    Sharon Goodson
  1. Import_debug.jpg
    111 kB
    19/Oct/08 12:06 PM
  2. Import_debug2.jpg
    99 kB
    19/Oct/08 12:10 PM

Issue Links

This issue will be resolved by:
MDL-9075 User Upload Failure when file contains utf-8 bom Major Closed

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Sharon Goodson added a comment -

This is a screen shot of the bedug message after another attempt.
Thanks Again!

Show
Sharon Goodson added a comment - This is a screen shot of the bedug message after another attempt. Thanks Again!
Hide
Permalink
Sharon Goodson added a comment -

And this is a screennshot of the list view after two attempts (each attempt included three record, as attached)

Show
Sharon Goodson added a comment - And this is a screennshot of the list view after two attempts (each attempt included three record, as attached)
Hide
Permalink
Sharon Goodson added a comment -

I just realized I do have an older version of mod/data/import.php. I replaced wit v.1.32, but now the white page with only:

Notice: Undefined variable: DB in /hermes/web05/b980/moo.intern32/ihs/moodle/mod/data/import.php on line 49

Fatal error: Call to a member function get_record() on a non-object in /hermes/web05/b980/moo.intern32/ihs/moodle/mod/data/import.php on line 49

I will wait until I hear from someone, thus I break Moodle lol

Show
Sharon Goodson added a comment - I just realized I do have an older version of mod/data/import.php. I replaced wit v.1.32, but now the white page with only: Notice: Undefined variable: DB in /hermes/web05/b980/moo.intern32/ihs/moodle/mod/data/import.php on line 49 Fatal error: Call to a member function get_record() on a non-object in /hermes/web05/b980/moo.intern32/ihs/moodle/mod/data/import.php on line 49 I will wait until I hear from someone, thus I break Moodle lol
Hide
Permalink
Robert Allerstorfer added a comment -

"I just realized I do have an older version of mod/data/import.php. I replaced wit v.1.32, but now..."

That seems to be your source of problems. Your Moodle installation seems to be messed up somehow. Before reporting a "bug", please always ensure to have updated Moodle properly to the latest stable version. Overwriting some files by hand is not the proper way...

Show
Robert Allerstorfer added a comment - "I just realized I do have an older version of mod/data/import.php. I replaced wit v.1.32, but now..." That seems to be your source of problems. Your Moodle installation seems to be messed up somehow. Before reporting a "bug", please always ensure to have updated Moodle properly to the latest stable version. Overwriting some files by hand is not the proper way...
Hide
Permalink
Sharon Goodson added a comment -

I must not have explained the situation very well, and I apologize for the misunderstanding. My remark about 'breaking Moodle' was in jest.

My Moodle version was downloaded 10/8/08 and I assumed was the latest stable at the time. I have not modified anything in the mod /data folder, and only tried replacing the mod/data/import.php filter after the fact.

AFTER I reported the problem illustrated in the screenshots I noticed that the mod/data/import.php file included in my Moodle installation was revision 1.21.2.7. It was my understanding that when you run into problems you should check if various files or modules had been updated to address bugs, etc.

In attempt to do this I checked the log for moodle/mod/data/import.php at <http://cvs.moodle.org/moodle/mod/data/import.php> and noted there had since been three revisions to the file included in my installation. Your 1.3 revision states it addresses a "reopened bug for importing database records as CSV."

Considering that that was my problem, I made a copy of the originally installed file 1.21.2.7( just in case something weird happened, which it did) and replaced it with the newest revised file 1.32. At that point I received the error I noted in my last comment. Since it obviously did not solve my problem, I deleted the 1.32 file and restored the original 1.21.2.7 file. Now the problem is once again as I originally stated and as I illustrated in the screenshots. I added the comment of what happened when I used the 1.32 simply to record that I had tried this action and it failed.

At no point since the Moodle installation did I overwrite or modify any files related to the issue I submitted. AFTER the initial problem was discovered and reported, I simply tried what I assumed (incorrectly) would be suggested, and got a new error. Restoring the original file simply restored me to the original problem. Replacing the file may have been the incorrect way to try to address the problem (please explain the proper procedure or let me know where I can find it), but it is irrelevant to what I initially reported and the problem that remains.

Thank you!

Show
Sharon Goodson added a comment - I must not have explained the situation very well, and I apologize for the misunderstanding. My remark about 'breaking Moodle' was in jest. My Moodle version was downloaded 10/8/08 and I assumed was the latest stable at the time. I have not modified anything in the mod /data folder, and only tried replacing the mod/data/import.php filter after the fact. AFTER I reported the problem illustrated in the screenshots I noticed that the mod/data/import.php file included in my Moodle installation was revision 1.21.2.7. It was my understanding that when you run into problems you should check if various files or modules had been updated to address bugs, etc. In attempt to do this I checked the log for moodle/mod/data/import.php at <http://cvs.moodle.org/moodle/mod/data/import.php> and noted there had since been three revisions to the file included in my installation. Your 1.3 revision states it addresses a "reopened bug for importing database records as CSV." Considering that that was my problem, I made a copy of the originally installed file 1.21.2.7( just in case something weird happened, which it did) and replaced it with the newest revised file 1.32. At that point I received the error I noted in my last comment. Since it obviously did not solve my problem, I deleted the 1.32 file and restored the original 1.21.2.7 file. Now the problem is once again as I originally stated and as I illustrated in the screenshots. I added the comment of what happened when I used the 1.32 simply to record that I had tried this action and it failed. At no point since the Moodle installation did I overwrite or modify any files related to the issue I submitted. AFTER the initial problem was discovered and reported, I simply tried what I assumed (incorrectly) would be suggested, and got a new error. Restoring the original file simply restored me to the original problem. Replacing the file may have been the incorrect way to try to address the problem (please explain the proper procedure or let me know where I can find it), but it is irrelevant to what I initially reported and the problem that remains. Thank you!
Hide
Permalink
Sharon Goodson added a comment -

Update: The import works if from notepad I save the csv file using ANSI encoding. I discovered this by accident, as it was my understanding that it should be saved with UTF-8 encoding (which i had been doing). I followed Jack McDowell's suggestion and exported a working database and opened that in notepad. I made some changes, saved it and it worked. I was even more stumped because the successful file looked identical to the unsuccessful file. After some more head-banging I finally noticed that one was defaulting to ANSI encoding on 'save as,' the other defaulted to UTF-8. I'm thrilled I can upload, but still seriously confused as to why this would be, since my Moodle database is UTF-8.

Show
Sharon Goodson added a comment - Update: The import works if from notepad I save the csv file using ANSI encoding. I discovered this by accident, as it was my understanding that it should be saved with UTF-8 encoding (which i had been doing). I followed Jack McDowell's suggestion and exported a working database and opened that in notepad. I made some changes, saved it and it worked. I was even more stumped because the successful file looked identical to the unsuccessful file. After some more head-banging I finally noticed that one was defaulting to ANSI encoding on 'save as,' the other defaulted to UTF-8. I'm thrilled I can upload, but still seriously confused as to why this would be, since my Moodle database is UTF-8.
Hide
Permalink
Sharon Goodson added a comment -

false positive

Show
Sharon Goodson added a comment - false positive
Hide
Permalink
Frank Ralf added a comment -

If the problem only shows with UTF-8 files the culprit might be the BOM (http://en.wikipedia.org/wiki/Byte-order_mark).

See this forum discussion for an example in Hebrew http://moodle.org/mod/forum/discuss.php?d=62251#p559428

Show
Frank Ralf added a comment - If the problem only shows with UTF-8 files the culprit might be the BOM (http://en.wikipedia.org/wiki/Byte-order_mark). See this forum discussion for an example in Hebrew http://moodle.org/mod/forum/discuss.php?d=62251#p559428
Hide
Permalink
Frank Ralf added a comment -

With the Database Activity There still seems to be a problem importing UTF-8 files with BOM (http://en.wikipedia.org/wiki/Byte-order_mark).

See this forum discussion for an example in Hebrew
http://moodle.org/mod/forum/discuss.php?d=62251#p559428

Show
Frank Ralf added a comment - With the Database Activity There still seems to be a problem importing UTF-8 files with BOM (http://en.wikipedia.org/wiki/Byte-order_mark). See this forum discussion for an example in Hebrew http://moodle.org/mod/forum/discuss.php?d=62251#p559428

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Moodle. Try JIRA - bug tracking software for your team.