Moodle
  1. Moodle
  2. MDL-30871

Flat File (CSV) plug in message_provider issue

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.2, 2.3.2
    • Fix Version/s: 2.2.7, 2.3.4
    • Component/s: Enrolments
    • Labels:
    • Environment:
      Ubuntu
    • Database:
      MySQL
    • Testing Instructions:
      Hide

      Test pre-requisites

      • 1 course with ID number 123 (no users enrolled)
      • 5 users with ID number 1 to 5
      • A CSV file as following
      add, student, 1, 123
      add, student, 2, 123
      add, student, 3, 123
      add, student, 4, 123
      add, student, 5, 123
      

      Test steps

      1. Enable, and configure the Flat File enrolment plugin
        • Enable notifications for each user
        • Set the path to the CSV file mentioned above
      2. Run the cron and make sure
        • The line "Running cron for enrol_flatfile..." is present but no errors occured
        • The users have been enrolled on the course
        • An email has been sent to the admin, and the students

      You might have to run the cron several times before the enrol_flatfile cron line appears
      Save your CSV file because it will be deleted after execution

      Show
      Test pre-requisites 1 course with ID number 123 (no users enrolled) 5 users with ID number 1 to 5 A CSV file as following add, student, 1, 123 add, student, 2, 123 add, student, 3, 123 add, student, 4, 123 add, student, 5, 123 Test steps Enable, and configure the Flat File enrolment plugin Enable notifications for each user Set the path to the CSV file mentioned above Run the cron and make sure The line "Running cron for enrol_flatfile..." is present but no errors occured The users have been enrolled on the course An email has been sent to the admin, and the students You might have to run the cron several times before the enrol_flatfile cron line appears Save your CSV file because it will be deleted after execution
    • Workaround:
      Hide

      Use another method for enrolment.

      Show
      Use another method for enrolment.
    • Affected Branches:
      MOODLE_22_STABLE, MOODLE_23_STABLE
    • Fixed Branches:
      MOODLE_22_STABLE, MOODLE_23_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-30871-master
    • Rank:
      33875

      Description

      When trying to use the flat file (CSV) plug in for course enrollments in Moodle 2.2, the following message is generated during the cron script:

      Running enrol crons if required...
      Running cron for enrol_flatfile...
      !!! Coding error detected, it must be fixed by a programmer: Could not load preference {$a}. Does the component and name you supplied to message_send() match a row in message_providers? Message providers must appear in the database so users can configure how they will be notified when they receive messages. !!!

        Activity

        Hide
        Brian King added a comment -

        Hi,

        I saw this too. The error message you see is not the real error, it's an error that occurs when trying to display the real error message.

        In my case, the real error message was "The text file you are using for file-based enrolments
        (/tmp/foo.csv) can not be deleted by the cron process. This usually
        means the permissions are wrong on it. Please fix the permissions so
        that Moodle can delete the file, otherwise it might be processed
        repeatedly."

        I changed the owner of the file to be the apache user (chmod 666 was not enough), and then cron ran through successfully.

        By putting a "debug_print_backtrace();" statement in lib/messagelib.php just above the line "throw new coding_exception($preferrormsg,'blah');", I get this output:

        #0  message_send(stdClass Object ([modulename] => moodle,[component] => course,[name] => flatfile_enrolment,[userfrom] => stdClass Object ([id] => 2,[auth] => manual,[confirmed] => 1,[policyagreed] => 0,[deleted] => 0,[suspended] => 0,[mnethostid] => 1,[username] => admin,[password] => nono,[idnumber] => ,[firstname] => Admin,[lastname] => User,[email] => none@example.com,[emailstop] => 0,[icq] => ,[skype] => ,[yahoo] => ,[aim] => ,[msn] => ,[phone1] => ,[phone2] => ,[institution] => ,[department] => ,[address] => ,[city] => Fribourg,[country] => CH,[lang] => en,[theme] => ,[timezone] => 99,[firstaccess] => 1318424785,[lastaccess] => 1326378224,[lastlogin] => 1326208594,[currentlogin] => 1326375404,[lastip] => 127.0.0.1,[secret] => blahfoobar,[picture] => 0,[url] => ,[description] => ,[descriptionformat] => 0,[mailformat] => 1,[maildigest] => 0,[maildisplay] => 1,[htmleditor] => 1,[ajax] => 1,[autosubscribe] => 1,[trackforums] => 0,[timecreated] => 0,[timemodified] => 1318424785,[trustbitmask] => 0,[imagealt] => ,[screenreader] => 0),[userto] => stdClass Object ([id] => 2,[auth] => manual,[confirmed] => 1,[policyagreed] => 0,[deleted] => 0,[suspended] => 0,[mnethostid] => 1,[username] => admin,[password] => nonono,[idnumber] => ,[firstname] => Admin,[lastname] => User,[email] => none@example.com,[emailstop] => 0,[icq] => ,[skype] => ,[yahoo] => ,[aim] => ,[msn] => ,[phone1] => ,[phone2] => ,[institution] => ,[department] => ,[address] => ,[city] => Fribourg,[country] => CH,[lang] => en,[theme] => ,[timezone] => 99,[firstaccess] => 1318424785,[lastaccess] => 1326378224,[lastlogin] => 1326208594,[currentlogin] => 1326375404,[lastip] => 127.0.0.1,[secret] => right,[picture] => 0,[url] => ,[description] => ,[descriptionformat] => 0,[mailformat] => 1,[maildigest] => 0,[maildisplay] => 1,[htmleditor] => 1,[ajax] => 1,[autosubscribe] => 1,[trackforums] => 0,[timecreated] => 0,[timemodified] => 1318424785,[trustbitmask] => 0,[imagealt] => ,[screenreader] => 0),[subject] => Important error: Enrolment file,[fullmessage] => The text file you are using for file-based enrolments (/tmp/foo.csv) can not be deleted by the cron process.  This usually means the permissions are wrong on it.  Please fix the permissions so that Moodle can delete the file, otherwise it might be processed repeatedly.,[fullmessageformat] => 2,[fullmessagehtml] => ,[smallmessage] => )) called at [/private/var/www/postmoodle/enrol/flatfile/lib.php:163]
        #1  enrol_flatfile_plugin->process_file() called at [/private/var/www/postmoodle/enrol/flatfile/lib.php:54]
        #2  enrol_flatfile_plugin->cron() called at [/private/var/www/postmoodle/lib/cronlib.php:239]
        #3  cron_run() called at [/private/var/www/postmoodle/admin/cron.php:85]
        
        Show
        Brian King added a comment - Hi, I saw this too. The error message you see is not the real error, it's an error that occurs when trying to display the real error message. In my case, the real error message was "The text file you are using for file-based enrolments (/tmp/foo.csv) can not be deleted by the cron process. This usually means the permissions are wrong on it. Please fix the permissions so that Moodle can delete the file, otherwise it might be processed repeatedly." I changed the owner of the file to be the apache user (chmod 666 was not enough), and then cron ran through successfully. By putting a "debug_print_backtrace();" statement in lib/messagelib.php just above the line "throw new coding_exception($preferrormsg,'blah');", I get this output: #0 message_send(stdClass Object ([modulename] => moodle,[component] => course,[name] => flatfile_enrolment,[userfrom] => stdClass Object ([id] => 2,[auth] => manual,[confirmed] => 1,[policyagreed] => 0,[deleted] => 0,[suspended] => 0,[mnethostid] => 1,[username] => admin,[password] => nono,[idnumber] => ,[firstname] => Admin,[lastname] => User,[email] => none@example.com,[emailstop] => 0,[icq] => ,[skype] => ,[yahoo] => ,[aim] => ,[msn] => ,[phone1] => ,[phone2] => ,[institution] => ,[department] => ,[address] => ,[city] => Fribourg,[country] => CH,[lang] => en,[theme] => ,[timezone] => 99,[firstaccess] => 1318424785,[lastaccess] => 1326378224,[lastlogin] => 1326208594,[currentlogin] => 1326375404,[lastip] => 127.0.0.1,[secret] => blahfoobar,[picture] => 0,[url] => ,[description] => ,[descriptionformat] => 0,[mailformat] => 1,[maildigest] => 0,[maildisplay] => 1,[htmleditor] => 1,[ajax] => 1,[autosubscribe] => 1,[trackforums] => 0,[timecreated] => 0,[timemodified] => 1318424785,[trustbitmask] => 0,[imagealt] => ,[screenreader] => 0),[userto] => stdClass Object ([id] => 2,[auth] => manual,[confirmed] => 1,[policyagreed] => 0,[deleted] => 0,[suspended] => 0,[mnethostid] => 1,[username] => admin,[password] => nonono,[idnumber] => ,[firstname] => Admin,[lastname] => User,[email] => none@example.com,[emailstop] => 0,[icq] => ,[skype] => ,[yahoo] => ,[aim] => ,[msn] => ,[phone1] => ,[phone2] => ,[institution] => ,[department] => ,[address] => ,[city] => Fribourg,[country] => CH,[lang] => en,[theme] => ,[timezone] => 99,[firstaccess] => 1318424785,[lastaccess] => 1326378224,[lastlogin] => 1326208594,[currentlogin] => 1326375404,[lastip] => 127.0.0.1,[secret] => right,[picture] => 0,[url] => ,[description] => ,[descriptionformat] => 0,[mailformat] => 1,[maildigest] => 0,[maildisplay] => 1,[htmleditor] => 1,[ajax] => 1,[autosubscribe] => 1,[trackforums] => 0,[timecreated] => 0,[timemodified] => 1318424785,[trustbitmask] => 0,[imagealt] => ,[screenreader] => 0),[subject] => Important error: Enrolment file,[fullmessage] => The text file you are using for file-based enrolments (/tmp/foo.csv) can not be deleted by the cron process. This usually means the permissions are wrong on it. Please fix the permissions so that Moodle can delete the file, otherwise it might be processed repeatedly.,[fullmessageformat] => 2,[fullmessagehtml] => ,[smallmessage] => )) called at [/private/var/www/postmoodle/enrol/flatfile/lib.php:163] #1 enrol_flatfile_plugin->process_file() called at [/private/var/www/postmoodle/enrol/flatfile/lib.php:54] #2 enrol_flatfile_plugin->cron() called at [/private/var/www/postmoodle/lib/cronlib.php:239] #3 cron_run() called at [/private/var/www/postmoodle/admin/cron.php:85]
        Hide
        Michael Mathers added a comment - - edited

        See this issue: http://tracker.moodle.org/browse/MDL-27059

        Here is a better fix than previous alternatives. This creates the email provider and sets the defaults so you do not have to set the preference for each user. The fix only requires inserting a few records into the Moodle database. The only thing missing is that the "nice" string description doesn't exist and the setting is displayed as [[messageprovider:flatfile_enrolment]]

        insert into mdl_message_providers (name, component) values ('flatfile_enrolment','course');
        
        insert into mdl_config_plugins (plugin, name, value) values ('message','email_provider_course_flatfile_enrolment_permitted', 'permitted');
        insert into mdl_config_plugins (plugin, name, value) values ('message','jabber_provider_course_flatfile_enrolment_permitted', 'permitted');
        insert into mdl_config_plugins (plugin, name, value) values ('message','popup_provider_course_flatfile_enrolment_permitted', 'permitted');
        
        insert into mdl_config_plugins (plugin, name, value) values ('message','message_provider_course_flatfile_enrolment_loggedin', 'email');
        insert into mdl_config_plugins (plugin, name, value) values ('message','message_provider_course_flatfile_enrolment_loggedoff', 'email');
        

        Now if you navigate to 'Plugins --> Message outputs --> Default message outputs' you will see the setting that will be used as default for every user. And, most importantly, the flatfile enrollment will no longer fail with the email options enabled.

        Show
        Michael Mathers added a comment - - edited See this issue: http://tracker.moodle.org/browse/MDL-27059 Here is a better fix than previous alternatives. This creates the email provider and sets the defaults so you do not have to set the preference for each user. The fix only requires inserting a few records into the Moodle database. The only thing missing is that the "nice" string description doesn't exist and the setting is displayed as [[messageprovider:flatfile_enrolment]] insert into mdl_message_providers (name, component) values ('flatfile_enrolment','course'); insert into mdl_config_plugins (plugin, name, value) values ('message','email_provider_course_flatfile_enrolment_permitted', 'permitted'); insert into mdl_config_plugins (plugin, name, value) values ('message','jabber_provider_course_flatfile_enrolment_permitted', 'permitted'); insert into mdl_config_plugins (plugin, name, value) values ('message','popup_provider_course_flatfile_enrolment_permitted', 'permitted'); insert into mdl_config_plugins (plugin, name, value) values ('message','message_provider_course_flatfile_enrolment_loggedin', 'email'); insert into mdl_config_plugins (plugin, name, value) values ('message','message_provider_course_flatfile_enrolment_loggedoff', 'email'); Now if you navigate to 'Plugins --> Message outputs --> Default message outputs' you will see the setting that will be used as default for every user. And, most importantly, the flatfile enrollment will no longer fail with the email options enabled.
        Hide
        Susan Montgomery added a comment - - edited

        Hi Michael, tried your fix, still getting the same error. this is on our 2.3 instance. Doesn't seem to matter whether we have e-mail students/teachers/admin checked in the settings or not.

        Show
        Susan Montgomery added a comment - - edited Hi Michael, tried your fix, still getting the same error. this is on our 2.3 instance. Doesn't seem to matter whether we have e-mail students/teachers/admin checked in the settings or not.
        Hide
        Michael Mathers added a comment -

        This may be a different problem. I have not ever tried to run Moodle 2.3 either. Specifically my issue fixes (at least for me) a problem that only appeared when the email options were checked. When the email options were unchecked the flat-file enrollment worked normally. If that was not the case for you then you may have a different (or additional) issue.

        If you do get it working without the email options, but my fix still does not work, one alternative might be (if you have shell access) is to have your cron job email you the output. This will include the flat-file enrollment output that you would normally get in the email (and a bunch of other stuff). Unfortunately, this won't send welcome notices to your students, however.

        I wish I could be more helpful.

        Show
        Michael Mathers added a comment - This may be a different problem. I have not ever tried to run Moodle 2.3 either. Specifically my issue fixes (at least for me) a problem that only appeared when the email options were checked. When the email options were unchecked the flat-file enrollment worked normally. If that was not the case for you then you may have a different (or additional) issue. If you do get it working without the email options, but my fix still does not work, one alternative might be (if you have shell access) is to have your cron job email you the output. This will include the flat-file enrollment output that you would normally get in the email (and a bunch of other stuff). Unfortunately, this won't send welcome notices to your students, however. I wish I could be more helpful.
        Hide
        Susan Montgomery added a comment -

        It's the same on our 2.2.3. We don't have any e-mail checked, but getting the error, plus it stops cron, so no other updates, etc happen.

        Show
        Susan Montgomery added a comment - It's the same on our 2.2.3. We don't have any e-mail checked, but getting the error, plus it stops cron, so no other updates, etc happen.
        Hide
        Koen Roggemans added a comment - - edited

        Same problem in 2.3.2
        The workaround from Michael Mathers worked for me

        Show
        Koen Roggemans added a comment - - edited Same problem in 2.3.2 The workaround from Michael Mathers worked for me
        Hide
        Debbie McDonald added a comment -

        We had the same problem, when email notification check boxes were checked the flatfile would not run. When the patch above was applied, it fixed the problem. Thank you Michael!!!

        Show
        Debbie McDonald added a comment - We had the same problem, when email notification check boxes were checked the flatfile would not run. When the patch above was applied, it fixed the problem. Thank you Michael!!!
        Hide
        Frédéric Massart added a comment -

        This patch fixes the issue. In fact, the preference was not found because the component was not properly defined in the enrolment plugin.

        • I have updated the component name passed to message_send()
        • I removed the use of get_string() in coding_exception()
        • (master only) I removed the original string used in the exception. It is not used anywhere else.
        Show
        Frédéric Massart added a comment - This patch fixes the issue. In fact, the preference was not found because the component was not properly defined in the enrolment plugin. I have updated the component name passed to message_send() I removed the use of get_string() in coding_exception() (master only) I removed the original string used in the exception. It is not used anywhere else.
        Hide
        Rajesh Taneja added a comment -

        Patch looks good Fred, pushing for integration review.
        [y] Syntax
        [-] Output
        [y] Whitespace
        [y] Language
        [-] Databases
        [-] Testing
        [-] Security
        [-] Documentation
        [y] Git
        [y] Sanity check

        Show
        Rajesh Taneja added a comment - Patch looks good Fred, pushing for integration review. [y] Syntax [-] Output [y] Whitespace [y] Language [-] Databases [-] Testing [-] Security [-] Documentation [y] Git [y] Sanity check
        Hide
        Dan Poltawski added a comment -

        The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.

        TIA and ciao

        Show
        Dan Poltawski added a comment - The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week. TIA and ciao
        Hide
        Dan Poltawski added a comment -

        Integrated, thanks Fred.

        Show
        Dan Poltawski added a comment - Integrated, thanks Fred.
        Hide
        Michael de Raadt added a comment -

        Test result: Success!

        Show
        Michael de Raadt added a comment - Test result: Success!
        Hide
        Dan Poltawski added a comment -

        Congratulations! Another bug solved.. only another 7330 to go, thanks for contributing to contributing to 0.8% of all bugs being fixed this week!

        ciao
        Dan

        Show
        Dan Poltawski added a comment - Congratulations! Another bug solved.. only another 7330 to go, thanks for contributing to contributing to 0.8% of all bugs being fixed this week! ciao Dan

          People

          • Votes:
            12 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: