Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-30871

Flat File (CSV) plug in message_provider issue

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2, 2.3.2
    • Fix Version/s: 2.2.7, 2.3.4
    • Component/s: Enrolments
    • Labels:
    • Environment:
      Ubuntu
    • Database:
      MySQL
    • Testing Instructions:
      Hide

      Test pre-requisites

      • 1 course with ID number 123 (no users enrolled)
      • 5 users with ID number 1 to 5
      • A CSV file as following

      add, student, 1, 123
      add, student, 2, 123
      add, student, 3, 123
      add, student, 4, 123
      add, student, 5, 123

      Test steps

      1. Enable, and configure the Flat File enrolment plugin
        • Enable notifications for each user
        • Set the path to the CSV file mentioned above
      2. Run the cron and make sure
        • The line "Running cron for enrol_flatfile..." is present but no errors occured
        • The users have been enrolled on the course
        • An email has been sent to the admin, and the students

      You might have to run the cron several times before the enrol_flatfile cron line appears
      Save your CSV file because it will be deleted after execution

      Show
      Test pre-requisites 1 course with ID number 123 (no users enrolled) 5 users with ID number 1 to 5 A CSV file as following add, student, 1, 123 add, student, 2, 123 add, student, 3, 123 add, student, 4, 123 add, student, 5, 123 Test steps Enable, and configure the Flat File enrolment plugin Enable notifications for each user Set the path to the CSV file mentioned above Run the cron and make sure The line "Running cron for enrol_flatfile..." is present but no errors occured The users have been enrolled on the course An email has been sent to the admin, and the students You might have to run the cron several times before the enrol_flatfile cron line appears Save your CSV file because it will be deleted after execution
    • Workaround:
      Hide

      Use another method for enrolment.

      Show
      Use another method for enrolment.
    • Affected Branches:
      MOODLE_22_STABLE, MOODLE_23_STABLE
    • Fixed Branches:
      MOODLE_22_STABLE, MOODLE_23_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-30871-master

      Description

      When trying to use the flat file (CSV) plug in for course enrollments in Moodle 2.2, the following message is generated during the cron script:

      Running enrol crons if required...
      Running cron for enrol_flatfile...
      !!! Coding error detected, it must be fixed by a programmer: Could not load preference {$a}. Does the component and name you supplied to message_send() match a row in message_providers? Message providers must appear in the database so users can configure how they will be notified when they receive messages. !!!

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            brianking Brian King added a comment -

            Hi,

            I saw this too. The error message you see is not the real error, it's an error that occurs when trying to display the real error message.

            In my case, the real error message was "The text file you are using for file-based enrolments
            (/tmp/foo.csv) can not be deleted by the cron process. This usually
            means the permissions are wrong on it. Please fix the permissions so
            that Moodle can delete the file, otherwise it might be processed
            repeatedly."

            I changed the owner of the file to be the apache user (chmod 666 was not enough), and then cron ran through successfully.

            By putting a "debug_print_backtrace();" statement in lib/messagelib.php just above the line "throw new coding_exception($preferrormsg,'blah');", I get this output:

            #0  message_send(stdClass Object ([modulename] => moodle,[component] => course,[name] => flatfile_enrolment,[userfrom] => stdClass Object ([id] => 2,[auth] => manual,[confirmed] => 1,[policyagreed] => 0,[deleted] => 0,[suspended] => 0,[mnethostid] => 1,[username] => admin,[password] => nono,[idnumber] => ,[firstname] => Admin,[lastname] => User,[email] => none@example.com,[emailstop] => 0,[icq] => ,[skype] => ,[yahoo] => ,[aim] => ,[msn] => ,[phone1] => ,[phone2] => ,[institution] => ,[department] => ,[address] => ,[city] => Fribourg,[country] => CH,[lang] => en,[theme] => ,[timezone] => 99,[firstaccess] => 1318424785,[lastaccess] => 1326378224,[lastlogin] => 1326208594,[currentlogin] => 1326375404,[lastip] => 127.0.0.1,[secret] => blahfoobar,[picture] => 0,[url] => ,[description] => ,[descriptionformat] => 0,[mailformat] => 1,[maildigest] => 0,[maildisplay] => 1,[htmleditor] => 1,[ajax] => 1,[autosubscribe] => 1,[trackforums] => 0,[timecreated] => 0,[timemodified] => 1318424785,[trustbitmask] => 0,[imagealt] => ,[screenreader] => 0),[userto] => stdClass Object ([id] => 2,[auth] => manual,[confirmed] => 1,[policyagreed] => 0,[deleted] => 0,[suspended] => 0,[mnethostid] => 1,[username] => admin,[password] => nonono,[idnumber] => ,[firstname] => Admin,[lastname] => User,[email] => none@example.com,[emailstop] => 0,[icq] => ,[skype] => ,[yahoo] => ,[aim] => ,[msn] => ,[phone1] => ,[phone2] => ,[institution] => ,[department] => ,[address] => ,[city] => Fribourg,[country] => CH,[lang] => en,[theme] => ,[timezone] => 99,[firstaccess] => 1318424785,[lastaccess] => 1326378224,[lastlogin] => 1326208594,[currentlogin] => 1326375404,[lastip] => 127.0.0.1,[secret] => right,[picture] => 0,[url] => ,[description] => ,[descriptionformat] => 0,[mailformat] => 1,[maildigest] => 0,[maildisplay] => 1,[htmleditor] => 1,[ajax] => 1,[autosubscribe] => 1,[trackforums] => 0,[timecreated] => 0,[timemodified] => 1318424785,[trustbitmask] => 0,[imagealt] => ,[screenreader] => 0),[subject] => Important error: Enrolment file,[fullmessage] => The text file you are using for file-based enrolments (/tmp/foo.csv) can not be deleted by the cron process.  This usually means the permissions are wrong on it.  Please fix the permissions so that Moodle can delete the file, otherwise it might be processed repeatedly.,[fullmessageformat] => 2,[fullmessagehtml] => ,[smallmessage] => )) called at [/private/var/www/postmoodle/enrol/flatfile/lib.php:163]
            #1  enrol_flatfile_plugin->process_file() called at [/private/var/www/postmoodle/enrol/flatfile/lib.php:54]
            #2  enrol_flatfile_plugin->cron() called at [/private/var/www/postmoodle/lib/cronlib.php:239]
            #3  cron_run() called at [/private/var/www/postmoodle/admin/cron.php:85]

            Show
            brianking Brian King added a comment - Hi, I saw this too. The error message you see is not the real error, it's an error that occurs when trying to display the real error message. In my case, the real error message was "The text file you are using for file-based enrolments (/tmp/foo.csv) can not be deleted by the cron process. This usually means the permissions are wrong on it. Please fix the permissions so that Moodle can delete the file, otherwise it might be processed repeatedly." I changed the owner of the file to be the apache user (chmod 666 was not enough), and then cron ran through successfully. By putting a "debug_print_backtrace();" statement in lib/messagelib.php just above the line "throw new coding_exception($preferrormsg,'blah');", I get this output: #0 message_send(stdClass Object ([modulename] => moodle,[component] => course,[name] => flatfile_enrolment,[userfrom] => stdClass Object ([id] => 2,[auth] => manual,[confirmed] => 1,[policyagreed] => 0,[deleted] => 0,[suspended] => 0,[mnethostid] => 1,[username] => admin,[password] => nono,[idnumber] => ,[firstname] => Admin,[lastname] => User,[email] => none@example.com,[emailstop] => 0,[icq] => ,[skype] => ,[yahoo] => ,[aim] => ,[msn] => ,[phone1] => ,[phone2] => ,[institution] => ,[department] => ,[address] => ,[city] => Fribourg,[country] => CH,[lang] => en,[theme] => ,[timezone] => 99,[firstaccess] => 1318424785,[lastaccess] => 1326378224,[lastlogin] => 1326208594,[currentlogin] => 1326375404,[lastip] => 127.0.0.1,[secret] => blahfoobar,[picture] => 0,[url] => ,[description] => ,[descriptionformat] => 0,[mailformat] => 1,[maildigest] => 0,[maildisplay] => 1,[htmleditor] => 1,[ajax] => 1,[autosubscribe] => 1,[trackforums] => 0,[timecreated] => 0,[timemodified] => 1318424785,[trustbitmask] => 0,[imagealt] => ,[screenreader] => 0),[userto] => stdClass Object ([id] => 2,[auth] => manual,[confirmed] => 1,[policyagreed] => 0,[deleted] => 0,[suspended] => 0,[mnethostid] => 1,[username] => admin,[password] => nonono,[idnumber] => ,[firstname] => Admin,[lastname] => User,[email] => none@example.com,[emailstop] => 0,[icq] => ,[skype] => ,[yahoo] => ,[aim] => ,[msn] => ,[phone1] => ,[phone2] => ,[institution] => ,[department] => ,[address] => ,[city] => Fribourg,[country] => CH,[lang] => en,[theme] => ,[timezone] => 99,[firstaccess] => 1318424785,[lastaccess] => 1326378224,[lastlogin] => 1326208594,[currentlogin] => 1326375404,[lastip] => 127.0.0.1,[secret] => right,[picture] => 0,[url] => ,[description] => ,[descriptionformat] => 0,[mailformat] => 1,[maildigest] => 0,[maildisplay] => 1,[htmleditor] => 1,[ajax] => 1,[autosubscribe] => 1,[trackforums] => 0,[timecreated] => 0,[timemodified] => 1318424785,[trustbitmask] => 0,[imagealt] => ,[screenreader] => 0),[subject] => Important error: Enrolment file,[fullmessage] => The text file you are using for file-based enrolments (/tmp/foo.csv) can not be deleted by the cron process. This usually means the permissions are wrong on it. Please fix the permissions so that Moodle can delete the file, otherwise it might be processed repeatedly.,[fullmessageformat] => 2,[fullmessagehtml] => ,[smallmessage] => )) called at [/private/var/www/postmoodle/enrol/flatfile/lib.php:163] #1 enrol_flatfile_plugin->process_file() called at [/private/var/www/postmoodle/enrol/flatfile/lib.php:54] #2 enrol_flatfile_plugin->cron() called at [/private/var/www/postmoodle/lib/cronlib.php:239] #3 cron_run() called at [/private/var/www/postmoodle/admin/cron.php:85]
            Hide
            mbmather Michael Mathers added a comment - - edited

            See this issue: http://tracker.moodle.org/browse/MDL-27059

            Here is a better fix than previous alternatives. This creates the email provider and sets the defaults so you do not have to set the preference for each user. The fix only requires inserting a few records into the Moodle database. The only thing missing is that the "nice" string description doesn't exist and the setting is displayed as [[messageprovider:flatfile_enrolment]]

            insert into mdl_message_providers (name, component) values ('flatfile_enrolment','course');
             
            insert into mdl_config_plugins (plugin, name, value) values ('message','email_provider_course_flatfile_enrolment_permitted', 'permitted');
            insert into mdl_config_plugins (plugin, name, value) values ('message','jabber_provider_course_flatfile_enrolment_permitted', 'permitted');
            insert into mdl_config_plugins (plugin, name, value) values ('message','popup_provider_course_flatfile_enrolment_permitted', 'permitted');
             
            insert into mdl_config_plugins (plugin, name, value) values ('message','message_provider_course_flatfile_enrolment_loggedin', 'email');
            insert into mdl_config_plugins (plugin, name, value) values ('message','message_provider_course_flatfile_enrolment_loggedoff', 'email');

            Now if you navigate to 'Plugins --> Message outputs --> Default message outputs' you will see the setting that will be used as default for every user. And, most importantly, the flatfile enrollment will no longer fail with the email options enabled.

            Show
            mbmather Michael Mathers added a comment - - edited See this issue: http://tracker.moodle.org/browse/MDL-27059 Here is a better fix than previous alternatives. This creates the email provider and sets the defaults so you do not have to set the preference for each user. The fix only requires inserting a few records into the Moodle database. The only thing missing is that the "nice" string description doesn't exist and the setting is displayed as [[messageprovider:flatfile_enrolment]] insert into mdl_message_providers ( name , component) values ( 'flatfile_enrolment' , 'course' );   insert into mdl_config_plugins (plugin, name , value) values ( 'message' , 'email_provider_course_flatfile_enrolment_permitted' , 'permitted' ); insert into mdl_config_plugins (plugin, name , value) values ( 'message' , 'jabber_provider_course_flatfile_enrolment_permitted' , 'permitted' ); insert into mdl_config_plugins (plugin, name , value) values ( 'message' , 'popup_provider_course_flatfile_enrolment_permitted' , 'permitted' );   insert into mdl_config_plugins (plugin, name , value) values ( 'message' , 'message_provider_course_flatfile_enrolment_loggedin' , 'email' ); insert into mdl_config_plugins (plugin, name , value) values ( 'message' , 'message_provider_course_flatfile_enrolment_loggedoff' , 'email' ); Now if you navigate to 'Plugins --> Message outputs --> Default message outputs' you will see the setting that will be used as default for every user. And, most importantly, the flatfile enrollment will no longer fail with the email options enabled.
            Hide
            suobrien Susan Montgomery added a comment - - edited

            Hi Michael, tried your fix, still getting the same error. this is on our 2.3 instance. Doesn't seem to matter whether we have e-mail students/teachers/admin checked in the settings or not.

            Show
            suobrien Susan Montgomery added a comment - - edited Hi Michael, tried your fix, still getting the same error. this is on our 2.3 instance. Doesn't seem to matter whether we have e-mail students/teachers/admin checked in the settings or not.
            Hide
            mbmather Michael Mathers added a comment -

            This may be a different problem. I have not ever tried to run Moodle 2.3 either. Specifically my issue fixes (at least for me) a problem that only appeared when the email options were checked. When the email options were unchecked the flat-file enrollment worked normally. If that was not the case for you then you may have a different (or additional) issue.

            If you do get it working without the email options, but my fix still does not work, one alternative might be (if you have shell access) is to have your cron job email you the output. This will include the flat-file enrollment output that you would normally get in the email (and a bunch of other stuff). Unfortunately, this won't send welcome notices to your students, however.

            I wish I could be more helpful.

            Show
            mbmather Michael Mathers added a comment - This may be a different problem. I have not ever tried to run Moodle 2.3 either. Specifically my issue fixes (at least for me) a problem that only appeared when the email options were checked. When the email options were unchecked the flat-file enrollment worked normally. If that was not the case for you then you may have a different (or additional) issue. If you do get it working without the email options, but my fix still does not work, one alternative might be (if you have shell access) is to have your cron job email you the output. This will include the flat-file enrollment output that you would normally get in the email (and a bunch of other stuff). Unfortunately, this won't send welcome notices to your students, however. I wish I could be more helpful.
            Hide
            suobrien Susan Montgomery added a comment -

            It's the same on our 2.2.3. We don't have any e-mail checked, but getting the error, plus it stops cron, so no other updates, etc happen.

            Show
            suobrien Susan Montgomery added a comment - It's the same on our 2.2.3. We don't have any e-mail checked, but getting the error, plus it stops cron, so no other updates, etc happen.
            Hide
            koen Koen Roggemans added a comment - - edited

            Same problem in 2.3.2
            The workaround from Michael Mathers worked for me

            Show
            koen Koen Roggemans added a comment - - edited Same problem in 2.3.2 The workaround from Michael Mathers worked for me
            Hide
            dmcdonald Debbie McDonald added a comment -

            We had the same problem, when email notification check boxes were checked the flatfile would not run. When the patch above was applied, it fixed the problem. Thank you Michael!!!

            Show
            dmcdonald Debbie McDonald added a comment - We had the same problem, when email notification check boxes were checked the flatfile would not run. When the patch above was applied, it fixed the problem. Thank you Michael!!!
            Hide
            fred Frédéric Massart added a comment -

            This patch fixes the issue. In fact, the preference was not found because the component was not properly defined in the enrolment plugin.

            • I have updated the component name passed to message_send()
            • I removed the use of get_string() in coding_exception()
            • (master only) I removed the original string used in the exception. It is not used anywhere else.
            Show
            fred Frédéric Massart added a comment - This patch fixes the issue. In fact, the preference was not found because the component was not properly defined in the enrolment plugin. I have updated the component name passed to message_send() I removed the use of get_string() in coding_exception() (master only) I removed the original string used in the exception. It is not used anywhere else.
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Patch looks good Fred, pushing for integration review.
            [y] Syntax
            [-] Output
            [y] Whitespace
            [y] Language
            [-] Databases
            [-] Testing
            [-] Security
            [-] Documentation
            [y] Git
            [y] Sanity check

            Show
            rajeshtaneja Rajesh Taneja added a comment - Patch looks good Fred, pushing for integration review. [y] Syntax [-] Output [y] Whitespace [y] Language [-] Databases [-] Testing [-] Security [-] Documentation [y] Git [y] Sanity check
            Hide
            poltawski Dan Poltawski added a comment -

            The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.

            TIA and ciao

            Show
            poltawski Dan Poltawski added a comment - The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week. TIA and ciao
            Hide
            poltawski Dan Poltawski added a comment -

            Integrated, thanks Fred.

            Show
            poltawski Dan Poltawski added a comment - Integrated, thanks Fred.
            Hide
            salvetore Michael de Raadt added a comment -

            Test result: Success!

            Show
            salvetore Michael de Raadt added a comment - Test result: Success!
            Hide
            poltawski Dan Poltawski added a comment -

            Congratulations! Another bug solved.. only another 7330 to go, thanks for contributing to contributing to 0.8% of all bugs being fixed this week!

            ciao
            Dan

            Show
            poltawski Dan Poltawski added a comment - Congratulations! Another bug solved.. only another 7330 to go, thanks for contributing to contributing to 0.8% of all bugs being fixed this week! ciao Dan

              People

              • Votes:
                12 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  14/Jan/13