Moodle
  1. Moodle
  2. MDL-12321

Forum digest sending breaks the cron task and don't send the messages

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 1.5, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.6, 1.6.1, 1.6.2, 1.6.3, 1.6.4, 1.6.5, 1.7, 1.7.1, 1.7.2, 1.7.3, 1.8, 1.8.1, 1.8.2, 1.8.3
    • Fix Version/s: 1.8.4
    • Component/s: Forum, Performance
    • Labels:
      None
    • Environment:
      Debian 4.0 / Apache 2.2.3 / PHP 5.2.0 / MySQl 5.0.32
    • Database:
      MySQL
    • Affected Branches:
      MOODLE_15_STABLE, MOODLE_16_STABLE, MOODLE_17_STABLE, MOODLE_18_STABLE
    • Fixed Branches:
      MOODLE_18_STABLE
    • Rank:
      29863

      Description

      We have a Moodle installation with more than 3000 users and 300 courses.
      Most users want the forum digest sending feature but it doesn't work well for us.
      Every attemp with the cron task hangs without explanation when trying to send digests.
      The cron report ends at "Sending forum digests:" line
      The normal message sending works well.
      After increasing "memory_limit" to 256M and "max_execution_time" to 600 the problem persists. The script uses a lot of CPU and time without success.
      Another installation with the same code in the same server using the same SMTP, but with fewer users works well.

      1. patch-12231.txt
        23 kB
        Mathieu Petit-Clair
      2. patch-12321.txt
        28 kB
        Mathieu Petit-Clair
      3. patch-12321.txt
        28 kB
        Mathieu Petit-Clair

        Issue Links

          Activity

          Hide
          Martin Dougiamas added a comment -

          Hi, can you please use the VERY latest 1.8.3+ CVS version and tell us if it's better?

          If not, Mat will help debug the situation.

          Show
          Martin Dougiamas added a comment - Hi, can you please use the VERY latest 1.8.3+ CVS version and tell us if it's better? If not, Mat will help debug the situation.
          Hide
          Wen Hao Chuang added a comment - - edited

          Martin just curious what improvement did 1.8.3+ make that led you to believe that it might be better for this? We (SFSU) also encountered this problem although for our main production site we are still running a heavily customized 1.5.4+ version of Moodle... thanks!

          Show
          Wen Hao Chuang added a comment - - edited Martin just curious what improvement did 1.8.3+ make that led you to believe that it might be better for this? We (SFSU) also encountered this problem although for our main production site we are still running a heavily customized 1.5.4+ version of Moodle... thanks!
          Hide
          Generazion Consulting S.L. added a comment - - edited

          There was no luck. Using the moodle available this morning with this command

          cvs -z3 -d:pserver:anonymous@es.cvs.moodle.org:/cvsroot/moodle co -r MOODLE_18_STABLE moodle

          and raising the memory limit to 512MB we had the next error message after 41 minutes:

          [Wed Nov 28 13:41:05 2007] [notice] child pid 9846 exit signal Segmentation fault (11)

          The cron report message finish with the "Sending forum digests:" line.
          It's too much time. We are on a Dual Opteron Dual Core with 2 Gb of RAM and the load was always below 1.2

          In previous days with 128 MB we had this error message:
          Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 35 bytes) in lib/adodb/adodb.inc.php on line 29

          With 192 MB:
          Fatal error: Allowed memory size of 201326592 bytes exhausted (tried to allocate 32 bytes) in lib/dmllib.php on line 714

          With 256 MB:
          Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 42 bytes) in lib/dmllib.php on line 714

          Show
          Generazion Consulting S.L. added a comment - - edited There was no luck. Using the moodle available this morning with this command cvs -z3 -d:pserver:anonymous@es.cvs.moodle.org:/cvsroot/moodle co -r MOODLE_18_STABLE moodle and raising the memory limit to 512MB we had the next error message after 41 minutes: [Wed Nov 28 13:41:05 2007] [notice] child pid 9846 exit signal Segmentation fault (11) The cron report message finish with the "Sending forum digests:" line. It's too much time. We are on a Dual Opteron Dual Core with 2 Gb of RAM and the load was always below 1.2 In previous days with 128 MB we had this error message: Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 35 bytes) in lib/adodb/adodb.inc.php on line 29 With 192 MB: Fatal error: Allowed memory size of 201326592 bytes exhausted (tried to allocate 32 bytes) in lib/dmllib.php on line 714 With 256 MB: Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 42 bytes) in lib/dmllib.php on line 714
          Hide
          Wen Hao Chuang added a comment -

          If I may I have changed the priority of this ticket to "Critical" as it really made large sites paralyzed - all "forum digest" were not sending out at all! The key file (IMHO) is the /mod/forum/lib.php, and we probably need to look into the function forum_cron(). For this function I compared with v1.536.2.33 (released with Moodle 1.8.2) and the MOST recent v.536.2.40 (2007/11/28), found that there were no change for this function. Just a quick FYI

          Show
          Wen Hao Chuang added a comment - If I may I have changed the priority of this ticket to "Critical" as it really made large sites paralyzed - all "forum digest" were not sending out at all! The key file (IMHO) is the /mod/forum/lib.php, and we probably need to look into the function forum_cron(). For this function I compared with v1.536.2.33 (released with Moodle 1.8.2) and the MOST recent v.536.2.40 (2007/11/28), found that there were no change for this function. Just a quick FYI
          Hide
          Martin Dougiamas added a comment -

          See MDL-11657 and MDL-11606 ... there are some patches there that probably should be backported to 1.8

          Show
          Martin Dougiamas added a comment - See MDL-11657 and MDL-11606 ... there are some patches there that probably should be backported to 1.8
          Hide
          Wen Hao Chuang added a comment -

          by the way we temporarily bump up the memory_limit in this script to 1024MB and it seems to work fine now. I will take a look at the two tickets Martin mentioned. Thanks Martin!

          Show
          Wen Hao Chuang added a comment - by the way we temporarily bump up the memory_limit in this script to 1024MB and it seems to work fine now. I will take a look at the two tickets Martin mentioned. Thanks Martin!
          Hide
          Mathieu Petit-Clair added a comment -

          I'm trying to reproduce your setup here... Can you tell us how many forums you have, and an estimation of how many posts are sent through digests every day? This will let me run some tests after backporting the patches mentionned by Martin.

          Show
          Mathieu Petit-Clair added a comment - I'm trying to reproduce your setup here... Can you tell us how many forums you have, and an estimation of how many posts are sent through digests every day? This will let me run some tests after backporting the patches mentionned by Martin.
          Hide
          Mathieu Petit-Clair added a comment -

          This is a backport of the patches mentionned earlier by Martin. This patch applies to moodle-1.8.3+, on November 30. It was made against cvs (MOODLE_18_STABLE) today.

          It also includes a backport of the modifications made to print_user_picture() (in lib/weblib.php) in 1.9.

          Show
          Mathieu Petit-Clair added a comment - This is a backport of the patches mentionned earlier by Martin. This patch applies to moodle-1.8.3+, on November 30. It was made against cvs (MOODLE_18_STABLE) today. It also includes a backport of the modifications made to print_user_picture() (in lib/weblib.php) in 1.9.
          Hide
          Generazion Consulting S.L. added a comment -

          Our bigger Moodle has 1261 forums. When all of our users have the digest option disabled we are sending more than 10.000 messages every day. But due to a management imposition we need to force everybody to use the digest feature.

          I am going to try your patch. Thank you for your efforts.

          Show
          Generazion Consulting S.L. added a comment - Our bigger Moodle has 1261 forums. When all of our users have the digest option disabled we are sending more than 10.000 messages every day. But due to a management imposition we need to force everybody to use the digest feature. I am going to try your patch. Thank you for your efforts.
          Hide
          Generazion Consulting S.L. added a comment -

          After applying your patch: "PHP Fatal error: Call to undefined function rs_EOF() in mod/forum/lib.php on line 503"

          Show
          Generazion Consulting S.L. added a comment - After applying your patch: "PHP Fatal error: Call to undefined function rs_EOF() in mod/forum/lib.php on line 503"
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Yep, that function was added for Moodle 1.9.

          Under Moodle 1.8 you can replace safely:

          if (!rs_EOF($digestposts_rs)) {

          by:

          if (!$digestposts_rs->EOF)) {

          Note for devs: I wouldn't backport the rs_EOF() function to 1.8

          Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - Yep, that function was added for Moodle 1.9. Under Moodle 1.8 you can replace safely: if (!rs_EOF($digestposts_rs)) { by: if (!$digestposts_rs->EOF)) { Note for devs: I wouldn't backport the rs_EOF() function to 1.8 Ciao
          Hide
          Mathieu Petit-Clair added a comment -

          A new version of the patch, with the fix sent by Eloy.

          Show
          Mathieu Petit-Clair added a comment - A new version of the patch, with the fix sent by Eloy.
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Hi Mathieu,

          just reviewed quickly the patch and... uhm... it's BIG ! And include DB modifications (warning!).

          Looks correct code and I think it can fit into 18_STABLE. Just noticing about it's a big backport, not only modifying digests but other forum parts.

          +1 for MD to decide.

          Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - Hi Mathieu, just reviewed quickly the patch and... uhm... it's BIG ! And include DB modifications (warning!). Looks correct code and I think it can fit into 18_STABLE. Just noticing about it's a big backport, not only modifying digests but other forum parts. +1 for MD to decide. Ciao
          Hide
          Mathieu Petit-Clair added a comment -

          This is a new (and hopefully, final) version of the patch, excluding the database changes and some whitespace that crept in the first files. It also incorporates the fix Eloy sent, for the "->EOF" problem.

          You probably need to use "patch -p1" to apply.

          Show
          Mathieu Petit-Clair added a comment - This is a new (and hopefully, final) version of the patch, excluding the database changes and some whitespace that crept in the first files. It also incorporates the fix Eloy sent, for the "->EOF" problem. You probably need to use "patch -p1" to apply.
          Hide
          Generazion Consulting S.L. added a comment -

          This time it works. It's a very slow process and maybe I need to tune my email server.
          Big thanks to everybody.

          Gustavo

          Show
          Generazion Consulting S.L. added a comment - This time it works. It's a very slow process and maybe I need to tune my email server. Big thanks to everybody. Gustavo
          Hide
          Wen Hao Chuang added a comment -

          quick question, does the final patch works for earlier version of moodle, like 1.7, 1.6, or even 1.5? Many thanks!

          Show
          Wen Hao Chuang added a comment - quick question, does the final patch works for earlier version of moodle, like 1.7, 1.6, or even 1.5? Many thanks!
          Hide
          Mathieu Petit-Clair added a comment -

          I doubt it would apply to earlier versions, considering the amout of changes, but possibly... Which version are you using?

          Show
          Mathieu Petit-Clair added a comment - I doubt it would apply to earlier versions, considering the amout of changes, but possibly... Which version are you using?
          Hide
          Wen Hao Chuang added a comment -

          Mathieu, we are using a customized 1.5.4+ for our main production site. Thanks!

          Show
          Wen Hao Chuang added a comment - Mathieu, we are using a customized 1.5.4+ for our main production site. Thanks!
          Hide
          Mathieu Petit-Clair added a comment -

          The last commit should fix it for version 1.8.3+. The patch doesn't apply to 1.7, and the backport would probably (I had a quick look) require significant work.

          Show
          Mathieu Petit-Clair added a comment - The last commit should fix it for version 1.8.3+. The patch doesn't apply to 1.7, and the backport would probably (I had a quick look) require significant work.

            People

            • Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: