Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-20614

The admin/cron.php does not be finished when global search function is enabled

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.9.5
    • Fix Version/s: None
    • Component/s: Global search
    • Labels:
      None
    • Difficulty:
      Easy
    • Affected Branches:
      MOODLE_19_STABLE

      Description

      Global search is good function for me and other moodlers, so I think heartly. Thanks!

      --------------

      > <!-StartFragment-> Delete: Your E-mail Address (database id = 510486, index id = 1090, moodle instance id = 3743)
      > Delete: Your E-mail Address (database id = 510491, index id = 1091, moodle instance id = 3743)
      > <br />
      > <b>Fatal error</b>: Allowed memory size of 50331648 bytes exhausted (tried to allocate 44 bytes) in <b>C:\xampplite\htdocs\mech\moodle\search\Zend\Search\Lucene.php</b> on line ><b>997</b><br />

      The cron process halted on my server. By the way, I had set 'memory_limit' = 500M in php.ini. 50331648 is 48M.
      I checked and found the cause of this problem. I propose a solution as follows;

      I also think that '48M' is too smll for executing search/delete.php in my site.

      (serach/cron_php5.php)

      (Debug030) means my comment.

      <?php
      /*

      • Moodle global search engine
      • This is a special externalized code for cron handling in PHP5.
      • Should never be called by a php 4.3.0 implementation.
        */
        // (Debug030): Add from here
        // http://www.php.net/manual/ja/function.ini-get.php
        function return_bytes($val) {
        if (is_int($val)) return $val;
        $val = trim($val);
        $last = strtolower($val[strlen($val)-1]);
        switch($last) { case 'g': $val *= 1024; case 'm': $val *= 1024; case 'k': $val *= 1024; }

        return $val;
        }
        // (Debug030): Add to here

      try{
      // overrides php limits
      $maxtimelimit = ini_get('max_execution_time');
      ini_set('max_execution_time', 300);
      $maxmemoryamount = ini_get('memory_limit');
      // (Debug030): Comment out this line
      // ini_set('memory_limit', '48M');
      // (Debug030): Add from here
      $temp_maxmemory = '48M';
      if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory))

      { ini_set('memory_limit', $temp_maxmemory); }

      // (Debug030): Add to here

      mtrace("\n-DELETE---");

        Gliffy Diagrams

          Activity

          Hide
          gnov Garrett Novotny added a comment -

          I am seeing this bug in 1.9.9. If I hide global search cron completes successfully. Here is a tail for my cron...

          Starting index update (additions)...
          Index size before: 1112

          9 modules to search in / 20 modules found.
          0 blocks to search in / 32 blocks found.
          1 additional to search in.
          Checking assignment module for additions.
          Finished assignment.

          Checking chat module for additions.
          No types to add.

          Finished chat.

          Checking data module for additions.
          Finished data.

          Checking forum module for additions.

          Show
          gnov Garrett Novotny added a comment - I am seeing this bug in 1.9.9. If I hide global search cron completes successfully. Here is a tail for my cron... Starting index update (additions)... Index size before: 1112 9 modules to search in / 20 modules found. 0 blocks to search in / 32 blocks found. 1 additional to search in. Checking assignment module for additions. Finished assignment. Checking chat module for additions. No types to add. Finished chat. Checking data module for additions. Finished data. Checking forum module for additions.
          Hide
          brugger Gisele Brugger added a comment -

          Hello,
          I had the same problem using moodle 1.9.10 and windows server 2008 with php limit_memory=1024M
          The first error when execute moodle/admin/cron.php

          PHP Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 44 bytes) in C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\moodle\search\Zend\Search\Lucene.php on line 970

          if i change code adding function

          // (Debug030): Add from here
          // http://www.php.net/manual/ja/function.ini-get.php
          function return_bytes($val) {
          if (is_int($val)) return $val;
          $val = trim($val);
          $last = strtolower($val[strlen($val)-1]);
          switch($last)

          { case 'g': $val *= 1024; case 'm': $val *= 1024; case 'k': $val *= 1024; }

          return $val;
          }
          // (Debug030): Add to here

          i have this error :

          PHP Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 24 bytes) in C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\moodle\search\Zend\Search\Lucene\Storage\File.php on line 268

          I think the script should use the php.ini limit_memory (1024M is more than enough or no ?)

          My resume index search:

          Documents in index 30608
          Deletions in index 1598
          Documents in database 29010
          Documents for 'Assignments' 1737
          Documents for 'Chats' 0
          Documents for 'Databases' 0
          Documents for 'Forums' 7464
          Documents for 'Glossaries' 213
          Documents for 'Labels' 1046
          Documents for 'Lessons' 42
          Documents for 'Resources' 12137
          Documents for 'Wikis' 145

          Any sugestion?

          Thank you

          Show
          brugger Gisele Brugger added a comment - Hello, I had the same problem using moodle 1.9.10 and windows server 2008 with php limit_memory=1024M The first error when execute moodle/admin/cron.php PHP Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 44 bytes) in C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\moodle\search\Zend\Search\Lucene.php on line 970 if i change code adding function // (Debug030): Add from here // http://www.php.net/manual/ja/function.ini-get.php function return_bytes($val) { if (is_int($val)) return $val; $val = trim($val); $last = strtolower($val [strlen($val)-1] ); switch($last) { case 'g': $val *= 1024; case 'm': $val *= 1024; case 'k': $val *= 1024; } return $val; } // (Debug030): Add to here i have this error : PHP Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 24 bytes) in C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\moodle\search\Zend\Search\Lucene\Storage\File.php on line 268 I think the script should use the php.ini limit_memory (1024M is more than enough or no ?) My resume index search: Documents in index 30608 Deletions in index 1598 Documents in database 29010 Documents for 'Assignments' 1737 Documents for 'Chats' 0 Documents for 'Databases' 0 Documents for 'Forums' 7464 Documents for 'Glossaries' 213 Documents for 'Labels' 1046 Documents for 'Lessons' 42 Documents for 'Resources' 12137 Documents for 'Wikis' 145 Any sugestion? Thank you
          Hide
          vf Valery Fremaux added a comment -

          Hi

          I think you have really a lot of stuff to index.

          Unfortunately, the memory is not an infinite resource and the indexer needs
          to open all documents and process all content.

          Indexer can do good stuff when starting a Moodle platform to run as further processing
          of new stuff is incremental.

          The way the items are processed may suggest me to let the global search indexer fail
          and rerun it untill all documents are processed : the indexer should index a first
          piece of resources, record them in DB and fail for memory reason, the second pass
          should start on the first failed resource and continue indexing things.

          The second pass and following should be done launching manually the cron, that will
          ask for the Global Search incremental updater to complete what is missing, and
          not using the indexer function that would always restart from scratch.

          If i can have a small bit of time to spend here I will post a patch to solve
          the "long tail stuff accumulation processing" effect by providing a segmentated
          indexer solution...

          Cheers

          Show
          vf Valery Fremaux added a comment - Hi I think you have really a lot of stuff to index. Unfortunately, the memory is not an infinite resource and the indexer needs to open all documents and process all content. Indexer can do good stuff when starting a Moodle platform to run as further processing of new stuff is incremental. The way the items are processed may suggest me to let the global search indexer fail and rerun it untill all documents are processed : the indexer should index a first piece of resources, record them in DB and fail for memory reason, the second pass should start on the first failed resource and continue indexing things. The second pass and following should be done launching manually the cron, that will ask for the Global Search incremental updater to complete what is missing, and not using the indexer function that would always restart from scratch. If i can have a small bit of time to spend here I will post a patch to solve the "long tail stuff accumulation processing" effect by providing a segmentated indexer solution... Cheers
          Hide
          shirai Tatsuya Shirai added a comment -

          Hi Gisele and Valery!

          First of all, "50331648 is equal to 48M".
          The root cause of this problem is "// overrides php limits" in search/cron_php5.php.
          If you set "limit_memory=1024M" at php.ini, search/cron_php5.php will override it to 48M.
          I guess... a long time ago, Servers do not have enought memory, initial value of limit_memory is less than 48M.

          Gisele, you miss second part of my patch;

          try{
          // overrides php limits
          $maxtimelimit = ini_get('max_execution_time');
          ini_set('max_execution_time', 300);
          $maxmemoryamount = ini_get('memory_limit');
          // (Debug030): Comment out this line
          // ini_set('memory_limit', '48M');
          // (Debug030): Add from here
          $temp_maxmemory = '48M';
          if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory))

          { ini_set('memory_limit', $temp_maxmemory); }

          // (Debug030): Add to here

          mtrace("\n-DELETE---");

          This part is main of this patch!

          Show
          shirai Tatsuya Shirai added a comment - Hi Gisele and Valery! First of all, "50331648 is equal to 48M". The root cause of this problem is "// overrides php limits" in search/cron_php5.php. If you set "limit_memory=1024M" at php.ini, search/cron_php5.php will override it to 48M. I guess... a long time ago, Servers do not have enought memory, initial value of limit_memory is less than 48M. Gisele, you miss second part of my patch; try{ // overrides php limits $maxtimelimit = ini_get('max_execution_time'); ini_set('max_execution_time', 300); $maxmemoryamount = ini_get('memory_limit'); // (Debug030): Comment out this line // ini_set('memory_limit', '48M'); // (Debug030): Add from here $temp_maxmemory = '48M'; if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory)) { ini_set('memory_limit', $temp_maxmemory); } // (Debug030): Add to here mtrace("\n-DELETE---"); This part is main of this patch!
          Hide
          shirai Tatsuya Shirai added a comment -

          http://www.suzuka-ct.ac.jp/mech/moodle/mod/wiki/view.php?id=320&page=view/search%2Fcron_php5.php

          Sorry, this page is written by Japanese.
          You can obtain correct patches from the page, red colored characters are additional codes.

          Show
          shirai Tatsuya Shirai added a comment - http://www.suzuka-ct.ac.jp/mech/moodle/mod/wiki/view.php?id=320&page=view/search%2Fcron_php5.php Sorry, this page is written by Japanese. You can obtain correct patches from the page, red colored characters are additional codes.
          Hide
          brugger Gisele Brugger added a comment -

          Hello Tatsuya and Valery,

          I had to increase value to : 512M

          try{
          // overrides php limits
          $maxtimelimit = ini_get('max_execution_time');
          ini_set('max_execution_time', 300);
          $maxmemoryamount = ini_get('memory_limit');
          ini_set('memory_limit', '512M');
          // (Debug030): Comment out this line
          // ini_set('memory_limit', '48M');
          // (Debug030): Add from here
          $temp_maxmemory = '512M';
          if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory))

          { ini_set('memory_limit', $temp_maxmemory); }

          // (Debug030): Add to here

          mtrace("\n-DELETE---");

          only with this value I could run the cron.php

          thanks for the help

          Show
          brugger Gisele Brugger added a comment - Hello Tatsuya and Valery, I had to increase value to : 512M try{ // overrides php limits $maxtimelimit = ini_get('max_execution_time'); ini_set('max_execution_time', 300); $maxmemoryamount = ini_get('memory_limit'); ini_set('memory_limit', '512M'); // (Debug030): Comment out this line // ini_set('memory_limit', '48M'); // (Debug030): Add from here $temp_maxmemory = '512M'; if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory)) { ini_set('memory_limit', $temp_maxmemory); } // (Debug030): Add to here mtrace("\n-DELETE---"); only with this value I could run the cron.php thanks for the help
          Hide
          joseph.jacelone Joseph Jacelone added a comment -

          I have found this issue to still be present in Moodle 1.9.14 (Build: 20111010). With Extra PHP memory limit set to 1024MB was still unable to get the cron to run successfully. As soon as I hid Global Search cron finished. Any updates on this functionality being improved?

          Show
          joseph.jacelone Joseph Jacelone added a comment - I have found this issue to still be present in Moodle 1.9.14 (Build: 20111010). With Extra PHP memory limit set to 1024MB was still unable to get the cron to run successfully. As soon as I hid Global Search cron finished. Any updates on this functionality being improved?
          Hide
          poltawski Dan Poltawski added a comment -

          Hi,

          We removed Global Search from Moodle 2.2 onwards because it was not working reliably. I'm closing this issue as it reffers to that removed search.

          Please follow MDL-31989 for a new global search for Moodle 2.4.

          Show
          poltawski Dan Poltawski added a comment - Hi, We removed Global Search from Moodle 2.2 onwards because it was not working reliably. I'm closing this issue as it reffers to that removed search. Please follow MDL-31989 for a new global search for Moodle 2.4.

            People

            • Votes:
              3 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: