Moodle
  1. Moodle
  2. MDL-20614

The admin/cron.php does not be finished when global search function is enabled

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.9.5
    • Fix Version/s: None
    • Component/s: Global search
    • Labels:
      None
    • Difficulty:
      Easy
    • Affected Branches:
      MOODLE_19_STABLE
    • Rank:
      2493

      Description

      Global search is good function for me and other moodlers, so I think heartly. Thanks!

      --------------

      > <!-StartFragment-> Delete: Your E-mail Address (database id = 510486, index id = 1090, moodle instance id = 3743)
      > Delete: Your E-mail Address (database id = 510491, index id = 1091, moodle instance id = 3743)
      > <br />
      > <b>Fatal error</b>: Allowed memory size of 50331648 bytes exhausted (tried to allocate 44 bytes) in <b>C:\xampplite\htdocs\mech\moodle\search\Zend\Search\Lucene.php</b> on line ><b>997</b><br />

      The cron process halted on my server. By the way, I had set 'memory_limit' = 500M in php.ini. 50331648 is 48M.
      I checked and found the cause of this problem. I propose a solution as follows;

      I also think that '48M' is too smll for executing search/delete.php in my site.

      (serach/cron_php5.php)

      (Debug030) means my comment.

      <?php
      /*

      • Moodle global search engine
      • This is a special externalized code for cron handling in PHP5.
      • Should never be called by a php 4.3.0 implementation.
        */
        // (Debug030): Add from here
        // http://www.php.net/manual/ja/function.ini-get.php
        function return_bytes($val) {
        if (is_int($val)) return $val;
        $val = trim($val);
        $last = strtolower($val[strlen($val)-1]);
        switch($last) { case 'g': $val *= 1024; case 'm': $val *= 1024; case 'k': $val *= 1024; }

        return $val;
        }
        // (Debug030): Add to here

      try{
      // overrides php limits
      $maxtimelimit = ini_get('max_execution_time');
      ini_set('max_execution_time', 300);
      $maxmemoryamount = ini_get('memory_limit');
      // (Debug030): Comment out this line
      // ini_set('memory_limit', '48M');
      // (Debug030): Add from here
      $temp_maxmemory = '48M';
      if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory))

      { ini_set('memory_limit', $temp_maxmemory); }

      // (Debug030): Add to here

      mtrace("\n-DELETE---");

        Activity

        Hide
        Garrett Novotny added a comment -

        I am seeing this bug in 1.9.9. If I hide global search cron completes successfully. Here is a tail for my cron...

        Starting index update (additions)...
        Index size before: 1112

        9 modules to search in / 20 modules found.
        0 blocks to search in / 32 blocks found.
        1 additional to search in.
        Checking assignment module for additions.
        Finished assignment.

        Checking chat module for additions.
        No types to add.

        Finished chat.

        Checking data module for additions.
        Finished data.

        Checking forum module for additions.

        Show
        Garrett Novotny added a comment - I am seeing this bug in 1.9.9. If I hide global search cron completes successfully. Here is a tail for my cron... Starting index update (additions)... Index size before: 1112 9 modules to search in / 20 modules found. 0 blocks to search in / 32 blocks found. 1 additional to search in. Checking assignment module for additions. Finished assignment. Checking chat module for additions. No types to add. Finished chat. Checking data module for additions. Finished data. Checking forum module for additions.
        Hide
        Gisele Brugger added a comment -

        Hello,
        I had the same problem using moodle 1.9.10 and windows server 2008 with php limit_memory=1024M
        The first error when execute moodle/admin/cron.php

        PHP Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 44 bytes) in C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\moodle\search\Zend\Search\Lucene.php on line 970

        if i change code adding function

        // (Debug030): Add from here
        // http://www.php.net/manual/ja/function.ini-get.php
        function return_bytes($val) {
        if (is_int($val)) return $val;
        $val = trim($val);
        $last = strtolower($val[strlen($val)-1]);
        switch($last)

        { case 'g': $val *= 1024; case 'm': $val *= 1024; case 'k': $val *= 1024; }

        return $val;
        }
        // (Debug030): Add to here

        i have this error :

        PHP Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 24 bytes) in C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\moodle\search\Zend\Search\Lucene\Storage\File.php on line 268

        I think the script should use the php.ini limit_memory (1024M is more than enough or no ?)

        My resume index search:

        Documents in index 30608
        Deletions in index 1598
        Documents in database 29010
        Documents for 'Assignments' 1737
        Documents for 'Chats' 0
        Documents for 'Databases' 0
        Documents for 'Forums' 7464
        Documents for 'Glossaries' 213
        Documents for 'Labels' 1046
        Documents for 'Lessons' 42
        Documents for 'Resources' 12137
        Documents for 'Wikis' 145

        Any sugestion?

        Thank you

        Show
        Gisele Brugger added a comment - Hello, I had the same problem using moodle 1.9.10 and windows server 2008 with php limit_memory=1024M The first error when execute moodle/admin/cron.php PHP Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 44 bytes) in C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\moodle\search\Zend\Search\Lucene.php on line 970 if i change code adding function // (Debug030): Add from here // http://www.php.net/manual/ja/function.ini-get.php function return_bytes($val) { if (is_int($val)) return $val; $val = trim($val); $last = strtolower($val [strlen($val)-1] ); switch($last) { case 'g': $val *= 1024; case 'm': $val *= 1024; case 'k': $val *= 1024; } return $val; } // (Debug030): Add to here i have this error : PHP Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 24 bytes) in C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\moodle\search\Zend\Search\Lucene\Storage\File.php on line 268 I think the script should use the php.ini limit_memory (1024M is more than enough or no ?) My resume index search: Documents in index 30608 Deletions in index 1598 Documents in database 29010 Documents for 'Assignments' 1737 Documents for 'Chats' 0 Documents for 'Databases' 0 Documents for 'Forums' 7464 Documents for 'Glossaries' 213 Documents for 'Labels' 1046 Documents for 'Lessons' 42 Documents for 'Resources' 12137 Documents for 'Wikis' 145 Any sugestion? Thank you
        Hide
        Valery Fremaux added a comment -

        Hi

        I think you have really a lot of stuff to index.

        Unfortunately, the memory is not an infinite resource and the indexer needs
        to open all documents and process all content.

        Indexer can do good stuff when starting a Moodle platform to run as further processing
        of new stuff is incremental.

        The way the items are processed may suggest me to let the global search indexer fail
        and rerun it untill all documents are processed : the indexer should index a first
        piece of resources, record them in DB and fail for memory reason, the second pass
        should start on the first failed resource and continue indexing things.

        The second pass and following should be done launching manually the cron, that will
        ask for the Global Search incremental updater to complete what is missing, and
        not using the indexer function that would always restart from scratch.

        If i can have a small bit of time to spend here I will post a patch to solve
        the "long tail stuff accumulation processing" effect by providing a segmentated
        indexer solution...

        Cheers

        Show
        Valery Fremaux added a comment - Hi I think you have really a lot of stuff to index. Unfortunately, the memory is not an infinite resource and the indexer needs to open all documents and process all content. Indexer can do good stuff when starting a Moodle platform to run as further processing of new stuff is incremental. The way the items are processed may suggest me to let the global search indexer fail and rerun it untill all documents are processed : the indexer should index a first piece of resources, record them in DB and fail for memory reason, the second pass should start on the first failed resource and continue indexing things. The second pass and following should be done launching manually the cron, that will ask for the Global Search incremental updater to complete what is missing, and not using the indexer function that would always restart from scratch. If i can have a small bit of time to spend here I will post a patch to solve the "long tail stuff accumulation processing" effect by providing a segmentated indexer solution... Cheers
        Hide
        Tatsuya Shirai added a comment -

        Hi Gisele and Valery!

        First of all, "50331648 is equal to 48M".
        The root cause of this problem is "// overrides php limits" in search/cron_php5.php.
        If you set "limit_memory=1024M" at php.ini, search/cron_php5.php will override it to 48M.
        I guess... a long time ago, Servers do not have enought memory, initial value of limit_memory is less than 48M.

        Gisele, you miss second part of my patch;

        try{
        // overrides php limits
        $maxtimelimit = ini_get('max_execution_time');
        ini_set('max_execution_time', 300);
        $maxmemoryamount = ini_get('memory_limit');
        // (Debug030): Comment out this line
        // ini_set('memory_limit', '48M');
        // (Debug030): Add from here
        $temp_maxmemory = '48M';
        if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory))

        { ini_set('memory_limit', $temp_maxmemory); }

        // (Debug030): Add to here

        mtrace("\n-DELETE---");

        This part is main of this patch!

        Show
        Tatsuya Shirai added a comment - Hi Gisele and Valery! First of all, "50331648 is equal to 48M". The root cause of this problem is "// overrides php limits" in search/cron_php5.php. If you set "limit_memory=1024M" at php.ini, search/cron_php5.php will override it to 48M. I guess... a long time ago, Servers do not have enought memory, initial value of limit_memory is less than 48M. Gisele, you miss second part of my patch; try{ // overrides php limits $maxtimelimit = ini_get('max_execution_time'); ini_set('max_execution_time', 300); $maxmemoryamount = ini_get('memory_limit'); // (Debug030): Comment out this line // ini_set('memory_limit', '48M'); // (Debug030): Add from here $temp_maxmemory = '48M'; if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory)) { ini_set('memory_limit', $temp_maxmemory); } // (Debug030): Add to here mtrace("\n-DELETE---"); This part is main of this patch!
        Hide
        Tatsuya Shirai added a comment -

        http://www.suzuka-ct.ac.jp/mech/moodle/mod/wiki/view.php?id=320&page=view/search%2Fcron_php5.php

        Sorry, this page is written by Japanese.
        You can obtain correct patches from the page, red colored characters are additional codes.

        Show
        Tatsuya Shirai added a comment - http://www.suzuka-ct.ac.jp/mech/moodle/mod/wiki/view.php?id=320&page=view/search%2Fcron_php5.php Sorry, this page is written by Japanese. You can obtain correct patches from the page, red colored characters are additional codes.
        Hide
        Gisele Brugger added a comment -

        Hello Tatsuya and Valery,

        I had to increase value to : 512M

        try{
        // overrides php limits
        $maxtimelimit = ini_get('max_execution_time');
        ini_set('max_execution_time', 300);
        $maxmemoryamount = ini_get('memory_limit');
        ini_set('memory_limit', '512M');
        // (Debug030): Comment out this line
        // ini_set('memory_limit', '48M');
        // (Debug030): Add from here
        $temp_maxmemory = '512M';
        if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory))

        { ini_set('memory_limit', $temp_maxmemory); }

        // (Debug030): Add to here

        mtrace("\n-DELETE---");

        only with this value I could run the cron.php

        thanks for the help

        Show
        Gisele Brugger added a comment - Hello Tatsuya and Valery, I had to increase value to : 512M try{ // overrides php limits $maxtimelimit = ini_get('max_execution_time'); ini_set('max_execution_time', 300); $maxmemoryamount = ini_get('memory_limit'); ini_set('memory_limit', '512M'); // (Debug030): Comment out this line // ini_set('memory_limit', '48M'); // (Debug030): Add from here $temp_maxmemory = '512M'; if (return_bytes($maxmemoryamount) < return_bytes($temp_maxmemory)) { ini_set('memory_limit', $temp_maxmemory); } // (Debug030): Add to here mtrace("\n-DELETE---"); only with this value I could run the cron.php thanks for the help
        Hide
        Joseph Jacelone added a comment -

        I have found this issue to still be present in Moodle 1.9.14 (Build: 20111010). With Extra PHP memory limit set to 1024MB was still unable to get the cron to run successfully. As soon as I hid Global Search cron finished. Any updates on this functionality being improved?

        Show
        Joseph Jacelone added a comment - I have found this issue to still be present in Moodle 1.9.14 (Build: 20111010). With Extra PHP memory limit set to 1024MB was still unable to get the cron to run successfully. As soon as I hid Global Search cron finished. Any updates on this functionality being improved?
        Hide
        Dan Poltawski added a comment -

        Hi,

        We removed Global Search from Moodle 2.2 onwards because it was not working reliably. I'm closing this issue as it reffers to that removed search.

        Please follow MDL-31989 for a new global search for Moodle 2.4.

        Show
        Dan Poltawski added a comment - Hi, We removed Global Search from Moodle 2.2 onwards because it was not working reliably. I'm closing this issue as it reffers to that removed search. Please follow MDL-31989 for a new global search for Moodle 2.4.

          People

          • Votes:
            3 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: