Issue Details (XML | Word | Printable)

Key: MDL-14141
Type: Bug Bug
Status: Open Open
Priority: Major Major
Assignee: Valery Fremaux
Reporter: Sean Keogh
Votes: 1
Watchers: 4
Operations

Add/Edit UI Mockup to this issue
If you were logged in you would be able to see more operations.
Moodle

Global search indexer suffers a PHP Fatal error almost straight away

Created: 02/Apr/08 01:14 AM   Updated: 16/Jul/09 08:09 AM
Return to search
Component/s: Global search
Affects Version/s: 1.9
Fix Version/s: None

Environment: Centos Linux 4.5, MySQL 5, Apache 2, php 5.2.5, Moodle 1.9+
Issue Links:
Relates
 

Database: MySQL
Participants: Anthony Borrow, Jonathan Moore, Martin Dougiamas, Sean Keogh and Valery Fremaux
Security Level: None
Affected Branches: MOODLE_19_STABLE


 Description  « Hide
Having enabled global search, and having ensured that the extra libraries were in the correct place, I ran the iondexer test - it appeared to show no problems, so I tried to run the indexer for the first time. All I got was output similar to this:

Server Time: Tue, 01 Apr 2008 18:11:21 +0100

Warning: Indexing was not successfully completed last time, restarting.

Using /home/httpd/vhosts/web2/moodledata/search as data directory.
Deleting old index entries.
Starting activity modules

28 modules found.
41 blocks found.
Processing module function chat_get_content_for_index ...

...and nothing else.

Looking at the php error logs shows this:

[Tue Apr 01 18:11:21 2008] [error] [client 81.179.17.193] PHP Fatal error: Uncaught exception 'Zend_Search_Lucene_Exception' with m
essage 'Index compound file doesn't contain _0.del file.' in /home/httpd/vhosts/web2/web/search/Zend/Search/Lucene/Index/SegmentInfo
.php:258\nStack trace:\n#0 /home/httpd/vhosts/web2/web/search/Zend/Search/Lucene/Index/SegmentInfo.php(206): Zend_Search_Lucene_Inde
x_SegmentInfo->openCompoundFile('.del')\n#1 /home/httpd/vhosts/web2/web/search/Zend/Search/Lucene/Index/SegmentWriter/DocumentWriter
.php(209): Zend_Search_Lucene_Index_SegmentInfo->__construct('_0', 3, Object(Zend_Search_Lucene_Storage_Directory_Filesystem))\n#2 /
home/httpd/vhosts/web2/web/search/Zend/Search/Lucene/Index/Writer.php(434): Zend_Search_Lucene_Index_SegmentWriter_DocumentWriter->c
lose()\n#3 /home/httpd/vhosts/web2/web/search/Zend/Search/Lucene.php(1010): Zend_Search_Lucene_Index_Writer->commit()\n#4 /home/http
d/vhosts/web2/web/search/indexer.php(178): Zend_Search_Lucene->commit()\n#5 {main}\n thrown in /home/httpd/vhosts/web2/web/search/Z
end/Search/Lucene/Index/SegmentInfo.php on line 258, referer: http://edu-demo.pteppic.net/search/stats.php

Any ideas?

I know that this is still experimental, and I am not relying on this for any production sites, but I can't even get it partly working for demonstrations...and yet other people in the forums and tracker seem to have got much further than this.

Am I missing some libraries, php extensions or something?

Sean K



 All   Comments   Change History   Version Control      Sort Order: Ascending order - Click to sort in descending order
Martin Dougiamas added a comment - 03/Apr/08 09:29 AM
Valery, any ideas?

Valery Fremaux added a comment - 04/Apr/08 01:23 AM
These are the quite opaque Zend exceptino reporting we all fear to receive one day ?:->!

I will check ASAP if the code of the chat module is Ok for indexing contents. Maybe I suspect some empty content being indexed.

I'll search in.


Jonathan Moore added a comment - 28/Jun/08 06:31 AM
We are seeing the same issue. I finally tracked it down to an eAccellerator bug.

http://eaccelerator.net/ticket/248

Looks like there may be a fix for Lucene in the upstream feed. Not sure if this impacts other accellerators. But lots of Moodle users seem to be using eAccellerator and are probably running into this issue.


Valery Fremaux added a comment - 28/Jun/08 07:06 AM
What package level of eAccelerator are you using ?

we are using

http://bart.eaccelerator.net/source/0.9.5.3/eaccelerator-0.9.5.3.tar.bz2

for Debian Etch and having no noticed this error (here is the report of our indexing base)

Base de données mdl_block_search_documents
Documents dans la base de données 1447
Documents de 'Chats' 16
Documents de 'Bases de données' 0
Documents de 'Forums' 336
Documents de 'Glossaires' 210
Documents de 'Leçons' 29
Documents de 'Ressources' 821
Documents de 'Projets techniques' 0
Documents de 'Wikis' 35


Jonathan Moore added a comment - 28/Jun/08 07:27 AM
We appear to be on the same version. 0.9.5.3. I just turned it back on to see the version reporting from phpinfo and they match.

We did run into an issue in the past that if phpsuexec is enabled it blocks eaccelleartor from being active. is there any chance that is happening on your setup?

I just retested it by turning it back on and it doesn't work. Disabled it again and the indexer worked immediately. Maybe there is something particular to our setup that is throwing an exception that is silently dealt with without the eaccel exception bug active. I have tried this on two separate servers on 1.9 base code, including latest CVS from today and it seems pretty consistent for our setup

centos 4.x
cpanel
eaccellearator
php 5.2.5


Sean Keogh added a comment - 28/Jun/08 05:50 PM
We are running e-accelerator on all of our servers, so that would explain it. The one I've been testing on is running php 5.2.6 wih e-accelerator version 0.9.5.3. With e-accelerator switched off indexing and searching works fine.

The bug report you mentioned said something about the problem being fixed "in the next mini release", which as far as I can see IS 0.9.5.3 - anyone else running e-accelerator and got this working? If so, what version, the development snapshot? The latest snapshot I can find is versioned as 0.9.6 - so maybe worth trying. I'll compile it up and give it a try and report back.

Sean K


Sean Keogh added a comment - 28/Jun/08 06:15 PM
Well, that didn't work. I downloaded the latest 0.9.6 snapshot and compiled and installed it as normal - but it breaks PHP - it segfaults as soon as you try and use it.

The size of the library file is only about 60% of the one compiled for 0.9.5.3, so maybe there is something wrong with the source at the moment?

Would be good to get this fixed.

If I get a chance I'll try APC instead, but I've never had all that much luck with it so far.

Sean K


Sean Keogh added a comment - 28/Jun/08 06:46 PM
Ok, me again. Still no joy with e-accelerator, but I have just successfully compiled and installed apc 3.0.19 from pecl.php.net - it is enabled and working without errors and indexer and search still work.

So it looks like I might have to pull e-accelerator and replace with apc on our servers if we are to get global search to work...unless the e-accelerator people put that fix in really soon...as more and more people are asking us about global search.

The only problem with the above is that you need to have apxs present on your system (it is part of the httpd-devel package) and this can entail installing a few more things as prerequisites of httpd-devel, such as openldap, db4 and a couple of other things.

I'm guessing that once you have the apc.so file, you don't need the rest, so you could probably compile the module on one machine and distribute to all the others (as long as you are running the same versions of Apache and PHP on all of those other machines). I will test out this theory and report back.

Sean K


Jonathan Moore added a comment - 28/Jun/08 08:35 PM
I think the reference to a "fix" at the eaccellerator site was actually someone posting a fix to the Lucene upstream code to make it work around the bug in eaccelerator.

Sean Keogh added a comment - 28/Jun/08 09:19 PM
Right, I tried copying the apc.so library file across to another server that previously had e-accelerator running, and disabled e-accelerator, and enabled apc. It seems to work ok. This machine is running php 5.2.6, the same as the machine I compiled the extension on, but is running Apache 2.0.52 rather than 2.0.59 - it still seems to work OK. And this machine does not have all the httpd-devel stuff on it.

So, it looks like apc might be a viable alternative.

I'm going to watch it for a while for performance, and see what happens.

Sean K


Sean Keogh added a comment - 07/Jan/09 02:19 AM
Well, we've been using APC sucessfully for six months or so now, so I'd say that for us anyway,m the issue is no longer relevant.

Jonathan Moore added a comment - 07/Jan/09 05:50 AM
It would be nice to see this fixed before global search goes out of experimental. eAccellarator is a pretty common optimization. In particular doesn't this defualt to on in some XAMPP builds? Seems like we will have a lot of support problems in the community if this goes into stable without a fix.

Sean Keogh added a comment - 07/Jan/09 05:59 PM
Good point. I've not looked at e-accelerator and recent PHP releases since we went to APC, so I'm not currently in a position to test this. But it is still either a problem with PHP's Lucene code or e-accelerator as far as I know. So it isn't really a moodle problem as such.

Unless the global search code can do some kind of conditional check and apply a work-around if the troublesome eaccelerator / PHP version combination is found?

Sean K


Valery Fremaux added a comment - 09/Jan/09 06:24 PM
Thanks guys for the track.

I'm neither in real position to make such experimentations, but will straight available to include any change in code that could improve the process.

I'm using the global search engine upon eAccelerator on both Debian and RedHat 5.0 distribs without apparent issues so it is a very "environment bound" problem.

Cheers.

Valery.


Anthony Borrow added a comment - 16/Jul/09 08:08 AM
I second Valery's thanks to those and especially to Jonathan for tracking down this issue. Especially considering that I am experiencing it on one of my production sites (well one that is ready to go into production - we are still putting the finishing touches on things) is now experiencing this. I was going to suggest APC and am happy to hear that some testing with APC has proven successful for Sean. While I hope that eAccelerator does manage to work it out so that folks have a choice, I have preferred APC on my other sites in part because it is slated (so I've heard) to be included as standard for PHP6. At some point I was chatting with Martin Langhoff about getting things in Moodle setup to take advantage of APC but I forget how or if that issue was resolved. I'm going to go ahead and link to that issue as folks experiencing this option may want to vote or at least be aware of the need for the APC plugin. Peace - Anthony

Anthony Borrow added a comment - 16/Jul/09 08:09 AM
For those considering switching to APC, I figured they may want to be aware of this issue. Peace - Anthony