Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-60555

search_manager_testcase::test_partial_indexing random failure

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      Apart from confirming unit test pass (they do; see Travis result), we should confirm indexing is not broken when running outside unit tests, in particular the partial indexing with time limit. You will need a system with search set up and working.

      1. Go to the search areas page (admin/searchareas.php; plugins/search/search areas).
      2. Select a search area that contains a non-trivial amount of content, and click 'Delete index'. (You want something that will take at least, say, 30 seconds to index.)
      3. Go to the main search options page (plugins/search/manage global search)
      4. Set indexing time limit to 15 seconds
      5. Go to the scheduled tasks page (server/scheduled tasks)
      6. Click 'Run now' next to the 'Global search indexing' task, and confirm the prompt.
        • Expected: It should stop after approximately 15 seconds (usually slightly over, e.g. 16-20), and there should be a 'not complete' note against the area where you deleted the index.
      7. Run it again (you can hit reload).
        • Expected: It should again run for approximately 15 seconds (unless it finishes indexing the area).
      Show
      Apart from confirming unit test pass (they do; see Travis result), we should confirm indexing is not broken when running outside unit tests, in particular the partial indexing with time limit. You will need a system with search set up and working. Go to the search areas page (admin/searchareas.php; plugins/search/search areas). Select a search area that contains a non-trivial amount of content, and click 'Delete index'. (You want something that will take at least, say, 30 seconds to index.) Go to the main search options page (plugins/search/manage global search) Set indexing time limit to 15 seconds Go to the scheduled tasks page (server/scheduled tasks) Click 'Run now' next to the 'Global search indexing' task, and confirm the prompt. Expected: It should stop after approximately 15 seconds (usually slightly over, e.g. 16-20), and there should be a 'not complete' note against the area where you deleted the index. Run it again (you can hit reload). Expected: It should again run for approximately 15 seconds (unless it finishes indexing the area).
    • Affected Branches:
      MOODLE_34_STABLE
    • Fixed Branches:
      MOODLE_34_STABLE
    • Pull Master Branch:
      MDL-60555-master

      Description

      Copying comments from MDL-59039 comments:

      David Mo:
      Today we got a random failure related to this issue:

      1) search_manager_testcase::test_partial_indexing
      Failed asserting that actual size 0 matches expected size 1.
       
      /Users/Shared/Jenkins/Home/git_repositories/master/search/tests/manager_test.php:243
      /Users/Shared/Jenkins/Home/git_repositories/master/lib/phpunit/classes/advanced_testcase.php:80
       
      To re-run:
       vendor/bin/phpunit search_manager_testcase search/tests/manager_test.php
      

      It is the first time it fails, I haven't checked how it all works but:
      1. the label is created
      2. we wait for the next second (waitForSecond)
      3. we index contents for 1 second
      4. ERROR 0 docs indexed
      the only possible cause I can think of is that 2 full seconds pass from the second the label is created and the second we index, I imagine that, although not likely, it could be possible, any ideas?

      Sam Marshall reply:
      Sorry I was away on holiday but I've now had a look at this and I think I've figured it out. It doesn't matter how long time passes after the label being created (as long as it is at least the next second).
      When we index contents with 1 second time limit, this part is supposed to test that the forum is done last (because it took longer last time). It is supposed to work as follows:
      1 Index all the other search areas apart from forum
      2 Most of these areas have no new data it should take less than 1 second to check all of them (ASSUMPTION!)
      3 When it hits label, it will 'index' the label, and also do the fake 1.2 second delay that the mock code adds in to simulate indexing taking time
      4 Because that took 1.2 seconds, indexing will now stop
      It checks the time limit after processing each search area (even if no documents were added there), and after adding each document within a search area.
      I think what is happening is that it's taking more than 1 second to check some other search area or areas (which contain no new documents but it still has to do the query) before it even gets to label. Therefore it exits before doing a single document. In other words the assumption is not always true.
      I can think of three approaches to fix this:
      1 In the test, increase all the times (throughout the test) a bit to reduce the chance of a failure. I tried to get the times fairly tight so this doesn't add ages to phpunit run time but we could make them a bit bigger...
      or
      2 In the real code, change the timeout logic so that it won't stop indexing before it indexes at least one document (i.e. if you set a stupid small limit and it takes that long to just look at all the search areas with nothing new in).
      or
      3 Change the real code so that there is a way to override the current time value, for use in unit tests, so that the timeout part is independent of real time (i.e. it would only 'advance' time when the mock search engine indexes a document, and nothing would really have to wait any more). This will probably mean using the PHPUNIT_TEST define and a global variable for the current fake time. Could be a little complicated and makes the tests fractionally less 'realistic', but maybe the most comprehensive solution.
      Obviously there is also option 4 to do nothing if the failures are sufficiently rare.
      I'm OK to code a fix if you want, but not certain which approach is best?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                quen Sam Marshall
                Reporter:
                dmonllao David Monllaó
                Peer reviewer:
                David Monllaó
                Integrator:
                Jake Dallimore
                Tester:
                Simey Lameze
                Participants:
                Component watchers:
                Amaia Anabitarte, Bas Brands, Carlos Escobedo, Sara Arjona (@sarjona), Víctor Déniz Falcón, Andrew Nicols, Mathew May, Michael Hawkins, Shamim Rezaie, Simey Lameze
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  13/Nov/17