Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-53271

Solr server document failures kill indexing

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 3.1
    • 3.1
    • Global search
    • MOODLE_31_STABLE
    • MOODLE_31_STABLE
    • MDL-53271-master
    • Hide
      1. Enable Global Search with a Solr backend setup. Make sure Assignments are indexed.
      2. Add 3 Assignments with the following descriptions, in order, with the following descriptions:
        1. "Findme1"
        2. "各項目の1を取ります。"
        3. "Findme2"
      3. Run the indexing command: 

        php admin/tool/task/cli/schedule_task.php --execute="\\core\\task\\search_task"

      4. You should see a "Solr server" error, but indexing should complete successfully.
      5. Search for Fineme1 and Findme2 and confirm you can find them.
      Show
      Enable Global Search with a Solr backend setup. Make sure Assignments are indexed. Add 3 Assignments with the following descriptions, in order, with the following descriptions: "Findme1" "各項目の1を取ります。" "Findme2" Run the indexing command: php admin/tool/task/cli/schedule_task.php --execute="\\core\\task\\search_task" You should see a "Solr server" error, but indexing should complete successfully. Search for Fineme1 and Findme2 and confirm you can find them.

    Description

      Solr seems to have some issues with certain unicode characters. I discover that it chokes on:

      項目 (U+9805 U+76EE), "item" in Japanese.

      It is actually the first character it has a problem with, and many following characters will cause a failure.

      That throws a SolrServerException with the message Invalid UTF-8 start byte 0xae
      We have also had other server errors, like Exception writing document id mod_page-activity-65414 to the index; possible analysis error

      I don't think there is much we can do about that, but, right now in search_solr\engine::add_document we catch SolrClientException errors, and just debug them, but when the server chokes on a doc, it throws a SolrServerException which we don't catch, and kills all the indexing.

      Attachments

        Issue Links

          has a non-specific relationship to

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. MDL-31989 Global Search 2

          • Major - Major loss of function, incorrect output
          • Closed

          Activity

            People

              emerrill Eric Merrill
              emerrill Eric Merrill
              David Monllaó David Monllaó
              Andrew Lyons Andrew Lyons
              Damyon Wiese Damyon Wiese
              David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                23/May/16