Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-53271

Solr server document failures kill indexing

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.1
    • Fix Version/s: 3.1
    • Component/s: Global search
    • Labels:
    • Testing Instructions:
      Hide
      1. Enable Global Search with a Solr backend setup. Make sure Assignments are indexed.
      2. Add 3 Assignments with the following descriptions, in order, with the following descriptions:
        1. "Findme1"
        2. "各項目の1を取ります。"
        3. "Findme2"
      3. Run the indexing command:

        php admin/tool/task/cli/schedule_task.php --execute="\\core\\task\\search_task"

      4. You should see a "Solr server" error, but indexing should complete successfully.
      5. Search for Fineme1 and Findme2 and confirm you can find them.
      Show
      Enable Global Search with a Solr backend setup. Make sure Assignments are indexed. Add 3 Assignments with the following descriptions, in order, with the following descriptions: "Findme1" "各項目の1を取ります。" "Findme2" Run the indexing command: php admin/tool/task/cli/schedule_task.php --execute="\\core\\task\\search_task" You should see a "Solr server" error, but indexing should complete successfully. Search for Fineme1 and Findme2 and confirm you can find them.
    • Affected Branches:
      MOODLE_31_STABLE
    • Fixed Branches:
      MOODLE_31_STABLE
    • Pull Master Branch:
      MDL-53271-master

      Description

      Solr seems to have some issues with certain unicode characters. I discover that it chokes on:

      項目 (U+9805 U+76EE), "item" in Japanese.

      It is actually the first character it has a problem with, and many following characters will cause a failure.

      That throws a SolrServerException with the message Invalid UTF-8 start byte 0xae
      We have also had other server errors, like Exception writing document id mod_page-activity-65414 to the index; possible analysis error

      I don't think there is much we can do about that, but, right now in search_solr\engine::add_document we catch SolrClientException errors, and just debug them, but when the server chokes on a doc, it throws a SolrServerException which we don't catch, and kills all the indexing.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  23/May/16