Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-53226

Add moodle DB search engine

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      To test this issue you can use https://github.com/moodlehq/moodle-docker as we usually do for issues that should be tested in multiple databases. In this case though you can not use it for mariadb < 10.0.5 and for mssql with full-text search support because these combinations are not there. You can use https://github.com/dmonllao/moodle-docker instead, where I added those combinations (alternatively you can copy db.mssql.fts.yml and db.mariadb.10.0.yml from my repo to your moodle-docker checkout). You can use them with MOODLE_DOCKER_DB=mariadb.10.0 and MOODLE_DOCKER_DB=mssql.fts

      Test this using the following database versions:

      • postgres (version is not important)
      • oracle (version is not important)
      • mysql (lower than 5.6.4)
      • mysql (higher or equal to 5.6.4)
      • mariadb (lower than 10.0.5)
      • mariadb (higher than 10.0.5)
      • mssql using sqlsrv driver (with fulltext search enabled)
      • mssql using sqlsrv dirver (with fulltext search disabled)
      Test 1
      1. Unit tests should pass in all combinations above
      Test 2 (in all combinations above, sorry)
      1. Log in as admin
      2. Create a course named Admin user
      3. Go to site admin and search search you can find here all settings you need to change:
        • Tick "Enable global search"
        • "Select search engine" to "Simple search". This setting should already contain this value if you installed a new site or if you upgraded a site without solr search engine configuration
      4. Execute php search/cli/indexer.php to index your site contents
      5. Go to search/index.php and search for "Admin user" (without double quotes)
      6. You should see at least 2 results: one with the admin user account and one with the course link. You may see other results here, it depends on whether full-text search is enabled or not...
      7. Try to search for strings containing characters that tend to bring problems (single quote, double quote...) the results depends on the full-text support and the database but no exceptions should be triggered
      Show
      To test this issue you can use https://github.com/moodlehq/moodle-docker as we usually do for issues that should be tested in multiple databases. In this case though you can not use it for mariadb < 10.0.5 and for mssql with full-text search support because these combinations are not there. You can use https://github.com/dmonllao/moodle-docker instead, where I added those combinations (alternatively you can copy db.mssql.fts.yml and db.mariadb.10.0.yml from my repo to your moodle-docker checkout). You can use them with MOODLE_DOCKER_DB=mariadb.10.0 and MOODLE_DOCKER_DB=mssql.fts Test this using the following database versions: postgres (version is not important) oracle (version is not important) mysql (lower than 5.6.4) mysql (higher or equal to 5.6.4) mariadb (lower than 10.0.5) mariadb (higher than 10.0.5) mssql using sqlsrv driver (with fulltext search enabled) mssql using sqlsrv dirver (with fulltext search disabled) Test 1 Unit tests should pass in all combinations above Test 2 (in all combinations above, sorry) Log in as admin Create a course named Admin user Go to site admin and search search you can find here all settings you need to change: Tick "Enable global search" "Select search engine" to "Simple search". This setting should already contain this value if you installed a new site or if you upgraded a site without solr search engine configuration Execute php search/cli/indexer.php to index your site contents Go to search/index.php and search for "Admin user" (without double quotes) You should see at least 2 results: one with the admin user account and one with the course link. You may see other results here, it depends on whether full-text search is enabled or not... Try to search for strings containing characters that tend to bring problems (single quote, double quote...) the results depends on the full-text support and the database but no exceptions should be triggered
    • Affected Branches:
      MOODLE_31_STABLE, MOODLE_35_STABLE
    • Fixed Branches:
      MOODLE_35_STABLE
    • Pull Master Branch:
      MDL-53226-master

      Description

      In MDL-31989 we added global search to moodle but only Solr is currently supported as search engine, it is good to have an external system for it because millions and millions of records can be potentially indexed from moodle, and apache lucene is designed to scale properly.

      This issue is about adding a search engine for moodle database so small sites can use moodle's global search without requiring extra systems like solr. The full-text search available operators (like *, ?...) depend on the search engine, so we could have simple %$q% selects and make it cross-db. Talking about it with Damyon Wiese he said that a cross-db search_db will not be possible if we want all the full-text search operators power, because it changes between db vendors, and we would need a different search engine for each db vendor. Another option would be to expand dml for it. Tomasz Muras your thoughts would be greatly appreciated.

      This may help us to refine the search engine API as there is margin to abstract more stuff that now should be implemented separately.

      As a related point (although probably better as a separate issue) we could expand the search engine API to allow search engines to set their capabilities (like highlighting or more like this) and abstract more stuff if possible (I don't have much experience with search engines so not even sure if we can abstract much)

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                12 Vote for this issue
                Watchers:
                28 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  17/May/18

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 5 hours, 34 minutes
                  5h 34m