Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-53412

Solr search does not work with SSL

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.1
    • Fix Version/s: 3.1
    • Component/s: Global search
    • Labels:
    • Testing Instructions:
      Hide

      A note for for Mac testers

      If you are running Mac OS X 10.9 or above for your PHP server, you have a problem... The SSL engine in OS X doesn't allow PEM private keys, and that is all that pecl-solr supports.
      You need to install openssl, compile curl against openssl, compile PHP against that version of curl. If you are using brew, it's pretty easy:

      $ brew rm curl ; brew install curl --with-openssl
      $ brew uninstall php56
      $ brew install homebrew/php/php56 --with-homebrew-curl --with-postgresql --with-fpm
      

      You will want to change the build options after with-homebrew-curl as needed.

      Then restart your web service.

      The actual testing:

      Creating Certs/Keys

      In a shell/terminal, create a directory, and move into it. It should probably be a path with no spaces in it.

      Now create a key/cert pair, with a passphrase of "secret":

      $ keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass secret -storepass secret -validity 9999 -keystore solr-ssl.keystore.jks -ext SAN=DNS:localhost,IP:127.0.0.1 -dname "CN=localhost, OU=Organizational Unit, O=Organization, L=Location, ST=State, C=Country"
      

      Make an intermediate pkcs12 file:

      $ keytool -importkeystore -srckeystore solr-ssl.keystore.jks -destkeystore solr-ssl.keystore.p12 -srcstoretype jks -deststoretype pkcs12
      

      Now make a key only PEM file - remember the passphrase is "secret":

      $ openssl pkcs12 -nocerts -in solr-ssl.keystore.p12 -out solr-ssl.key.pem
      

      Now create a cert only PEM file:

      $ openssl pkcs12 -nokeys -in solr-ssl.keystore.p12 -out solr-ssl.cacert.pem
      

      Setting up Solr

      Next we need to setup solr.in.sh. The location will very by system, with a brew solr install, we can do ~/.solr.in.sh. Other places may be /var/solr/solr.in.sh, or bin/solr.in.sh, located where the solr binary is located.

      We want to add:

      SOLR_SSL_KEY_STORE=/path/to/certs/solr-ssl.keystore.jks
      SOLR_SSL_KEY_STORE_PASSWORD=secret
      SOLR_SSL_TRUST_STORE=/path/to/certs/solr-ssl.keystore.jks
      SOLR_SSL_TRUST_STORE_PASSWORD=secret
      # Require clients to authenticate
      SOLR_SSL_NEED_CLIENT_AUTH=false
      # Enable clients to authenticate (but not require)
      SOLR_SSL_WANT_CLIENT_AUTH=true
      

      Now we can restart solr:

      $ solr stop
      $ solr start
      

      Now you can confirm it is working by going to https://localhost:8983/, you will get a invalid cert error, which you can just accept in your browser and continue.

      Once you confirm that works, change two lines in the solr.in.sh file:

      SOLR_SSL_NEED_CLIENT_AUTH=true
      SOLR_SSL_WANT_CLIENT_AUTH=false
      

      And restart solr. This will put solr into a more restrictive mode, and your browser will no longer be able to connect.

      Setting and testing Moodle

      1. Enable global search.
      2. Go to the solr setup page, and the the normal settings (if not already there).
      3. Add these settings:
        • Secure -> On
        • SSL certificate -> /path/to/certs/solr-ssl.cacert.pem
        • SSL key -> /path/to/certs/solr-ssl.key.pem
        • SSL key password -> "secret"
        • SSL CA certificates name -> /path/to/certs/solr-ssl.cacert.pem
      4. Save.
      5. Back on the search management page, next to setup search engine, confirm you see Yes or that you need to install schema.
      6. Install schema if needed.
      7. Go to the index data page and select index all and delete.
      8. Run the index, hopefully without error.
      9. Do a search you should get results for.

      PHP Unit running

      Configure your config.php with:

      define('TEST_SEARCH_SOLR_HOSTNAME', '127.0.0.1');
      define('TEST_SEARCH_SOLR_PORT', '8983');
      define('TEST_SEARCH_SOLR_INDEXNAME', 'unittest');
      define('TEST_SEARCH_SOLR_SSLCERT', '/path/to/certs/solr-ssl.cacert.pem');
      define('TEST_SEARCH_SOLR_SSLKEY', '/path/to/certs/solr-ssl.key.pem');
      define('TEST_SEARCH_SOLR_KEYPASSWORD', 'secret');
      define('TEST_SEARCH_SOLR_CAINFOCERT', '/path/to/certs/solr-ssl.cacert.pem');
      

      Run phpunit against the search_solr_testsuite.

      Comment out the lines in solr.in.sh, restart solr, remove the last 4 defines in config.php, and rerun unit tests.

      Show
      A note for for Mac testers If you are running Mac OS X 10.9 or above for your PHP server, you have a problem... The SSL engine in OS X doesn't allow PEM private keys, and that is all that pecl-solr supports. You need to install openssl, compile curl against openssl, compile PHP against that version of curl. If you are using brew, it's pretty easy: $ brew rm curl ; brew install curl --with-openssl $ brew uninstall php56 $ brew install homebrew/php/php56 --with-homebrew-curl --with-postgresql --with-fpm You will want to change the build options after with-homebrew-curl as needed. Then restart your web service. The actual testing: Creating Certs/Keys In a shell/terminal, create a directory, and move into it. It should probably be a path with no spaces in it. Now create a key/cert pair, with a passphrase of "secret": $ keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass secret -storepass secret -validity 9999 -keystore solr-ssl.keystore.jks -ext SAN=DNS:localhost,IP:127.0.0.1 -dname "CN=localhost, OU=Organizational Unit, O=Organization, L=Location, ST=State, C=Country" Make an intermediate pkcs12 file: $ keytool -importkeystore -srckeystore solr-ssl.keystore.jks -destkeystore solr-ssl.keystore.p12 -srcstoretype jks -deststoretype pkcs12 Now make a key only PEM file - remember the passphrase is "secret": $ openssl pkcs12 -nocerts -in solr-ssl.keystore.p12 -out solr-ssl.key.pem Now create a cert only PEM file: $ openssl pkcs12 -nokeys -in solr-ssl.keystore.p12 -out solr-ssl.cacert.pem Setting up Solr Next we need to setup solr.in.sh. The location will very by system, with a brew solr install, we can do ~/.solr.in.sh. Other places may be /var/solr/solr.in.sh, or bin/solr.in.sh, located where the solr binary is located. We want to add: SOLR_SSL_KEY_STORE=/path/to/certs/solr-ssl.keystore.jks SOLR_SSL_KEY_STORE_PASSWORD=secret SOLR_SSL_TRUST_STORE=/path/to/certs/solr-ssl.keystore.jks SOLR_SSL_TRUST_STORE_PASSWORD=secret # Require clients to authenticate SOLR_SSL_NEED_CLIENT_AUTH=false # Enable clients to authenticate (but not require) SOLR_SSL_WANT_CLIENT_AUTH=true Now we can restart solr: $ solr stop $ solr start Now you can confirm it is working by going to https://localhost:8983/ , you will get a invalid cert error, which you can just accept in your browser and continue. Once you confirm that works, change two lines in the solr.in.sh file: SOLR_SSL_NEED_CLIENT_AUTH=true SOLR_SSL_WANT_CLIENT_AUTH=false And restart solr. This will put solr into a more restrictive mode, and your browser will no longer be able to connect. Setting and testing Moodle Enable global search. Go to the solr setup page, and the the normal settings (if not already there). Add these settings: Secure -> On SSL certificate -> /path/to/certs/solr-ssl.cacert.pem SSL key -> /path/to/certs/solr-ssl.key.pem SSL key password -> "secret" SSL CA certificates name -> /path/to/certs/solr-ssl.cacert.pem Save. Back on the search management page, next to setup search engine, confirm you see Yes or that you need to install schema. Install schema if needed. Go to the index data page and select index all and delete. Run the index, hopefully without error. Do a search you should get results for. PHP Unit running Configure your config.php with: define('TEST_SEARCH_SOLR_HOSTNAME', '127.0.0.1'); define('TEST_SEARCH_SOLR_PORT', '8983'); define('TEST_SEARCH_SOLR_INDEXNAME', 'unittest'); define('TEST_SEARCH_SOLR_SSLCERT', '/path/to/certs/solr-ssl.cacert.pem'); define('TEST_SEARCH_SOLR_SSLKEY', '/path/to/certs/solr-ssl.key.pem'); define('TEST_SEARCH_SOLR_KEYPASSWORD', 'secret'); define('TEST_SEARCH_SOLR_CAINFOCERT', '/path/to/certs/solr-ssl.cacert.pem'); Run phpunit against the search_solr_testsuite. Comment out the lines in solr.in.sh, restart solr, remove the last 4 defines in config.php, and rerun unit tests.
    • Affected Branches:
      MOODLE_31_STABLE
    • Fixed Branches:
      MOODLE_31_STABLE
    • Pull Master Branch:
      MDL-53412-master

      Description

      There are a number of issues with using solr search with SSL, and it does not work - both using the SolrClient and the curl handle for schema.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  23/May/16