Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-63789

Solr engine core implementation doesn't support authenticated proxies

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.4.5, 3.5.2, 3.6
    • Fix Version/s: 3.4.6, 3.5.3
    • Component/s: Global search
    • Labels:
    • Testing Instructions:
      Hide

      Run the related unit tests:

      php admin/tool/phpunit/cli/util.php --buildcomponentconfigs
      php vendor/bin/phpunit -c search
      

      You need to configure an HTTP Proxy Service w/ authentication like Squid.

      You can use the HQ Moodle DEV env, by making the following changes - beware that the proxy service used here is for just dev/testing purposes, it's actually a really old container:

      # git diff base.yml
      diff --git a/base.yml b/base.yml
      index 24f3882..b765d6c 100755
      --- a/base.yml
      +++ b/base.yml
      @@ -1,9 +1,15 @@
       version: "2"
       services:
      +  proxy:
      +    image: "robhaswell/squid-authenticated"
      +    environment:
      +      SQUID_USERNAME: moodle
      +      SQUID_PASSWORD: foobar
         webserver:
           image: "moodlehq/moodle-php-apache:${MOODLE_DOCKER_PHP_VERSION}"
           depends_on:
             - db
      +      - proxy
           volumes:
             - "${MOODLE_DOCKER_WWWROOT}:/var/www/html"
             - "${ASSETDIR}/web/apache2_faildumps.conf:/etc/apache2/conf-enabled/apache2_faildumps.conf"
       
      # git diff config.docker-template.php
      diff --git a/config.docker-template.php b/config.docker-template.php
      index 4d54a48..fd72fe9 100755
      --- a/config.docker-template.php
      +++ b/config.docker-template.php
      @@ -4,6 +4,11 @@ unset($CFG);
       global $CFG;
       $CFG = new stdClass();
       
      +$CFG->proxyhost = 'proxy';
      +$CFG->proxyport = 3128;
      +$CFG->proxyuser = 'moodle';
      +$CFG->proxypassword = 'foobar';
      +
       $CFG->dbtype    = getenv('MOODLE_DOCKER_DBTYPE');
       $CFG->dblibrary = 'native';
       $CFG->dbhost    = 'db';
      

      and the you can run the following commands:

      # cp config.docker-template.php $MOODLE_DOCKER_WWWROOT/config.php
      # export MOODLE_DOCKER_WWWROOT=/path/to/moodle/code/
      # export MOODLE_DOCKER_DB=mysql
      # export MOODLE_DOCKER_PHPUNIT_EXTERNAL_SERVICES=1
      # bin/moodle-docker-compose up -d
      # bin/moodle-docker-wait-for-db
      # bin/moodle-docker-compose exec webserver php admin/tool/phpunit/cli/init.php
      # bin/moodle-docker-compose exec -T webserver php admin/tool/phpunit/cli/util.php --buildcomponentconfigs
      # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search
      # bin/moodle-docker-compose down -v
      

      If you don't apply the PR you'll get:

      # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search
      Moodle 3.6dev+ (Build: 20181031), 8b019fb70930c48cda037968d3257edc222f8143
      Php: 7.1.22, mysqli: 5.7.24, OS: Linux 3.10.0-862.14.4.el7.x86_64 x86_64
      PHPUnit 6.5.8 by Sebastian Bergmann and contributors.
       
      ...........EEEEEEEEEEEEEEEEEEEEEEEEEEEEE......................... 65 / 92 ( 70%)
      ...........................                                       92 / 92 (100%)
       
      Time: 43.53 seconds, Memory: 76.00MB
       
      There were 29 errors:
      [...]
      

      and if you tail the Squid logs within the proxy container you'll get:

      # tail -f /var/log/squid3/access.log
      1541196411.322      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196411.976      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196412.601      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196413.232      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196413.843      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196414.465      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196415.101      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196415.741      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196416.375      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196416.993      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196417.620      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196418.246      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196418.876      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196419.502      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196420.129      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196420.765      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196421.391      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196422.021      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196422.659      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196423.294      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196423.916      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196424.550      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196425.176      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196425.812      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196426.451      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196427.077      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196427.716      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196428.355      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196428.983      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      

      Show
      Run the related unit tests: php admin/tool/phpunit/cli/util.php --buildcomponentconfigs php vendor/bin/phpunit -c search You need to configure an HTTP Proxy Service w/ authentication like Squid. You can use the HQ Moodle DEV env, by making the following changes - beware that the proxy service used here is for just dev/testing purposes, it's actually a really old container: # git diff base.yml diff --git a/base.yml b/base.yml index 24f3882..b765d6c 100755 --- a/base.yml +++ b/base.yml @@ -1,9 +1,15 @@ version: "2" services: + proxy: + image: "robhaswell/squid-authenticated" + environment: + SQUID_USERNAME: moodle + SQUID_PASSWORD: foobar webserver: image: "moodlehq/moodle-php-apache:${MOODLE_DOCKER_PHP_VERSION}" depends_on: - db + - proxy volumes: - "${MOODLE_DOCKER_WWWROOT}:/var/www/html" - "${ASSETDIR}/web/apache2_faildumps.conf:/etc/apache2/conf-enabled/apache2_faildumps.conf"   # git diff config.docker-template.php diff --git a/config.docker-template.php b/config.docker-template.php index 4d54a48..fd72fe9 100755 --- a/config.docker-template.php +++ b/config.docker-template.php @@ -4,6 +4,11 @@ unset($CFG); global $CFG; $CFG = new stdClass();   +$CFG->proxyhost = 'proxy'; +$CFG->proxyport = 3128; +$CFG->proxyuser = 'moodle'; +$CFG->proxypassword = 'foobar'; + $CFG->dbtype = getenv('MOODLE_DOCKER_DBTYPE'); $CFG->dblibrary = 'native'; $CFG->dbhost = 'db'; and the you can run the following commands: # cp config.docker-template.php $MOODLE_DOCKER_WWWROOT/config.php # export MOODLE_DOCKER_WWWROOT=/path/to/moodle/code/ # export MOODLE_DOCKER_DB=mysql # export MOODLE_DOCKER_PHPUNIT_EXTERNAL_SERVICES=1 # bin/moodle-docker-compose up -d # bin/moodle-docker-wait-for-db # bin/moodle-docker-compose exec webserver php admin/tool/phpunit/cli/init.php # bin/moodle-docker-compose exec -T webserver php admin/tool/phpunit/cli/util.php --buildcomponentconfigs # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search # bin/moodle-docker-compose down -v If you don't apply the PR you'll get: # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search Moodle 3.6dev+ (Build: 20181031), 8b019fb70930c48cda037968d3257edc222f8143 Php: 7.1.22, mysqli: 5.7.24, OS: Linux 3.10.0-862.14.4.el7.x86_64 x86_64 PHPUnit 6.5.8 by Sebastian Bergmann and contributors.   ...........EEEEEEEEEEEEEEEEEEEEEEEEEEEEE......................... 65 / 92 ( 70%) ........................... 92 / 92 (100%)   Time: 43.53 seconds, Memory: 76.00MB   There were 29 errors: [...] and if you tail the Squid logs within the proxy container you'll get: # tail -f /var/log/squid3/access.log 1541196411.322 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196411.976 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196412.601 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196413.232 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196413.843 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196414.465 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196415.101 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196415.741 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196416.375 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196416.993 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196417.620 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196418.246 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196418.876 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196419.502 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196420.129 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196420.765 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196421.391 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196422.021 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196422.659 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196423.294 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196423.916 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196424.550 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196425.176 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196425.812 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196426.451 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196427.077 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196427.716 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196428.355 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196428.983 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
    • Affected Branches:
      MOODLE_34_STABLE, MOODLE_35_STABLE, MOODLE_36_STABLE
    • Fixed Branches:
      MOODLE_34_STABLE, MOODLE_35_STABLE
    • Pull from Repository:
    • Pull 3.5 Branch:
      m35_MDL-63789_Search_SOLR_Fixes_Proxy_Auth
    • Pull Master Branch:
      m36_MDL-63789_Search_SOLR_Fixes_Proxy_Auth

      Description

      While working on understanding why I'm not able to let the PHP solr extension pass Moodle tests on the docker dev env I've found that \search_solr\engine::get_search_client() doesn't support authenticated proxies.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                1 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  12/Nov/18