Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-63789

Solr engine core implementation doesn't support authenticated proxies

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 3.4.5, 3.5.2, 3.6
    • 3.4.6, 3.5.3
    • Global search
    • MOODLE_34_STABLE, MOODLE_35_STABLE, MOODLE_36_STABLE
    • MOODLE_34_STABLE, MOODLE_35_STABLE
    • m36_MDL-63789_Search_SOLR_Fixes_Proxy_Auth
    • Hide

      Run the related unit tests:

      php admin/tool/phpunit/cli/util.php --buildcomponentconfigs
      php vendor/bin/phpunit -c search
      

      You need to configure an HTTP Proxy Service w/ authentication like Squid.

      You can use the HQ Moodle DEV env, by making the following changes - beware that the proxy service used here is for just dev/testing purposes, it's actually a really old container:

      # git diff base.yml
      diff --git a/base.yml b/base.yml
      index 24f3882..b765d6c 100755
      --- a/base.yml
      +++ b/base.yml
      @@ -1,9 +1,15 @@
       version: "2"
       services:
      +  proxy:
      +    image: "robhaswell/squid-authenticated"
      +    environment:
      +      SQUID_USERNAME: moodle
      +      SQUID_PASSWORD: foobar
         webserver:
           image: "moodlehq/moodle-php-apache:${MOODLE_DOCKER_PHP_VERSION}"
           depends_on:
             - db
      +      - proxy
           volumes:
             - "${MOODLE_DOCKER_WWWROOT}:/var/www/html"
             - "${ASSETDIR}/web/apache2_faildumps.conf:/etc/apache2/conf-enabled/apache2_faildumps.conf"
       
      # git diff config.docker-template.php
      diff --git a/config.docker-template.php b/config.docker-template.php
      index 4d54a48..fd72fe9 100755
      --- a/config.docker-template.php
      +++ b/config.docker-template.php
      @@ -4,6 +4,11 @@ unset($CFG);
       global $CFG;
       $CFG = new stdClass();
       
      +$CFG->proxyhost = 'proxy';
      +$CFG->proxyport = 3128;
      +$CFG->proxyuser = 'moodle';
      +$CFG->proxypassword = 'foobar';
      +
       $CFG->dbtype    = getenv('MOODLE_DOCKER_DBTYPE');
       $CFG->dblibrary = 'native';
       $CFG->dbhost    = 'db';
      

      and the you can run the following commands:

      # cp config.docker-template.php $MOODLE_DOCKER_WWWROOT/config.php
      # export MOODLE_DOCKER_WWWROOT=/path/to/moodle/code/
      # export MOODLE_DOCKER_DB=mysql
      # export MOODLE_DOCKER_PHPUNIT_EXTERNAL_SERVICES=1
      # bin/moodle-docker-compose up -d
      # bin/moodle-docker-wait-for-db
      # bin/moodle-docker-compose exec webserver php admin/tool/phpunit/cli/init.php
      # bin/moodle-docker-compose exec -T webserver php admin/tool/phpunit/cli/util.php --buildcomponentconfigs
      # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search
      # bin/moodle-docker-compose down -v
      

      If you don't apply the PR you'll get:

      # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search
      Moodle 3.6dev+ (Build: 20181031), 8b019fb70930c48cda037968d3257edc222f8143
      Php: 7.1.22, mysqli: 5.7.24, OS: Linux 3.10.0-862.14.4.el7.x86_64 x86_64
      PHPUnit 6.5.8 by Sebastian Bergmann and contributors.
       
      ...........EEEEEEEEEEEEEEEEEEEEEEEEEEEEE......................... 65 / 92 ( 70%)
      ...........................                                       92 / 92 (100%)
       
      Time: 43.53 seconds, Memory: 76.00MB
       
      There were 29 errors:
      [...]
      

      and if you tail the Squid logs within the proxy container you'll get:

      # tail -f /var/log/squid3/access.log
      1541196411.322      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196411.976      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196412.601      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196413.232      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196413.843      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196414.465      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196415.101      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196415.741      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196416.375      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196416.993      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196417.620      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196418.246      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196418.876      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196419.502      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196420.129      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196420.765      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196421.391      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196422.021      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196422.659      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196423.294      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196423.916      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196424.550      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196425.176      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196425.812      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196426.451      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196427.077      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196427.716      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196428.355      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      1541196428.983      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      

      Show
      Run the related unit tests: php admin/tool/phpunit/cli/util.php --buildcomponentconfigs php vendor/bin/phpunit -c search You need to configure an HTTP Proxy Service w/ authentication like Squid. You can use the HQ Moodle DEV env, by making the following changes - beware that the proxy service used here is for just dev/testing purposes, it's actually a really old container: # git diff base.yml diff --git a/base.yml b/base.yml index 24f3882..b765d6c 100755 --- a/base.yml +++ b/base.yml @@ -1,9 +1,15 @@ version: "2" services: + proxy: + image: "robhaswell/squid-authenticated" + environment: + SQUID_USERNAME: moodle + SQUID_PASSWORD: foobar webserver: image: "moodlehq/moodle-php-apache:${MOODLE_DOCKER_PHP_VERSION}" depends_on: - db + - proxy volumes: - "${MOODLE_DOCKER_WWWROOT}:/var/www/html" - "${ASSETDIR}/web/apache2_faildumps.conf:/etc/apache2/conf-enabled/apache2_faildumps.conf"   # git diff config.docker-template.php diff --git a/config.docker-template.php b/config.docker-template.php index 4d54a48..fd72fe9 100755 --- a/config.docker-template.php +++ b/config.docker-template.php @@ -4,6 +4,11 @@ unset($CFG); global $CFG; $CFG = new stdClass();   +$CFG->proxyhost = 'proxy'; +$CFG->proxyport = 3128; +$CFG->proxyuser = 'moodle'; +$CFG->proxypassword = 'foobar'; + $CFG->dbtype = getenv('MOODLE_DOCKER_DBTYPE'); $CFG->dblibrary = 'native'; $CFG->dbhost = 'db'; and the you can run the following commands: # cp config.docker-template.php $MOODLE_DOCKER_WWWROOT/config.php # export MOODLE_DOCKER_WWWROOT=/path/to/moodle/code/ # export MOODLE_DOCKER_DB=mysql # export MOODLE_DOCKER_PHPUNIT_EXTERNAL_SERVICES=1 # bin/moodle-docker-compose up -d # bin/moodle-docker-wait-for-db # bin/moodle-docker-compose exec webserver php admin/tool/phpunit/cli/init.php # bin/moodle-docker-compose exec -T webserver php admin/tool/phpunit/cli/util.php --buildcomponentconfigs # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search # bin/moodle-docker-compose down -v If you don't apply the PR you'll get: # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search Moodle 3.6dev+ (Build: 20181031), 8b019fb70930c48cda037968d3257edc222f8143 Php: 7.1.22, mysqli: 5.7.24, OS: Linux 3.10.0-862.14.4.el7.x86_64 x86_64 PHPUnit 6.5.8 by Sebastian Bergmann and contributors.   ...........EEEEEEEEEEEEEEEEEEEEEEEEEEEEE......................... 65 / 92 ( 70%) ........................... 92 / 92 (100%)   Time: 43.53 seconds, Memory: 76.00MB   There were 29 errors: [...] and if you tail the Squid logs within the proxy container you'll get: # tail -f /var/log/squid3/access.log 1541196411.322 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196411.976 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196412.601 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196413.232 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196413.843 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196414.465 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196415.101 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196415.741 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196416.375 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196416.993 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196417.620 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196418.246 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196418.876 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196419.502 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196420.129 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196420.765 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196421.391 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196422.021 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196422.659 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196423.294 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196423.916 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196424.550 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196425.176 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196425.812 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196426.451 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196427.077 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196427.716 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196428.355 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196428.983 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html

    Description

      While working on understanding why I'm not able to let the PHP solr extension pass Moodle tests on the docker dev env I've found that \search_solr\engine::get_search_client() doesn't support authenticated proxies.

      Attachments

        Issue Links

          Activity

            People

              matteo Matteo Scaramuccia
              matteo Matteo Scaramuccia
              Victor Déniz Falcón Victor Déniz Falcón
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Ilya Tregubov, Laurent David, Raquel Ortega, Sara Arjona (@sarjona)
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                12/Nov/18