Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-63789

Solr engine core implementation doesn't support authenticated proxies

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 3.4.5, 3.5.2, 3.6
    • 3.4.6, 3.5.3
    • Global search
    • MOODLE_34_STABLE, MOODLE_35_STABLE, MOODLE_36_STABLE
    • MOODLE_34_STABLE, MOODLE_35_STABLE
    • m36_MDL-63789_Search_SOLR_Fixes_Proxy_Auth
    • Hide

      Run the related unit tests:

      php admin/tool/phpunit/cli/util.php --buildcomponentconfigs
      		 
      php vendor/bin/phpunit -c search

      You need to configure an HTTP Proxy Service w/ authentication like Squid.

      You can use the HQ Moodle DEV env, by making the following changes - beware that the proxy service used here is for just dev/testing purposes, it's actually a really old container:

      # git diff base.yml
      		 
      diff --git a/base.yml b/base.yml
      		 
      index 24f3882..b765d6c 100755
      		 
      --- a/base.yml
      		 
      +++ b/base.yml
      		 
      @@ -1,9 +1,15 @@
      		 
       version: "2"
      		 
       services:
      		 
      +  proxy:
      		 
      +    image: "robhaswell/squid-authenticated"
      		 
      +    environment:
      		 
      +      SQUID_USERNAME: moodle
      		 
      +      SQUID_PASSWORD: foobar
      		 
         webserver:
      		 
           image: "moodlehq/moodle-php-apache:${MOODLE_DOCKER_PHP_VERSION}"
      		 
           depends_on:
      		 
             - db
      		 
      +      - proxy
      		 
           volumes:
      		 
             - "${MOODLE_DOCKER_WWWROOT}:/var/www/html"
      		 
             - "${ASSETDIR}/web/apache2_faildumps.conf:/etc/apache2/conf-enabled/apache2_faildumps.conf"
      		 
       
      		 
      # git diff config.docker-template.php
      		 
      diff --git a/config.docker-template.php b/config.docker-template.php
      		 
      index 4d54a48..fd72fe9 100755
      		 
      --- a/config.docker-template.php
      		 
      +++ b/config.docker-template.php
      		 
      @@ -4,6 +4,11 @@ unset($CFG);
      		 
       global $CFG;
      		 
       $CFG = new stdClass();
      		 
       
      		 
      +$CFG->proxyhost = 'proxy';
      		 
      +$CFG->proxyport = 3128;
      		 
      +$CFG->proxyuser = 'moodle';
      		 
      +$CFG->proxypassword = 'foobar';
      		 
      +
      		 
       $CFG->dbtype    = getenv('MOODLE_DOCKER_DBTYPE');
      		 
       $CFG->dblibrary = 'native';
      		 
       $CFG->dbhost    = 'db';

      and the you can run the following commands:

      # cp config.docker-template.php $MOODLE_DOCKER_WWWROOT/config.php
      		 
      # export MOODLE_DOCKER_WWWROOT=/path/to/moodle/code/
      		 
      # export MOODLE_DOCKER_DB=mysql
      		 
      # export MOODLE_DOCKER_PHPUNIT_EXTERNAL_SERVICES=1
      		 
      # bin/moodle-docker-compose up -d
      		 
      # bin/moodle-docker-wait-for-db
      		 
      # bin/moodle-docker-compose exec webserver php admin/tool/phpunit/cli/init.php
      		 
      # bin/moodle-docker-compose exec -T webserver php admin/tool/phpunit/cli/util.php --buildcomponentconfigs
      		 
      # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search
      		 
      # bin/moodle-docker-compose down -v

      If you don't apply the PR you'll get:

      # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search
      		 
      Moodle 3.6dev+ (Build: 20181031), 8b019fb70930c48cda037968d3257edc222f8143
      		 
      Php: 7.1.22, mysqli: 5.7.24, OS: Linux 3.10.0-862.14.4.el7.x86_64 x86_64
      		 
      PHPUnit 6.5.8 by Sebastian Bergmann and contributors.
      		 
       
      		 
      ...........EEEEEEEEEEEEEEEEEEEEEEEEEEEEE......................... 65 / 92 ( 70%)
      		 
      ...........................                                       92 / 92 (100%)
      		 
       
      		 
      Time: 43.53 seconds, Memory: 76.00MB
      		 
       
      		 
      There were 29 errors:
      		 
      [...]

      and if you tail the Squid logs within the proxy container you'll get:

      # tail -f /var/log/squid3/access.log
      		 
      1541196411.322      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196411.976      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196412.601      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196413.232      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196413.843      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196414.465      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196415.101      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196415.741      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196416.375      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196416.993      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196417.620      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196418.246      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196418.876      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196419.502      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196420.129      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196420.765      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196421.391      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196422.021      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196422.659      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196423.294      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196423.916      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196424.550      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196425.176      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196425.812      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196426.451      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196427.077      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196427.716      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196428.355      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html
      		 
      1541196428.983      0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html

      Show
      Run the related unit tests: php admin/tool/phpunit/cli/util.php --buildcomponentconfigs php vendor/bin/phpunit -c search You need to configure an HTTP Proxy Service w/ authentication like Squid. You can use the HQ Moodle DEV env, by making the following changes - beware that the proxy service used here is for just dev/testing purposes, it's actually a really old container: # git diff base.yml diff --git a/base.yml b/base.yml index 24f3882..b765d6c 100755 --- a/base.yml +++ b/base.yml @@ -1,9 +1,15 @@ version: "2" services: + proxy: + image: "robhaswell/squid-authenticated" + environment: + SQUID_USERNAME: moodle + SQUID_PASSWORD: foobar webserver: image: "moodlehq/moodle-php-apache:${MOODLE_DOCKER_PHP_VERSION}" depends_on: - db + - proxy volumes: - "${MOODLE_DOCKER_WWWROOT}:/var/www/html" - "${ASSETDIR}/web/apache2_faildumps.conf:/etc/apache2/conf-enabled/apache2_faildumps.conf"   # git diff config.docker-template.php diff --git a/config.docker-template.php b/config.docker-template.php index 4d54a48..fd72fe9 100755 --- a/config.docker-template.php +++ b/config.docker-template.php @@ -4,6 +4,11 @@ unset($CFG); global $CFG; $CFG = new stdClass();   +$CFG->proxyhost = 'proxy'; +$CFG->proxyport = 3128; +$CFG->proxyuser = 'moodle'; +$CFG->proxypassword = 'foobar'; + $CFG->dbtype = getenv('MOODLE_DOCKER_DBTYPE'); $CFG->dblibrary = 'native'; $CFG->dbhost = 'db'; and the you can run the following commands: # cp config.docker-template.php $MOODLE_DOCKER_WWWROOT/config.php # export MOODLE_DOCKER_WWWROOT=/path/to/moodle/code/ # export MOODLE_DOCKER_DB=mysql # export MOODLE_DOCKER_PHPUNIT_EXTERNAL_SERVICES=1 # bin/moodle-docker-compose up -d # bin/moodle-docker-wait-for-db # bin/moodle-docker-compose exec webserver php admin/tool/phpunit/cli/init.php # bin/moodle-docker-compose exec -T webserver php admin/tool/phpunit/cli/util.php --buildcomponentconfigs # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search # bin/moodle-docker-compose down -v If you don't apply the PR you'll get: # bin/moodle-docker-compose exec webserver php vendor/bin/phpunit -c search Moodle 3.6dev+ (Build: 20181031), 8b019fb70930c48cda037968d3257edc222f8143 Php: 7.1.22, mysqli: 5.7.24, OS: Linux 3.10.0-862.14.4.el7.x86_64 x86_64 PHPUnit 6.5.8 by Sebastian Bergmann and contributors.   ...........EEEEEEEEEEEEEEEEEEEEEEEEEEEEE......................... 65 / 92 ( 70%) ........................... 92 / 92 (100%)   Time: 43.53 seconds, Memory: 76.00MB   There were 29 errors: [...] and if you tail the Squid logs within the proxy container you'll get: # tail -f /var/log/squid3/access.log 1541196411.322 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196411.976 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196412.601 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196413.232 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196413.843 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196414.465 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196415.101 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196415.741 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196416.375 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196416.993 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196417.620 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196418.246 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196418.876 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196419.502 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196420.129 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196420.765 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196421.391 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196422.021 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196422.659 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196423.294 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196423.916 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196424.550 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196425.176 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196425.812 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196426.451 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196427.077 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196427.716 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196428.355 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html 1541196428.983 0 172.19.0.12 TCP_DENIED/407 3962 POST http://solr:8983/solr/test/update/? - HIER_NONE/- text/html

    Description

      While working on understanding why I'm not able to let the PHP solr extension pass Moodle tests on the docker dev env I've found that \search_solr\engine::get_search_client() doesn't support authenticated proxies.

      Attachments

        Issue Links

          has a non-specific relationship to

          Bug - A problem which impairs or prevents Moodle from functioning correctly. MDL-60346 core_search: Solr connection ignores proxy settings

          • Minor - Minor loss of function where workaround is possible
          • Closed

          Activity

            People

              matteo Matteo Scaramuccia
              matteo Matteo Scaramuccia
              Victor Déniz Falcón Victor Déniz Falcón
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                12/Nov/18