Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-74096

Various errors and failures in mlbackend phpunit tests

    XMLWordPrintable

Details

    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MOODLE_310_STABLE, MOODLE_311_STABLE, MOODLE_39_STABLE
    • MDL-74096-311
    • MDL-74096-master
    • Hide

      Setup

      1. Run Moodle ML backend via docker:
        • For master

          docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:3.0.4-python3.7.5

        • For 311 and below: (when testing on other mlbackend container, you may need to stop the container running the 3.0.4 version. "docker stop [container_name]")

          docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:2.6.5-python3.7.5

      2. Setup the needed constants in config.php:

        define('TEST_MLBACKEND_PYTHON_HOST', 'localhost');
        define('TEST_MLBACKEND_PYTHON_PORT', 5000);
        define('TEST_MLBACKEND_PYTHON_USERNAME', 'default');
        define('TEST_MLBACKEND_PYTHON_PASSWORD', 'sshhhh');
        define('PHPUNIT_LONGTEST', true);
        

      Phpunit

      1. Run analytics unit tests

        vendor/bin/phpunit -v --testsuite core_analytics_testsuite
        

      2. Ensure tests are passing

      Behat

      1. mdk behat -r -n "Create a model"
      2. mdk behat -r -n "Evaluate a model"
      3. Ensure tests are passing
      Show
      Setup Run Moodle ML backend via docker: For master docker run -d -p 5000 : 5000 moodlehq/moodle-mlbackend-python: 3.0 . 4 -python3. 7.5 For 311 and below: (when testing on other mlbackend container, you may need to stop the container running the 3.0.4 version. " docker stop [container_name] ") docker run -d -p 5000 : 5000 moodlehq/moodle-mlbackend-python: 2.6 . 5 -python3. 7.5 Setup the needed constants in config.php: define( 'TEST_MLBACKEND_PYTHON_HOST' , 'localhost' ); define( 'TEST_MLBACKEND_PYTHON_PORT' , 5000 ); define( 'TEST_MLBACKEND_PYTHON_USERNAME' , 'default' ); define( 'TEST_MLBACKEND_PYTHON_PASSWORD' , 'sshhhh' ); define( 'PHPUNIT_LONGTEST' , true ); Phpunit Run analytics unit tests vendor/bin/phpunit -v --testsuite core_analytics_testsuite Ensure tests are passing Behat mdk behat -r -n "Create a model" mdk behat -r -n "Evaluate a model" Ensure tests are passing
    • Moppies Kanban

    Description

      It seems that unit tests are not passing against the python mlbackend.

      To reproduce

      • Run the mlbackend docker image:

        docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:3.0.2-python3.7.5
        

      • Setup the needed constants in config.php:

        define('TEST_MLBACKEND_PYTHON_HOST', 'localhost');
        define('TEST_MLBACKEND_PYTHON_PORT', 5000);
        define('TEST_MLBACKEND_PYTHON_USERNAME', 'default');
        define('TEST_MLBACKEND_PYTHON_PASSWORD', 'sshhhh');
        

      • Init PHPUnit and run the core_analytics_prediction_testcase testcase:

        vendor/bin/phpunit -v --filter core_analytics_prediction_testcase
        

      Expected

      The tests pass ok.

      Current status

      There are errors (2) and failures (2)

      Moodle 4.0beta (Build: 20220303), 87791a13a413498116fc5f081f5e48256f6e492f
      Php: 8.0.16, mysqli: 8.0.27, OS: Darwin 20.6.0 x86_64
      PHPUnit 9.5.13 by Sebastian Bergmann and contributors.
       
      Runtime:       PHP 8.0.16
      Configuration: /Users/stronk7/git_moodle/testing/phpunit.xml
       
      .....F.E.........F.....E..                                        26 / 26 (100%)
       
      Time: 02:49.630, Memory: 291.00 MB
       
      There were 2 errors:
       
      1) core_analytics_prediction_testcase::test_ml_export_import with data set "case-\mlbackend_python\processor-server" ('\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      moodle_exception: Server error HTTP status code 500: 1
       
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:642
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:401
      /Users/stronk7/git_moodle/testing/analytics/classes/model_config.php:91
      /Users/stronk7/git_moodle/testing/analytics/classes/model.php:1605
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:392
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
       
      2) core_analytics_prediction_testcase::test_ml_evaluation_trained_model with data set "case-\mlbackend_python\processor-server" ('\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      moodle_exception: Server error HTTP status code 500: 1
       
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:642
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:401
      /Users/stronk7/git_moodle/testing/analytics/classes/model_config.php:91
      /Users/stronk7/git_moodle/testing/analytics/classes/model.php:1605
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:684
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
       
      --
       
      There were 2 failures:
       
      1) core_analytics_prediction_testcase::test_ml_training_and_prediction with data set "quarters-\mlbackend_python\processor-server" ('\core\analytics\time_splittin...arters', 3, 4, '\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      Failed asserting that '0' matches expected 1.
       
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:254
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
       
      2) core_analytics_prediction_testcase::test_ml_multi_classifier with data set "notimesplitting-\mlbackend_python\processor-server" ('\core\analytics\time_splittin...itting', '\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      Failed asserting that '1' matches expected 2.
       
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:578
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
       
      ERRORS!
      Tests: 26, Assertions: 213, Errors: 2, Failures: 2.
      

      This has not been tested using local server mlbackend instalation. Only using the docker one. Also, it may affect older versions.

      Attachments

        1. MDL-74096_master_behat_1.png
          MDL-74096_master_behat_1.png
          122 kB
        2. MDL-74096_master_behat_2.png
          MDL-74096_master_behat_2.png
          121 kB
        3. MDL-74096_master_phpunit.png
          MDL-74096_master_phpunit.png
          51 kB
        4. MDL-74096_v310_behat_1.png
          MDL-74096_v310_behat_1.png
          130 kB
        5. MDL-74096_v310_behat_2.png
          MDL-74096_v310_behat_2.png
          131 kB
        6. MDL-74096_v310_phpunit.png
          MDL-74096_v310_phpunit.png
          52 kB
        7. MDL-74096_v311_behat_1.png
          MDL-74096_v311_behat_1.png
          123 kB
        8. MDL-74096_v311_behat_2.png
          MDL-74096_v311_behat_2.png
          124 kB
        9. MDL-74096_v311_phpunit.png
          MDL-74096_v311_phpunit.png
          51 kB
        10. MDL-74096_v39_behat_1.png
          MDL-74096_v39_behat_1.png
          121 kB
        11. MDL-74096_v39_behat_2.png
          MDL-74096_v39_behat_2.png
          121 kB
        12. MDL-74096_v39_phpunit.png
          MDL-74096_v39_phpunit.png
          56 kB

        Issue Links

          Activity

            People

              ilyatregubov Ilya Tregubov
              stronk7 Eloy Lafuente (stronk7)
              Amaia Anabitarte Amaia Anabitarte
              Jun Pataleta Jun Pataleta
              Angelia Dela Cruz Angelia Dela Cruz
              David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                9/May/22

                Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 3 days, 1 hour, 20 minutes
                  3d 1h 20m