Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-74096

Various errors and failures in mlbackend phpunit tests

    XMLWordPrintable

Details

    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MOODLE_310_STABLE, MOODLE_311_STABLE, MOODLE_39_STABLE
    • MDL-74096-master
    • Hide

      Setup

      1. Run Moodle ML backend via docker:
        • For master

          docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:3.0.4-python3.7.5

        • For 311 and below: (when testing on other mlbackend container, you may need to stop the container running the 3.0.4 version. "docker stop [container_name]")

          docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:2.6.5-python3.7.5

      2. Setup the needed constants in config.php:

        define('TEST_MLBACKEND_PYTHON_HOST', 'localhost');
        define('TEST_MLBACKEND_PYTHON_PORT', 5000);
        define('TEST_MLBACKEND_PYTHON_USERNAME', 'default');
        define('TEST_MLBACKEND_PYTHON_PASSWORD', 'sshhhh');
        define('PHPUNIT_LONGTEST', true);
        

      Phpunit

      1. Run analytics unit tests

        vendor/bin/phpunit -v --testsuite core_analytics_testsuite
        

      2. Ensure tests are passing

      Behat

      1. mdk behat -r -n "Create a model"
      2. mdk behat -r -n "Evaluate a model"
      3. Ensure tests are passing
      Show
      Setup Run Moodle ML backend via docker: For master docker run -d -p 5000 : 5000 moodlehq/moodle-mlbackend-python: 3.0 . 4 -python3. 7.5 For 311 and below: (when testing on other mlbackend container, you may need to stop the container running the 3.0.4 version. " docker stop [container_name] ") docker run -d -p 5000 : 5000 moodlehq/moodle-mlbackend-python: 2.6 . 5 -python3. 7.5 Setup the needed constants in config.php: define( 'TEST_MLBACKEND_PYTHON_HOST' , 'localhost' ); define( 'TEST_MLBACKEND_PYTHON_PORT' , 5000 ); define( 'TEST_MLBACKEND_PYTHON_USERNAME' , 'default' ); define( 'TEST_MLBACKEND_PYTHON_PASSWORD' , 'sshhhh' ); define( 'PHPUNIT_LONGTEST' , true ); Phpunit Run analytics unit tests vendor/bin/phpunit -v --testsuite core_analytics_testsuite Ensure tests are passing Behat mdk behat -r -n "Create a model" mdk behat -r -n "Evaluate a model" Ensure tests are passing
    • Moppies Kanban

    Description

      It seems that unit tests are not passing against the python mlbackend.

      To reproduce

      • Run the mlbackend docker image:

        docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:3.0.2-python3.7.5
        

      • Setup the needed constants in config.php:

        define('TEST_MLBACKEND_PYTHON_HOST', 'localhost');
        define('TEST_MLBACKEND_PYTHON_PORT', 5000);
        define('TEST_MLBACKEND_PYTHON_USERNAME', 'default');
        define('TEST_MLBACKEND_PYTHON_PASSWORD', 'sshhhh');
        

      • Init PHPUnit and run the core_analytics_prediction_testcase testcase:

        vendor/bin/phpunit -v --filter core_analytics_prediction_testcase
        

      Expected

      The tests pass ok.

      Current status

      There are errors (2) and failures (2)

      Moodle 4.0beta (Build: 20220303), 87791a13a413498116fc5f081f5e48256f6e492f
      Php: 8.0.16, mysqli: 8.0.27, OS: Darwin 20.6.0 x86_64
      PHPUnit 9.5.13 by Sebastian Bergmann and contributors.
       
      Runtime:       PHP 8.0.16
      Configuration: /Users/stronk7/git_moodle/testing/phpunit.xml
       
      .....F.E.........F.....E..                                        26 / 26 (100%)
       
      Time: 02:49.630, Memory: 291.00 MB
       
      There were 2 errors:
       
      1) core_analytics_prediction_testcase::test_ml_export_import with data set "case-\mlbackend_python\processor-server" ('\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      moodle_exception: Server error HTTP status code 500: 1
       
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:642
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:401
      /Users/stronk7/git_moodle/testing/analytics/classes/model_config.php:91
      /Users/stronk7/git_moodle/testing/analytics/classes/model.php:1605
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:392
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
       
      2) core_analytics_prediction_testcase::test_ml_evaluation_trained_model with data set "case-\mlbackend_python\processor-server" ('\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      moodle_exception: Server error HTTP status code 500: 1
       
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:642
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:401
      /Users/stronk7/git_moodle/testing/analytics/classes/model_config.php:91
      /Users/stronk7/git_moodle/testing/analytics/classes/model.php:1605
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:684
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
       
      --
       
      There were 2 failures:
       
      1) core_analytics_prediction_testcase::test_ml_training_and_prediction with data set "quarters-\mlbackend_python\processor-server" ('\core\analytics\time_splittin...arters', 3, 4, '\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      Failed asserting that '0' matches expected 1.
       
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:254
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
       
      2) core_analytics_prediction_testcase::test_ml_multi_classifier with data set "notimesplitting-\mlbackend_python\processor-server" ('\core\analytics\time_splittin...itting', '\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      Failed asserting that '1' matches expected 2.
       
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:578
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
       
      ERRORS!
      Tests: 26, Assertions: 213, Errors: 2, Failures: 2.
      

      This has not been tested using local server mlbackend instalation. Only using the docker one. Also, it may affect older versions.

      Attachments

        1. MDL-74096_v311_phpunit.png
          MDL-74096_v311_phpunit.png
          51 kB
        2. MDL-74096_master_phpunit.png
          MDL-74096_master_phpunit.png
          51 kB
        3. MDL-74096_v310_phpunit.png
          MDL-74096_v310_phpunit.png
          52 kB
        4. MDL-74096_v311_behat_2.png
          MDL-74096_v311_behat_2.png
          124 kB
        5. MDL-74096_v39_phpunit.png
          MDL-74096_v39_phpunit.png
          56 kB
        6. MDL-74096_master_behat_1.png
          MDL-74096_master_behat_1.png
          122 kB
        7. MDL-74096_master_behat_2.png
          MDL-74096_master_behat_2.png
          121 kB
        8. MDL-74096_v310_behat_1.png
          MDL-74096_v310_behat_1.png
          130 kB
        9. MDL-74096_v310_behat_2.png
          MDL-74096_v310_behat_2.png
          131 kB
        10. MDL-74096_v311_behat_1.png
          MDL-74096_v311_behat_1.png
          123 kB
        11. MDL-74096_v39_behat_1.png
          MDL-74096_v39_behat_1.png
          121 kB
        12. MDL-74096_v39_behat_2.png
          MDL-74096_v39_behat_2.png
          121 kB

        Issue Links

          Activity

            People

              ilyatregubov Ilya Tregubov
              stronk7 Eloy Lafuente (stronk7)
              Amaia Anabitarte Amaia Anabitarte
              Jun Pataleta Jun Pataleta
              Angelia Dela Cruz Angelia Dela Cruz
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 3 days, 1 hour, 20 minutes
                  3d 1h 20m

                  Clockify

                    Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.