Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-74096

Various errors and failures in mlbackend phpunit tests

    XMLWordPrintable

Details

    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MOODLE_310_STABLE, MOODLE_311_STABLE, MOODLE_39_STABLE
    • MDL-74096-311
    • MDL-74096-master
    • Hide

      Setup

      1. Run Moodle ML backend via docker:
        • For master 

          docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:3.0.4-python3.7.5

        • For 311 and below: (when testing on other mlbackend container, you may need to stop the container running the 3.0.4 version. "docker stop [container_name]") 

          docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:2.6.5-python3.7.5

      2. Setup the needed constants in config.php: 

        define('TEST_MLBACKEND_PYTHON_HOST', 'localhost');
        		 
        define('TEST_MLBACKEND_PYTHON_PORT', 5000);
        		 
        define('TEST_MLBACKEND_PYTHON_USERNAME', 'default');
        		 
        define('TEST_MLBACKEND_PYTHON_PASSWORD', 'sshhhh');
        		 
        define('PHPUNIT_LONGTEST', true);

      Phpunit

      1. Run analytics unit tests 

        vendor/bin/phpunit -v --testsuite core_analytics_testsuite

      2. Ensure tests are passing

      Behat

      1. mdk behat -r -n "Create a model"
      2. mdk behat -r -n "Evaluate a model"
      3. Ensure tests are passing
      Show
      Setup Run Moodle ML backend via docker: For master docker run -d -p 5000 : 5000 moodlehq/moodle-mlbackend-python: 3.0 . 4 -python3. 7.5 For 311 and below: (when testing on other mlbackend container, you may need to stop the container running the 3.0.4 version. " docker stop [container_name] ") docker run -d -p 5000 : 5000 moodlehq/moodle-mlbackend-python: 2.6 . 5 -python3. 7.5 Setup the needed constants in config.php: define( 'TEST_MLBACKEND_PYTHON_HOST' , 'localhost' ); define( 'TEST_MLBACKEND_PYTHON_PORT' , 5000 ); define( 'TEST_MLBACKEND_PYTHON_USERNAME' , 'default' ); define( 'TEST_MLBACKEND_PYTHON_PASSWORD' , 'sshhhh' ); define( 'PHPUNIT_LONGTEST' , true ); Phpunit Run analytics unit tests vendor/bin/phpunit -v --testsuite core_analytics_testsuite Ensure tests are passing Behat mdk behat -r -n "Create a model" mdk behat -r -n "Evaluate a model" Ensure tests are passing
    • Moppies Kanban

    Description

      It seems that unit tests are not passing against the python mlbackend.

      To reproduce

      • Run the mlbackend docker image: 

        docker run -d -p 5000:5000 moodlehq/moodle-mlbackend-python:3.0.2-python3.7.5

      • Setup the needed constants in config.php: 

        define('TEST_MLBACKEND_PYTHON_HOST', 'localhost');
        		 
        define('TEST_MLBACKEND_PYTHON_PORT', 5000);
        		 
        define('TEST_MLBACKEND_PYTHON_USERNAME', 'default');
        		 
        define('TEST_MLBACKEND_PYTHON_PASSWORD', 'sshhhh');

      • Init PHPUnit and run the core_analytics_prediction_testcase testcase: 

        vendor/bin/phpunit -v --filter core_analytics_prediction_testcase

      Expected

      The tests pass ok.

      Current status

      There are errors (2) and failures (2)

      Moodle 4.0beta (Build: 20220303), 87791a13a413498116fc5f081f5e48256f6e492f
      		 
      Php: 8.0.16, mysqli: 8.0.27, OS: Darwin 20.6.0 x86_64
      		 
      PHPUnit 9.5.13 by Sebastian Bergmann and contributors.
      		 
       
      		 
      Runtime:       PHP 8.0.16
      		 
      Configuration: /Users/stronk7/git_moodle/testing/phpunit.xml
      		 
       
      		 
      .....F.E.........F.....E..                                        26 / 26 (100%)
      		 
       
      		 
      Time: 02:49.630, Memory: 291.00 MB
      		 
       
      		 
      There were 2 errors:
      		 
       
      		 
      1) core_analytics_prediction_testcase::test_ml_export_import with data set "case-\mlbackend_python\processor-server" ('\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      		 
      moodle_exception: Server error HTTP status code 500: 1
      		 
       
      		 
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:642
      		 
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:401
      		 
      /Users/stronk7/git_moodle/testing/analytics/classes/model_config.php:91
      		 
      /Users/stronk7/git_moodle/testing/analytics/classes/model.php:1605
      		 
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:392
      		 
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      		 
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
      		 
       
      		 
      2) core_analytics_prediction_testcase::test_ml_evaluation_trained_model with data set "case-\mlbackend_python\processor-server" ('\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      		 
      moodle_exception: Server error HTTP status code 500: 1
      		 
       
      		 
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:642
      		 
      /Users/stronk7/git_moodle/testing/lib/mlbackend/python/classes/processor.php:401
      		 
      /Users/stronk7/git_moodle/testing/analytics/classes/model_config.php:91
      		 
      /Users/stronk7/git_moodle/testing/analytics/classes/model.php:1605
      		 
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:684
      		 
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      		 
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
      		 
       
      		 
      --
      		 
       
      		 
      There were 2 failures:
      		 
       
      		 
      1) core_analytics_prediction_testcase::test_ml_training_and_prediction with data set "quarters-\mlbackend_python\processor-server" ('\core\analytics\time_splittin...arters', 3, 4, '\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      		 
      Failed asserting that '0' matches expected 1.
      		 
       
      		 
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:254
      		 
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      		 
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
      		 
       
      		 
      2) core_analytics_prediction_testcase::test_ml_multi_classifier with data set "notimesplitting-\mlbackend_python\processor-server" ('\core\analytics\time_splittin...itting', '\mlbackend_python\processor', array(array(true, '127.0.0.1', 5000, false, 'default', 'sshhhh')))
      		 
      Failed asserting that '1' matches expected 2.
      		 
       
      		 
      /Users/stronk7/git_moodle/testing/analytics/tests/prediction_test.php:578
      		 
      /Users/stronk7/git_moodle/testing/lib/phpunit/classes/advanced_testcase.php:80
      		 
      /private/tmp/composerbase/testing/400/vendor/phpunit/phpunit/phpunit:98
      		 
       
      		 
      ERRORS!
      		 
      Tests: 26, Assertions: 213, Errors: 2, Failures: 2.

      This has not been tested using local server mlbackend instalation. Only using the docker one. Also, it may affect older versions.

      Attachments

        1. MDL-74096_master_behat_1.png
          MDL-74096_master_behat_1.png
          122 kB
        2. MDL-74096_master_behat_2.png
          MDL-74096_master_behat_2.png
          121 kB
        3. MDL-74096_master_phpunit.png
          MDL-74096_master_phpunit.png
          51 kB
        4. MDL-74096_v310_behat_1.png
          MDL-74096_v310_behat_1.png
          130 kB
        5. MDL-74096_v310_behat_2.png
          MDL-74096_v310_behat_2.png
          131 kB
        6. MDL-74096_v310_phpunit.png
          MDL-74096_v310_phpunit.png
          52 kB
        7. MDL-74096_v311_behat_1.png
          MDL-74096_v311_behat_1.png
          123 kB
        8. MDL-74096_v311_behat_2.png
          MDL-74096_v311_behat_2.png
          124 kB
        9. MDL-74096_v311_phpunit.png
          MDL-74096_v311_phpunit.png
          51 kB
        10. MDL-74096_v39_behat_1.png
          MDL-74096_v39_behat_1.png
          121 kB
        11. MDL-74096_v39_behat_2.png
          MDL-74096_v39_behat_2.png
          121 kB
        12. MDL-74096_v39_phpunit.png
          MDL-74096_v39_phpunit.png
          56 kB

        Issue Links

          blocks

          Functional Test - A test of a single Moodle feature MDLQA-16576 CLONE - Admins can change analytics settings and setup different machine learning backends

          • Minor - Minor loss of function where workaround is possible
          • Passed

          Functional Test - A test of a single Moodle feature MDLQA-16623 CLONE - An admin can set up the Python machine learning backend on a separate server

          • Minor - Minor loss of function where workaround is possible
          • Passed
          Discovered while testing

          Functional Test - A test of a single Moodle feature MDLQA-16576 CLONE - Admins can change analytics settings and setup different machine learning backends

          • Minor - Minor loss of function where workaround is possible
          • Passed

          Activity

            People

              ilyatregubov Ilya Tregubov
              stronk7 Eloy Lafuente (stronk7)
              Amaia Anabitarte Amaia Anabitarte
              Jun Pataleta Jun Pataleta
              Angelia Dela Cruz Angelia Dela Cruz
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                9/May/22

                Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 3 days, 1 hour, 20 minutes
                  3d 1h 20m