Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-71036

Upgrade PHPUnit to 9.5.x

XMLWordPrintable

    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MOODLE_311_STABLE
    • Hide

      Requirements:

      • Ability to edit files.
      • Some coverage extension/tool enabled (xdebug, pcov...)

      Xdebug installation in Ubuntu/LinuxMint

      1. Update your system 

        sudo apt update && sudo apt upgrade -y

      2. Install Xdebug appropriate to your PHP version. e.g. on PHP 7.3: 

        sudo apt install php7.3-xdebug

      3. Check that Xdebug is enabled: 

        php -v

        You should see something like "with Xdebug v3.0.3, Copyright (c) 2002-2021, by Derick Rethans" at the bottom of the output.

      4. Export Xdebug mode environment variable

        export XDEBUG_MODE=coverage

      Automated tests (Integrator to confirm this)

      1. Ensure that all CI test pass ok (Behat included because this issue comes with a noticeable composer.lock changeset).
      2. Also, check that there aren't "W" (warnings about deprecated stuff) in the PHPUnit output.

      Manual tests

      Coverage-related changes

      A - Ensure that you've some coverage extension available.

      1. Run vendor/bin/phpunit -v --coverage-html /tmp --filter mod_h5pactivity_generator_testcase
      2. Verify that, after running the tests you see "Generating code coverage report in HTML format ... done"
      3. Verify that if you open /tmp/index.html in the browser you see a coverage report with 6 directories: admin, grade, h5p, lib, mod, privacy.

      B - Check that the deprecation message when old properties are used is working.

      1. Edit mod/h5pactivity/tests/coverage.php and change $includelistfolders by $whitelistfolders.
      2. Run php admin/tool/phpunit/cli/init.php
      3. Verify that you see the message: "Warning: $whitelistfolders and $whitelistfiles in coverage.php files are deprecated since Moodle 3.11. Please, replace them with $includelistfolders and $includelistfiles in mod/h5pactivity/tests/coverage.php"
      4. Run vendor/bin/phpunit -v --coverage-html /tmp --filter mod_h5pactivity_generator_testcase
      5. Verify that, after running the tests you see "Generating code coverage report in HTML format ... done"
      6. Verify that if you open /tmp/index.html in the browser you see a coverage report with 6 directories: admin, grade, h5p, lib, mod, privacy.
      7. Verify that if you follow the "lib" element, you don't see neither the "xmldb" dir or the "weblib.php" file.
      8. Edit mod/h5pactivity/tests/coverage.php and change back $whitelistfolders to $includelistfolders.

      C - check that adding includes add stuff to coverage report.

      1. Edit lib/tests/coverage.php and add "xmldb" to the $includelistfolders property and add "weblib.php" to the $includelistfiles property.
      2. Run php admin/tool/phpunit/cli/init.php
      3. Verify that there aren't warnings.
      4. Run vendor/bin/phpunit -v --coverage-html /tmp --filter mod_h5pactivity_generator_testcase
      5. Verify that if you open /tmp/index.html in the browser you see a coverage report with 6 directories: admin, grade, h5p, lib, mod, privacy.
      6. Verify that if you follow the "lib" element, you see both the "xmldb" dir and the "weblib.php" file (with some coverage %).
      7. Verify that if you follow the "xmldb" element, you see 8 files, one of them being "xmldb_constants.php".

      D - Check that adding excludes remove stuff from coverage report.

      1. Without un-doing the changes peformed in previous test.
      2. Edit lib/tests/coverage.php and add "classes/access" to the $excludelistfolders property and add "xmldb/xmldb_constants.php" to the $excludelistfiles property.
      3. Run php admin/tool/phpunit/cli/init.php
      4. Verify that there aren't warnings.
      5. Run vendor/bin/phpunit -v --coverage-html /tmp --filter mod_h5pactivity_generator_testcase
      6. Verify that if you open /tmp/index.html in the browser you see a coverage report with 6 directories: admin, grade, h5p, lib, mod, privacy.
      7. Verify that if you follow the lib -> classes links... you don't see the "access" directory.
      8. Verify that if you follow the lib -> xmldb links... you don't see the "xmldb_constants.php" file (but still see another 7 files).

      Execution options related

      E - Verify that suite/config/filter options continue working.

      1. Run php admin/tool/phpunit/cli/init.php
      2. Run php admin/tool/phpunit/cli/util.php --buildcomponentconfigs
      3. Run vendor/bin/phpunit --testsuite mod_h5pactivity_testsuite.
      4. Verify it works and shows around "270 tests, 2467 assertions" (numbers may change a bit).
      5. Run vendor/bin/phpunit -c mod/h5pactivity
      6. Verify it works and shows the same numbers than the previous run.
      7. Run vendor/bin/phpunit --filter mod_h5pactivity_generator_testcase
      8. Verify it works and shows around "17 tests, 191 assertions" (numbers may change a bit).
      9. Run vendor/bin/phpunit --filter ::test_create_file_exception
      10. Verify it works and shows "1 test, 1 assertion"
      11. Run vendor/bin/phpunit --filter mod_h5pactivity_generator_testcase::test_create_file_exception
      12. Verify it works and shows "1 test, 1 assertion"
      Show
      Requirements: Ability to edit files. Some coverage extension/tool enabled (xdebug, pcov...) Xdebug installation in Ubuntu/LinuxMint Update your system sudo apt update && sudo apt upgrade -y Install Xdebug appropriate to your PHP version. e.g. on PHP 7.3: sudo apt install php7.3-xdebug Check that Xdebug is enabled: php -v You should see something like " with Xdebug v3.0.3, Copyright (c) 2002-2021, by Derick Rethans " at the bottom of the output. Export Xdebug mode environment variable export XDEBUG_MODE=coverage Automated tests (Integrator to confirm this) Ensure that all CI test pass ok (Behat included because this issue comes with a noticeable composer.lock changeset). Also, check that there aren't "W" (warnings about deprecated stuff) in the PHPUnit output. Manual tests Coverage-related changes A - Ensure that you've some coverage extension available. Run vendor/bin/phpunit -v --coverage-html /tmp --filter mod_h5pactivity_generator_testcase Verify that, after running the tests you see "Generating code coverage report in HTML format ... done" Verify that if you open /tmp/index.html in the browser you see a coverage report with 6 directories: admin, grade, h5p, lib, mod, privacy. B - Check that the deprecation message when old properties are used is working. Edit mod/h5pactivity/tests/coverage.php and change $includelistfolders by $whitelistfolders . Run php admin/tool/phpunit/cli/init.php Verify that you see the message: "Warning: $whitelistfolders and $whitelistfiles in coverage.php files are deprecated since Moodle 3.11. Please, replace them with $includelistfolders and $includelistfiles in mod/h5pactivity/tests/coverage.php" Run vendor/bin/phpunit -v --coverage-html /tmp --filter mod_h5pactivity_generator_testcase Verify that, after running the tests you see "Generating code coverage report in HTML format ... done" Verify that if you open /tmp/index.html in the browser you see a coverage report with 6 directories: admin, grade, h5p, lib, mod, privacy. Verify that if you follow the "lib" element, you don't see neither the "xmldb" dir or the "weblib.php" file. Edit mod/h5pactivity/tests/coverage.php and change back $whitelistfolders to $includelistfolders . C - check that adding includes add stuff to coverage report. Edit lib/tests/coverage.php and add "xmldb" to the $includelistfolders property and add "weblib.php" to the $includelistfiles property. Run php admin/tool/phpunit/cli/init.php Verify that there aren't warnings. Run vendor/bin/phpunit -v --coverage-html /tmp --filter mod_h5pactivity_generator_testcase Verify that if you open /tmp/index.html in the browser you see a coverage report with 6 directories: admin, grade, h5p, lib, mod, privacy. Verify that if you follow the "lib" element, you see both the "xmldb" dir and the "weblib.php" file (with some coverage %). Verify that if you follow the "xmldb" element, you see 8 files, one of them being "xmldb_constants.php". D - Check that adding excludes remove stuff from coverage report. Without un-doing the changes peformed in previous test. Edit lib/tests/coverage.php and add "classes/access" to the $excludelistfolders property and add "xmldb/xmldb_constants.php" to the $excludelistfiles property. Run php admin/tool/phpunit/cli/init.php Verify that there aren't warnings. Run vendor/bin/phpunit -v --coverage-html /tmp --filter mod_h5pactivity_generator_testcase Verify that if you open /tmp/index.html in the browser you see a coverage report with 6 directories: admin, grade, h5p, lib, mod, privacy. Verify that if you follow the lib -> classes links... you don't see the "access" directory. Verify that if you follow the lib -> xmldb links... you don't see the "xmldb_constants.php" file (but still see another 7 files). Execution options related E - Verify that suite/config/filter options continue working. Run php admin/tool/phpunit/cli/init.php Run php admin/tool/phpunit/cli/util.php --buildcomponentconfigs Run vendor/bin/phpunit --testsuite mod_h5pactivity_testsuite . Verify it works and shows around "270 tests, 2467 assertions" (numbers may change a bit). Run vendor/bin/phpunit -c mod/h5pactivity Verify it works and shows the same numbers than the previous run. Run vendor/bin/phpunit --filter mod_h5pactivity_generator_testcase Verify it works and shows around "17 tests, 191 assertions" (numbers may change a bit). Run vendor/bin/phpunit --filter ::test_create_file_exception Verify it works and shows "1 test, 1 assertion" Run vendor/bin/phpunit --filter mod_h5pactivity_generator_testcase::test_create_file_exception Verify it works and shows "1 test, 1 assertion"

      Right now we are running phpunit 8.5.x (MDL-67673). It's time to bump to newer version.

      This is a "clone" of that issue, with all the still open linked issues dragged here.

      Candidates are, right now: (source link):

      • 9.x: supports php 7.3, 7.4 and 8.0 (that aligns 100% with our supported PHP versions). Mid-size changes required. Fully supported till Feb 2022.
      • 10.x: still not released. Not enough info right now. Maybe to be considered next year.

      Because of a better alignment with Moodle 3.11 PHP supported versions (see ongoing work @ MDL-70745) this issue is about to upgrade to phpunit 9.5. Knowing that changes aren't trivial.

      Sources of information:

      1. Release announcement: https://phpunit.de/announcements/phpunit-9.html
      2. Migrating to phpunit 9 post: https://thephp.cc/news/2020/02/migrating-to-phpunit-9
      3. Changes / deprecations in various PHPUnit releases, not documented above:
        1. PHPUnit 9.0: https://github.com/sebastianbergmann/phpunit/blob/9.0.0/ChangeLog-9.0.md
        2. PHPUnit 9.1: https://github.com/sebastianbergmann/phpunit/blob/9.1.0/ChangeLog-9.1.md
        3. PHPUnit 9.3: https://github.com/sebastianbergmann/phpunit/blob/9.3.0/ChangeLog-9.3.md
        4. PHPUnit 9.5: https://github.com/sebastianbergmann/phpunit/blob/9.5.0/ChangeLog-9.5.md
      4. Attached summary created with all the info above, with the bits affecting us: phpunit95-progress.rtf

      SUMMARY:

      1. Upgrade composer to the target phpunit version (link to procedure).
      2. Fullfill phpunit API required changes.
      3. Fix other incompatibilities detected.
      4. Add a note @ upgrade.txt, pointing to the docs (and maybe main points).
      5. Verify that the new code is passing against ALL php supported versions.
      6. Verify that behat continues passing without problem
      7. Verify that incomplete and skipped tests are shown using the -v[erbose] switch.
      8. Send the issue to peer-review, yay!

      Some notes to reviewer(s).

      • It's recommended to review the patch commit by commit. I've tried to introduce all the important information within each commit message.
      • Don't become crazy about checking the hundreds of changes to assertions 1 by 1. PHPUnit execution will tell you if there is any change missing (will emit warning or failure). Better focus on commits with logic.
      • Verify that there are commits corresponding to all the "DONE" points in the comment used to trace the progress.

      TODO (after rolled upstream):

      1. Create a new section like this @ the development Moodle Docs, explaining the main differences and linking to complete information.
      2. Also fill this table with the new version just added.
      3. Document the coverage_info changes coming with this release. (link)

        1. MDL-71036.jpg
          MDL-71036.jpg
          76 kB
        2. MDL-71036 (2).jpg
          MDL-71036 (2).jpg
          63 kB
        3. MDL-71036 (3).jpg
          MDL-71036 (3).jpg
          52 kB
        4. MDL-71036 (4).jpg
          MDL-71036 (4).jpg
          91 kB
        5. phpunit95-progress.rtf
          12 kB

            stronk7 Eloy Lafuente (stronk7)
            stronk7 Eloy Lafuente (stronk7)
            Nobody Nobody
            Jun Pataleta Jun Pataleta
            Anna Carissa Sadia Anna Carissa Sadia
            Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 3 days, 6 hours, 5 minutes
                3d 6h 5m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.