Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-81698

Fix all unit tests that aren't running ok one by one

XMLWordPrintable

    • MOODLE_403_STABLE, MOODLE_404_STABLE, MOODLE_405_STABLE
    • MOODLE_403_STABLE, MOODLE_404_STABLE
    • MDL-81698_403
    • MDL-81698_404
    • Hide

      This can be considered automatically tested:

      1. Verify that CI jobs are passing without problem.
      2. Verify that all the GHA jobs in the issue are passing without problem.

      If, still, you want to test this manually:

      1. Alternative #1 (Locally):
        1. Download the "phpunit_run_indvidual_tests.sh" script attached
        2. Copy it to your Moodle root dir.
          • You might need to give execution permissions: chmod +x phpunit_run_indvidual_tests.sh
        3. Launch it
        4. Verify the the execution (can a good number of hours!) ends without any "Error:" in the output.
      1. Alternative #2 (GitHub):
        1. Edit ".github/workflows/onebyone.yml"
        2. Add a new line with "push:" contents after the "on:" line, so it looks like: 

          ...
          		 
          ...
          		 
          # own isolation or any other option.
          		 
           
          on:
          		 
            push:
          		 
            workflow_dispatch:
          		 
              inputs:
          		 
                phpunit_extra_options:
          		 
          ...
          		 
          ...

        3. Make a commit with the changes above.
        4. Push the branch to your GitHub moodle.git clone.
        5. Examine your running actions @ https://github.com/YOURUSERNAME/moodle/actions
        6. Verify that both the "Core" and the "One by One Testing" workflows finish without error.
      Show
      This can be considered automatically tested: Verify that CI jobs are passing without problem. Verify that all the GHA jobs in the issue are passing without problem. If, still, you want to test this manually: Alternative #1 (Locally): Download the "phpunit_run_indvidual_tests.sh" script attached Copy it to your Moodle root dir. You might need to give execution permissions: chmod +x phpunit_run_indvidual_tests.sh Launch it Verify the the execution (can a good number of hours!) ends without any "Error:" in the output. Alternative #2 (GitHub): Edit ".github/workflows/onebyone.yml" Add a new line with "push:" contents after the "on:" line, so it looks like: ... ... # own isolation or any other option. on: push: workflow_dispatch: inputs: phpunit_extra_options: ... ... Make a commit with the changes above. Push the branch to your GitHub moodle.git clone. Examine your running actions @ https://github.com/YOURUSERNAME/moodle/actions Verify that both the "Core" and the "One by One Testing" workflows finish without error.

      This comes from MDLSITE-6625, where some scripts and GH workflows have been created to be able to detect which tests have dependencies not satisfied when running individually.

      There are 2 ways to run tests individually:

      A) Manually, when you just want to run one test (with --filter + file, for example).
      B) Configured, relying on PHPUnit's own --process-isolation feature

      In this issue we are focussing on the first ones (A), to guarantee that all the tests can be run individually and there aren't missing dependencies, normally not detected when all the tests are run together. Work on the laters (B) will continue in the linked MDLSITE.

      So this issue is about to:

      • Include a new GH workflow, that can be run under demand (pretty much like the current windows one) that will be able to detect any new problem by running the tests one by one.
      • Fix all cases in core until we get the workflow passing.

      As of April 26th 2024, we have 38/39 failures detected, they are listed in this GH workflow run. We'll take the information there as the initial point to go fixing tests.

      Ciao

      PS: Some runs of the new workflow (manual triggered/dispatched only) with the patches already applied:

        1. phpunit_run_indvidual_tests.sh
          3 kB

            stronk7 Eloy Lafuente (stronk7)
            stronk7 Eloy Lafuente (stronk7)
            Simey Lameze Simey Lameze
            Andrew Lyons Andrew Lyons
            CiBoT CiBoT
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 day, 6 hours, 44 minutes
                1d 6h 44m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.