Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-68405

Regression: grunt gherkinlint not providing correct exit status anymore

XMLWordPrintable

    • MOODLE_37_STABLE, MOODLE_38_STABLE, MOODLE_39_STABLE
    • MOODLE_37_STABLE, MOODLE_38_STABLE
    • MDL-68405-master
    • Hide

      Setup

      1. Open your terminal and go to your Moodle root directory
      2. Run "npm install"

      Test all is good

      1. Run Gherkinlint 

        grunt gherkinlint

        1. Confirm that no errors were reported
      2. Immediately check the exit code: 

        echo $?

        1. Confirm that it was zero
      3. Change directory into mod/forum and run the check again: 

        cd mod/forum
        		 
        grunt gherkinlint ; echo $?

        1. Confirm that there were no errors
        2. Confirm that the exit code was zero

      Master only:

      1. Change directory into mod/h5pactivity and run the check again: 

        cd ../h5pactivity
        		 
        grunt gherkinlint ; echo $?

        1. Confirm that there were no errors
        2. Confirm that the exit code was zero

      Test an introduced failure

      1. Open mod/forum/tests/behat/add_forum.feature in your editor
      2. At the first empty line, add another empty line and save the file
      3. Ensure that you are in the root of the moodle installation
      4. Run Gherkinlint 

        grunt gherkinlint

        1. Confirm that the test reported a failing line
      5. Immediately check the exit code: 

        echo $?

        1. Confirm that it was non-zero (probably 3)
      6. Change directory into mod/forum and run the check again: 

        cd mod/forum
        		 
        grunt gherkinlint ; echo $?

        1. Confirm that errors were found
        2. Confirm the non-zero exit code again
      7. Change directory into mod/chat and run the check again: 

        cd ../chat
        		 
        grunt gherkinlint ; echo $?

        1. Confirm that there were no errors
        2. Confirm that the exit code was zero
      Show
      Setup Open your terminal and go to your Moodle root directory Run " npm install " Test all is good Run Gherkinlint grunt gherkinlint Confirm that no errors were reported Immediately check the exit code: echo $? Confirm that it was zero Change directory into mod/forum and run the check again: cd mod/forum grunt gherkinlint ; echo $? Confirm that there were no errors Confirm that the exit code was zero Master only: Change directory into mod/h5pactivity and run the check again: cd ../h5pactivity grunt gherkinlint ; echo $? Confirm that there were no errors Confirm that the exit code was zero Test an introduced failure Open mod/forum/tests/behat/add_forum.feature in your editor At the first empty line, add another empty line and save the file Ensure that you are in the root of the moodle installation Run Gherkinlint grunt gherkinlint Confirm that the test reported a failing line Immediately check the exit code: echo $? Confirm that it was non-zero (probably 3) Change directory into mod/forum and run the check again: cd mod/forum grunt gherkinlint ; echo $? Confirm that errors were found Confirm the non-zero exit code again Change directory into mod/chat and run the check again: cd ../chat grunt gherkinlint ; echo $? Confirm that there were no errors Confirm that the exit code was zero

      This was detected when working on MDLSITE-6103 because I was getting non-consistent results in some of the tests being added there.

      In that issue (see details) the problem was bisected and found that the change of behavior came with MDL-67953.

      Basically, before that patch... whenever the gherkinlint task finds errors, they are reported, then some warning text is shown and the execution ends with error and exit code (not 0).

      After the patch, the errors are reported exactly the same, but the warning text is not shown and the execution ends without error and with exit code (= 0).

      To reproduce:

      • edit any feature file, change some indentation (to force a gherkin-lint error).
      • run grunt gherkinklint; echo $?
      • expectation: exit code should not be zero.
      • current: exit code is zero.

      Have confirmed that reverting the commit does lead to the expected result.

      Haven't checked if there are other tasks affected the same way. Say stylelint, eslint... all them should lead to error exit code if there are errors.

      Ciao

      PS: I think this explains pretty much why, recently, some gherkin violations have met core and we haven't detected them (in fact that's ultimately the cause I prepared MDLSITE-6103). Since this regression, grunt is not failing any more... hence CiBoT says all ok, so there wasn't any reason to look. Before the regression, grunt was failing and, although it was hard to find the cause (well-hidden in log/xml files in the job) it was enough ro raise alarms and make us look for it.

        1. MDL-68405.jpg
          MDL-68405.jpg
          50 kB
        2. MDL-68405 (2).jpg
          MDL-68405 (2).jpg
          68 kB

            dobedobedoh Andrew Lyons
            stronk7 Eloy Lafuente (stronk7)
            Simey Lameze Simey Lameze
            Jun Pataleta Jun Pataleta
            Anna Carissa Sadia Anna Carissa Sadia
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 days, 6 hours, 24 minutes
                2d 6h 24m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.