Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41134

failures in the windows build are ignored

    Details

    • Similar Issues:

      Description

      the Jenkisfile to build core ignores unit test failures on windows.

      https://ci.jenkins.io/job/Core/job/jenkins/job/master/209/console

      02:49:52.718 [Windows] Tests run: 9198, Failures: 53, Errors: 11, Skipped: 34, Flakes: 3
      02:49:52.718 [Windows] 
      02:49:52.718 [Windows] [ERROR] There are test failures.
      02:49:52.718 [Windows] 
      

      yet https://ci.jenkins.io/job/Core/job/jenkins/job/master/209 is PASSED

        Attachments

          Issue Links

            Activity

            teilo James Nord created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Labels testing
            teilo James Nord made changes -
            Attachment Untitled.png [ 35498 ]
            teilo James Nord made changes -
            Priority Major [ 3 ] Blocker [ 1 ]
            teilo James Nord made changes -
            Component/s junit-plugin [ 15499 ]
            teilo James Nord made changes -
            Description the Jenkisnfile to build core ignores unit test failures on windows.

            https://ci.jenkins.io/job/Core/job/jenkins/job/master/209/console
            {noformat}
            02:49:52.718 [Windows] Tests run: 9198, Failures: 53, Errors: 11, Skipped: 34, Flakes: 3
            02:49:52.718 [Windows]
            02:49:52.718 [Windows] [ERROR] There are test failures.
            02:49:52.718 [Windows]
            {noformat}
            yet https://ci.jenkins.io/job/Core/job/jenkins/job/master/209 is PASSED
            the Jenkisfile to build core ignores unit test failures on windows.

            https://ci.jenkins.io/job/Core/job/jenkins/job/master/209/console
            {noformat}
            02:49:52.718 [Windows] Tests run: 9198, Failures: 53, Errors: 11, Skipped: 34, Flakes: 3
            02:49:52.718 [Windows]
            02:49:52.718 [Windows] [ERROR] There are test failures.
            02:49:52.718 [Windows]
            {noformat}
            yet https://ci.jenkins.io/job/Core/job/jenkins/job/master/209 is PASSED
            Hide
            teilo James Nord added a comment -

            Looks like an issue in the junit plugin?

            there are test failures and it shows them - but at the same time it says "no test failures"

            Show
            teilo James Nord added a comment - Looks like an issue in the junit plugin? there are test failures and it shows them - but at the same time it says "no test failures"
            Hide
            jglick Jesse Glick added a comment -

            You are passing -Dmaven.test.failure.ignore=true so the build correctly moves on. Why the junit step is not marking the build UNSTABLE I am not sure offhand. Would need to narrow it down to a reproducible test case perhaps.

            Show
            jglick Jesse Glick added a comment - You are passing -Dmaven.test.failure.ignore=true so the build correctly moves on. Why the junit step is not marking the build UNSTABLE I am not sure offhand. Would need to narrow it down to a reproducible test case perhaps.
            Hide
            teilo James Nord added a comment -

            I filed this at first as I thought the Jenkinsfile was wrong and was going to do a quick PR. Then I saw that it looked correct and hence my update above.

            Show
            teilo James Nord added a comment - I filed this at first as I thought the Jenkinsfile was wrong and was going to do a quick PR. Then I saw that it looked correct and hence my update above.
            Hide
            danielbeck Daniel Beck added a comment -

            It's JUnit Plugin 1.19 on ci.j.io.

            Show
            danielbeck Daniel Beck added a comment - It's JUnit Plugin 1.19 on ci.j.io.
            Hide
            slide_o_mix Alex Earl added a comment -

            I added a bit of logging to the junit plugin and when it gets to this point:

            https://github.com/jenkinsci/junit-plugin/blob/master/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java#L196

            getFailCount() is returning 0.

            I'll continue to dig in.

            Show
            slide_o_mix Alex Earl added a comment - I added a bit of logging to the junit plugin and when it gets to this point: https://github.com/jenkinsci/junit-plugin/blob/master/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java#L196 getFailCount() is returning 0. I'll continue to dig in.
            Hide
            danielbeck Daniel Beck added a comment -

            Maybe a missing tally() call in mergeResult()?

            Show
            danielbeck Daniel Beck added a comment - Maybe a missing tally() call in mergeResult()?
            Hide
            slide_o_mix Alex Earl added a comment -

            I added a call to tally() in the merge method of TestResult (which amounts to the same thing as putting it in mergeResult(). Rerunning...takes about 3 hours.

            Show
            slide_o_mix Alex Earl added a comment - I added a call to tally() in the merge method of TestResult (which amounts to the same thing as putting it in mergeResult(). Rerunning...takes about 3 hours.
            Hide
            slide_o_mix Alex Earl added a comment -

            That's looks like it is the problem. With the call to tally() in TestResult.merge(), the failures show up correctly and the build is marked as unstable. Not sure if the location I put the call is correct for all use cases, if people think it is, I can create a PR for it for the junit-plugin.

            Show
            slide_o_mix Alex Earl added a comment - That's looks like it is the problem. With the call to tally() in TestResult.merge(), the failures show up correctly and the build is marked as unstable. Not sure if the location I put the call is correct for all use cases, if people think it is, I can create a PR for it for the junit-plugin.
            Hide
            slide_o_mix Alex Earl added a comment -

            This is the change I made to TestResult.java

                void merge(TestResult other) {
                    for (SuiteResult suite : other.suites) {
                        suite.setParent(null); // otherwise freeze ignores it
                        add(suite);
                    }
                    tally();
                }
            

            The call to tally() at the end is the addition.

            Show
            slide_o_mix Alex Earl added a comment - This is the change I made to TestResult.java void merge(TestResult other) { for (SuiteResult suite : other.suites) { suite.setParent( null ); // otherwise freeze ignores it add(suite); } tally(); } The call to tally() at the end is the addition.
            Hide
            danielbeck Daniel Beck added a comment -

            That's where I'd have put it.

            Should be straightforward to write a test for.

            Show
            danielbeck Daniel Beck added a comment - That's where I'd have put it. Should be straightforward to write a test for.
            Show
            slide_o_mix Alex Earl added a comment - See https://github.com/jenkinsci/junit-plugin/pull/63
            slide_o_mix Alex Earl made changes -
            Assignee Alex Earl [ slide_o_mix ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Alex Earl
            Path:
            src/test/java/hudson/tasks/junit/TestResultTest.java
            src/test/resources/hudson/tasks/junit/JENKINS-41134/TestSuite_first.xml
            src/test/resources/hudson/tasks/junit/JENKINS-41134/TestSuite_second.xml
            http://jenkins-ci.org/commit/junit-plugin/c0f470f994693d50d78491bde7deb938903ccd7e
            Log:
            Add test for JENKINS-41134

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Alex Earl Path: src/test/java/hudson/tasks/junit/TestResultTest.java src/test/resources/hudson/tasks/junit/ JENKINS-41134 /TestSuite_first.xml src/test/resources/hudson/tasks/junit/ JENKINS-41134 /TestSuite_second.xml http://jenkins-ci.org/commit/junit-plugin/c0f470f994693d50d78491bde7deb938903ccd7e Log: Add test for JENKINS-41134
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Alex Earl
            Path:
            src/main/java/hudson/tasks/junit/TestResult.java
            src/test/java/hudson/tasks/junit/TestResultTest.java
            src/test/resources/hudson/tasks/junit/JENKINS-41134/TestSuite_first.xml
            src/test/resources/hudson/tasks/junit/JENKINS-41134/TestSuite_second.xml
            src/test/resources/hudson/tasks/junit/JENKINS-41134/TestSuite_second_dup_first.xml
            http://jenkins-ci.org/commit/junit-plugin/631c41350f6eb24520af19083a9f2fe27163f572
            Log:
            Merge pull request #63 from slide/master

            Fix JENKINS-41134

            Compare: https://github.com/jenkinsci/junit-plugin/compare/90e7ddbda961...631c41350f6e

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Alex Earl Path: src/main/java/hudson/tasks/junit/TestResult.java src/test/java/hudson/tasks/junit/TestResultTest.java src/test/resources/hudson/tasks/junit/ JENKINS-41134 /TestSuite_first.xml src/test/resources/hudson/tasks/junit/ JENKINS-41134 /TestSuite_second.xml src/test/resources/hudson/tasks/junit/ JENKINS-41134 /TestSuite_second_dup_first.xml http://jenkins-ci.org/commit/junit-plugin/631c41350f6eb24520af19083a9f2fe27163f572 Log: Merge pull request #63 from slide/master Fix JENKINS-41134 Compare: https://github.com/jenkinsci/junit-plugin/compare/90e7ddbda961...631c41350f6e
            slide_o_mix Alex Earl made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-39696 [ JENKINS-39696 ]
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal OSS-1861 (Web Link)" [ 18536 ]

              People

              • Assignee:
                slide_o_mix Alex Earl
                Reporter:
                teilo James Nord
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: