Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48020

Warning log spam "[hudson.tasks.junit.CaseResult getRun] In getOwner(), suiteResult.getParent() is null."

    Details

    • Similar Issues:

      Description

      After upgrading Jenkins plugins (presumably irrelevant durable-task => 1.16): junit 1.21 => 1.22.1

      ***@***:~$ grep "In getOwner" /var/log/jenkins/jenkins.log
      ...
      2017-11-15 10:54:05 WARNING [hudson.tasks.junit.CaseResult getRun]   In getOwner(), suiteResult.getParent() is null.
      ***@***:~$ grep "In getOwner" /var/log/jenkins/jenkins.log | wc -l
      820
      

       

      Actually I had hoped to benefit from JENKINS-27395 (due to having two different stages using the junit step for recording the test results), but in fact everything looks like before, except the new excessive warning log spam. Since the number of tests is in fact 678, I am also wondering about the number of 820 logs.

      The junit step is called like this: junit(keepLongStdio: true, testResults: phpUnitXmlFilename); as aforementioned in two different stages for different XML Files, each one stemming from PHPUnit 6.4.3

        Attachments

          Issue Links

            Activity

            Hide
            abayer Andrew Bayer added a comment -

            Any chance you could post your Jenkinsfile and your test reports xml?

            Show
            abayer Andrew Bayer added a comment - Any chance you could post your Jenkinsfile and your test reports xml?
            Hide
            abayer Andrew Bayer added a comment -

            Ok, got one part - the failed tests lists don't include the stage name. If you drill down in the full test list to a particular test class, you'll see the test names including the stage name, but not in the list of failed tests, or the list of packages or classes. The latter two are sadly just what we're stuck with - it's entirely possible for a package or class result to contain tests from multiple stages, so there's no viable way I've found yet to connect package/class results with a specific stage. The failed tests, though, should have the stage included, but they pull their display name differently than in the case list, so I need to fix that.

            I can't reproduce the warnings, though - as mentioned, your Jenkinsfile, and particularly your test results XML file that the step is reading, would be hugely helpful for that.

            Show
            abayer Andrew Bayer added a comment - Ok, got one part - the failed tests lists don't include the stage name. If you drill down in the full test list to a particular test class, you'll see the test names including the stage name, but not in the list of failed tests, or the list of packages or classes. The latter two are sadly just what we're stuck with - it's entirely possible for a package or class result to contain tests from multiple stages, so there's no viable way I've found yet to connect package/class results with a specific stage. The failed tests, though, should have the stage included, but they pull their display name differently than in the case list, so I need to fix that. I can't reproduce the warnings, though - as mentioned, your Jenkinsfile, and particularly your test results XML file that the step is reading, would be hugely helpful for that.
            Hide
            abayer Andrew Bayer added a comment -

            Check that, got the warning. It shows up when getting trends. Working on it.

            Show
            abayer Andrew Bayer added a comment - Check that, got the warning. It shows up when getting trends. Working on it.
            Hide
            reinholdfuereder Reinhold Füreder added a comment -

            Well, the Jenkinsfile actually depends heavily on a shared pipeline library, but the basic principle is that in two different stages ("docker-unit-tests" and "docker-integration-tests") PHPUnit (v6.4.3) is executed in a Docker Container via "phpunit --log-junit phpunit-result.xml ..."
            ... and the resulting XML file processed like this:

            def phpUnitXmlFilename = 'phpunit-result.xml'
            ...
            junit(keepLongStdio: true, testResults: phpUnitXmlFilename)
            

            (Maybe mind that it is "only" one single XML file in each run, and that the filename does not change.)

            Which of the test reports XML do you need: the one written by Jenkins in "/var/lib/jenkins/jobs/<job name>/builds/<build number>/junitResult.xml" or the PHPUnit output files?

            The former one looks like this:

            <?xml version='1.0' encoding='UTF-8'?>
            <result plugin="junit@1.22.1">
              <suites>
                <suite>
                  <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file>
                  <name>ACME_Accounting_Accountings_FileImporters_UBSTest</name>
                  <duration>0.009598</duration>
                  <time>0.009598</time>
                  <nodeId>51</nodeId>
                  <enclosingBlocks>
                    <string>34</string>
                  </enclosingBlocks>
                  <enclosingBlockNames>
                    <string>docker-unit-tests</string>
                  </enclosingBlockNames>
                  <cases>
                    <case>
                      <duration>0.00624</duration>
                      <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className>
                      <testName>testCheckFile</testName>
                      <skipped>false</skipped>
                      <failedSince>0</failedSince>
                    </case>
                    <case>
                      <duration>0.002944</duration>
                      <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className>
                      <testName>testImport</testName>
                      <skipped>false</skipped>
                      <failedSince>0</failedSince>
                    </case>
                    <case>
                      <duration>4.14E-4</duration>
                      <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className>
                      <testName>testPossibleTenantsKeys</testName>
                      <skipped>false</skipped>
                      <failedSince>0</failedSince>
                    </case>
                  </cases>
                </suite>
                <suite>
                  <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file>
                  <name>ACME_Accounting_Export_Abacus_GeneratorTest</name>
            ...
                <suite>
                  <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file>
                  <name>TestFramework_DbFixtureGenerator_MeasuringModelBuilderTest</name>
                  <duration>0.017027</duration>
                  <time>0.017027</time>
                  <nodeId>80</nodeId>
                  <enclosingBlocks>
                    <string>61</string>
                  </enclosingBlocks>
                  <enclosingBlockNames>
                    <string>docker-integration-tests</string>
                  </enclosingBlockNames>
                  <cases>
                    <case>
                      <duration>0.004158</duration>
                      <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className>
                      <testName>testCreate</testName>
                      <skipped>false</skipped>
                      <failedSince>0</failedSince>
                    </case>
                    <case>
                      <duration>0.00341</duration>
                      <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className>
                      <testName>testCreateNotCurrent</testName>
                      <skipped>false</skipped>
                      <failedSince>0</failedSince>
                    </case>
                    <case>
                      <duration>0.004291</duration>
                      <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className>
                      <testName>testCreateWithOneDefaultPoint</testName>
                      <skipped>false</skipped>
                      <failedSince>0</failedSince>
                    </case>
                    <case>
                      <duration>0.005168</duration>
                      <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className>
                      <testName>testCreateWithPoints</testName>
                      <skipped>false</skipped>
                      <failedSince>0</failedSince>
                    </case>
                  </cases>
                </suite>
              </suites>
              <duration>16.892485</duration>
              <keepLongStdio>true</keepLongStdio>
            </result>
            

            The latter ones I don't have now (they are thrown away), but I could of course try to get them if needed...

            Oh, I just found one from yesterday which should IMHO still be up-to-date:

            <?xml version="1.0" encoding="UTF-8"?>
            <testsuites>
              <testsuite name="./unittests/unit/" tests="482" assertions="925" errors="0" failures="0" skipped="0" time="0.780566">
                <testsuite name="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" tests="3" assertions="76" errors="0" failures="0" skipped="0" time="0.010088">
                  <testcase name="testCheckFile" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="11" assertions="2" time="0.006981"/>
                  <testcase name="testImport" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="21" assertions="73" time="0.002706"/>
                  <testcase name="testPossibleTenantsKeys" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="45" assertions="1" time="0.000401"/>
                </testsuite>
                <testsuite name="ACME\Accounting\Export\Abacus\GeneratorTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/export/abacus/GeneratorTest.php" tests="1" assertions="1" errors="0" failures="0" skipped="0" time="0.013121">
            ...
              </testsuite>
            </testsuites>
            
            Show
            reinholdfuereder Reinhold Füreder added a comment - Well, the Jenkinsfile actually depends heavily on a shared pipeline library, but the basic principle is that in two different stages ("docker-unit-tests" and "docker-integration-tests") PHPUnit (v6.4.3) is executed in a Docker Container via " phpunit --log-junit phpunit-result.xml ... " ... and the resulting XML file processed like this: def phpUnitXmlFilename = 'phpunit-result.xml' ... junit(keepLongStdio: true , testResults: phpUnitXmlFilename) (Maybe mind that it is "only" one single XML file in each run, and that the filename does not change.) Which of the test reports XML do you need: the one written by Jenkins in "/var/lib/jenkins/jobs/<job name>/builds/<build number>/junitResult.xml" or the PHPUnit output files? The former one looks like this: <?xml version='1.0' encoding='UTF-8'?> <result plugin="junit@1.22.1"> <suites> <suite> <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file> <name>ACME_Accounting_Accountings_FileImporters_UBSTest</name> <duration>0.009598</duration> <time>0.009598</time> <nodeId>51</nodeId> <enclosingBlocks> <string>34</string> </enclosingBlocks> <enclosingBlockNames> <string>docker-unit-tests</string> </enclosingBlockNames> <cases> <case> <duration>0.00624</duration> <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className> <testName>testCheckFile</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>0.002944</duration> <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className> <testName>testImport</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>4.14E-4</duration> <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className> <testName>testPossibleTenantsKeys</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> </cases> </suite> <suite> <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file> <name>ACME_Accounting_Export_Abacus_GeneratorTest</name> ... <suite> <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file> <name>TestFramework_DbFixtureGenerator_MeasuringModelBuilderTest</name> <duration>0.017027</duration> <time>0.017027</time> <nodeId>80</nodeId> <enclosingBlocks> <string>61</string> </enclosingBlocks> <enclosingBlockNames> <string>docker-integration-tests</string> </enclosingBlockNames> <cases> <case> <duration>0.004158</duration> <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className> <testName>testCreate</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>0.00341</duration> <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className> <testName>testCreateNotCurrent</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>0.004291</duration> <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className> <testName>testCreateWithOneDefaultPoint</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>0.005168</duration> <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className> <testName>testCreateWithPoints</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> </cases> </suite> </suites> <duration>16.892485</duration> <keepLongStdio>true</keepLongStdio> </result> The latter ones I don't have now (they are thrown away), but I could of course try to get them if needed... Oh, I just found one from yesterday which should IMHO still be up-to-date: <?xml version="1.0" encoding="UTF-8"?> <testsuites> <testsuite name="./unittests/unit/" tests="482" assertions="925" errors="0" failures="0" skipped="0" time="0.780566"> <testsuite name="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" tests="3" assertions="76" errors="0" failures="0" skipped="0" time="0.010088"> <testcase name="testCheckFile" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="11" assertions="2" time="0.006981"/> <testcase name="testImport" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="21" assertions="73" time="0.002706"/> <testcase name="testPossibleTenantsKeys" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="45" assertions="1" time="0.000401"/> </testsuite> <testsuite name="ACME\Accounting\Export\Abacus\GeneratorTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/export/abacus/GeneratorTest.php" tests="1" assertions="1" errors="0" failures="0" skipped="0" time="0.013121"> ... </testsuite> </testsuites>
            Hide
            abayer Andrew Bayer added a comment -

            Perfect - thanks!

            Show
            abayer Andrew Bayer added a comment - Perfect - thanks!
            Hide
            reinholdfuereder Reinhold Füreder added a comment - - edited

            Frankly I am not sure that I understood your comment in https://issues.jenkins-ci.org/browse/JENKINS-48020?focusedCommentId=319785&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-319785

            Note that in my example:

            • there were no failing tests
            • there are definitely different test cases in same test classes (= test classes with same name) executed in the two different PHPUnit runs (filtered executions based on "@group" test annotations)
              • but I would strongly assume there are some "unique" test classes per PHPUnit run as well
            Show
            reinholdfuereder Reinhold Füreder added a comment - - edited Frankly I am not sure that I understood your comment in https://issues.jenkins-ci.org/browse/JENKINS-48020?focusedCommentId=319785&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-319785 Note that in my example: there were no failing tests there are definitely different test cases in same test classes (= test classes with same name) executed in the two different PHPUnit runs (filtered executions based on " @group " test annotations) but I would strongly assume there are some "unique" test classes per PHPUnit run as well
            Hide
            abayer Andrew Bayer added a comment -

            Basically, I was saying that the stage info isn't shown except for on individual tests - so, not in the list of packages on the initial test results screen, not on the list of classes when you click on one of the packages. You will see it in the list of cases within an individual test class, though. And with https://github.com/jenkinsci/junit-plugin/pull/89, the lists of failed tests will have the stage info as well.

            (this is all in the classic UI - Blue Ocean isn't yet updated to take advantage of this information)

            Show
            abayer Andrew Bayer added a comment - Basically, I was saying that the stage info isn't shown except for on individual tests - so, not in the list of packages on the initial test results screen, not on the list of classes when you click on one of the packages. You will see it in the list of cases within an individual test class, though. And with https://github.com/jenkinsci/junit-plugin/pull/89 , the lists of failed tests will have the stage info as well. (this is all in the classic UI - Blue Ocean isn't yet updated to take advantage of this information)
            Hide
            abayer Andrew Bayer added a comment -

            Also, I slept badly last night so may not be fully coherent. My apologies. =)

            Show
            abayer Andrew Bayer added a comment - Also, I slept badly last night so may not be fully coherent. My apologies. =)
            Hide
            reinholdfuereder Reinhold Füreder added a comment -

            OK, now I got it, thanks for your patience; and I can actually see/confirm that ("shown for individual tests")

            Just out of interest: are there any plans to pimp the visualization and not merge all test results together, but – frankly I don't know what the best UX would be – have one TestResult tab/entry per run or stage or so?

             

            Show
            reinholdfuereder Reinhold Füreder added a comment - OK, now I got it, thanks for your patience; and I can actually see/confirm that ("shown for individual tests") Just out of interest: are there any plans to pimp the visualization and not merge all test results together, but – frankly I don't know what the best UX would be – have one TestResult tab/entry per run or stage or so?  
            Hide
            abayer Andrew Bayer added a comment -

            Maaaaybe? That's definitely on the roadmap for Blue Ocean, and I'll see if I can figure out something that makes sense for classic UI as well.

            Show
            abayer Andrew Bayer added a comment - Maaaaybe? That's definitely on the roadmap for Blue Ocean, and I'll see if I can figure out something that makes sense for classic UI as well.
            Hide
            abayer Andrew Bayer added a comment -

            Releasing 1.22.2 with this fix right now.

            Show
            abayer Andrew Bayer added a comment - Releasing 1.22.2 with this fix right now.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Andrew Bayer
            Path:
            src/main/java/hudson/tasks/junit/CaseResult.java
            src/main/java/hudson/tasks/junit/SuiteResult.java
            src/test/java/hudson/tasks/junit/pipeline/JUnitResultsStepTest.java
            http://jenkins-ci.org/commit/junit-plugin/9ec93955137b9eeba7715e6f118e558be7dbb712
            Log:
            [FIXED JENKINS-48020] Show stage in failed tests, fix warnings

            First, this changes `CaseResult#getFullDisplayName()` to include
            stage/branch names like `CaseResult#getDisplayName()`, so that the
            full information will show up in lists of failed tests, for example.

            Second, stop using `CaseResult#getDisplayName()` for indexing
            `CaseResult`s in `SuiteResult`. That method will spam warnings if
            called too early in the run now due to `CaseResult#getRun()` being
            called before the `SuiteResult`'s parent is set.

            Third, fix `JUnitResultsStepTest#testTrends` to actually test what
            it's supposed to, to fail if it doesn't find the expected display
            names, to test full display names as well, and to use the same
            filename for the test results each time due to one of the stages' test
            file not containing an explicit suite name, leading to its suite's
            name being set to the filename, breaking `SuiteResult#getPreviousResult()`.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: src/main/java/hudson/tasks/junit/CaseResult.java src/main/java/hudson/tasks/junit/SuiteResult.java src/test/java/hudson/tasks/junit/pipeline/JUnitResultsStepTest.java http://jenkins-ci.org/commit/junit-plugin/9ec93955137b9eeba7715e6f118e558be7dbb712 Log: [FIXED JENKINS-48020] Show stage in failed tests, fix warnings First, this changes `CaseResult#getFullDisplayName()` to include stage/branch names like `CaseResult#getDisplayName()`, so that the full information will show up in lists of failed tests, for example. Second, stop using `CaseResult#getDisplayName()` for indexing `CaseResult`s in `SuiteResult`. That method will spam warnings if called too early in the run now due to `CaseResult#getRun()` being called before the `SuiteResult`'s parent is set. Third, fix `JUnitResultsStepTest#testTrends` to actually test what it's supposed to, to fail if it doesn't find the expected display names, to test full display names as well, and to use the same filename for the test results each time due to one of the stages' test file not containing an explicit suite name, leading to its suite's name being set to the filename, breaking `SuiteResult#getPreviousResult()`.
            Hide
            reinholdfuereder Reinhold Füreder added a comment -

            Thanks a lot for the quick fixes => looks good to me (at a first quick glance)

            Show
            reinholdfuereder Reinhold Füreder added a comment - Thanks a lot for the quick fixes => looks good to me (at a first quick glance)

              People

              • Assignee:
                abayer Andrew Bayer
                Reporter:
                reinholdfuereder Reinhold Füreder
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: