Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46193

New test failures are reported as both new test failures and existing test failures.

    Details

    • Similar Issues:
    • Epic Link:
    • Sprint:
      Blue Ocean - Candidates
    • Released As:
      1.15.0

      Description

      BlueOcean reports new test failures as both new test failures and existing test failures.

      Example

      Test pipeline and runs here -
      https://ci.blueocean.io/blue/organizations/jenkins/scratch%2Fbwalding%2F3-minute-avocados/detail/3-minute-avocados/12/tests

      Reproduction

      Consider a job with 2 builds

      Build 1, test A fails
      Build 2, test A / B fails

      Expected

      When displaying Build 2 test results, BlueOcean should report the following

      # New Failures - 1
      Test B
      # Existing Failures - 1
      Test A
      

      Actual

      However BlueOcean actually displays

      # New Failures - 1
      Test B
      # Existing Failures - 1
      Test B
      Test A
      

      This seems redundant (and probably wrong) to me. Also note - the count is correct, but the display is wrong.

        Attachments

          Activity

          bwalding Ben Walding created issue -
          bwalding Ben Walding made changes -
          Field Original Value New Value
          Description BlueOcean reports new test failures as both new test failures and existing test failures.

          h4. Example

          Test pipeline and runs here -
          https://ci.blueocean.io/blue/organizations/jenkins/scratch%2Fbwalding%2F3-minute-avocados/detail/3-minute-avocados/12/tests

           !image-2017-08-15-12-31-24-597.png|thumbnail!


          h4. Reproduction

          Consider a job with 2 builds

          Build 1, test A fails
          Build 2, test A / B fails

          h4. Expected
          When displaying Build 2 test results, BlueOcean should report the following

          {code}
          # New Failures
          Test B
          # Existing Failures
          Test A
          {code}

          h4. Actual

          However BlueOcean actually displays


          {code}
          # New Failures
          Test B
          # Existing Failures
          Test B
          Test A
          {code}

          This seems redundant (and probably wrong) to me.
          BlueOcean reports new test failures as both new test failures and existing test failures.
          h4. Example

          Test pipeline and runs here -
           [https://ci.blueocean.io/blue/organizations/jenkins/scratch%2Fbwalding%2F3-minute-avocados/detail/3-minute-avocados/12/tests]

          !image-2017-08-15-12-31-24-597.png|thumbnail!
          h4. Reproduction

          Consider a job with 2 builds

          Build 1, test A fails
           Build 2, test A / B fails
          h4. Expected

          When displaying Build 2 test results, BlueOcean should report the following
          {code:java}
          # New Failures - 1
          Test B
          # Existing Failures - 1
          Test A
          {code}
          h4. Actual

          However BlueOcean actually displays
          {code:java}
          # New Failures - 1
          Test B
          # Existing Failures - 1
          Test B
          Test A
          {code}
          This seems redundant (and probably wrong) to me. Also note - the count is correct, but the display is wrong.
          michaelneale Michael Neale made changes -
          Epic Link JENKINS-35761 [ 171656 ]
          jamesdumay James Dumay made changes -
          Sprint Blue Ocean 1.4 - beta 2 [ 416 ]
          jamesdumay James Dumay made changes -
          Rank Ranked higher
          jamesdumay James Dumay made changes -
          Rank Ranked lower
          jamesdumay James Dumay made changes -
          Sprint Blue Ocean 1.4 - beta 3 [ 416 ] Blue Ocean 1.4 - beta 4 [ 441 ]
          Hide
          seanwhyte Sean Whyte added a comment - - edited

          This is happening to me team too. 

          In addition to this, we sometimes get a passed run followed by the new failing tests showing up in "Existing failures".

          Let me know if that seems related or should be a new separate defect.

          I am using Jenkins 2.84 with Blue Ocean 1.3.3 (also occurred in 1.3.0)

          Show
          seanwhyte Sean Whyte added a comment - - edited This is happening to me team too.  In addition to this, we sometimes get a passed run followed by the new failing tests showing up in "Existing failures". Let me know if that seems related or should be a new separate defect. I am using Jenkins 2.84 with Blue Ocean 1.3.3 (also occurred in 1.3.0)
          Hide
          doispi v o added a comment -

          Our team is also experiencing this problem in some of our pipelines.  Both "New failing" and "Fixed" lists are wrong. Some tests show up in these lists even if we simply re-run a build.   

          Hopefully this will be fixed soon because this feature is very useful (one of the main reasons our team is using Blue Ocean).  I am using Blue Ocean 1.4.0.

           

           

           

          Show
          doispi v o added a comment - Our team is also experiencing this problem in some of our pipelines.  Both "New failing" and "Fixed" lists are wrong. Some tests show up in these lists even if we simply re-run a build.    Hopefully this will be fixed soon because this feature is very useful (one of the main reasons our team is using Blue Ocean).  I am using Blue Ocean 1.4.0.      
          Hide
          borisivan boris ivan added a comment -

          Same here. I have so many people ask me about this when I send them links to that test page. For this to be the case since 2017 seems avoidable.

          Show
          borisivan boris ivan added a comment - Same here. I have so many people ask me about this when I send them links to that test page. For this to be the case since 2017 seems avoidable.
          halkeye Gavin Mogan made changes -
          Assignee Gavin Mogan [ halkeye ]
          Show
          halkeye Gavin Mogan added a comment - https://github.com/jenkinsci/blueocean-plugin/pull/1938
          halkeye Gavin Mogan made changes -
          Status Open [ 1 ] Fixed but Unreleased [ 10203 ]
          Resolution Fixed [ 1 ]
          halkeye Gavin Mogan made changes -
          Status Fixed but Unreleased [ 10203 ] Resolved [ 5 ]
          Released As 1.15.0
          Hide
          doispi v o added a comment -

          Thanks for the update Gavin Mogan.  The issue regarding the reporting of new test failures seems to be fixed on our side. That's great.  However,  We still see problems with the list of 'Fixed' tests.   Sometimes it will list tests that were not even failing before as 'Fixed', which is confusing. 

          Show
          doispi v o added a comment - Thanks for the update Gavin Mogan .  The issue regarding the reporting of new test failures seems to be fixed on our side. That's great.  However,  We still see problems with the list of 'Fixed' tests.   Sometimes it will list tests that were not even failing before as 'Fixed', which is confusing. 

            People

            • Assignee:
              halkeye Gavin Mogan
              Reporter:
              bwalding Ben Walding
            • Votes:
              7 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: