Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26523

Allow yellow "unstable" state from a step

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Won't Fix
    • Component/s: pipeline
    • Labels:
      None
    • Similar Issues:

      Description

      The xUnit plugin allows for a yellow "unstable" state, between red "FAIL" and blue "OK". This is useful for test-suite steps, to help developers immediately distinguish between "I broke one of 40,000 unit tests" and "I caused compilation to fail entirely".

      It would help if there were an easy way to assign an unstable state as the result of a step, especially a shell step (e.g. if we could check for a magic exit code to mark a step as unstable rather than failed)

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            The more commonly used JUnit plugin also has an UNSTABLE state—for the Run as a whole, which Workflow supports if the step wishes to set it. Jenkins generally just has a notion of a whole build being stable, unstable, or failed.

            Would rather leave this to JENKINS-26522 for marking up detailed information, and the existing option for a step to set a Run.result.

            Show
            jglick Jesse Glick added a comment - The more commonly used JUnit plugin also has an UNSTABLE state—for the Run as a whole, which Workflow supports if the step wishes to set it. Jenkins generally just has a notion of a whole build being stable, unstable, or failed. Would rather leave this to JENKINS-26522 for marking up detailed information, and the existing option for a step to set a Run.result .
            Hide
            jglick Jesse Glick added a comment -

            I should add that FlowNode.getIconColor does exist, which renders the red or blue ball in the Running Steps view for example, but this is not based on a stored Result; it is red if there is an ErrorAction (exception of some kind), blue otherwise. So even allowing a step to set a custom result on its node would require at least a modest API change, such as a new ResultAction encoding a Result that getIconColor would use as an override, or a similar added option to ErrorAction.

            And to be useful, any new per-step status would need to be shown prominently. Running Steps would show it, but this is intended more as a detailed diagnostic view than something a user would normally be browsing. Visualizations like the stage view in Jenkins Enterprise do not show individual steps for the most part, so status would need to be set per stage to be useful.

            For the originally described use case, no new feature is necessary, since any integration of the xUnit plugin (presumably via SimpleBuildStep, like the JUnit plugin uses) would set the Run.result to UNSTABLE if there are test failures, which already captures the distinction between a compilation error halting the build early and one or more test failures recorded near the end. A per-FlowNode status would only be useful if you wished to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and if the console log were inadequate to note this (even with specialized markup such as colors).

            Show
            jglick Jesse Glick added a comment - I should add that FlowNode.getIconColor does exist, which renders the red or blue ball in the Running Steps view for example, but this is not based on a stored Result ; it is red if there is an ErrorAction (exception of some kind), blue otherwise. So even allowing a step to set a custom result on its node would require at least a modest API change, such as a new ResultAction encoding a Result that getIconColor would use as an override, or a similar added option to ErrorAction . And to be useful, any new per-step status would need to be shown prominently. Running Steps would show it, but this is intended more as a detailed diagnostic view than something a user would normally be browsing. Visualizations like the stage view in Jenkins Enterprise do not show individual steps for the most part, so status would need to be set per stage to be useful. For the originally described use case, no new feature is necessary, since any integration of the xUnit plugin (presumably via SimpleBuildStep , like the JUnit plugin uses) would set the Run.result to UNSTABLE if there are test failures, which already captures the distinction between a compilation error halting the build early and one or more test failures recorded near the end. A per- FlowNode status would only be useful if you wished to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and if the console log were inadequate to note this (even with specialized markup such as colors).
            Hide
            directhex Jo Shields added a comment - - edited

            "A per-FlowNode status would only be useful if you wished to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and if the console log were inadequate to note this (even with specialized markup such as colors)."

            I wish to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and the console log is inadequate to note this (even with specialized markup such as colors).

            Show
            directhex Jo Shields added a comment - - edited "A per-FlowNode status would only be useful if you wished to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and if the console log were inadequate to note this (even with specialized markup such as colors)." I wish to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and the console log is inadequate to note this (even with specialized markup such as colors).

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                directhex Jo Shields
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: