Currently failed builds are assigned the GitHub status of Failure.
The GitHub statuses seem to be modelled on the JUnit test result states.
In JUnit a failure corresponds to the assertions being incorrect, whereas an error is an unexpected outcome from the test.
If we upscale that to the unit of an entire build:
- The build completes with test failures (i.e. Jenkins UNSTABLE) is analogous to an individual tests assertions being incorrect thus this should map to GitHub's FAILURE status
- The build does not complete (i.e. Jenkins FAILURE) is analogous to an unexpected outcome thus this should map to GitHub's ERROR status
Thus the change is to make a Jenkins build FAILURE report as a GitHub ERROR
Here is the current (and proposed) github status for a build that has test failures (i.e. Jenkins UNSTABLE)
Here is the current github status for a build that cannot be built (e.g. a compiler error or such)
Here is the proposed github status for a build that cannot be built:
And on the overall PRs tab you will see tooltips like: