Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6344

fail build if any empty files exist, or provide option to do it

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Component/s: xunit-plugin
    • Labels:
      None

      Description

      Currently empty test results are just a warning, but it would be great if they could be configured as failures. In the case of phpunit, when it starts running, it creates an empty xml file. So if there is a fatal php error, it will just result in an empty, but new, file, and xUnit will happily warn but ignore this problem. For us it makes sense to always consider these errors, but maybe there should be a checkbox for "Fail the build if there are empty test files" or something.

      Attached is the patch we are using locally to accomplish this. Along with the previous patch, it makes our xUnit runs really robust and is the only way to ensure nothing is going wrong with the builds.

      Thanks for any comments!

        Activity

        Hide
        gbois Gregory Boissinot added a comment -

        In my opinion, we can't set a fail status for one empty test result file.
        Nevertheless, if there are only empty test result files, set the result status to failed would be great.

        Regarding adding a checkbox, I think that's would be good idea but there already lots of checkboxes. Adding new checkboxes could make the plugin too complex.

        Show
        gbois Gregory Boissinot added a comment - In my opinion, we can't set a fail status for one empty test result file. Nevertheless, if there are only empty test result files, set the result status to failed would be great. Regarding adding a checkbox, I think that's would be good idea but there already lots of checkboxes. Adding new checkboxes could make the plugin too complex.
        Hide
        mcrooney mcrooney added a comment -

        Hm yeah, I can see your point. In our case though with phpunit fatal errors it is absolutely a failure and we want to consider it as such, and I can see where others who try to keep a clean/robust build would consider that a failure as well.

        Can you think of a way to have it such that we don't have to apply a patch on each release, while keeping the plugin to your liking? You don't find it worthy of a configuration option?

        Show
        mcrooney mcrooney added a comment - Hm yeah, I can see your point. In our case though with phpunit fatal errors it is absolutely a failure and we want to consider it as such, and I can see where others who try to keep a clean/robust build would consider that a failure as well. Can you think of a way to have it such that we don't have to apply a patch on each release, while keeping the plugin to your liking? You don't find it worthy of a configuration option?
        Hide
        gbois Gregory Boissinot added a comment -

        Ok for the option.
        I need to include it in the new xUnit architecture.

        Show
        gbois Gregory Boissinot added a comment - Ok for the option. I need to include it in the new xUnit architecture.
        Hide
        gbois Gregory Boissinot added a comment -

        Available in trunk?
        Are the possibility to test it?

        Show
        gbois Gregory Boissinot added a comment - Available in trunk? Are the possibility to test it?
        Hide
        gbois Gregory Boissinot added a comment -

        Available with xUnit 1.15

        Show
        gbois Gregory Boissinot added a comment - Available with xUnit 1.15

          People

          • Assignee:
            gbois Gregory Boissinot
            Reporter:
            mcrooney mcrooney
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: