Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-25214

New error cancelled by another resolved error in the same build

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: cppcheck-plugin
    • Labels:
      None
    • Environment:
      Jenkins 1.86
      CppCheck 1.20
    • Similar Issues:

      Description

      If the number of new errors is 1 but the number of resolved errors is 1 or more in the same build, then the new error is ignored when evaluating the severity levels. You can reproduce this by setting up a simple build that generates a dummy cppcheck.xml containing 1 error that is unique for every build, which generates +1 and -1 errors for each build. For the first / initial build, leave the new failure threshold unset to generate a stable build from which you can base the behaviour on. After this initial build has been created, set the new failure threshold level to 0 (any more than zero errors is a failure).

      Add a shell command as a build step and run the following:

      ------------------

      dt=$(date)
      msg="An error with a date of $dt to make it unique"
      errno="$BUILD_NUMBER"
      file="/usr/bin/splain"
      lineno=$(( $RANDOM % $(wc -l <$file) + 1 ))

      cat >cppcheck.xml <<HERE
      <?xml version="1.0" encoding="UTF-8"?>
      <results version="2">
      <cppcheck version="1.53"/>
      <errors>
      <error id="PerlErrors" severity="warning" msg="$msg" verbose="WARNING $errno:$msg">
      <location file="$file" line="$lineno"/>
      </error>
      </errors>
      </results>
      HERE

      ---------------------

      This will create a new cppcheck.xml file for each build, that will be shown as one error resolved and one new error. Observe that the build is still stable and is never marked as unstable. Also observe the message in the log:

      [Cppcheck] Not changing build status, since no threshold has been exceeded.

      If you inspect the report, you can observe the delta:

      new /usr/bin/splain 583 warning PerlErrors false An error with a date of Mon Oct 20 11:30:03 BST 2014 to make it unique
      solved /usr/bin/splain 498 warning PerlErrors false An error with a date of Mon Oct 20 11:29:23 BST 2014 to make it unique

      This is obviously a major issue, because it is allowing new errors to go unnoticed in CI builds.

        Attachments

          Activity

          Hide
          iwonbigbro Craig Phillips added a comment -

          If you haven't the time to look at this, please assign back to me and I will hopefully create a pull request on Thursday.

          Show
          iwonbigbro Craig Phillips added a comment - If you haven't the time to look at this, please assign back to me and I will hopefully create a pull request on Thursday.
          Hide
          mixalturek Michal Turek added a comment -

          Craig, you are right, the current implementation only compares number of issues. It seems you are quite interested in this plugin, what about to become a maintainer with commit and release rights? Feel free to ask in the developer email conference. I can help you with anything if it will be needed.

          Show
          mixalturek Michal Turek added a comment - Craig, you are right, the current implementation only compares number of issues. It seems you are quite interested in this plugin, what about to become a maintainer with commit and release rights? Feel free to ask in the developer email conference. I can help you with anything if it will be needed.
          Hide
          iwonbigbro Craig Phillips added a comment -

          I have nothing against being a maintainer, it would be a privilege. Thanks. I'll put forward a request.

          Show
          iwonbigbro Craig Phillips added a comment - I have nothing against being a maintainer, it would be a privilege. Thanks. I'll put forward a request.

            People

            • Assignee:
              iwonbigbro Craig Phillips
              Reporter:
              iwonbigbro Craig Phillips
            • Votes:
              3 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: