Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62198

Warnings Next Generation: Links not created for SpotBugs

    Details

    • Similar Issues:

      Description

      I have two Jobs using Warnings Next Generation Plugin for code analysis purposes. Both use the following analysis tools:

      • Checkstyle
      • CPD
      • PMD
      • SpotBugs

      The project is built with maven using the corresponding plugins for the tools.

      One of the Jobs is analyzing the master branch of a Git repository, the other one analyzes ticket branches and compares the result against the first job using a quality gate. As I need to send a mail to the author of the last git commit of the ticket branch, I had to switch to a pipeline job for the second job, while the first is still a classic jenkins job configured through the ui.

      Now I have noticed that for SpotBugs the links to the issues in the code is not set, while it is working for the first job for creating the baseline. I made second version for the first job using a pipeline job and can reproduce the same problem, so it seems that:

      • The problem is not related with the comparism between the two jobs.
      • The problem occurs only with pipeline jobs.

      There is nothing special with the recordIssues-statement IMO:

      recordIssues(
                  aggregatingResults: true, enabledForFailure: true, ignoreQualityGate: true, sourceCodeEncoding: 'UTF-8', sourceDirectory: 'src/main/java',
                  tools: [
                      checkStyle(pattern: 'target/checkstyle-result.xml', reportEncoding: 'UTF-8'),
                      mavenConsole(),
                      java(),
                      cpd(pattern: 'target/cpd.xml', reportEncoding: 'UTF-8'),
                      pmdParser(pattern: 'target/pmd.xml', reportEncoding: 'UTF-8'),
                      spotBugs(pattern: 'target/spotbugsXml.xml', reportEncoding: 'UTF-8', useRankAsPriority: true)
                  ]
              )
      

      In the logs of the alternative version of job one, I can see that the files are found to be not in the workspace anymore, yet they are there and there is nothing special about them.

      As of the other tools, their links all work correctly in both jobs. That is, if I use sed commands to relativize the paths to the issues in their resulting files as otherwise they are not recognized as being in the workspace. However, there is no such command for SpotBugs as the links seemed to work at first sight.

      BTW: the plugin should provide a way to relativize the paths for the other tools in future. The tools use absolute paths, there's nothing the users could do about it. Having to trick with sed commands feels nasty, the plugin should be able to deal with that.

        Attachments

          Activity

          Hide
          drulli Ulli Hafner added a comment -

          The plugin is reporting errors in the console log (or in the view http://localhost:8080/job/[job-name]/[build-number]/spotbugs/info/). What is shown there?

          Show
          drulli Ulli Hafner added a comment - The plugin is reporting errors in the console log (or in the view http://localhost:8080/job/[job-name]/[build-number]/spotbugs/info/ ). What is shown there?
          Hide
          drulli Ulli Hafner added a comment -

          BTW: the plugin should provide a way to relativize the paths for the other tools in future. The tools use absolute paths, there's nothing the users could do about it. Having to trick with sed commands feels nasty, the plugin should be able to deal with that.

          The plugin automatically creates relative paths, there is nothing you should care about. What exactly is not working with the other tools?

          Show
          drulli Ulli Hafner added a comment - BTW: the plugin should provide a way to relativize the paths for the other tools in future. The tools use absolute paths, there's nothing the users could do about it. Having to trick with sed commands feels nasty, the plugin should be able to deal with that. The plugin automatically creates relative paths, there is nothing you should care about. What exactly is not working with the other tools?
          Hide
          thedentist Daniel Kepes added a comment -

          As of SpotBugs, the log gives the following information:

          [SpotBugs] Searching for all files in '[PATH_TO_JENKINS]/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace' that match the pattern 'target/spotbugsXml.xml'
          [SpotBugs] -> found 1 file
          [SpotBugs] Successfully parsed file [PATH_TO_JENKINS]/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace/target/spotbugsXml.xml
          [SpotBugs] -> found 21 issues (skipped 0 duplicates)
          [SpotBugs] Post processing issues on 'Master' with source code encoding 'UTF-8'
          [SpotBugs] Creating SCM blamer to obtain author and commit information for affected files
          [SpotBugs] Creating SCM miner to obtain statistics for affected repository files
          [SpotBugs] Resolving file names for all issues in source directory '[PATH_TO_JENKINS]/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace/src/main/java'
          [SpotBugs] -> resolved paths in source directory (6 found, 0 not found)
          [SpotBugs] Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files)
          [SpotBugs] -> all issues already have a valid module name
          [SpotBugs] Resolving package names (or namespaces) by parsing the affected files
          [SpotBugs] -> all affected files already have a valid package name
          [SpotBugs] No filter has been set, publishing all 21 issues
          [SpotBugs] Creating fingerprints for all affected code blocks to track issues over different builds
          [SpotBugs] -> created fingerprints for 0 issues (skipped 21 issues)
          [SpotBugs] Copying affected files to Jenkins' build folder '[PATH_TO_JENKINS]/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/builds/5/files-with-issues'
          [SpotBugs] -> 0 copied, 21 not in workspace, 0 not-found, 0 with I/O error
          

          I have censored the path to the jenkins home, there doesn't seem to be anything wrong with it anyways. So, there doesn't seem to be anything wrong to me until the last line where it states that there were no files copied, but 21 were outside the workspace. But I can go through the workspace from jenkins and the files are there.

          Using your URL pattern, I get rerouted to the builds home. Nothing special there, just a summary about the git revision, the analysis tools used and a link to the warnings report.

          Regarding the other bugs, the problems were similar, the logs also stated that the files would not be in the workspace. But as I figured out that those tools use absolute paths in contrast to spotbugs (which gives only the packages and class names), I tested around with relative paths and then I got the links working.

          Maybe a short hint to avoid misunderstandings: I get the report about the warnings and all the warnings / issues are there but when I am on the lowest level and try to look at the code, there is only the name of the file and the linenumber of the issue, without a link I could follow.

          Show
          thedentist Daniel Kepes added a comment - As of SpotBugs, the log gives the following information: [SpotBugs] Searching for all files in '[PATH_TO_JENKINS]/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace' that match the pattern 'target/spotbugsXml.xml' [SpotBugs] -> found 1 file [SpotBugs] Successfully parsed file [PATH_TO_JENKINS]/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace/target/spotbugsXml.xml [SpotBugs] -> found 21 issues (skipped 0 duplicates) [SpotBugs] Post processing issues on 'Master' with source code encoding 'UTF-8' [SpotBugs] Creating SCM blamer to obtain author and commit information for affected files [SpotBugs] Creating SCM miner to obtain statistics for affected repository files [SpotBugs] Resolving file names for all issues in source directory '[PATH_TO_JENKINS]/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace/src/main/java' [SpotBugs] -> resolved paths in source directory (6 found, 0 not found) [SpotBugs] Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files) [SpotBugs] -> all issues already have a valid module name [SpotBugs] Resolving package names (or namespaces) by parsing the affected files [SpotBugs] -> all affected files already have a valid package name [SpotBugs] No filter has been set, publishing all 21 issues [SpotBugs] Creating fingerprints for all affected code blocks to track issues over different builds [SpotBugs] -> created fingerprints for 0 issues (skipped 21 issues) [SpotBugs] Copying affected files to Jenkins ' build folder ' [PATH_TO_JENKINS]/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/builds/5/files-with-issues' [SpotBugs] -> 0 copied, 21 not in workspace, 0 not-found, 0 with I/O error I have censored the path to the jenkins home, there doesn't seem to be anything wrong with it anyways. So, there doesn't seem to be anything wrong to me until the last line where it states that there were no files copied, but 21 were outside the workspace. But I can go through the workspace from jenkins and the files are there. Using your URL pattern, I get rerouted to the builds home. Nothing special there, just a summary about the git revision, the analysis tools used and a link to the warnings report. Regarding the other bugs, the problems were similar, the logs also stated that the files would not be in the workspace. But as I figured out that those tools use absolute paths in contrast to spotbugs (which gives only the packages and class names), I tested around with relative paths and then I got the links working. Maybe a short hint to avoid misunderstandings: I get the report about the warnings and all the warnings / issues are there but when I am on the lowest level and try to look at the code, there is only the name of the file and the linenumber of the issue, without a link I could follow.
          Hide
          thedentist Daniel Kepes added a comment -

          I took a closer look at this and found a few things out. First of all, I set up a fresh jenkins instance, created a dummy-project with a spotbugs issue and two jobs doing the same thing, but one being a pipeline job, the other not. Result: everything worked properly, no problems with linking, so I couldn't reproduce the issue on the test instance. Then I installed all the plugins we use on our real jenkins, no changes. Lastly, I noticed that the jobs directory of our real instance is only a symlink to another directory. Even more, the workspaces of the jobs are under the jobs directory, so are also affected by the symlink. The latter is not the case for my test instance, where I could not reproduce the problem even when using a symlink to the jobs directory. In any case, on my real instance the symlink leads to different paths in the output of the maven plugins from what Jenkins believes to be the workspace. Clearly a problem with my setup, nevertheless, my story doesn't end here.

          I tried to get maven use the right paths by changing the working directory before calling the goals, but no luck, still the wrong paths. Well, this is beyond the scope of the Warnings plugin of course. However, I then set an absolute path to the source directory, which I had to register globally as well. This did the trick for the pipeline job, but guess what? It stopped working for PMD in the jobs that are not set up the pipeline way (again, no links). At this point I returned to my approach using sed, which I could set up properly for SpotBugs as well.

          Although I found a workaround, I have a bad feeling about it expecting it to fail in future. All in all, the symlink is the trigger, which is clearly a problem on my side. Yet it doesn't explain why the analyzis tools work differently. Interestingly, the symlink does not cause any trouble for SpotBugs as long as I don't use pipeline scripts. If the pipelin script and the other tools would work the same way, everything would be fine for me.

          Show
          thedentist Daniel Kepes added a comment - I took a closer look at this and found a few things out. First of all, I set up a fresh jenkins instance, created a dummy-project with a spotbugs issue and two jobs doing the same thing, but one being a pipeline job, the other not. Result: everything worked properly, no problems with linking, so I couldn't reproduce the issue on the test instance. Then I installed all the plugins we use on our real jenkins, no changes. Lastly, I noticed that the jobs directory of our real instance is only a symlink to another directory. Even more, the workspaces of the jobs are under the jobs directory, so are also affected by the symlink. The latter is not the case for my test instance, where I could not reproduce the problem even when using a symlink to the jobs directory. In any case, on my real instance the symlink leads to different paths in the output of the maven plugins from what Jenkins believes to be the workspace. Clearly a problem with my setup, nevertheless, my story doesn't end here. I tried to get maven use the right paths by changing the working directory before calling the goals, but no luck, still the wrong paths. Well, this is beyond the scope of the Warnings plugin of course. However, I then set an absolute path to the source directory, which I had to register globally as well. This did the trick for the pipeline job, but guess what? It stopped working for PMD in the jobs that are not set up the pipeline way (again, no links). At this point I returned to my approach using sed, which I could set up properly for SpotBugs as well. Although I found a workaround, I have a bad feeling about it expecting it to fail in future. All in all, the symlink is the trigger, which is clearly a problem on my side. Yet it doesn't explain why the analyzis tools work differently. Interestingly, the symlink does not cause any trouble for SpotBugs as long as I don't use pipeline scripts. If the pipelin script and the other tools would work the same way, everything would be fine for me.
          Hide
          drulli Ulli Hafner added a comment -

          I see. I'm using several low level API methods to resolve the real file name of the sources. I think in most of those places I set the option to resolve symbolic links to false. Maybe this can be changed without breaking something else.

          What still is not clear for me: why is the fingerprinting not working;

          [SpotBugs] Creating fingerprints for all affected code blocks to track issues over different builds
          [SpotBugs] -> created fingerprints for 0 issues (skipped 21 issues)

          There should be some errors reported here as well. What errors are shown in the screen http://localhost:8080/job/your-job/your-build/spotbugs/info/

          Show
          drulli Ulli Hafner added a comment - I see. I'm using several low level API methods to resolve the real file name of the sources. I think in most of those places I set the option to resolve symbolic links to false . Maybe this can be changed without breaking something else. What still is not clear for me: why is the fingerprinting not working; [SpotBugs] Creating fingerprints for all affected code blocks to track issues over different builds [SpotBugs] -> created fingerprints for 0 issues (skipped 21 issues) There should be some errors reported here as well. What errors are shown in the screen http://localhost:8080/job/your-job/your-build/spotbugs/info/
          Hide
          thedentist Daniel Kepes added a comment - - edited

          For a "broken" build, the whole block given under the url you stated is as follows:

           
          Searching for all files in '/proj/kr1/tools/jenkins/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace' that match the pattern 'target/spotbugsXml.xml'
          -> found 1 file
          Successfully parsed file /proj/kr1/tools/jenkins/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace/target/spotbugsXml.xml
          -> found 21 issues (skipped 0 duplicates)
          Post processing issues on 'Master' with source code encoding 'UTF-8'
          Creating SCM blamer to obtain author and commit information for affected files
          Creating SCM miner to obtain statistics for affected repository files
          Resolving file names for all issues in source directory '/proj/kr1/tools/jenkins/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace/src/main/java'
          -> resolved paths in source directory (6 found, 0 not found)
          Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files)
          -> all issues already have a valid module name
          Resolving package names (or namespaces) by parsing the affected files
          -> all affected files already have a valid package name
          No filter has been set, publishing all 21 issues
          Creating fingerprints for all affected code blocks to track issues over different builds
          -> created fingerprints for 0 issues (skipped 21 issues)
          Copying affected files to Jenkins' build folder '/proj/kr1/tools/jenkins/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/builds/25/files-with-issues'
          -> 0 copied, 21 not in workspace, 0 not-found, 0 with I/O error
          No valid reference build found that meets the criteria (NO_JOB_FAILURE - IGNORE_QUALITY_GATE)
          All reported issues will be considered outstanding
          No quality gates have been set - skipping
          Health report is disabled - skipping
          Created analysis result for 21 issues (found 0 new issues, fixed 0 issues)
          Attaching ResultAction with ID 'spotbugs' to run 'KR1-ScnDrv-NG-Codeanalyse_Tmp #25'.
           
          Couldn't say I could see an error there.
           
           

          Show
          thedentist Daniel Kepes added a comment - - edited For a "broken" build, the whole block given under the url you stated is as follows:   Searching for all files in '/proj/kr1/tools/jenkins/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace' that match the pattern 'target/spotbugsXml.xml' -> found 1 file Successfully parsed file /proj/kr1/tools/jenkins/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace/target/spotbugsXml.xml -> found 21 issues (skipped 0 duplicates) Post processing issues on 'Master' with source code encoding 'UTF-8' Creating SCM blamer to obtain author and commit information for affected files Creating SCM miner to obtain statistics for affected repository files Resolving file names for all issues in source directory '/proj/kr1/tools/jenkins/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/workspace/src/main/java' -> resolved paths in source directory (6 found, 0 not found) Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files) -> all issues already have a valid module name Resolving package names (or namespaces) by parsing the affected files -> all affected files already have a valid package name No filter has been set, publishing all 21 issues Creating fingerprints for all affected code blocks to track issues over different builds -> created fingerprints for 0 issues (skipped 21 issues) Copying affected files to Jenkins' build folder '/proj/kr1/tools/jenkins/jobs/KR1-ScnDrv-NG-Codeanalyse_Tmp/builds/25/files-with-issues' -> 0 copied, 21 not in workspace, 0 not-found, 0 with I/O error No valid reference build found that meets the criteria (NO_JOB_FAILURE - IGNORE_QUALITY_GATE) All reported issues will be considered outstanding No quality gates have been set - skipping Health report is disabled - skipping Created analysis result for 21 issues (found 0 new issues, fixed 0 issues) Attaching ResultAction with ID 'spotbugs' to run 'KR1-ScnDrv-NG-Codeanalyse_Tmp #25'.   Couldn't say I could see an error there.    
          Hide
          drulli Ulli Hafner added a comment -

          Ah, ok. Normally there should be some errors as in https://ci.jenkins.io/job/Plugins/job/warnings-ng-plugin/job/master/860/checkstyle/info/. Seems that this part is missing for your job

          Show
          drulli Ulli Hafner added a comment - Ah, ok. Normally there should be some errors as in https://ci.jenkins.io/job/Plugins/job/warnings-ng-plugin/job/master/860/checkstyle/info/ . Seems that this part is missing for your job
          Hide
          drulli Ulli Hafner added a comment -

          Sorry, forget about the fingerprinting problem: SpotBugs creates Fingerprints on its own so I'm simply skipping that part.

          So we actually have only the problem about the wrong detection of the path due to links.

          Show
          drulli Ulli Hafner added a comment - Sorry, forget about the fingerprinting problem: SpotBugs creates Fingerprints on its own so I'm simply skipping that part. So we actually have only the problem about the wrong detection of the path due to links.

            People

            • Assignee:
              drulli Ulli Hafner
              Reporter:
              thedentist Daniel Kepes
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: