Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57085

Stack Traces with complex file names cannot be deserialized by XStream

    XMLWordPrintable

    Details

    • Similar Issues:
    • Released As:
      workflow-cps 2.68, groovy-cps 1.27

      Description

      Not sure about component, might be groovy-cps, might be upstream Groovy, might be upstream XStream, but nice to have tracking ticket here anyway. Marked as core for now assuming we'd need to backport a fix from some upstream library.

      Problem:

      If XStream is used to deserialize a StackTraceElement containing a complex file name, such as the following one seen in a stack trace in a Declarative Pipeline, the exception will be unable to be deserialized:

      org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)
      

      The file name for this example is:

      jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy
      

      Example exception:

      com.thoughtworks.xstream.converters.ConversionException: Could not parse StackTraceElement : org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)
      ---- Debugging information ----
      class               : java.lang.StackTraceElement
      required-type       : java.lang.StackTraceElement
      converter-type      : com.thoughtworks.xstream.converters.SingleValueConverterWrapper
      wrapped-converter   : com.thoughtworks.xstream.converters.extended.StackTraceElementConverter
      path                : /Tag/actions/wf.a.ErrorAction/error/stackTrace/trace[26]
      line number         : 46
      class[1]            : [Ljava.lang.StackTraceElement;
      converter-type[1]   : com.thoughtworks.xstream.converters.collections.ArrayConverter
      class[2]            : hudson.AbortException
      converter-type[2]   : com.thoughtworks.xstream.converters.extended.ThrowableConverter
      class[3]            : org.jenkinsci.plugins.workflow.actions.ErrorAction
      converter-type[3]   : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
      class[4]            : [Lhudson.model.Action;
      class[5]            : org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage$Tag
      version             : 1.4.7-jenkins-1
      -------------------------------
      	at com.thoughtworks.xstream.converters.extended.StackTraceElementConverter.fromString(StackTraceElementConverter.java:93)
      	at com.thoughtworks.xstream.converters.SingleValueConverterWrapper.fromString(SingleValueConverterWrapper.java:41)
      	at com.thoughtworks.xstream.converters.SingleValueConverterWrapper.unmarshal(SingleValueConverterWrapper.java:49)
      

      The reason this happens is that XStream uses a regex to extract the file name from the serialized format, and that regex assumes the file name does not contain a colon. However, in some cases stack traces for Groovy code contain the full path to the jar. I am not yet sure exactly why this is. CompilationUnit.java appears to have fixed this exact issue 14 years ago, but perhaps this is specific to MethodLocation in groovy-cps. Here is an excerpt of the serialized stack trace in case it helps identify the source of the file name:

      ...
      <trace>WorkflowScript.run(WorkflowScript:14)</trace>
      <trace>org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/Projects/pipeline-graph-analysis-plugin/work/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)</trace>
      ...
      <trace>org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(jar:file:/Users/dnusbaum/Projects/pipeline-graph-analysis-plugin/work/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:248)</trace>
      <trace>___cps.transform___(Native Method)</trace>
      <trace>com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)</trace>
      ...
      <trace>org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)</trace>
      

      Reproduction Steps:

      1. Start an instance of Jenkins with Blue Ocean and Declarative Pipeline installed.
      2. Create a new Pipeline with the following script:
        pipeline {
          agent none
          stages {
            stage('Fails') {
              steps {
                error 'oops'
              }
            }
          }
        }
        
      3. Run the build and view the visualization in Blue Ocean.
      4. Observe that the stage Fails is shown as failing.
      5. Restart Jenkins
      6. Open the visualization for the previous build.
      7. Observe that the stage Fails is shown as succeeding.

      Impact:

      I first noticed this for a Declarative Pipeline, and the effect was that actions for FlowNodes failed to be deserialized after restarting Jenkins, so RobustReflectionConverter just nulled out the actions, which changed the visualization of the Pipeline in Blue Ocean because ErrorActions were no longer present on the nodes, so stages that actually failed were displayed as successful.

      I imagine there could be other similar issues for restarted Pipelines. I have only encountered stack traces with these kinds of file names in Declarative Pipelines, but perhaps non-Pipeline code could be affected by the same issue.

      Solution

      This could be fixed with a patch to XStream like this, but I think it would be better to figure out what code is emitting these stack traces in the first place and modify that code in a similar way to https://github.com/apache/groovy/commit/4c42d503f6592c6ecc511c30bb49599316a937ff, unless displaying these kinds of file names in stack traces is normal in other contexts as well.

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            Offhand it sounds like groovy-cps is creating poor bytecode, and XStream is also behaving poorly. Ideally both would be fixed. Probably best to focus on groovy-cps to start with, since getting fixes into upstream XStream could take a while. (We have a fork anyway, but I would rather not add to the diff.)

            Show
            jglick Jesse Glick added a comment - Offhand it sounds like groovy-cps is creating poor bytecode, and XStream is also behaving poorly. Ideally both would be fixed. Probably best to focus on groovy-cps to start with, since getting fixes into upstream XStream could take a while. (We have a fork anyway, but I would rather not add to the diff.)
            Hide
            jglick Jesse Glick added a comment -

            Filed the XStream patch to start with, since it was easy enough.

            Show
            jglick Jesse Glick added a comment - Filed the XStream patch to start with, since it was easy enough.
            Hide
            jglick Jesse Glick added a comment -

            It is most easily visible with Declarative but not limited to that. Using

            @Library('github.com/jglick/sample-pipeline-library@JENKINS-57085-diag') _
            com.github.jglick.sample.Err.fail(this)
            

            which calls

            package com.github.jglick.sample
            class Err {
                static void fail(script) {
                    script.error 'oops'
                }
            }
            

            you can see

            <wf.a.ErrorAction plugin="workflow-api@2.33">
              <error class="hudson.AbortException">
                <detailMessage>oops</detailMessage>
                <stackTrace>
                  <trace>org.jenkinsci.plugins.workflow.steps.ErrorStep$Execution.run(ErrorStep.java:63)</trace>
                  <!-- … -->
                  <trace>org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)</trace>
                  <!-- … -->
                  <trace>com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)</trace>
                  <trace>com.github.jglick.sample.Err.fail(file:/…/jobs/p/builds/…/libs/github.com/jglick/sample-pipeline-library/src/com/github/jglick/sample/Err.groovy:4)</trace>
                  <trace>WorkflowScript.run(WorkflowScript:2)</trace>
                  <trace>___cps.transform___(Native Method)</trace>
                  <trace>com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)</trace>
                  <!-- … -->
            
            Show
            jglick Jesse Glick added a comment - It is most easily visible with Declarative but not limited to that. Using @Library( 'github.com/jglick/sample-pipeline-library@JENKINS-57085-diag' ) _ com.github.jglick.sample.Err.fail( this ) which calls package com.github.jglick.sample class Err { static void fail(script) { script.error 'oops' } } you can see <wf.a.ErrorAction plugin= "workflow-api@2.33" > <error class= "hudson.AbortException" > <detailMessage> oops </detailMessage> <stackTrace> <trace> org.jenkinsci.plugins.workflow.steps.ErrorStep$Execution.run(ErrorStep.java:63) </trace> <!-- … --> <trace> org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) </trace> <!-- … --> <trace> com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) </trace> <trace> com.github.jglick.sample.Err.fail(file:/…/jobs/p/builds/…/libs/github.com/jglick/sample-pipeline-library/src/com/github/jglick/sample/Err.groovy:4) </trace> <trace> WorkflowScript.run(WorkflowScript:2) </trace> <trace> ___cps.transform___(Native Method) </trace> <trace> com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) </trace> <!-- … -->
            Hide
            jglick Jesse Glick added a comment -

            For SourceUnit.name to have this form seems normal:

            /**
             * Initializes the SourceUnit from the specified URL.
             */
            public SourceUnit(URL source, CompilerConfiguration configuration, GroovyClassLoader loader, ErrorCollector er) {
                this(source.toExternalForm(), new URLReaderSource(source, configuration), configuration, loader, er);
            }
            

            CpsTransformer.makeBuilder then compiles it into a runtime construction of a MethodLocation with a URL as its fileName, from which later a StackTraceElement would be constructed.

            Show
            jglick Jesse Glick added a comment - For SourceUnit.name to have this form seems normal: /** * Initializes the SourceUnit from the specified URL. */ public SourceUnit(URL source, CompilerConfiguration configuration, GroovyClassLoader loader, ErrorCollector er) { this (source.toExternalForm(), new URLReaderSource(source, configuration), configuration, loader, er); } CpsTransformer.makeBuilder then compiles it into a runtime construction of a MethodLocation with a URL as its fileName , from which later a StackTraceElement would be constructed.
            Hide
            jglick Jesse Glick added a comment -

            Solved with test in workflow-cps-global-lib, but manually confirmed that the fix also corrects stack traces for the original Declarative example. (I did not bother to go through the full reproduction case involving Blue Ocean.)

            Show
            jglick Jesse Glick added a comment - Solved with test in workflow-cps-global-lib , but manually confirmed that the fix also corrects stack traces for the original Declarative example. (I did not bother to go through the full reproduction case involving Blue Ocean.)
            Hide
            dnusbaum Devin Nusbaum added a comment -

            A fix for this issue was just released in Pipeline: Groovy 2.68.

            Show
            dnusbaum Devin Nusbaum added a comment - A fix for this issue was just released in Pipeline: Groovy 2.68.

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                dnusbaum Devin Nusbaum
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: