Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57085

Stack Traces with complex file names cannot be deserialized by XStream

    Details

    • Similar Issues:
    • Released As:
      workflow-cps 2.68, groovy-cps 1.27

      Description

      Not sure about component, might be groovy-cps, might be upstream Groovy, might be upstream XStream, but nice to have tracking ticket here anyway. Marked as core for now assuming we'd need to backport a fix from some upstream library.

      Problem:

      If XStream is used to deserialize a StackTraceElement containing a complex file name, such as the following one seen in a stack trace in a Declarative Pipeline, the exception will be unable to be deserialized:

      org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)
      

      The file name for this example is:

      jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy
      

      Example exception:

      com.thoughtworks.xstream.converters.ConversionException: Could not parse StackTraceElement : org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)
      ---- Debugging information ----
      class               : java.lang.StackTraceElement
      required-type       : java.lang.StackTraceElement
      converter-type      : com.thoughtworks.xstream.converters.SingleValueConverterWrapper
      wrapped-converter   : com.thoughtworks.xstream.converters.extended.StackTraceElementConverter
      path                : /Tag/actions/wf.a.ErrorAction/error/stackTrace/trace[26]
      line number         : 46
      class[1]            : [Ljava.lang.StackTraceElement;
      converter-type[1]   : com.thoughtworks.xstream.converters.collections.ArrayConverter
      class[2]            : hudson.AbortException
      converter-type[2]   : com.thoughtworks.xstream.converters.extended.ThrowableConverter
      class[3]            : org.jenkinsci.plugins.workflow.actions.ErrorAction
      converter-type[3]   : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
      class[4]            : [Lhudson.model.Action;
      class[5]            : org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage$Tag
      version             : 1.4.7-jenkins-1
      -------------------------------
      	at com.thoughtworks.xstream.converters.extended.StackTraceElementConverter.fromString(StackTraceElementConverter.java:93)
      	at com.thoughtworks.xstream.converters.SingleValueConverterWrapper.fromString(SingleValueConverterWrapper.java:41)
      	at com.thoughtworks.xstream.converters.SingleValueConverterWrapper.unmarshal(SingleValueConverterWrapper.java:49)
      

      The reason this happens is that XStream uses a regex to extract the file name from the serialized format, and that regex assumes the file name does not contain a colon. However, in some cases stack traces for Groovy code contain the full path to the jar. I am not yet sure exactly why this is. CompilationUnit.java appears to have fixed this exact issue 14 years ago, but perhaps this is specific to MethodLocation in groovy-cps. Here is an excerpt of the serialized stack trace in case it helps identify the source of the file name:

      ...
      <trace>WorkflowScript.run(WorkflowScript:14)</trace>
      <trace>org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/Projects/pipeline-graph-analysis-plugin/work/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)</trace>
      ...
      <trace>org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(jar:file:/Users/dnusbaum/Projects/pipeline-graph-analysis-plugin/work/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:248)</trace>
      <trace>___cps.transform___(Native Method)</trace>
      <trace>com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)</trace>
      ...
      <trace>org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)</trace>
      

      Reproduction Steps:

      1. Start an instance of Jenkins with Blue Ocean and Declarative Pipeline installed.
      2. Create a new Pipeline with the following script:
        pipeline {
          agent none
          stages {
            stage('Fails') {
              steps {
                error 'oops'
              }
            }
          }
        }
        
      3. Run the build and view the visualization in Blue Ocean.
      4. Observe that the stage Fails is shown as failing.
      5. Restart Jenkins
      6. Open the visualization for the previous build.
      7. Observe that the stage Fails is shown as succeeding.

      Impact:

      I first noticed this for a Declarative Pipeline, and the effect was that actions for FlowNodes failed to be deserialized after restarting Jenkins, so RobustReflectionConverter just nulled out the actions, which changed the visualization of the Pipeline in Blue Ocean because ErrorActions were no longer present on the nodes, so stages that actually failed were displayed as successful.

      I imagine there could be other similar issues for restarted Pipelines. I have only encountered stack traces with these kinds of file names in Declarative Pipelines, but perhaps non-Pipeline code could be affected by the same issue.

      Solution: 

      This could be fixed with a patch to XStream like this, but I think it would be better to figure out what code is emitting these stack traces in the first place and modify that code in a similar way to https://github.com/apache/groovy/commit/4c42d503f6592c6ecc511c30bb49599316a937ff, unless displaying these kinds of file names in stack traces is normal in other contexts as well.

        Attachments

          Issue Links

            Activity

            dnusbaum Devin Nusbaum created issue -
            dnusbaum Devin Nusbaum made changes -
            Field Original Value New Value
            Description Not sure about component, might be groovy-cps, might be upstream Groovy, might be upstream XStream, but nice to have tracking ticket here anyway. Marked as core for now assuming we'd need to backport a fix from some upstream library.

            *Problem:*

            If XStream is used to deserialize a {{StackTraceElement}} containing a complex file name, such as the following one seen in a stack trace in a Declarative Pipeline, the exception will be unable to be deserialized:
            {noformat}
            org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)
            {noformat}
            The file name for this example is:
            {noformat}
            jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy
            {noformat}

            Example exception:

            {noformat}
            com.thoughtworks.xstream.converters.ConversionException: Could not parse StackTraceElement : org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)
            ---- Debugging information ----
            class : java.lang.StackTraceElement
            required-type : java.lang.StackTraceElement
            converter-type : com.thoughtworks.xstream.converters.SingleValueConverterWrapper
            wrapped-converter : com.thoughtworks.xstream.converters.extended.StackTraceElementConverter
            path : /Tag/actions/wf.a.ErrorAction/error/stackTrace/trace[26]
            line number : 46
            class[1] : [Ljava.lang.StackTraceElement;
            converter-type[1] : com.thoughtworks.xstream.converters.collections.ArrayConverter
            class[2] : hudson.AbortException
            converter-type[2] : com.thoughtworks.xstream.converters.extended.ThrowableConverter
            class[3] : org.jenkinsci.plugins.workflow.actions.ErrorAction
            converter-type[3] : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
            class[4] : [Lhudson.model.Action;
            class[5] : org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage$Tag
            version : 1.4.7-jenkins-1
            -------------------------------
            at com.thoughtworks.xstream.converters.extended.StackTraceElementConverter.fromString(StackTraceElementConverter.java:93)
            at com.thoughtworks.xstream.converters.SingleValueConverterWrapper.fromString(SingleValueConverterWrapper.java:41)
            at com.thoughtworks.xstream.converters.SingleValueConverterWrapper.unmarshal(SingleValueConverterWrapper.java:49)
            {noformat}

            The reason this happens is that XStream uses a regex to extract the file name from the serialized format, and that regex assumes the file name does not contain a colon. However, in some cases stack traces for Groovy code contain the full path to the jar. I am not yet sure exactly why this is. [CompilationUnit.java|https://github.com/apache/groovy/blame/master/src/main/java/org/codehaus/groovy/control/CompilationUnit.java#L851-L852] appears to have fixed this exact issue 14 years ago, but perhaps this is specific to [MethodLocation|https://github.com/cloudbees/groovy-cps/blob/846d8d8eed10c57063d50894d2949e9a8fd21ae0/lib/src/main/java/com/cloudbees/groovy/cps/MethodLocation.java] in groovy-cps. Here is an excerpt of the serialized stack trace in case it helps identify the source of the file name:

            {noformat}
            ...
            <trace>WorkflowScript.run(WorkflowScript:14)</trace>
            <trace>org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/Projects/pipeline-graph-analysis-plugin/work/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)</trace>
            ...
            <trace>org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(jar:file:/Users/dnusbaum/Projects/pipeline-graph-analysis-plugin/work/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:248)</trace>
            <trace>___cps.transform___(Native Method)</trace>
            <trace>com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)</trace>
            ...
            <trace>org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)</trace>
            {noformat}

            *Impact:*

            I first noticed this for a Declarative Pipeline, and the effect was that actions for {{FlowNodes}} failed to be deserialized after restarting Jenkins, so {{RobustReflectionConverter}} just nulled out the actions, which changed the visualization of the Pipeline in Blue Ocean because {{ErrorActions}} were no longer present on the nodes, so stages that actually failed were displayed as successful.

            I imagine there could be other similar issues for restarted Pipelines. I have only encountered stack traces with these kinds of file names in Declarative Pipelines, but perhaps non-Pipeline code could be affected by the same issue.

            *Solution*: 

            This could be fixed with a patch to XStream like [this|https://github.com/x-stream/xstream/compare/master...dwnusbaum:stack-traces], but I think it would be better to figure out what code is emitting these stack traces in the first place and modify that code in a similar way to https://github.com/apache/groovy/commit/4c42d503f6592c6ecc511c30bb49599316a937ff, unless displaying these kinds of file names in stack traces is normal in other contexts as well.
            Not sure about component, might be groovy-cps, might be upstream Groovy, might be upstream XStream, but nice to have tracking ticket here anyway. Marked as core for now assuming we'd need to backport a fix from some upstream library.

            *Problem:*

            If XStream is used to deserialize a {{StackTraceElement}} containing a complex file name, such as the following one seen in a stack trace in a Declarative Pipeline, the exception will be unable to be deserialized:
            {noformat}
            org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)
            {noformat}
            The file name for this example is:
            {noformat}
            jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy
            {noformat}

            Example exception:

            {noformat}
            com.thoughtworks.xstream.converters.ConversionException: Could not parse StackTraceElement : org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/.m2/repository/org/jenkinsci/plugins/pipeline-model-definition/1.3.7/pipeline-model-definition-1.3.7.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)
            ---- Debugging information ----
            class : java.lang.StackTraceElement
            required-type : java.lang.StackTraceElement
            converter-type : com.thoughtworks.xstream.converters.SingleValueConverterWrapper
            wrapped-converter : com.thoughtworks.xstream.converters.extended.StackTraceElementConverter
            path : /Tag/actions/wf.a.ErrorAction/error/stackTrace/trace[26]
            line number : 46
            class[1] : [Ljava.lang.StackTraceElement;
            converter-type[1] : com.thoughtworks.xstream.converters.collections.ArrayConverter
            class[2] : hudson.AbortException
            converter-type[2] : com.thoughtworks.xstream.converters.extended.ThrowableConverter
            class[3] : org.jenkinsci.plugins.workflow.actions.ErrorAction
            converter-type[3] : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
            class[4] : [Lhudson.model.Action;
            class[5] : org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage$Tag
            version : 1.4.7-jenkins-1
            -------------------------------
            at com.thoughtworks.xstream.converters.extended.StackTraceElementConverter.fromString(StackTraceElementConverter.java:93)
            at com.thoughtworks.xstream.converters.SingleValueConverterWrapper.fromString(SingleValueConverterWrapper.java:41)
            at com.thoughtworks.xstream.converters.SingleValueConverterWrapper.unmarshal(SingleValueConverterWrapper.java:49)
            {noformat}

            The reason this happens is that XStream uses a regex to extract the file name from the serialized format, and that regex assumes the file name does not contain a colon. However, in some cases stack traces for Groovy code contain the full path to the jar. I am not yet sure exactly why this is. [CompilationUnit.java|https://github.com/apache/groovy/blame/master/src/main/java/org/codehaus/groovy/control/CompilationUnit.java#L851-L852] appears to have fixed this exact issue 14 years ago, but perhaps this is specific to [MethodLocation|https://github.com/cloudbees/groovy-cps/blob/846d8d8eed10c57063d50894d2949e9a8fd21ae0/lib/src/main/java/com/cloudbees/groovy/cps/MethodLocation.java] in groovy-cps. Here is an excerpt of the serialized stack trace in case it helps identify the source of the file name:

            {noformat}
            ...
            <trace>WorkflowScript.run(WorkflowScript:14)</trace>
            <trace>org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(jar:file:/Users/dnusbaum/Projects/pipeline-graph-analysis-plugin/work/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:134)</trace>
            ...
            <trace>org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateStage(jar:file:/Users/dnusbaum/Projects/pipeline-graph-analysis-plugin/work/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:248)</trace>
            <trace>___cps.transform___(Native Method)</trace>
            <trace>com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)</trace>
            ...
            <trace>org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)</trace>
            {noformat}

            *Reproduction Steps:*

            # Start an instance of Jenkins with Blue Ocean and Declarative Pipeline installed.
            # Create a new Pipeline with the following script:
            {code}
            pipeline {
              agent none
              stages {
                stage('Fails') {
                  steps {
                    error 'oops'
                  }
                }
              }
            }
            {code}
            # Run the build and view the visualization in Blue Ocean.
            # Observe that the stage {{Fails}} is shown as failing.
            # Restart Jenkins
            # Open the visualization for the previous build.
            # Observe that the stage {{Fails}} is shown as succeeding.

            *Impact:*

            I first noticed this for a Declarative Pipeline, and the effect was that actions for {{FlowNodes}} failed to be deserialized after restarting Jenkins, so {{RobustReflectionConverter}} just nulled out the actions, which changed the visualization of the Pipeline in Blue Ocean because {{ErrorActions}} were no longer present on the nodes, so stages that actually failed were displayed as successful.

            I imagine there could be other similar issues for restarted Pipelines. I have only encountered stack traces with these kinds of file names in Declarative Pipelines, but perhaps non-Pipeline code could be affected by the same issue.

            *Solution*: 

            This could be fixed with a patch to XStream like [this|https://github.com/x-stream/xstream/compare/master...dwnusbaum:stack-traces], but I think it would be better to figure out what code is emitting these stack traces in the first place and modify that code in a similar way to https://github.com/apache/groovy/commit/4c42d503f6592c6ecc511c30bb49599316a937ff, unless displaying these kinds of file names in stack traces is normal in other contexts as well.
            jglick Jesse Glick made changes -
            Component/s workflow-cps-plugin [ 21713 ]
            Labels pipeline pipeline xstream
            jglick Jesse Glick made changes -
            Assignee Jesse Glick [ jglick ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "xstream #145 (Web Link)" [ 22724 ]
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "workflow-cps-global-lib #69 (Web Link)" [ 22726 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-31838 [ JENKINS-31838 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "groovy-cps #97 (Web Link)" [ 22727 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "workflow-cps #284 (Web Link)" [ 22728 ]
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            dnusbaum Devin Nusbaum made changes -
            Status In Review [ 10005 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Released As workflow-cps 2.68, groovy-cps 1.27

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                dnusbaum Devin Nusbaum
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: