Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53030

Intermittent hudson.util.IOException2 + org.xml.sax.SAXParseException

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Major Major

      I am working on a build + test + deployment script for multiple platforms. So far the script is still mostly a dummy with the actual work still left to fill in. Nevertheless, when running the script I get intermittent (about one in three or four runs) failures caused by always the same two exceptions:

       

      org.xml.sax.SAXParseException; systemId: file:/var/lib/jenkins/jobs/Temp.ScriptTest/branches/branches-nodenesting.skpm19/builds/109/changelog2.xml; lineNumber: 1; columnNumber: 1; Premature end of file.
       at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1239)
       at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
       at org.apache.commons.digester.Digester.parse(Digester.java:1871)
       at hudson.scm.SubversionChangeLogParser.parse(SubversionChangeLogParser.java:76)
      Caused: hudson.util.IOException2: Failed to parse /var/lib/jenkins/jobs/Temp.ScriptTest/branches/branches-nodenesting.skpm19/builds/109/changelog2.xml
       at hudson.scm.SubversionChangeLogParser.parse(SubversionChangeLogParser.java:80)
       at hudson.scm.SubversionChangeLogParser.parse(SubversionChangeLogParser.java:43)
       at org.jenkinsci.plugins.workflow.job.WorkflowRun.onCheckout(WorkflowRun.java:1024)
       at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$1400(WorkflowRun.java:143)
       at org.jenkinsci.plugins.workflow.job.WorkflowRun$SCMListenerImpl.onCheckout(WorkflowRun.java:1232)
       at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:127)
       at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:85)
       at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:75)
       at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
       at hudson.security.ACL.impersonate(ACL.java:290)
       at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
       at java.lang.Thread.run(Thread.java:745)

       

      The file involved is always changelog2.xml.

      The script I am running is this:

       

      // --------------------------------------------------------------------------
      // Project definitions used in the declarative pipeline below
      // --------------------------------------------------------------------------
      def maintainers = "${env.MAIL_VGIMPLE}"
          // these people will be notified when the build fails; comma separated
      def platforms = util.StringToArray(env.CONAN_PLATFORMS)
          // alternative: ['win32', 'x64', ...]
      def upstreamjobs = "BuildSystem.CMake.all"
          // these jobs will trigger this job; comma separated
      def parallelbuild = true
          // set to true to execute builds parallel;
      def crossplatformbuild = true
          // use cross platform builds where possible (i.e. Linux ARM builds will
          // be carried out on Linux PC nodes)
      
      // --------------------------------------------------------------------------
      // Diagnostic output and preparations
      // --------------------------------------------------------------------------
      echo "**** Building for the following platforms:"
      print platforms
      
      builds = [:]
      builds.failFast = true
      
      stashNameArtifacts = "artifacts." + env.BUILD_NUMBER + "."
      stashNameTests = "tests." + env.BUILD_NUMBER + "."
      
      // --------------------------------------------------------------------------
      // The build and test step
      // --------------------------------------------------------------------------
      def BuildAndTestStep(platform) {
          def isCrossPlatformBuild = util.IsCrossPlatformBuild(platform)
          try {        testplatform = platform.startsWith('build.') ? platform - 'build.' : platform
              node (platform) {
                  checkout scm
                  echo "--------> build step for platform " + platform
                  echo "--------> stash " + stashNameArtifacts + testplatform
                  if (isCrossPlatformBuild) {
                      echo "--------> stash " + stashNameTests + testplatform
                  } else {
                      echo "--------> execute tests directly on the node where the build ran"
                  }
              }
              if (isCrossPlatformBuild) {
                  node (testplatform) {
                      echo "--------> unstash " + stashNameTests + testplatform
                      echo "--------> execute cross platform test"
                  }
              }
          } catch(Exception e) {
              echo "**** An error occurred on build platform " + platform
              print e
              throw e
          }
      }
      
      // --------------------------------------------------------------------------
      // Pipeline Declaration
      // --------------------------------------------------------------------------
      pipeline {
          agent none
          triggers {
              pollSCM('@hourly')
              upstream(upstreamProjects: upstreamjobs, threshold: hudson.model.Result.SUCCESS)
          }
          stages {
              // ------------------------------------------------------------------
              // Build and Test step
              // ------------------------------------------------------------------
              stage ('build & test') {
                  steps {
                      script {
                          // now execute the builds as determined - either parallel
                          // or sequentially
                          prefix = crossplatformbuild ? "build." : ""
                          if (parallelbuild) {
                              for (int i = 0; i < platforms.size(); ++i) {
                                  def plt = prefix + platforms[i]
                                  builds[plt] = { -> BuildAndTestStep(plt) }
                              }
                              parallel builds
                          } else {
                              for (int i = 0; i < platforms.size(); ++i) {
                                  BuildAndTestStep(prefix + platforms[i])
                              }
                          }
                      }
                  }
              }
              // ------------------------------------------------------------------
              // Deployment step
              // ------------------------------------------------------------------
              stage ('deploy') {
                  steps {
                      script {
                          for (int i = 0; i < platforms.size(); ++i) {
                              node { // actual node does not matter!
                                  echo "--------> unstash " + stashNameArtifacts + platforms[i]
                                  echo "--------> install unstashed artifacts to either cvdev2 or conan"
                              }
                          }
                      }
                  }
              }
          }
          post {
              failure {
                  echo "*** Build Regression ***"
                  script {
                      mail (to: "${env.MAIL_VGIMPLE}",
                      subject: "Build failed in Jenkins: ${JOB_NAME} #${BUILD_NUMBER}",
                      body: 'See ' + currentBuild.absoluteUrl)
                  }
              }
              fixed {
                  echo "*** Build Fixed ***"
                  script {
                      mail (to: "${env.MAIL_VGIMPLE}",
                      subject: "Jenkins build is back to normal: ${JOB_NAME} #${BUILD_NUMBER}",
                      body: 'See ' + currentBuild.absoluteUrl)
                  }
              }
          }
      }

      A typical output log is attached (file Log114.txt).

      Looking at the failed builds it looks like in most cases the (generally slower) ARM-based test nodes are the ones where the exception is raised. The exceptions never occur when executing the script sequentially (i.e. when setting 'parallelbuild' to false and the 'parallel' block is never hit).

      My next move will be to try and filter out these particular exceptions and see if I have a chance at completing the build regardless but an approach where this is not necessary would of course be my preference.

       

            vaimr Denis Saponenko
            vgimple Volker Gimple
            Votes:
            3 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: