Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42974

Build occasionally throws an exception and crashes during execution of shell command when running on zOS Unix Service slave.

    Details

    • Similar Issues:

      Description

      Jenkins version:
      Jenkins ver. 1.642.4

      Jenkins slave machine info:
      uname -a
      OS/390 SA2A 24.00 04 2964
      System running on zOS V2R1
      Java version on slave machine:
      java version "1.8.0"
      Java(TM) SE Runtime Environment (build pmz6480sr3fp11-20160817_02(SR3 FP11))
      IBM J9 VM (build 2.8, JRE 1.8.0 z/OS s390x-64 Compressed References 20160816_315341 (JIT enabled, AOT enabled)
      J9VM - R28_20160816_1459_B315341
      JIT - tr.r14.java.green_20160726_121883
      GC - R28_20160816_1459_B315341_CMPRSS
      J9CL - 20160816_315341)
      JCL - 20160816_01 based on Oracle jdk8u101-b13

      We have a build slave set up under zOS Unix System Service.
      During the build, we execute a shell script mounted via NFS mount from a server.
      From time to time, the execution of such shell script throws an exception

      Note: the error does not ALWAYS occur, the build runs fine most of the time, but occurs only occasionally. However when it occurs, the build will ABEND and no workaround is found. So I am opening it as Priority "Major".

      The following is what's defined in Execute Shell Command in the Build tab in configuration for the project:
      #!/bin/sh
      sh -x <a_prebuild_shell_script_located_on_nfs_server>

      The following is the error from the console output:
      ....
      [EnvInject] - Injecting environment variables from a build step.
      [EnvInject] - Injecting as environment variables the properties content
      _BPXK_AUTOCVT=ON

      [EnvInject] - Variables injected successfully.
      ERROR: Build step failed with exception
      java.security.ProviderException: nextBytes() failed
      at com.ibm.crypto.provider.G.b(Unknown Source)
      at com.ibm.crypto.provider.G.b(Unknown Source)
      at com.ibm.crypto.provider.NativePRNG_Unix.engineNextBytes(Unknown Source)
      at java.security.SecureRandom.nextBytes(SecureRandom.java:471)
      at java.security.SecureRandom.next(SecureRandom.java:494)
      at java.util.Random.nextLong(Random.java:435)
      at java.io.File$TempDirectory.generateFile(File.java:1916)
      at java.io.File.createTempFile(File.java:2021)
      at hudson.FilePath$17.invoke(FilePath.java:1366)
      at hudson.FilePath$17.invoke(FilePath.java:1356)
      at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2719)
      at hudson.remoting.UserRequest.perform(UserRequest.java:120)
      at hudson.remoting.UserRequest.perform(UserRequest.java:48)
      at hudson.remoting.Request$2.run(Request.java:326)
      at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
      at java.util.concurrent.FutureTask.run(FutureTask.java:277)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      at java.lang.Thread.run(Thread.java:785)
      at ......remote call to <our_zos_slave_host_name>(Native Method)
      at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
      at hudson.remoting.UserResponse.retrieve(UserRequest.java:220)
      at hudson.remoting.Channel.call(Channel.java:781)
      at hudson.FilePath.act(FilePath.java:979)
      at hudson.FilePath.act(FilePath.java:968)
      at hudson.FilePath.createTextTempFile(FilePath.java:1356)
      at hudson.tasks.CommandInterpreter.createScriptFile(CommandInterpreter.java:142)
      at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:80)
      at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:64)
      at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
      at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
      at hudson.model.Build$BuildExecution.build(Build.java:205)
      at hudson.model.Build$BuildExecution.doRun(Build.java:162)
      at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
      at hudson.model.Run.execute(Run.java:1738)
      at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
      at hudson.model.ResourceController.execute(ResourceController.java:98)
      at hudson.model.Executor.run(Executor.java:410)
      Caused by: java.io.EOFException: File(s) closed?
      at com.ibm.crypto.provider.G.a(Unknown Source)
      at com.ibm.crypto.provider.G.b(Unknown Source)
      at com.ibm.crypto.provider.G.b(Unknown Source)
      at com.ibm.crypto.provider.G.b(Unknown Source)
      at com.ibm.crypto.provider.NativePRNG_Unix.engineNextBytes(Unknown Source)
      at java.security.SecureRandom.nextBytes(SecureRandom.java:471)
      at java.security.SecureRandom.next(SecureRandom.java:494)
      at java.util.Random.nextLong(Random.java:435)
      at java.io.File$TempDirectory.generateFile(File.java:1916)
      at java.io.File.createTempFile(File.java:2021)
      at hudson.FilePath$17.invoke(FilePath.java:1366)
      at hudson.FilePath$17.invoke(FilePath.java:1356)
      at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2719)
      at hudson.remoting.UserRequest.perform(UserRequest.java:120)
      at hudson.remoting.UserRequest.perform(UserRequest.java:48)
      at hudson.remoting.Request$2.run(Request.java:326)
      at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
      at java.util.concurrent.FutureTask.run(FutureTask.java:277)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      at java.lang.Thread.run(Thread.java:785)
      Build step 'Execute shell' marked build as failure
      Archiving artifacts
      [BFA] Scanning build for known causes...
      [BFA] No failure causes found
      [BFA] Done. 0s
      Finished: FAILURE

        Attachments

          Issue Links

            Activity

            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            I suppose it is an issue specific to IBM Java 9 SecureRandom implementation. I doubt we can do something specific about it. In the bottom of https://issues.apache.org/jira/browse/AXIS-2568 you may find the same issue, but there is no resolution there for this specific issue. I suppose the best workaround is to specify another random number provider in JVM settings

            Show
            oleg_nenashev Oleg Nenashev added a comment - I suppose it is an issue specific to IBM Java 9 SecureRandom implementation. I doubt we can do something specific about it. In the bottom of https://issues.apache.org/jira/browse/AXIS-2568 you may find the same issue, but there is no resolution there for this specific issue. I suppose the best workaround is to specify another random number provider in JVM settings
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            See other issues referenced in JENKINS-23176

            Show
            oleg_nenashev Oleg Nenashev added a comment - See other issues referenced in JENKINS-23176

              People

              • Assignee:
                Unassigned
                Reporter:
                stevench2000 Steven Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: