Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-71227

Agent on AIX stops working with Jenkins 2.387.3

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Critical Critical
    • core, ssh-agent-plugin
    • None
    • AIX 7.2, JDK Semeru 11.0.17

      After upgrading Jenkins 2.387.2 to 2.387.3 the SSH agent on an AIX 7.2 machine sort of stops working. It is still connected, but unable to do any file operations. Starting a FreeStyle job fails. Even listing a workspace content is impossible.

      The log of the agent contains the following entries:

      Remoting version: 3107.v665000b_51092
Launcher: SSHLauncher
Communication Protocol: Standard in/out
This is a Unix agent
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jenkins.slaves.StandardOutputSwapper$ChannelSwapper to constructor java.io.FileDescriptor(int)
WARNING: Please consider reporting this to the maintainers of jenkins.slaves.StandardOutputSwapper$ChannelSwapper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Evacuated stdout
May 08, 2023 2:20:02 PM hudson.remoting.UserRequest perform
WARNING: LinkageError while performing UserRequest:hudson.FilePath$Exists@33285262
java.lang.ExceptionInInitializerError
    at java.base/java.lang.J9VMInternals.ensureError(J9VMInternals.java:206)
    at java.base/java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:195)
    at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
    at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1445)
    at java.base/jdk.internal.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    at java.base/jdk.internal.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:186)
    at java.base/java.lang.reflect.Field.acquireFieldAccessor(Field.java:1105)
    at java.base/java.lang.reflect.Field.getFieldAccessor(Field.java:1086)
    at java.base/java.lang.reflect.Field.getLong(Field.java:630)
    at java.base/java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1637)
    at java.base/java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:367)
    at java.base/java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:355)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:691)
    at java.base/java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:355)
    at java.base/java.io.ObjectStreamClass$Caches$1.computeValue(ObjectStreamClass.java:98)
    at java.base/java.io.ObjectStreamClass$Caches$1.computeValue(ObjectStreamClass.java:95)
    at java.base/java.io.ClassCache$1.computeValue(ClassCache.java:73)
    at java.base/java.io.ClassCache$1.computeValue(ClassCache.java:70)
    at java.base/java.lang.ClassValue.getFromHashMap(ClassValue.java:234)
    at java.base/java.lang.ClassValue.getFromBackup(ClassValue.java:216)
    at java.base/java.lang.ClassValue.get(ClassValue.java:122)
    at java.base/java.io.ClassCache.get(ClassCache.java:84)
    at java.base/java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:336)
    at java.base/java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:542)
    at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2141)
    at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1991)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2322)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1808)
    at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2624)
    at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2516)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2349)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1808)
    at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:573)
    at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:483)
    at hudson.remoting.UserRequest.deserialize(UserRequest.java:289)
    at hudson.remoting.UserRequest.perform(UserRequest.java:189)
    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
    at hudson.remoting.Request$2.run(Request.java:377)
    at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:839)
Caused by: sun.nio.fs.UnixException: A file, file system or message queue is no longer available.

      Trying to start a job on the agent results in the following stack traces in the job's log:

      00:35:24 Building remotely on aix[...] (OSSL30-AIX7_64 AIX7_64) in workspace /[...]
00:35:24 sun.nio.fs.UnixException: No such file or directory
00:35:24 Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to aix[...]
00:35:24 		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
00:35:24 		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
00:35:24 		at hudson.remoting.Channel.call(Channel.java:1000)
00:35:24 		at hudson.FilePath.act(FilePath.java:1192)
00:35:24 		at hudson.FilePath.act(FilePath.java:1181)
00:35:24 		at hudson.FilePath.mkdirs(FilePath.java:1372)
00:35:24 		at hudson.model.AbstractProject.checkout(AbstractProject.java:1234)
00:35:24 		at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:649)
00:35:24 		at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:85)
00:35:24 		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:521)
00:35:24 		at hudson.model.Run.execute(Run.java:1900)
00:35:24 		at hudson.matrix.MatrixRun.run(MatrixRun.java:153)
00:35:24 		at hudson.model.ResourceController.execute(ResourceController.java:101)
00:35:24 		at hudson.model.Executor.run(Executor.java:442)
00:35:24 Caused: java.lang.NoClassDefFoundError: hudson.FilePath (initialization failure)
00:35:24 	at java.base@11.0.17/java.lang.J9VMInternals.initializationAlreadyFailed(J9VMInternals.java:156)
00:35:24 	at java.base@11.0.17/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
00:35:24 	at java.base@11.0.17/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1445)
00:35:24 	at java.base@11.0.17/jdk.internal.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
00:35:24 	at java.base@11.0.17/jdk.internal.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:186)
00:35:24 	at java.base@11.0.17/java.lang.reflect.Field.acquireFieldAccessor(Field.java:1105)
00:35:24 	at java.base@11.0.17/java.lang.reflect.Field.getFieldAccessor(Field.java:1086)
00:35:24 	at java.base@11.0.17/java.lang.reflect.Field.getLong(Field.java:630)
00:35:24 	at java.base@11.0.17/java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1637)
00:35:24 	at java.base@11.0.17/java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:367)
00:35:24 	at java.base@11.0.17/java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:355)
00:35:24 	at java.base@11.0.17/java.security.AccessController.doPrivileged(AccessController.java:691)
00:35:24 	at java.base@11.0.17/java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:355)
00:35:24 	at java.base@11.0.17/java.io.ObjectStreamClass$Caches$1.computeValue(ObjectStreamClass.java:98)
00:35:24 	at java.base@11.0.17/java.io.ObjectStreamClass$Caches$1.computeValue(ObjectStreamClass.java:95)
00:35:24 	at java.base@11.0.17/java.io.ClassCache$1.computeValue(ClassCache.java:73)
00:35:24 	at java.base@11.0.17/java.io.ClassCache$1.computeValue(ClassCache.java:70)
00:35:24 	at java.base@11.0.17/java.lang.ClassValue.getFromHashMap(ClassValue.java:234)
00:35:24 	at java.base@11.0.17/java.lang.ClassValue.getFromBackup(ClassValue.java:216)
00:35:24 	at java.base@11.0.17/java.lang.ClassValue.get(ClassValue.java:122)
00:35:24 	at java.base@11.0.17/java.io.ClassCache.get(ClassCache.java:84)
00:35:24 	at java.base@11.0.17/java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:336)
00:35:24 	at java.base@11.0.17/java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:542)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2141)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1991)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2322)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1808)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2624)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2516)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2349)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1808)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.readObject(ObjectInputStream.java:573)
00:35:24 	at java.base@11.0.17/java.io.ObjectInputStream.readObject(ObjectInputStream.java:483)
00:35:24 	at app//hudson.remoting.UserRequest.deserialize(UserRequest.java:289)
00:35:24 	at app//hudson.remoting.UserRequest.perform(UserRequest.java:189)
00:35:24 	at app//hudson.remoting.UserRequest.perform(UserRequest.java:54)
00:35:24 	at app//hudson.remoting.Request$2.run(Request.java:377)
00:35:24 	at app//hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
00:35:24 	at java.base@11.0.17/java.util.concurrent.FutureTask.run(FutureTask.java:264)
00:35:24 	at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
00:35:24 	at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
00:35:24 	at java.base@11.0.17/java.lang.Thread.run(Thread.java:839)
00:35:24 Caused: java.io.IOException: Remote call on aix[...] failed
00:35:24 	at hudson.remoting.Channel.call(Channel.java:1004)
00:35:24 	at hudson.FilePath.act(FilePath.java:1192)
00:35:24 	at hudson.FilePath.act(FilePath.java:1181)
00:35:24 	at hudson.FilePath.mkdirs(FilePath.java:1372)
00:35:24 	at hudson.model.AbstractProject.checkout(AbstractProject.java:1234)
00:35:24 	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:649)
00:35:24 	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:85)
00:35:24 	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:521)
00:35:24 	at hudson.model.Run.execute(Run.java:1900)
00:35:24 	at hudson.matrix.MatrixRun.run(MatrixRun.java:153)
00:35:24 	at hudson.model.ResourceController.execute(ResourceController.java:101)
00:35:24 	at hudson.model.Executor.run(Executor.java:442)

      After returning to Jenkins 2.387.2 everything is fine again. The plugins are all up-to-date.

      : The illegal reflective access warning still appears, though:

      Remoting version: 3107.v665000b_51092
Launcher: SSHLauncher
Communication Protocol: Standard in/out
This is a Unix agent
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jenkins.slaves.StandardOutputSwapper$ChannelSwapper to constructor java.io.FileDescriptor(int)
WARNING: Please consider reporting this to the maintainers of jenkins.slaves.StandardOutputSwapper$ChannelSwapper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Evacuated stdout
ERROR: Unexpected error in launching an agent. This is probably a bug in Jenkins
java.lang.NoSuchMethodException: java.lang.Class.getDeclaredFields0(boolean)
	at java.base@11.0.17/java.lang.Class.newNoSuchMethodException(Class.java:649)
	at java.base@11.0.17/java.lang.Class.throwExceptionOrReturnNull(Class.java:1447)
	at java.base@11.0.17/java.lang.Class.getMethodHelper(Class.java:1533)
	at java.base@11.0.17/java.lang.Class.getDeclaredMethod(Class.java:1121)
	at org.jenkinsci.plugins.envinject.service.EnvInjectMasterEnvVarsSetter.getModifiers(EnvInjectMasterEnvVarsSetter.java:28)
	at org.jenkinsci.plugins.envinject.service.EnvInjectMasterEnvVarsSetter.call(EnvInjectMasterEnvVarsSetter.java:55)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to aix[...]
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:1000)
		at hudson.FilePath.act(FilePath.java:1283)
		at org.jenkinsci.plugins.envinject.EnvInjectComputerListener.onOnline(EnvInjectComputerListener.java:144)
		at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:748)
		at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:438)
		at hudson.plugins.sshslaves.SSHLauncher.startAgent(SSHLauncher.java:641)
		at hudson.plugins.sshslaves.SSHLauncher.lambda$launch$0(SSHLauncher.java:458)
		at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
		at java.base/java.lang.Thread.run(Thread.java:829)
Caused: java.lang.RuntimeException
	at org.jenkinsci.plugins.envinject.service.EnvInjectMasterEnvVarsSetter.call(EnvInjectMasterEnvVarsSetter.java:62)
	at org.jenkinsci.plugins.envinject.service.EnvInjectMasterEnvVarsSetter.call(EnvInjectMasterEnvVarsSetter.java:19)
	at app//hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at app//hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at app//hudson.remoting.Request$2.run(Request.java:377)
	at app//hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
	at java.base@11.0.17/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base@11.0.17/java.lang.Thread.run(Thread.java:839)
Agent successfully connected and online

      Please advise, thank you. – Matthias

       

            Unassigned Unassigned
            makr Matthias Kraft
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: