Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-25734

Massive executors death on a node, causeOfDeath is null

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: core
    • Labels:
      None
    • Environment:
      Jenkins-1.532
      JNLP slaves
    • Similar Issues:

      Description

      Executors started dying on one node with unknown reason. Other nodes are not affected
      According to Jenkins UI, all executors (excepting ones with a running build) continuously die and restore

      • No error info even on the FINEST log level
      • Executor::causeOfDeath field is null when I open the death descriptor
      • causeOfDeath page is accessible, hence Executor::isAlive() returns false

      Probably, a node user decreased the number of executors (to be checked)

      WARNING: Caught exception evaluating: h.printThrowable(it.causeOfDeath) in /computer/XXXX/executors/4/causeOfDeath. Reason: java.lang.NullPointerException
      java.lang.NullPointerException
      at hudson.Functions.printThrowable(Functions.java:1184)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
      at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
      at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
      at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
      at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
      at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
      at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
      at org.apache.commons.jelly.parser.EscapingExpression.evaluate(EscapingExpression.java:24)
      at org.apache.commons.jelly.impl.ExpressionScript.run(ExpressionScript.java:66)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)

        Attachments

          Activity

          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          The code has been reworked in https://github.com/jenkinsci/jenkins/commit/73073387c85d90bb5543a856af24bfcb5590352a
          I'm not sure the issue may happen in newest core versions.
          BTW, it makes sense to handle NPE at least

          Show
          oleg_nenashev Oleg Nenashev added a comment - The code has been reworked in https://github.com/jenkinsci/jenkins/commit/73073387c85d90bb5543a856af24bfcb5590352a I'm not sure the issue may happen in newest core versions. BTW, it makes sense to handle NPE at least
          Hide
          oleg_nenashev Oleg Nenashev added a comment -
          Show
          oleg_nenashev Oleg Nenashev added a comment - Created PR: https://github.com/jenkinsci/jenkins/pull/1468 (WiP)
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          core/src/main/java/hudson/Functions.java
          core/src/main/java/hudson/model/Executor.java
          core/src/main/resources/hudson/Messages.properties
          http://jenkins-ci.org/commit/jenkins/bf7f14b165f235fed69ebabec0a852bff1874965
          Log:
          JENKINS-25734 - Prevent NPE in Executor/causeOfDeath page if the field is null

          The change does not fix the issue, but it suppresses NPE at least.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/hudson/Functions.java core/src/main/java/hudson/model/Executor.java core/src/main/resources/hudson/Messages.properties http://jenkins-ci.org/commit/jenkins/bf7f14b165f235fed69ebabec0a852bff1874965 Log: JENKINS-25734 - Prevent NPE in Executor/causeOfDeath page if the field is null The change does not fix the issue, but it suppresses NPE at least.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          core/src/main/java/hudson/Functions.java
          core/src/main/java/hudson/model/Executor.java
          core/src/main/resources/hudson/Messages.properties
          http://jenkins-ci.org/commit/jenkins/d5d8463e3759f00059d0586795952cdb9b1a4a8d
          Log:
          Merge pull request #1468 from synopsys-arc-oss/JENKINS-25734

          JENKINS-25734 - Prevent NPE in Executor/causeOfDeath page on "expected thread death"

          Compare: https://github.com/jenkinsci/jenkins/compare/194fa5655b28...d5d8463e3759

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/hudson/Functions.java core/src/main/java/hudson/model/Executor.java core/src/main/resources/hudson/Messages.properties http://jenkins-ci.org/commit/jenkins/d5d8463e3759f00059d0586795952cdb9b1a4a8d Log: Merge pull request #1468 from synopsys-arc-oss/ JENKINS-25734 JENKINS-25734 - Prevent NPE in Executor/causeOfDeath page on "expected thread death" Compare: https://github.com/jenkinsci/jenkins/compare/194fa5655b28...d5d8463e3759
          Hide
          dogfood dogfood added a comment -

          Integrated in jenkins_main_trunk #4177

          Result = SUCCESS

          Show
          dogfood dogfood added a comment - Integrated in jenkins_main_trunk #4177 Result = SUCCESS
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          The fix has been released in Jenkins 1.618

          Show
          oleg_nenashev Oleg Nenashev added a comment - The fix has been released in Jenkins 1.618

            People

            • Assignee:
              oleg_nenashev Oleg Nenashev
              Reporter:
              oleg_nenashev Oleg Nenashev
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: