Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30975

Jobs not firing; NPE sending event to GerritMissedEventsPlaybackManager

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Labels:
      None
    • Environment:
      Jenkins 1.609.3
      gerrit-trigger 2.16.0
      Gerrit 2.11.3 w/ events-log v2.11.3-8-g318953f
      OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.14.04.1)
      Ubuntu 14.04.3 LTS
    • Similar Issues:

      Description

      We installed the events-log and enabled the REST integration this morning. Almost immediately, jobs stopped firing on gerrit events. We noted high CPU usage on the Jenkins process, and looking at the javamelody monitoring we saw that 2 of the 3 GerritMissedEventsPlaybackManager receive threads were blocked. In our jenkins.log we saw entries similar to the following:

      Oct 15, 2015 10:41:04 AM com.sonymobile.tools.gerrit.gerritevents.GerritHandler notifyListeners
      SEVERE: When notifying listener: com.sonyericsson.hudson.plugins.gerrit.trigger.playback.GerritMissedEventsPlaybackManager@7a9a6d56 about event: ManualPatchsetCreated Change: Change-Id for #36488: Ia70accbcf0ef07628b6ea861af546235adc2c1fb PatchSet: PatchSet: 1
      Oct 15, 2015 10:41:04 AM com.sonymobile.tools.gerrit.gerritevents.GerritHandler notifyListeners
      SEVERE: Notify-error:
      java.lang.NullPointerException
      at com.sonyericsson.hudson.plugins.gerrit.trigger.playback.GerritMissedEventsPlaybackManager.persist(GerritMissedEventsPlaybackManager.java:411)
      at com.sonyericsson.hudson.plugins.gerrit.trigger.playback.GerritMissedEventsPlaybackManager.gerritEvent(GerritMissedEventsPlaybackManager.java:256)
      at com.sonymobile.tools.gerrit.gerritevents.GerritHandler.notifyListener(GerritHandler.java:317)
      at com.sonymobile.tools.gerrit.gerritevents.GerritHandler.notifyListeners(GerritHandler.java:285)
      at com.sonyericsson.hudson.plugins.gerrit.trigger.JenkinsAwareGerritHandler.notifyListeners(JenkinsAwareGerritHandler.java:77)
      at com.sonymobile.tools.gerrit.gerritevents.workers.AbstractGerritEventWork.perform(AbstractGerritEventWork.java:46)
      at com.sonymobile.tools.gerrit.gerritevents.workers.GerritEventWork.perform(GerritEventWork.java:48)
      at com.sonymobile.tools.gerrit.gerritevents.workers.EventThread.run(EventThread.java:66)
      at com.sonyericsson.hudson.plugins.gerrit.trigger.SystemEventThread.run(SystemEventThread.java:66)

      Looking at persist(), it looks like the passed-in Gerrit event is null or missing GetEventCreatedOn(), but that doesn't make sense.

      We've disabled REST for the time being, because this behavior effectively breaks our Jenkins deployment.

        Attachments

          Activity

          Hide
          scoheb Scott Hebert added a comment -

          Can you run a:

          ssh -p 29418 <your gerrit server hostname> gerrit stream-events

          and see if events do in fact contain eventCreatedOn attributes?

          I added protection in that method by checking if eventCreatedOn.getTime() was 0. That was supposed to handle Gerrit Servers that did not support that attribute.

          I can further protect it by doing some null checks.

          Show
          scoheb Scott Hebert added a comment - Can you run a: ssh -p 29418 <your gerrit server hostname> gerrit stream-events and see if events do in fact contain eventCreatedOn attributes? I added protection in that method by checking if eventCreatedOn.getTime() was 0. That was supposed to handle Gerrit Servers that did not support that attribute. I can further protect it by doing some null checks.
          Show
          scoheb Scott Hebert added a comment - Created PR https://github.com/jenkinsci/gerrit-trigger-plugin/pull/261
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Scott Hebert
          Path:
          src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/playback/GerritMissedEventsPlaybackManager.java
          src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/playback/GerritMissedEventsLoadPersistTest.java
          src/test/resources/com/sonyericsson/hudson/plugins/gerrit/trigger/playback/DeserializeEventCreatedOnTest.json
          http://jenkins-ci.org/commit/gerrit-trigger-plugin/7fd089691116567def282f47b8059d647e5834f9
          Log:
          Protect PlaybackManager from null EventCreatedOn

          It is possible that Gerrit might return a null eventCreated attribute for some events.

          This protects the persistence of events from this situation.

          [FIXED JENKINS-30975]

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Scott Hebert Path: src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/playback/GerritMissedEventsPlaybackManager.java src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/playback/GerritMissedEventsLoadPersistTest.java src/test/resources/com/sonyericsson/hudson/plugins/gerrit/trigger/playback/DeserializeEventCreatedOnTest.json http://jenkins-ci.org/commit/gerrit-trigger-plugin/7fd089691116567def282f47b8059d647e5834f9 Log: Protect PlaybackManager from null EventCreatedOn It is possible that Gerrit might return a null eventCreated attribute for some events. This protects the persistence of events from this situation. [FIXED JENKINS-30975]

            People

            • Assignee:
              scoheb Scott Hebert
              Reporter:
              wsaxon Will Saxon
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: