Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44568

Gerrit Trigger version 23.2 fails to trigger Jenkins jobs on Jenkins version 2.46.3!!!

    Details

    • Similar Issues:

      Description

      Hello,

      Last week we upgraded our Jenkins instance to version 2.46.3 (on/around 05/25/2017).  Upon successfully completing/testing that upgrade, we upgraded our Gerrit trigger plug-in version from 2.23.0 -> 2.23.2.

      Since performing that Gerrit trigger plug-in upgrade, our Jenkins server has completely stopped processing ANY of the stream events generated by code pushes by our users, period.  I've pushed code a number of times myself and the Gerrit trigger plug-in simply ignores all of the events, and this is in spite of the fact that I am monitoring the events coming across myself (in a separate window).

      Our configuration is simple; we are not filtering-out any specific event types.  This software simply isn't working.

      The only way to get things working again was to back-out the 2.23.2 version of the Gerrit trigger plug-in and go back to the 2.23.0 version.  Once we bounced the Jenkins server and re-pushed some of the previously-failed changes, everything started working again like clockwork.

      Please let me know if there is any additional information I can provide you to help troubleshoot this.  We don't have access to the log files on our Gerrit servers (it's centralized in a datacenter in France), but I do have access to our Jenkins server's logs.

      I will assist you in any way I can.

      I created this tracker as 'Major' because I believe it deserves investigation and prior to the creation of this tracker – literally only an hour ago – it was blocking our ability to deliver our product.  I consider the fallback to 2.23.0 to be a fortunate work-around, but it's not optimal.  We like to keep our plug-ins updated with the latest versions whenever possible.

      Any help you can offer is greatly appreciated!

      Kind Regards,

      Marcus E. Harris

        Attachments

          Issue Links

            Activity

            Hide
            mawinter69 Markus Winter added a comment -

            Actually with the current Unit tests I can easily reproduce the current problem.

            In GerritConnectionTest just add the line

            connection.setSshRxBufferSize(20);

            The test will then never finish, because the position is set after the clear to the capacity of the buffer and we have one event that is bigger than the buffer.

            If I remove the change of position after the clear, the test fails because data is lost.

            Show
            mawinter69 Markus Winter added a comment - Actually with the current Unit tests I can easily reproduce the current problem. In GerritConnectionTest just add the line connection.setSshRxBufferSize(20); The test will then never finish, because the position is set after the clear to the capacity of the buffer and we have one event that is bigger than the buffer. If I remove the change of position after the clear, the test fails because data is lost.
            Hide
            ssbarnea Sorin Sbarnea added a comment - - edited

            I was able to obtain GerritHub messages of 78KB in less than a day of saving the event stream so we know at least one of the bugs with the current code. I propose configuring a buffer of at least 256KB.

            This should be implemented and released ASAP and after this we should have enough time to rewrite the implementation without having daily outages.

            PR at https://github.com/sonyxperiadev/gerrit-events/pull/67 – seems like a good enough reason for a dot release.

            Show
            ssbarnea Sorin Sbarnea added a comment - - edited I was able to obtain GerritHub messages of 78KB in less than a day of saving the event stream so we know at least one of the bugs with the current code. I propose configuring a buffer of at least 256KB. This should be implemented and released ASAP and after this we should have enough time to rewrite the implementation without having daily outages. PR at https://github.com/sonyxperiadev/gerrit-events/pull/67 – seems like a good enough reason for a dot release.
            Hide
            mawinter69 Markus Winter added a comment - - edited

            Locally my fixed version is running stable since Friday while the buggy version stopped twice since then.

            PR: https://github.com/sonyxperiadev/gerrit-events/pull/68

            Show
            mawinter69 Markus Winter added a comment - - edited Locally my fixed version is running stable since Friday while the buggy version stopped twice since then. PR: https://github.com/sonyxperiadev/gerrit-events/pull/68
            Hide
            epkaxma Martin Wiklundh added a comment -

            Any progress on this? Any way to test some of the suggested solutions?

            Show
            epkaxma Martin Wiklundh added a comment - Any progress on this? Any way to test some of the suggested solutions?
            Hide
            mawinter69 Markus Winter added a comment -

            The fix is in since 2.25.0

            Show
            mawinter69 Markus Winter added a comment - The fix is in since 2.25.0

              People

              • Assignee:
                rsandell rsandell
                Reporter:
                harrism Marcus Harris
              • Votes:
                12 Vote for this issue
                Watchers:
                19 Start watching this issue

                Dates

                • Created:
                  Updated: