Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54046

GitHub branch souce plugin fails to move temporary files

    Details

    • Similar Issues:
    • Released As:
      github-branch-source-plugin 2.4.1

      Description

       

      After Jenkins restart I can build a project/branch/PR exactly once. After that I can see:

       

      INFO: sample-jenkins-pipeline-2/jisr.documentation/experimenting-with-testing-with-Jenkins #1 completed: ABORTED
      Oct 12, 2018 11:08:31 AM hudson.model.listeners.RunListener report
      WARNING: RunListener failed
      java.lang.IllegalStateException: closed
      {{ at okio.RealBufferedSink.writeUtf8(RealBufferedSink.java:56)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache.get(DiskLruCache.java:433)}}
      {{ at com.squareup.okhttp.Cache.get(Cache.java:184)}}
      {{ at com.squareup.okhttp.Cache$1.get(Cache.java:139)}}
      {{ at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:207)}}
      {{ at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:450)}}
      {{ at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:399)}}
      {{ at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getHeaders(HttpURLConnectionImpl.java:155)}}
      {{ at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getHeaderField(HttpURLConnectionImpl.java:199)}}
      {{ at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getHeaderField(DelegatingHttpsURLConnection.java:190)}}
      {{ at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getHeaderField(HttpsURLConnectionImpl.java:25)}}
      {{ at org.kohsuke.github.Requester.noteRateLimit(Requester.java:350)}}
      {{ at org.kohsuke.github.Requester._to(Requester.java:300)}}
      {{ at org.kohsuke.github.Requester.to(Requester.java:239)}}
      {{ at org.kohsuke.github.GitHub.getRepository(GitHub.java:474)}}
      {{ at org.jenkinsci.plugins.github_branch_source.GitHubBuildStatusNotification.lookUpRepo(GitHubBuildStatusNotification.java:146)}}
      {{ at org.jenkinsci.plugins.github_branch_source.GitHubBuildStatusNotification.createBuildCommitStatus(GitHubBuildStatusNotification.java:79)}}
      {{ at org.jenkinsci.plugins.github_branch_source.GitHubBuildStatusNotification.access$400(GitHubBuildStatusNotification.java:68)}}
      {{ at org.jenkinsci.plugins.github_branch_source.GitHubBuildStatusNotification$JobCompletedListener.onCompleted(GitHubBuildStatusNotification.java:306)}}
      {{ at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:211)}}
      {{ at org.jenkinsci.plugins.workflow.job.WorkflowRun.finish(WorkflowRun.java:807)}}
      {{ at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$1100(WorkflowRun.java:143)}}
      {{ at org.jenkinsci.plugins.workflow.job.WorkflowRun$GraphL.onNewHead(WorkflowRun.java:1220)}}
      {{ at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1437)}}
      {{ at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:417)}}
      {{ at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)}}
      {{ at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)}}
      {{ at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)}}
      {{ at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)}}
      {{ at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)}}
      {{ at java.util.concurrent.FutureTask.run(Unknown Source)}}
      {{ at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)}}
      {{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)}}
      {{ at java.lang.Thread.run(Unknown Source)}}Oct 12, 2018 11:08:32 AM jenkins.scm.api.SCMHeadEvent$DispatcherImpl log
      WARNING: SCMEventListener.onSCMHeadEvent(SCMHeadEvent) jenkins.branch.MultiBranchProject$SCMEventListenerImpl@1ceef7f propagated an exception
      java.lang.IllegalStateException: closed
      {{ at okio.RealBufferedSink.writeUtf8(RealBufferedSink.java:56)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache.get(DiskLruCache.java:433)}}
      {{ at com.squareup.okhttp.Cache.get(Cache.java:184)}}
      {{ at com.squareup.okhttp.Cache$1.get(Cache.java:139)}}
      {{ at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:207)}}
      {{ at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:450)}}
      {{ at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:399)}}
      {{ at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getHeaders(HttpURLConnectionImpl.java:155)}}
      {{ at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getHeaderField(HttpURLConnectionImpl.java:199)}}
      {{ at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getHeaderField(DelegatingHttpsURLConnection.java:190)}}
      {{ at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getHeaderField(HttpsURLConnectionImpl.java:25)}}
      {{ at org.kohsuke.github.Requester.noteRateLimit(Requester.java:350)}}
      {{ at org.kohsuke.github.Requester._to(Requester.java:300)}}
      {{ at org.kohsuke.github.Requester.to(Requester.java:239)}}
      {{ at org.kohsuke.github.GitHub.getRepository(GitHub.java:474)}}
      {{ at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:878)}}
      {{ at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:374)}}
      {{ at jenkins.scm.api.SCMSource.fetch(SCMSource.java:328)}}
      {{ at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.processHeadUpdate(MultiBranchProject.java:1589)}}
      {{ at jenkins.branch.MultiBranchProject$SCMEventListenerImpl.onSCMHeadEvent(MultiBranchProject.java:1190)}}
      {{ at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:247)}}
      {{ at jenkins.scm.api.SCMHeadEvent$DispatcherImpl.fire(SCMHeadEvent.java:230)}}
      {{ at jenkins.scm.api.SCMEvent$Dispatcher.run(SCMEvent.java:481)}}
      {{ at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)}}
      {{ at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)}}
      {{ at java.util.concurrent.FutureTask.run(Unknown Source)}}
      {{ at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)}}
      {{ at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)}}
      {{ at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)}}
      {{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)}}
      {{ at java.lang.Thread.run(Unknown Source)}}Oct 12, 2018 11:19:00 AM hudson.init.impl.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler uncaughtException
      SEVERE: A thread (OkHttp DiskLruCache/1443) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code.
      java.lang.RuntimeException: java.io.IOException: failed to rename C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal to C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal.bkp
      {{ at com.squareup.okhttp.internal.DiskLruCache$1.run(DiskLruCache.java:181)}}
      {{ at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)}}
      {{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)}}
      {{ at java.lang.Thread.run(Unknown Source)}}
      Caused by: java.io.IOException: failed to rename C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal to C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal.bkp
      {{ at com.squareup.okhttp.internal.io.FileSystem$1.rename(FileSystem.java:85)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache.rebuildJournal(DiskLruCache.java:407)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache.access$400(DiskLruCache.java:88)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache$1.run(DiskLruCache.java:177)}}
      {{ ... 3 more}}Oct 12, 2018 11:19:00 AM com.squareup.okhttp.internal.Platform$JdkWithJettyBootPlatform getSelectedProtocol
      INFO: ALPN callback dropped: SPDY and HTTP/2 are disabled. Is alpn-boot on the boot class path?
      Oct 12, 2018 11:19:00 AM jenkins.branch.OrganizationFolder$OrganizationScan run
      INFO: CSD organization #20181012.111900 organization scan action completed: FAILURE in 0.14 sec
      Oct 12, 2018 11:20:00 AM jenkins.branch.OrganizationFolder$OrganizationScan run
      INFO: sample-jenkins-pipeline-2 #20181012.112000 organization scan action completed: FAILURE in 11 ms
      Oct 12, 2018 11:31:00 AM hudson.init.impl.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler uncaughtException
      SEVERE: A thread (OkHttp DiskLruCache/1563) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code.
      java.lang.RuntimeException: java.io.IOException: failed to rename C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal to C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal.bkp
      {{ at com.squareup.okhttp.internal.DiskLruCache$1.run(DiskLruCache.java:181)}}
      {{ at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)}}
      {{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)}}
      {{ at java.lang.Thread.run(Unknown Source)}}
      Caused by: java.io.IOException: failed to rename C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal to C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal.bkp
      {{ at com.squareup.okhttp.internal.io.FileSystem$1.rename(FileSystem.java:85)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache.rebuildJournal(DiskLruCache.java:407)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache.access$400(DiskLruCache.java:88)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache$1.run(DiskLruCache.java:177)}}
      {{ ... 3 more}}Oct 12, 2018 11:31:00 AM com.squareup.okhttp.internal.Platform$JdkWithJettyBootPlatform getSelectedProtocol
      INFO: ALPN callback dropped: SPDY and HTTP/2 are disabled. Is alpn-boot on the boot class path?
      Oct 12, 2018 11:31:00 AM jenkins.branch.OrganizationFolder$OrganizationScan run
      INFO: CSD organization #20181012.113100 organization scan action completed: FAILURE in 0.16 sec
      Oct 12, 2018 11:32:00 AM jenkins.branch.OrganizationFolder$OrganizationScan run
      INFO: sample-jenkins-pipeline-2 #20181012.113200 organization scan action completed: FAILURE in 18 ms
      Oct 12, 2018 11:38:25 AM hudson.model.AsyncPeriodicWork$1 run
      INFO: Started Gravatar periodic lookup
      Oct 12, 2018 11:38:25 AM hudson.model.AsyncPeriodicWork$1 run
      INFO: Finished Gravatar periodic lookup. 1 ms
      Oct 12, 2018 11:43:00 AM hudson.init.impl.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler uncaughtException
      SEVERE: A thread (OkHttp DiskLruCache/1693) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code.
      java.lang.RuntimeException: java.io.IOException: failed to rename C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal to C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal.bkp
      {{ at com.squareup.okhttp.internal.DiskLruCache$1.run(DiskLruCache.java:181)}}
      {{ at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)}}
      {{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)}}
      {{ at java.lang.Thread.run(Unknown Source)}}
      Caused by: java.io.IOException: failed to rename C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal to C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache\D06sSK4O0xohU7Y5_vBnnFPSVeKSfbwBwHAxbgK0kH0\journal.bkp
      {{ at com.squareup.okhttp.internal.io.FileSystem$1.rename(FileSystem.java:85)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache.rebuildJournal(DiskLruCache.java:407)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache.access$400(DiskLruCache.java:88)}}
      {{ at com.squareup.okhttp.internal.DiskLruCache$1.run(DiskLruCache.java:177)}}
      {{ ... 3 more}}

       

      Removing temp directory C:\Jenkins\org.jenkinsci.plugins.github_branch_source.GitHubSCMProbe.cache helps so that I can build again, but again - only once

        Attachments

        1. screenshot-1.png
          screenshot-1.png
          434 kB
        2. screenshot-2.png
          screenshot-2.png
          489 kB
        3. screenshot-3.png
          screenshot-3.png
          554 kB

          Issue Links

            Activity

            Hide
            kshultz Karl Shultz added a comment -

            No. When set via the script console, you'll be back to 20 after a restart.

            To have the change persist, you'll want to set a system property. On my box (which is running Ubuntu Linux), that lives in /etc/default/jenkins. In JAVA_ARGS, mine now looks like this.

            # arguments to pass to java
            
            # Allow graphs etc. to work even when an X server is present
            # JAVA_ARGS="-Djava.awt.headless=true"
            JAVA_ARGS="-Djava.awt.headless=true -Dorg.jenkinsci.plugins.github_branch_source.GitHubSCMSource.cacheSize=0"
            

            Once the property is set, restart Jenkins via sudo service jenkins restart, or by whatever means you use on Windows to do the same thing. If you're running via java -jar..., you should be able to use the -Dorg.jenkinsci.plugins.github_branch_source.GitHubSCMSource.cacheSize=0 property right there.

            There's a second way to confirm that the setting has been changed. Under Manage Jenkins, go to "System Information" and look for the property. Should look like this now (I've drawn a red box around it):

            Show
            kshultz Karl Shultz added a comment - No. When set via the script console, you'll be back to 20 after a restart. To have the change persist, you'll want to set a system property. On my box (which is running Ubuntu Linux), that lives in /etc/default/jenkins . In JAVA_ARGS , mine now looks like this. # arguments to pass to java # Allow graphs etc. to work even when an X server is present # JAVA_ARGS="-Djava.awt.headless=true" JAVA_ARGS="-Djava.awt.headless=true -Dorg.jenkinsci.plugins.github_branch_source.GitHubSCMSource.cacheSize=0" Once the property is set, restart Jenkins via sudo service jenkins restart , or by whatever means you use on Windows to do the same thing. If you're running via java -jar... , you should be able to use the -Dorg.jenkinsci.plugins.github_branch_source.GitHubSCMSource.cacheSize=0 property right there. There's a second way to confirm that the setting has been changed. Under Manage Jenkins, go to "System Information" and look for the property. Should look like this now (I've drawn a red box around it):
            Hide
            kshultz Karl Shultz added a comment - - edited

            Looks like the default file on a Windows system would be C:\Program Files\Jenkins\jenkins.xml. Judging by what I see in the traces that Nick Jones and Lukasz Wiechec provided, I'm guessing both of yours will be in C:\Jenkins\jenkins.xml.

            Edit that file, find the <arguments> tag, and add the system property disabling the cache there, without removing any of the settings you already had there:

            <arguments>** leave the existing settings alone and add ** -Dorg.jenkinsci.plugins.github_branch_source.GitHubSCMSource.cacheSize=0</arguments>
            

            Personally, I hate editing xml files, so maybe take a backup of that file before you do so.

            Show
            kshultz Karl Shultz added a comment - - edited Looks like the default file on a Windows system would be C:\Program Files\Jenkins\jenkins.xml . Judging by what I see in the traces that Nick Jones and Lukasz Wiechec provided, I'm guessing both of yours will be in C:\Jenkins\jenkins.xml . Edit that file, find the <arguments> tag, and add the system property disabling the cache there, without removing any of the settings you already had there: <arguments> ** leave the existing settings alone and add ** -Dorg.jenkinsci.plugins.github_branch_source.GitHubSCMSource.cacheSize=0 </arguments> Personally, I hate editing xml files, so maybe take a backup of that file before you do so.
            Hide
            medianick Nick Jones added a comment -

            Thanks for your help, Karl Shultz! In our test Jenkins environment, I've updated the jenkins.xml file to set the system property, upgraded back to 2.4.0, and confirmed that the Script Console reports the cacheSize as being 0 after a restart. I'll keep an eye on it to see whether the error recurs there, but will otherwise hold off on upgrading our production Jenkins instance to 2.4.0 until I'm quite certain the issue is resolved by this system property (or there's a newer version of the plugin that fixes the underlying issue).

            Show
            medianick Nick Jones added a comment - Thanks for your help, Karl Shultz ! In our test Jenkins environment, I've updated the jenkins.xml file to set the system property, upgraded back to 2.4.0, and confirmed that the Script Console reports the cacheSize as being 0 after a restart. I'll keep an eye on it to see whether the error recurs there, but will otherwise hold off on upgrading our production Jenkins instance to 2.4.0 until I'm quite certain the issue is resolved by this system property (or there's a newer version of the plugin that fixes the underlying issue).
            Hide
            kshultz Karl Shultz added a comment -

            I'm so glad that helped fix you guys up. Thanks for reporting the bug, and thanks for all the followup information. That really helps! Sometimes bug reporters vanish, so thanks for sticking around.

            There's not a new version of the plugin yet - 2.4.0 is the latest version available. We'll need to get this fixed up, though. I've got an idea for how to do that, but it's more of a workaround than an actual fix. It would make the default cache behavior conditional.

            • Windows master? Cache disabled by default, but can be turned back on via the same means we've been using in this ticket.
            • !=Windows master? Cache enabled, still set to the previous default of 20MB. Can still be changed or disabled as you want.
            Show
            kshultz Karl Shultz added a comment - I'm so glad that helped fix you guys up. Thanks for reporting the bug, and thanks for all the followup information. That really helps! Sometimes bug reporters vanish, so thanks for sticking around. There's not a new version of the plugin yet - 2.4.0 is the latest version available. We'll need to get this fixed up, though. I've got an idea for how to do that, but it's more of a workaround than an actual fix. It would make the default cache behavior conditional. Windows master? Cache disabled by default, but can be turned back on via the same means we've been using in this ticket. !=Windows master? Cache enabled, still set to the previous default of 20MB. Can still be changed or disabled as you want.
            Hide
            kshultz Karl Shultz added a comment -

            Lukasz Wiechec and Nick Jones - a new version of the plugin, 2.4.1, has been released, and is in the update center. You can read about what I did in the pull request, but essentially, the default behavior for a Windows master is now for caching to be disabled. It can be turned back on, by any of the means we used on Friday to turn it back off.

            Show
            kshultz Karl Shultz added a comment - Lukasz Wiechec and Nick Jones - a new version of the plugin, 2.4.1, has been released, and is in the update center. You can read about what I did in the pull request , but essentially, the default behavior for a Windows master is now for caching to be disabled. It can be turned back on, by any of the means we used on Friday to turn it back off.

              People

              • Assignee:
                kshultz Karl Shultz
                Reporter:
                lwiechec Lukasz Wiechec
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: