Uploaded image for project: 'Jenkins Website'
  1. Jenkins Website
  2. WEBSITE-363

Plugin site stopped updating 4-6 days ago

    Details

    • Similar Issues:

      Description

      HOSTING-348 mentions that a new plugin doesn't get listed on plugins.jenkins.io despite having been released four days ago.

      The most recently updated plugin on plugins.j.io was updated 6 days ago. Something stopped.

      https://ci.jenkins.io/job/Infra/job/plugin-site-api/job/generate-data/ looks good.

        Attachments

          Issue Links

            Activity

            Hide
            mmccaskill Michael McCaskill added a comment -

            I have confirmed the index currently being used is well behind:

            Michaels-MacBook-Pro:~ mmccaskill $ http https://plugins.jenkins.io/api/health/elasticsearch
            HTTP/1.1 200 OK
            Access-Control-Allow-Credentials: true
            Access-Control-Allow-Headers: origin, content-type, accept, authorization
            Access-Control-Allow-Methods: GET
            Access-Control-Allow-Origin: *
            Connection: keep-alive
            Content-Length: 207
            Content-Type: application/json
            Date: Tue, 06 Jun 2017 14:01:23 GMT
            Server: nginx/1.11.10
            Strict-Transport-Security: max-age=15724800; preload
            
            {
                "createdAt": "2017-05-31T16:46:40",
                "elasticsearchVersion": "50ac9ff046e4e9a23f4804d4df5ab1681cbf0542263993bb8c2ca89108608967",
                "mappingVersion": "deddc38a424f0fc617d5203fffec0cb1398d3c8db4278cb32b4d3880a0a9948e"
            }
            

            I have also deployed locally and ensured the elasticsearchVersion and mappingVersion matches so it's not that. I don't have access to any of the underlying infrastructure so I cannot see any logs to determine the cause of the issue. Given that searching is still working it leads me to believe Elasticsearch is still running and it must be something to do with the process that re-indexes the data.

            Show
            mmccaskill Michael McCaskill added a comment - I have confirmed the index currently being used is well behind: Michaels-MacBook-Pro:~ mmccaskill $ http https://plugins.jenkins.io/api/health/elasticsearch HTTP/1.1 200 OK Access-Control-Allow-Credentials: true Access-Control-Allow-Headers: origin, content-type, accept, authorization Access-Control-Allow-Methods: GET Access-Control-Allow-Origin: * Connection: keep-alive Content-Length: 207 Content-Type: application/json Date: Tue, 06 Jun 2017 14:01:23 GMT Server: nginx/1.11.10 Strict-Transport-Security: max-age=15724800; preload { "createdAt": "2017-05-31T16:46:40", "elasticsearchVersion": "50ac9ff046e4e9a23f4804d4df5ab1681cbf0542263993bb8c2ca89108608967", "mappingVersion": "deddc38a424f0fc617d5203fffec0cb1398d3c8db4278cb32b4d3880a0a9948e" } I have also deployed locally and ensured the elasticsearchVersion and mappingVersion matches so it's not that. I don't have access to any of the underlying infrastructure so I cannot see any logs to determine the cause of the issue. Given that searching is still working it leads me to believe Elasticsearch is still running and it must be something to do with the process that re-indexes the data.
            Hide
            mmccaskill Michael McCaskill added a comment - - edited

            Daniel gave me a snippet of the logs.

            2017-05-31T18:05:34.157399299Z 18:05:34.156 [jersey-background-task-scheduler-0] ERROR i.j.p.s.i.DefaultConfigurationService - Data file not found
            2017-05-31T18:05:34.157446102Z 18:05:34.156 [jersey-background-task-scheduler-0] ERROR i.j.p.s.i.DefaultConfigurationService - Problem getting data file
            2017-05-31T18:05:34.157450202Z java.lang.RuntimeException: Data file not found
            2017-05-31T18:05:34.157453102Z  at io.jenkins.plugins.services.impl.DefaultConfigurationService.getIndexData(DefaultConfigurationService.java:79)
            2017-05-31T18:05:34.157462302Z  at io.jenkins.plugins.services.impl.ElasticsearchPrepareDatastoreService.populateDataStore(ElasticsearchPrepareDatastoreService.java:51)
            2017-05-31T18:05:34.157465403Z  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            2017-05-31T18:05:34.157468203Z  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
            2017-05-31T18:05:34.157471103Z  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
            2017-05-31T18:05:34.157474103Z  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
            2017-05-31T18:05:34.157477303Z  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            2017-05-31T18:05:34.157479903Z  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            2017-05-31T18:05:34.157489704Z  at java.lang.Thread.run(Thread.java:745)
            2017-05-31T18:05:34.158206441Z 18:05:34.157 [jersey-background-task-scheduler-0] ERROR i.j.p.s.i.ElasticsearchPrepareDatastoreService - Problem populating index
            2017-05-31T18:05:34.158215041Z io.jenkins.plugins.services.ServiceException: Problem getting data file
            2017-05-31T18:05:34.158218241Z  at io.jenkins.plugins.services.impl.DefaultConfigurationService.getIndexData(DefaultConfigurationService.java:83)
            2017-05-31T18:05:34.158221241Z  at io.jenkins.plugins.services.impl.ElasticsearchPrepareDatastoreService.populateDataStore(ElasticsearchPrepareDatastoreService.java:51)
            2017-05-31T18:05:34.158224342Z  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            2017-05-31T18:05:34.158227242Z  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
            2017-05-31T18:05:34.158230242Z  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
            2017-05-31T18:05:34.158233142Z  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
            2017-05-31T18:05:34.158236242Z  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            2017-05-31T18:05:34.158239042Z  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            2017-05-31T18:05:34.158241742Z  at java.lang.Thread.run(Thread.java:745)
            2017-05-31T18:05:34.158244743Z Caused by: java.lang.RuntimeException: Data file not found
            2017-05-31T18:05:34.158247443Z  at io.jenkins.plugins.services.impl.DefaultConfigurationService.getIndexData(DefaultConfigurationService.java:79)
            2017-05-31T18:05:34.158250343Z  ... 8 common frames omitted
            

            We both agree it seems the thread responsible for checking for data file updates crashed and didn't recover. I will add in some recovery code.

            Show
            mmccaskill Michael McCaskill added a comment - - edited Daniel gave me a snippet of the logs. 2017-05-31T18:05:34.157399299Z 18:05:34.156 [jersey-background-task-scheduler-0] ERROR i.j.p.s.i.DefaultConfigurationService - Data file not found 2017-05-31T18:05:34.157446102Z 18:05:34.156 [jersey-background-task-scheduler-0] ERROR i.j.p.s.i.DefaultConfigurationService - Problem getting data file 2017-05-31T18:05:34.157450202Z java.lang.RuntimeException: Data file not found 2017-05-31T18:05:34.157453102Z  at io.jenkins.plugins.services.impl.DefaultConfigurationService.getIndexData(DefaultConfigurationService.java:79) 2017-05-31T18:05:34.157462302Z  at io.jenkins.plugins.services.impl.ElasticsearchPrepareDatastoreService.populateDataStore(ElasticsearchPrepareDatastoreService.java:51) 2017-05-31T18:05:34.157465403Z  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 2017-05-31T18:05:34.157468203Z  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 2017-05-31T18:05:34.157471103Z  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) 2017-05-31T18:05:34.157474103Z  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) 2017-05-31T18:05:34.157477303Z  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2017-05-31T18:05:34.157479903Z  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2017-05-31T18:05:34.157489704Z  at java.lang.Thread.run(Thread.java:745) 2017-05-31T18:05:34.158206441Z 18:05:34.157 [jersey-background-task-scheduler-0] ERROR i.j.p.s.i.ElasticsearchPrepareDatastoreService - Problem populating index 2017-05-31T18:05:34.158215041Z io.jenkins.plugins.services.ServiceException: Problem getting data file 2017-05-31T18:05:34.158218241Z  at io.jenkins.plugins.services.impl.DefaultConfigurationService.getIndexData(DefaultConfigurationService.java:83) 2017-05-31T18:05:34.158221241Z  at io.jenkins.plugins.services.impl.ElasticsearchPrepareDatastoreService.populateDataStore(ElasticsearchPrepareDatastoreService.java:51) 2017-05-31T18:05:34.158224342Z  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 2017-05-31T18:05:34.158227242Z  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 2017-05-31T18:05:34.158230242Z  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) 2017-05-31T18:05:34.158233142Z  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) 2017-05-31T18:05:34.158236242Z  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2017-05-31T18:05:34.158239042Z  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2017-05-31T18:05:34.158241742Z  at java.lang.Thread.run(Thread.java:745) 2017-05-31T18:05:34.158244743Z Caused by: java.lang.RuntimeException: Data file not found 2017-05-31T18:05:34.158247443Z  at io.jenkins.plugins.services.impl.DefaultConfigurationService.getIndexData(DefaultConfigurationService.java:79) 2017-05-31T18:05:34.158250343Z  ... 8 common frames omitted We both agree it seems the thread responsible for checking for data file updates crashed and didn't recover. I will add in some recovery code.

              People

              • Assignee:
                mmccaskill Michael McCaskill
                Reporter:
                danielbeck Daniel Beck
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: