Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-810

Make update center build logs available

    Details

    • Type: Improvement
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Won't Fix
    • Component/s: ci
    • Labels:
      None
    • Similar Issues:

      Description

      With the recent infra refactorings, the update center job was moved into the secured Jenkins instance.

      While I totally understand the reasons, in that case this makes helping plugin developers harder, almost impossible actually.

      The typical question we have, as you know, is "I released my plugin and it's not available, did something go wrong?".

      Before, we were available to crawl the UC build logs to find out if, say, it was ignored because of a missing wiki page, and so on.

      Today, we're unable to analyse it and have to bother one of the admins to know more about that.

      I'm not (necessarily) asking for access to that private Jenkins instance, but more of any way we could make those logs securely accessible for everybody (to let people find out the solution by themselves, granted we document how to do it w/ the wiki).

      Thanks!

        Attachments

          Issue Links

            Activity

            Hide
            batmat Baptiste Mathus added a comment -

            Depends on what your question relates to exactly.
            We have set up a RELK (Redis in front + ELK) installation in my current company for logs aggregation. We didn't fine tune anything, but I have used and configured logstash quite a few times (that cluster receives around 20 millions of lines of logs a week).

            I suppose that if Azure offers that solution, the only thing that would be left to us would be pushing the logs, right? In that case, I guess using logstash for that would be a no-brainer. Possibly, we would even want to use filebeat as it's designed to be lighter than logstash in terms of (memory, mostly) footprint.

            Show
            batmat Baptiste Mathus added a comment - Depends on what your question relates to exactly. We have set up a RELK (Redis in front + ELK) installation in my current company for logs aggregation. We didn't fine tune anything, but I have used and configured logstash quite a few times (that cluster receives around 20 millions of lines of logs a week). I suppose that if Azure offers that solution, the only thing that would be left to us would be pushing the logs, right? In that case, I guess using logstash for that would be a no-brainer. Possibly, we would even want to use filebeat as it's designed to be lighter than logstash in terms of (memory, mostly) footprint.
            Hide
            integer Kanstantsin Shautsou added a comment - - edited

            I would say it blocker as it doesn't allow plugin developers to have CD and understand even documented case https://wiki.jenkins-ci.org/display/JENKINS/Hosting+Plugins#HostingPlugins-Help%5C%21Mypluginisnotshowingupintheupdatecenter (keeping in mind wiki enforcement and other things).

            Show
            integer Kanstantsin Shautsou added a comment - - edited I would say it blocker as it doesn't allow plugin developers to have CD and understand even documented case https://wiki.jenkins-ci.org/display/JENKINS/Hosting+Plugins#HostingPlugins-Help%5C%21Mypluginisnotshowingupintheupdatecenter (keeping in mind wiki enforcement and other things).
            Hide
            rtyler R. Tyler Croy added a comment -

            Kanstantsin Shautsou, I agree that this is important. Are you volunteering to help or just setting the priorities for other contributors?

            Show
            rtyler R. Tyler Croy added a comment - Kanstantsin Shautsou , I agree that this is important. Are you volunteering to help or just setting the priorities for other contributors?
            Hide
            integer Kanstantsin Shautsou added a comment - - edited

            I can't debug why plugin is missing (though it seems missing only in root `update-center.json`), the same development blocking case may have anybody.

            As i mentioned in email (that in moderating queue) you can simply push logs with long existing jenkins feature https://wiki.jenkins-ci.org/display/JENKINS/Monitoring+external+jobs .

            Filebeat replaced logstash. In FOSS i see only ELK and graylog2 as good usable things. You would need to make clusters replicated for HA of course (with anti-affinity).

            Show
            integer Kanstantsin Shautsou added a comment - - edited I can't debug why plugin is missing (though it seems missing only in root `update-center.json`), the same development blocking case may have anybody. As i mentioned in email (that in moderating queue) you can simply push logs with long existing jenkins feature https://wiki.jenkins-ci.org/display/JENKINS/Monitoring+external+jobs . Filebeat replaced logstash. In FOSS i see only ELK and graylog2 as good usable things. You would need to make clusters replicated for HA of course (with anti-affinity).
            Hide
            danielbeck Daniel Beck added a comment -

            Given that the need for this has largely been obsoleted since we got rid of the link to the wiki, I think we don't need this anymore. It's been a while since anyone asked about plugin availability.

            Show
            danielbeck Daniel Beck added a comment - Given that the need for this has largely been obsoleted since we got rid of the link to the wiki, I think we don't need this anymore. It's been a while since anyone asked about plugin availability.

              People

              • Assignee:
                danielbeck Daniel Beck
                Reporter:
                batmat Baptiste Mathus
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: