Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-559

Move update stats census generation off of cucumber

    Details

    • Similar Issues:

      Description

      The update center census stuff takes a stupidly long time. I can get it to go faster, but that requires bumping the Mongo version and more memory (and probably more disk!) - I'd like to move the actual generation to another host, with rsyncing of /var/log/usage-stats from cucumber to the other host, a new job that actually runs the census updating on that other host, then stashes/archives it, and finally unstashes/unarchives it on cucumber. Or something. I'll work on it.

        Attachments

          Issue Links

            Activity

            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: R. Tyler Croy
            Path:
            manifests/site.pp
            http://jenkins-ci.org/commit/jenkins-infra/e578795189558b5bff818cb77205c1cd620e4014
            Log:
            Make kelp a census::agent

            References INFRA-559 INFRA-738

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: manifests/site.pp http://jenkins-ci.org/commit/jenkins-infra/e578795189558b5bff818cb77205c1cd620e4014 Log: Make kelp a census::agent References INFRA-559 INFRA-738
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: R. Tyler Croy
            Path:
            core/src/main/resources/hudson/model/UsageStatistics/footer.jelly
            http://jenkins-ci.org/commit/jenkins/01db7d36c83674dcf8fdfea9ba4e8df6a6d17a54
            Log:
            [FIXED JENKINS-35641] - Always send usage statistics over HTTPs to the new usage.jenkins.io hostname(#2398)

            References INFRA-559

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: core/src/main/resources/hudson/model/UsageStatistics/footer.jelly http://jenkins-ci.org/commit/jenkins/01db7d36c83674dcf8fdfea9ba4e8df6a6d17a54 Log: [FIXED JENKINS-35641] - Always send usage statistics over HTTPs to the new usage.jenkins.io hostname(#2398) References INFRA-559
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: R. Tyler Croy
            Path:
            Jenkinsfile
            http://jenkins-ci.org/commit/infra-statistics/db44840c4902f01b7fef916f740c94e350df167b
            Log:
            Add a Jenkinsfile for defining the full statistics processing pipeline

            Previously we used a series of chained jobs together, now all that logic can
            live in one place. This commit also makes use of the CloudBees Docker Pipeline
            plugin for spinning up MongoDB for parsing usage stats.

            References INFRA-559

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: Jenkinsfile http://jenkins-ci.org/commit/infra-statistics/db44840c4902f01b7fef916f740c94e350df167b Log: Add a Jenkinsfile for defining the full statistics processing pipeline Previously we used a series of chained jobs together, now all that logic can live in one place. This commit also makes use of the CloudBees Docker Pipeline plugin for spinning up MongoDB for parsing usage stats. References INFRA-559
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: R. Tyler Croy
            Path:
            Jenkinsfile
            http://jenkins-ci.org/commit/infra-statistics/755e5fc174863d951e529af58a55fe7caeaffcfb
            Log:
            Add a Jenkinsfile for defining the full statistics processing pipeline

            Previously we used a series of chained jobs together, now all that logic can
            live in one place. This commit also makes use of the CloudBees Docker Pipeline
            plugin for spinning up MongoDB for parsing usage stats.

            References INFRA-559

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: Jenkinsfile http://jenkins-ci.org/commit/infra-statistics/755e5fc174863d951e529af58a55fe7caeaffcfb Log: Add a Jenkinsfile for defining the full statistics processing pipeline Previously we used a series of chained jobs together, now all that logic can live in one place. This commit also makes use of the CloudBees Docker Pipeline plugin for spinning up MongoDB for parsing usage stats. References INFRA-559
            Hide
            rtyler R. Tyler Croy added a comment -

            I believe with the finalization of this job in ci.jenkins.io census and stats generation is finally back up and operational

            Show
            rtyler R. Tyler Croy added a comment - I believe with the finalization of this job in ci.jenkins.io census and stats generation is finally back up and operational

              People

              • Assignee:
                rtyler R. Tyler Croy
                Reporter:
                abayer Andrew Bayer
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: