Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45150

large console logging can take Jenkins down or affecting performance

    Details

    • Similar Issues:

      Description

      Huge (too verbose) console logs can easily take an entire Jenkins master down and in order to assure that the system is stable we should limit the amount of log that can be generated by a single sh() script.

      This is also a security issue because one can easily do DDOS with a simple looping jobs. Jenkins stability with be serious affected in multiple ways: JVM memory, getting out of disk space, log processors blocking CPU for endless number of hours.

      While I do not find a DDOS high-likely to happen in real life, I do find high-likely to have badly designed jobs that do become too verbose. In some cases they may have almost-loops caused by retries or just because they were started with a too much verbosity.

      Based on my experience with one specific Jenkins instance, it seems that we got at least one partial-outage that required maintenance per month caused only due to having too verbose console logs.

      I think that the only wait possible to avoid these is to put some limits regarding how much sh() would send to the console. Once that limit is reached no output should be given.

      From my point of view it would even be acceptable to abandon the job, but maybe others would want to continue with it silenced or output saved to a file. The core team should be able to make a decision on this.

       

       

        Attachments

          Issue Links

            Activity

            ssbarnea Sorin Sbarnea created issue -
            danielbeck Daniel Beck made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-38313 [ JENKINS-38313 ]

              People

              • Assignee:
                Unassigned
                Reporter:
                ssbarnea Sorin Sbarnea
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: