Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-29198

Console logs get truncated without reason

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Blocker
    • Resolution: Unresolved
    • Component/s: core
    • Environment:
      Jenkins ver. 1.553, Chrome browser
      Jenkins ver. 2.150.1 from "Official" Docker Image running on Ubuntu 16.04 with jenkins_home mounted from the local file system on a RAID 0 setup.
    • Similar Issues:

      Description

      I have a bug which looks similar to JENKINS-14899:
      Sometimes, when a job runs for a long time without anything getting output, the jenkins page stops updating until the job finishes. Then it add the completion lines (SUCCESS or FAILURE) to the job.
      The difference is that the number of line does not matter (last bug occurence was on a job that output 264 lines (+ an unknown number of non-output lines)

      This results in a truncated /console, /consoleFull and /consoleText.

      All files do end with the two lines (if it succeeds):
      Notifying upstream projects of job completion
      Finished: SUCCESS
      but hundreds of lines didn't get stored by jenkins (over 10 minutes of logs for my last job!)

      I can often see this bug when I'm running a long impala query, where the output stops before the status line (something like "Returned 0 row(s) in 1987.42s") and the rest of the console logs never get output.

      This happens approximately one time out of 20 for the jobs I'm running myself on this Jenkins server.

        Attachments

          Activity

          Hide
          webrat Andreas Sieferlinger added a comment - - edited

          We are seeing this issue aswell. Its seems like this happens mostly with pipelines that have multiple parallel steps.

          Running on 2.150.3

          We are using also the Timestamper plugin, just curious if anybody else facing the issue is using it aswell and if it might be related.

          Show
          webrat Andreas Sieferlinger added a comment - - edited We are seeing this issue aswell. Its seems like this happens mostly with pipelines that have multiple parallel steps. Running on 2.150.3 We are using also the Timestamper plugin, just curious if anybody else facing the issue is using it aswell and if it might be related.
          Hide
          laiph Gabriel Rusu added a comment -

          Indeed. For us, this happens when we run like 6 builds in parallel. However, if we break them in two, having only 3 builds run at a time, same setup, just two sequence of 3 builds, we get full logs.

          Show
          laiph Gabriel Rusu added a comment - Indeed. For us, this happens when we run like 6 builds in parallel. However, if we break them in two, having only 3 builds run at a time, same setup, just two sequence of 3 builds, we get full logs.
          Hide
          elliot_nelson Elliot Nelson added a comment -

          Recently seen this a couple times on Jenkins 2.193.

           

          In our case, it's a multijob that ends like this:

          ...
          Finished Build : #18084 of Job : fvt-worker-5 with status : SUCCESS
          Finished Build : #18084 of Job : fvt-worker-20 with status : SUCCESS
          Finished Build : #18085 of Job : fvt-worker-7 with status : SUCCESS
          Finished Build : #18085 of Job : fvt-worker-23 with status : SUCCESS
          Finished Build : #18084 of Job : fvt-worker-8 with status : SUCCESS
          Finished Build : Finished Build :  

           

          It's clear the job finished successfully (including another dozen subjobs that should be listed but aren't), and that it also completed all the publisher steps as expected, but there's about 120 lines of log at the end there that are missing (affects /console, /consoleFull, and /consoleText).

           

          Show
          elliot_nelson Elliot Nelson added a comment - Recently seen this a couple times on Jenkins 2.193.   In our case, it's a multijob that ends like this: ... Finished Build : #18084 of Job : fvt-worker-5 with status : SUCCESS Finished Build : #18084 of Job : fvt-worker-20 with status : SUCCESS Finished Build : #18085 of Job : fvt-worker-7 with status : SUCCESS Finished Build : #18085 of Job : fvt-worker-23 with status : SUCCESS Finished Build : #18084 of Job : fvt-worker-8 with status : SUCCESS Finished Build : Finished Build :   It's clear the job finished successfully (including another dozen subjobs that should be listed but aren't), and that it also completed all the publisher steps as expected, but there's about 120 lines of log at the end there that are missing (affects /console, /consoleFull, and /consoleText).  
          Hide
          akmjenkins ASHOK MOHANTY added a comment -

          In our case - "Console logs get truncated when job runs for a long time". Are there any log size limit !?

          Show
          akmjenkins ASHOK MOHANTY added a comment - In our case - "Console logs get truncated when job runs for a long time". Are there any log size limit !?
          Hide
          akmjenkins ASHOK MOHANTY added a comment -

          We are using core 2.164.3 version. Ay idea, by upgrading to any specific version - will it help !! 

          And/or any workaround we can try !!

          Show
          akmjenkins ASHOK MOHANTY added a comment - We are using core 2.164.3 version. Ay idea, by upgrading to any specific version - will it help !!  And/or any workaround we can try !!

            People

            • Assignee:
              Unassigned
              Reporter:
              camusensei Namikaze Minato
            • Votes:
              15 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated: