Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44699

Limit the number of lines that are shown for the log in the results page

    Details

    • Similar Issues:
    • Sprint:
      Blue Ocean 1.1, Blue Ocean 1.2, Blue Ocean 1.3, Blue Ocean 1.4 - beta 1, Blue Ocean 1.4 - beta 3, Blue Ocean 1.4 - beta 2, Blue Ocean 1.6 - beta 2, Blue Ocean - 1.6 - beta 4

      Description

      Blue Ocean streams an exceptional amount of logs to the client.

      In Scope

      • Limit all steps to show a maximum of 100 lines (last 100)

      Out of scope

      • Any functional changes to the amount of logs shown for non-pipeline

      Notes
      There may be a frontend and backend component to this.

      This means that there would only be 100 lines visible on screen at all times and the only way to see the whole log for the step is the "show more" button.

        Attachments

          Issue Links

            Activity

            Hide
            jamesdumay James Dumay added a comment -

            Vivek Pandey Josh McDonald it would be good if you two could collaborate on this. Some frontend work to change the behaviour of our log lines and some backend changes to the behaviour of the page of logs we get back to server.

            Show
            jamesdumay James Dumay added a comment - Vivek Pandey Josh McDonald it would be good if you two could collaborate on this. Some frontend work to change the behaviour of our log lines and some backend changes to the behaviour of the page of logs we get back to server.
            Hide
            vivek Vivek Pandey added a comment -

            Serving last n number of lines could be expensive, we can end up reading all file. Instead Logging API for step or for run serves last 150KB log data by default. It's controllable using query param thresholdInKB, its default value is 150. Basically it results in to computing offset, `offset=logFileLength-thresholdInKB` in to the log file.

            For example, this is default rest API called from UI, it fetches last 150KB of log, in terms of line this particular log is about 1900 lines.

            https://ci.blueocean.io/blue/rest/organizations/jenkins/pipelines/blueocean/branches/bug%252FJENKINS-45754-github-enterprise-url-validation-2/runs/2/nodes/35/steps/38/log/
            

            with threasholdInKB=1, we get about 10 lines:

            https://ci.blueocean.io/blue/rest/organizations/jenkins/pipelines/blueocean/branches/bug%252FJENKINS-45754-github-enterprise-url-validation-2/runs/2/nodes/35/steps/38/log/?thresholdInKB=1
            
            ESS [04:52 min]
            [INFO] Events API for Blue Ocean .......................... SUCCESS [ 41.221 s]
            [INFO] Dashboard for Blue Ocean ........................... SUCCESS [03:37 min]
            [INFO] Personalization for Blue Ocean ..................... SUCCESS [02:27 min]
            [INFO] Config API for Blue Ocean .......................... SUCCESS [01:03 min]
            [INFO] GitHub Pipeline for Blue Ocean ..................... SUCCESS [03:47 min]
            [INFO] Git Pipeline for Blue Ocean ........................ SUCCESS [01:08 min]
            [INFO] Bitbucket Pipeline for Blue Ocean .................. SUCCESS [03:20 min]
            [INFO] Blue Ocean ......................................... SUCCESS [ 50.354 s]
            [INFO] ------------------------------------------------------------------------
            [INFO] BUILD SUCCESS
            [INFO] ------------------------------------------------------------------------
            [INFO] Total time: 30:21 min
            [INFO] Finished at: 2017-08-15T20:21:04+00:00
            [INFO] Final Memory: 174M/1599M
            [INFO] ------------------------------------------------------------------------
            
            Show
            vivek Vivek Pandey added a comment - Serving last n number of lines could be expensive, we can end up reading all file. Instead Logging API for step or for run serves last 150KB log data by default. It's controllable using query param thresholdInKB, its default value is 150. Basically it results in to computing offset, `offset=logFileLength-thresholdInKB` in to the log file. For example, this is default rest API called from UI, it fetches last 150KB of log, in terms of line this particular log is about 1900 lines. https: //ci.blueocean.io/blue/ rest /organizations/jenkins/pipelines/blueocean/branches/bug%252FJENKINS-45754-github-enterprise-url-validation-2/runs/2/nodes/35/steps/38/log/ with threasholdInKB=1, we get about 10 lines: https: //ci.blueocean.io/blue/ rest /organizations/jenkins/pipelines/blueocean/branches/bug%252FJENKINS-45754-github-enterprise-url-validation-2/runs/2/nodes/35/steps/38/log/?thresholdInKB=1 ESS [04:52 min] [INFO] Events API for Blue Ocean .......................... SUCCESS [ 41.221 s] [INFO] Dashboard for Blue Ocean ........................... SUCCESS [03:37 min] [INFO] Personalization for Blue Ocean ..................... SUCCESS [02:27 min] [INFO] Config API for Blue Ocean .......................... SUCCESS [01:03 min] [INFO] GitHub Pipeline for Blue Ocean ..................... SUCCESS [03:47 min] [INFO] Git Pipeline for Blue Ocean ........................ SUCCESS [01:08 min] [INFO] Bitbucket Pipeline for Blue Ocean .................. SUCCESS [03:20 min] [INFO] Blue Ocean ......................................... SUCCESS [ 50.354 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 30:21 min [INFO] Finished at: 2017-08-15T20:21:04+00:00 [INFO] Final Memory: 174M/1599M [INFO] ------------------------------------------------------------------------
            Hide
            sophistifunk Josh McDonald added a comment - - edited
            1. What's our main concern here? Reducing the number of lines we see, or network traffic?
            2. Do we want to discard lines at the front-end if we end up fetching more than the threshold because they're short?
            3. Is there a particular benefit to having a different line count for live vs completed steps to justify the complexity? If so, what is it, and do we want to discard 50 lines when the step you're watching is complete?
            4. Do the line numbers in the gutter have any value to the user when we don't have the entire log for that step? Unless we've fetched the entire thing, they're going to be wrong, and will change when the user gets more data, either by requesting more or watching a live log.
            Show
            sophistifunk Josh McDonald added a comment - - edited What's our main concern here? Reducing the number of lines we see, or network traffic? Do we want to discard lines at the front-end if we end up fetching more than the threshold because they're short? Is there a particular benefit to having a different line count for live vs completed steps to justify the complexity? If so, what is it, and do we want to discard 50 lines when the step you're watching is complete? Do the line numbers in the gutter have any value to the user when we don't have the entire log for that step? Unless we've fetched the entire thing, they're going to be wrong, and will change when the user gets more data, either by requesting more or watching a live log.
            Hide
            jamesdumay James Dumay added a comment - - edited

            Good questions

            1. The driver is that are displaying too much useless information on screen. The last 100-150 lines contains the context.
            2. Yes, that would prevent things from jumping around too I suspect.
            3. Lets kill the line numbers. They just don't make any sense.
            4. Ditto. Eventually we should have a timestamp but I suspect that is going to be hard (I suspect core changes, Vivek Pandey?)

            EDIT: lets simplify here. Lets stick with 100 lines for both running and finalized log steps.

            Show
            jamesdumay James Dumay added a comment - - edited Good questions The driver is that are displaying too much useless information on screen. The last 100-150 lines contains the context. Yes, that would prevent things from jumping around too I suspect. Lets kill the line numbers. They just don't make any sense. Ditto. Eventually we should have a timestamp but I suspect that is going to be hard (I suspect core changes, Vivek Pandey ?) EDIT: lets simplify here. Lets stick with 100 lines for both running and finalized log steps.
            Hide
            sophistifunk Josh McDonald added a comment -

            Do we want the users to be able to request "more" log, or just "all" the log if they need more info? My vote is on "all" for implementation simplicity and fewer choices the user has to make, but I thought I should clarify that.

            Show
            sophistifunk Josh McDonald added a comment - Do we want the users to be able to request "more" log, or just "all" the log if they need more info? My vote is on "all" for implementation simplicity and fewer choices the user has to make, but I thought I should clarify that.
            Hide
            jamesdumay James Dumay added a comment -

            Josh McDonald I believe the "all" behaviour is what we have today. I agree, lets keep it that way unless we get a strong signal to change it.

            Show
            jamesdumay James Dumay added a comment - Josh McDonald I believe the "all" behaviour is what we have today. I agree, lets keep it that way unless we get a strong signal to change it.
            Hide
            michaelneale Michael Neale added a comment -

            Josh McDonald any moar progress on this or should we take this out of the queue? 

            Show
            michaelneale Michael Neale added a comment - Josh McDonald any moar progress on this or should we take this out of the queue? 
            Hide
            sophistifunk Josh McDonald added a comment -

            There is some progress, it's bit-rotting in a local branch (slowly though, it's mostly new files) while waiting on some fixes Keith Zantow is sitting on [cough]

            Show
            sophistifunk Josh McDonald added a comment - There is some progress, it's bit-rotting in a local branch (slowly though, it's mostly new files) while waiting on some fixes Keith Zantow is sitting on [cough]
            Hide
            michaelneale Michael Neale added a comment -

            np - just put this one back behind https://issues.jenkins-ci.org/browse/JENKINS-45182

            Show
            michaelneale Michael Neale added a comment - np - just put this one back behind  https://issues.jenkins-ci.org/browse/JENKINS-45182
            Hide
            sophistifunk Josh McDonald added a comment -

            Back onto this now

            Show
            sophistifunk Josh McDonald added a comment - Back onto this now

              People

              • Assignee:
                sophistifunk Josh McDonald
                Reporter:
                jamesdumay James Dumay
              • Votes:
                5 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated: