Details

    • Similar Issues:

      Description

      Channel already keeps some rudimentary statistics about class loading (classLoadingCount and so on), which can give insight into slow initial builds. But there are many cases where we suspect that a performance or reliability problem is ultimately due to excessive traffic on the channel, yet cannot prove this except via the crude tool of a network traffic analyzer, and have no visibility into which features (plugins) are responsible and whether the traffic is really necessary.

      Propose that we add some more statistics (counts, mean, median, standard deviation) maintained per Channel for each of the three types of Request/Response.

      • For ProxyInputStream/ProxyOutputStream, record number of pipes opened and closed in each direction, and bytes sent on them. Ideally we would like to get some information about what code opened the stream, though I am not sure how to aggregate this well, since stack traces are too variable.
      • For RPCRequest, record the implementation class name of the proxied object, perhaps the method name/signature being called, the number of calls, their duration, the byte size of the request, the byte size of the response.
      • For UserRequest, record the implementation class name of the Callable (or delegate of a FileCallableWrapper), number of calls, duration, byte sizes of request and response.

      These statistics could be reported in various places:

      • Slave system information page.
      • support-core bundles. (Of course aggregate data across channels should be recorded, so it is still available after slave disconnections.)
      • metrics data.

        Attachments

          Issue Links

            Activity

            jglick Jesse Glick created issue -
            schristou Steven Christou made changes -
            Field Original Value New Value
            Assignee Steven Christou [ schristou ]
            Hide
            jglick Jesse Glick added a comment -

            Wireshark plugin or other detailed diagnostics would be good too.

            Show
            jglick Jesse Glick added a comment - Wireshark plugin or other detailed diagnostics would be good too.
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 161226 ] JNJira + In-Review [ 180618 ]
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            +1 for the Wireshark plugin, but we cannot easily do it with Encrypted protocols like JNLP4. BTW we still can extract some data

            Show
            oleg_nenashev Oleg Nenashev added a comment - +1 for the Wireshark plugin, but we cannot easily do it with Encrypted protocols like JNLP4. BTW we still can extract some data
            jglick Jesse Glick made changes -
            Assignee Steven Christou [ schristou ]
            jglick Jesse Glick made changes -
            Assignee Jesse Glick [ jglick ]
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "remoting PR 204 (Web Link)" [ 17842 ]
            jglick Jesse Glick made changes -
            Component/s core [ 15593 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "core PR 3071 (Web Link)" [ 17843 ]
            Hide
            jglick Jesse Glick added a comment -

            Implemented raw event gathering infrastructure; initially the information is only available as a logger.

            Show
            jglick Jesse Glick added a comment - Implemented raw event gathering infrastructure; initially the information is only available as a logger.
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-45897 [ JENKINS-45897 ]
            Hide
            jglick Jesse Glick added a comment -

            Might also be useful to introduce a utility in jenkins-test-harness which would allow you to assert that a given (little-W) workflow run on an agent transfers at most a given number of packets, at most a given total size, includes only the specified packet types, etc. For example, proof of the effectiveness of JENKINS-38381 could include a test that no packets are being transmitted during the course of a long-running sh step.

            Show
            jglick Jesse Glick added a comment - Might also be useful to introduce a utility in jenkins-test-harness which would allow you to assert that a given (little-W) workflow run on an agent transfers at most a given number of packets, at most a given total size, includes only the specified packet types, etc. For example, proof of the effectiveness of JENKINS-38381 could include a test that no packets are being transmitted during the course of a long-running sh step.
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-38381 [ JENKINS-38381 ]
            jglick Jesse Glick made changes -
            Link This issue relates to HOSTING-464 [ HOSTING-464 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "support-core PR 128 (Web Link)" [ 19424 ]
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            Hide
            jglick Jesse Glick added a comment -

            Have a proposed integration in support-core which seems interesting enough to justify the API. The listener could certainly tally more detailed information, and/or present the results more compellingly, but I wanted to keep the scope small for now; other people may want to refine it now that the plumbing is in place.

            Show
            jglick Jesse Glick added a comment - Have a proposed integration in support-core which seems interesting enough to justify the API. The listener could certainly tally more detailed information, and/or present the results more compellingly, but I wanted to keep the scope small for now; other people may want to refine it now that the plumbing is in place.
            oleg_nenashev Oleg Nenashev made changes -
            Epic Link JENKINS-46154 [ 184393 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "remoting PR 250 (Web Link)" [ 19996 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/FilePath.java
            core/src/main/java/hudson/slaves/SlaveComputer.java
            pom.xml
            test/src/test/java/hudson/bugs/JnlpAccessWithSecuredHudsonTest.java
            test/src/test/java/jenkins/security/Security218Test.java
            test/src/test/java/jenkins/security/s2m/DefaultFilePathFilterTest.java
            http://jenkins-ci.org/commit/jenkins/f4b78921e87c0a265c633674bc3ca3ec0458c19a
            Log:
            JENKINS-27035 Gathering command read/write events (#3071)

            • Slave.JnlpJar.getURL did not work in some modes when core had a snapshot dependency on Remoting.
            • Starting to implement Channel.Listener.read/write.
            • Now obtaining response timing statistics.
            • For now, avoiding timestamped snapshots, as it caused problems for #3120 which I have asked for help from @stephenc diagnosing.
            • Simplified logging a bit.
            • onResponse
            • hudson.FilePath$Mkdirs is a lot more readable than hudson.FilePath$13.
            • Specific snapshot.
            • onJar
            • LoggingChannelListener
            • remoting.version=3.17
            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/slaves/SlaveComputer.java pom.xml test/src/test/java/hudson/bugs/JnlpAccessWithSecuredHudsonTest.java test/src/test/java/jenkins/security/Security218Test.java test/src/test/java/jenkins/security/s2m/DefaultFilePathFilterTest.java http://jenkins-ci.org/commit/jenkins/f4b78921e87c0a265c633674bc3ca3ec0458c19a Log: JENKINS-27035 Gathering command read/write events (#3071) Slave.JnlpJar.getURL did not work in some modes when core had a snapshot dependency on Remoting. Starting to implement Channel.Listener.read/write. Now obtaining response timing statistics. For now, avoiding timestamped snapshots, as it caused problems for #3120 which I have asked for help from @stephenc diagnosing. Simplified logging a bit. onResponse hudson.FilePath$Mkdirs is a lot more readable than hudson.FilePath$13. Specific snapshot. onJar LoggingChannelListener remoting.version=3.17 Making a few test assertions more lenient to adapt to https://github.com/jenkinsci/remoting/pull/247 .
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Merged towards 2.106

            I will update the changelog entries

            Show
            oleg_nenashev Oleg Nenashev added a comment - Merged towards 2.106 I will update the changelog entries
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            It has been released in Remoting 3.17 and Jenkins 2.106

            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited It has been released in Remoting 3.17 and Jenkins 2.106
            oleg_nenashev Oleg Nenashev made changes -
            Status In Review [ 10005 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Owen Wood
            Path:
            Jenkinsfile
            pom.xml
            src/main/java/com/cloudbees/jenkins/support/impl/SlaveCommandStatistics.java
            src/test/java/com/cloudbees/jenkins/support/impl/SlaveCommandStatisticsTest.java
            http://jenkins-ci.org/commit/support-core-plugin/299ed0983c66c315ff99f6c8917599f16d9577c8
            Log:
            Merge pull request #128 from jglick/CommandListener-JENKINS-27035

            JENKINS-27035 Tallying command read/write events

            Compare: https://github.com/jenkinsci/support-core-plugin/compare/5eb81933d6aa...299ed0983c66

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Owen Wood Path: Jenkinsfile pom.xml src/main/java/com/cloudbees/jenkins/support/impl/SlaveCommandStatistics.java src/test/java/com/cloudbees/jenkins/support/impl/SlaveCommandStatisticsTest.java http://jenkins-ci.org/commit/support-core-plugin/299ed0983c66c315ff99f6c8917599f16d9577c8 Log: Merge pull request #128 from jglick/CommandListener- JENKINS-27035 JENKINS-27035 Tallying command read/write events Compare: https://github.com/jenkinsci/support-core-plugin/compare/5eb81933d6aa...299ed0983c66
            escoem Emilio Escobar made changes -
            Remote Link This issue links to "Page (Jenkins Wiki)" [ 20261 ]
            Hide
            escoem Emilio Escobar added a comment -

            Released in support-core:2.46

            Show
            escoem Emilio Escobar added a comment - Released in support-core:2.46
            jglick Jesse Glick made changes -
            Remote Link This issue links to "Page (Jenkins Wiki)" [ 20296 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "remoting PR 287 (Web Link)" [ 21810 ]

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                jglick Jesse Glick
              • Votes:
                2 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: