Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41827

JenkinsRule mode to use realistic class loading

    Details

    • Similar Issues:

      Description

      JenkinsRule-based tests generally load not only Jenkins core but also all plugins from ${java.class.path}, i.e., the Maven test-scoped classpath. This is convenient in some ways but also means that class loading is done in a very different way than in an actual production instance, or even mvn hpi:run, making it very hard to test changes like JENKINS-26192 without using the much clunkier, slower, and flakier acceptance-test-harness. It also means that we are forever fighting with Maven's weird transitive dependency management (JENKINS-41631, JENKINS-39301, etc.).

      There should be an option to run JenkinsRule in a mode whereby the test classpath is only used to load Jetty; Jenkins core and its WEB-INF/lib/*.jar (including modules!) is loaded in a derivative class loader; and then plugins are discovered from the test classpath, transitive dependencies computed and verified, and finally loaded in the real plugin class loader (probably using *.jpl files to allow us to still load from unpacked development trees). Finally, the test suite itself needs to be reloaded in a fresh loader parented to UberClassLoader, and executed from there.

      NbModuleSuite demonstrates the concept, though the details would be rather different for Jenkins.

      There are a lot of details to be checked, such as

      • @TestExtension handling
      • PluginWorkspaceMap support for coördinated snapshot development
      • moving InjectedTest from HudsonTestCase so that it can use the new mode

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            More ambitiously, there could be an option to start Jenkins and run the tests in another JVM, in a Docker container running an image specified in the test case. This would allow tests to reproduce issues specific to certain on-master operations, such as native executables used for SCM caching, which docker-fixtures cannot simulate. On the other hand perhaps this indicates that the feature in question needs to be rewritten to use an agent! Another use case is bugs specific to a certain JVM.

            Show
            jglick Jesse Glick added a comment - More ambitiously, there could be an option to start Jenkins and run the tests in another JVM, in a Docker container running an image specified in the test case. This would allow tests to reproduce issues specific to certain on-master operations, such as native executables used for SCM caching, which docker-fixtures cannot simulate. On the other hand perhaps this indicates that the feature in question needs to be rewritten to use an agent! Another use case is bugs specific to a certain JVM.
            Hide
            jglick Jesse Glick added a comment -

            Another use case for out-of-process testing is to reproduce bugs specific to a non-Jetty container, such as JENKINS-31068.

            Show
            jglick Jesse Glick added a comment - Another use case for out-of-process testing is to reproduce bugs specific to a non-Jetty container, such as JENKINS-31068 .
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/test/java/hudson/plugins/mercurial/MercurialSCMSource2Test.java
            http://jenkins-ci.org/commit/mercurial-plugin/8a429bf3d682c73b451697e4bf7cb8f35b8b9efa
            Log:
            Test fails on CI when it should merely be skipped (pending JENKINS-41827 rewrite to keep master in a container).

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/test/java/hudson/plugins/mercurial/MercurialSCMSource2Test.java http://jenkins-ci.org/commit/mercurial-plugin/8a429bf3d682c73b451697e4bf7cb8f35b8b9efa Log: Test fails on CI when it should merely be skipped (pending JENKINS-41827 rewrite to keep master in a container).
            Hide
            jglick Jesse Glick added a comment -

            Would make reproducing things like JENKINS-46754 easier.

            Show
            jglick Jesse Glick added a comment - Would make reproducing things like  JENKINS-46754 easier.
            Hide
            jglick Jesse Glick added a comment -

            In discussion with Sam Van Oort we realized this could be useful for testing durability guarantees & recovery scenarios in Pipeline, for example if there is a SIGKILL delivered to the master at a specific point in the build.

            Show
            jglick Jesse Glick added a comment - In discussion with Sam Van Oort we realized this could be useful for testing durability guarantees & recovery scenarios in Pipeline, for example if there is a SIGKILL delivered to the master at a specific point in the build.
            Hide
            jglick Jesse Glick added a comment -

            Durability tests turned out to be feasible with a simple addition to RestartableJenkinsRule.

            Show
            jglick Jesse Glick added a comment - Durability tests turned out to be feasible with a simple addition to RestartableJenkinsRule .
            Hide
            jvz Matt Sicker added a comment -

            Perhaps consider Arquillian as a framework for that?

            Show
            jvz Matt Sicker added a comment - Perhaps consider Arquillian as a framework for that?
            Hide
            jglick Jesse Glick added a comment -

            I suspect Arquillian would be much more trouble than it is worth, since Jenkins uses only a little bit of the Servlet system and then has its own big complex module system completely outside of that.

            Show
            jglick Jesse Glick added a comment - I suspect Arquillian would be much more trouble than it is worth, since Jenkins uses only a little bit of the Servlet system and then has its own big complex module system completely outside of that.

              People

              • Assignee:
                Unassigned
                Reporter:
                jglick Jesse Glick
              • Votes:
                4 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated: