Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41827

JenkinsRule mode to use realistic class loading

    Details

    • Similar Issues:

      Description

      JenkinsRule-based tests generally load not only Jenkins core but also all plugins from ${java.class.path}, i.e., the Maven test-scoped classpath. This is convenient in some ways but also means that class loading is done in a very different way than in an actual production instance, or even mvn hpi:run, making it very hard to test changes like JENKINS-26192 without using the much clunkier, slower, and flakier acceptance-test-harness. It also means that we are forever fighting with Maven's weird transitive dependency management (JENKINS-41631, JENKINS-39301, etc.).

      There should be an option to run JenkinsRule in a mode whereby the test classpath is only used to load Jetty; Jenkins core and its WEB-INF/lib/*.jar (including modules!) is loaded in a derivative class loader; and then plugins are discovered from the test classpath, transitive dependencies computed and verified, and finally loaded in the real plugin class loader (probably using *.jpl files to allow us to still load from unpacked development trees). Finally, the test suite itself needs to be reloaded in a fresh loader parented to UberClassLoader, and executed from there.

      NbModuleSuite demonstrates the concept, though the details would be rather different for Jenkins.

      There are a lot of details to be checked, such as

      • @TestExtension handling
      • PluginWorkspaceMap support for coördinated snapshot development
      • moving InjectedTest from HudsonTestCase so that it can use the new mode

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            More ambitiously, there could be an option to start Jenkins and run the tests in another JVM, in a Docker container running an image specified in the test case. This would allow tests to reproduce issues specific to certain on-master operations, such as native executables used for SCM caching, which docker-fixtures cannot simulate. On the other hand perhaps this indicates that the feature in question needs to be rewritten to use an agent! Another use case is bugs specific to a certain JVM.

            Show
            jglick Jesse Glick added a comment - More ambitiously, there could be an option to start Jenkins and run the tests in another JVM, in a Docker container running an image specified in the test case. This would allow tests to reproduce issues specific to certain on-master operations, such as native executables used for SCM caching, which docker-fixtures cannot simulate. On the other hand perhaps this indicates that the feature in question needs to be rewritten to use an agent! Another use case is bugs specific to a certain JVM.
            Hide
            jglick Jesse Glick added a comment -

            Another use case for out-of-process testing is to reproduce bugs specific to a non-Jetty container, such as JENKINS-31068.

            Show
            jglick Jesse Glick added a comment - Another use case for out-of-process testing is to reproduce bugs specific to a non-Jetty container, such as JENKINS-31068 .
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/test/java/hudson/plugins/mercurial/MercurialSCMSource2Test.java
            http://jenkins-ci.org/commit/mercurial-plugin/8a429bf3d682c73b451697e4bf7cb8f35b8b9efa
            Log:
            Test fails on CI when it should merely be skipped (pending JENKINS-41827 rewrite to keep master in a container).

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/test/java/hudson/plugins/mercurial/MercurialSCMSource2Test.java http://jenkins-ci.org/commit/mercurial-plugin/8a429bf3d682c73b451697e4bf7cb8f35b8b9efa Log: Test fails on CI when it should merely be skipped (pending JENKINS-41827 rewrite to keep master in a container).
            Hide
            jglick Jesse Glick added a comment -

            Would make reproducing things like JENKINS-46754 easier.

            Show
            jglick Jesse Glick added a comment - Would make reproducing things like  JENKINS-46754 easier.
            Hide
            jglick Jesse Glick added a comment -

            In discussion with Sam Van Oort we realized this could be useful for testing durability guarantees & recovery scenarios in Pipeline, for example if there is a SIGKILL delivered to the master at a specific point in the build.

            Show
            jglick Jesse Glick added a comment - In discussion with Sam Van Oort we realized this could be useful for testing durability guarantees & recovery scenarios in Pipeline, for example if there is a SIGKILL delivered to the master at a specific point in the build.
            Hide
            jglick Jesse Glick added a comment -

            Durability tests turned out to be feasible with a simple addition to RestartableJenkinsRule.

            Show
            jglick Jesse Glick added a comment - Durability tests turned out to be feasible with a simple addition to RestartableJenkinsRule .
            Hide
            jvz Matt Sicker added a comment -

            Perhaps consider Arquillian as a framework for that?

            Show
            jvz Matt Sicker added a comment - Perhaps consider Arquillian as a framework for that?
            Hide
            jglick Jesse Glick added a comment -

            I suspect Arquillian would be much more trouble than it is worth, since Jenkins uses only a little bit of the Servlet system and then has its own big complex module system completely outside of that.

            Show
            jglick Jesse Glick added a comment - I suspect Arquillian would be much more trouble than it is worth, since Jenkins uses only a little bit of the Servlet system and then has its own big complex module system completely outside of that.
            Hide
            drulli Ulli Hafner added a comment - - edited

            Can this issue be also the root case for my failing tests? I moved the JS files of my warnings plugin into separate plugins to simplify the reuse in other plugins. Now all UI tests fail since the JS files are not found any more: example failure - 404. I get the 404 for all JS scripts that are part of one of my dependencies (the JS files in my plugin seem to work). Everything works fine in a real Jenkins instance.

            java.io.IOException: Unable to download JavaScript from 'http://localhost:39309/plugin/data-tables-api/js/table.js' (status 404).
            	at com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1054)
            	at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:982)
            	at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:362)
            	at com.gargoylesoftware.htmlunit.html.HtmlScript$2.execute(HtmlScript.java:236)
            	at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:257)
            	at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:792)
            
            Show
            drulli Ulli Hafner added a comment - - edited Can this issue be also the root case for my failing tests? I moved the JS files of my warnings plugin into separate plugins to simplify the reuse in other plugins. Now all UI tests fail since the JS files are not found any more: example failure - 404 . I get the 404 for all JS scripts that are part of one of my dependencies (the JS files in my plugin seem to work). Everything works fine in a real Jenkins instance. java.io.IOException: Unable to download JavaScript from 'http://localhost:39309/plugin/data-tables-api/js/table.js' (status 404). at com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1054) at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:982) at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:362) at com.gargoylesoftware.htmlunit.html.HtmlScript$2.execute(HtmlScript.java:236) at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:257) at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:792)

              People

              • Assignee:
                Unassigned
                Reporter:
                jglick Jesse Glick
              • Votes:
                4 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated: