Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52309

Release Candidate Testing: Run ATH of Jenkins WAR with Java 11 support

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Acceptance criteria

      • Run ATH over 2.164 with all the changes
        • Both on Java 8 and Java 11, so we can compare potential failures

        Attachments

          Issue Links

            Activity

            Hide
            mramonleon Ramon Leon added a comment - - edited

            I've compared the results of the Jenkins CI. One comparison on Jenkins 2.164 and latest ATH, and the other between Jenkins 2.164 and latest ATH and Jenkins 2.156 and ATH with params (latest at that moment).

             All the info is in the comparison spreadsheet, for reference here:

            The comparison has been made between these two executions on Jenkins CI:

            https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/128/testReport/
            Jenkins 2.164 (no need of arguments: enable..., modules), ATH without params:

            https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/124/testReport/
            Jenkins 2.156 (need arguments), ATH with params:

            env JAVA_OPTS="-p /home/ath-user/jdk11-libs/jaxb-api.jar:/home/ath-user/jdk11-libs/javax.activation.jar --add-modules java.xml.bind,java.activation -cp /home/ath-user/jdk11-libs/jaxb-impl.jar:/home/ath-user/jdk11-libs/jaxb-core.jar" JENKINS_OPTS="--enable-future-java" ./run.sh firefox latest -DforkCount=1 -Dmaven.test.failure.ignore=true -B -Dtest=...

            The oldest execution throws 38 differences between Java8 and Java11.

            The latest execution throws 1 difference between Java8 and Java11. And are the same differences between Java11 in this execution and Java8 in the oldest execution.

            The difference is JobDslPluginTest.should_fail_on_missing_plugin

            "Expected: a string containing ""Warning: (script, line 3) plugin 'chucknorris' needs to be installed""
             but: was ""Started by user unknown or anonymous
            Building in workspace /mnt/agent-workspace/workspace/e_acceptance-test-harness_master/target/jenkins11653942047609402639home/workspace/Seed
            Processing provided DSL script
            Warning: (NativeMethodAccessorImpl.java) plugin 'chucknorris' needs to be installed
            Added items:
             GeneratedJob{name='New_Job'}
            Build step 'Process Job DSLs' changed build result to UNSTABLE
            Finished: UNSTABLE"""

            Explanation:

            The test execution is the same on both environments, but the text printed on console is different and the Assert checking the output fails. Is there any difference on the libraries parsing the DSL between Java8 and Java11?

            Assert failing: https://github.com/jenkinsci/acceptance-test-harness/blob/4c989fb384929b4ff019195122966ef694fbabcf/src/test/java/plugins/JobDslPluginTest.java#L473

            Show
            mramonleon Ramon Leon added a comment - - edited I've compared the results of the Jenkins CI. One comparison on Jenkins 2.164 and latest ATH, and the other between Jenkins 2.164 and latest ATH and Jenkins 2.156 and ATH with params (latest at that moment).  All the info is in the comparison spreadsheet , for reference here: The comparison has been made between these two executions on Jenkins CI: https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/128/testReport/ Jenkins 2.164 (no need of arguments: enable..., modules), ATH without params: https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/124/testReport/ Jenkins 2.156 (need arguments), ATH with params: env JAVA_OPTS="-p /home/ath-user/jdk11-libs/jaxb-api.jar:/home/ath-user/jdk11-libs/javax.activation.jar --add-modules java.xml.bind,java.activation -cp /home/ath-user/jdk11-libs/jaxb-impl.jar:/home/ath-user/jdk11-libs/jaxb-core.jar" JENKINS_OPTS="--enable-future-java" ./run.sh firefox latest -DforkCount=1 -Dmaven.test.failure.ignore=true -B -Dtest=... The oldest execution throws 38 differences between Java8 and Java11. The latest execution throws 1 difference between Java8 and Java11. And are the same differences between Java11 in this execution and Java8 in the oldest execution. The difference is JobDslPluginTest.should_fail_on_missing_plugin "Expected: a string containing ""Warning: (script, line 3) plugin 'chucknorris' needs to be installed"" but: was ""Started by user unknown or anonymous Building in workspace /mnt/agent-workspace/workspace/e_acceptance-test-harness_master/target/jenkins11653942047609402639home/workspace/Seed Processing provided DSL script Warning: (NativeMethodAccessorImpl.java) plugin 'chucknorris' needs to be installed Added items: GeneratedJob{name='New_Job'} Build step 'Process Job DSLs' changed build result to UNSTABLE Finished: UNSTABLE""" Explanation : The test execution is the same on both environments, but the text printed on console is different and the Assert checking the output fails. Is there any difference on the libraries parsing the DSL between Java8 and Java11? Assert failing: https://github.com/jenkinsci/acceptance-test-harness/blob/4c989fb384929b4ff019195122966ef694fbabcf/src/test/java/plugins/JobDslPluginTest.java#L473
            Hide
            mramonleon Ramon Leon added a comment -

            I don't understand the blocked link because we don't need the docker image to run the ATH with Jenkins 2.164 and to compare the results.

            If we need to check the results running on the docker image, then we need to have the new docker image.

            Oliver Gondža Oleg Nenashev

            Show
            mramonleon Ramon Leon added a comment - I don't understand the blocked link because we don't need the docker image to run the ATH with Jenkins 2.164 and to compare the results. If we need to check the results running on the docker image, then we need to have the new docker image. Oliver Gondža Oleg Nenashev
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Closing that since Java 11 RC testing has been performed before the 2.164.1 release

            Show
            oleg_nenashev Oleg Nenashev added a comment - Closing that since Java 11 RC testing has been performed before the 2.164.1 release

              People

              • Assignee:
                mramonleon Ramon Leon
                Reporter:
                oleg_nenashev Oleg Nenashev
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: