Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49362

Unable to connect to GitHub using IBM JDK 8 - Unable to find acceptable protocols

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Environment:
    • Similar Issues:

      Description

      Running on IBM z13 mainframe, after installing IBM JDK 8, I am unable to pull the branch source from GitHub.    This is really a critical/major problem, since the only other JDK for the z13 would be OpenJDK, which uses Zero VM (interpreted mode only), so performance is terrible, so bad in fact that the machine runs at 200-300%, and cannot keeps up with long running jobs on the agents. 

      // code placeholderjava.net.UnknownServiceException: Unable to find acceptable protocols. isFallback=false, modes=[ConnectionSpec(cipherSuites=[TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA], tlsVersions=[TLS_1_2, TLS_1_1, TLS_1_0], supportsTlsExtensions=true), ConnectionSpec(cipherSuites=[TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA], tlsVersions=[TLS_1_0], supportsTlsExtensions=true), ConnectionSpec()], supported protocols=[TLSv1]
      	at com.squareup.okhttp.internal.ConnectionSpecSelector.configureSecureSocket(ConnectionSpecSelector.java:73)
      	at com.squareup.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:185)
      	at com.squareup.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:149)
      	at com.squareup.okhttp.internal.io.RealConnection.connect(RealConnection.java:112)
      	at com.squareup.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
      	at com.squareup.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
      	at com.squareup.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
      	at com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:281)
      	at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
      	at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:450)
      	at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:399)
      	at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:527)
      	at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
      	at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:25)
      	at org.kohsuke.github.Requester.parse(Requester.java:602)
      Caused: org.kohsuke.github.HttpException: Server returned HTTP response code: -1, message: 'null' for URL: https://github.ibm.com/api/v3/
      	at org.kohsuke.github.Requester.parse(Requester.java:633)
      	at org.kohsuke.github.Requester.parse(Requester.java:594)
      	at org.kohsuke.github.Requester._to(Requester.java:272)
      	at org.kohsuke.github.Requester.to(Requester.java:234)
      	at org.kohsuke.github.GitHub.checkApiUrlValidity(GitHub.java:703)
      	at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.checkApiUrlValidity(GitHubSCMSource.java:1291)
      Caused: java.io.IOException: It seems https://github.ibm.com/api/v3 is unreachable
      	at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.checkApiUrlValidity(GitHubSCMSource.java:1294)
      	at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1151)
      	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:598)
      	at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrieve(SCMSourceRetriever.java:80)
      	at org.jenkinsci.plugins.workflow.libs.LibraryAdder.retrieve(LibraryAdder.java:153)
      	at org.jenkinsci.plugins.workflow.libs.LibraryAdder.add(LibraryAdder.java:134)
      	at org.jenkinsci.plugins.workflow.libs.LibraryDecorator$1.call(LibraryDecorator.java:125)
      	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1065)
      	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
      	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
      	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
      	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
      	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
      	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:129)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:123)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:517)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:480)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:268)
      	at hudson.model.ResourceController.execute(ResourceController.java:97)
      	at hudson.model.Executor.run(Executor.java:421)
      org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
      WorkflowScript: Loading libraries failed
      
      1 error
      
      	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
      	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
      	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
      	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
      	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
      	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
      	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
      	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:129)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:123)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:517)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:480)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:268)
      	at hudson.model.ResourceController.execute(ResourceController.java:97)
      	at hudson.model.Executor.run(Executor.java:421)
      

      It looks like this issue: https://github.com/watson-developer-cloud/java-sdk/issues/603  Where the a fix may have gone into okhttp..  The issue seems quite old, but it seems the problem lies in that area. 

        Attachments

          Activity

          Hide
          vpaprots Volodymyr Paprotski added a comment - - edited

          (In trying to keep 'same Java runs everywhere' conspiracy/rumours going) I spent much of the day today on this issue and am now past at least this issue. I think.

          As Joe suspected, issue appears to be in okhttp. I believe this was fixed in https://github.com/square/okhttp/issues/3173 okhttp3 3.7 according to the comments. There were quite a few fixes in there, from mismatch of names (TLS_ vs SSL_) to some dangerous reflection code reaching straight into the JSSE provider internals to grab some internal secrets (IBM JCE team runs obfuscators on all their release builds, making reflection possible but impractical)

          Long story short, several jenkins pluggins, that use okhttp need to update their dependencies and/or code.

           


          What follows is a record of my 'experiment' to prove above point. (Hopefully useful to somebody)

          Important note: this is my first foray into Maven, take mvn commands with grain of salt. There are likely better/correct/faster way of doing what I did.

          There are three pluggins involved. I cloned and checked out the version that matches my jenkins install. First two, just update pom.xml to use okhttp3 3.8.1 (only one month newer then 3.7 in issue above). Last one involves code change.

          1. git clone https://github.com/jenkinsci/github-plugin

          2. git clone https://github.com/jenkinsci/github-api-plugin

          3. git clone https://github.com/jenkinsci/github-branch-source-plugin


          ~ Verbatim 

          Github-plugin

          (Might not be actually involved in this issue, but is using okhttp 2.7 so, updated)

          $ git checkout v1.29.0

          $ vim pom.xml:

          <dependency>
          - <groupId>com.squareup.okhttp</groupId>
          + <groupId>com.squareup.okhttp3</groupId>
          {{ <artifactId>okhttp-urlconnection</artifactId>}}
          - <version>2.7.5</version>
          + <version>3.8.1</version>$ mvn hpi:hpi  # mkdir target/classes? 

          # Might I say thanks to github-plugin developers to be the only ones with a usable README.md as their front page! Saved me a lot of mvn grief

          Grab target/*.hpi file and upload it to your jenkins via Manage Pluggins->Advanced

          Github-api-plugin

          Curiously, kohsuke of CloudBees Inc. already is using okhttp 3.9 https://github.com/kohsuke/github-api/blob/master/pom.xml#L157

          $ git checkout github-api-1.90

          $ vim pom.xml

          {{ <groupId>org.kohsuke</groupId>}}
          {{ <artifactId>github-api</artifactId>}}
          - <version>1.90</version>
          + <version>1.93</version>

          ...{{ }}

          <dependency>
          - <groupId>com.squareup.okhttp</groupId>
          + <groupId>com.squareup.okhttp3</groupId>
          {{ <artifactId>okhttp-urlconnection</artifactId>}}
          - <version>2.7.5</version>
          + <version>3.8.1</version>

          $ mvn hpi:hpi

          $ mvn install -DskipTests # install locally, needed by github branch-source

          Grab target/*.hpi file and upload it to your jenkins via Manage Pluggins->Advanced

          Github-branch-source-plugin

          $ git checkout github-branch-source-2.3.4

          $ vim pom.xml # resolving build rule failure by explicitly specifying newer package at root

          + <dependency>
          + <groupId>org.apache.commons</groupId>
          + <artifactId>commons-lang3</artifactId>
          + <version>3.7</version>
          + </dependency>

          $ vim src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java

          -import com.squareup.okhttp.OkHttpClient;
          -import com.squareup.okhttp.OkUrlFactory;
          +import okhttp3.OkHttpClient;
          +import okhttp3.OkUrlFactory;

          ...

          -import org.kohsuke.github.extras.OkHttpConnector;
          +import org.kohsuke.github.extras.OkHttp3Connector;

          ...

          - OkHttpClient client = new OkHttpClient().setProxy(getProxy(host));
          + OkHttpClient client = new OkHttpClient.Builder().proxy(getProxy(host)).build();

          - gb.withConnector(new OkHttpConnector(new OkUrlFactory(client)));
          + gb.withConnector(new OkHttp3Connector(new OkUrlFactory(client)));

          $ mvn compile

          $ mvn hpi:hpi

          Grab target/*.hpi file and upload it to your jenkins via Manage Pluggins->Advanced

          Now, I am able to index my Github Enterprise branches!

          PS: Jenkins was started with ./ibm-java-s390x-80/bin/java -Dcom.ibm.jsse2.overrideDefaultProtocol=TLSv12 -Djavax.net.debug=ssl,handshake,data,verbose, -Djava.library.path=/usr/lib/jni -Dcom.ibm.jsse2.overrideDefaultTLS=true  -jar jenkins.war

          I haven't bothered removing any of the tracing and override options as of yet to see if they have any effect on the fix.

          Show
          vpaprots Volodymyr Paprotski added a comment - - edited (In trying to keep 'same Java runs everywhere' conspiracy/rumours going) I spent much of the day today on this issue and am now past at least this issue. I think. As Joe suspected, issue appears to be in okhttp. I believe this was fixed in https://github.com/square/okhttp/issues/3173  okhttp3 3.7 according to the comments. There were quite a few fixes in there, from mismatch of names (TLS_ vs SSL_) to some dangerous reflection code reaching straight into the JSSE provider internals to grab some internal secrets (IBM JCE team runs obfuscators on all their release builds, making reflection possible but impractical) Long story short, several jenkins pluggins, that use okhttp need to update their dependencies and/or code.   What follows is a record of my 'experiment' to prove above point. (Hopefully useful to somebody) Important note: this is my first foray into Maven, take mvn commands with grain of salt. There are likely better/correct/faster way of doing what I did. There are three pluggins involved. I cloned and checked out the version that matches my jenkins install. First two, just update pom.xml to use okhttp3 3.8.1 (only one month newer then 3.7 in issue above). Last one involves code change. 1. git clone https://github.com/jenkinsci/github-plugin 2. git clone https://github.com/jenkinsci/github-api-plugin 3. git clone https://github.com/jenkinsci/github-branch-source-plugin ~ Verbatim  Github-plugin (Might not be actually involved in this issue, but is using okhttp 2.7 so, updated) $ git checkout v1.29.0 $ vim pom.xml: <dependency> - <groupId>com.squareup.okhttp</groupId> + <groupId>com.squareup.okhttp3</groupId> {{ <artifactId>okhttp-urlconnection</artifactId>}} - <version>2.7.5</version> + <version>3.8.1</version> $ mvn hpi:hpi  # mkdir target/classes?  # Might I say thanks to github-plugin developers to be the only ones with a usable README.md as their front page! Saved me a lot of mvn grief Grab target/*.hpi file and upload it to your jenkins via Manage Pluggins->Advanced Github-api-plugin Curiously, kohsuke of CloudBees Inc. already is using okhttp 3.9 https://github.com/kohsuke/github-api/blob/master/pom.xml#L157 $ git checkout github-api-1.90 $ vim pom.xml {{ <groupId>org.kohsuke</groupId>}} {{ <artifactId>github-api</artifactId>}} - <version>1.90</version> + <version>1.93</version> ... {{ }} <dependency> - <groupId>com.squareup.okhttp</groupId> + <groupId>com.squareup.okhttp3</groupId> {{ <artifactId>okhttp-urlconnection</artifactId>}} - <version>2.7.5</version> + <version>3.8.1</version> $ mvn hpi:hpi $ mvn install -DskipTests # install locally, needed by github branch-source Grab target/*.hpi file and upload it to your jenkins via Manage Pluggins->Advanced Github-branch-source-plugin $ git checkout github-branch-source-2.3.4 $ vim pom.xml # resolving build rule failure by explicitly specifying newer package at root + <dependency> + <groupId>org.apache.commons</groupId> + <artifactId>commons-lang3</artifactId> + <version>3.7</version> + </dependency> $ vim src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java -import com.squareup.okhttp.OkHttpClient; -import com.squareup.okhttp.OkUrlFactory; +import okhttp3.OkHttpClient; +import okhttp3.OkUrlFactory; ... -import org.kohsuke.github.extras.OkHttpConnector; +import org.kohsuke.github.extras.OkHttp3Connector; ... - OkHttpClient client = new OkHttpClient().setProxy(getProxy(host)); + OkHttpClient client = new OkHttpClient.Builder().proxy(getProxy(host)).build(); - gb.withConnector(new OkHttpConnector(new OkUrlFactory(client))); + gb.withConnector(new OkHttp3Connector(new OkUrlFactory(client))); $ mvn compile $ mvn hpi:hpi Grab target/*.hpi file and upload it to your jenkins via Manage Pluggins->Advanced Now, I am able to index my Github Enterprise branches! PS: Jenkins was started with ./ibm-java-s390x-80/bin/java -Dcom.ibm.jsse2.overrideDefaultProtocol=TLSv12 -Djavax.net.debug=ssl,handshake,data,verbose, -Djava.library.path=/usr/lib/jni -Dcom.ibm.jsse2.overrideDefaultTLS=true  -jar jenkins.war I haven't bothered removing any of the tracing and override options as of yet to see if they have any effect on the fix.

            People

            • Assignee:
              Unassigned
              Reporter:
              sj98ta Joe Fowler
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: