Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8207029

Unable to use custom SSLEngine with default TrustManagerFactory after updating to JDK 11 b21

    Details

    • Subcomponent:
    • Introduced In Build:
      b20
    • Introduced In Version:
      11
    • Resolved In Build:
      b23
    • CPU:
      generic
    • OS:
      generic
    • Verification:
      Fix failed

      Backports

        Description

        ADDITIONAL SYSTEM INFORMATION :
        # java -version
        java version "11-ea" 2018-09-25
        Java(TM) SE Runtime Environment 18.9 (build 11-ea+21)
        Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11-ea+21, mixed mode)


        A DESCRIPTION OF THE PROBLEM :
        Hi all,

        I just tried to run netty[1] testsuite with the latest jdk11 EA release (21) and saw some class-cast-exception with our custom SSLEngine implementation:


        Caused by: java.lang.ClassCastException: class io.netty.handler.ssl.OpenSslEngine cannot be cast to class sun.security.ssl.SSLEngineImpl (io.netty.handler.ssl.OpenSslEngine is in unnamed module of loader 'app'; sun.security.ssl.SSLEngineImpl is in module java.base of loader 'bootstrap')
        at java.base/sun.security.ssl.SSLAlgorithmConstraints.<init>(SSLAlgorithmConstraints.java:93)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:270)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
        at io.netty.handler.ssl.ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify(ReferenceCountedOpenSslClientContext.java:237)
        at io.netty.handler.ssl.ReferenceCountedOpenSslContext$AbstractCertificateVerifier.verify(ReferenceCountedOpenSslContext.java:621)
        ... 27 more


        This change seems to be related to:
        http://hg.openjdk.java.net/jdk/jdk11/rev/68fa3d4026ea

        I think you miss an instanceof check here in SSLAlgorithmConstraints before try to cast to SSLEngineImpl, as otherwise it will be impossible to use custom implementations of SSLEngine (which we have in netty) with the default TrustManagerFactory.

        REGRESSION : Last worked in version 11

        STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
        Create custom SSLEngine implementation and invoke sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(...) with it as argument.

        EXPECTED VERSUS ACTUAL BEHAVIOR :
        EXPECTED -
        Should not throw ClassCastException, X509TrustManagerImpl should not depend on the fact that SSLEngineImpl is used.
        ACTUAL -
        Throws an ClassCastException like this:

        Caused by: java.lang.ClassCastException: class io.netty.handler.ssl.OpenSslEngine cannot be cast to class sun.security.ssl.SSLEngineImpl (io.netty.handler.ssl.OpenSslEngine is in unnamed module of loader 'app'; sun.security.ssl.SSLEngineImpl is in module java.base of loader 'bootstrap')
        at java.base/sun.security.ssl.SSLAlgorithmConstraints.<init>(SSLAlgorithmConstraints.java:93)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:270)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
        at io.netty.handler.ssl.ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify(ReferenceCountedOpenSslClientContext.java:237)
        at io.netty.handler.ssl.ReferenceCountedOpenSslContext$AbstractCertificateVerifier.verify(ReferenceCountedOpenSslContext.java:621)
        ... 27 more


        CUSTOMER SUBMITTED WORKAROUND :
        There is no workaround, like stated it worked in ea19 last. That said the fix should be as simple as adding an instanceof SSLEngineImpl check before trying to cast.

        FREQUENCY : always


          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  xuelei Xue-Lei Fan
                  Reporter:
                  webbuggrp Webbug Group
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  7 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: