Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8223940

TLSv1.2 client auth: private key not supported by chosen signature algorithm

    Details

      Description

      A DESCRIPTION OF THE PROBLEM :
      When using a PKCS11 keystore (using the SunPKCS11 Provider) for client authentication in TLSv1.2, the client chooses a signature algorithm which does not support the P11PrivateKey.

      The server sends a list of supported SignatureSchemes for use in client authentication. The method sun.security.ssl.SignatureScheme.getPreferableAlgorithm(List<SignatureScheme>, PrivateKey, ProtocolVersion) chooses which SignatureScheme to use in client authentication. It can choose any algorithm for which a provider exists, even if the algorithm's java.security.Provider$Service.supportsParameter(privateKey) == false. In this case it chooses RSASSA-PSS from the provider sun.security.rsa.SunRsaSign.

      Because of this, signing with P11PrivateKey fails (with a misleading message stating no providers support the key, when it's a question of the <algorithm name> implementations not supporting the key):

      Caused by: javax.net.ssl.SSLHandshakeException: Cannot produce CertificateVerify signature
      at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:128)
      at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
      at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
      at java.base/sun.security.ssl.CertificateVerify$T12CertificateVerifyMessage.<init>(CertificateVerify.java:590)
      at java.base/sun.security.ssl.CertificateVerify$T12CertificateVerifyProducer.produce(CertificateVerify.java:740)
      at java.base/sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:436)
      at java.base/sun.security.ssl.ServerHelloDone$ServerHelloDoneConsumer.consume(ServerHelloDone.java:173)
      at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
      at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
      at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
      at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:178)
      at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
      at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152)
      at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063)
      at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
      at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
      at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
      at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1356)
      at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1331)
      at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:241)
      <snip>
      Caused by: java.security.InvalidKeyException: No installed provider supports this key: sun.security.pkcs11.P11Key$P11PrivateKey
      at java.base/java.security.Signature$Delegate.chooseProvider(Signature.java:1163)
      at java.base/java.security.Signature$Delegate.engineInitSign(Signature.java:1204)
      at java.base/java.security.Signature.initSign(Signature.java:546)
      at java.base/sun.security.ssl.SignatureScheme.getSignature(SignatureScheme.java:473)
      at java.base/sun.security.ssl.CertificateVerify$T12CertificateVerifyMessage.<init>(CertificateVerify.java:580)
      ... 26 more

      The problem cannot be worked around by adding RSASSA-PSS to the java.security configuration jdk.tls.disabledAlgorithms property.

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Regular Java TLS client using PKCS11 KeyStore from the SunPKCS11 provider for client authentication.

      TLSv1.2 server with client authentication, with supported_signature_algorithms preferring RSASSA-PSS (not supported by PKCS11 provider) over RSA_PKCS1_SHA256 (supported by PKCS11 provider).

      Open connection from client to server.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      The client chooses the RSA_PKCS1_SHA256 signature algorithm. The TLS connection is established with PKCS11 for client authentication.
      ACTUAL -
      The client chooses the RSASSA-PSS signature algorithm. The client cannot complete TLS handshake because it cannot use the P11PrivateKey in the signature algorithm.

      CUSTOMER SUBMITTED WORKAROUND :
      Configure server so that supported_signature_algorithms prefers signature algorithms supported by the SunPKCS11 provider (RSA_PKCS1_SHA256, RSA_PKCS1_SHA384, RSA_PKCS1_SHA_512, RSA_SHA224, RSA_PKCS1_SHA1).

      FREQUENCY : always


        Attachments

          Activity

            People

            • Assignee:
              valeriep Valerie Peng
              Reporter:
              webbuggrp Webbug Group
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: