Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8180583

Specifying maxdepth for registryFilter added in Java8u121 seems to have no effect

    Details

    • Subcomponent:
    • Introduced In Version:
    • CPU:
      x86_64
    • OS:
      generic

      Description

      FULL PRODUCT VERSION :
      java -version'の結果
      java version "1.8.0_131"
      Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
      Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)


      ADDITIONAL OS VERSION INFORMATION :
      Linux xxx 2.6.32-641.11.1.el6.x86_64 #1 SMP Wed Oct 26 10:25:23 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux


      EXTRA RELEVANT SYSTEM CONFIGURATION :
      Introduction.

      Our system uses RMI, and some RMI stubs include original classes. (* 1)
      After updating to Java8u121, this class was rejected to bind to rmiregistry as follows by registryFilter that is added .

      java.io.ObjectInputStream filterCheck
      INFO: ObjectInputFilter REJECTED: class <Target Class>, array length: -1, nRefs: 8, depth: 2, bytes: 507, ex: n/a

      Therefore, in Java8u121, we could avoid this issue by setting registryFilter in the java.security file as follows.

      sun.rmi.registry.registryFilter=<Target Class>

      (*1) It uses the original InvocationHandler class which is not the java.rmi.server.RemoteObjectInvocationHandler class.

      <Note>
      The binding of another RMI stub that does not contain an original InvocationHandler class is not rejected.


      A DESCRIPTION OF THE PROBLEM :
      <Problem contents>
      After updating to Java8u131, binding was rejected by registryFilter as follows when RMI stub containing classes that need to be set in registryFilter was bound to rmiregisty.

      java.io.ObjectInputStream filterCheck
      INFO: ObjectInputFilter REJECTED: null, array length: -1, nRefs: 16, depth: 6, bytes: 692, ex: n/a

      Based on logs, we supposed that it was rejected because depth exceeded the maximum value of 5.
      Therefore, based on JEP 290 (*1), the maxdepth value was added as 7 to the registryFilter.
      Below setting image to java.security file.

      sun.rmi.registry.registryFilter=<Target Class>;\
      maxdepth=7

      However, despite specifying maxdepth as 7, it was REJECTed at depth: 6, and it appeared to be ineffective.
      There is the our initial investigation result.

      <Our primary view>
      There is the our initial investigation result.
      Based on the source code, in the RegistryImpl class (*2), maxdepth of registryFilter is defined as 5 (*3) as a fixed value, so specifying maxdepth of registryFilter seems to have no effect in the java.security file.

      (*1) : <http://openjdk.java.net/jeps/290>
      (*2) : src/share/classes/sun/rmi/registry/RegistryImpl.java
       <http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/75f31e0bd829/src/share/classes/sun/rmi/registry/RegistryImpl.java>
      (*3) : There are two point on RegistryImpl.java
       Line 99: private static int REGISTRY_MAX_DEPTH = 5;
       Line 397: if (filterInfo.depth() > REGISTRY_MAX_DEPTH) {

      <Related issue>
      After updating to Java8u131, the bind to rmiregistry is rejected by registryFilter even though registryFilter is set


      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      In Java8u131, bind an RMI stub that requires setting registryFilter to rmiregistry.


      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      If maxdepth is specified for registryFilter, maxdepth is checked with the specified value instead of 5.

      ACTUAL -
      Although maxdepth is specified for registryFilter, it seems that maxdepth is checked with a fixed value as 5.


      ERROR MESSAGES/STACK TRACES THAT OCCUR :
      <rmiregistry log (standard output)>
      java.io.ObjectInputStream filterCheck
      INFO: ObjectInputFilter REJECTED: null, array length: -1, nRefs: 16, depth: 6, bytes: 692, ex: n/a

      <Part of stack trace>
      Caused by: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
       java.io.InvalidClassException: filter status: REJECTED
       at sun.rmi.registry.RegistryImpl_Skel.dispatch(Unknown Source) [rt.jar:1.8.0_131]
       at sun.rmi.server.UnicastServerRef.oldDispatch(UnicastServerRef.java:450) [rt.jar:1.8.0_131]
       at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:294) [rt.jar:1.8.0_131]
       at sun.rmi.transport.Transport$1.run(Transport.java:200) [rt.jar:1.8.0_131]
       at sun.rmi.transport.Transport$1.run(Transport.java:197) [rt.jar:1.8.0_131]
       at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.8.0_131]
       at sun.rmi.transport.Transport.serviceCall(Transport.java:196) [rt.jar:1.8.0_131]
       at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) [rt.jar:1.8.0_131]
       at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) [rt.jar:1.8.0_131]
       at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) [rt.jar:1.8.0_131]
       at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.8.0_131]
       at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) [rt.jar:1.8.0_131]
       ... 3 more
      Caused by: java.io.InvalidClassException: filter status: REJECTED
       at java.io.ObjectInputStream.filterCheck(ObjectInputStream.java:1244) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readHandle(ObjectInputStream.java:1664) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1515) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) [rt.jar:1.8.0_131]
       at sun.rmi.server.MarshalInputStream.readLocation(MarshalInputStream.java:313) [rt.jar:1.8.0_131]
       at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:189) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) [rt.jar:1.8.0_131]
       at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) [rt.jar:1.8.0_131]
       ... 15 more


      REPRODUCIBILITY :
      This bug can be reproduced always.

      ---------- BEGIN SOURCE ----------
      Unfortunately we can not submit it now.
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      Do not update to Java8u131.
      Set registryFilter in Java8u121.


        Issue Links

          Activity

          Hide
          rriggs Roger Riggs added a comment - - edited
          In the configurable RMI filter (sun.rmi.registry.registryFilter), limits can only be reduced to REJECT additional cases.
          If the configurable filter does not reject, then the built-in filter is applied.

          Both the javadoc for ObjectInputFilter.createFilter and JEP 290 make this point as:
          "If any of the limits are exceeded, the filter returns Status.REJECTED."

          The built-in depth of 5 was based on expecting simple graphs to be bound.
          The depth can be raised significantly.
          Show
          rriggs Roger Riggs added a comment - - edited In the configurable RMI filter (sun.rmi.registry.registryFilter), limits can only be reduced to REJECT additional cases. If the configurable filter does not reject, then the built-in filter is applied. Both the javadoc for ObjectInputFilter.createFilter and JEP 290 make this point as: "If any of the limits are exceeded, the filter returns Status.REJECTED." The built-in depth of 5 was based on expecting simple graphs to be bound. The depth can be raised significantly.
          Hide
          rriggs Roger Riggs added a comment -
          As noted, the configurable filter can reduce limits not increase them.
          The related issue 8180582 should increase the limits of the built-in registry filter.
          Show
          rriggs Roger Riggs added a comment - As noted, the configurable filter can reduce limits not increase them. The related issue 8180582 should increase the limits of the built-in registry filter.

            People

            • Assignee:
              rriggs Roger Riggs
              Reporter:
              webbuggrp Webbug Group
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: