Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8136353

ClassValue preventing class unloading

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P3
    • Resolution: Not an Issue
    • Affects Version/s: 8, 9
    • Fix Version/s: 9
    • Component/s: core-libs
    • Subcomponent:
    • CPU:
      x86_64
    • OS:
      linux_ubuntu

      Description

      FULL PRODUCT VERSION :
      any JDK supporting ClassValue

      FULL OS VERSION :
      Linux 3.13.0-29-generic #53~precise1-Ubuntu SMP Wed Jun 4 22:06:25 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

      A DESCRIPTION OF THE PROBLEM :
      ClassValue internal structures seem to prevent the garbage collection of classes, if those keep a reference to the ClassValue instance. I have tested these against several JDK 7, 8 and 9 versions, including for example JDK9 b78. And all of them fail to unload the classes from the supplied jar. It does not matter what class is used to attach a value to (my test uses Integer.TYPE ), nor does it matter if the field is static or not. If the field is changed to use a WeakReferece for example, garbage collection will work.

      This bug is quite problematic for Gradle an Groovy

      THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Yes

      THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      I run the provided CVTest program with very low memory settings for total memory/permgen. I tested with 4m and got a OOME afer less then 100 iterations. For this there has to be a "t/t.jar" with the test classes Dummy and MyClassValue.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      Since the controlling program CVTest does not keep any references to the class from t.jar or the ClassValue, all of this must be due for garbage collection at some point. Instead an OOME is thrown.
      REPRODUCIBILITY :
      This bug can be reproduced always.

      ---------- BEGIN SOURCE ----------

      public class CVTest {

          public static void main(String[] args) throws Throwable {
              for (long i = 0; i<10000000; i++) {
                  File dir = new File("t/t.jar");
                  URLClassLoader classLoader = new URLClassLoader(new URL[]{dir.toURI().toURL()});
                  ClassValue cv = (ClassValue) classLoader.loadClass("MyClassValue").newInstance();
                  Object value = cv.get(Integer.TYPE);
                  assert value !=null;
                  assert value.getClass().getClassLoader() == classLoader;
                  classLoader.close();
              }

          }
      }

      And for "t/t.jar":

      public class MyClassValue extends ClassValue {
          protected Object computeValue(Class type) {
              Dummy ret = new Dummy();
              Other.o = this;
              return ret;
          }
      }

      class Dummy {
        static Object o;
      }


      ---------- END SOURCE ----------

      1. CVTest.java
        0.7 kB
        Balchandra Vaidya
      2. MyClassValue.java
        0.3 kB
        Balchandra Vaidya

        Issue Links

          Activity

          Hide
          stefank Stefan Karlsson added a comment - - edited
          It's not obvious that this is a GC weak reference problem. While running the repro, the Full GCs don't clear the weak references, so it seems more likely a problem with the reproducer or the ClassValue.

          core-libs, could you take a look at the reproducer and the ClassValue class and figure out if this is the expected behavior or not.

          Show
          stefank Stefan Karlsson added a comment - - edited It's not obvious that this is a GC weak reference problem. While running the repro, the Full GCs don't clear the weak references, so it seems more likely a problem with the reproducer or the ClassValue. core-libs, could you take a look at the reproducer and the ClassValue class and figure out if this is the expected behavior or not.
          Hide
          stefank Stefan Karlsson added a comment -
          I just realized that the problem is probably that MyClassValue is strongly reachable from Dummy.

          We have the following reference chain:
          Dummy instance -(via class pointer)-> Dummy.class -(via Dummy.o static)-> MyClassValue instance -(via ClassValue.identity field)-> ClassValue$Identity instance

          The ClassValue class uses a WeakHashmap to semantically map between classes and values. Instances of ClassValue$Identity are used as 'keys' and the Dummy instances are used as the 'values'. Since the 'values' in WeakHashMap are held strongly, the keys are also strongly reachable and will not be cleared.
          Show
          stefank Stefan Karlsson added a comment - I just realized that the problem is probably that MyClassValue is strongly reachable from Dummy. We have the following reference chain: Dummy instance -(via class pointer)-> Dummy.class -(via Dummy.o static)-> MyClassValue instance -(via ClassValue.identity field)-> ClassValue$Identity instance The ClassValue class uses a WeakHashmap to semantically map between classes and values. Instances of ClassValue$Identity are used as 'keys' and the Dummy instances are used as the 'values'. Since the 'values' in WeakHashMap are held strongly, the keys are also strongly reachable and will not be cleared.
          Hide
          stefank Stefan Karlsson added a comment -
          From Jesse Glick:
          "While investigating a leak of `ClassLoader`s in Groovy 2.x due to the
          default behavior of `GroovyClassValueFactory`, I ran into
          https://bugs.openjdk.java.net/browse/JDK-8136353. Offhand that does
          not look like a bug to me (rather a mistake in Groovy), though my
          longstanding RFE https://bugs.openjdk.java.net/browse/JDK-6389107
          would probably allow it to be fixed more easily."
          Show
          stefank Stefan Karlsson added a comment - From Jesse Glick: "While investigating a leak of `ClassLoader`s in Groovy 2.x due to the default behavior of `GroovyClassValueFactory`, I ran into https://bugs.openjdk.java.net/browse/JDK-8136353 . Offhand that does not look like a bug to me (rather a mistake in Groovy), though my longstanding RFE https://bugs.openjdk.java.net/browse/JDK-6389107 would probably allow it to be fixed more easily."
          Hide
          psandoz Paul Sandoz added a comment -
          AFAICT Groovy fixed the problem:

            https://issues.apache.org/jira/browse/GROOVY-7683
            https://github.com/apache/groovy/commit/a8fb776023253ebc2da35538f25eccd7d59997ed

          Groovy's ClassValue implementation computed instances of ClassInfo that held a strong reference to the class associated with the computed value.

          The commit, linked to above, modified things such that ClassInfo now holds a weak reference to the class associated with the computed value.

          Groovy, when computing associated values, also added instances of ClassInfo to a "global" (static) set. Thus strong refs to computed values were retained for the life time of Groovy's ClassInfo class.
          Show
          psandoz Paul Sandoz added a comment - AFAICT Groovy fixed the problem:    https://issues.apache.org/jira/browse/GROOVY-7683    https://github.com/apache/groovy/commit/a8fb776023253ebc2da35538f25eccd7d59997ed Groovy's ClassValue implementation computed instances of ClassInfo that held a strong reference to the class associated with the computed value. The commit, linked to above, modified things such that ClassInfo now holds a weak reference to the class associated with the computed value. Groovy, when computing associated values, also added instances of ClassInfo to a "global" (static) set. Thus strong refs to computed values were retained for the life time of Groovy's ClassInfo class.
          Hide
          psandoz Paul Sandoz added a comment -
          This was definitely a problem with Groovy (holding ClassInfo instances is a static set), and in this case unrelated to ClassValue.

          If associated values hold a strong reference to the ClassValue instances there will be an issue, but values can hold a strong reference to the class itself and there should be no issue with that, since the value has no reference to the key utilized by ClassValue.ClassValueMap.

          This bug can be closed as not an issue. I will open another issue to add an api note to ClassValue warning users that values should not hold references to ClassValue instance.
          Show
          psandoz Paul Sandoz added a comment - This was definitely a problem with Groovy (holding ClassInfo instances is a static set), and in this case unrelated to ClassValue. If associated values hold a strong reference to the ClassValue instances there will be an issue, but values can hold a strong reference to the class itself and there should be no issue with that, since the value has no reference to the key utilized by ClassValue.ClassValueMap. This bug can be closed as not an issue. I will open another issue to add an api note to ClassValue warning users that values should not hold references to ClassValue instance.

            People

            • Assignee:
              psandoz Paul Sandoz
              Reporter:
              webbuggrp Webbug Group
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: