Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-5047199

(ref) WeakReferences are not always properly enqueued -- WeakRefs are leaked

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: P4
    • Resolution: Not an Issue
    • Affects Version/s: 1.3.1_12, 1.4.2, 5.0
    • Fix Version/s: None
    • Component/s: core-libs
    • Labels:
    • Subcomponent:
    • CPU:
      x86
    • OS:
      linux, windows_2000

      Description



      Name: rmT116609 Date: 05/13/2004


      FULL PRODUCT VERSION :
      java version "1.4.2_01"
      Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_01-b06)
      Java HotSpot(TM) Client VM (build 1.4.2_01-b06, mixed mode)

      FULL OS VERSION :
      Microsoft Windows 2000 [Version 5.00.2195]

      A DESCRIPTION OF THE PROBLEM :
      I was attempting to track resources using a ReferenceQueue of SoftRefs.
      No entries were showing up in queue. Used a profiler (OptimizeIt) to confirm that all strong refs were correctly cleared. They where. Wrote a small test program to confirm I was correctly using the Reference API. This program demonstrated that not all Refs where enqueued. Adding a finalizer to the Refs demonstrated that they were GC'ed but not enqueued.

      Reviewing the Bug database turned up two related issues: 4268317 and 4243978.
      However, neither points out the actual problem.

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Run the provided program.
      It runs two threads: one producing WeakRefs and another consuming them off a ReferenceQueue. This program will eventually consume 2000 "lost" references and exit.

      This is a sample of the output

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      This is a sample of the output:
          nothing in queue
          dequeued: 35
          refset: size=86 first=0 last=86
          dequeued: 86 85
          refset: size=84 first=0 last=84
          dequeued: 137 188 187 186 185 136 135
          refset: size=245 first=0 last=254
          nothing in queue
          sleeping...wakeup call
          dequeued: 290
          refset: size=280 first=0 last=289

      refset is Set tracking "resources" allocation but not released via the ReferenceQueue. It keeps growing.
      There is also some code to throttle the produce as the size of the set grows. This only slows down the eventual failure.

      If you uncomment the finalizer on MyWeakRef the program will run indefinitely.
      ACTUAL -
      Eventual "resource" exhaustion

      ERROR MESSAGES/STACK TRACES THAT OCCUR :
      java.lang.IllegalStateException: exceeded arbitrary resource limit
      at ReferenceLeak$MyWeakReference.<init>(ReferenceLeak.java:129)
      at ReferenceLeak$MyWeakReference.<init>(ReferenceLeak.java:114)
      at ReferenceLeak$1.produce(ReferenceLeak.java:48)
      at ReferenceLeak$1.run(ReferenceLeak.java:37)

      REPRODUCIBILITY :
      This bug can be reproduced always.

      ---------- BEGIN SOURCE ----------
      import java.lang.ref.*;
      import java.util.*;

      /**
       * Class to demostrate how WeakRefs may be leaked by the GC.
       * This class enqueues a medium number of WeakRefs tracking creations.
       * It then tracks what it dequeues. Some Refs never show up in queue.
       * They may even be finalized but still never show up in queue.
       *
       * BugIds 4268317 and 4243978 are probably related to this but don't stress the
       * the problem.
       *
       * If you can't count on SoftRefs being properly enqueued you can't really
       * count on them at all.
       *
       * @author rmp
       */
      public class ReferenceLeak {

          // RefQueue<MyWeakRefs>
          private static ReferenceQueue refQueue = new ReferenceQueue();

          // SortedSet<Integer> -- tracks MyWeakRefs that have been created.
          // Entries are removed when dequeued
          // Also removed if the finalize() code below is uncommented.
          // Should hit a steady state -- 0 in a perfect world.
          // However, unless we also cleanup in finalize() Set keeps growing. I.e
          // some WeakRefs are never enqueued.
          static SortedSet refset = Collections.synchronizedSortedSet(new TreeSet());

          public static void main(String[] args)
          {
              // this Thread is a producer of WeakRefs
              new Thread() {
                  public void run() {
                      try {
                          produce();
                      } catch (Exception e) {
                          e.printStackTrace();
                          System.exit(1);
                      }
                  }

                  public void produce() {
                      for (int i = 0; ; i++)
                      {
                          // create some junk -- a lot of junk
                          Object[] junk = new Object[] {
                              new MyWeakReference(new byte[10000], refQueue),
                              new MyWeakReference(new byte[10000], refQueue),
                              new MyWeakReference(new byte[10000], refQueue),
                              new MyWeakReference(new byte[10000], refQueue),
                              new MyWeakReference(new byte[10000], refQueue),
                          };

                          if (i % 50 == 0)
                          {
                              System.err.print("sleeping...");
                              try {
                                  Thread.sleep(1000);
                                  System.err.println("wakeup call");
                              } catch (InterruptedException e) {
                                  System.err.println("interrupted");
                              }
                          }
                      }
                  }
              }.start();

              // main thread is the consumer of WeakRefs
              while (true) {
                  MyWeakReference ref;

                  // block for the first dequeue
                  try {
                      ref = (MyWeakReference) refQueue.remove(500);
                  } catch (InterruptedException e) {
                      System.out.println("interrupted...");
                      return;
                  }

                  if (ref == null)
                  {
                      System.out.println("nothing in queue");

                      // GC or not doesn't seem to matter
      // System.gc();
      // System.runFinalization();
                      continue;
                  }

                  // output what we dequeue
                  System.out.print("dequeued:");
                  do { // poll for rest
                      int k = ref.getK();
                      boolean missing = !refset.remove(new Integer(k)); // already removed
                      if (missing)
                          System.err.print("(missing " + k + ")");
                      System.out.print(" " + k);

                  } while (null != (ref = (MyWeakReference) refQueue.poll()));

                  // output current state of refset -- produced but not consumed yet
                  System.out.println();
                  System.out.print("refset: size="+refset.size());
                  if (refset.size() > 0)
                     System.out.print(" first=" + refset.first() + " last=" + refset.last());
                  System.out.println();
              }
          }

          // Simple WeakRef has no other strong refs just an ivar
          // In any event, instances are lost -- not properly enqueued
          private static class MyWeakReference extends WeakReference
          {
              private static int __counter;
              private int _k;

              private MyWeakReference(Object ref, ReferenceQueue queue) {
                  super(ref, queue);
                  _k = __counter++;
                  refset.add(new Integer(_k)); // track ref

                  //
                  // add some arbitrary code to throttle down any producer
                  //

                  if (refset.size() > 2000) {
                      throw new IllegalStateException("exceeded arbitrary resource limit");
                  }

                  if (refset.size() > 1900) {
                      System.err.println("gc()");
                      System.gc();
                  }

                  if (refset.size() > 1800) {
                      System.err.println("runFinalization()");
                      System.runFinalization();
                  }

                  // If all refs are actually enqueued this lets consumer catch up
                  if (refset.size() > 1500)
                  {
                      // backoff 10-60ms (more as we approach limit)
                      int backoff = ((refset.size() - 2000 + 500) / 10) + 10;
                      System.err.println("backing off " + backoff);
                      try { Thread.sleep(backoff); } catch (InterruptedException e) { }
                  }

              }

              public int getK() {
                  return _k;
              }

      // enabling the finalizer code here will stop resource exhaustion issues
      // as we are no longer relying on WeakRef alone :(

      /**
              public void finalize() throws Throwable {
                  try {
                      super.finalize();
                  } finally {
                      boolean removed = refset.remove(new Integer(_k));

                      // VERY verbose if enabled
                      // if (removed)
                      // System.err.println("finalizer removed: " + _k);
                  }
              }
      /**/
          }
      }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      Use a combination of WeakRef and finalization.

      (Incident Review ID: 208488)
      ======================================================================

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              gafter Neal Gafter
              Reporter:
              rmandalasunw Ranjith Mandala (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Imported:
                Indexed: