Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8175883

bytecode generated for the enhanced for loop may block memory garbage collecting

    Details

    • Subcomponent:
    • Resolved In Build:
      b33
    • CPU:
      generic
    • OS:
      generic

      Description

      A DESCRIPTION OF THE REQUEST :
      If a "for" loop iterates over a large list/array it keeps a temporary (implicit auto-generated) reference to the memory, and this memory might be locked for garbage collecting in the next steps which causes OutOfMemoryError.

      The problem might happen if one uses:
        - a large list or arrays which allocates, for example, more than half of available heap space
        - a new list/array should be allocated immediately after the "for" loop and the old one expected to be discarded

      JUSTIFICATION :
      As soon as explicit reference to any (large) memory block is released it should be available for collecting as a garbage, otherwise an application may fall to OutOfMemoryError.

      I found this example when i needed to process two large XML files: after parsing the first one i fell to OutOfMemoryError when i tried to parse the second one, and i discarded on declared references to the previously parsed XML file. Note, if i avoid "for" loop and use explicit "Iterator-while-hasNext-next-iterator=null" construction - the application works fine.

      In the examples below i simplified the issue demonstration.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      No OutOfMemoryError when i don't have any explicit references to allocated memory.

      As soon as "for" block is finished the platform (not sure javac or JVM) must release those auto-generated implicit reference and make the memory block available for garbage collecting.
      ACTUAL -
      The program falls to OutOfMemoryError.

      The program keeps it's own implicit auto-generated reference to the memory block till next implicit usage. If one tries to allocate a memory before that "next implicit usage" - OutOfMemoryError occurs.

      ---------- BEGIN SOURCE ----------
      /**
      * See more examples here: https://bitbucket.org/snippets/radistao/geerj
      * <p>
       * Run with:
       * <pre>javac IteratorInOneScope.java && java IteratorInOneScope</pre>
       *
       * and try two cases:<ul>
       * <li>with the <b>for</b> loop: falls to <i>java.lang.OutOfMemoryError: Java heap space</i></li>
       * <li>without <b>for</b> loop: completes successfully</li>
       * </ul>
       * <p>
       * <b>Conclusion</b>: <b>for</b> loop created automatically a (temporary) reference to a memory block,
       * but has not released it after usage before the next memory allocation.
       */
      public class IteratorInOneScope {

          private static final int HALF_OF_MEMORY = (int) (Runtime.getRuntime().maxMemory() * 0.5);

          public static void main(String[] args) {
              byte[] data = new byte[HALF_OF_MEMORY];

              for (byte b : data); // <-- if you comment this line - the application finished successfully
              data = null; // this expects to discard reference -> allow to release the memory

              byte[] data2 = new byte[HALF_OF_MEMORY]; // the memory can't be allocated second time, if the "for" loop statement above is used

              System.out.println("Success");
          }
      }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      See more examples and workarounds in this snippet:
      https://bitbucket.org/snippets/radistao/geerj

      1. If you avoid those "for" loops - it works fine (no OutOfMemoryError)

      2. If you use explicit code block with final nullify, like
          Iterator itr = list.iterator();
          while (itr.hasNext()) {
              Object t = itr.next();
          }
          itr = null; // <-- saves our souls
        
      3. If you make intermediate operations between the "for" loop and the next memory allocation - it may work fine, but guess there is not guarantee that exactly the same reference will be reused in those intermediate operation (see example https://bitbucket.org/snippets/radistao/geerj#file-IteratorInOtherScope.java where "System.gc()" or "double d = Math.random()" is used before second allocation)

      4. The first open investigation started here
      http://stackoverflow.com/questions/42403347/java-for-statement-implementation-prevents-garbage-collecting
      as the explantation why those implicit reference occur.

      5. Tested on versions from 1.8.0_111 and 1.8.0_121, macos and windows, + 3 online java compilers (like this one: https://ideone.com/GJ1qoI )

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vromero Vicente Arturo Romero Zaldivar
                Reporter:
                webbuggrp Webbug Group
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: