Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8175883

bytecode generated for the enhanced for loop may block memory garbage collecting

    Details

    • Subcomponent:
    • Resolved In Build:
      b33
    • CPU:
      generic
    • OS:
      generic

      Description

      A DESCRIPTION OF THE REQUEST :
      If a "for" loop iterates over a large list/array it keeps a temporary (implicit auto-generated) reference to the memory, and this memory might be locked for garbage collecting in the next steps which causes OutOfMemoryError.

      The problem might happen if one uses:
        - a large list or arrays which allocates, for example, more than half of available heap space
        - a new list/array should be allocated immediately after the "for" loop and the old one expected to be discarded

      JUSTIFICATION :
      As soon as explicit reference to any (large) memory block is released it should be available for collecting as a garbage, otherwise an application may fall to OutOfMemoryError.

      I found this example when i needed to process two large XML files: after parsing the first one i fell to OutOfMemoryError when i tried to parse the second one, and i discarded on declared references to the previously parsed XML file. Note, if i avoid "for" loop and use explicit "Iterator-while-hasNext-next-iterator=null" construction - the application works fine.

      In the examples below i simplified the issue demonstration.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      No OutOfMemoryError when i don't have any explicit references to allocated memory.

      As soon as "for" block is finished the platform (not sure javac or JVM) must release those auto-generated implicit reference and make the memory block available for garbage collecting.
      ACTUAL -
      The program falls to OutOfMemoryError.

      The program keeps it's own implicit auto-generated reference to the memory block till next implicit usage. If one tries to allocate a memory before that "next implicit usage" - OutOfMemoryError occurs.

      ---------- BEGIN SOURCE ----------
      /**
      * See more examples here: https://bitbucket.org/snippets/radistao/geerj
      * <p>
       * Run with:
       * <pre>javac IteratorInOneScope.java && java IteratorInOneScope</pre>
       *
       * and try two cases:<ul>
       * <li>with the <b>for</b> loop: falls to <i>java.lang.OutOfMemoryError: Java heap space</i></li>
       * <li>without <b>for</b> loop: completes successfully</li>
       * </ul>
       * <p>
       * <b>Conclusion</b>: <b>for</b> loop created automatically a (temporary) reference to a memory block,
       * but has not released it after usage before the next memory allocation.
       */
      public class IteratorInOneScope {

          private static final int HALF_OF_MEMORY = (int) (Runtime.getRuntime().maxMemory() * 0.5);

          public static void main(String[] args) {
              byte[] data = new byte[HALF_OF_MEMORY];

              for (byte b : data); // <-- if you comment this line - the application finished successfully
              data = null; // this expects to discard reference -> allow to release the memory

              byte[] data2 = new byte[HALF_OF_MEMORY]; // the memory can't be allocated second time, if the "for" loop statement above is used

              System.out.println("Success");
          }
      }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      See more examples and workarounds in this snippet:
      https://bitbucket.org/snippets/radistao/geerj

      1. If you avoid those "for" loops - it works fine (no OutOfMemoryError)

      2. If you use explicit code block with final nullify, like
          Iterator itr = list.iterator();
          while (itr.hasNext()) {
              Object t = itr.next();
          }
          itr = null; // <-- saves our souls
        
      3. If you make intermediate operations between the "for" loop and the next memory allocation - it may work fine, but guess there is not guarantee that exactly the same reference will be reused in those intermediate operation (see example https://bitbucket.org/snippets/radistao/geerj#file-IteratorInOtherScope.java where "System.gc()" or "double d = Math.random()" is used before second allocation)

      4. The first open investigation started here
      http://stackoverflow.com/questions/42403347/java-for-statement-implementation-prevents-garbage-collecting
      as the explantation why those implicit reference occur.

      5. Tested on versions from 1.8.0_111 and 1.8.0_121, macos and windows, + 3 online java compilers (like this one: https://ideone.com/GJ1qoI )

        Issue Links

          Activity

          Hide
          dholmes David Holmes added a comment -
          Moved to tools->javac
          Show
          dholmes David Holmes added a comment - Moved to tools->javac
          Hide
          jjg Jonathan Gibbons added a comment -
          Any change in behavior for javac should be coordinated with JLS 14.14.2.
          Show
          jjg Jonathan Gibbons added a comment - Any change in behavior for javac should be coordinated with JLS 14.14.2.
          Hide
          darcy Joe Darcy added a comment -
          A reminder that a spec change should have a CSR before being pushed.
          Show
          darcy Joe Darcy added a comment - A reminder that a spec change should have a CSR before being pushed.
          Hide
          vromero Vicente Arturo Romero Zaldivar added a comment -
          comment from Dan:
          JLS 14.14.2 is only concerned with semantics of the statement, not garbage collection behavior. Compilers are free to generate whatever bytecode they want that produces the specified behavior. So, if javac wants to set some unused locals to null, it's free to do so. No spec change necessary, and it would be a mistake to include it in the spec, because it doesn't impact the statement's semantics.
          Show
          vromero Vicente Arturo Romero Zaldivar added a comment - comment from Dan: JLS 14.14.2 is only concerned with semantics of the statement, not garbage collection behavior. Compilers are free to generate whatever bytecode they want that produces the specified behavior. So, if javac wants to set some unused locals to null, it's free to do so. No spec change necessary, and it would be a mistake to include it in the spec, because it doesn't impact the statement's semantics.
          Hide
          hgupdate HG Updates added a comment -
          URL: http://hg.openjdk.java.net/jdk/jdk/rev/e29ae57c6421
          User: vromero
          Date: 2017-11-20 22:12:51 +0000
          Show
          hgupdate HG Updates added a comment - URL: http://hg.openjdk.java.net/jdk/jdk/rev/e29ae57c6421 User: vromero Date: 2017-11-20 22:12:51 +0000

            People

            • Assignee:
              vromero Vicente Arturo Romero Zaldivar
              Reporter:
              webbuggrp Webbug Group
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: