Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8258956

Memory Leak in StringCoding on ThreadLocal resultCached StringCoding.Result

    XMLWordPrintable

    Details

    • Subcomponent:
    • Resolved In Build:
      b06
    • CPU:
      generic
    • OS:
      generic
    • Verification:
      Not verified

      Description

      ADDITIONAL SYSTEM INFORMATION :
      Running on Window 10 - 19019 but happens also on Linux distributions
      Tests using openjdk version "11.0.9.1" 2020-11-04


      A DESCRIPTION OF THE PROBLEM :
      In JDK 11 the class StringCoding has been change and has introduced a new ThreadLocal resultCached showing StringCoding.Result

      Line 512
           /* The cached Result for each thread */
          private static final ThreadLocal<StringCoding.Result>
              resultCached = new ThreadLocal<>() {
                  protected StringCoding.Result initialValue() {
                      return new StringCoding.Result();
                  }};

      Since the StringCoding.Result() is not encapsulated in a SoftReference the Garbage Collector cannot cleanup the internal byte[] even if choosing a very small value for -XX:SoftRefLRUPolicyMSPerMB=<ms>

      I have done a simple test to reproduce the error. Therfore I write a simple Spring Boot Web Application providng a Rest interface. Within the rest method I created a simple huge byte array and have called the String constructure which does a encoding.

          byte[] testArray = //some huge byte array
          new String(testArray, 0, testArray.length, "UTF-8");

      Internally java.lang.String calls StringCoding.decode
          
           StringCoding.Result ret = StringCoding.decode(charset, bytes, offset, length);

      Within StringCoding the StringCoding.Result class helds an byte[] of the given data.

      Now call the Rest interface most of the time the caller get a new http-nio-8080-exec-n thread which is processing the request.
      Now, after the StringCoding.decode method has been called the Result is cached in the ThreadLocal variable and not cleaned up after processing.
      This results in a memory leak if Thread Pools are used which is very usual in Spring Boot or Applications Server in general. If the values of the testArray are very huge (e.g. XML or JSON messages) while different Thread are taken from a Thread Pool it will result in a Out Of Memory Error after a view requests.

      Profilers Heap Walker says:
      org.apache.tomcat.util.threads.TaskThread 195 MB (48 %)
       195 MB (100.0%) threadLocals (declared by java.lang.Thread) java.lang.ThreadLocal$ThreadLocalMap
       195 MB (100.0%) table java.lang.ThreadLocal$ThreadLocalMap$Entry[ ]
       195 MB (100.0%) element java.lang.ThreadLocal$ThreadLocalMap$Entry
       195 MB (100.0%) value java.lang.StringCoding$Result
       195 MB (100.0%) value byte[ ]
       Another 8 instances with a total retained size of 784 bytes and a maximum single retained size of 200 bytes
       Another 4 instances with a total retained size of 256 bytes and a maximum single retained size of 104 bytes
      org.apache.tomcat.util.threads.TaskThread 195 MB (48 %)
       195 MB (100.0%) threadLocals (declared by java.lang.Thread) java.lang.ThreadLocal$ThreadLocalMap
       195 MB (100.0%) table java.lang.ThreadLocal$ThreadLocalMap$Entry[ ]
       195 MB (100.0%) element java.lang.ThreadLocal$ThreadLocalMap$Entry
       195 MB (100.0%) value java.lang.StringCoding$Result
       195 MB (100.0%) value byte[ ]
       Another 8 instances with a total retained size of 784 bytes and a maximum single retained size of 200 bytes
       Another 4 instances with a total retained size of 256 bytes and a maximum single retained size of 104 bytes

      ThreadLocal for StringCoding.Result has been introduced in JDK 11
      All version since JDK 11 are effected, also have took a look on JDK 17 and tested with JDK 17


      REGRESSION : Last worked in version 8

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      I have done a simple test to reproduce the error. Therfore I write a simple Spring Boot Web Application providng a Rest interface. Within the rest method I created a simple huge byte array and have called the String constructure which does a encoding.

          byte[] testArray = //some huge byte array
          new String(testArray, 0, testArray.length, "UTF-8");

      Internally java.lang.String calls StringCoding.decode
          
           StringCoding.Result ret = StringCoding.decode(charset, bytes, offset, length);

      Within StringCoding the StringCoding.Result class helds an byte[] of the given data.

      Now call the Rest interface most of the time the caller get a new http-nio-8080-exec-n thread which is processing the request.
      Now, after the StringCoding.decode method has been called the Result is cached in the ThreadLocal variable and not cleaned up after processing.
      This results in a memory leak if Thread Pools are used which is very usual in Spring Boot or Applications Server in general. If the values of the testArray are very huge (e.g. XML or JSON messages) while different Thread are taken from a Thread Pool it will result in a Out Of Memory Error after a view requests.

      Profilers Heap Walker says:
      org.apache.tomcat.util.threads.TaskThread 195 MB (48 %)
       195 MB (100.0%) threadLocals (declared by java.lang.Thread) java.lang.ThreadLocal$ThreadLocalMap
       195 MB (100.0%) table java.lang.ThreadLocal$ThreadLocalMap$Entry[ ]
       195 MB (100.0%) element java.lang.ThreadLocal$ThreadLocalMap$Entry
       195 MB (100.0%) value java.lang.StringCoding$Result
       195 MB (100.0%) value byte[ ]
       Another 8 instances with a total retained size of 784 bytes and a maximum single retained size of 200 bytes
       Another 4 instances with a total retained size of 256 bytes and a maximum single retained size of 104 bytes
      org.apache.tomcat.util.threads.TaskThread 195 MB (48 %)
       195 MB (100.0%) threadLocals (declared by java.lang.Thread) java.lang.ThreadLocal$ThreadLocalMap
       195 MB (100.0%) table java.lang.ThreadLocal$ThreadLocalMap$Entry[ ]
       195 MB (100.0%) element java.lang.ThreadLocal$ThreadLocalMap$Entry
       195 MB (100.0%) value java.lang.StringCoding$Result
       195 MB (100.0%) value byte[ ]
       Another 8 instances with a total retained size of 784 bytes and a maximum single retained size of 200 bytes
       Another 4 instances with a total retained size of 256 bytes and a maximum single retained size of 104 bytes

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      StringCoding.Result is cleaned up after usage or encapsulated in a SoftReference to avoid OOM
      ACTUAL -
      Memory Leak on each new Thread

      FREQUENCY : always


        Attachments

          Issue Links

            Activity

              People

              Assignee:
              naoto Naoto Sato
              Reporter:
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: