Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-6697161

Compiler memory leak

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: P2
    • Resolution: Duplicate
    • Affects Version/s: 5.0u14
    • Fix Version/s: None
    • Component/s: hotspot
    • Labels:
      None
    • Subcomponent:
    • CPU:
      sparc
    • OS:
      solaris_10

      Description

      Customer reported native memory leak during compilation.
      (refer to radiance case 11209644 for additional details)

      * Observed growing native heap eventually leads to OOM

      root@kmcicap2 # more pmap-x-29561.log

      29561: /usr/jdk1.5.0_14/bin/sparcv9/java -server -Dweblogic.Name=mcis3
      -Xms20
      -Xms20
      Address Kbytes RSS Anon Locked Mode Mapped File
      0000000100114000 2992 1760 1256 - rwx-- [ heap ]
      0000000100400000 3956736 2575432 1961984 - rwx-- [ heap ]
      00000001F1C00000 3891200 2453624 1802240 - rwx-- [ heap ]
      00000002DF400000 4190208 3452952 3145728 - rwx-- [ heap ]
      00000003DF000000 3579904 2973528 2621440 - rwx-- [ heap ]
      00000004B9800000 4190208 2988688 2314240 - rwx-- [ heap ]
      00000005B9400000 1327104 1327104 1327104 - rwx-- [ heap ]


      * At that time, there were many brk() system calls on t@19 to allocate memory

      root@kmcicap2 #truss -dD -p 29561

      Base time stamp: 1208961051.2012 [ Wed Apr 23 23:30:51 KST 2008 ]
      /22: 184.5903 184.5903 pollsys(0x00000000, 0, 0x7FFFFFFF713FF970, 0x00000000) = 0
      /19: 184.6064 184.6064 brk(0x1C073E000) = 0
      /19: 184.6206 0.0142 brk(0x1C074E000) = 0
      /19: 184.6336 0.0130 brk(0x1C075E000) = 0
      /19: 184.6402 0.0066 brk(0x1C076E000) = 0
      /22: 184.6468 0.0565 pollsys(0x00000000, 0, 0x7FFFFFFF713FF970, 0x00000000) = 0
      /19: 184.6527 0.0125 brk(0x1C077E000) = 0
      /19: 184.6662 0.0135 brk(0x1C078E000) = 0
      /53: 184.6668 184.6668 pollsys(0x7FFFFFFF5EAFEE68, 1, 0x7FFFFFFF5EAFED90, 0x00000000) = 0
      /19: 184.6839 0.0177 brk(0x1C079E000) = 0
      /19: 184.6989 0.0150 brk(0x1C07AE000) = 0
      /22: 184.7069 0.0601 pollsys(0x00000000, 0, 0x7FFFFFFF713FF970, 0x00000000) = 0
      /19: 184.7136 0.0147 brk(0x1C07BE000) = 0
      /19: 184.7305 0.0169 brk(0x1C07CE000) = 0
      /19: 184.7447 0.0142 brk(0x1C07DE000) = 0
      /19: 184.7609 0.0162 brk(0x1C07EE000)

      * t@19 from pstack:

      ----------------- lwp# 19 / thread# 19 --------------------
      7fffffff7e5fe3cc __1cENodeFclone6kM_p0_ (26b7fff20, 30, 26b7ffff0, 26b7fff38, 3fffffffbfc849, 7fffffff719febc0) + 11c
      7fffffff7e657480 __1cHAddNodeFIdeal6MpnIPhaseGVN_i_pnENode__ (26b7fff88, 7fffffff719fe2a8, 26b7fff20, 0, e, 26b7fff50) + 614
      7fffffff7e5eb2ec __1cMPhaseIterGVNNtransform_old6MpnENode__2_ (7fffffff719fe2a8, 26b7fff88, 1, 1d1358020, 7ffffff, 7fffffff7e662118) + a4
      7fffffff7e6e4480 __1cMPhaseIterGVNIoptimize6M_v_ (7fffffff719fe2a8, 26b7fff88, 26b7fff88, 0, 7fffffff719fe518, ffffffffffffffff) + b8
      7fffffff7e784218 __1cHCompileIOptimize6M_v_ (7fffffff719febc0, 10050f470, 6a1ea4, 7fffffff7e63c738, fc00, 7fffffff7eca8abf) + c8
      7fffffff7e78747c __1cHCompile2t5B6MpnFciEnv_pnKC2Compiler_pnIciMethod_ii_v_ (7fffffff719febc0, 0, 7fffffff719ffb28, 103dd3ff8, 0, d 80) + c3c
      7fffffff7e77daec __1cKC2CompilerOcompile_method6MpnFciEnv_pnIciMethod_i_v_ (7fffffff719ffb28, 7fffffff7ec91290, 1040d4560, 25, 1005 39d50, 7fffffff719fed60) + b4
      7fffffff7e77e0b4 __1cNCompileBrokerZinvoke_compiler_on_method6FpnLCompileTask__v_ (1040d4560, 10050f470, 1002266d0, 0, 7fffffff7eec 7e10, 7fffffff7ee26000) + 504
      7fffffff7e84b3d0 __1cNCompileBrokerUcompiler_thread_loop6F_v_ (7fffffff7eec7e10, 10050f600, 1002266d0, 10054c720, 7fffffff7ee b3548, 7fffffff7eeb354c) + 450
      7fffffff7e7e4734 __1cKJavaThreadDrun6M_v_ (1002266d0, f000, 7fffffff7eebdd0c, 0, 0, 7fffffff7ee26000) + 2ac
      7fffffff7ebbd0d0 __1cG_start6Fpv_0_ (1002266d0, d800, b000, b388, 7fffffff7eec8f64, 7fffffff7ee26000) + 210
      7fffffff7f5d2298 _lwp_start (0, 0, 0, 0, 0, 0)

      * Successful workaround by excluding one method from compilation

      -XX:CompileCommand=exclude,org/aspectj/apache/bcel/classfile/Attribute,readAttribute
      Looks like a bug in JIT optimization


      * Customer has also opened an incident
        http://webbugs.sfbay/rt/incidentDisplay?incidentID=1229942

        where they provide a testcase and instructions to reproduce the problem
        (with tomcat/BCEL)

        Here's summary of customer's observation:
        (for details, refer to the above incident)

        1. This native memory leak did not occur when using -Xint option.
        2. During the booting stage, if class that initially compiles the hotspot
           (by registering .hotspot_complier and excluding it) using the
           PrintCompliation option, the problem did not occur.
        3. The problem did not occur consistently, the problem occurred occasionally
           after multiple restarts.
        4. In another system, I found cases that are related to this problem.
           In those c ases the native memory leak occurred 10+ hours after the
           server was rebooted. (Solaris 32bit/64bit, Linux 32bit, etc...)
        5. java heap memory did not increase while monitoring using jconsole.
        6. I couldn't determine the root-cause using gcore/mdb:: findleaks.
        7. I noticed that there were a lot of brk() methods used when native memory
           leak occurs after Tomcat' booting is completed.
        8. I do not think that this is a bytecode logic problem because test program
           doesn't transform the original class.
        9. The native memory leak happened on the latest Solaris 10(SPARC 64bit)
           with al l fixes installed, so it is probably not related to OS
           environment problem.
       10. After 3~30 minutes passes from the time native memory leak occurs, the
           JAVA process shuts down with message
           "Exception in thread "CompilerThread1" java.lang .OutOfMemoryError: requested 134217736 bytes for Chunk::new. Out of swap space"


      * In addition, a core file has been collected
        /net/cores.central/cores/dir20/11209644 with pkgcore.
      escalated to sustaining Apollo 1-23815642

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              minqi Yumin Qi
              Reporter:
              lkchow Lawrence Chow
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Imported:
                Indexed: