Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-6315640

(process) Depleting native Win32 process handles when running on large Java heap

    Details

    • Subcomponent:
    • CPU:
      x86
    • OS:
      windows_2000

      Description

      Class java.lang.Win32Process is depleting native Win32 process handles
      when running on a large Java heap. The Win32 API process handles are
      kept until the next minor collection is run. When running on a large
      Java heap, Windows may run of out resources before the next minor
      collection will appear. What is considered a "large" Java heap is
      depending on the machine's load.

      The problem is strictly reproducible.

      0. Equipment
      ------------
       All tests have been made on a Pentium4 Win2000 SP4 with 512MB of RAM,
       only running Windows.

      1. Testcase
      -----------
      The testcase is fairly simple and consists of the following files:
      23/08/2005 19:54 31 native.cpp
      23/08/2005 20:37 525 Exec.java
      24/08/2005 23:32 1,454 make.bat
      You'll need sysinternals pslist (www.sysinternals.com/Utilities/PsList.html):
      23/05/2003 19:21 86,016 pslist.exe


      % more native.cpp

      int main() {
              return 0;
      }

      % more Exec.java
      class Exec {
        public static void main(String args[]) {
          int i = 0;
          while (true) {
            try {
              Process p = Runtime.getRuntime().exec("native.exe");
              p.waitFor();
              p.exitValue();
              p.getErrorStream().close();
              p.getOutputStream().close();
              p.getInputStream().close();
              p.destroy();
            } catch (Exception e) {
              e.printStackTrace();
              break;
            }

            if (i%100 == 0) System.out.println(i);
            i++;

          } // end while
        }
      }


      2. Compile
      ----------
       Used Borland's free C++ compiler:
      E:\>make ccompile
      E:\>bcc32 -enative.exe -LC:\bcc55\lib native.cpp
      Borland C++ 5.5.1 for Win32 Copyright (c) 1993, 2000 Borland
      native.cpp:
      Turbo Incremental Link 5.00 Copyright (c) 1997, 2000 Borland
      E:\>make compile
      E:\>java -version
      java version "1.4.2_09"
      Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_09-b05)
      Java HotSpot(TM) Client VM (build 1.4.2_09-b05, mixed mode)
      E:\>javac Exec.java
      E:\>

      3. Run on default config
      ------------------------
       Everything is fine when running on default or small heapsize:
       Example: default heapsize:
      E:\37443767>make run
      E:\37443767>java -server -verbosegc -XX:+PrintGCDetails Exec
      0
      100
      [GC [DefNew: 511K->64K(576K), 0.0033800 secs] 511K->188K(1984K), 0.0036278 secs]
      200
      [GC [DefNew: 575K->63K(576K), 0.0019318 secs] 700K->323K(1984K), 0.0021575 secs]
      300
      400
      [GC [DefNew: 575K->63K(576K), 0.0014337 secs] 835K->395K(1984K), 0.0016561 secs]
      500
      600
      [GC [DefNew: 575K->63K(576K), 0.0011099 secs] 907K->468K(1984K), 0.0013306 secs]
      700
      [GC [DefNew: 573K->63K(576K), 0.0012275 secs] 978K->541K(1984K), 0.0015496 secs]
      800
      900
      [GC [DefNew: 575K->63K(576K), 0.0010415 secs] 1053K->612K(1984K), 0.0012650 secs]
      1000
      [GC [DefNew: 575K->63K(576K), 0.0010423 secs] 1124K->684K(1984K), 0.0012655 secs]
      1100
      [ ... keeps running forever ]
      Terminate batch job (Y/N)? y

       Use "pslist.exe" in a second shell window to monitor handle utilisation:
       
      E:\37443767>pslist java

      PsList 1.23 - Process Information Lister
      Copyright (C) 1999-2002 Mark Russinovich
      Sysinternals - www.sysinternals.com

      Process information for WIN2000:

      Name Pid Pri Thd Hnd Mem User Time Kernel Time Elapsed Time
      java 1808 8 10 264 10100 0:00:06.259 0:00:11.125 0:01:06.205

      E:\>pslist java
      [ ... ]
      Process information for WIN2000:
      Name Pid Pri Thd Hnd Mem User Time Kernel Time Elapsed Time
      java 1808 8 10 334 10100 0:00:06.289 0:00:11.196 0:01:06.645

      E:\>pslist java
      [ ... ]
      Process information for WIN2000:
      Name Pid Pri Thd Hnd Mem User Time Kernel Time Elapsed Time
      java 1808 8 10 395 10100 0:00:06.309 0:00:11.256 0:01:07.066

      E:\>pslist java

      PsList 1.23 - Process Information Lister
      Copyright (C) 1999-2002 Mark Russinovich
      Sysinternals - www.sysinternals.com

      Process information for WIN2000:

      Name Pid Pri Thd Hnd Mem User Time Kernel Time Elapsed Time
      java 1808 8 10 304 10100 0:00:06.319 0:00:11.366 0:01:07.507

      E:\>

       The handle count seen is within a small range of 250 - 450 handles.


      4. Testrow: varying -Xms over -Xms1024m
      ----------------------------------------
       The following tests have been run on 1.4.2_09, while -Xmx1024 was
       left unchanged. The following table gives the maximum amount of
       handles seen before minor collection used to kick in:

         -Xms client server
         -----------------------------------
           64m 1450 2050
          128m 2700 4000
          256m 5300 7600
          512m 10400 14900
          768m 15500 fail
         1024m fail fail
         default ~500 ~500
         -----------------------------------
       The minimum number of handles was about 250, seen after
       GC (minor collection most times) had run.

      5. Failure case
      ---------------
       The failure can be most easily reproduced on the above given machine with
       -Xms1024m -Xmx1024m.

      E:\37443767>make run
      E:\37443767>java -server -Xms1024m -Xmx1024m -verbosegc -XX:+PrintGCDetails Exec
      0
      [ ... ]
      18200
      18300
      18400
      18500
      18600
      18700
      java.io.IOException: CreateProcess: native.exe error=1450
              at java.lang.Win32Process.create(Native Method)
              at java.lang.Win32Process.<init>(Win32Process.java:66)
              at java.lang.Runtime.execInternal(Native Method)
              at java.lang.Runtime.exec(Runtime.java:566)
              at java.lang.Runtime.exec(Runtime.java:428)
              at java.lang.Runtime.exec(Runtime.java:364)
              at java.lang.Runtime.exec(Runtime.java:326)
              at Exec.main(Exec.java:6)

      E:\>

       This leads to Windows error 1450 "Insufficient System Resources".
       Obviously Windows ran out of process handles.

       "pslist" monitor showed that the process just used 18948 handles:

      E:\>pslist java

      PsList 1.23 - Process Information Lister
      Copyright (C) 1999-2002 Mark Russinovich
      Sysinternals - www.sysinternals.com

      Process information for WIN2000:

      Name Pid Pri Thd Hnd Mem User Time Kernel Time Elapsed Time
      java 60380 8 10 18948 69112 0:00:10.214 0:00:39.717 0:02:20.782

      C:\>pslist java

       So, this shows that the Java process piled up for its over 18700 iterations
       18948 Windows' handles, until it effectively ran out of Windows' resources.
       Most of the handles in use are expected to be Win32 process handles, as the
       figures just correspond very nicely.

       The exact failure parameters vary with the actual utilisation of the resources
       on the machine. It was reported that the limit was already hit by a little over
       2000 handles being occupied by the Java process.

      6. Documentation
      ----------------
       Looking into Microsoft's documentation for "CreateProcess"
      http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/createprocess.asp
      ------------
      "lpProcessInformation
          [out] Pointer to a PROCESS_INFORMATION structure that receives identification
                information about the new process.

          Handles in PROCESS_INFORMATION must be closed with CloseHandle when they are
          no longer needed."
      ------------
       So, it is required ("must") to close the handles as soon as they are no longer
       needed. The JVM however is stacking up the handles until the objects of type
       java.lang.Win32Process get collected.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                minqi Yumin Qi
                Reporter:
                thlenz Thomas Lenz (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Imported:
                  Indexed: