Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-4724038

(fs) Add unmap method to MappedByteBuffer

    Details

    • Subcomponent:
    • Understanding:
      Cause Known
    • Introduced In Version:
    • CPU:
      generic, x86
    • OS:
      generic, solaris_8, windows_2000, windows_vista

      Description

      Name: nt126004 Date: 07/31/2002


      FULL PRODUCT VERSION :
      java version "1.4.0"
      Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-b92)
      Java HotSpot(TM) Client VM (build 1.4.0-b92, mixed mode)

      FULL OPERATING SYSTEM VERSION :
      Microsoft Windows 2000 [Version 5.00.2195]

      A DESCRIPTION OF THE PROBLEM :
      Once a file has been mapped a number of operations on that
      file will fail until the mapping has been released (e.g.
      delete, truncating to a size less than the mapped area).
      However the programmer can't control accurately the time at
      which the unmapping takes place --- typically it depends on
      the processing of finalization or a PhantomReference queue.

      The danger associated with unmapping that is noted in the
      JavaDoc could be avoided if when an area is unmapped, the
      memory is not made available for reallocation but instead
      is reserved (e.g. by using VirtualAlloc under Windows).
      Then any unwanted reference to the previously mapped area
      would result in an exception without incurring performance
      penalties. The finalization process could release the
      reservation when there were no remaining references to the
      buffer.

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      1. Run the attached code.
      2. A 1KB file will remain --- the File.delete call fails
      unless the System.gc() is uncommented. However we can't
      rely on the GC call actually doing anything.


      EXPECTED VERSUS ACTUAL BEHAVIOR :
      There is no way of ensuring that the delete will work even
      though we are no longer interested in the mapping.
      The deleteOnExit doesn't work either (with far less excuse)
      and I have submitted a bug report for that.


      REPRODUCIBILITY :
      This bug can be reproduced often.

      ---------- BEGIN SOURCE ----------
      import java.io.*;
      import java.nio.MappedByteBuffer;
      import java.nio.channels.FileChannel;

      class TestMemoryMapping
      {
      public static void main(String[] args)
      {
      try
      {
      File f = File.createTempFile("Test", null);
      f.deleteOnExit();
      RandomAccessFile raf = new RandomAccessFile(f, "rw");
      raf.setLength(1024);
      FileChannel channel = raf.getChannel();
      MappedByteBuffer buffer = channel.map
      (FileChannel.MapMode.READ_WRITE, 0, 1024);
      channel.close();
      raf.close();
      buffer = null;
      // System.gc();
      if (f.delete())
      System.out.println("Temporary file
      deleted: "+f);
      else
      System.out.println("Not yet deleted: "+f);
      }
      catch (IOException ex)
      {
      ex.printStackTrace();
      }
      }
      }
      ---------- END SOURCE ----------
      (Review ID: 153923)
      ======================================================================

        Issue Links

          Activity

          Hide
          mr Mark Reinhold added a comment -
          BT2:EVALUATION

          There is no unmap() method on mapped byte buffers because there is no known
          technique for providing one without running into insurmountable security and
          performance issues.

          Suppose that a thread operating on behalf of Alice maps a file into memory and
          then unmaps it. A second thread operating on behalf of Bob then maps some
          other file that the underlying operating system happens to assign to the same
          memory address. Now Alice's thread can read, and possibly even modify, the
          contents of Bob's file. Oops.

          This problem could be avoided by defining a private volatile field in a mapped
          buffer to indicate whether or not the mapping is still valid, and having every
          access to the buffer's content first check this field and throw an appropriate
          exception if the mapping is no longer valid. This solution would significantly
          slow down access, however, and fast access is the whole point of supporting
          mapped buffers in the first place.

          Another potential solution is to implement unmap() by unmapping the buffer's
          memory region and then immediately remapping it from /dev/null, setting the
          page protections so that any further access to the region would cause a SIGSEGV
          (or the equivalent) to be raised and then translated into an appropriate
          Java-level exception.

          The remapped region would need to remain valid for as long as the original
          buffer object is valid, eventually being released by the cleaner thread. This
          solution therefore wouldn't reduce virtual-memory footprint but it would at
          least allow the underlying file to be manipulated on those operating systems
          (e.g., Windows) that forbid certain file manipulations when valid mappings
          exist.

          To avoid the race condition between the unmap and remap operations we'd need to
          globally synchronize all mapping operations within the same VM; this could,
          theoretically, limit scalability on some higher-end systems.

          The downfall of this second approach is that while we can avoid this race
          condition within the VM we can't avoid it everywhere within the VM's process.
          Memory-mapped files are pretty widely used these days, by everything from C's
          stdio functions (on some systems) to dynamic linkers (on most Unix systems).
          If a mapping operation were initiated by one of these subsystems, or by native
          application code (perhaps unknowingly, via an intermediate library), at just
          the wrong moment, and it happened to overlap our soon-to-be-remapped region,
          then the remap operation would fail.

          We at Sun have given this problem a lot of thought, both during the original
          development of NIO and in the time since. We have yet to come up with a way to
          implement an unmap() method that's safe, efficient, and plausibly portable
          across operating systems. We've explored several other alternatives aside from
          the two described above, but all of them were even more problematic. We'd be
          thrilled if someone could come up with a workable solution, so we'll leave this
          bug open in the hope that it will attract attention from someone more clever
          than we are.

          With regard to the "workaround", described in the JDC comments, of (ab)using
          reflection in order to invoke a mapped buffer's cleaner object: This is highly
          inadvisable, to put it mildly. It is exceedingly dangerous to forcibly unmap a
          mapped byte buffer that's visible to Java code. Doing so risks both the
          security and stability of the system.

          ###@###.### 2005-03-23 02:41:14 GMT
          Show
          mr Mark Reinhold added a comment - BT2:EVALUATION There is no unmap() method on mapped byte buffers because there is no known technique for providing one without running into insurmountable security and performance issues. Suppose that a thread operating on behalf of Alice maps a file into memory and then unmaps it. A second thread operating on behalf of Bob then maps some other file that the underlying operating system happens to assign to the same memory address. Now Alice's thread can read, and possibly even modify, the contents of Bob's file. Oops. This problem could be avoided by defining a private volatile field in a mapped buffer to indicate whether or not the mapping is still valid, and having every access to the buffer's content first check this field and throw an appropriate exception if the mapping is no longer valid. This solution would significantly slow down access, however, and fast access is the whole point of supporting mapped buffers in the first place. Another potential solution is to implement unmap() by unmapping the buffer's memory region and then immediately remapping it from /dev/null, setting the page protections so that any further access to the region would cause a SIGSEGV (or the equivalent) to be raised and then translated into an appropriate Java-level exception. The remapped region would need to remain valid for as long as the original buffer object is valid, eventually being released by the cleaner thread. This solution therefore wouldn't reduce virtual-memory footprint but it would at least allow the underlying file to be manipulated on those operating systems (e.g., Windows) that forbid certain file manipulations when valid mappings exist. To avoid the race condition between the unmap and remap operations we'd need to globally synchronize all mapping operations within the same VM; this could, theoretically, limit scalability on some higher-end systems. The downfall of this second approach is that while we can avoid this race condition within the VM we can't avoid it everywhere within the VM's process. Memory-mapped files are pretty widely used these days, by everything from C's stdio functions (on some systems) to dynamic linkers (on most Unix systems). If a mapping operation were initiated by one of these subsystems, or by native application code (perhaps unknowingly, via an intermediate library), at just the wrong moment, and it happened to overlap our soon-to-be-remapped region, then the remap operation would fail. We at Sun have given this problem a lot of thought, both during the original development of NIO and in the time since. We have yet to come up with a way to implement an unmap() method that's safe, efficient, and plausibly portable across operating systems. We've explored several other alternatives aside from the two described above, but all of them were even more problematic. We'd be thrilled if someone could come up with a workable solution, so we'll leave this bug open in the hope that it will attract attention from someone more clever than we are. With regard to the "workaround", described in the JDC comments, of (ab)using reflection in order to invoke a mapped buffer's cleaner object: This is highly inadvisable, to put it mildly. It is exceedingly dangerous to forcibly unmap a mapped byte buffer that's visible to Java code. Doing so risks both the security and stability of the system. ###@###.### 2005-03-23 02:41:14 GMT
          Hide
          alanb Alan Bateman added a comment -
          BT2:EVALUATION

          --

          (2005-05-24) In mustang b86 there is a small update that may provide some interim relief for cases where the map method currently fails due to a resource issue. The bugID for this work is 6417205.
          Show
          alanb Alan Bateman added a comment - BT2:EVALUATION -- (2005-05-24) In mustang b86 there is a small update that may provide some interim relief for cases where the map method currently fails due to a resource issue. The bugID for this work is 6417205.
          Hide
          aph Andrew Haley added a comment -
          http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-September/035055.html

          "I have two suggested fixes:

          "1. Have an unmap method that throws if a SecurityManager is present.
          Non-secure code that owns its own JVM (i.e. nearly all of it) would then be
          able to unmap files and delete them on Windows. Sandboxed code would still
          suffer, but OK, so be it.

          "2. Have an internal subclass or wrapper object around MappedByteBuffer that
          is only used when a SecurityManager is present, which does the check
          against the volatile field. The JVM should optimise out the virtual method
          call or inline the wrapper code, thus ensuring only sandboxed code pays the
          cost."
          Show
          aph Andrew Haley added a comment - http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-September/035055.html "I have two suggested fixes: "1. Have an unmap method that throws if a SecurityManager is present. Non-secure code that owns its own JVM (i.e. nearly all of it) would then be able to unmap files and delete them on Windows. Sandboxed code would still suffer, but OK, so be it. "2. Have an internal subclass or wrapper object around MappedByteBuffer that is only used when a SecurityManager is present, which does the check against the volatile field. The JVM should optimise out the virtual method call or inline the wrapper code, thus ensuring only sandboxed code pays the cost."

            People

            • Assignee:
              mr Mark Reinhold
              Reporter:
              nthompsosunw Nathanael Thompson (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Imported:
                Indexed: