Details
-
Type:
Enhancement
-
Status: Open
-
Priority:
P4
-
Resolution: Unresolved
-
Affects Version/s: 1.4.0, 5.0, 7
-
Fix Version/s: None
-
Component/s: core-libs
Description
Name: nt126004 Date: 07/31/2002
FULL PRODUCT VERSION :
java version "1.4.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-b92)
Java HotSpot(TM) Client VM (build 1.4.0-b92, mixed mode)
FULL OPERATING SYSTEM VERSION :
Microsoft Windows 2000 [Version 5.00.2195]
A DESCRIPTION OF THE PROBLEM :
Once a file has been mapped a number of operations on that
file will fail until the mapping has been released (e.g.
delete, truncating to a size less than the mapped area).
However the programmer can't control accurately the time at
which the unmapping takes place --- typically it depends on
the processing of finalization or a PhantomReference queue.
The danger associated with unmapping that is noted in the
JavaDoc could be avoided if when an area is unmapped, the
memory is not made available for reallocation but instead
is reserved (e.g. by using VirtualAlloc under Windows).
Then any unwanted reference to the previously mapped area
would result in an exception without incurring performance
penalties. The finalization process could release the
reservation when there were no remaining references to the
buffer.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Run the attached code.
2. A 1KB file will remain --- the File.delete call fails
unless the System.gc() is uncommented. However we can't
rely on the GC call actually doing anything.
EXPECTED VERSUS ACTUAL BEHAVIOR :
There is no way of ensuring that the delete will work even
though we are no longer interested in the mapping.
The deleteOnExit doesn't work either (with far less excuse)
and I have submitted a bug report for that.
REPRODUCIBILITY :
This bug can be reproduced often.
---------- BEGIN SOURCE ----------
import java.io.*;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
class TestMemoryMapping
{
public static void main(String[] args)
{
try
{
File f = File.createTempFile("Test", null);
f.deleteOnExit();
RandomAccessFile raf = new RandomAccessFile(f, "rw");
raf.setLength(1024);
FileChannel channel = raf.getChannel();
MappedByteBuffer buffer = channel.map
(FileChannel.MapMode.READ_WRITE, 0, 1024);
channel.close();
raf.close();
buffer = null;
// System.gc();
if (f.delete())
System.out.println("Temporary file
deleted: "+f);
else
System.out.println("Not yet deleted: "+f);
}
catch (IOException ex)
{
ex.printStackTrace();
}
}
}
---------- END SOURCE ----------
(Review ID: 153923)
======================================================================
FULL PRODUCT VERSION :
java version "1.4.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-b92)
Java HotSpot(TM) Client VM (build 1.4.0-b92, mixed mode)
FULL OPERATING SYSTEM VERSION :
Microsoft Windows 2000 [Version 5.00.2195]
A DESCRIPTION OF THE PROBLEM :
Once a file has been mapped a number of operations on that
file will fail until the mapping has been released (e.g.
delete, truncating to a size less than the mapped area).
However the programmer can't control accurately the time at
which the unmapping takes place --- typically it depends on
the processing of finalization or a PhantomReference queue.
The danger associated with unmapping that is noted in the
JavaDoc could be avoided if when an area is unmapped, the
memory is not made available for reallocation but instead
is reserved (e.g. by using VirtualAlloc under Windows).
Then any unwanted reference to the previously mapped area
would result in an exception without incurring performance
penalties. The finalization process could release the
reservation when there were no remaining references to the
buffer.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Run the attached code.
2. A 1KB file will remain --- the File.delete call fails
unless the System.gc() is uncommented. However we can't
rely on the GC call actually doing anything.
EXPECTED VERSUS ACTUAL BEHAVIOR :
There is no way of ensuring that the delete will work even
though we are no longer interested in the mapping.
The deleteOnExit doesn't work either (with far less excuse)
and I have submitted a bug report for that.
REPRODUCIBILITY :
This bug can be reproduced often.
---------- BEGIN SOURCE ----------
import java.io.*;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
class TestMemoryMapping
{
public static void main(String[] args)
{
try
{
File f = File.createTempFile("Test", null);
f.deleteOnExit();
RandomAccessFile raf = new RandomAccessFile(f, "rw");
raf.setLength(1024);
FileChannel channel = raf.getChannel();
MappedByteBuffer buffer = channel.map
(FileChannel.MapMode.READ_WRITE, 0, 1024);
channel.close();
raf.close();
buffer = null;
// System.gc();
if (f.delete())
System.out.println("Temporary file
deleted: "+f);
else
System.out.println("Not yet deleted: "+f);
}
catch (IOException ex)
{
ex.printStackTrace();
}
}
}
---------- END SOURCE ----------
(Review ID: 153923)
======================================================================
Issue Links
- duplicates
-
JDK-6893654 Mapping large files leads to system "death" due to catastrophic swapping
-
-
JDK-8144174 Potential off-heap memory leak, while FileChannel.map
-
-
JDK-6558368 (bf) MappedByteBuffer.release()/close() to release system resources
-
-
JDK-8028683 Memory-Mapped files cannot be renamed
-
- relates to
-
JDK-6359560 (fs) File.deleteOnExit() doesn't work when MappedByteBuffer exists (win)
-
-
JDK-6417205 (fc) Out of address space because mapped ByteBuffers are not freed
-
Show 1 more links (1 relates to)
There is no unmap() method on mapped byte buffers because there is no known
technique for providing one without running into insurmountable security and
performance issues.
Suppose that a thread operating on behalf of Alice maps a file into memory and
then unmaps it. A second thread operating on behalf of Bob then maps some
other file that the underlying operating system happens to assign to the same
memory address. Now Alice's thread can read, and possibly even modify, the
contents of Bob's file. Oops.
This problem could be avoided by defining a private volatile field in a mapped
buffer to indicate whether or not the mapping is still valid, and having every
access to the buffer's content first check this field and throw an appropriate
exception if the mapping is no longer valid. This solution would significantly
slow down access, however, and fast access is the whole point of supporting
mapped buffers in the first place.
Another potential solution is to implement unmap() by unmapping the buffer's
memory region and then immediately remapping it from /dev/null, setting the
page protections so that any further access to the region would cause a SIGSEGV
(or the equivalent) to be raised and then translated into an appropriate
Java-level exception.
The remapped region would need to remain valid for as long as the original
buffer object is valid, eventually being released by the cleaner thread. This
solution therefore wouldn't reduce virtual-memory footprint but it would at
least allow the underlying file to be manipulated on those operating systems
(e.g., Windows) that forbid certain file manipulations when valid mappings
exist.
To avoid the race condition between the unmap and remap operations we'd need to
globally synchronize all mapping operations within the same VM; this could,
theoretically, limit scalability on some higher-end systems.
The downfall of this second approach is that while we can avoid this race
condition within the VM we can't avoid it everywhere within the VM's process.
Memory-mapped files are pretty widely used these days, by everything from C's
stdio functions (on some systems) to dynamic linkers (on most Unix systems).
If a mapping operation were initiated by one of these subsystems, or by native
application code (perhaps unknowingly, via an intermediate library), at just
the wrong moment, and it happened to overlap our soon-to-be-remapped region,
then the remap operation would fail.
We at Sun have given this problem a lot of thought, both during the original
development of NIO and in the time since. We have yet to come up with a way to
implement an unmap() method that's safe, efficient, and plausibly portable
across operating systems. We've explored several other alternatives aside from
the two described above, but all of them were even more problematic. We'd be
thrilled if someone could come up with a workable solution, so we'll leave this
bug open in the hope that it will attract attention from someone more clever
than we are.
With regard to the "workaround", described in the JDC comments, of (ab)using
reflection in order to invoke a mapped buffer's cleaner object: This is highly
inadvisable, to put it mildly. It is exceedingly dangerous to forcibly unmap a
mapped byte buffer that's visible to Java code. Doing so risks both the
security and stability of the system.
###@###.### 2005-03-23 02:41:14 GMT