Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-6265734

(fc) single FileChannel slower than using multiple FileChannels (win)

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: P3
    • Resolution: Unresolved
    • Affects Version/s: 5.0
    • Fix Version/s: tbd_major
    • Component/s: core-libs
    • Labels:
    • Subcomponent:
    • Understanding:
      Cause Known
    • CPU:
      x86
    • OS:
      windows_xp

      Description

      FULL PRODUCT VERSION :
      fails with 1.5_02 and 1.4.2_08

      ADDITIONAL OS VERSION INFORMATION :
      Windows XP SP 2

      A DESCRIPTION OF THE PROBLEM :
      The nio package is designed for multithreaded access, but there is a strange performance problem with FileChannel. Using multiple FileChannels with multiple RandomAccessFiles (against the same physical file), is faster than using a single FileChannel against the file. How come?

      I originally thought it might be due to internal syncrhonization in the FileChannel, but according to the JavaDoc, the methods are synchronized only if you use the non-positional reads.

      Under Linux the times for both are nearly identical - which is what is expected.


      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      run the supplied test program. you can change the usage of multiple FileChannel by changing the MULTICHANNEL variable.

      You can also alter the RANDOM variable to control whether or not random vs. sequential reads are performed. In either case, the multichannel version is faster.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      I would expect the time to be be similar, or that the multiple FileChannel be slower, since the OS may need to do some synchronization due to the multiple file descriptors.
      ACTUAL -
      The 'multiple' FileChannel version is faster than the single FileChannel version by almost 2 to 1 on my hardware. Having others test it, the difference was not so great, but the multiple channel version always performs much better.

      REPRODUCIBILITY :
      This bug can be reproduced always.

      ---------- BEGIN SOURCE ----------
      import java.nio.ByteBuffer;
      import java.nio.channels.FileChannel;
      import java.util.Random;

      public class MultiThreadIO {
          static final int NTHREADS = 8;
          static final boolean MULTICHANNEL = true;
          static final boolean RANDOM = true;
          static final int BLOCKSIZE = 1024;
          static final int NBLOCKS = 200000;
          
          static File file = new File("testfile");

          public static void main(String[] args) throws Exception {
              
              RandomAccessFile f = new RandomAccessFile(file,"rw");
              FileChannel ch = f.getChannel();
              
              byte[] buffer = new byte[BLOCKSIZE];
              
              System.err.println("writing...");
              
              for(int i=0;i<NBLOCKS;i++){
                  ch.write(ByteBuffer.wrap(buffer),i*(long)BLOCKSIZE);
              }
              
              System.err.println("starting readers");
              
              long stime = System.currentTimeMillis();
              
              Thread[] thread = new Thread[NTHREADS];
              for(int i=0;i<NTHREADS;i++) {
                  thread[i] = new Reader(file,ch,i);
              }
              for(int i=0;i<NTHREADS;i++) {
                  thread[i].start();
              }
              for(int i=0;i<NTHREADS;i++) {
                  thread[i].join();
              }
              
              System.err.println("multichannel="+MULTICHANNEL+", time = "+(System.currentTimeMillis()-stime));
          }
          
          static class Reader extends Thread {
              File f;
              FileChannel ch;
              RandomAccessFile ra;
              int n;
              
              Random r = new Random();
              
              public Reader(File file, FileChannel channel,int n) throws FileNotFoundException {
                  f = file;
                  ch = channel;
                  this.n = n;
                  
                  if(MULTICHANNEL)
                      ch = new RandomAccessFile(file,"rw").getChannel();
              }

              public void run() {
                  byte[] buffer = new byte[BLOCKSIZE];
                  for(int i=0;i<NBLOCKS;i++){
                      try {
                          if(RANDOM)
                              ch.read(ByteBuffer.wrap(buffer),r.nextInt(NBLOCKS)*BLOCKSIZE);
                          else
                              ch.read(ByteBuffer.wrap(buffer),i*BLOCKSIZE);
                      } catch (IOException e) {
                          e.printStackTrace();
                      }
                  }
                  System.err.println("reader "+n+" done");
              }
          }
      }

      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      write code to manage a pool of FileChannel objects and use a different FileChannel from each thread.

      This does not seem to be how nio was designed to be used.

      Possible cause... the multiple file descriptiors cause Windows to view them as different files, so each is going to get its own buffer, but this would lead to consistency problems unless the FileChannels are extenally synchronized, and the writer "syncs" its data to disk (before the readers attempt to read the file), but if this were the case there could be buffering on the readers, since each would need to always read from disk. Strange.
      ###@###.### 2005-05-05 05:42:25 GMT

        Issue Links

          Activity

          Hide
          alanb Alan Bateman added a comment -
          BT2:EVALUATION

          Most likely the issue here is that Windows doesn't have true pread/pwrite-like support that doesn't impact the global file position. The implication is that the Windows implementation requires additional synchronization that is not required on other platforms.
          Show
          alanb Alan Bateman added a comment - BT2:EVALUATION Most likely the issue here is that Windows doesn't have true pread/pwrite-like support that doesn't impact the global file position. The implication is that the Windows implementation requires additional synchronization that is not required on other platforms.
          Hide
          alanb Alan Bateman added a comment -
          BT2:WORK AROUND

          For those using jdk7 builds then AsynchronousFileChannel does not suffer from this issue (no global file position).
          Show
          alanb Alan Bateman added a comment - BT2:WORK AROUND For those using jdk7 builds then AsynchronousFileChannel does not suffer from this issue (no global file position).
          Hide
          alanb Alan Bateman added a comment -
          BT2:EVALUATION

          The issue we have with ReadFile/WriteFile changing the global file position still remains but if we change the positionLock to be a RW lock then we should be able to allow concurrent read/write if we synchronize against operations that depend on the global file position.
          Show
          alanb Alan Bateman added a comment - BT2:EVALUATION The issue we have with ReadFile/WriteFile changing the global file position still remains but if we change the positionLock to be a RW lock then we should be able to allow concurrent read/write if we synchronize against operations that depend on the global file position.

            People

            • Assignee:
              dxu Dan Xu (Inactive)
              Reporter:
              ndcosta Nelson Dcosta
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Imported:
                Indexed: