javabytebufferfilechannel

Reading and writing file in number of Byte Buffer chunks of smaller length


I am trying to read file in ByteBuffer chunks of fixed length and then store it to a list of ByteBuffer and then after some operations read those ByteBuffer chunks in a sequential order to reconstruct the file. Problem is that while writing output file channel position is not increasing. I do not want to use byte arrays, as they are fixed length and file reconstruction does not work properly. So I would like to know how to increase size of file write channel position, or any other way to do this operation. Sample code would be appreciated. Here are my code snippets,

file = new File(fileName);  // hello.txt - 20 MB size
fis = new FileInputStream(file);
inChannel = fis.getChannel();
double maxChunkSequenceNoFloat = ((int)inChannel.size()) / chunkSize;
int maxChunkSequenceNo = 1;
if(maxChunkSequenceNoFloat%10 > 0) {
    maxChunkSequenceNo = ((int)maxChunkSequenceNoFloat)+1;
} else if(maxChunkSequenceNoFloat%10 < 0) {
    maxChunkSequenceNo = 1;
} else {
    maxChunkSequenceNo = (int)maxChunkSequenceNoFloat;
}
maxChunkSequenceNo = (maxChunkSequenceNo == 0) ? 1 : maxChunkSequenceNo;            
ByteBuffer buffer = ByteBuffer.allocate(chunkSize);
buffer.clear();

while(inChannel.read(buffer) > 0) {
    buffer.flip();
    bufferList.add(buffer);
    buffer.clear();
    chunkSequenceNo++;
}
maxChunkSequenceNo = chunkSequenceNo;

// write
File file1 = new File("hello2.txt") ; 
buffer.clear();
FileOutputStream fos = new FileOutputStream(file1);
FileChannel outChannel = fos.getChannel();
chunkSequenceNo = 1;
for(ByteBuffer test : bufferList) {
    writeByteCount += outChannel.write(test);
    //outChannel.position() += writeByteCount;'
    System.out.println("main - channelPosition: "+outChannel.position()
                        +" tempBuffer.Position: "+test.position()
                        +" limit: "+test.limit()
                        +" remaining: "+test.remaining()
                        +" capacity: "+test.capacity());              
}
BufferedReader br = new BufferedReader(new FileReader(file1));
String line = null;
while ((line = br.readLine()) != null) {
    System.out.println(line);
}
outChannel.close();
fos.close();

Bytebuffer position is correct but outChannel position remains at 1048 which is chunk size.


Solution

  • The following would indeed maintain a list of ByteBuffers as requested.

    String fileName = "hello.txt";
    final int chunkSize = 256;
    List<ByteBuffer> bufferList = new ArrayList<>();
    Path path = Paths.get(fileName);
    try (SeekableByteChannel inChannel = Files.newByteChannel(path,
            EnumSet.of(StandardOpenOption.READ))) {
        long size = inChannel.size();
        while (size > 0) {
            ByteBuffer buffer = ByteBuffer.allocate(chunkSize);
    
            int nread = inChannel.read(buffer);
            if (nread <= 0) {
                break;
            }
            buffer.flip();
            bufferList.add(buffer);
            size -= nread;
        }
    }
    
    // write
    Path file1 = Paths.get("hello2.txt") ;
    try (SeekableByteChannel outChannel = Files.newByteChannel(file1,
            EnumSet.of(StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE))) {
        for (ByteBuffer buffer : bufferList) {
            int nwritten = outChannel.write(buffer);
        }
    }
    

    Try-with-resources takes care of closing the channels/files. Files (utility functions) and Path (more generic than File) are useful.

    One needs to add new instances of a ByteBuffer when having a chunkSize limit. (So one also might have added the underlying byte arrays.)

    Better not use floating point, even here.