javafilerandomaccessfilebufferedoutputstream

How does RandomAccessFile.seek() work?

As per the API, these are the facts:

However, the situation I'm curious about is: When there is a file with no data (0 bytes) and I execute the following code:

file.seek(100000-1);
file.write(0);

All the 100,000 bytes are filled with 0 almost instantly. I can clock over 200GB in say, 10 ms.

But when I try to write 100000 bytes using other methods such as BufferedOutputStream the same process takes an almost infinitely longer time.

What is the reason for this difference in time? Is there a more efficient way to create a file of n bytes and fill it with 0s?

EDIT: If the data is not actually written, how is the file filled with data? Sample this code:

RandomAccessFile out=new RandomAccessFile("D:/out","rw");
out.seek(100000-1);
out.write(0);
out.close();

This is the output:

Output

Plus, If the file is huge enough I can no longer write to the disk due to lack of space.

Solution

  • When you write 100,000 bytes to a BufferedOutputStream, your program is explicitly accessing each byte of the file and writing a zero.

    When you use a RandomAccessFile.seek() on a local file, you are indirectly using the C system call fseek(). How that gets handled depends on the operating system.

    In most modern operating systems, sparse files are supported. This means that if you ask for an empty 100,000 byte file, 100,000 bytes of disk space are not actually used. When you write to byte 100,001, the OS still doesn't use 100,001 bytes of disk. It allocates a small amount of space for the block containing "real" data, and keeps track of the empty space separately.

    When you read a sparse file, for example, by fseek()ing to byte 50,000, then reading, the OS can say "OK, I have not allocated disk space for byte 50,000 because I have noted that bytes 0 to 100,000 are empty. Therefore I can return 0 for this byte.". This is invisible to the caller.

    This has the dual purpose of saving disk space, and improving speed. You have noticed the speed improvement.

    More generally, fseek() goes directly to a position in a file, so it's O(1) rather than O(n). If you compare a file to an array, it's like doing x = arr[n] instead of for(i = 0; i<=n; i++) { x = arr[i]; }

    This description, and that on Wikipedia, is probably sufficient to understand why seeking to byte 100,000 then writing is faster than writing 100,000 zeros. However you can read the Linux kernel source code to see how sparse files are implemented, you can read the RandomAccessFile source code in the JDK, and the JRE source code, to see how they interact. However, this is probably more detail than you need.