[SOLVED] 32-bit JVM maximum memory size on 64-bit Windows not as large as expected

32-bit JVM maximum memory size on 64-bit Windows not as large as expected

I've been having a tough time memory mapping a 550MB file. I understand that 32-bit JVM can allocate a maximum memory size of around 1.4G, so I need to map a large file by parts. However, this is a 550MB file but I still cannot map it all into memory. The best I can do is a buffer size of around 333MB.

Below is my test code:

void testMap() throws IOException{
    long buffer = 500000000; // CAUSES ERROR. best I can do is 350000000
    RandomAccessFile srcFile = new RandomAccessFile("550MBFile", "r");
    ByteBuffer srcbb = srcFile.getChannel().map(MapMode.READ_ONLY, 0, buffer);
}

And the error is as following:

Exception in thread "main" java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(Unknown Source) at TestSpliter.testMap(TestSpliter.java:22) at TestSpliter.main(TestSpliter.java:14) Caused by: java.lang.OutOfMemoryError: Map failed ...

JVM argument: -Xms1024m

Can anyone explain why I can only use 300MB out of 1.4GB? Thanks.

Note that this is not a duplicate question with those ones asking about the maximum heap size for 32-bit JVM.

Solution

Most likely, your address space is fragmented and there isn't more than 550MB available of contiguous address space. If you need to memory map large files, you need to be able to map them in segments if that becomes necessary.