cubuntumemory-managementubuntu-17.04

Fail to allocate a large amount of virtual memory

I read that when you try to allocate more bytes than are available in RAM using malloc(), it allocates virtual memory. At least on Linux.

I want to allocate a huge amount of virtual memory, lets say 100 GB. So, I wrote something like this:

void* virtual_memory = malloc(100 gb int);

But the returned pointer is NULL.

I execute this code on a 64-bit Ubuntu virtual machine.

What am I doing wrong?

EDIT What I'm trying to achieve is to make htop tool displaying 100GB in VIRT column for my process.

UPDATE I CAN call malloc to allocate 2 GB at once 50 times

Solution

  • I read that when you try to allocate more bytes than are available in RAM using malloc(), it allocates virtual memory

    To start with, this is not correct. You always allocate virtual memory. This virtual memory is mapped to some area on the Physical memory(RAM) or the swap space. If the swap space + physical memory is less than 100 GBs, your allocation will fail. Also, the libc implementation might fail to allocate such a large amount, if it has some (programmable) limit set.

    but I have a strange task to show up 100gb of virtual memory for the process in htop tool. And it's claimed to be achievable via single line of code.

    Yes if you just need this much virtual memory, you can reserve memory but not commit it. You can read upon how mmap(*NIX) or VirtualAlloc(Windows) can be used for the same.

    When you reserve a particular Virtual Address range, you tell the operating system that you intend to use this range, so other code can't use that. But it doesn't mean you can actually use it. This also means that it doesn't need a RAM/Swap backing. So you will be able to reserve arbitrarily large amount (less than 2^48 bytes on your 64 bit system of course).

    Although I am not sure if htop will include that in the value it shows, you will have to try that out.

    If this doesn't indeed add to your virtual memory count, you can map it to a file, instead of mapping it anonymously. This might create a 100 GB file on your system (assuming you have that much space), but you should even be able to read/write to it.

    Following code can be used on linux - 

    int fd = open("temp.txt", O_RDWR | O_CREAT);
void* addr = mmap(NULL, 100 * GBS, PROT_WRITE | PROT_READ, MAP_PRIVATE, fd, 0);