cmemorymemory-managementdynamic-memory-allocationmemory-fragmentation

What algorithm to apply for continuious reallocation of small memory chunks?


In C program I face transactions that require to have alot of memory chunks, I need to know if there is an algorithm or best practice teqnique used to handle all these malloc/free, I've used arrays to store these memory chunks but at some point the array itself gets full and reallocating the array is just more waste, what is the elegant way to handle this issue?


Solution

  • The best algorithm in this case would be free list allocator + binary search tree.

    You are asking one big memory chunk from system, and then allocating memory blocks of fixed size from this chunk. When chunk is full you are allocating another one, and link them into red-black or AVL binary search interval tree (otherwise searching for a chunk during free by iterating over the chunks list becomes a bottleneck)

    And in the same time, multi-threading and thread synchronization becomes important. If you will simply use mutexes or trivial spin-locks, you will found you libc malloc works much faster than your custom memory allocator. Solution can be found in Hazard pointer, i.e. each thread have it's own memory allocator (or one allocator per CPU core). This will add another problem - when one thread allocates memory and another releasing it, this will require searching for the exact allocator during free function, and strick locking or lock-free data structures.

    The best option - you can use jemalloc, tcmalloc or any another generic propose fast memory allocator to replace your libc (i.e. pdmalloc) default allocator completely or partially.