How does C prevent heap growing indefinitely?

I’m investigating how C’s runtime memory allocator behaves in a long-running process that repeatedly:

• allocates many blocks of variable size with malloc(),
• finishes its work, and
• calls free() on all these blocks.

Although everything is freed up, with heap fragmentation, I'm guessing that when the blocks are allocated a second time, it won't perfectly match what's available on the heap. So I'm guessing that C will request memory with brk at some point, thus growing the size of heap.

But by doing this, the heap can grow indefinitely if, for example, a server handles requests 24/7.

What does C do to prevent the heap from growing indefinitely?

My first guess is that C periodically executes functions, perhaps similar to malloc_trim.

How does C prevent heap growing indefinitely?

The C language specifies no mechanism for this. In fact, "heap" is not even a C-language concept. The C language specification does not specify a partitioning of memory into areas of different types or having different characteristics. It does not address how its memory management functions are implemented. On the other hand, C does acknowledge and make provision for the fact that the system might not be able to satisfy dynamic memory allocation requests under some (unspecified) circumstances.

I’m investigating how C’s runtime memory allocator behaves

There is no "C’s runtime memory allocator", only the allocator used by your particular C implementation in support of your particular program. Different C implementations have different allocators with different characteristics. Some are tunable in various ways. Nothing prevents an implementation from having more than one to select from.

that repeatedly:

• allocates many blocks of variable size with malloc(),
• finishes its work, and
• calls free() on all these blocks.

Although everything is freed up, with heap fragmentation, I'm guessing that when the blocks are allocated a second time, it won't perfectly match what's available on the heap.

The particular scenario you describe is a very friendly one. It could present a fragmentation problem, but a good allocator will coalesce adjacent free blocks, at least from time to time or at need. With such an allocator, there is effectively no heap fragmentation when nothing is allocated on the heap, allocation history notwithstanding.

So I'm guessing that C will request memory with brk at some point, thus growing the size of heap.

C will do no such thing. Your particular C implementation might do, but that depends on a multitude of details. The specifics of how that plays out, even more so.

But by doing this, the heap can grow indefinitely if, for example, a server handles requests 24/7.

It is possible that a poor allocator implementation in conjunction with a malicious or infellicitous program and workload would produce that effect. In practice, however, unbounded memory usage is almost always associated with objects remaining allocated.

What does C do to prevent the heap from growing indefinitely?

Competent allocators use a variety of techniques to minimize or eliminate such problems. Some of them are:

• Managing actual allocation sizes. That you successfully allocate n bytes does not imply that the allocator has reserved exactly n bytes for you. It may have reserved more, such that even if that block alone is freed, it is reusable to serve a variety of requests, possibly including some for more than n bytes, and likely including many for less than n bytes.

• Automatically coalescing adjacent free regions, typically as part of the operation of free() itself. With an allocator that does this, the allocation pattern described in the question does not leave any residual fragmentation at the end of a cycle, when all the allocated blocks have been freed.

• Employing allocation patterns and strategies intended to improve the ability to do the aforementioned coalescing.

• Making larger allocations in different ways or from different areas than smaller ones. Among other things, this avoids a few small, poorly positioned allocations from creating fragmentation that blocks large allocations, because the small allocations just don't happen in the same part of the address space that large ones do.

It's unclear what particular allocator you're investigating, or whether you're looking at a particular one at all. You may be able to find documentation about the behavior of allocators of interest. If you happen to be looking at Glibc's in particular, then the documentation contains an overview, and there is a separate detailed description.

My first guess is that C periodically executes functions, perhaps similar to malloc_trim.

Again C does no such thing. Moreover, although I can't speak to C implementations in general, it would be surprising to find that one did GC-like periodic malloc_trim()s or equivalent on a timer. It is plausible, however, that some implementations' free()s sometimes have such an additional effect. Again, this is implementation-specific.