I've been given a 8051 based board with an embedded in-house operating system. I am using SDCC to create applications above the OS. And malloc is not available so I have to allocate memory statically. Why is that? Isn't malloc supposed to be on a dynamic library within the compiler?
TL;DR:
Why and when malloc() will not be available in C?
The only thing that can be said in general is that malloc() will be provided by every conforming, hosted C implementation, but there are other kinds, including another conforming kind.
Isn't malloc supposed to be on a dynamic library within the compiler?
Not exactly. malloc() is part of the C standard library, therefore it is provided by every conforming, hosted C implementation. A C implementation comprises a system for translating C source code into executable programs and a mechanism and environment for running the resulting programs. The former typically revolves around a compiler. The latter includes as much of the C standard library as the implementation provides, and this part is where malloc resides if it is available. Thus, no, malloc is technically not part of the compiler.
I'm sure that's not a distinction you meant to invoke, but it does bear on the answer. Note well that I said that malloc is provided by hosted implementations. These are the kind you ordinarily run into on general-purpose operating systems. They create programs that are launched in a standard way via the host OS, and they provide all the features of the C standard library in conjunction with the OS. But there are also freestanding implementations. One of the key differences is that freestanding implementations are excused from providing most of the standard library, including malloc().
You will commonly find freestanding implementations in use for and on embedded systems, such as yours. They are also used for OS kernels, boot loaders, and other such programs than run directly on bare metal. That your programs run on top of an OS makes your environment a bit of a Cadillac among embedded systems, but does not ensure that the C implementation is a hosted one. Inasmuch as it does not provide malloc, it cannot be a conforming hosted implementation, but it can be a conforming freestanding implementation. It ought to document which, if either, it claims to be. If it is freestanding but provides other standard library functions then you can consider that a luxury.