For example, with 32 bit denormalized floats, the exponent would be -126. What is the significance of -126 in this context?
The top bit of the mantissa (sgnificand) can not be binary 1. anymore, so the top bit is at most binary 0.1.
With an exponent of -126, the highest denormal is 0.1111... x 2^-126. The next higher value is the smallest normal, with value 1.000... x 2^-126.
They chose that value so the ranges of denormals and normals neither overlap, nor leave an ugly gap between them.