Everything I look up just tells me how to do complement operations/calculations in C.
I want to know what representation does C use internally and how does it handle with overflow.
C allows 3 representation for signed integers (https://port70.net/~nsz/c/c11/n1570.html#6.2.6.2p2):
- the corresponding value with sign bit 0 is negated (sign and magnitude);
- the sign bit has the value -(2M) (two's complement);
- the sign bit has the value -(2M- 1) (ones' complement).
Two's complement is most common.
Unsigned overflow wraps around the maximum value for the unsigned.
Signed overflow causes undefined behavior. I.e., it is assumed not to happen, and if you do make it happen, no guarantees can be made about your program's behavior.
Overflow in signed atomics is an exception: it is well defined and two's complement is mandated there: https://port70.net/~nsz/c/c11/n1570.html#7.17.7.5p3.