Why is the "sign bit" included in the calculation of negative numbers?

I'm new to Swift and is trying to learn the concept of "shifting behavior for signed integers". I saw this example from "the swift programming language 2.1".

My question is: Why is the sign bit included in the calculation as well?

I experienced with several number combinations and they all works, but I don't seem to get reasons behind including sign bit in the calculation.

To add -1 to -4, simply by performing a standard binary addition of all eight bits (including the sign bit), and discarding anything that doesn't fit in the eight bits once you are done:

This is not unique to Swift. Nevertheless, historically, computers didn't subtract, they just added. To calculate A-B, do a two's-complement of B and add it to A. The two's-complement (negate all the bits and add 1) means that the highest order bit will be 0 for positive numbers (and zero), or 1 for negative numbers.