Here is mostly what we do, firstly we transform signal from time domain to frequency domain, then we will operate on the frequency domain with different filters, such as band filter, low pass filter, all pass filter.
What I don't understand is that all pass filter is mostly for phase shifting and magnitude gain/loss, I think for phase shifting and magnitude operation, we can also do these in time domain, then why do we need to do it in frequency domain?
you seem to be missing a important point here: Nearly every operation can be done in frequency domain OR in time domain. Multipling becomes convulution and vice versa. So (digital) filtering can be done by multipication of the Signalspectrum with the Transferfunction of your filter OR by convoluting your time domain signal with the Impulseresponse of your filter.