I am trying to remove spikes noise from a fuel level signal and for that, I want to implement a low pass filter in python, I have my frequency domain plot of the signal but I cannot understand how to choose the cutoff frequency or bandpass in case I should be using a band pass filter. My data's sampling frequency is 1sample/3min. Here is my code to generate the frequency domain response and its output.
#Fourier Transform of signal
fuel_vol_fft=np.fft.fft((fuel_vol-np.mean(fuel_vol)),axis=0) / fuel_vol.shape[0]
freq=np.fft.fftfreq(fuel_vol.shape[0],d=3) #sampling time step = 3 min- sampling rate 1/3 cycle/min
plt.figure(figsize=(20,7))
plt.plot(freq,fuel_vol_fft.real)
plt.grid()
plt.xlabel('Frequency (Hz)')
plt.ylabel('Amplitude')
plt.title('Frequency Domain Fuel Signal - Presuming regulary sampled')
plt.show()
Here is the frequency domain view of my signal:
With a closeup of frequency domain: And finally here is a close-up of a portion of my original time-domain signal with spikes showing noise.
I would suggest using a moving average filter here.
It also has a Lowpass-Effect, since it is basically integrating the discrete input signal by summing up a defined number of values and normalizing them to the number of values that have been processed:
https://en.wikipedia.org/wiki/Moving_average
It is also very easy to implement, it is just a weighted sum, and for the cause of smoothening a signal it is perfectly well-suited. Probably the moving average over 10 or 20 samples should be good for your application.