i am currently filtering some data and ran into trouble, when filtering smaller frequencies from a large trend.The Reverse FFTs seem to have large spikes at the beginning and the ending. Here is the Data before and after filtering smaller frequencies.
I have looked into the mathematic phenomenon and it is called the Gibbs phenomenon. Is there a way around this to clear the data of some overlying frequencies without getting this effect. Or is there even a workaround to keep the spikes as small as possible.
Here is the code BTW:
fourier_transformation= np.fft.fft(Sensor_4)
frequencies = np.fft.fftfreq(len(time), d=1/Values_per_second)
fourier_transformation[np.abs(frequencies) > 0.18] = 0
Sensor_4 = np.fft.ifft(fourier_transformation)
Following the suggestion of Martin Brown's comment, the following code subtracts a ramp before FFT and adds it back after IFFT (I needed to make up my own values for Sensor_4
, Values_per_second
, and time
, as the corresponding variables were missing in the question, so you might need to tune the parameters to match your actual signal):
import matplotlib.pyplot as plt
import numpy as np
# Create signal of Sensor_4 (similar to question)
T = 600
Values_per_second = 100 # Just to have a value here
time = np.arange(0, T, 1/Values_per_second)
Sensor_4 = 2 * np.sin(.25 * (2 * np.pi * time / T)) - 2.25
Sensor_4 += .1 * np.sin(1.9 * (2 * np.pi * time / T))
Sensor_4 += .1 * np.sin(2 * np.pi * time * 0.18)
frequencies = np.fft.fftfreq(len(time), d=1/Values_per_second)
# Original version
fourier_transformation = np.fft.fft(Sensor_4)
fourier_transformation[np.abs(frequencies) > 0.18] = 0
filtered_no_ramp = np.real(np.fft.ifft(fourier_transformation))
# Using ramp, as suggested in
# https://stackoverflow.com/questions/78788533#comment138912162_78788533
# (1) Create ramp, (2) subtract ramp, (3) FFT, (4) filter, (5) IFFT, (6) add ramp back
ramp = np.linspace(Sensor_4[0], Sensor_4[-1], len(Sensor_4)) # (1)
fourier_transformation = np.fft.fft(Sensor_4 - ramp) # (2, 3)
fourier_transformation[np.abs(frequencies) > 0.18] = 0 # (4)
filtered_ramp = np.real(np.fft.ifft(fourier_transformation)) + ramp # (5, 6)
# Show result
ax1 = plt.subplot(311, title="unfiltered")
plt.plot(time, Sensor_4)
plt.subplot(312, sharey=ax1, title="w/o ramp")
plt.plot(time, filtered_no_ramp)
plt.subplot(313, sharey=ax1, title="with ramp")
plt.plot(time, filtered_ramp)
plt.show()
Here is the result, before filtering (top), filtering without (center) and with (bottom) ramp subtraction: