I am taking 1-D wavelet transform of data. How can I make this faster? I have 1.4 million samples and 32 features.
def apply_wavelet_transform(data):
ca,cd=pywt.dwt(data[0,:],'haar')
for i in range(1,data.shape[0]):
ca_i,__=pywt.dwt(data[i,:],'haar')
ca=np.vstack((ca,ca_i))
return ca
Consider I don't care about memory usage as much as speed of execution.
This is a common mistake. You don't want to append rows to an array one at a time, because each iteration requires copying the entire array. Complexity: O(N**2). Much better to keep the intermediate results in a list and form the array at the end. This is better because lists do not require their elements to be contiguous in memory, so no copying is required.
def apply_wavelet_transform(data):
results_list = []
for row in data:
ca, cd = pywt.dwt(row, 'haar')
results_list.append(ca)
result = np.array(results_list)
return result