I am suing different methods such as wavelet(different parameters) , FFT to denoise 1 D single . I using skimage.metrics
to evaluate denoising as shown in snippet below. (here signal
is original noisy signal)
import skimage.metrics as M
def get_denoise_metrics(signal, clean_signal):
data_range = (min(signal), max(signal))
d = {
'normalized_root_mse' : [np.round(M.normalized_root_mse(signal, clean_signal), 4)],
'peak_signal_noise_ratio' : [round(M.peak_signal_noise_ratio(signal, clean_signal, data_range =data_range[1]), 4)],
'structural_similarity' : [np.round(M.structural_similarity(signal, clean_signal, data_range =data_range[1]), 4)],
}
return d
Since I have 3 metrices for each denoising method (total no. of methods are greater than 10) I am using , How can I create a composite score so based on that I select based method .
All depends on what you want to achieve, there is no best answer, because for someone RMSE will be most important, for others peak SNR
That being said, I give you three propositions:
Score methods and create ranking for each metric separately.
composite_score = place_in_ranking_metric1 + place_in_ranking_metric2 + ...
Use your knowledge and intuition to find arbitrary weights.
composite_score = w1 * score_metric1 + w2 * score_metric2 + ...
If you think it through this should work better than simple ranking. For each method normalize score based on best result.
score_metric1 = score_metric1 / best_score_metric1
score_metric2 = score_metric2 / best_score_metric1
...
composite_score = score_metric1 + score_metric2 + ...
Sometimes, you may want to transform some scores using either log
, exp
, sqrt
or pow
. Or some other way. It is, again, very arbitrary.