amazon-web-servicesmachine-learningclassificationamazon-machine-learning

How to interpret performance results of AWS Machine Learning Service?


I am working on a pilot with Amazon Web Service Machine Learning service and I have some soubts.

I have used a Binary Classifier model and, in my opinion, the histogram of the results obtained does not match the numerical results. According to the histogram, the distribution of False Positives is higher than the distribution of True Negatives but the numerical results do not present this behavior.

Histogram

Anyone can bring some insights into this matter?

Thank you,


Solution

  • This is the answer to my question from the Amazon Web Services Support team through their forums:

    After doing some digging around, I found that the Y-axis scaling is logarithmic for the histograms, which explains why a direct 1:1 area comparison of the true negatives and false positives would not be consistent with the numerical results. If we didn't display a logarithmic scale, my guess would be that most of your Y-axis would be dominated by the true negative and true positive results and the false positives and false negatives could be too small to noticeably see.

    Reference: https://forums.aws.amazon.com/message.jspa?messageID=733706

    If the Y-axis is logarithmic the results DO match with the provided histograms.