Machine Learning model Wrong Prediction

I have a machine learning classifier from create ML. The model is trained with 3400 samples and overall is impressive in how accurate it is. However The model ever so often makes a prediction that is wrong and I can't seem to figure out how I add this into the model so that it does not have such high confidence on some of these wrong prediction. How/can I re-train the model with this new piece of data that is wrong in its classification that it is not of this classification? Should I be making a new classification folder and add these wrong classification to it or is there a way in training the model to pass in as a not classification type so it can try to understand the difference between them?

Disclaimer: So far I have not been working with createML. As I understand from the question you provide your training data via a folder structure and training and evaluation is then done by pressing a button. Correct me if I have made some bad assumption.

It would be nice to know what kind of model / architecture you are using and how your training samples look like.

To me your issue sounds like these poorly predicted samples might be underrepresented in your overall dataset. There are a few tricks you can try here:

1. Just duplicate (copy-paste within your training sample folder) these samples for your training process, so as to double the error feedback on those particular samples.
2. A more sophisticated approach would be to apply data augmentation strategies on those samples, and then add the augmented samples to your training data set.

Depending on your sample type, there are augmentation packages for Python available, and they are pretty easy and straight forward to use.