In the Deep Learning Specialization course by Dr. Andrew Ng, he mentions that stuff like ensemble models or boosting, though work fine in practice and are good for winning competitions, he recommends firmly against using them for developing real-world applications. Why is that? What's wrong with them?
I think you need to run multiple algos over your dataset and see which performs best, on the given data that is fed into the model. Ensemble models probably work best on most datasets, but that will certainly not be the case all the time. See the links below for some examples of how to test several models, to pick the best, based on the outcome of each, and not based on some kind of preconceived notion.