machine-learningartificial-intelligenceimage-classificationsemisupervised-learningself-supervised-learning

What are the disadvantages of self-supervised learning in ML?


Self-supervised learning has been on the rise over the past few years. Compared to other learning methods such as supervised and semi-supervised, it does have an edge since it does not require labeled data.

I would like to know if self-supervised learning has any disadvantages and in what ways semi-supervised learning is better than it.


Solution

  • I think that the best way to illustrate this problem is to cite the great Yann LeCun:

    If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning (RL).

    The different types of ML can be very good or not depending on the case. For example, for robotics or autonomous driving problems, RL would be the ideal solution given the nature of these algorithms. However, for a recommender system or a stock price predictor, you could probably find better (and simpler) solutions in supervised and unsupervised learning.

    Reinforcement learning is very different from supervised and unsupervised learning in that it needs to be defined in terms of agent, states, and environment, rather than simply data (and labels in the case of supervised learning). Therefore, you will need those elements and define the interactions between them very carefully to train a good and reliable system that, as I mentioned above, might not be the most optimal (or even feasible) solution for the problem you are trying to solve.