machine-learningsvmsoftmaxpacket-loss

Effect of addition of a data point on the loss of SVM v.s. Softmax


Is it possible to add a new data point to a training set that would leave the SVM loss unchanged, but not the Softmax classifier loss?


Solution

  • In the SVM if the new data point has a score that is out of the margin range from the correct class score the loss wouldn't change but in the Softmax loss if the score of the new added datapoint be close to +infinity it will adversely affect the loss, but definitely the loss of Softmax will change.