algorithmmachine-learningclassificationlogistic-regressionsoftmax

Difference between logistic regression and softmax regression


I know that logistic regression is for binary classification and softmax regression for multi-class problem. Would it be any differences if I train several logistic regression models with the same data and normalize their results to get a multi-class classifier instead of using one softmax model. I assume the result is of the same. Can I say : "all the multi-class classifier is the cascading result of binary classifiers". (except neuron network)


Solution

  • There are minor differences in multiple logistic regression models and a softmax output.

    Essentially you can map an input of size d to a single output k times, or map an input of size d to k outputs a single time. However, multiple logistic regression models are confusing, and perform poorer in practice. This is because most libraries (TensorFlow, Caffe, Theano) are implemented in low level compiled languages and are highly optimized. Since managing the multiple logistic regression models is likely handled at a higher level, it should be avoided.