Comparing MSE loss and cross-entropy loss in terms of convergence
For a very simple classification problem where I have a target vector [0,0,0,....0] and a prediction vector [0,0.1,0.2,....1] would cross-entropy loss converge better/faster or would MSE loss?
When I plot them it seems to me that MSE loss has a lower error margin. Why would that be?
Or for example when I have the target as [1,1,1,1....1] I get the following:
You sound a little confused...
- Comparing the values of MSE & cross-entropy loss and saying that one is lower than the other is like comparing apples to oranges
- MSE is for regression problems, while cross-entropy loss is for classification ones; these contexts are mutually exclusive, hence comparing the numerical values of their corresponding loss measures makes no sense
- When your prediction vector is like
[0,0.1,0.2,....1](i.e. with non-integer components), as you say, the problem is a regression (and not a classification) one; in classification settings, we usually use one-hot encoded target vectors, where only one component is 1 and the rest are 0 - A target vector of
[1,1,1,1....1]could be the case either in a regression setting, or in a multi-label multi-class classification, i.e. where the output may belong to more than one class simultaneously
On top of these, your plot choice, with the percentage (?) of predictions in the horizontal axis, is puzzling - I have never seen such plots in ML diagnostics, and I am not quite sure what exactly they represent or why they can be useful...
If you like a detailed discussion of the cross-entropy loss & accuracy in classification settings, you may have a look at this answer of mine.
- KMeans clustering in PySpark
- Is there a way to extract information from contracts using ML with including contract files and targeted strings as inputs and outputs?
- Training different stage of model with different loss
- How does one set the pad token correctly (not to eos) during fine-tuning to avoid model not predicting EOS?
- How can I properly split imbalanced dataset to train and test set?
- Deep learning for inferences in sequences
- How is log_softmax() implemented to compute its value (and gradient) with better speed and numerical stability?
- Is a neural network a lazy or eager learning method?
- bnlearn wrong dimensions for node
- Why is the accuracy for my Keras model always 0 when training?
- Can we use Pydantic models (BaseModel) directly inside model.predict() using FastAPI, and if not ,why?
- How to use MFCCs in Weka for audio classification?
- Creating a train, test split for data nested in multiple folders
- Custom Encoder and Decoder Layers within Keras Model Show As Unbuilt
- Bee swarm plot with SHAP values for Random forest
- Multiclass classifier using Keras from 'Deep Learning with Python' yields very different accuracy compared to what's in the textbook
- `TypeError: Unsupported integer size (0)` when attempted to save custom Keras model
- MLmetrics F1_Score function
- tensorflow TypeError: cannot unpack non-iterable float object
- AttributeError : 'tuple' has no attribute 'to'
- Understanding How YOLO is trained
- Confused about YOLO Process
- Columns are missing after imputing and creating dummy variables. How should I fix this?
- Implementing GridSearchCV and Pipelines to perform Hyperparameters Tuning for KNN Algorithm
- how do i set ‘random_state’ correctly so that my results are always the same?
- Pyinstaller created exe file cannot load Decision Tree model using joblib
- Training a CNN using ImageDataGenerator and training fails after the 2nd epoch
- Need to segment each number from the image separately
- Efficient way to compare two similar images
- Random Choice with Pytorch?