Comparing MSE loss and cross-entropy loss in terms of convergence
For a very simple classification problem where I have a target vector [0,0,0,....0] and a prediction vector [0,0.1,0.2,....1] would cross-entropy loss converge better/faster or would MSE loss?
When I plot them it seems to me that MSE loss has a lower error margin. Why would that be?
Or for example when I have the target as [1,1,1,1....1] I get the following:
You sound a little confused...
- Comparing the values of MSE & cross-entropy loss and saying that one is lower than the other is like comparing apples to oranges
- MSE is for regression problems, while cross-entropy loss is for classification ones; these contexts are mutually exclusive, hence comparing the numerical values of their corresponding loss measures makes no sense
- When your prediction vector is like
[0,0.1,0.2,....1](i.e. with non-integer components), as you say, the problem is a regression (and not a classification) one; in classification settings, we usually use one-hot encoded target vectors, where only one component is 1 and the rest are 0 - A target vector of
[1,1,1,1....1]could be the case either in a regression setting, or in a multi-label multi-class classification, i.e. where the output may belong to more than one class simultaneously
On top of these, your plot choice, with the percentage (?) of predictions in the horizontal axis, is puzzling - I have never seen such plots in ML diagnostics, and I am not quite sure what exactly they represent or why they can be useful...
If you like a detailed discussion of the cross-entropy loss & accuracy in classification settings, you may have a look at this answer of mine.
- Keras: Difference between Kernel and Activity regularizers
- How to fix the learning-rate for Huggingface´s Trainer?
- Roboflow Vs. Darknet for generating weight file and creating the model
- What is the difference between backpropagation and reverse-mode autodiff?
- Load Registered Component in Azure ML for Pipeline using Python sdk v2
- classifiers in scikit-learn that handle nan/null
- Encoding two categorial data present in same dataset in Deep learning
- Machine learning in q/kdb+
- Why do the sensitivity (recall) values differ between classification_report and precision_recall_fscore_support in a loop?
- How do you specify the bfloat16 mixed precision with the Intel Extension for PyTorch?
- mlflow doesn't autolog artifacts while logging images
- Choosing top k models using GridSearchCV in scikit-learn
- Calculating pairwise distances between entries in a `torch.tensor`
- Reusing a feature to split regression decision tree's nodes
- Fine tuning LayoutLmv3 using Cord-V2 dataset
- What does model.eval() do in pytorch?
- ElasticNetCV in Python: Get full grid of hyperparameters with corresponding MSE?
- Why is the accuracy for my Keras model always 0 when training?
- Polars - issues with performance - attempting to create a new dataframe per row
- Keras model.export() fails because of NoneType shapes in model
- Meta-feature analysis: split data for computation on available memory
- Single-headed transformer model, output and target tensor sizes don't match
- How to add the bounding box values to the labels text files during prediction with a trained YOLO-V8 instance segmentation model?
- How to calculate correlation of colours in a dataset?
- Convert array of indices to one-hot encoded array in NumPy
- Import custom prediction routine to vertex AI pipeline
- what is the difference between bigram and unigram text features extraction
- How to map the results of Principal Component Analysis back to the actual features that were fed into the model?
- How to calculate optimal batch size?
- Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample