pythontensorflowloss-functionprecision-recall

How to use F-score as error function to train neural networks?


I am pretty new to neural networks. I am training a network in tensorflow, but the number of positive examples is much much less than negative examples in my dataset (it is a medical dataset). So, I know that F-score calculated from precision and recall is a good measure of how well the model is trained. I have used error functions like cross-entropy loss or MSE before, but they are all based on accuracy calculation (if I am not wrong). But how do I use this F-score as an error function? Is there a tensorflow function for that? Or I have to create a new one?

Thanks in advance.


Solution

  • I think you are confusing model evaluation metrics for classification with training losses.

    Accuracy, precision, F-scores etc. are evaluation metrics computed from binary outcomes and binary predictions.

    For model training, you need a function that compares a continuous score (your model output) with a binary outcome - like cross-entropy. Ideally, this is calibrated such that it is minimised if the predicted mean matches the population mean (given covariates). These rules are called proper scoring rules, and the cross-entropy is one of them.

    Also check the thread is-accuracy-an-improper-scoring-rule-in-a-binary-classification-setting

    If you want to weigh positive and negative cases differently, two methods are

    I recommend just using simple oversampling in practice.