rkerasmontecarloconv-neural-networkdropout

Monte Carlo (MC) dropout in Keras with R

How to implement Monte Carlo dropout with Keras in Convolutional neural networks to estimate predictive uncertainty as suggested by YARIN GAL? I am using R.R-Code is here

I am fitting the model in small batches and want to evaluate the model in small batches as well with Monte Carlo dropout.Could not find any hint in Keras documentation.BTW, I trained my model with flag training=TRUE.

Thanks

Solution

  • Regular dropout only drops neurons randomly at training time, not a test time, so this is the default behavior of the Dropout class. If you want MC dropout, you need to use training=TRUE at test time as well, and you must run the forward pass multiple times: this will give you a distribution of predictions, which you can use as you please, for example to compute the mean.

    I'm not familiar enough with R, so here is the class I use instead of the standard Dropout class:

    class MCDropout(keras.layers.Dropout):
    def call(self, inputs, training=None):
        return super(MCDropout, self).call(inputs, training=True)