I am training my first transferred learning model(yay!) and I am having trouble getting the model to stop training when the validation loss hasn't changed by more than 0.1 in more than 3 epochs.
Here is the relevant block of code
early_stopping = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3, min_delta = 0.1)
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'],
callbacks=[early_stopping])
EPOCHS = 100
history = model.fit(training_batches,
epochs=EPOCHS,
validation_data=validation_batches)
And here are some of the logs:
Epoch 32/100
155/155 [==============================] - 21s 134ms/step - loss: 0.0042 - accuracy: 0.9998 - val_loss: 0.3407 - val_accuracy: 0.9012
Epoch 33/100
155/155 [==============================] - 21s 133ms/step - loss: 0.0040 - accuracy: 0.9998 - val_loss: 0.3443 - val_accuracy: 0.9000
Epoch 34/100
155/155 [==============================] - 21s 134ms/step - loss: 0.0037 - accuracy: 0.9998 - val_loss: 0.3393 - val_accuracy: 0.9019
Epoch 35/100
155/155 [==============================] - 21s 135ms/step - loss: 0.0031 - accuracy: 1.0000 - val_loss: 0.3396 - val_accuracy: 0.9000
Epoch 36/100
155/155 [==============================] - 21s 134ms/step - loss: 0.0028 - accuracy: 1.0000 - val_loss: 0.3390 - val_accuracy: 0.9000
Epoch 37/100
155/155 [==============================] - 21s 133ms/step - loss: 0.0026 - accuracy: 1.0000 - val_loss: 0.3386 - val_accuracy: 0.9025
Epoch 38/100
155/155 [==============================] - 21s 133ms/step - loss: 0.0024 - accuracy: 1.0000 - val_loss: 0.3386 - val_accuracy: 0.8994
Epoch 39/100
155/155 [==============================] - 21s 133ms/step - loss: 0.0022 - accuracy: 1.0000 - val_loss: 0.3386 - val_accuracy: 0.9019
Questions:
EDIT
It doesn't work because you placed the callback parameter in the wrong method call. (and in fact, I received an invalid argument error when fitting the model with callbacks passed to compile. Thus I'm not sure why your model compiled without problem.)
It should be inside your fit method, as shown below. Note that it is recommended to set verbose = 1 in your early stopping configuration, so that it will prints out the early stopping log.
early_stopping = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3, min_delta = 0.1, verbose = 1)
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
EPOCHS = 100
history = model.fit(training_batches,
epochs=EPOCHS,
callbacks=[early_stopping], # call back should be here!
validation_data=validation_batches)
On your second question, custom callback is possible, you can refer to the example from documentation here. Basically you got to define your early stopping logic in on_epoch_end.
On a side note, I think you shouldn't be early stopping on multiple metrics, pick one that matters (i.e. the metric that you are optimizing - val_accuracy) and just monitor that. There are even sources which discourage early stopping & instead, treat epoch as a tunable hyperparameter. See this discussion thread on Reddit which I found useful.