How to solve infinity/NaN loss for CTC? Is there something similar to zero_infinity in Pytorch for Keras/Tensorflow?

I am using a CTC loss for handwriting recognition in Tensorflow/Keras. However, just a few seconds after the model starts fitting, the loss goes to infinity.

I think this is because the input size isn't much bigger than the output size. Pytorch has an argument in its CTC loss called zero_infinity, but the Keras version of CTC does't have that. The Keras version is also very obscure and his little documentation. How do I go about solving this?

Edit: I have tried using gradient clipping with clipnorm and clipvalue as parameters for the optimizer, increasing the batch_size, using Adam instead of SGD, and using regularization and normalization. This has helped a bit, and it seemed like the loss was going down, but after a few batches, it always jumps to infinity.

I figured out why. The output size was not always greater than two times the input, so CTC could not generate blanks.