I've coded a Neural Network from scratch in Python and I am using Google Colaboratory to train it. However, if I enable GPU or TPU acceleration, the training is not faster.
When you search for examples online, all of them use Tensorflow and other libraries, and their training times are shorter with GPU than without it.
Am I doing it correctly or am I missing something and the GPU is not being used?
Just enabling GPU or TPU won't help your problem, you need to explicitly code them to run on GPU if you are not using any frameworks or libraries.