I think a recent update with either Colab or Cuda is throwing off the YOLOv4 model. I have previously built this notebook and trained a complete model with it with no problems but now when I run the exact same code, with no changes, I get this problem:
672 x 672
try to allocate additional workspace_size = 65.03 MB
CUDA allocate done!
Loaded: 2.962808 seconds - performance bottleneck on CPU or Disk HDD/SSD
CUDA Error Prev: an illegal memory access was encountered
Darknet error location: ./src/dark_cuda.c, check_error, line #81
CUDA Error Prev: an illegal memory access was encountered: File exists
Has anyone else had this problem? It seems to be very recent because no one has been able to figure it out so far. This is how I'm running my darknet:
!./darknet detector train data/obj.data cfg/custom-yolov4-detector.cfg yolov4.conv.137 -dont_show -map -clear
It appears that Google Colab has reduced the memory allowed for free notebooks.
Try changing the batch size and subdivisions in your config.
batch=32 and subdivisions=8 solved this problem for me.