I am testing an FCN-VGG16 network in Caffe. Instead of resizing the input images to a fixed size, I reshape the network each time to the image size. I tried this in both matcaffe and pycaffe. In both cases, it seems that it is able to run for small images (for example, 500x500). However, when I have a larger sized image (e.g. 1920 x 1080), I get this error "Check failed: error == cudaSuccess (2 vs. 0) out of memory". I am running this on a Quadro M6000 GPU with 12 GB memory. Any help/advice would be appreciated.
I know it's obvious from the message, but you do have to reduce the input size: the GPU doesn't have enough on-board memory for everything you fed it. Can you reduce your batch size? If not, do you really need the entire 2M pixels? Most models do quite well with cropped or reduced images. Do this as part of the input processing (on the CPU), and parametrize it for any sine input.
Resizing the network is expensive in memory, especially if you resize all layers, scaling with the input dimensions. In the case you gave, this will increase your memory requirements by roughly 8x.
As a check on the size you can have, watch the memory figures as you initialize the network; I believe that Caffe reports memory requirements by layer.