caffepycaffecaffe2

Place some nodes of the same network on GPU and others on CPU?

When defining a network in Caffe/Caffe2, can you place some of the nodes on the CPU and others on GPU? If so, how?

(If your answer pertains a specific version of Caffe, please specify which)

Solution

  • Generally speaking, the answer is NO: you cannot configure the device for each layer independently for the reasons Pooya Davoodi and Harsh Wardhan described.

    However, if you look at specific layers, you might sometimes get the behavior you look for. For instance, if your solver is configured to run on GPU, but you have a layer in your net that does not have a GPU implementation, then this layer will run on the CPU (with all the overhead described in Harsh Wardhan's answer).
    One such layer is a "Python" layer: This layer runs only on CPU and you may have your word2vec implementation there.
    Alternatively, you may write your own layers without GPU implementation making sure they run only on CPU.

    BTW, are you using caffe2? Are you okay with their PATENTS clause?!
    UPDATE: it seems like fb decided to soften caffe2's license. Well done!