My goal is augment my pre-existing image processing pipeline (written in Halide) with OpenCV functions such as NL means denoising. OpenCV functions will not be capable of using Halide's scheduling functionality, so my plan is to realize each Halide Func before each OpenCV stage. The remaining question is how to best convert from a Halide Image (the result of the Func realization) to an OpenCV Mat (as input to an OpenCV function) and from OpenCV Mat to Halide Image when done. My Halide Images are of type float and have 3 channels.
One obvious solution to this is to write functions which copy the data from one data type to the other, but this strikes me as wasteful. Not only will it take precious time to copy over the data, but it will also waste memory since the image will then be stored as two different data types. Is there a way to use pointers or data buffers to simply re-wrap the image data in a new format? Hopefully this process would be reversible so I can go from Halide to OpenCV, and then after the OpenCV function is done back to Halide.
Yes, you can avoid copying data. I see two possible approaches: either allocate memory yourself and refer to that memory in both an OpenCV Mat instance and a Halide buffer_t structure; or let OpenCV's Mat class allocate the memory and refer to that memory in a buffer_t structure.
For the first approach, you can use a Mat constructor that takes a data pointer:
float* data = new float[3 * width * height];
cv::Mat image(height, width, CV_32FC3, data, AUTO_STEP);
For the second approach, you can use the usual constructor or Mat::create method:
cv::Mat image(height, width, CV_32FC3);
Either way, you can use something like the following code to wrap the memory in a Halide buffer_t structure:
buffer_t buffer;
memset(&buffer, 0, sizeof(buffer));
buffer.host = image.data;
buffer.elem_size = image.elemSize1();
buffer.extent[0] = image.cols;
buffer.extent[1] = image.rows;
buffer.extent[2] = image.channels();
buffer.stride[0] = image.step1(1);
buffer.stride[1] = image.step1(0);
buffer.stride[2] = 1;
Now you should be able to operate on the same memory with both OpenCV and Halide functions.