Azure Kinect Recording Color Format

I'm attempting to read an Azure Kinect recording and save images from the frames. But, it is not possible to set the color_format, which causes problems when using imwrite.

I have read the recording documentation here: https://learn.microsoft.com/en-us/azure/Kinect-dk/azure-kinect-recorder.

By default, the format seems to be K4A_IMAGE_FORMAT_COLOR_MJPG. But I am unsure what parameter to pass in when creating the material. For BGRA32 it is CV_8UC4 and for depth images it is CV_16U.

I assume there are two ways to solve this problem, either by setting the color_format or figuring out what parameter is correct for the default format made by the recording.

You can access the rgb with OpenCV as if it were a normal webcam:

VideoCapture cap(0); // open the default camera
cap.set(CV_CAP_PROP_FRAME_WIDTH, 3840);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 2160);
if (!cap.isOpened())  // check if we succeeded
    return -1;
Mat frame, img;
for (;;)
{

    cap >> frame; // get a new frame from camera
    cout << frame.cols << " x " << frame.rows << endl;
    resize(frame, img, Size(), 0.25, 0.25);
    imshow("frame", img);
    if (waitKey(30) >= 0) break;
}

No k4a function is called, no need to set the color format.

If you want to use their SDK with jpeg format, they provide a function in one of their sample codes:

long WriteToFile(const char *fileName, void *buffer, size_t bufferSize)
{
    cout << bufferSize << endl;
    assert(buffer != NULL);

    std::ofstream hFile;
    hFile.open(fileName, std::ios::out | std::ios::trunc | std::ios::binary);
    if (hFile.is_open())
    {
        hFile.write((char *)buffer, static_cast<std::streamsize>(bufferSize));
        hFile.close();
    }
    std::cout << "[Streaming Service] Color frame is stored in " << fileName << std::endl;

    return 0;
}

You just call:

image = k4a_capture_get_color_image(capture);
WriteToFile("color.jpg", k4a_image_get_buffer(image), k4a_image_get_size(image));

Finally, you can set the format to RGBA32:

config.color_format = K4A_IMAGE_FORMAT_COLOR_BGRA32;

and convert it into a OpenCV Mat:

color_image = k4a_capture_get_color_image(capture);
    if (color_image)
    {
        uint8_t* buffer = k4a_image_get_buffer(color_image);        // get raw buffer
        cv::Mat colorMat(Hrgb, Wrgb, CV_8UC4, (void*)buffer, cv::Mat::AUTO_STEP);
        //do something with colorMat
        k4a_image_release(color_image);
    }

More details on the last option here: How to convert k4a_image_t to opencv matrix? (Azure Kinect Sensor SDK)

The data is slightly better with the last solution, but the buffer is significantly larger (33M vs ~1.5M) for 3840x2160.