ffmpeg

Trying to get pixels from a transparent AVI file to use them in a Raylib image but AVFrame comes out

I was trying to open a video file with FFmpeg's API using C++. So far all the steps up until calling av_read_frame has no errors:

This following code is where I do the rest of the checks, to get the pixel data:

int bufferSize = av_codec_ctx->width * av_codec_ctx->height * 4;
unsigned char* data = new unsigned char[bufferSize];
    
    while (av_read_frame(av_format_ctx, av_packet) >= 0) {
        if (av_packet->stream_index == video_stream_idx) {
            if (avcodec_send_packet(av_codec_ctx, av_packet) == 0) {
                if (avcodec_receive_frame(av_codec_ctx, av_frame) == 0) {

                    memcpy(data, av_frame->data[0], bufferSize);

                    break; // We only need the first frame
                }
            }
        }
    }

The issue is, up until ... if (avcodec_receive_frame(av_codec_ctx, av_frame) == 0) { ... and while inspecting the data, the packet does bear information in the buffer, but once it's received into the frame, it turns into all zeroes.

None of the function calls return any error so it should be working, yet the packet has no data, and consequently the frame comes out empty.

Why could this be? Should I keep looping on avcodec_receive_frame(av_codec_ctx, av_frame) until it comes out not empty?

Solution

  • Ok so, depending on the type of video you're using, you'll want to approach this differently, but it's important to check if the video data is encoded or not.

    If the data is encoded, skip both avcodec_send_packet and avcodec_receive_frame, since the information is already available inside the av_packet->data array.

    Important things to consider: