c++videowebmvp8vp9

vpx_codec_decode return true but vpx_codec_get_frame return no frame


I want to make a player that can run webm(vp8/vp9) files.

I make the player so that every time it finds a cluster send the cluster to the decoder and put all frames into memory.

Something strange happens, if I call the same cluster more times the decoder "vpx_codec_get_frame()" stop finding some frames and if I repeated this process multiple times no frames are decoded from the cluster but every time before I try to decode a frame I call "vpx_codec_decode()" and it returns true even if vpx_codec_get_frame() returns a null pointer;

Also, the decoding process is influenced by how many threads I use, for example, I get a different number of frames from a cluster if I use one thread vs 8.

I suspect that maybe is possible that threads are not finished at the same time a new buffer frame is coming so that can cause the problem.

Also, I test my player on an Mkv video that is obtained by merging three videos using mkvmerge tool, my player is supposed to open an mkv file with multiple tracks in it and display all track at the same time.

Here is the code : Initialized the decoder :

 VPXDecoder::VPXDecoder(const WebMDemuxer &demuxer, unsigned threads) :
    m_ctx(NULL),
    m_iter(NULL),
    m_delay(0),
    m_last_space(VPX_CS_UNKNOWN)
{
    if (threads > 8)
        threads = 8;
    else if (threads < 1)
        threads = 1;


    const vpx_codec_dec_cfg_t codecCfg = {
        threads,
        0,
        0
    };
    vpx_codec_iface_t *codecIface = NULL;

    switch (demuxer.getVideoCodec())
    {
        case WebMDemuxer::VIDEO_VP8:
            codecIface = vpx_codec_vp8_dx();
            break;
        case WebMDemuxer::VIDEO_VP9:
            codecIface = vpx_codec_vp9_dx();
            m_delay = threads - 1;
            break;
        default:
            return;
    }

    m_ctx = new vpx_codec_ctx_t;
    if (vpx_codec_dec_init(m_ctx, codecIface, &codecCfg, m_delay > 0 ? VPX_CODEC_USE_FRAME_THREADING : 0))
    {
        delete m_ctx;
        m_ctx = NULL;
    }

}

Here Is the code that calls vpx_codec_decode():

bool VPXDecoder::decode(const WebMFrame &frame)
{
    m_iter = NULL;
    this->decodead = !vpx_codec_decode(m_ctx, frame.buffer, frame.bufferSize, NULL, 0);
    return  decodead;
}

And finally where the image should be decoded :

VPXDecoder::IMAGE_ERROR VPXDecoder::getImage(Image &image)
{
    IMAGE_ERROR err = NO_FRAME;
    vpx_image_t *img = NULL;
    img = vpx_codec_get_frame(m_ctx, &m_iter);

    if (/*vpx_image_t *img = vpx_codec_get_frame(m_ctx, &m_iter)*/ img != NULL)
    {
        // It seems to be a common problem that UNKNOWN comes up a lot, yet FFMPEG is somehow getting accurate colour-space information.
        // After checking FFMPEG code, *they're* getting colour-space information, so I'm assuming something like this is going on.
        // It appears to work, at least.
        if (img->cs != VPX_CS_UNKNOWN)
            m_last_space = img->cs;
        if ((img->fmt & VPX_IMG_FMT_PLANAR) && !(img->fmt & (VPX_IMG_FMT_HAS_ALPHA | VPX_IMG_FMT_HIGHBITDEPTH)))
        {
            if (img->stride[0] && img->stride[1] && img->stride[2])
            {
                const int uPlane = !!(img->fmt & VPX_IMG_FMT_UV_FLIP) + 1;
                const int vPlane =  !(img->fmt & VPX_IMG_FMT_UV_FLIP) + 1;

                image.w = img->d_w;
                image.h = img->d_h;
                image.cs = m_last_space;
                image.chromaShiftW = img->x_chroma_shift;
                image.chromaShiftH = img->y_chroma_shift;

                image.planes[0] = img->planes[0];
                image.planes[1] = img->planes[uPlane];
                image.planes[2] = img->planes[vPlane];

                image.linesize[0] = img->stride[0];
                image.linesize[1] = img->stride[uPlane];
                image.linesize[2] = img->stride[vPlane];

                err = NO_ERROR;
            }
        }
        else
        {
            err = UNSUPPORTED_FRAME;
        }
    }
    return err;
}

In my project, I use the code from :

https://github.com/zaps166/libsimplewebm

Can some recommend another way to decode a vp8/9 frame or fined the problem to my code?


Solution

  • I found the problem, I build my player such as it request only one cluster at a time, but I didn't know that webm files need to have a keyframe at the beginning of every cluster, so the solution was to make a keyframe at the start of every cluster and use the decoder from there .