[SOLVED] -12909 error decoding h264 stream with intra-refresh

-12909 error decoding h264 stream with intra-refresh

I'm making an iOS app that decodes an h264 stream using video-toolbox. I create the stream with ffmpeg on a PC and send it to an iPhone using RTP. It's working nicely when I use this command to create it:

ffmpeg -y -f:v rawvideo -c:v rawvideo -s 1280x720 -pix_fmt bgra -r 30 -an -i - -pix_fmt yuv420p -c:v libx264 -tune zerolatency -preset fast -b:v 5M -refs 1 -g 30 -profile:v high -level 4.1 -f rtp rtp://192.168.1.100:5678

The iPhone receives and displays all the frames. However, when I enable intra-refresh

-intra-refresh 1

decoding fails with error code -12909 (-8969 on simulator) when VTDecompressionSessionDecodeFrame() is called.

I take care of processing UDP packets to extract NAL Units, so I triple checked this process and discarded a problem with this part of the code.

I didn't find any info about Video-toolbox not supporting intra-refresh, so the question is, does Video-toolbox support intra-refresh? and if it does, am I missing something in the ffmpeg side that makes the stream not supported by Video-toolbox? Do I have to add something to the CMVideoFormatDescriptionRef apart from creating it with SPS and PPS data using CMVideoFormatDescriptionCreateFromH264ParameterSets()?

Solution

Yes, Video-toolbox supports intra-refresh
No, nothing to do with ffmpeg
No, don't need to do anything special with CMVideoFormatDescriptionRef

I figured it out, I was creating a new VTDecompressionSession each time I was receiving SPS and PPS NALUs, so the decoder was loosing the context.

It was working without intra-refresh because in that case a complete I-Frame is received right after SPS and PPS, so it doesn't need context from previous frames.

With intra-refresh enabled, only the first frame is a complete I-Frame, then the decoder relies on context from previous frames and must use the same VTDecompressionSession.