androidvideo-streamingh.264rtmpflv

h264 to FLV conversion, ambiguous FrameType definition: What differentiates a Key Frame vs an Inter Frame?


I'm trying to write out an Annex B formatted h264 bitstream in an FLV container over RTMP in an Android app. After reading this SO answer and several more, I understand my Annex B bitstream first has to be converted to AVCC before being wrapped in an FLV container.

That said, the FLV container spec is a bit unclear in the wrapping of its video tags. It requires a FrameType which can be:

  1. keyframe (For AVC a seekable frame)
  2. interframe (For AVC a non seekable frame)
  3. Disposable interframe (h263 only)
  4. Generated keyframe (serverside only)
  5. Video info / command frame (The message string contains a byte, 0 or 1 meaning either start of client seeking video frame sequence, or end of client seeking frame sequence)

When writing IDR or non-IDR AVC frames, i.e VCL NAL units, it's clear that options 1 and 2 should be used respectively. However, my h264 bitstream also contains non-VCL NAL units, specifically SEI payload type 47 (Display Orientation). It's not clear whether this fits when going from AVC to FLV per the spec. Do I simply consider the SEI a non seekable frame? What is a "frame" from FLV's point of view?


Solution

  • It’s not ambiguous. What flv calls a frame, h264 calls an access unit. Like the link you posted said, an access unit can contain more than one NAL. An SEI by itself is not an access unit, hence it is not a frame, hence it should never be by itself in an flv tag, hence the question does not make sense.

    An SEI is sent along as part of the same frame (AU) payload and the flv frame type is that if the VCL nalus in that au.