[SOLVED] In Video Does Sample Means mdat data?

In Video Does Sample Means mdat data?

I use isobmff Library for Muxing. (https://github.com/MPEGGroup/isobmff) I make h264(avc) to mp4 file. (muxing) when I muxing, I use ISOAddMediaSamples this function.

In Video, Does Sample Means only mdat data ? or Is it possible that Sample includes SEI etc.. ?

What box does sei information matching to in the mp4 box? (avcc box? trak box etc...)

Solution

If you wrap (mux) an H.264 (AVC) stream into an MP4 - you (can) remove some information from the H.264 stream.

For example PPS/SPS: you have to put them into the MP4 container (avcc). You can remove them from the stream.

PPS/SPS can be missing from the Media Data (mdat).

Stream Enhancement Information (SEI) stay in the H.264 stream and will be in the Media Data (mdat).

Stream Enhancement Information (SEI) are diverse and can contain many different information and even vendor specific information. For example:

timecode
close captions
camera make and model
etc.

You could parse Stream Enhancement Information (SEI), look for close captions and then create a separate close caption track inside your MP4.

Typically you don't touch Stream Enhancement Information (SEI) and just pass them through and SEI will be in mdat.