videoffmpegmp4quicktimefmp4

Fragmented MP4 not playing in ffplay/QuickTime/Safari, but in VLC

I encoded a fMP4-Video (HEVC) in Swift using VideoToolbox and CoreMedia. The resulting fragmented MP4 is only playing in VLC.

The library I am using to write the fMP4 is an HEVC-adapted version of this GitHub-Project: https://github.com/krad/morsel

Process of encoding and writing:

  1. VideoToolbox: Encoding SampleBuffer from the camera (VTCompressionSession, as described in WWDC 2014 Session 513)
  2. Using CoreMedia-Functions (e.g. CMVideoFormatDescriptionGetHEVCParameterSetAtIndex, CMVideoFormatDescriptionGetDimensions) to get the encoded streams metadata. I am writing the content of CMFormatDescriptionGetExtension(description, extensionKey: "SampleDescriptionExtensionAtoms" as CFString)["hvcC"] directly to the hvcC box. This box is obviously written correctly, when I alter one bit of the box of the mp4, QuickTime throws an error.
  3. I am appending the sample buffer data to the morsel-library, which manages fragmentation and creates the moof and mdat atoms.

The result file is playable in VLC, when I just let the playback "run" without doing anything. When I scroll the time in VLC, VLC crashes and the playback stops.

It is also sort of "playable" in Safari and QuickTime (no error message shown, the playback window opens and the correct length of the file is shown, I can even change to playback time / play/pause, but there is no video shown. The window remains empty. And that is the problem I have. I need to get an fMP4 running in Safari and QuickTime.

The file is not playing at all in ffplay (same problem when converting with ffmpeg). The line where the playback time is shown remains nan M-V: nan fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0 , there are no error messages. The playback just does not start. When playing the file with ffplay, this is the output:

ffplay version 4.3 Copyright (c) 2003-2020 the FFmpeg developers
  built with Apple clang version 11.0.3 (clang-1103.0.32.62)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: mp41mp42isomhlsf
    creation_time   : 2020-08-03T20:41:08.000000Z
  Duration: N/A, bitrate: N/A
    Stream #0:0(und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv), 1920x1080, SAR 1:1 DAR 16:9, 1000000000.00 tbr, 1000000000.00 tbn, 1000000000.00 tbc (default)
    Metadata:
      creation_time   : 2020-08-03T20:41:08.000000Z
      handler_name    : video
    nan M-V:    nan fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0

My file has this structure:

ftyp
moov
    mvhd
    trak
        tkhd
        mdia
            mdhd
            hdlr
            minf
                vmhd
                dinf
                    dref
                stbl
                    stsd
                    stts
                    stsc
                    stsz
                    stco
    mvex
        trex
moof-(1)
mdat
moof-(2)
mdat 
...

The result file I want to create should be like the fMP4 served within this HLS playlist: (Example: https://developer.apple.com/streaming/examples/advanced-stream-hevc.html, Playlist: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_adv_example_hevc/master.m3u8) This file has the following structure:

ftyp
moov
    mvhd
    trak
        tkhd
        mdia
            mdhd
            hdlr
            minf
                vmhd
                dinf
                    dref
                stbl
                    stsd
                    stts
                    stsc
                    stsz
                    stco
    trak (same structure as above)
    mvex
        trex
moof-(1)
mdat
moof-(2)
mdat 
...

This file from Apple plays perfectly in ffmpeg and QuickTime/Safari. Except for the second trak atom, it has the same structure.

The "good" file is: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_adv_example_hevc/v14/main.mp4 (It is intended for use in a HLS Playlist I've downloaded it with curl)

The "bad" file is: https://www.transfernow.net/ddl/fmp4_bad (link should work, tested it just now :) ). The video shows a slowly rotating sheet of paper.

Any suggestions what the problem with my file is? Thanks in advance!

Solution

  • The duration in moof->traf->trun->entries->duration is wrong.

    So is the moof->traf->tfhd->default_sample_duration.

    It looks more like a timestamp but it should be the frame duration.

    You only have one sample per fragment which is legal but pretty wasteful.