I am encoding a live 25 FPS 950x540 H.264 video-only stream using the latest (built from source) FFmpeg, 10 segments in the manifest, each of 5 seconds, and consuming that stream on another machine on my local LAN in [incognito to avoid cache issues] Chrome with the latest hls.js (cdn.jsdelivr.net/npm/hls.js@latest). The encoded video plays but not reliably; I have two problems:
hls.js client starts to be unable to sync with the video stream after a few minutes, complaining of being beyond the end of the sync window and, on re-sync, getting empty buffers.hls.js client will not sync to the stream unless I start the stream and the client at the same time: if the hls.js client tries to use the stream a few 10's of seconds after it has started it again fails to sync with the stream. The only way I can start the stream successfully is if the .m3u8 file is deleted first, so the browser cannot find the file, then the client reads the file and begins streaming. Chrome's F12 window says things like "[stream-controller]: Playback: 0.000 is located too far from the end of live sliding playlist: 49.92, reset currentTime to : 44.920", which might be expected but then more mysteriously says "[abr] buffer is empty, optimal quality level 0", something which definitely doesn't happen in the succese case.Obviously I have something fundamentally wrong, I just can't figure out what. My HLS flags to FFmpeg are discont_start and omit_endlist and the client has been directed not to cache. Here is a sample .m3u8 output file:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:NO
#EXT-X-TARGETDURATION:5
#EXT-X-MEDIA-SEQUENCE:30
#EXTINF:5.120000,
http://10.10.1.16/watchdog30.ts
#EXTINF:4.880000,
http://10.10.1.16/watchdog31.ts
#EXTINF:5.120000,
http://10.10.1.16/watchdog32.ts
#EXTINF:4.880000,
http://10.10.1.16/watchdog33.ts
#EXTINF:5.120000,
http://10.10.1.16/watchdog34.ts
#EXTINF:4.880000,
http://10.10.1.16/watchdog35.ts
#EXTINF:5.120000,
http://10.10.1.16/watchdog36.ts
#EXTINF:4.880000,
http://10.10.1.16/watchdog37.ts
#EXTINF:5.120000,
http://10.10.1.16/watchdog38.ts
#EXTINF:4.880000,
http://10.10.1.16/watchdog39.ts
I've tried also adding the flags independent_segments and inclusion of program_date_time, which have not helped.
What have I missed?
My full code (FFMpeg is driven at its C API) can be found here: https://github.com/RobMeades/watchdog/blob/83e718390f779a7ef98d2be009784e8be5deaf5f/software/watchdog.cpp#L933. This code is running on a Raspberry Pi 5 under Raspbian latest.
And the answer was... tell the FFmpeg libx264 video codec to -tune zerolatency.
At the FFmpeg C API this is done with av_dict_set(&codecOptions, "tune", "zerolatency", 0), where codecOptions is the AVDictionary you will then pass as the last parameter to avcodec_open2().
Why? I couldn't tell you. It has taken me nearly a week of trying everything I could try before I found this. With this single option added the hls.js client synchronizes, and re-synchronizes, with the HLS stream every time, under all circumstances. Without it, hls.js will not gain initial sync to a HLS stream if it is started just a few seconds after the stream has begun and, won't regain synchronization if it should lose it.
Note that I did try running hls.js with lowLatency: false but that did not fix the problem.
We live and learn.