ffmpegmpeg-dashdash.jsshaka

FFMPEG/DASH-LL creates audio and video chunks at different rates; player is confused (404 errors)

I'm trying to create a MPEG-DASH "live" stream from a static file to test various low latency modes. The DASH muxer in FFmpeg creates two AdaptationSets, one for video chunks and one for audio chunks.

However, the audio and video chunk files are not created at the same rate (should they be?). ie, here stream0 are the video chunks and stream1 are the audio chunks. After a few seconds of running, the webroot directory contains:

chunk-stream0-00001.m4s  chunk-stream1-00001.m4s  
chunk-stream0-00002.m4s  chunk-stream1-00002.m4s  
chunk-stream0-00003.m4s  chunk-stream1-00003.m4s  
chunk-stream0-00004.m4s  chunk-stream1-00004.m4s  
                         chunk-stream1-00005.m4s  
                         chunk-stream1-00006.m4s  
                         chunk-stream1-00007.m4s  
                         chunk-stream1-00008.m4s  
                         chunk-stream1-00009.m4s  
master.mpd  
init-stream0.m4s  
init-stream1.m4s

The stream doesn't load (or play) on either dash.js or shaka-player and there are lots of 404 (Not Found) errors for the video chunks. The player is requesting chunks from both stream0 and stream1 in sequence, ie, stream0-001 + stream1-001, then stream0-002 + stream1-002 and so on.

But since stream0 only goes from 001 to 004, there are lots of 404 errors as it tries to load stream0-005 through 009.

The gap gets wider after letting FFmpeg run for a while. eg, stream0 is 62 to 75 but stream1 is 174 to 187. Reloading the player page at this point fails with dash.all.debug.js:15615 [2055][FragmentController] No video bytes to push or stream is inactive. and shows 404 errors stream0 chunk 188 (which doesn't exist yet!)

enter image description here

The FFmpeg command was adopted from DASH streaming from the top-down:

ffmpeg -re -i /mnt/swdevel/TestStreams/H264/ThreeHourMovie.mp4 \
-c:v libx264 -x264-params keyint=120:scenecut=0 -b:v 1M -c:a copy \
-f dash -dash_segment_type mp4 \
 -seg_duration 2 \
 -target_latency 3 \
 -frag_type duration \
 -frag_duration 0.2 \
 -window_size 10 \
 -extra_window_size 3 \
 -streaming 1 \
 -ldash 1 \
 -use_template 1 \
 -use_timeline 0 \
 -write_prft 1 \
 -fflags +nobuffer+flush_packets \
 -format_options "movflags=+cmaf" \
 -utc_timing_url "/pelican/testPlayers/time.php" \
 master.mpd

And the dash.js player code is very simple:

const srcUrl = "../ottWebRoot/playerTest/master.mpd"; 

var player = dashjs.MediaPlayer().create();

let autoPlay = false;
player.initialize(document.querySelector("#videoTagId"), srcUrl, autoPlay);

player.updateSettings(
{
    streaming :
    {
        lowLatencyEnabled : true,
        liveDelay : 2,
        jumpGaps : true,
        jumpLargeGaps : true,
        smallGapLimit : 1.5,
    }
});

To provide the UTCTiming element in the manifest, the small time.php URL returns a UTC time from the web server:

<?php
    print gmdate("Y-m-d\TH:i:s\Z");
?>

(It also shows 404 errors for the latest stream1/audio chunk, that's likely a different problem)

I'm not sure what to try next. Any and suggestions greatly appreciated.

EDIT I

The suggestion by @Anonymous Coward to change the key interval improved things a lot. The chunks for stream0 and stream1 are in lock-step and have identical sequence numbers.

However, there are still many 404 errors, both on initial page load (without pressing play) and during playback.

I ran watch -n 1 ls -lt <webRootFolder and compared side-by-side to the errors in the browser console. It's hard to compare but it looks like the browser is trying to fetch files "on the play edge" which haven't yet been created by FFmpeg. See the pic below.

How do I instruct the browser to wait just a bit more before fetching the edge chunks?

enter image description here

EDIT II

Using shaka-player instead of dash.js plays properly without 404 errors. Configured as:

    player.configure(
    {
        streaming: 
        {
            lowLatencyMode: true,
            inaccurateManifestTolerance: 0,
            rebufferingGoal: 0.1,
        }
        
    });

Client

Server

(For reference, here is the mpd file generated by FFmpeg)

<?xml version="1.0" encoding="utf-8"?>
<MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns="urn:mpeg:dash:schema:mpd:2011"
    xmlns:xlink="http://www.w3.org/1999/xlink"
    xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd"
    profiles="urn:mpeg:dash:profile:isoff-live:2011"
    type="dynamic"
    minimumUpdatePeriod="PT500S"
    availabilityStartTime="2021-05-24T14:50:00.263Z"
    publishTime="2021-05-24T15:22:45.335Z"
    timeShiftBufferDepth="PT50.0S"
    maxSegmentDuration="PT2.0S"
    minBufferTime="PT5.0S">
    <ProgramInformation>
    </ProgramInformation>
    <ServiceDescription id="0">
        <Latency target="3000" referenceId="0"/>
    </ServiceDescription>
    <Period id="0" start="PT0.0S">
        <AdaptationSet id="0" contentType="video" startWithSAP="1" segmentAlignment="true" bitstreamSwitching="true" frameRate="24/1" maxWidth="1280" maxHeight="682" par="15:8" lang="und">
            <Resync dT="200000" type="0"/>
            <Representation id="0" mimeType="video/mp4" codecs="avc1.64081f" bandwidth="1000000" width="1280" height="682" sar="1023:1024">
                <ProducerReferenceTime id="0" inband="true" type="captured" wallClockTime="2021-05-24T14:50:00.263Z" presentationTime="0">
                    <UTCTiming schemeIdUri="urn:mpeg:dash:utc:http-xsdate:2014" value="/pelican/testPlayers/time.php"/>
                </ProducerReferenceTime>
                <Resync dT="5000000" type="1"/>
                <SegmentTemplate timescale="1000000" duration="2000000" availabilityTimeOffset="1.800" availabilityTimeComplete="false" initialization="init-stream$RepresentationID$.m4s" media="chunk-stream$RepresentationID$-$Number%05d$.m4s" startNumber="1">
                </SegmentTemplate>
            </Representation>
        </AdaptationSet>
        <AdaptationSet id="1" contentType="audio" startWithSAP="1" segmentAlignment="true" bitstreamSwitching="true" lang="und">
            <Resync dT="200000" type="0"/>
            <Representation id="1" mimeType="audio/mp4" codecs="mp4a.40.2" bandwidth="116317" audioSamplingRate="48000">
                <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2" />
                <ProducerReferenceTime id="1" inband="true" type="captured" wallClockTime="2021-05-24T14:50:00.306Z" presentationTime="0">
                    <UTCTiming schemeIdUri="urn:mpeg:dash:utc:http-xsdate:2014" value="/pelican/testPlayers/time.php"/>
                </ProducerReferenceTime>
                <Resync dT="21333" type="1"/>
                <SegmentTemplate timescale="1000000" duration="2000000" availabilityTimeOffset="1.800" availabilityTimeComplete="false" initialization="init-stream$RepresentationID$.m4s" media="chunk-stream$RepresentationID$-$Number%05d$.m4s" startNumber="1">
                </SegmentTemplate>
            </Representation>
        </AdaptationSet>
    </Period>
    <UTCTiming schemeIdUri="urn:mpeg:dash:utc:http-xsdate:2014" value="/pelican/testPlayers/time.php"/>
</MPD>
Solution

  • The input, ThreeHourMovie.mp4, has a frame rate of 24Hz (see AdaptationSet@frameRate), so setting keyint to 120 will give an IDR every five seconds at the output of x264.

    You are instructing the DASH muxer (via seg_duration) to output and signal 2s segments, but this isn't possible since a segment must start with an IDR (as mentioned in the guide you linked). So the muxer outputs 5s segments, but signals 2s in the manifest, which is clearly incorrect.

    Change keyint to the correct multiple of seg_duration (ie keyint=48 in this case), and it will probably start working.