When I use ffmpeg to transcode some pure audio flv files to hls segments, the last .ts segment contains some decoding errors. It seems that it happens when the last segment's duration is short enough (about 1.5 second).
Here is my transcode command:
ffmpeg -i 111.flv -c:a aac -b:a 128k -f hls -hls_time 10.00 -hls_list_size 9999 tmp.m3u8
And output:
ffmpeg version n4.0.1-5-gb5106c5 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-4)
configuration: --prefix=/home/zx/ffmpeg_build --extra-cflags=-I/home/zx/ffmpeg_build/include --extra-ldflags='-L/home/zx/ffmpeg_build/lib -ldl' --bindir=/home/zx/bin --pkg-config-flags=--static --enable-gpl --enable-nonfree --enable-version3 --enable-libfdk_aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libspeex --enable-libvpx --enable-libx264 --disable-shared --enable-static --disable-debug
libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
libswscale 5. 1.100 / 5. 1.100
libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
Input #0, flv, from '111.flv':
Metadata:
major_brand : M4A
minor_version : 512
compatible_brands: isomiso2
date : 2018-05-31 01:31
encoder : Lavf57.25.100
Duration: 00:12:11.40, start: 0.000000, bitrate: 135 kb/s
Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[hls @ 0x25d4280] Opening 'tmp0.ts' for writing
[mpegts @ 0x26feb80] frame size not set
Output #0, hls, to 'tmp.m3u8':
Metadata:
major_brand : M4A
minor_version : 512
compatible_brands: isomiso2
date : 2018-05-31 01:31
encoder : Lavf58.12.100
Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:
encoder : Lavc58.18.100 aac
[hls @ 0x25d4280] Opening 'tmp1.ts' for writing
[hls @ 0x25d4280] Opening 'tmp.m3u8.tmp' for writing
[hls @ 0x25d4280] Opening 'tmp2.ts' for writing
[hls @ 0x25d4280] Opening 'tmp.m3u8.tmp' for writing
[hls @ 0x25d4280] Opening 'tmp3.ts' for writing
[hls @ 0x25d4280] Opening 'tmp.m3u8.tmp' for writing
[hls @ 0x25d4280] Opening 'tmp4.ts' for writing
......
[hls @ 0x25d4280] Opening 'tmp71.ts' for writing
[hls @ 0x25d4280] Opening 'tmp.m3u8.tmp' for writing
[hls @ 0x25d4280] Opening 'tmp72.ts' for writingx
[hls @ 0x25d4280] Opening 'tmp.m3u8.tmp' for writing
[hls @ 0x25d4280] Opening 'tmp73.ts' for writing
[hls @ 0x25d4280] Opening 'tmp.m3u8.tmp' for writing
[hls @ 0x25d4280] Opening 'tmp.m3u8.tmp' for writing
size=N/A time=00:12:11.39 bitrate=N/A speed=84.7x
video:0kB audio:11421kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[aac @ 0x25a0580] Qavg: 567.976
After transcoding, the duration of the last segment in m3u8 file is 1.39 second:
....
#EXTINF:9.984000,
tmp71.ts
#EXTINF:10.005333,
tmp72.ts
#EXTINF:1.386667,
tmp73.ts
#EXT-X-ENDLIST
But when I use ffprobe to parse the last segment, I got an error:
ffprobe -i tmp73.ts
ffprobe version n4.0.1-5-gb5106c5 Copyright (c) 2007-2018 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-4)
configuration: --prefix=/home/zx/ffmpeg_build --extra-cflags=-I/home/zx/ffmpeg_build/include --extra-ldflags='-L/home/zx/ffmpeg_build/lib -ldl' --bindir=/home/zx/bin --pkg-config-flags=--static --enable-gpl --enable-nonfree --enable-version3 --enable-libfdk_aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libspeex --enable-libvpx --enable-libx264 --disable-shared --enable-static --disable-debug
libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
libswscale 5. 1.100 / 5. 1.100
libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
tmp73.ts: Invalid data found when processing input
The input flv:
http://45.76.56.248/hls_transcode_fail.flv
md5sum: 4627bbb27bcb22c143ad4dfa47359650
Since my statistic module relies on the output of ffprobe, this exception will cause the entire transcoding task to fail. My temporary solution is to skip the last segment which is too short in the statistics phase. But I need a more reasonable and robust way to fix or bypass this issue. Any suggestion will be appreciated.
Somehow ffmpeg fails to add certain metadata to the last audio segment of an HLS stream, causing the Invalid data found when processing input error you're seeing when trying to probe the file with ffprobe or ffmpeg later, as they are unable to determine the format of the file, which you can see play out by setting -loglevel trace in your command.
The workaround is to add -f mpegts before your input to skip the format probing and force it to recognize the file as an MPEG-TS segment.
The solution is to run something like ffmpeg -f mpegts -i /tmp/original.ts -c copy /fixed.ts to fix the faulty segment.
You'll see the filesize increase in the fixed segment, as missing information is added to the file.
No idea why this is happening, but it has cost me a LOT of time to figure out what to do about it, hope it helps someone out.