[SOLVED] How do you display closed caption format of HLS video stream from an m3u8 URL

How do you display closed caption format of HLS video stream from an m3u8 URL

I'm working on a Roku and TVOS app that is going to play HLS videos (VOD and live) as well as MP4. According to the Roku docs EIA-608 is supported on both and should also work on TVOS.

My question is, given URL to the m3u8 how can I tell what specific format (EIA-608,WebVTT etc) of closed captioning is being used in each stream?

Contents of the main m3u8 (note 1st stream says no CC, but it really does have it):

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=380000,RESOLUTION=400x228,CODECS="avc1.66.30, mp4a.40.2",CLOSED-CAPTIONS=NONE
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/index_0_av.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=750000,RESOLUTION=640x360,CODECS="avc1.77.30, mp4a.40.2",CLOSED-CAPTIONS=NONE
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/index_1_av.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1044000,RESOLUTION=1280x720,CODECS="avc1.64001f, mp4a.40.2",CLOSED-CAPTIONS=NONE
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/index_2_av.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2127000,RESOLUTION=1280x720,CODECS="avc1.64001f, mp4a.40.2",CLOSED-CAPTIONS=NONE
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/index_3_av.m3u8

Contents of the 1st stream's m3u8

#EXTM3U
#EXT-X-TARGETDURATION:4
#EXT-X-ALLOW-CACHE:YES
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:4.000,
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/segment1_0_av.ts
...

I can use ffprobe -hide_banner to show the 1st program's stream has closed captioning. Ex:

Duration: 00:02:36.76, start: 0.100511, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 380000
    Stream #0:0: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 400x228 [SAR 1:1 DAR 100:57], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 380000
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, mono, fltp, 48 kb/s
    Metadata:
      variant_bitrate : 380000

However, as you can see, Program 0 > Stream 0 just says that is has Closed captions - it does not list the type/spec of closed captioning technology being used.

How do I display the format of the Closed Captions?

Solution

WebVTT is what is know as a 'side car', or 'out of band' format. Meaning captions are in a separate file that you must download. You can see the URL for this file in the m3u8. Here the caption are part of the video stream itself. The only supported format in this case is EIA-608. ffmpeg support for 608 is pretty limited. The best tool I know of to for dealing with 608 is libcaption (full disclosure, I wrote it). I recently added a ts2srt example program. Fair warning its still sorta betaish.