Is it possible to determine if a subtitle track is imaged based or text based with ffprobe

Question

I'm writing a script that burns subtitles into video files to prepare them for a personal stream I'm hosting. I'm having a hard time finding which type of subtitle is used in the file. I use ffprobe to get the files' information, and I can get stuff like the codec type, but I was wondering if there is a way to determine if a subtitle track is image based or text based. I can only think of getting a list of all possible codecs and match the codec type with this list but it would be very useful to have an info somewhere that can tell me "OK this is an image-based subtitle track", as when I burn I cannot use the same filters with ffmpeg to burn image vs. text subtitles.

Answer 1

Well, it partly depends on what OS you're using. In Linux, you can run the following command to get a list of all subtitle codecs supported by your version of ffmpeg:

ffmpeg -codecs | grep "^...S"

To narrow it down to which subtitle codecs your ffmpeg build is capable of encoding:

ffmpeg -codecs | grep "^..ES"

It sounds like you'll be interested foremost in which subtitles ffmpeg can decode:

ffmpeg -codecs | grep "^.D.S"

On my ffmpeg build (git-2020-08-31-4a11a6f), the command above displays the following result:

 DES... ass                  ASS (Advanced SSA) subtitle (decoders: ssa ass ) (encoders: ssa ass )
 DES... dvb_subtitle         DVB subtitles (decoders: dvbsub ) (encoders: dvbsub )
 DES... dvd_subtitle         DVD subtitles (decoders: dvdsub ) (encoders: dvdsub )
 D.S... eia_608              EIA-608 closed captions (decoders: cc_dec )
 D.S... hdmv_pgs_subtitle    HDMV Presentation Graphic Stream subtitles (decoders: pgssub )
 D.S... jacosub              JACOsub subtitle
 D.S... microdvd             MicroDVD subtitle
 DES... mov_text             MOV text
 D.S... mpl2                 MPL2 subtitle
 D.S... pjs                  PJS (Phoenix Japanimation Society) subtitle
 D.S... realtext             RealText subtitle
 D.S... sami                 SAMI subtitle
 D.S... stl                  Spruce subtitle format
 DES... subrip               SubRip subtitle (decoders: srt subrip ) (encoders: srt subrip )
 D.S... subviewer            SubViewer subtitle
 D.S... subviewer1           SubViewer v1 subtitle
 DES... text                 raw UTF-8 text
 D.S... vplayer              VPlayer subtitle
 DES... webvtt               WebVTT subtitle
 DES... xsub                 XSUB

Which of these are graphics-based/non-text? Most are text-based. Note that "text" can mean raw text (e.g. ASCII or UTF-8), XML, or HTML.

Image-based sub-title codecs in ffmpeg

• dvbsub
• dvdsub
• pgssub
• xsub

Text-based subtitle codecs in ffmpeg

• ssa,ass
• webvtt
• jacosub
• microdvd
• mov_text
• mpl2
• pjs
• realtext
• sami
• stl
• subrip
• subviewer
• subviewer1
• text
• vplayer
• webvtt

EIA Closed Captions EIA-608 is a Closed Caption format, and seems to be a bit of a bear to manage properly with ffmpeg.

eia_608              EIA-608 closed captions (decoders: cc_dec )

This Stack Overflow post offers one of the better explanations of how they function and how to manage them if you know they exist in a file: Can ffmpeg extract closed caption data