How to show HLS embedded captions on Exoplayer

Question

how do I enable and also select different subtitles that are embedded in a Vimeo video in HLS format using Exoplayer, ExoMedia or another player? In iOS this same video already presents the option of subtitles natively but in Android I cannot find means to implement it.

Answer 1

My answer here will look a lot like this one, so you might want to check that one out first.

ExoPlayer is the library you'll want for Android. It's a non- trivial task to get the subtitles to show, but the demo app for that library has all the code you'll need to get them working on an HLS video. More specifically the PlayerActivity class. You can go to HLS -> "Apple 16x9 basic stream" in the demo app and that video has a ton of subtitles (aka "text tracks").

Just to simplify their code so that it doesn't rely on the helper (and so you can see how it works just on closed captions) I've written/documented some of their code below.

private static class ClosedCaptionManager {

    ClosedCaptionManager(MappingTrackSelector mappingTrackSelector, SimpleExoPlayer player) {
        this.player = player;
        this.trackSelector = mappingTrackSelector;
    }

    SimpleExoPlayer player;
    MappingTrackSelector trackSelector;

    // These two could be fields OR passed around
    int textTrackIndex;
    TrackGroupArray trackGroups;

    ArrayList<Pair<Integer, Integer>> pairTrackList = new ArrayList<>();

    private boolean checkAndSetClosedCaptions() {
        // This is the body of the logic  for see if there are even video tracks
        // It also does some field setting
        MappedTrackInfo mappedTrackInfo = trackSelector.getCurrentMappedTrackInfo();
        if (mappedTrackInfo == null) {
            return false;
        }
        for (int i = 0; i < mappedTrackInfo.length; i++) {
            trackGroups = mappedTrackInfo.getTrackGroups(i);
            if (trackGroups.length != 0) {
                switch (player.getRendererType(i)) {
                    case C.TRACK_TYPE_TEXT:
                        textTrackIndex = i;
                        return true;
                }
            }
        }
        return false;
    }

    private void buildTrackList() {
        // This next part is actually about getting the list.
        // Below you'd be building up items in a list. This just does
        // views directly, but you could just have a list of track names (with indexes)
        for (int groupIndex = 0; groupIndex < trackGroups.length; groupIndex++) {
            TrackGroup group = trackGroups.get(groupIndex);
            for (int trackIndex = 0; trackIndex < group.length; trackIndex++) {
                if (trackIndex == 0) {
                    // Beginning of a new set, the demo app adds a divider
                }
                //CheckedTextView trackView = ...; // The TextView to show in the list
                // The below points to a util which extracts the quality from the TrackGroup
                //trackView.setText(DemoUtil.buildTrackName(group.getFormat(trackIndex)));
                Log.e("Thing", DemoUtil.buildTrackName(group.getFormat(trackIndex)));
                pairTrackList.add(new Pair<>(groupIndex, trackIndex));
            }
        }
    }

    private void onTrackViewClick(Pair<Integer, Integer> trackPair) {
        // Assuming you tagged the view with the groupIndex and trackIndex, you
        // can build your override with that info.
        Pair<Integer, Integer> tag = trackPair;
        int groupIndex = tag.first;
        int trackIndex = tag.second;
        // This is the override you'd use for something that isn't adaptive.
        // `override = new SelectionOverride(FIXED_FACTORY, groupIndex, trackIndex);`
        // Otherwise they call their helper for adaptives (HLS/DASH), which roughly does:
        int[] tracks = getTracksAdding(new MappingTrackSelector.SelectionOverride(
                        new FixedTrackSelection.Factory(), groupIndex, trackIndex),
                trackIndex
        );
        TrackSelection.Factory factory = tracks.length == 1
                ? new FixedTrackSelection.Factory()
                : new AdaptiveTrackSelection.Factory(BANDWIDTH_METER);

        MappingTrackSelector.SelectionOverride override =
                new MappingTrackSelector.SelectionOverride(factory, groupIndex, tracks);

        // Then we actually set our override on the selector to switch the text track
        trackSelector.setSelectionOverride(textTrackIndex, trackGroups, override);
    }

    private static int[] getTracksAdding(MappingTrackSelector.SelectionOverride override, int addedTrack) {
        int[] tracks = override.tracks;
        tracks = Arrays.copyOf(tracks, tracks.length + 1);
        tracks[tracks.length - 1] = addedTrack;
        return tracks;
    }
}

Then if you post the following code into the end of their initializePlayer() method, you'll get an idea of how these pieces fit together.

    final ClosedCaptionManager closedCaptionManager = new ClosedCaptionManager(trackSelector, player);

    new Handler().postDelayed(new Runnable() {
        @Override
        public void run() {
            boolean hasTracks = closedCaptionManager.checkAndSetClosedCaptions();
            if (hasTracks) {
                closedCaptionManager.buildTrackList();
                closedCaptionManager.onTrackViewClick(closedCaptionManager.pairTrackList.get(4));
            }
        }
    }, 2000);

The code above is super sloppy, but it should hopefully at least get you started in the right direction. I wouldn't recommend using any piece of what's written - it should mostly be to get an idea of how the different pieces fit together. What they have in the demo app is a little better in that their code is very reusable for the different track selection types (since you can have video, audio, and text tracks).

Answer 2

This works well!

TrackGroupArray trackGroups = mappedTrackInfo.getTrackGroups(rendererIndex);   

TrackSelectionArray currentTrackGroups = player.getCurrentTrackSelections();
TrackSelection currentTrackSelection = currentTrackGroups.get(rendererIndex);


    for (int groupIndex = 0; groupIndex < trackGroups.length; groupIndex++) {

        TrackGroup group = trackGroups.get(groupIndex);

        for (int trackIndex = 0; trackIndex < group.length; trackIndex++) {
            Format trackFormat = group.getFormat(trackIndex);


            if(currentTrackSelection!=null && currentTrackSelection.getSelectedFormat()==trackFormat){
                //THIS ONE IS SELECTED
            }




        }
    }

rendererIndex is 0 for Video, 1 for Audio and 2 for Subtitles/Text