Is there an ExoPlayer + Leanback library example for using captions?

Question

I have found a few examples that work with Leanback and ExoPlayer and I have all that working but I can't get subtitles/captions to work. The newest Google example I could find (https://github.com/android/tv-samples) has a captions button on the Java sample but they never show up. The Kotlin example has a comment that says // TODO(owahltinez): handle captions.

I've tried these changes to one of the samples but it didn't help:

private void prepareMediaForPlaying(Uri mediaSourceUri) {
        String userAgent = Util.getUserAgent(getActivity(), "VideoPlayerGlue");
        DefaultDataSourceFactory defaultDataSourceFactory = new DefaultDataSourceFactory(getActivity(), userAgent);
        MediaSource mediaSource =
                new ExtractorMediaSource(
                        mediaSourceUri,
                        defaultDataSourceFactory,
                        new DefaultExtractorsFactory(),
                        null,
                        null);
        String subtitle = "https://subtitledomain/sintel-en.vtt";
        Uri uriSubtitle = Uri.parse(subtitle);
        MediaSource subtitleMediaSource = new SingleSampleMediaSource.Factory(defaultDataSourceFactory)
                .createMediaSource(uriSubtitle, Format.createTextSampleFormat(null, MimeTypes.TEXT_VTT, C.SELECTION_FLAG_FORCED, "n/a"), C.TIME_UNSET);
        mediaSource = new MergingMediaSource(mediaSource, subtitleMediaSource);
        mPlayer.prepare(mediaSource);
    }

And also this change:

    mTrackSelector = new DefaultTrackSelector(videoTrackSelectionFactory);
    DefaultTrackSelector.Parameters parameters = mTrackSelector.getParameters();
    mTrackSelector.setParameters(parameters.withSelectUndeterminedTextLanguage(true));

I've tried changing the language on the subtitle to EN and that didn't help. I feel like I'm probably just missing something small but I just don't know what it could be.

Thanks.

Edit: I made a branch and removed all Leanback code and just left ExoPlayer stuff untouched and used com.google.android.exoplayer2.ui.PlayerView in my Fragment instead of VideoFragment and subtitles worked without making any other change. So it is like I just need to enable them on the Leanback side somehow.

Answer 1

To get captions working you need to override lb_playback_fragment.xml and add a SubtitleView.

app/src/main/res/layout/lb_playback_fragment.xml

<?xml version="1.0" encoding="utf-8"?> 
<!--
    Copied from android source so that we can add the subtitle view
-->
<FrameLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/playback_fragment_root"
    android:layout_width="match_parent"
    android:transitionGroup="false"
    android:layout_height="match_parent">

    <androidx.leanback.widget.NonOverlappingFrameLayout
        android:id="@+id/playback_fragment_background"
        android:transitionGroup="false"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

    <androidx.leanback.widget.NonOverlappingFrameLayout
        android:id="@+id/playback_controls_dock"
        android:transitionGroup="true"
        android:layout_height="match_parent"
        android:layout_width="match_parent"/>

    <com.google.android.exoplayer2.ui.AspectRatioFrameLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_gravity="center">

        <com.google.android.exoplayer2.ui.SubtitleView
            android:id="@+id/leanback_subtitles"
            android:layout_width="match_parent"
            android:layout_height="match_parent" />

    </com.google.android.exoplayer2.ui.AspectRatioFrameLayout>

</FrameLayout>

Once this is done you can set up a DefaultTrackSelector to select the text track you want.

You can see the full code, based on the sample leanback player, in https://github.com/bennettpeter/android-MythTV-Leanfront. The change to add subtitles is in this commit.

Answer 2

Here is another approach--renderIndex = 0 for video, =1 for audio, =2 for subtitles/text.

TrackGroupArray trackGroups = mappedTrackInfo.getTrackGroups(rendererIndex);   

TrackSelectionArray currentTrackGroups = player.getCurrentTrackSelections();
TrackSelection currentTrackSelection = currentTrackGroups.get(rendererIndex);


for (int groupIndex = 0; groupIndex < trackGroups.length; groupIndex++) {

    TrackGroup group = trackGroups.get(groupIndex);

    for (int trackIndex = 0; trackIndex < group.length; trackIndex++) {
        Format trackFormat = group.getFormat(trackIndex);


        if(currentTrackSelection!=null && 
          currentTrackSelection.getSelectedFormat()==trackFormat){
            //THIS ONE IS SELECTED
        }

    }
}