AVPlayer - Retrieve audio tracks in all formats

Question

I'm working on a tvOS application where I'm using the AVPlayer to play an HLS playlist which provides audio in two formats for some languages. For example:

• French (AAC)
• French (EC-3)
• English

I'm trying to display a custom dialog that would allow the users to select between each of these tracks.

The playlist looks like this:

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-mp4a.40.2",NAME="Français",DEFAULT=YES,AUTOSELECT=YES,LANGUAGE="fr",URI="..."
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-ec-3",NAME="Français",DEFAULT=NO,AUTOSELECT=YES,LANGUAGE="fr",URI="..."
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-mp4a.40.2",NAME="English",DEFAULT=NO,AUTOSELECT=YES,LANGUAGE="en",URI="..."

The problem is that, from what I can tell, the AVPlayer groups the tracks by language and it never returns all the 3 tracks.

(lldb) po player?.currentItem?.asset.mediaSelectionGroup(forMediaCharacteristic: .audible)

▿ Optional<AVMediaSelectionGroup>
  - some : <AVAssetMediaSelectionGroup: 0x283961940, options = (
    "<AVMediaSelectionKeyValueOption: 0x2839a5a00, language = fr, mediaType = 'soun', title = Français, default = YES>",
    "<AVMediaSelectionKeyValueOption: 0x2839a5b00, language = en, mediaType = 'soun', title = English>"
), allowsEmptySelection = YES>

I went deeper into the French item (player?.currentItem?.asset.mediaSelectionGroup(forMediaCharacteristic: .audible)?.options.first) but I still couldn't find anything useful. I also tried looking at other fields from the AVPlayer with no success.

Even when I use the AVPlayerViewController I only see two audio tracks to choose from.

Is there any way to get all the available audio tracks?

Answer 1

So the issue here is actually the playlist. If you check the HLS specification, there are some notes explaining this under the Rendition Groups subsection of EXT-X-MEDIA (https://datatracker.ietf.org/doc/html/draft-pantos-hls-rfc8216bis-07#section-4.4.6.1.1)

A Playlist MAY contain multiple Groups of the same TYPE in order to provide multiple encodings of that media type. If it does so, each Group of the same TYPE MUST have the same set of members, and each corresponding member MUST have identical attributes with the exception of the URI and CHANNELS attributes.

Generally, the way to think about it is, anything within a given GROUP-ID is selectable by the user (and so AVFoundation reveals it to you). But which GROUP-ID is playing is selected by the player, and (for this scenario) this is determined by the AUDIO attribute of the EXT-X-STREAM-INF that the player has selected.

If you want the surround audio track to be selectable, then it needs to exist in the same GROUP-ID as the rest of the AUDIO tracks. If you don't have control of the manifest, you can test this by re-writing the GROUP-ID of the surround French track (using something like Charles Proxy), from audio-ec-3 to audio-mp4a.40.2; it should appear in AVFoundation after that. But a word of warning, for the HLS to remain valid, the CODECS attribute of all of the EXT-X-STREAM-INF tags have to be updated to include the CODECS defined by the surround track (otherwise playback failure may occur).

If you want to leave it up to the player to select, and you do not have a surround English track, you still have to give the English option in the surround group to remain valid HLS, but you can just leave the URI identical to the one defined in the stereo group. Again CODECS will have to be updated in this scenario.

This video from WWDC gives a good explanation of all of this (relevant section starts around 42:39): https://developer.apple.com/videos/play/wwdc2018/502/