Prevent AVPlayerController from hijacking MPRemoteCommandCenter

Question

I have a view controller with two separate players, an audio player with AVAudioPlayer and a video player with AVPlayerViewController.
When the view is loaded both players are initialized as well with their sources but stay paused until the user interacts with one of them.

I'm trying to make this app able to play the audio from AVAudioPlayer in background, so I have set the option in the capabilities of the app and it's working but I have some major issues.

The first issue is caused by setting

AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playback, mode: .default)

which is required for background audio, but it enables videos to play in background as well, which is not what I want. Is there some way I can prevent that?

I then set the controls for the Lockscreen and Control Center with MPRemoteCommandCenter specifying that the audio player should start or pause when the user interacts with the controls. If I only use the audio without ever starting the player and move between Control Center, Lockscreen and in-app player everything works fine.

But if I start playing the video with AVPlayerViewController, pause it, start the audio player and go to the Lockscreen, the commands are wired to the video, even though the video isn't even playing.
Nowhere in the code that is run for the controls I mention the video player (I also tried removing the code altogether with no luck).

Is that expected behaviour? Is there some way to disable this?

TL;DR
I have an audio and a video player and want only the audio player to be able to play in background but for some reason the controls from MPRemoteCommandCenter give priority to the video player and I want to change/disable this behaviour.

Answer 1

Ok so for anyone trying to work this out, after a bit of fiddling I found out that AVPlayerController has a property updatesNowPlayingInfoCenter set by default on true.
Setting that to false prevents most of the issues I was having.

One issue I kept having was that when playing an audio file and then switching to video, closing the app would still show the player controls, with data mixed from both audio and video.
I worked around this by switching AVAudioSession.sharedInstance().category from ambient to playback when playing audio and switching back to ambient when playing video.