iOS WebRTC How to replace input audio with custom source

Question

I am trying to replace the A/V source using the native google WebRTC. I'm using the latest pod of GoogleWebRTC and using the current Google demo project.

Replacing the video source was pretty easy and quick, the audio input not so much.

The default mic input is setup in the ARDAppClient like so

  RTCMediaConstraints *constraints = [self defaultMediaAudioConstraints];
  RTCAudioSource *source = [_factory audioSourceWithConstraints:constraints];
  RTCAudioTrack *track = [_factory audioTrackWithSource:source
                                            trackId:kARDAudioTrackId];
  [_peerConnection addTrack:track streamIds:@[ kARDMediaStreamId ]];

I'd like to swap out the source with anything I want to, I want to take any audio source I want (A song, ReplayKit app audio), get into a require format, and hook into the WebRTC pipeline without making a mess of things.

The ReplayKit Broadcast extension doesn't seem to have any hooks on audio,

- (void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer
               withType:(RPSampleBufferType)sampleBufferType {
  switch (sampleBufferType) {
    case RPSampleBufferTypeVideo:
      [self.capturer didCaptureSampleBuffer:sampleBuffer];
      break;
    case RPSampleBufferTypeAudioApp:
      break;
    case RPSampleBufferTypeAudioMic:
      break;
    default:
      break;
  }
}

It seems like I should be able to send an AudioApp or AudioMic CMSamplebuffer somewhere...?

What I want to avoid is altering the WebRTC libs and compiling my own static framework, but it seems more and more likely I'll have to.

I can't find anywhere in the WebRTC libs that use the mic as a CaptureDevice via a CMSampleBufferRef. I have found a AVAudioSession capture flow, but hooking into that is not forthcoming.

Recent release notes state

Bug
8466
Objc interface for peer connection factory does not allow external    audio device module to be used.
PeerConnection (iOS)

This seems to state any external audio devices (like I want to do) is not by default accepted, so if I'm understanding this correctly I have to alter the libs.

Is there ANY way I can easily swap out the audio source? And if not, any guidance on where to look first with altering the WebRTC libs would be extremely helpful.

Answer 1

You can use this fork of WebRTC which was created specifically to solve this issue :~)

https://github.com/pixiv/webrtc/blob/87.0.4280.142-pixiv0/README.pixiv.en.md