iOS -- How to change video resolution in webRTC?

Question

I am trying to change local video resolution in webRTC. I used following method to create local video tracker:

-(RTCVideoTrack *)createLocalVideoTrack {
   RTCVideoTrack *localVideoTrack = nil;
   RTCMediaConstraints *mediaConstraints = [[RTCMediaConstraints alloc] initWithMandatoryConstraints:nil                                                                        optionalConstraints:nil];
   RTCAVFoundationVideoSource *source =
   [self.factory avFoundationVideoSourceWithConstraints:mediaConstraints];
   localVideoTrack =
   [self.factory videoTrackWithSource:source
                           trackId:@"ARDAMSv0"];
   return localVideoTrack;
}

I set the mandatory constraint as follow, but it doesn't work:

@{@"minFrameRate":@"20",@"maxFrameRate":@"30",@"maxWidth":@"320",@"minWidth":@"240",@"maxHeight":@"320",@"minHeight":@"240"};

Could anyone help me?

Answer 1

Latest SDK builds don't provide factory method to build capturer with constraints any more. Solution should be based on AVCaptureSession instead and WebRTC will take care about CPU and bandwidth utilization.

For this you need to keep reference to your RTCVideoSource that was passed to capturer. It has method:

- (void)adaptOutputFormatToWidth:(int)width height:(int)height fps:(int)fps;

Calling this function will cause frames to be scaled down to the requested resolution. Also, frames will be cropped to match the requested aspect ratio, and frames will be dropped to match the requested fps. The requested aspect ratio is orientation agnostic and will be adjusted to maintain the input orientation, so it doesn't matter if e.g. 1280x720 or 720x1280 is requested.

var localVideoSource: RTCVideoSource?

You may create your video track this way:

func createVideoTrack() -> RTCVideoTrack? {
    var source: RTCVideoSource
    if let localSource = self.localVideoSource {
        source = localSource
    } else {
        source = self.factory.videoSource()
        self.localVideoSource = source
    }
    let devices = RTCCameraVideoCapturer.captureDevices()
    if let camera = devices.first, 
        // here you can decide to use front or back camera

        let format = RTCCameraVideoCapturer.supportedFormats(for: camera).last, 
        // here you have a bunch of formats from tiny to up to 4k, find 1st that conforms your needs, i.e. if you usemax 1280x720, then no need to pick 4k

        let fps = format.videoSupportedFrameRateRanges.first?.maxFrameRate 
        // or take smth in between min..max, i.e. 24 fps and not 30, to reduce gpu/cpu use {

        let intFps = Int(fps)
        let capturer = RTCCameraVideoCapturer(delegate: source)
        capturer.startCapture(with: camera, format: format, fps: intFps)
        let videoTrack = self.factory.videoTrack(with: source, trackId: WebRTCClient.trackIdVideo)
        return videoTrack
    }
    retun nil
}

And when you need to change resolution, you can tell this video source to do "scaling".

func changeResolution(w: Int32, h: Int32) -> Bool {
    guard let videoSource = self.localVideoSource else {
        return false
    }
    // TODO: decide fps
    videoSource.adaptOutputFormat(toWidth: w, height: h, fps: 30)
    return true
}

Camera will still capture frames with resolution providd in format to startCapture. And if you care about resource utilization, then you can also use next methods prior to adaptOutputFormat.

// Stops the capture session asynchronously and notifies callback on completion.
- (void)stopCaptureWithCompletionHandler:(nullable void (^)(void))completionHandler;

// Starts the capture session asynchronously.
- (void)startCaptureWithDevice:(AVCaptureDevice *)device format:(AVCaptureDeviceFormat *)format fps:(NSInteger)fps;