AVFoundation capturing video with custom resolution

Question

I'm writing application on OS X, which will capture frames from camera. Is it possible to set capture setting using AVCaptureDevice.activeFormat property? I had tried this, but it didn't work (session preset overrides it).

I found that on IOS it is possible with setting SessionPreset in AVCaptureSession to AVCaptureSessionPresetInputPriority.

The main purpose is to choose more detailed video resolutions than presets.

Answer 1

Updated: April 08, 2020.

In macOS (unlike iOS), a capture session can automatically configure the capture format after you make changes. To prevent automatic changes to the capture format use lockForConfiguration() method. Then call the beginConfiguration() method, set properties (choose one preset out of a dozen, for instance AVCaptureSessionPresetiFrame960x540) and after that call the commitConfiguration() method. In the end you need to put unlockForConfiguration() after changing a device properties.

Or follow these steps:

• Call lockForConfiguration() to acquire access to the device’s config properties.

• Change the device’s activeFormat property (as mentioned above & below).

• Begin capture with the session’s startRunning() method.

• Unlock the device with the unlockForConfiguration().

startRunning() and stopRunning() methods must be invoked to start and stop the flow of your data from the inputs to the outputs, respectively.

You must also call lockForConfiguration() before calling the AVCaptureSession method startRunning(), or the session's preset will override the selected active format on the capture device.

However, you might hold onto a lock, without releasing that lock, if you require the device properties to remain unchanged.

Here are details in developer's documentation lockForConfiguration().

If you attempt to set the active format to one not present in the accessible formats, will throw an invalidArgumentException.

Also, there's an explanation how to change properties: macOS AVFoundation Video Capture

In AVCaptureDevice there are two properties. formats and activeFormat. format will return an NSArrary of AVCaptureDeviceFormat with contains all formats exposed by cam. You select any one format from this list and set it to activeFormat. Make sure that you set the format after you receive the exclusive access to the devlce by calling AVCaptureDevice lockForConfigration. After you set the format release the lock with AVCaptureDevice unlockForConfigration. Then start the AVCaptureSession which will give you the video frames of the format you set.

AVCaptureFormat is a wraper for CMFormatDescription. CMVideoFotmatDescription is the concreete subclass of CMFormatDescription. Use CMVideoFormatDescriptionGetDimentions() to get the width and height in the set format. Use CMFormatDescriptionGetMediaSubType() to get the video codec. For raw fotmats video codec mostly is yuvs or vuy2. For compressed formats its h264, dmb1(mjpeg) and many more.

Here's a macOS code snippet written in Swift:

import Cocoa
import AVFoundation

class ViewController: NSViewController, 
                      AVCaptureVideoDataOutputSampleBufferDelegate {

    override func viewDidAppear() {
        super.viewDidAppear()

        setupCameraSession()
        view.layer?.addSublayer(previewLayer)
        cameraSession.startRunning()
    }

    lazy var cameraSession: AVCaptureSession = {
        let session = AVCaptureSession()
        session.sessionPreset = AVCaptureSession.Preset.hd1280x720
        return session
    }()

    lazy var previewLayer: AVCaptureVideoPreviewLayer = {
        let preview =  AVCaptureVideoPreviewLayer(session: self.cameraSession)

        preview.bounds = CGRect(x: 0, 
                                y: 0, 
                            width: self.view.bounds.width, 
                           height: self.view.bounds.height)

        preview.position = CGPoint(x: self.view.bounds.midX, 
                                   y: self.view.bounds.midY)

        preview.videoGravity = AVLayerVideoGravity.resize
        return preview
    }()

    func setupCameraSession() {

        let captureDevice = AVCaptureDevice.default(for: AVMediaType.video)
    
        do {
            let deviceInput = try AVCaptureDeviceInput(device: captureDevice!)
        
            guard let camera = AVCaptureDevice.default(for: .video) 
            else { return }

            // acquire exclusive access to the device’s properties
            try camera.lockForConfiguration()            
            cameraSession.beginConfiguration()
        
            camera.focusMode = .continuousAutoFocus
            camera.flashMode = .on
            camera.whiteBalanceMode = .continuousAutoWhiteBalance
        
            if (cameraSession.canAddInput(deviceInput) == true) {
                cameraSession.addInput(deviceInput)
            }
        
            let dataOutput = AVCaptureVideoDataOutput()
            dataOutput.videoSettings = [(kCVPixelBufferPixelFormatTypeKey as NSString) : 
                                         NSNumber(value: kCVPixelFormatType_420YpCbCr8BiPlanarFullRange as UInt32)] as [String : Any]
            dataOutput.alwaysDiscardsLateVideoFrames = true
        
            if (cameraSession.canAddOutput(dataOutput) == true) {
                cameraSession.addOutput(dataOutput)
            }

            let preset: AVCaptureSession.Preset = .hd4K3840x2160
            cameraSession.sessionPreset = preset
        
            cameraSession.commitConfiguration()
            camera.unlockForConfiguration()
        
            let queue = DispatchQueue(label: "blah.blah.blah")
            dataOutput.setSampleBufferDelegate(self, queue: queue)

        } catch let error as NSError {
            NSLog("\(error.localizedDescription)")
        }
    }
}

And here's a code snippet written in Objective-C setting min and max fps:

myCamera = NULL;
 
if ( NULL != myCamera ) {

    if ( [ myCamera lockForConfiguration: NULL ] ) {

        [ myCamera setActiveVideoMinFrameDuration: CMTimeMake( 1, 12 ) ];
 
        [ myCamera setActiveVideoMaxFrameDuration: CMTimeMake( 1, 25 ) ];
 
        [ myCamera unlockForConfiguration ];
    }
}
return ( NULL != myCamera );