OpenCv iOS camera preview single orientation only without stretching

Question

I'm writing a OpenCv Cordova plugin for iOS. I needed the camera preview to be fullscreen and to maintain aspect ratio for all iOS devices (iPhone, iPads).

I was able to achieve Portrait mode (see code) and it works perfectly on iPhone 6/plus but the camera preview is stretched slightly on iPads possibly due to the fact that AVCaptureSessionPresetHigh's supported resolution is 1280x720. Anyone has any ideas on how to achieve single orientation (landscape or portrait only) and maintain aspect ratio?

My current code for starting a camera is

- (void)viewDidLoad {
    [super viewDidLoad];
    // Do any additional setup after loading the view.

    // Initialize imageView to fullscreen
    imageView = [[UIImageView alloc] initWithFrame:[UIScreen mainScreen].bounds];
    [self.view addSubview:imageView];
    [imageView setContentMode:UIViewContentModeScaleAspectFill];
    [imageView setClipsToBounds:YES];

    self.videoCamera = [[FixedCvVideoCamera alloc] initWithParentView:imageView];
    self.videoCamera.delegate = self;
    self.videoCamera.defaultAVCaptureDevicePosition = AVCaptureDevicePositionBack;
    self.videoCamera.defaultAVCaptureSessionPreset = AVCaptureSessionPresetHigh;
    self.videoCamera.defaultAVCaptureVideoOrientation =  AVCaptureVideoOrientationPortrait;
    self.videoCamera.defaultFPS = 30;
    [self.videoCamera start];
}

Notice I'm using FixedCvVideoCamera instead of the OpenCv CvVideoCamera class. The reason is that I'm subclassing and overriding CvVideoCamera's layoutPreviewLayer function to keep portrait mode.

 - (void)layoutPreviewLayer;
{
    if (self.parentView != nil)
    {
        CALayer* layer = self->customPreviewLayer;
        CGRect bounds = self->customPreviewLayer.bounds;
        int rotation_angle = 0;

        switch (defaultAVCaptureVideoOrientation) {
            case AVCaptureVideoOrientationLandscapeRight:
                rotation_angle = 270;
                break;
            case AVCaptureVideoOrientationPortraitUpsideDown:
                rotation_angle = 180;
                break;
            case AVCaptureVideoOrientationLandscapeLeft:
                rotation_angle = 90;
                break;
            case AVCaptureVideoOrientationPortrait:
            default:
                break;
        }

        layer.position = CGPointMake(self.parentView.frame.size.width/2., self.parentView.frame.size.height/2.);
        layer.affineTransform = CGAffineTransformMakeRotation( DEGREES_RADIANS(rotation_angle) );
        layer.bounds = bounds;
    }
}

Thanks in advance.

Answer 1

I have solved it myself. The code bellow will help those who wanted a fixed Portrait Mode only and supporting front and back camera.

ViewController

- (void)viewDidLoad {
    [super viewDidLoad];
    // Do any additional setup after loading the view.

    // Initialize imageView to fullscreen
    imageView = [[UIImageView alloc] initWithFrame:[UIScreen mainScreen].bounds];
    [self.view addSubview:imageView];
    [imageView setContentMode:UIViewContentModeScaleAspectFill];
    [imageView setClipsToBounds:YES];

    self.videoCamera = [[FixedCvVideoCamera alloc] initWithParentView:imageView];
    self.videoCamera.delegate = self;
    self.videoCamera.defaultAVCaptureDevicePosition = AVCaptureDevicePositionBack;
    self.videoCamera.defaultAVCaptureSessionPreset = AVCaptureSessionPresetHigh;
    self.videoCamera.defaultAVCaptureVideoOrientation =  AVCaptureVideoOrientationPortrait;
    self.videoCamera.defaultFPS = 30;
    self.videoCamera.grayscaleMode = NO;

    [self.videoCamera start];
}

FixedCvVideoCamera : CvVideoCamera

- (void)layoutPreviewLayer;
{
    if (self.parentView != nil)
    {
        CALayer* layer = self->customPreviewLayer;
        CGRect bounds = self->customPreviewLayer.bounds;
        NSLog(@"[FixedCvVideoCamera]Custom Preview Layer bounds %fx%f", bounds.size.width, bounds.size.height);

        float previewAspectRatio = bounds.size.height / bounds.size.width;
        NSLog(@"[FixedCvVideoCamera]Preview aspect ratio %f", previewAspectRatio);

        //int rotation_angle = 0;

        layer.position = CGPointMake(self.parentView.frame.size.width/2., self.parentView.frame.size.height/2.);
        //layer.affineTransform = CGAffineTransformMakeRotation( DEGREES_RADIANS(rotation_angle) );

        // Get video feed's resolutions
        NSArray* devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
        AVCaptureDevice* device = nil;
        for (AVCaptureDevice *d in devices) {
            // Get the default camera device - should be either front of back camera device
            if ([d position] == self.defaultAVCaptureDevicePosition) {
                device = d;
            }
        }

        // Set the default device if not found
        if (!device) {
            device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
        }

        CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(device.activeFormat.formatDescription);
        CGSize resolution = CGSizeMake(dimensions.width, dimensions.height);
        if (self.defaultAVCaptureVideoOrientation == AVCaptureVideoOrientationPortrait || self.defaultAVCaptureVideoOrientation == AVCaptureVideoOrientationPortraitUpsideDown) {
            resolution = CGSizeMake(resolution.height, resolution.width);
        }
        NSLog(@"[FixedCvVideoCamera]Video feed resolution is %fx%f", resolution.width, resolution.height);

        float videoFeedAspectRatio = resolution.height / resolution.width;
        NSLog(@"[FixedCvVideoCamera]Video feed's aspect ratio is %f", videoFeedAspectRatio);

        // Set layer bounds to ASPECT FILL by expanding either the width or the height
        if (previewAspectRatio > videoFeedAspectRatio) {
            NSLog(@"[FixedCvVideoCamera] Preview is more rectangular than the video feed aspect ratio. Expanding width to maintain aspect ratio.");
            float newWidth = bounds.size.height / videoFeedAspectRatio;
            layer.bounds = CGRectMake(0, 0, newWidth, bounds.size.height);
        } else {
            NSLog(@"[FixedCvVideoCamera] Preview is equally or less rectangular (wider) than the video feed's aspect ratio. Expanding height bound to maintain aspect ratio.");
            float newHeight = bounds.size.width * videoFeedAspectRatio;
            layer.bounds = CGRectMake(0, 0, bounds.size.width, newHeight);
        }
    }
}