How to use AVCaptureSession with Slide Over and Split View in iOS 9?

Question

My team is developing a set of SDKs for barcode scanning, ID scanning and OCR. We use device's camera, specifically, AVCaptureSession, to obtain video frames on which we perform our processing.

We're exploring new iOS 9 multitasking features Slide Over and Split View.

---

Apple suggests opting out of these features for camera-centric apps, where using the entire screen for preview and capturing a moment quickly is a primary feature (reference). This is the approach the use in their sample app AVCam.

However, our customers might have apps which don't fall into this category (e.g Mobile banking apps), so we cannot force them to opt out, instead, we need to handle new features in the SDK. We're exploring what would be the best approach to do that, since the docs at the moment aren't really telling us what to do.

---

We used our simple Camera sample app to analyse the use case. The sample app is available on Github and it's developed as of iOS 9 Beta 5.

From the sample app, it can be clearly seen which system events happen when Slide Over is used, and when Split View is used.

• When our app is primary, and Slide Over is used, we get UIApplicationWillResignActiveNotification and AVCaptureSessionDidStopRunningNotification
• When Slide Over is used, and our app is secondary, we get UIApplicationWillEnterForegroundNotification and AVCaptureSessionDidStopRunningNotification immediately after that
• When Split View is used, on each divider drag, our app gets UIApplicationWillResignActiveNotification.
• However, if the Camera is launched when in Split View, it immediately gets AVCaptureSessionDidStopRunningNotification

So, empirically, it looks like AVCaptureSession is immediately stopped when Slide Over or Split View are used.

What's confusing is that UIImagePickerController, which our sample app also supports, exhibits completely different behaviour.

UIImagePickerController isn't stopped when the app goes into Slide Over/ Split View, instead, it functions completely normally. One can normally take a photo in Split View. In fact, two apps, both of which present UIImagePickerController, can work side by side, with UIImagePickerController of the active app being active. (You can try that by running our sample app, and Contacts app -> New Contact -> Add photo)

---

With all this in mind, our questions are the following:

• If AVCaptureSession is immediately paused when Slide Over and Split View are used, is it a good idea to monitor AVCaptureSessionDidStopRunningNotification, and present a message "Camera Paused" to the user, so that he clearly knows that the app isn't performing scanning?

• Why is behaviour of UIImagePickerController different than AVCaptureSession?

• Can we expect from Apple than in future beta versions behaviour of AVCaptureSession changes to match UIImagePickerController?

Answer 1

In case you haven't found out yet. After some more investigation I can now answer your first question:

If AVCaptureSession is immediately paused when Slide Over and Split View are used, is it a good idea to monitor AVCaptureSessionDidStopRunningNotification, and present a message "Camera Paused" to the user, so that he clearly knows that the app isn't performing scanning?

The notification you actually want to observe is this one: AVCaptureSessionWasInterruptedNotification

And you want to check for the newly introduced in iOS9 reason: AVCaptureSessionInterruptionReason.VideoDeviceNotAvailableWithMultipleForegroundApps

override func viewWillAppear(animated: Bool) {     super.viewWillAppear(animated)     self.addObserverForAVCaptureSessionWasInterrupted() }  func addObserverForAVCaptureSessionWasInterrupted() {     let mainQueue = NSOperationQueue.mainQueue()     NSNotificationCenter.defaultCenter().addObserverForName(AVCaptureSessionWasInterruptedNotification, object: nil, queue: mainQueue)         { (notification: NSNotification) -> Void in              guard let userInfo = notification.userInfo else             {                 return             }              // Check if the current system is iOS9+ because AVCaptureSessionInterruptionReasonKey is iOS9+ (relates to Split View / Slide Over)             if #available(iOS 9.0, *)             {                 if let interruptionReason = userInfo[AVCaptureSessionInterruptionReasonKey] where Int(interruptionReason as! NSNumber) == AVCaptureSessionInterruptionReason.VideoDeviceNotAvailableWithMultipleForegroundApps.rawValue                 {                     // Warn the user they need to get back to Full Screen Mode                 }             }             else             {                 // Fallback on earlier versions. From iOS8 and below Split View and Slide Over don't exist, no need to handle anything then.             }         } }  override func viewWillDisappear(animated: Bool) {     super.viewWillDisappear(true)      NSNotificationCenter.defaultCenter().removeObserver(self) } 

You can also know when the interruption was ended by observing: AVCaptureSessionInterruptionEndedNotification

Answer based on these two links:

http://asciiwwdc.com/2015/sessions/211 https://developer.apple.com/library/ios/samplecode/AVCam/Introduction/Intro.html