ioscameramultitaskingavcapturesessionios9

How to use AVCaptureSession with Slide Over and Split View in iOS 9?

My team is developing a set of SDKs for barcode scanning, ID scanning and OCR. We use device's camera, specifically, AVCaptureSession, to obtain video frames on which we perform our processing.

We're exploring new iOS 9 multitasking features Slide Over and Split View.

Apple suggests opting out of these features for camera-centric apps, where using the entire screen for preview and capturing a moment quickly is a primary feature (reference). This is the approach the use in their sample app AVCam.

However, our customers might have apps which don't fall into this category (e.g Mobile banking apps), so we cannot force them to opt out, instead, we need to handle new features in the SDK. We're exploring what would be the best approach to do that, since the docs at the moment aren't really telling us what to do.

We used our simple Camera sample app to analyse the use case. The sample app is available on Github and it's developed as of iOS 9 Beta 5.

From the sample app, it can be clearly seen which system events happen when Slide Over is used, and when Split View is used.

So, empirically, it looks like AVCaptureSession is immediately stopped when Slide Over or Split View are used.

What's confusing is that UIImagePickerController, which our sample app also supports, exhibits completely different behaviour.

UIImagePickerController isn't stopped when the app goes into Slide Over/ Split View, instead, it functions completely normally. One can normally take a photo in Split View. In fact, two apps, both of which present UIImagePickerController, can work side by side, with UIImagePickerController of the active app being active. (You can try that by running our sample app, and Contacts app -> New Contact -> Add photo)

With all this in mind, our questions are the following:

Solution

  • In case you haven't found out yet. After some more investigation I can now answer your first question:

    If AVCaptureSession is immediately paused when Slide Over and Split View are used, is it a good idea to monitor AVCaptureSessionDidStopRunningNotification, and present a message "Camera Paused" to the user, so that he clearly knows that the app isn't performing scanning?

    The notification you actually want to observe is this one: AVCaptureSessionWasInterruptedNotification

    And you want to check for the newly introduced in iOS9 reason: AVCaptureSessionInterruptionReason.VideoDeviceNotAvailableWithMultipleForegroundApps

    override func viewWillAppear(animated: Bool)
{
    super.viewWillAppear(animated)
    self.addObserverForAVCaptureSessionWasInterrupted()
}

func addObserverForAVCaptureSessionWasInterrupted()
{
    let mainQueue = NSOperationQueue.mainQueue()
    NSNotificationCenter.defaultCenter().addObserverForName(AVCaptureSessionWasInterruptedNotification, object: nil, queue: mainQueue)
        { (notification: NSNotification) -> Void in

            guard let userInfo = notification.userInfo else
            {
                return
            }

            // Check if the current system is iOS9+ because AVCaptureSessionInterruptionReasonKey is iOS9+ (relates to Split View / Slide Over)
            if #available(iOS 9.0, *)
            {
                if let interruptionReason = userInfo[AVCaptureSessionInterruptionReasonKey] where Int(interruptionReason as! NSNumber) == AVCaptureSessionInterruptionReason.VideoDeviceNotAvailableWithMultipleForegroundApps.rawValue
                {
                    // Warn the user they need to get back to Full Screen Mode
                }
            }
            else
            {
                // Fallback on earlier versions. From iOS8 and below Split View and Slide Over don't exist, no need to handle anything then.
            }
        }
}

override func viewWillDisappear(animated: Bool)
{
    super.viewWillDisappear(true)

    NSNotificationCenter.defaultCenter().removeObserver(self)
}

    You can also know when the interruption was ended by observing: AVCaptureSessionInterruptionEndedNotification

    Answer based on these two links:

    http://asciiwwdc.com/2015/sessions/211 https://developer.apple.com/library/ios/samplecode/AVCam/Introduction/Intro.html