iosswiftimage-processingavfoundationapple-vision

CIDetector crashing while processing CMSampleBuffer


Problem: I am trying to get facial features through CIDetector from CMSampleBuffer from AVCaptureVideoDataOutput. On execution of the program, 9 out of 10 time the program crashes and only once it runs fine.

Expected Output: Run without crash and print "Happy" on feature detection.

Code:

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    
    let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
    let opaqueBuffer = Unmanaged<CVImageBuffer>.passUnretained(imageBuffer).toOpaque()
    let pixelBuffer = Unmanaged<CVPixelBuffer>.fromOpaque(opaqueBuffer).takeUnretainedValue()
    let sourceImage = CIImage(cvPixelBuffer: pixelBuffer, options: nil)
    let options = [CIDetectorSmile : true as AnyObject, CIDetectorEyeBlink: true as AnyObject, CIDetectorImageOrientation : 6 as AnyObject]
    
    // The detector is nil
    let detector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options) 
    let features = detector!.features(in: sourceImage, options: options)

        for feature in features as! [CIFaceFeature] {

            if (feature.hasSmile) {
                printLog(item: "HAPPY")
            }
        }
}

Crashlog : Unexpectedly found nil while unwrapping an Optional value. The detector is nil

Would love help and further pointers.


Solution

  • The return is optional and sampleBuffer is called way too heavy you can only do this

    if let detector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options) {
        let features = detector.features(in: sourceImage, options: options) 
        for feature in features as! [CIFaceFeature] { 
            if (feature.hasSmile) {
                printLog(item: "HAPPY")
            }
        }
    }