[SOLVED] get current fram as uiimage from ARWorldTrackingConfiguration?

get current fram as uiimage from ARWorldTrackingConfiguration?

I need to extract a frame that the physical camera sees (without SCNScene added) while ARWorldTrackingConfiguration is configured. I am new to ios so maybe I am missing something but I wasn't able to do this. Any help appreciated on how to do this.

I have tried extracting using

var buffer = self.sceneView.session.currentFrame?.capturedImage

where sceneView is my ARSCNView, I thought it would give me the image in YCbCr

I then tried to convert to RGB using

let ciImage = CIImage(cvPixelBuffer: buffer)
let context = CIContext(options: nil)
let cgImage = context.createCGImage(ciImage, from: ciImage.extent)
let uiImage = UIImage(cgImage: cgImage!)

But got nil error after calling pixelBufferToUIImage()

Solution

You can convert the ARSCNView to UIImage with this function:

func imageFrom(scene:ARSCNView) -> UIImage {

    UIGraphicsBeginImageContextWithOptions(scene.bounds.size, scene.isOpaque, 0.0)
    scene.drawHierarchy(in: scene.bounds, afterScreenUpdates: false)
    let image = UIGraphicsGetImageFromCurrentImageContext()
    UIGraphicsEndImageContext()
    return UIImage(cgImage: (image?.cgImage)!)

}

Usage:

@IBOutlet var sceneView: ARSCNView!

override func viewDidLoad() {
        super.viewDidLoad()
    let convertedToImage = imageFrom(scene: sceneView)

    // Do something with convertedToImage

}