swiftswiftuiaugmented-realityarkitrealitykit

Using ARKit anchor to place RealityKit Scene


I am currently struggling to add an RealityKit Scene to an ARKit anchor. As my previous question, it is not possible to directly add the RealityKit Scene to an ARKit Anchor Node.

But how can I use the ARKit Anchor to place my Scene? For example, I want to find an object in the room, and depending on which object I found, I want to use different scenes and anchor them in the room depending on the recognised objects position.

Thanks for the help.

Current try:

func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)
        
        // Load the "Box" scene from the "Experience" Reality File
        let boxAnchor = try! Experience.loadBox()
        
        // Add the box anchor to the scene
        arView.scene.anchors.append(boxAnchor)
        
        let configuration = ARWorldTrackingConfiguration()
        guard let referenceObjects = ARReferenceObject.referenceObjects(
                inGroupNamed: "AR Objects", bundle: nil) else {
            fatalError("Missing expected asset catalog resources.")
        }
        
        configuration.detectionObjects = referenceObjects
        arView.session.run(configuration) // not possible at arview, but i need arview for the .rcproject :-/
        
        return arView
        
    }

EDIT: add code


Solution

  • ARKit has no nodes. SceneKit does.

    However, if you need to place RealityKit's model into AR scene using ARKit's anchor, it's as simple as this:

    import ARKit
    import RealityKit
    
    class ViewController: UIViewController {
        
        @IBOutlet var arView: ARView!
        let boxScene = try! Experience.loadBox()
        
        override func viewDidLoad() {
            super.viewDidLoad()
            arView.session.delegate = self
            
            let entity = boxScene.steelBox
            
            var transform = simd_float4x4(diagonal: [1,1,1,1])
            transform.columns.3.z = -3.0  // three meters away
            
            let arkitAnchor = ARAnchor(name: "ARKit anchor",
                                  transform: transform)
            
            let anchorEntity = AnchorEntity(anchor: arkitAnchor)
            anchorEntity.name = "RealityKit anchor"
            
            arView.session.add(anchor: arkitAnchor)
            anchor.addChild(entity)
            arView.scene.anchors.append(anchorEntity)
        }
    }
    

    It's possible thanks to the AnchorEntity convenience initializer:

    convenience init(anchor: ARAnchor)
    


    Also you can use session(_:didAdd:) and session(_:didUpdate:) instance methods:

    extension ViewController: ARSessionDelegate {
            
        func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
                
            guard let objectAnchor = anchors.first as? ARObjectAnchor
            else { return }
                
            let anchor = AnchorEntity(anchor: objectAnchor)
            let entity = boxScene.steelBox
            anchor.addChild(entity)
            arView.session.add(anchor: objectAnchor)
            arView.scene.anchors.append(anchor)
        }
    }