iosswiftuiarkitreality-composer-pro

How to access a Reality Composer Pro scene in a UIViewRepresentable ARView?


I am creating an Augmented Reality iOS (Not VisionOS) app using scenes created in Reality Composer Pro.

I'd like my code to send a notification to a RCP scene that plays a timeline. The RCP interface has the option to set up a behaviour for this purpose:

notification behaviour field in Reality Composer Pro

and both this SO post Send Notification from SwiftUI Xcode 16 to trigger Reality Composer Pro Timeline Action Animation and this Forum thread https://developer.apple.com/forums/thread/756978 show the code I need for sending a notification is:

NotificationCenter.default.post(
  name: NSNotification.Name("RealityKit.NotificationTrigger"),
  object: nil,
  userInfo: [
    "RealityKit.NotificationTrigger.Scene": scene,
    "RealityKit.NotificationTrigger.Identifier": "HideCharacter"
  ]
)

but the 'scene' var needs to point to the relevant RCP scene, which is loaded within a UIViewRepresentable ARView (because I need to use ARKit) and I can't work out how to correctly access it. The examples in the links above are for working with RealityKit, VisionOS or just don't provide enough context.

Code for loading the scene is as follows. How can I get the notification code above, situated out of the scope of the ARView, to access this scene?

struct ARViewContainer: UIViewRepresentable {
    
    typealias UIViewType = ARView
    
    func makeUIView(context: Context) -> ARView {
        
        // Create an ARView
        let arView = ARView(frame: .zero)
        
        // Configure it
        let arConfiguration = ARWorldTrackingConfiguration()
        arConfiguration.planeDetection = [.horizontal]
        arConfiguration.environmentTexturing = .automatic
        arConfiguration.frameSemantics.insert(.personSegmentationWithDepth)
        arView.environment.sceneUnderstanding.options.insert(.occlusion)
        arView.session.run(arConfiguration)
        
        // Load in Reality Composer Pro scene
        let scene = try! Entity.load(named:"myScene)", in: realityKitContentBundle)
        
        // Create a horizontal plane anchor
        let anchor = AnchorEntity(.plane(.horizontal, classification: .any, minimumBounds: SIMD2<Float>(0.2, 0.2)))
        
        // Append the scene to the anchor
        anchor.children.append(scene)
        
        // Append the anchor to the ARView
        arView.scene.anchors.append(anchor)
        
        return arView
    }
    
    func updateUIView(_ uiView: ARView, context: Context) {
    }
}

Solution

  • The simplest way is to just take a Binding<Scene> and assign uiView.scene to it in updateUIView.

    struct ContentView: View {
        @State private var scene: RealityKit.Scene?
        
        var body: some View {
            ARViewContainer(scene: $scene)
        }
    }
    
    struct ARViewContainer: UIViewRepresentable {
        class MyARView: ARView {
            var onMoveToWindow: (() -> Void)?
            
            override func didMoveToWindow() {
                super.didMoveToWindow()
                onMoveToWindow?()
            }
        }
        
        @Binding var scene: RealityKit.Scene?
        
        func makeUIView(context: Context) -> MyARView {
            let arView = MyARView(frame: .zero)
            
            // ...
            
            return arView
        }
        
        func updateUIView(_ uiView: MyARView, context: Context) {
            if scene == nil {
                uiView.onMoveToWindow = { [weak uiView] in self.scene = uiView?.scene }
            }
        }
    }
    

    That said, I think a more idiomatic approach would be to expose a "proxy" struct, instead of a the Scene class itself. This is much like the existing ScrollViewProxy, MapProxy, and ChartProxy.

    struct SceneProxy: Hashable {
        // now SwiftUI can hold a *weak* reference to the scene, which makes more sense
        // because it is owned by the ARView
        fileprivate weak var scene: RealityKit.Scene?
        
        func playTimeline() {
            guard let scene else {
                print("Scene not found!")
                return
            }
            NotificationCenter.default.post(
              name: NSNotification.Name("RealityKit.NotificationTrigger"),
              object: nil,
              userInfo: [
                "RealityKit.NotificationTrigger.Scene": scene,
                "RealityKit.NotificationTrigger.Identifier": "HideCharacter"
              ]
            )
        }
    }
    
    struct ARViewContainer: UIViewRepresentable {
        class MyARView: ARView {
            var onMoveToWindow: (() -> Void)?
            
            override func didMoveToWindow() {
                super.didMoveToWindow()
                onMoveToWindow?()
            }
        }
        
        @Binding var sceneProxy: SceneProxy
        
        func makeUIView(context: Context) -> MyARView {
            let arView = MyARView(frame: .zero)
            
            // ...
            
            return arView
        }
        
        func updateUIView(_ uiView: MyARView, context: Context) {
            if sceneProxy.scene == nil {
                uiView.onMoveToWindow = { [weak uiView] in sceneProxy.scene = uiView?.scene }
            }
        }
    }
    

    If you want to go the extra mile, you can even write a SceneReader (akin to ScrollViewReader). For more info, see my answer here.