imagetrackingrealitykitvisionos

Slow Rendering During VisionOS Image Tracking


I am following this documentation on tracking images in 3D space on Vision Pro. https://developer.apple.com/documentation/visionos/tracking-images-in-3d-space

The expected result is that a 3D sphere will smoothly track a moving reference image. However, when I run the app, the 3D sphere will only update positions about twice a second or so. There's not a lot of documentation for troubleshooting. Is there some sort of render function I should be calling to have the RealityView render more quickly?

import SwiftUI
import ARKit
import RealityKit
import RealityKitContent

struct ImmersiveView: View {
    var arkitSession = ARKitSession()

    @State var entityMap : [UUID: ModelEntity] = [:]
    let imageInfo = ImageTrackingProvider(
        referenceImages: ReferenceImage.loadReferenceImages(inGroupNamed: "ref")
    )
    var material = SimpleMaterial()

    var rootEntity = Entity()

    func updateImage(_ anchor: ImageAnchor) {
        if entityMap[anchor.id] == nil {
            // Add a new entity to represent this image.
            let entity = ModelEntity(mesh: .generateSphere(radius: 0.05))
            entityMap[anchor.id] = entity
            rootEntity.addChild(entity)
        }

        if anchor.isTracked {
            entityMap[anchor.id]?.transform = Transform(matrix: anchor.originFromAnchorTransform)
        }

    }

    var body: some View {

        RealityView { content in
            // Add the initial RealityKit content
            content.add(rootEntity)

            Task {
                try await arkitSession.run([imageInfo]);
                for await update in imageInfo.anchorUpdates {
                    updateImage(update.anchor)
                }
            }
        }

    }
}

Solution

  • Per the Apple Developer Forums, ImageTrackingProvider on VisionOS currently seems to fire anchorUpdates quite slowly, leading to inability to smoothly track moving images. I filed an enhancement request via the Feedback Assistant and encourage others to do the same.

    https://forums.developer.apple.com/forums/thread/747118