swiftuiarkitrealitykitvisionosvision-pro

VisionOS detect lateral head tilt


Is it possible to detect a lateral head tilt within an Immersive Space?

When the AVP wearer tilts their head to the left or right, I want to detect it and trigger an animation.

enter image description here

As far as I can tell, it's not possible to replicate this movement in the Simulator. Is there a work-around to test it without owning a device?

I'm not sure if this is a viable input method as the Vision Pro has not been released in my country yet.


Solution

  • Here's how I ended up achieving a head roll trigger using RealityView and WorldTrackingProvider:

    VisionProPose:

    import ARKit
    import QuartzCore
    import Combine
    
    class VisionProPose: ObservableObject {
        
        let arkitSession = ARKitSession()
        let worldTracking = WorldTrackingProvider()
        
        func runARKitSession() async {
            do {
                try await arkitSession.run([worldTracking])
            } catch {
                print("Failed to run ARKit session: \(error)")
                return
            }
        }
    
        func queryDeviceRoll() async -> Double? {
            
            // Wait until the worldTracking state is running
            while worldTracking.state != .running {
                try? await Task.sleep(nanoseconds: 100_000_000) // 100 milliseconds
            }
            
            let deviceAnchor = worldTracking.queryDeviceAnchor(atTimestamp: CACurrentMediaTime())
            
            if let deviceAnchor = deviceAnchor {
                let transform = deviceAnchor.originFromAnchorTransform
                
                // Extract the elements for the 2x2 submatrix
                let m00 = transform.columns.0.x
                let m10 = transform.columns.1.x
                
                // Compute the rotation around the Z-axis
                let zRotation = atan2(m10, m00)
                
                // Convert the rotation to degrees and then to Double
                let zRotationDegrees = Double(zRotation * 180.0 / .pi)
                
                return zRotationDegrees
            } else {
                print("No device anchor found")
                return nil
            }
        }
    }
    

    I then used it in a view like this:

    ExampleView

    import SwiftUI
    import RealityKit
    
    struct ExampleView: View {
        
        // Load vision pro pose
        @StateObject private var visionProPose = VisionProPose()
        
        var body: some View {
            RealityView { content in
                
                // Run each time scene is drawn (90Hz)
                _ =  content.subscribe(to: SceneEvents.Update.self) { event in
                    
                    Task {
                        
                        // Check the current head position
                        if let currentHeadRoll = await visionProPose.queryDeviceRoll() {
                            
                            // Head roll is more than 14 degrees in either direction
                            if abs(currentHeadRoll) > 14 {
                                
                                // Default to left
                                var tiltDirection = "left"
                                
                                // But could be right
                                if currentHeadRoll > 14 {
                                    tiltDirection = "right"
                                }
                                
                                // Do something with head tilt
                                switch tiltDirection {
                                case "left": 
                                    print("Head tilting left!")
                                case "right":
                                    print("Head tilting right!")
                                default:
                                    break
                                }
                            }
                        }
                    }
                }
            }
        }
    }
    

    Together this code looks for a 14 degree head tilt, left or right and allows you to use it for whatever you want.