Is it possible to detect a lateral head tilt within an Immersive Space?
When the AVP wearer tilts their head to the left or right, I want to detect it and trigger an animation.
As far as I can tell, it's not possible to replicate this movement in the Simulator. Is there a work-around to test it without owning a device?
I'm not sure if this is a viable input method as the Vision Pro has not been released in my country yet.
Here's how I ended up achieving a head roll trigger using RealityView
and WorldTrackingProvider
:
VisionProPose:
import ARKit
import QuartzCore
import Combine
class VisionProPose: ObservableObject {
let arkitSession = ARKitSession()
let worldTracking = WorldTrackingProvider()
func runARKitSession() async {
do {
try await arkitSession.run([worldTracking])
} catch {
print("Failed to run ARKit session: \(error)")
return
}
}
func queryDeviceRoll() async -> Double? {
// Wait until the worldTracking state is running
while worldTracking.state != .running {
try? await Task.sleep(nanoseconds: 100_000_000) // 100 milliseconds
}
let deviceAnchor = worldTracking.queryDeviceAnchor(atTimestamp: CACurrentMediaTime())
if let deviceAnchor = deviceAnchor {
let transform = deviceAnchor.originFromAnchorTransform
// Extract the elements for the 2x2 submatrix
let m00 = transform.columns.0.x
let m10 = transform.columns.1.x
// Compute the rotation around the Z-axis
let zRotation = atan2(m10, m00)
// Convert the rotation to degrees and then to Double
let zRotationDegrees = Double(zRotation * 180.0 / .pi)
return zRotationDegrees
} else {
print("No device anchor found")
return nil
}
}
}
I then used it in a view like this:
ExampleView
import SwiftUI
import RealityKit
struct ExampleView: View {
// Load vision pro pose
@StateObject private var visionProPose = VisionProPose()
var body: some View {
RealityView { content in
// Run each time scene is drawn (90Hz)
_ = content.subscribe(to: SceneEvents.Update.self) { event in
Task {
// Check the current head position
if let currentHeadRoll = await visionProPose.queryDeviceRoll() {
// Head roll is more than 14 degrees in either direction
if abs(currentHeadRoll) > 14 {
// Default to left
var tiltDirection = "left"
// But could be right
if currentHeadRoll > 14 {
tiltDirection = "right"
}
// Do something with head tilt
switch tiltDirection {
case "left":
print("Head tilting left!")
case "right":
print("Head tilting right!")
default:
break
}
}
}
}
}
}
}
}
Together this code looks for a 14 degree head tilt, left or right and allows you to use it for whatever you want.