I'm getting the data from HoloLens 2 eye tracker using the SpatialPointerPose API like this (safety checks not given for brevity):
currentSpatialCoordinateSystem = PerceptionInterop.GetSceneCoordinateSystem(UnityEngine.Pose.identity) as SpatialCoordinateSystem;
SpatialPointerPose pointerPose = SpatialPointerPose.TryGetAtTimestamp(currentSpatialCoordinateSystem, PerceptionTimestampHelper.FromHistoricalTargetTime(DateTimeOffset.Now));
var eyes = pointerPose.Eyes;
GazeOrigin = eyes.Gaze.Value.Origin.ToUnityVector3(),
GazeDirection = eyes.Gaze.Value.Direction.ToUnityVector3()
This seems to also work while using Holographic Remoting. However, I have some doubts about the timing/synchronization of the gaze data in this case because there will be some delay between the remote server and the HoloLens?
Specifically, will TryGetAtTimestamp give me the correct timing here, considering that the code runs on the remote machine (either as a standalone app or using the remoting from play mode in Unity editor) ?
Edit: The above code only works while remoting in Unity play mode
Holographic Remoting uses clock synchronization which means that timing is correct. The result and accuracy of SpatialPointerPose::TryGetAtTimestamp depends on the given timestamp. For instance, if the given timestamp is older than the last received pose data on the remote side, then the returned SpatialPointerPose will be an exact interpolation of received pose data values. If the given timestamp is newer than the last received pose data on the remote side, then Holographic Remoting predicts the SpatialPointerPose based on the latest pose data available.