augmented-realityhololenswebview2mrtk

MouseEventsWebView.MouseEvent does nothing


Question:
I have WebView2 set up in my Unity project. Now I would like to transfer my clicks from a canvas to the WebView. However, nothing happens after passing a WebViewMouseEventData. Did I forget something or do I have to update something again myself?

I have read the following article and at the end it roughly shows how to do it.

And even if I send wrong coordinates, I would have to send randomly usable coordinates at some point so that something on the web page gets clicked. But that doesn't happen.

Does there need to be more done than just handing over the MouseEvent?

My Setup:

public void OnPointerClicked(MixedRealityPointerEventData eventData)
{
    var result = eventData.Pointer.Result;
    if (result.CurrentPointerTarget.layer != 30)
    {
         Debug.Log("OnPointerClicked>layer != 30");
         return;
    }

    var clickPosition = result.Details.PointLocalSpace;
    var x = (int)Math.Ceiling(clickPosition.x);
    var y = (int)Math.Ceiling(clickPosition.y);
    Debug.Log("OnPointerClicked > X: " + x + " Y: " + y);

    var mouseEventsWebView = _webView as IWithMouseEvents;
    WebViewMouseEventData mouseEvent = new WebViewMouseEventData
    {
          X = x,
          Y = y,
          Device = WebViewMouseEventData.DeviceType.Pointer,
          Type = WebViewMouseEventData.EventType.MouseDown,
          Button = WebViewMouseEventData.MouseButton.ButtonLeft,
          TertiaryAxisDeviceType = WebViewMouseEventData.TertiaryAxisDevice.PointingDevice
     };

     mouseEventsWebView.MouseEvent(mouseEvent);
}

WebView


Solution

  • Unfortunately, the documentation does not provide any information on the subject.
    The right answer to my question was posted on Github by Michael Farnsworth:

      public void OnPointerClicked(MixedRealityPointerEventData eventData)
      {
          var hitCoord = NormalizeWorldPoint(eventData.Pointer.Result.Details.Point);
    
          hitCoord.x *= _webView.Width;
          hitCoord.y *= _webView.Height;
    
          var mouseEventsWebView = _webView as IWithMouseEvents;
          WebViewMouseEventData mouseEvent = new WebViewMouseEventData
          {
              X = (int)hitCoord.x,
              Y = (int)hitCoord.y,
              Device = WebViewMouseEventData.DeviceType.Pointer,
              Type = WebViewMouseEventData.EventType.MouseDown,
              Button = WebViewMouseEventData.MouseButton.ButtonLeft,
              TertiaryAxisDeviceType = WebViewMouseEventData.TertiaryAxisDevice.PointingDevice
          };
    
          mouseEventsWebView.MouseEvent(mouseEvent);
    
          // To register as a click, the WebView needs to be a mouse-up event.
          mouseEvent.Type = WebViewMouseEventData.EventType.MouseUp;
          mouseEventsWebView.MouseEvent(mouseEvent);
      }
    
      private Vector2 NormalizeWorldPoint(Vector3 worldPoint)
      {
          // Convert the world point to our control's local space.
          Vector3 localPoint = transform.InverseTransformPoint(worldPoint);
    
          var boundsSize = collider.sharedMesh.bounds.size;
          var boundsExtents = collider.sharedMesh.bounds.max;
    
          // Adjust the point to be based on a 0,0 origin.
          var uvTouchPoint = new Vector2((localPoint.x + boundsExtents.x), -1.0f * (localPoint.y - boundsExtents.y));
    
          // Normalize the point so it can be mapped to the WebView's texture.
          var normalizedPoint = new Vector2(uvTouchPoint.x / boundsSize.x, uvTouchPoint.y / boundsSize.y);
    
          return normalizedPoint;
      }