Marks and axis labels are out of sync randomly while panning

I have a Vega-Lite chart, multi-layered with point and rule marks. Here is the configuration in Vega Editor.

To enable selection of points, zoom and panning on this chart, it has two selections named - select and zoom.

• Pointer click and drag is configured for selecting points
• Shift + mouse wheel is for zoom
• Shift + pointer drag is for pan, retaining the selection rectangle if present.
• Double click on the chart clears all the selections and restores the view to the original state.

This works good most of the time, except randomly while panning (with or without zoom) the axis labels and rules move along the mouse pointer but the point marks don't move in sync with the labels.

I couldn't determine the exact click sequence that causes this or whether there is something incorrect in the configuration itself. I am using vega v5.4.0, vega-lite v3.4.0 and vega-embed v4.2.5.

Thanks

This issue was fixed along with some other fixes in vega-lite and vega-embed, it's not reproducible after upgrading to vega-lite v4 and vega-embed v6.

A sample that works with the above versions.