Convert mouse position to Canvas Coordinates and back

I'm creating a canvas with an overlay div to add markers on click and I want markers to change position when I pan zoom the canvas or resize the window. I'm using https://github.com/timmywil/panzoom to pan zoom.

The problem is when I convert mouse position to canvas coordinates it worked correctly but when I convert it back to screen position to render markers on overlay div, the result is not as same as initialized mouse position and recalculate marker's position on resize also not correct.

This canvas is fullscreen with no scroll.

width = 823; height = 411;
scale = 2; panX = 60; panY = 10;
mouse.pageX = 467; mouse.pageY = 144; 

// {x: 475, y: 184} correct coords when I use ctx.drawImage(..) to test
canvasCoords = getCanvasCoords(mouse.pageX, mouse.pageY, scale);

// {x: 417, y: 124}
screenCoords = toScreenCoords(canvasCoords.x, canvasCoords.y, scale, panX, panY);
------------------------------
but with scale = 1; it worked correctly.

// convert mouse position to canvas coordinates
getCanvasCoords(pageX: number, pageY: number, scale: number) {
    var rect = this.pdfInfo.canvas.getBoundingClientRect();
    let x = (pageX - rect.left + this.scrollElement.scrollTop) / scale;
    let y = (pageY - rect.top + this.scrollElement.scrollLeft) / scale;
    return {
      x: Number.parseInt(x.toFixed(0)),
      y: Number.parseInt(y.toFixed(0)),
    };
}

// convert canvas coords to screen coords 
toScreenCoords(
    x: number,
    y: number,
    scale: number
  ) {
    var rect = this.pdfInfo.canvas.getBoundingClientRect();

    let wx =
      x * scale + rect.left - this.scrollElement.scrollTop / scale;
    let wy =
      y * scale + rect.top - this.scrollElement.scrollLeft / scale;
    return {
      x: Number.parseInt(wx.toFixed(0)),
      y: Number.parseInt(wy.toFixed(0)),
    };
}

getNewPos(x, oldV, newV) {
    return (x * oldV) / newV;
}

// update screen coords with new screen width and height
onResize(old, new) {
   this.screenCoordList.forEach(el => {
      el.x = getNewPos(el.x, old.width, new.width);
      el.y = getNewPos(el.y, old.height, new.height);
   })
}

How to get it worked with scale and pan? if you know any library can do the job please recommend, thank you.

Here's a code snippet that seems to be working, you can probably adapt it for your purposes.

What I used was:

function toCanvasCoords(pageX, pageY, scale) {
    var rect = canvas.getBoundingClientRect();
    let x = (pageX - rect.left) / scale;
    let y = (pageY - rect.top) / scale;
    return toPoint(x, y);
}

and

function toScreenCoords(x, y, scale) {
    var rect = canvas.getBoundingClientRect();
    let wx = x * scale + rect.left + scrollElement.scrollLeft;
    let wy = y * scale + rect.top + scrollElement.scrollTop;
    return toPoint(wx, wy);
}

I'm just getting the mouse position from the window object. I'm may be mistaken, but I think this is why scrollLeft and scrollTop don't appear in toCanvasCoords (since the position is relative to the client area of the window itself, the scroll doesn't come into it). But then when you transform back, you have to take it into account.

This ultimately just returns the mouse position relative to the window (which was the input), so it's not really necessary to do the whole transformation in a roundabout way if you just want to attach an element to the mouse pointer. But transforming back is useful if you want to have something attached to a certain point on the canvas image (say, a to feature on the map) - which I'm guessing is something that you're going for, since you said that you want to render markers on an overlay div.

In the code snippet bellow, the red circle is drawn on the canvas itself at the location returned by toCanvasCoords; you'll notice that it scales together with the background.

I didn't use an overlay div covering the entire map, I just placed a couple of small divs on top using absolute positioning. The black triangle is a div (#tracker) that basically tracks the mouse; it is placed at the result of toScreenCoords. It serves as a way to check if the transformations work correctly. It's an independent element, so it doesn't scale with the image.

The red triangle is another such div (#feature), and demonstrates the aforementioned "attach to feature" idea. Suppose the background is a something like a map, and suppose you want to attach a "map pin" icon to something on it, like to a particular intersection; you can take that location on the map (which is a fixed value), and pass it to toScreenCoords. In the code snippet below, I've aligned it with a corner of a square on the background, so that you can track it visually as you change scale and/or scroll. (After you click "Run code snippet", you can click "Full page", and then resize the window to get the scroll bars).

Now, depending on what exactly is going on in your code, you may have tweak a few things, but hopefully, this will help you. If you run into problems, make use of console.log and/or place some debug elements on the page that will display values live for you (e.g. mouse position, client rectangle, etc.), so that you can examine values. And take things one step at the time - e.g. first get the scale to work, but ignore scrolling, then try to get scrolling to work, but keep the scale at 1, etc.

const canvas = document.getElementById('canvas');
const context = canvas.getContext("2d");
const tracker = document.getElementById('tracker');
const feature = document.getElementById('feature');
const slider = document.getElementById("scale-slider");
const scaleDisplay = document.getElementById("scale-display");
const scrollElement = document.querySelector('html');

const bgImage = new Image();
bgImage.src = "https://i.sstatic.net/yxtqw.jpg"
var bgImageLoaded = false;
bgImage.onload = () => { bgImageLoaded = true; };

var mousePosition = toPoint(0, 0);
var scale = 1;

function updateMousePosition(evt) {
  mousePosition = toPoint(evt.clientX, evt.clientY);
}

function getScale(evt) {
  scale = evt.target.value;
  scaleDisplay.textContent = scale;
}

function toCanvasCoords(pageX, pageY, scale) {
    var rect = canvas.getBoundingClientRect();
    let x = (pageX - rect.left) / scale;
    let y = (pageY - rect.top) / scale;
    return toPoint(x, y);
}

function toScreenCoords(x, y, scale) {
    var rect = canvas.getBoundingClientRect();
    let wx = x * scale + rect.left + scrollElement.scrollLeft;
    let wy = y * scale + rect.top + scrollElement.scrollTop;
    return toPoint(wx, wy);
}

function toPoint(x, y) {
  return { x: x, y: y }
}

function roundPoint(point) {
  return {
    x: Math.round(point.x), 
    y: Math.round(point.y)
  }
}

function update() {
  context.clearRect(0, 0, 500, 500);
  context.save();
  context.scale(scale, scale);
  
  if (bgImageLoaded) 
    context.drawImage(bgImage, 0, 0);
  
  const canvasCoords = toCanvasCoords(mousePosition.x, mousePosition.y, scale);
  drawTarget(canvasCoords);
  
  const trackerCoords = toScreenCoords(canvasCoords.x, canvasCoords.y, scale);
  updateTrackerLocation(trackerCoords);

  updateFeatureLocation()
  
  context.restore();
  requestAnimationFrame(update);
}

function drawTarget(location) {
  context.fillStyle = "rgba(255, 128, 128, 0.8)";
  context.beginPath();
  context.arc(location.x, location.y, 8.5, 0, 2*Math.PI); 
  context.fill();
}

function updateTrackerLocation(location) {
  const canvasRectangle = offsetRectangle(canvas.getBoundingClientRect(), 
    scrollElement.scrollLeft, scrollElement.scrollTop);
  if (rectContains(canvasRectangle, location)) {
    tracker.style.left = location.x + 'px';
    tracker.style.top = location.y + 'px';
  }
}

function updateFeatureLocation() {
  // suppose the background is a map, and suppose there's a feature of interest
  // (e.g. a road intersection) that you want to place the #feature div over
  // (I roughly aligned it with a corner of a square).
  const featureLoc = toScreenCoords(84, 85, scale);
  feature.style.left = featureLoc.x + 'px';
  feature.style.top = featureLoc.y + 'px';
}

function offsetRectangle(rect, offsetX, offsetY) {
  // copying an object via the spread syntax or 
  // using Object.assign() doesn't work for some reason
  const result = JSON.parse(JSON.stringify(rect));
  result.left += offsetX;
  result.right += offsetX;
  result.top += offsetY;
  result.bottom += offsetY;
  result.x = result.left;
  result.y = result.top;
  
  return result;
}

function rectContains(rect, point) {
  const inHorizontalRange = rect.left <= point.x && point.x <= rect.right;
  const inVerticalRange = rect.top <= point.y && point.y <= rect.bottom;
  return inHorizontalRange && inVerticalRange;
}

window.addEventListener('mousemove', (e) => updateMousePosition(e), false);
slider.addEventListener('input', (e) => getScale(e), false);
requestAnimationFrame(update);

#canvas {
  border: 1px solid gray;
}

#tracker, #feature {
  position: absolute;
  left: 0;
  top: 0;  
  border-left: 5px solid transparent;
  border-right: 5px solid transparent;  
  border-bottom: 10px solid black;
  transform: translate(-4px, 0);
}

#feature {
  border-bottom: 10px solid red;
}

<div>
  <label for="scale-slider">Scale:</label>
  <input type="range" id="scale-slider" name="scale-slider" min="0.5" max="2" step="0.02" value="1">
  <span id="scale-display">1</span>
</div>
<canvas id="canvas" width="500" height="500"></canvas>
<div id="tracker"></div>
<div id="feature"></div>

P.S. Don't do Number.parseInt(x.toFixed(0)); generally, work with floating point for as long as possible to minimize accumulation of errors, and only convert to int at the last minute. I've included the roundPoint function that rounds the (x, y) coordinates of a point to the nearest integer (via Math.round), but ended up not needing to use it at all.

Note: The image below is used as the background in the code snippet, to serve as a reference point for scaling; it is included here just so that it is hosted on Stack Exchange's imgur.com account, so that the code is not referencing a (potentially volatile) 3rd-pary source.