Using jquery panzoom alongside jquery draggable

So I have this jquery .draggable items that work inside the .droppable and the .droppable parent uses the jquery .panzoom. There is also a image for background witch is inside .canvasimg.

Here is a tree of my html structure:

<div class="wrapper"> //panzoom
    <div class="canvasimg">
        <img src="somebackground">
    </div>
    <div class="droppable">

        <div class="draggable">
        </div>
        <div class="draggable">
        </div>

    </div>
</div>

What I have:

I have the zoom working and draggable working too. The problem is when I zoom in, the dragged item should be under the mouse but it kind of gets the "true" position, or the original position as you prefer..

If I start my drag from the top left of my .droppable with zoom in, it starts okay, the element is under the mouse, like this:

But as I go across the screen the element starts to "get away" from the mouse, the more I go right and bottom, more the element gets away from the mouse, like this:

What I'v tried:

To implement the answer on this question but since I'm saving the position via ajax, I'm not very sure how can I use it, if the answer to this question is in this code:

var m = map.panzoom('getMatrix');
var zoom = Math.sqrt(m[0] * m[3]); // zoom don't change   proportion and scale 
var original = ui.originalPosition;

ui.position = {
    left: (e.clientX - click.x + original.left) / zoom,
    top:  (e.clientY - click.y + original.top ) / zoom
};

I'v tried using it, but no success.

what I thought:

Can I invert the matrix of the panzoom and use the css transform property in the draggable item with the oposite matrix?

EDIT: So.. I successfully implemented this code:

 ui.position.top = Math.round(ui.position.top / zoom);
 ui.position.left = Math.round(ui.position.left / zoom);

In my function that gets the position of the element now the area that I have to drag the items reduces as I zoom in..

Probably I have to use a similar code in the .droppable div. to scale the .droppable as it scales the .wrapper?

EDIT 2: Created a question about the item here.

So I have succesfully solved this problem using:

        var realheight = $(".droppable").eq(0).height() * zoom;
        var realwidth = $(".droppable").eq(0).width() * zoom ; 

        ui.position.top =  ui.position.top / zoom ;
        ui.position.left = ui.position.left / zoom;     
        ui.position.left - (ui.helper.outerWidth()/2)

        xpos = (((parseFloat(ui.helper.css("left"))   /  realwidth) * 100) * zoom) * zoom ; 
        ypos = (((parseFloat(ui.helper.css("top"))   / realheight) * 100) * zoom) * zoom  ; 

        ui.helper.css({top : ypos+"%", left: xpos+"%"});