Measuring height of a playing card deck in an image

I am writing a module for a casino surveillance software.

A bit of context:

I have several images from a video extracted at different times. I have a reference deck always at a fixed position that I know the size of. Assuming that the distortion, focal length of the camera and the distance is not known (this will be a software package sold commercially so these parameters are unknown).

Currently working:

I have applied canny edge detection and houghlines to get the pixel height of the two decks. Also the coordinates for all corners of the deck in the front is known since it is always in a fixed position with the same orientation.

The problem starts when the reference deck is not at the same distance as the deck I am trying to measure but closer to the camera. In this case since one of the decks are closer to the camera both decks may appear to have the same pixel height although the one in the back contains more cards.

Here is a visual example:

In reality both decks have 52 cards but the one closer to the camera measures 83 pixels where as the one in the back measures 63 pixels.

How can I normalize the values I have for reference to the deck in the background? I have found a question which does something similar to calculate the size when moved within the image but this uses openGL which I am not familiar with (How do I reverse-project 2D points into 3D?). I am trying to achieve this using a computer vision/image manipulation library (I currently use ImageMagick for canny edge and hough lines) in Python(I am open to suggestions on using another programming language if you think python is not suitable).

EDIT:

To wrap up what I am trying to achieve: I want to know how the two objects compare height wise. How do I scale the one in the back to be the the size it would be if it was next to the one in the front so that I can compare the two?

I'm not familiar with ImageMagick so I can't write the codes for you but I explained what your going to do:

There is something that we are sure about it: The real size of each card! You know that all cards have the same width and height (in real world), so the only thing you need to do is to find the width of upper card on the first stack and also the second stack in the image. Since these two numbers are related to each other, now you can have the ratios of the dimensions around the deck 1 and deck 2.

With these ratio you can normalize the heights of decks and compare these numbers with each other.

For example, consider width of each card (the red arrows) is 5 cm and this width in the image for the first deck is 80 pixels and for the second deck is 40 pixels. So the ratio is 2. Now For comparing those black lines, just multiply the second line's length with 2 and compare the result number with the first line's length.

You have lots of way to estimate the width of each card deck. Extracting the widths might be hard so I suggest to compare the width of the patterns (the yellow arrows) and for this, I would do such method:

1- Do a canny on image (because of the drawings on the card, you will have lots of line on the surface of the card)

2- Do a dilation to stick all those lines of drawing on the surface of the card and make a unit object

3- Do an erosion to shrink that object a little, and then catch the width of that object (this would be a good estimation of the width of the card)