Using Python openCV to accurately find squares from processed image for Rubik's Cube solver

I am in the initial stages of writing a Rubik's cube solver and am stuck at the following challenge:

Using the following image-processing code gives me the following image:

import cv2 as cv
import glob
import numpy as np

for img in glob.glob("captured_images/*.jpg"):

    image = cv.imread(img)
    copy = image.copy()
    grey = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
    decrease_noise = cv.fastNlMeansDenoising(grey, 10, 15, 7, 21)
    blurred = cv.GaussianBlur(decrease_noise, (3, 3), 0)
    canny = cv.Canny(blurred, 20, 40)
    thresh = cv.threshold(canny, 0, 255, cv.THRESH_OTSU + cv.THRESH_BINARY)[1]
    contours = cv.findContours(thresh, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)

    contours = contours[0] if len(contours) == 2 else contours[1]

    for c in contours:
        # obtain the bounding rectangle coordinates for each square
        x, y, w, h = cv.boundingRect(c)
        # With the bounding rectangle coordinates we draw the green bounding boxes
        cv.rectangle(copy, (x, y), (x + w, y + h), (36, 255, 12), 2)

    cv.imshow('copy', copy)
    cv.waitKey(0)
    cv.destroyAllWindows()

There are numerous bound rectangles highlighted. Trying to filter out only the squares using this code:

contour_list = []
for contour in contours:
    approx = cv.approxPolyDP(contour, 0.01 * cv.arcLength(contour, True),  True)
    area = cv.contourArea(contour)
    if len(approx) == 4:
        (x, y, w, h) = cv.boundingRect(approx)
        if (float(w)/h) == 1:
            cv.rectangle(copy, (x, y), (x + w, y + h), (36, 255, 12), 2)
          
        contour_list.append(contour)

doesn't work as the squares aren't precise enough to fit the definition of "all sides of square are equal".

I though retaking the images against a white background might help to more easily find the relevant squares, however modifying the original image to a cube with a white background and using the original code causes only the larger cube to be recognised as a square:

My question is three-fold:

1a) How can I modify my original code for the original image to accurately measure only the relevant squares by using the following criteria for finding squares:

• There must be four corners
• All four lines must be roughly the same length
• All four corners must be roughly 90 degrees

1b) In the second image with the white background, how can I select everything outside the bound rectangle and convert that white background to black, which helps greatly in correctly detecting the appropriate squares?

1c) In general, why is a black background so much more beneficial than a white background in using the cv2.rectangle() function?

Any help in gaining some clearer understanding is much appreciated! :)

How can I modify my original code for the original image to accurately measure only the relevant squares by using the following criteria for finding squares:

Your code only accepts contours that are exactly square. You need to have a "squaredness" factor and then determine some acceptable threshold.

The "squaredness" factor is h/w if w > h else w/h. The closer that value to one, the more square the rectangle is. Then you can accept only rectangles with a factor of .9 or higher (or whatever works best).

In general, why is a black background so much more beneficial than a white background in using the cv2.rectangle() function?

The contour finding algorithm that OpenCV uses is actually:

Suzuki, S. and Abe, K., Topological Structural Analysis of Digitized Binary Images by Border Following. CVGIP 30 1, pp 32-46 (1985)

In your case, the algorithm might just have picked up the contours just fine, but you have set the RETR_EXTERNAL flag, which will cause OpenCV to only report the outermost contours. Try changing it to RETR_LIST.

Find the OpenCV docs with regards to contour finding here: https://docs.opencv.org/master/d9/d8b/tutorial_py_contours_hierarchy.html