pythonopencvcomputer-visioncamera-calibrationaruco

Charuco Calibration is not detecting any corners


I have a class in python in which using opencv calibrates the charuco board from multiple images

class CharucoCalibrator():
    def __init__(self, filepaths, squaresX=8, squaresY=11, squareLength=0.015, markerLength=0.011, dict=cv2.aruco.DICT_5X5_100):
        self.filepaths = filepaths
        self.squaresX = squaresX
        self.squaresY = squaresY
        self.squareLength = squareLength
        self.markerLength = markerLength

        self.aruco_dict = cv2.aruco.getPredefinedDictionary(dict)

        self.rotation_vectors = {}
        self.rotation_matrices = {}
        self.translation_vectors = {}
        self.projection_matricies = {}

        self.images = [cv2.imread(filepath) for filepath in self.filepaths]
        self.calibrated_images = []
        self.img_size = self.images[0].shape[:2][::-1]

    def calibrate(self):

        board = cv2.aruco.CharucoBoard((self.squaresX, self.squaresY), self.squareLength, self.markerLength, self.aruco_dict)
        params = cv2.aruco.DetectorParameters()

        all_charuco_corners = []
        all_charuco_ids = []

        self.successful_filepaths = []

        for filepath in self.filepaths:

            image = cv2.imread(filepath)
            calib_image = image.copy()
            # Detect ArUco markers
            corners, ids, _ = cv2.aruco.detectMarkers(image, self.aruco_dict, parameters=params)
            cv2.aruco.drawDetectedMarkers(calib_image, corners, ids)

            # Interpolate ChArUco corners
            if ids is not None and len(ids) > 0:
                retval, charuco_corners, charuco_ids = cv2.aruco.interpolateCornersCharuco(
                    markerCorners=corners,
                    markerIds=ids,
                    image=image,
                    board=board
                )

                if retval:
                    all_charuco_corners.append(charuco_corners)
                    all_charuco_ids.append(charuco_ids)

                    # Draw corners
                    cv2.aruco.drawDetectedCornersCharuco(calib_image, charuco_corners, charuco_ids)
                    self.successful_filepaths.append(filepath)

            self.calibrated_images.append(calib_image)


        if len(all_charuco_corners) >= 2:
            ret, self._camera_matrix, self._dist_coeffs, self._rvecs, self._tvecs = cv2.aruco.calibrateCameraCharuco(
                charucoCorners=all_charuco_corners,
                charucoIds=all_charuco_ids,
                board=board,
                imageSize=self.img_size,
                cameraMatrix=None,
                distCoeffs=None
            )

I'm using 10 images to do the process. Example :

Calib1

For some reason, the code manages to detect all the markers successfully, however not even a single corner is detected. I have done many other example images, but nothing seems to change.

I'm using opencv version 4.11.0


Solution

  • There seems to be a persisting problem with how charuco boards are interpreted. See here and here.

    First of all, note that you switched the dimensions, it should be squaresX=11, squaresY=8.

    Your board generated by www.calib.io does not seem to match the expected specifications of OpenCV. Also, setting board.setLegacyPattern(True) did not solve your issue for me. To get started you can generate a board natively via OpenCV adding a method to your class:

    def generate_charuco_board(self, output_path="charuco_board.png"):
        board = cv2.aruco.CharucoBoard(
            (self.squaresX, self.squaresY),
            self.squareLength,
            self.markerLength,
            self.aruco_dict
        )
        img = board.generateImage((800, 600))
        cv2.imwrite(output_path, img)
    

    If you use this image or a print of it as input you will get successful detections.

    charuco corners

    If you want to use your generator, you can go to an uneven number of rows, i.e. 7x11 and run your detection on that pattern with:

    calibrator = CharucoCalibrator(
        filepaths=filepaths,
        squaresX=11,
        squaresY=7,
        squareLength=0.015,
        markerLength=0.011,
        dict=cv2.aruco.DICT_5X5_100
    )
    

    detection on calib.io image