I am attempting to align timelapse images using skimage.feature.orb
to extract keypoints and then filtering them using skimage.measure.ransac
. The transform modelled by RANSAC should then be able to align my images.
The process appears to work well, I get plenty of keypoint matches that are then filtered well by RANSAC. The modelled transformation corrects the rotation perfectly but the translation is way off every time.
Am I simply misunderstanding how the transformation should be applied, or how it is modelled by RANSAC?
# Extract and match features from both images
descriptor_extractor = ORB(n_keypoints = 400, harris_k = 0.0005)
descriptor_extractor.detect_and_extract(image_ref)
descriptors_ref, keypoints_ref = descriptor_extractor.descriptors, descriptor_extractor.keypoints
descriptor_extractor.detect_and_extract(image)
descriptors, keypoints = descriptor_extractor.descriptors, descriptor_extractor.keypoints
# Match features in both images
matches = match_descriptors(descriptors_ref, descriptors, cross_check = True)
# Filter keypoints to remove non-matching
matches_ref, matches = keypoints_ref[matches[:, 0]], keypoints[matches[:, 1]]
# Robustly estimate transform model with RANSAC
transform_robust, inliers = ransac((matches_ref, matches), EuclideanTransform, min_samples = 5, residual_threshold = 0.5, max_trials = 1000)
# Apply transformation to image
image = warp(image, transform_robust.inverse, order = 1, mode = "constant", cval = 0, clip = True, preserve_range = True)
I get similar results with other images. I have also tried using the inliers from RANSAC with skimage.transform.estimate_transform
but it provides identical results to using transform_robust
directly.
It turns out that I needed to invert the translation before applying the transform:
# Robustly estimate transform model with RANSAC
transform_robust, inliers = ransac((matches_ref, matches), EuclideanTransform, min_samples = 5, residual_threshold = 0.5, max_trials = 1000)
# Invert the translation
transform_robust = transform(rotation = transform_robust.rotation) + transform(translation = -flip(transform_robust.translation))
# Apply transformation to image
image = warp(image, transform_robust.inverse, order = 1, mode = "constant", cval = 0, clip = True, preserve_range = True)
The result is not perfect, but adjusting my keypoint selection should get it lined up