matlabimage-processingcomputer-visionvisual-odometry

Image Rectification of Stereo Images


My final goal is to make visual odometry. For this, I am going to collect my own image dataset. But first of all, I am triying to do image rectification with raw images of Malaga Dataset . The dataset is stereo images dataset. I already downloaded the malaga-urban-dataset-extract-07.zip file and I tryed to do image rectification with raw images. I used the MATLAB code :

path='C:\Users\Jasmine\Desktop\malaga_raw';
for i=0:2120
I1 = imread([path '\image_0\' sprintf('%06d.jpg',i)]); % left camera images
I2= imread([path '\image_1\' sprintf('%06d.jpg',i)]);  % right camera images
I1gray = histeq(rgb2gray(I1));
I2gray = histeq(rgb2gray(I2));
blobs1 = detectSURFFeatures(I1gray, 'MetricThreshold', 2000);
blobs2 = detectSURFFeatures(I2gray, 'MetricThreshold', 2000);
[features1, validBlobs1] = extractFeatures(I1gray, blobs1);
[features2, validBlobs2] = extractFeatures(I2gray, blobs2);
indexPairs = matchFeatures(features1, features2, 'Metric', 'SAD', ...
    'MatchThreshold', 5);
matchedPoints1 = validBlobs1(indexPairs(:,1),:);
matchedPoints2 = validBlobs2(indexPairs(:,2),:);
[fMatrix, epipolarInliers, status] = estimateFundamentalMatrix(...
    matchedPoints1, matchedPoints2, 'Method', 'RANSAC', ...
    'NumTrials', 10000, 'DistanceThreshold', 0.1, 'Confidence', 99.99);

if status ~= 0 || isEpipoleInImage(fMatrix, size(I1)) ...
        || isEpipoleInImage(fMatrix', size(I2))
    error(['Either not enough matching points were found or '...
        'the epipoles are inside the images. You may need to '...
        'inspect and improve the quality of detected features ',...
        'and/or improve the quality of your images.']);
end

inlierPoints1 = matchedPoints1(epipolarInliers, :);
inlierPoints2 = matchedPoints2(epipolarInliers, :);

[t1, t2] = estimateUncalibratedRectification(fMatrix, ...
    inlierPoints1.Location, inlierPoints2.Location, size(I2));
tform1 = projective2d(t1);
tform2 = projective2d(t2);
[I1Rect, I2Rect] = rectifyStereoImages(I1, I2, tform1, tform2, 'OutputView', 'Full');
folder = 'C:\Users\Jasmine\Desktop\malaga_raw';
newimagename_1 = [folder '\i_0\' sprintf('%06d.jpg',i)]; % left camera images
imwrite(I1Rect,newimagename_1)
newimagename_2 = [folder '\i_1\' sprintf('%06d.jpg',i)]; %right camera images
imwrite(I2Rect,newimagename_2)
end

But It gaves me the error in 152th frame:

if status ~= 0 || isEpipoleInImage(fMatrix, size(I1)) ...
        || isEpipoleInImage(fMatrix', size(I2))
    error(['Either not enough matching points were found or '...
        'the epipoles are inside the images. You may need to '...
        'inspect and improve the quality of detected features ',...
        'and/or improve the quality of your images.']);
end

Except from this error, already the perious (from 1th frame to 151th frame) rectifited images aren't resemble with the orginal rectifited images of Malaga dataset. For example; If I show to you respectiely the 20th frame of Malaga Raw Image Dataset, the rectification result of the Matlab code that I used and the original rectifited image of Malaga Dataset : The 20th frame of Malaga Dataset Raw Images 20th frame rectification result of the Matlab code Original Rectificated Image of 20th frame of Malaga Dataset

I know that I used the Matlab rectification code for uncalibrated cameras. But, it is already calibrated raw image dataset in Malaga Dataset. Another rectification matlab code is this But the Malaga dataset images (or the dataset that I will collect) doesn't contain the Checker board in it. So, I couldn't understand that how can i use this code with images that doesn't contain a Checker board.

I have three questions :

  1. How can I solve the error problem that I wrote in the above?
  2. Why the result of the Matlab rectification isn't resemble to the original rectification image?
  3. How can i rectify the raw images with another way ?

Solution

  • The difference most probably comes from the fact that you used estimateUncalibratedRectification() whereas the rectified images of the Malaga dataset rely on a proper rectification using calibration parameters.

    In the Malaga dataset webpage, you can find the stereo calibration parameters (e.g. http://ingmec.ual.es/~jlblanco/malaga-urban-dataset/calibration/camera_params_rectified_a=0_800x600.txt .

    Using this approach, you should get the same results.