pythonopencvcomputer-visionimage-stitchingopencv-stitching

Stitching of video frames from two calibrated CCTV cameras


I have two CCTV cameras setup in a room with sufficient overlapping region. The stream from these cameras is calibrated. I used OpenCV stitching python code from here to stitch two respective frames of videos from two cameras. The stitching output has shaky movements in it. How to overcome these movements? Please suggest.


Solution

  • Shaky movements from stitching image streams from cameras were already reported here. The stitching pipeline will calibrate its internal cameras independently on each stitch. In case of video stitching, the issue is that this results in some warping from one frame to the other. The solution is to calibrate the cameras only with the first frame and then afterwards only use those cameras. This will also save the time needed for feature matching for each frame. To do this, you could inherit a VideoStitcher from Stitcher where you overwrite initialize_stitcher() and stitch()

    from stitching import Stitcher
    from stitching.images import Images
    
    class VideoStitcher(Stitcher):
    
        def initialize_stitcher(self, **kwargs):
            super().initialize_stitcher(kwargs)
            self.cameras = None
            self.cameras_registered = False
            
        def stitch(self, images, feature_masks=[]):
            self.images = Images.of(
                images, self.medium_megapix, self.low_megapix, self.final_megapix
            )
    
            if not self.cameras_registered:
                imgs = self.resize_medium_resolution()
                features = self.find_features(imgs, feature_masks)
                matches = self.match_features(features)
                imgs, features, matches = self.subset(imgs, features, matches)
                cameras = self.estimate_camera_parameters(features, matches)
                cameras = self.refine_camera_parameters(features, matches, cameras)
                cameras = self.perform_wave_correction(cameras)
                self.estimate_scale(cameras)
                self.cameras = cameras
                self.cameras_registered = True
    
            imgs = self.resize_low_resolution()
            imgs, masks, corners, sizes = self.warp_low_resolution(imgs, self.cameras)
            self.prepare_cropper(imgs, masks, corners, sizes)
            imgs, masks, corners, sizes = self.crop_low_resolution(
                imgs, masks, corners, sizes
            )
            self.estimate_exposure_errors(corners, imgs, masks)
            seam_masks = self.find_seam_masks(imgs, corners, masks)
    
            imgs = self.resize_final_resolution()
            imgs, masks, corners, sizes = self.warp_final_resolution(imgs, self.cameras)
            imgs, masks, corners, sizes = self.crop_final_resolution(
                imgs, masks, corners, sizes
            )
            self.set_masks(masks)
            imgs = self.compensate_exposure_errors(corners, imgs)
            seam_masks = self.resize_seam_masks(seam_masks)
    
            self.initialize_composition(corners, sizes)
            self.blend_images(imgs, seam_masks, corners)
            return self.create_final_panorama()