c++opencvimage-processingoverlayalpha-transparency

Draw cv::Mat onto another cv::Mat at a certain position? (OpenCV C++)


I'm attempting to overlay an image stored in a cv::Mat object onto another cv::Mat image at a certain position using OpenCV's C++ library.

Most of the existing solutions are documented using OpenCV's Python library, and I'm encountering some difficulties translating those solutions to C++.


As a minimal test, suppose I have a simple test image test_1.png of size 100x100:

enter image description here

which I want to overlay in the middle of a black background of size 300x300 as such:

enter image description here

Most of the answers, including this one, make use of the addWeighted function. Attempting to implement the function as such:

#include <opencv2/opencv.hpp>
#include <opencv2/core/mat.hpp>
#include <opencv2/core/core.hpp>

int main()
{
    cv::Mat base = cv::Mat::zeros(300, 300, CV_8UC3);
    cv::Mat overlay = cv::imread("test_1.png", 1);
    cv::Mat result;

    cv::addWeighted(base, 1, overlay, 0.1, 0, result);

    cv::imshow("test", result);
    cv::waitKey(1);
    std::system("pause");

    return 0;
}

Results in a C++ memory error at the addWeighted call, I'm assuming because base and overlay aren't the same size.


Another post answer that asked about the same issue in C++ specificaly provides the following answer:

#include <iostream>

#include <opencv2/opencv.hpp>
#include <opencv2/core/mat.hpp>
#include <opencv2/core/core.hpp>

void OverlayImage(cv::Mat& base, cv::Mat& overlay, std::pair<int, int> pos)
{
    cv::Size fsize{overlay.size()};
    cv::Mat gray;
    cv::Mat mask;
    cv::Mat maskInv;
    cv::cvtColor(overlay, gray, cv::COLOR_BGR2GRAY);
    cv::threshold(gray, mask, 0, 255, cv::THRESH_BINARY);
    cv::bitwise_not(mask, maskInv);

    cv::Mat roi{base(cv::Range(pos.first, pos.second + fsize.width), cv::Range(pos.second, fsize.height))};
    cv::Mat backBg;
    cv::Mat frontBg;
    cv::bitwise_and(roi, roi, backBg, maskInv);
    cv::bitwise_and(overlay, overlay, frontBg, mask);

    cv::Mat result;
    cv::add(backBg, frontBg, result);
    cv::addWeighted(roi, 0.1, result, 0.9, 0.0, result);
    result.copyTo(base(cv::Rect(pos.first, pos.second, fsize.width, fsize.height)));
}

int main()
{
    cv::Mat base = cv::Mat::zeros(300, 300, CV_8UC3);
    cv::Mat overlay = cv::imread("test_1.png", 1);
    
    OverlayImage(base, overlay, std::pair<int, int>(0, 0));

    cv::imshow("test", base);
    cv::waitKey(1);
    std::system("pause");

    return 0;
}

While this solution appears to work with a position of (0, 0):

enter image description here

It doesn't appear to work for any other position other than (0, 0).

If either of the position components are positive, a C++ memory error occurs at the line:

cv::bitwise_and(roi, roi, backBg, maskInv);

If either of the position components are negative, an error occurs within the cv::Mat constructor, from the line:

cv::Mat roi{base(cv::Range(pos.first, pos.second + fsize.width), cv::Range(pos.second, fsize.height))};

What is the proper way to overlay a cv::Mat image onto another cv::Mat image at a certain position using OpenCV's C++ library? (sidenote: the background will not always be black as in my example, and I wish for the image to be partially visible if I place it close to the base images' edges)

Thanks for reading my post, any guidance is appreciated.


Solution

  • cv::Mat roi{base(cv::Range(pos.first, pos.second + fsize.width), cv::Range(pos.second, fsize.height))};
    

    This line is a mess. Maybe, should be

    cv::Mat roi{base(cv::Range(pos.second, pos.second + fsize.height), cv::Range(pos.first, pos.first+fsize.width))};
    

    If you want to support the situation that overlay is not completely included in base, like this:

    inline cv::Range OverlapRange( const cv::Range &A, const cv::Range &B )
    {   return cv::Range( std::max( A.start, B.start ), std::min( A.end, B.end ) ); }
    
    inline bool IsEmpty( const cv::Range &R ){  return ( R.end <= R.start );    }
    
    void OverlayImage( cv::Mat& base, cv::Mat& overlay, std::pair<int, int> pos )
    {
        auto BaseRowRange = OverlapRange( cv::Range{ 0, base.rows }, { pos.second, pos.second+overlay.rows } );
        auto BaseColRange = OverlapRange( cv::Range{ 0, base.cols }, { pos.first, pos.first+overlay.cols } );
        if( IsEmpty( BaseRowRange ) || IsEmpty( BaseColRange ) )return;
    
        auto Base_roi = base(BaseRowRange,BaseColRange);
        auto OV_roi = overlay(
            cv::Rect(
                BaseColRange.start - pos.first,
                BaseRowRange.start - pos.second,
                BaseColRange.end - BaseColRange.start,
                BaseRowRange.end - BaseRowRange.start
            )
        );
    
        cv::Mat BkgndMask;
        cv::inRange( OV_roi, cv::Scalar(0,0,0), cv::Scalar(0,0,0), BkgndMask );
    
        auto Result = Base_roi.clone();
        OV_roi.copyTo( Result, ~BkgndMask );
    
        cv::addWeighted( Base_roi, 0.1, Result, 0.9, 0, Base_roi );
    }
    

    For example, when main() is below, the result becomes:

    enter image description here

    int main()
    {
        //base is small green image
        cv::Mat base = cv::Mat(60, 60, CV_8UC3);
        base = cv::Scalar( 0,128,0 );
        //This is the your test image
        cv::Mat overlay = cv::imread("test1.png", 1);
    
        OverlayImage( base, overlay, std::pair<int, int>(-30, 25) );
        cv::imshow("test", base);
        cv::waitKey(0);
        return 0;
    }