opencvcolorsbgr

after merging and extracting the BGR channel, compilation is successful,but there is a run-time error: ILLEGAL OPERATION


I am trying to create 1 CV_8UC3 image from 3 different CV_8UC1 images that I already have, i.e I am trying to allocate the different single channel images that I already have into a single 1 Multi-Dimensional Image. Likely the below code worked flawlessly directly with 3-channel image but If am merging and extracting it comes up with a Runtime error. ILLEGAL OPERATION

#include <opencv2/opencv.hpp>
#include <stdio.h>
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include<vector>
typedef cv::Vec3b Pixel;  //  correct


struct Operator {
    void operator ()(cv::Vec3b &pixel, const int * position) const 
    {    
            pixel[2]*=0.5; 
    }
}; 

int main(int argc, char** argv )
{
    cv::VideoCapture cap(0);
    if(!cap.isOpened())  
        return -1;
    cv::Mat frame1,frame2,for_each,cblue, cgreen, cred; 
    std::vector<cv::Mat> channels { cblue, cgreen, cred};
    for(;;)
    {
            cap >> frame1;    
            cvtColor(frame1, frame1, cv::COLOR_BGR2GRAY); 
            frame1.convertTo(frame2,CV_8U);

            frame2.copyTo(cblue);
            frame2.copyTo(cgreen);
            frame2.copyTo(cred);

            cv::merge(channels, for_each);

            double t1 = (double)cv::getTickCount();
            for_each.forEach<Pixel>(Operator());
            t1 = ((double)cv::getTickCount() - t1)/cv::getTickFrequency();
            std::cout<< "Parallel TEST time " << t1 << std::endl;

            cv::extractChannel (for_each, cblue, 0 );
            cv::imshow("cropped_BGR",frame1);
            cv::imshow("mod_BLUE",cblue);

           if (cv::waitKey(30) == 27) 
           {
                std::cout << "esc key is pressed by user" <<std::endl;
                 break; 
           }
    }
    return 0;

}

I am getting no idea from where this error is coming, any help will be really appreciated , TIA.


Solution

  • The Problem:

    When you do this:

    cv::Mat frame1,frame2,for_each,cblue, cgreen, cred; 
    std::vector<cv::Mat> channels { cblue, cgreen, cred};
    

    channels will have a shallow copy of the cv::Mat cblue, cgreen and cred. This means, that they both will have the same headers with a data pointer that will point to the same place.

    Then you do:

    frame2.copyTo(cblue);
    frame2.copyTo(cgreen);
    frame2.copyTo(cred);
    

    which does a deep copy of frame2 to each of the cv::Mat. The documentation of copyTo says:

    m – Destination matrix. If it does not have a proper size or type before the operation, it is reallocated.

    This means, the pointer to data will change, however it won't change to the cv::Mat inside the vector, they will still point to nullptr but cblue, cgreen and cred will point to the other place.

    I tested it with this code:

      cv::Mat frame(500, 500, CV_8UC3, cv::Scalar::all(111));
      cv::Mat frame1, frame2, cblue, cgreen, cred;
      std::vector<cv::Mat> channels{ cblue, cgreen, cred };
      // at this point all data members of mat will point to nullptr except frame
      cv::cvtColor(frame, frame, cv::COLOR_BGR2GRAY);
      frame.convertTo(frame2, CV_8U);
    
      frame2.copyTo(cblue);
      frame2.copyTo(cgreen);
      frame2.copyTo(cred);
    
      // at this point all point to another place except the ones inside the vector
    

    Possible solutions:

    1) Create references and not copies:

      cv::Mat frame1, frame2;
      std::vector<cv::Mat> channels(3);
      cv::Mat& cblue = channels[0], &cgreen=channels[1], &cred=channels[2];
    

    2) Use the channels directly instead of using other variables

            frame2.copyTo(channels[0]);
            frame2.copyTo(channels[1]);
            frame2.copyTo(channels[2]);
    

    3) Create the vector inside the loop

            frame2.copyTo(cblue);
            frame2.copyTo(cgreen);
            frame2.copyTo(cred);
            std::vector<cv::Mat> channels { cblue, cgreen, cred};
            cv::merge(channels, for_each);
    

    4) Your code is equivalent to:

    cvtColor(frame1, frame1, cv::COLOR_BGR2GRAY);
    cvtColor(frame1, for_each, cv::COLOR_GRAY2BGR);
    

    This will create a 3 channel image of the grey values, which is basically a copy of the grey mat in each channel...


    5) One more thing:

    frame1.convertTo(frame2,CV_8U);
    

    This is not necessary, because it is already a CV_8U mat, because the previous instruction converted it to greyscale which is CV_8U and then you can even create a vector there without doing deep copy (it will deep copy to for_each).

        std::vector<cv::Mat> channels { frame1, frame1, frame1};
        cv::merge(channels, for_each);
    

    And one additional thing, not related to the error:

            cv::extractChannel (for_each, cblue, 0 );
            cv::imshow("cropped_BGR",frame1);
            cv::imshow("mod_BLUE",cblue);
    

    will display exactly the same image :) or should at least.