I'm writing a QT GUI application in which a live stream of a connected camera is shown on a QGraphicsview. Therefore an openCV image is first converted to a QImage and than to a QPixmap. This is added to the QGraphicsScene of the QGraphicsView.
The bandwidth is not a problem, the cameras are connected via ethernet or USB.
I am testing the performance with the Analyze Toole build in Visual Studio 2012 and it shows that the conversion to the QPixmap is very slow and takes 60% of the computation time (of displaying the image), so that I end up with 1 FPS or so. The images are 2560 by 1920 or even bigger. Scaling the cv::Ptr stream_image before converting it to a QImage improves the performance significantly but I need all the image detail in the image.
EDIT Here is some code how I do the conversion:
cv::Ptr<IplImage> color_image;
// stream_image is a cv::Ptr<IplImage> and holds the current image from the camera
if (stream_image->nChannels != 3) {
color_image = cvCreateImage(cvGetSize(stream_image), IPL_DEPTH_8U, 3);
cv::Mat gr(stream_image);
cv::Mat col(color_image);
cv::cvtColor(gr, col, CV_GRAY2BGR);
}
else {
color_image = stream_image;
}
QImage *tmp = new QImage(color_image->width, color_image->height, QImage::Format_RGB888);
memcpy(tmp->bits(), color_image->imageData, color_image->width * color_image->height * 3);
// update Scene
m_pixmap = QPixmap::fromImage(*tmp); // this line takes the most time!!!
m_scene->clear();
QGraphicsPixmapItem *item = m_scene->addPixmap(m_pixmap);
m_scene->setSceneRect(0,0, m_pixmap.width(), m_pixmap.height());
delete tmp;
m_ui->graphicsView->fitInView(m_scene.sceneRect(),Qt::KeepAspectRatio);
m_ui->graphicsView->update();
EDIT 2 I tested the method from from Thomas answer, but it is as slow as my method.
QPixmap m_pixmap = QPixmap::fromImage(QImage(reinterpret_cast<uchar const*>(color_image->imageData),
color_image->width,
color_image->height,
QImage::Format_RGB888));
EDIT 3 I tried to incorporate Thomas second suggestion:
color_image = cvCreateImage(cvGetSize(resized_image), IPL_DEPTH_32F, 3);
//[...]
QPixmap m_pixmap = QPixmap::fromImage(QImage(
reinterpret_cast<uchar const*>( color_image->imageData),
color_image->width,
color_image->height,
QImage::Format_RGB32));
But that crashes when the drawEvent of the Widget is called.
Q: Is there a way to display the image stream in a QGraphicsView without converting it to a QPixmap first or any other fast/performant way? The QGraphicsView is importent since I want to add overlays to the image.
I have figured out a solution that works for me but also tested a little with different methods and how they perform:
Method one is performant even in debug mode and takes only 23.7 % of the execution time of the drawing procedure (using ANALYZE in VS2012):
color_image = cvCreateImage(cvGetSize(stream_image), IPL_DEPTH_8U, 4);
cv::Mat gr(stream_image);
cv::Mat col(color_image);
cv::cvtColor(gr, col, CV_GRAY2RGBA,4);
QPixmap m_pixmap = QPixmap::fromImage(QImage(reinterpret_cast<uchar const*>( color_image->imageData),
color_image->width,
color_image->height,
QImage::Format_ARGB32));
Method two is still performant in debug mode taking 42,1% of the execution time. when the following enum is used in QPixmap::fromeImage instead
QImage::Format_RGBA8888
Method three is the one I showed in my question and it is very slow in debug builds being responsible for 68,3% of the drawing workload.
However, when I compile in release all three methods are seamingly equally performant.