c++qtvideochat

Display on screen using QAbstractVideoSurface

I am trying to display camera picture on screen using a subclass of QAbstractVideoSurface, and I've no experience on this.

I'd appreciate if anyone can explain how to do it.

Solution

  • The QAbstractVideoSurface is an interface between the producer and consumer of the video frames. You only have two functions to implement to begin with:

    1. supportedPixelFormats so that the producer can select an appropriate format for the QVideoFrame
    2. present which is more generic wording for show\display this frame

    Lets say you want to use a classic QWidget for the display. In this case, you may choose to use a QImage to draw on the widget.

    First Qt is guaranteed to paint a QImage which is RGB24 (or BGR24) on most platforms. So 

    QList<QVideoFrame::PixelFormat> LabelBasedVideoSurface::supportedPixelFormats(
        QAbstractVideoBuffer::HandleType handleType) const
{
    if (handleType == QAbstractVideoBuffer::NoHandle) {
        return QList<QVideoFrame::PixelFormat>()
                << QVideoFrame::Format_RGB24;
    } else {
        return QList<QVideoFrame::PixelFormat>();
    }
}

    Now to present the QVideoFrame, you map its data to a QImage, and paint the QImage to the widget. For simplicity I will use a QLabel, that I access directly (no signal no slot).

    bool LabelBasedVideoSurface::present(const QVideoFrame &frame)
{
    if (notMyFormat(frame.pixelFormat())) {
        setError(IncorrectFormatError);
        return false;
    } else {

        QVideoFrame frametodraw(frame);

        if(!frametodraw.map(QAbstractVideoBuffer::ReadOnly))
        {
           setError(ResourceError);
           return false;
        } 

         //this is a shallow operation. it just refer the frame buffer
         QImage image(
                frametodraw.bits(),
                frametodraw.width(),
                frametodraw.height(),
                frametodraw.bytesPerLine(),
                QImage::Format_RGB444);

        mylabel->resize(image.size());

        //QPixmap::fromImage create a new buffer for the pixmap
        mylabel->setPixmap(QPixmap::fromImage(image));

        //we can release the data
        frametodraw.unmap();

        mylabel->update();

        return true;
    }
}

    This example is obviously not optimal.

    1. It doesnt take cash on the fact that a QVideoFrame might be stored in video memory, because we are drawing using a pixmap.
    2. The conversion from image to pixmap is a unnecessary hit.

    You can write your own widget, and implement a paintEvent for better performance. Also, you have several design liberties on how present() behave. For instance :