I'm working in a application that's use opencv to capture frames and show it's in a window. The code use's the frame to record a video too, but, when I show the frame and record at the same time, the frame rate in screen decreases. Can I use a atomic variable to store the frames? like that:
std::atomic<cv::Mat> atomicFrame;
atomicFrame.store(newFrame);
cv::Mat sameFrame = atomicFrame.load();
My idea is uses the atomic variable to access the same frame at the same time in show_frame_on_screen
function and record_frames
function, but, when i set the variable in my header file, my cpp program get some errors.
A part of my header file:
#include <iostream>
#include <iomanip>
#include <chrono>
#include <ctime>
#include <thread>
#include <atomic>
class MainWindow : public QMainWindow
{
Q_OBJECT
private:
Ui::MainWindow *ui;
QFutureWatcher<void> watcher_record_frame;
QFutureWatcher<void> watcher_show_frame;
QFutureWatcher<void> watcher_get_frame;
QProcess *processKeypad;
QTimer *savingTimer;
QTimer *framesRecordTimer;
cv::VideoWriter *videoWriter;
std::queue<cv::Mat> framesQueue;
std::atomic<cv::Mat> frameAtomic;
};
A part of my .cpp file:
#include "mainwindow.h"
#include "ui_mainwindow.h"
MainWindow::MainWindow(QWidget *parent)
: QMainWindow(parent)
, ui(new Ui::MainWindow)
{
ui->setupUi(this);
}
In line ui->setupUi(this);
i get the error when passing the this
parameter:
mainwindow.cpp:10:17: error: cannot initialize a parameter of type 'QMainWindow *' with an rvalue of type 'MainWindow *'
ui_mainwindow.h:40:31: note: passing argument to parameter 'MainWindow' here
If I comment the line std::atomic<cv::Mat> frameAtomic;
or change to std::atomic<int> frameAtomic;
the error in cpp file disappear.
This application is running in a RaspBerry Pi 4 8gb, with raspbian OS.
EDIT: At this moment, my code have this functions to refresh the frames, show then and record then.
void MainWindow::update_frame()
{
while (fvs.more())
{
cv::Mat frame = fvs.read();
if (recording)
{
framesQueue.push(frame);
}
emit show_frame_on_screen(frame);
}
}
void MainWindow::show_frame_on_screen(cv::Mat frame)
{
cv::resize(frame, frame, cv::Size(1024, 600));
QImage image(frame.data, frame.cols,
frame.rows, frame.step,
QImage::Format_BGR888);
ui->layer_frame->setPixmap(QPixmap::fromImage(image));
}
void MainWindow::thread_record_frame()
{
while (!framesQueue.empty()){
cv::Mat newFrame = framesQueue.front();
framesQueue.pop();
newFrame = write_text_on_frame(newFrame);
videoWriter->write(newFrame);
if (!recording && framesQueue.empty()){
savingTimer->stop();
ui->layer_loading->setText("");
videoWriter->release();
}
}
}
I'm using std::queue<cv::Mat> framesQueue;
to storage all frames and record then.
std::atomic<cv::Mat> atomicFrame;
makes little sense.
cv::Mat
is an n-dimensional array, and making it atomic isn't feasible.
Atomics are only meant for very small objects.
For anything large, reading or writing the atomic object wouldn't be lock-free and might be no better than guarding all access with a std:mutex
.
Also, you're most likely interested in accessing small parts of the atomicFrame
, not just replacing it in its entirety. std::atomic
does not give you that option.
Instead, you should double-buffer, which means that you use two separate buffers to show_frame_on_screen
and record_frames
.
This allows you to simultaneously draw and record on separate threads without contention.
When a frame is done drawing, you swap the buffers.
std::atomic<cv::Mat>
is not useful for this; you could instead use a pair of cv::Mat*
, guarded by a std::atomic<bool>
.
std::array<cv::Mat, 2> buffers;
cv::Mat* screen_buffer = &buffers[0];
cv::Mat* record_buffer = &buffers[1];
std::atomic<bool> request_frame = false;
void show_frame_on_screen() {
// if an old request for a frame is still active, block until it isn't
request_frame.wait(true);
do_show(screen_buffer);
request_frame.store(true);
request_frame.notify_one();
}
There are two approaches to implementing record_frames
:
// low-energy variant: only record a frame when needed
void record_frames() {
request_frame.wait(false);
do_record(record_buffer);
std::swap(screen_buffer, record_buffer);
request_frame.store(false);
request_frame.notify_one();
}
// low-latency variant: never wait for the frame to be shown on screen, always draw
// (this variant is wait-free)
void record_frames() {
do_record(record_buffer);
if (request_frame.load()) {
std::swap(screen_buffer, record_buffer);
request_frame.store(false);
request_frame.notify_one();
}
}
As for the error caused by ui->setupUi(this);
, it's unclear why this happens, without seeing your full code, or a minimal reproducible example.