I have bought a stereo camera with global shutter and a frame rate of at most 120 fps. https://www.amazon.com/dp/B0D8T3ZSL4?ref_=pe_386300_442618370_TE_sc_as_ri_0#
My next step is to write a program that can show and record a video with desired fps and resolution.
use opencv::{
core, highgui,
prelude::*,
videoio::{self, VideoCapture},
Result,
};
fn open_camera() -> Result<VideoCapture> {
let capture = videoio::VideoCapture::new(2, videoio::CAP_ANY)?;
return Ok(capture);
}
fn main() -> Result<()> {
let window = "video capture";
highgui::named_window(window, highgui::WINDOW_AUTOSIZE)?;
let mut cam = open_camera()?;
let opened = videoio::VideoCapture::is_opened(&cam)?;
if !opened {
panic!("Unable to open default camera!");
}
let width = 3200.0;
let height = 1200.0;
cam.set(videoio::CAP_PROP_FRAME_WIDTH, width)?;
cam.set(videoio::CAP_PROP_FRAME_HEIGHT, height)?;
// Set the frame rate (FPS)
let fps = 60.0;
let fourcc = videoio::VideoWriter::fourcc('M', 'J', 'P', 'G')?;
let mut writer = videoio::VideoWriter::new(
"video_output.avi",
fourcc,
fps,
core::Size::new(width as i32, height as i32),
true,
)?;
if !writer.is_opened()? {
println!("Error: Could not open the video writer.");
}
let mut frame = core::Mat::default();
let mut ctr = 0;
while cam.read(&mut frame)? {
if frame.empty() {
break;
}
writer.write(&frame)?;
highgui::imshow(window, &frame)?;
let key = highgui::wait_key(1)?;
if key > 0 {
break;
}
ctr += 1;
if ctr == 600 {
break;
}
}
cam.release()?;
writer.release()?;
Ok(())
}
When I run this code the frame rate is terrible. Like 1 fps or something. For debugging I tried to run in cheese. There I got 30 fps with full resolution 3200x1200. But I cannot change the fps to 60 fps what I can see.
Then I tried to capture a video using ffmpeg:
ffmpeg -f v4l2 -framerate 60 -video_size 3200x1200 -i /dev/video2 output.mp4
With the following output:
[video4linux2,v4l2 @ 0x5a72cbbd1400] The driver changed the time per frame from 1/60 to 1/2
Input #0, video4linux2,v4l2, from '/dev/video2':
Duration: N/A, start: 2744.250608, bitrate: 122880 kb/s
Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 3200x1200, 122880 kb/s, 2 fps, 2 tbr, 1000k tbn
File 'output.mp4' already exists. Overwrite? [y/N]
The frame rate is lowered to 2 fps.
Then I tried to run v4l2-ctl --list-formats-ext -d 2 with the following output:
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'MJPG' (Motion-JPEG, compressed)
Size: Discrete 3200x1200
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 2560x720
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 1600x600
Interval: Discrete 0.008s (120.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
I then tried to open the camera using qv4land there it seemed to work. Does not seem like I can record a video though.
I am using Rust to learn. I want to be able to programmatically be able to record a video somehow and then do computer vision. The easiest would be to do it in Rust. But other solutions are ok.
Edit I have found some more this morning:
v4l2-ctl -d 2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'MJPG' (Motion-JPEG, compressed)
Size: Discrete 3200x1200
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
[1]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 3200x1200
Interval: Discrete 0.500s (2.000 fps)
Size: Discrete 2560x720
Interval: Discrete 0.500s (2.000 fps)
Size: Discrete 1600x600
Interval: Discrete 0.100s (10.000 fps)
I also found here that order of flags was important for ffmpeg. Running this I can actually record a video with 60 fps:
ffmpeg -framerate 60 -f v4l2 -video_size 3200x1200 -input_format mjpeg -i /dev/video2 output.avi
A drawback is that the images does not look very sharp. You can clearly see the pixels. (I am new to video formats etc as well. Before it has just worked.)
If I change from avito mkvit is slow again.
In the link above I also saw a suggestion to first do:
ffmpeg -framerate 60 -f v4l2 -video_size 3200x1200 -input_format mjpeg -i /dev/video2 -c copy mjpeg.mkv
and then:
ffmpeg -i mjpeg.mkv -c:v libx264 -crf 23 -preset medium -pix_fmt yuv420p out.mkv
which worked. But I am not sure those flags are ideal for the camera I have. I think it is a good start to make it run as expected using command line and ffmpeg. So I know what format to use and that it actually works as intended before doing it programmatically.
There's a limit to how fast raw data can be sent on USB. Uncompressed YUYV takes 16 bits per pixel, so 3200×1200@120fps would be close to 7 Gb/s, which is way above the limit (your camera is USB 2.0, so limited to 480 Mb/s).
MJPEG compresses the images, allowing bigger images to be transmitted faster, but it introduces artefacts that become more visible as the compression ratio increases. Those artefacts are visible as square blocks (usually 16×16 pixels in size). For this reason, you may be better off choosing a lower resolution that will require less compression and result in a higher subjective image quality with fewer visible blocks.