javawebcam-capture

Webcam api : Maximizing the 720p capture


I am attempting to capture a video recording through an external camera, Logitec C922. Using java, I can make this possible through webcam api.

    <dependency>
        <groupId>com.github.sarxos</groupId>
        <artifactId>webcam-capture</artifactId>
        <version>0.3.10</version>
    </dependency>
    <dependency>
        <groupId>xuggle</groupId>
        <artifactId>xuggle-xuggler</artifactId>
        <version>5.4</version>
    </dependency>

However, for the life of me, I cannot make it record at 60FPS. The video randomly stutters when stored, and is not smooth at all.

I can connect to the camera, using the following details.

final List<Webcam> webcams = Webcam.getWebcams();

for (final Webcam cam : webcams) {
    if (cam.getName().contains("C922")) {
        System.out.println("### Logitec C922 cam found");
        webcam = cam;
        break;
    }
}

I set the size of the cam to the following:

final Dimension[] nonStandardResolutions = new Dimension[] { WebcamResolution.HD720.getSize(), };
webcam.setCustomViewSizes(nonStandardResolutions);
webcam.setViewSize(WebcamResolution.HD720.getSize());
webcam.open(true);

And then I capture the images:

while (continueRecording) {
    // capture the webcam image
    final BufferedImage webcamImage = ConverterFactory.convertToType(webcam.getImage(),
                    BufferedImage.TYPE_3BYTE_BGR);
    final Date timeOfCapture = new Date();

    // convert the image and store
    final IConverter converter = ConverterFactory.createConverter(webcamImage, IPixelFormat.Type.YUV420P);
    final IVideoPicture frame = converter.toPicture(webcamImage,
                    (System.currentTimeMillis() - start) * 1000);

    frame.setKeyFrame(false);
    frame.setQuality(0);
    writer.encodeVideo(0, frame);

}

My writer is defined as follows:

final Dimension size = WebcamResolution.HD720.getSize();
final IMediaWriter writer = ToolFactory.makeWriter(videoFile.getName());
writer.addVideoStream(0, 0, ICodec.ID.CODEC_ID_H264, size.width, size.height);

I am honestly not sure what in my code could be causing this. Given that I lower the resolution, I get no problems. ( 480p ) Could the issue be with the codes I am using?


Solution

  • As some of the comments mentioned, introducing queues does solve the problem. Here is the general logic to performs the needed steps. Note, I've setup my code for a lower resolution, as it allows me to capture 100FPS per sec. Adjust as needed.

    Class to link the image/video capture and class to edit it :

    public class WebcamRecorder {
    
        final Dimension size = WebcamResolution.QVGA.getSize();
        final Stopper stopper = new Stopper();
    
        public void startRecording() throws Exception {
    
            final Webcam webcam = Webcam.getDefault();
            webcam.setViewSize(size);
            webcam.open(true);
    
            final BlockingQueue<CapturedFrame> queue = new LinkedBlockingQueue<CapturedFrame>();
            final Thread recordingThread = new Thread(new RecordingThread(queue, webcam, stopper));
            final Thread imageProcessingThread = new Thread(new ImageProcessingThread(queue, size));
    
            recordingThread.start();
            imageProcessingThread.start();
        }
    
        public void stopRecording() {
            stopper.setStop(true);
        }
    
    }
    

    RecordingThread :

    public void run() {
        try {
            System.out.println("## capturing images began");
            while (true) {
                final BufferedImage webcamImage = ConverterFactory.convertToType(webcam.getImage(),
                        BufferedImage.TYPE_3BYTE_BGR);
                final Date timeOfCapture = new Date();
                queue.put(new CapturedFrame(webcamImage, timeOfCapture, false));
                if (stopper.isStop()) {
                    System.out.println("### signal to stop capturing images received");
                    queue.put(new CapturedFrame(null, null, true));
                    break;
                }
            }
        } catch (InterruptedException e) {
            System.out.println("### threading issues during recording:: " + e.getMessage());
        } finally {
            System.out.println("## capturing images end");
            if (webcam.isOpen()) {
                webcam.close();
            }
        }
    }
    

    ImageProcessingThread:

    public void run() {
        writer.addVideoStream(0, 0, ICodec.ID.CODEC_ID_H264, size.width, size.height);
        try {
            int frameIdx = 0;
            final long start = System.currentTimeMillis();
            while (true) {
                final CapturedFrame capturedFrame = queue.take();
                if (capturedFrame.isEnd()) {
                    break;
                }
                final BufferedImage webcamImage = capturedFrame.getImage();
                size.height);
    
                // convert the image and store
                final IConverter converter = ConverterFactory.createConverter(webcamImage, IPixelFormat.Type.YUV420P);
                final long end = System.currentTimeMillis();
                final IVideoPicture frame = converter.toPicture(webcamImage, (end - start) * 1000);
    
                frame.setKeyFrame((frameIdx++ == 0));
                frame.setQuality(0);
                writer.encodeVideo(0, frame);
            }
        } catch (final InterruptedException e) {
            System.out.println("### threading issues during image processing:: " + e.getMessage());
        } finally {
            if (writer != null) {
                writer.close();
            }
        }
    

    The way it works is pretty simple. The WebcamRecord class creates an instance of a queue that is shared between the video capture and the image processing. The RecordingThread sends bufferedImages to the queue ( in my case, it a pojo, called CapturedFrame ( which has a BufferedImage in it ) ). The ImageProcessingThread will listen and pull data from the queue. If it does not receive a signal that the writing should end, the loop never dies.