swingkotlinjavafxqr-codejavacv

How to display OpenCV webcam capture frame in JavaFX app


I'm working on creating desktop app using JavaFX, which allows you to scan qr codes from a webcam.

I decided to choose JavaCV to handle webcam capturning. However, the problem is that the CanvasFrame class creates a Swing JFrame. My main goal is to find the best way to integrate this with JavaFX components.

My question is whether it is possible to create CanvasFrame in JPanel(or other Swing/JavaFx component), not in JFrame. In this option I would wrap JPanel into SwingNode - it's solve my integration problem.

I'm also asking for other suggestions that solves JavaFX with JavaCV integration problem in my case. Maybe there is a direct way to embed a camera screen into a JavaFx component.

I'm pasting the test code below. My code is written in kotlin, but it doesn't affect the problem:

import com.google.zxing.*
import com.google.zxing.client.j2se.BufferedImageLuminanceSource
import com.google.zxing.common.HybridBinarizer
import org.bytedeco.javacv.*
import java.awt.image.BufferedImage
import java.util.*
import java.util.concurrent.Executors

class Test {
    companion object {
        @JvmStatic
        fun main(args: Array<String>) {
            Executors.newSingleThreadExecutor().execute { testWebcam() }
        }

        private fun testWebcam() {
            val grabber: OpenCVFrameGrabber = OpenCVFrameGrabber(0);
            val canvasFrame: CanvasFrame = CanvasFrame("Cam")
            grabber.start()

            while (canvasFrame.isVisible) {
                val frame: Frame = grabber.grabFrame() ?: break
                canvasFrame.showImage(frame)
                decodeQrCode(grabber)
            }
        }

        private fun decodeQrCode(grabber: OpenCVFrameGrabber) {
            val java2DFrameConverter: Java2DFrameConverter = Java2DFrameConverter()

            val frame: Frame = grabber.grabFrame()
            val image = java2DFrameConverter.getBufferedImage(frame)

            val decodedQr = parseQr(image)
            println(decodedQr)
        }

        private fun parseQr(image: BufferedImage): String? {

            val reader: MultiFormatReader = MultiFormatReader()
            val binaryBitmap: BinaryBitmap =
                BinaryBitmap(HybridBinarizer(BufferedImageLuminanceSource(image)))

            val hints: Hashtable<DecodeHintType, Any> = Hashtable()
            hints[DecodeHintType.CHARACTER_SET] = "UTF-8"
            hints[DecodeHintType.POSSIBLE_FORMATS] = listOf(BarcodeFormat.QR_CODE)

            return try {
                reader.decode(binaryBitmap, hints).text
            } catch (e: NotFoundException) {
                null;
            }

        }
    }
}

Solution

  • I solved my problem. In my case, the best solution was to use Java2DFrameConverter:

    import javafx.application.Application
    import javafx.embed.swing.SwingFXUtils
    import javafx.scene.Scene
    import javafx.scene.image.ImageView
    import javafx.scene.image.WritableImage
    import javafx.scene.layout.VBox
    import javafx.stage.Stage
    import org.bytedeco.javacv.Frame
    import org.bytedeco.javacv.Java2DFrameConverter
    import org.bytedeco.javacv.OpenCVFrameGrabber
    import java.awt.image.BufferedImage
    import java.util.concurrent.Executors
    
    class StackOverflow : Application() {
        private val java2DFrameConverter: Java2DFrameConverter = Java2DFrameConverter()
    
        companion object {
            @JvmStatic
            fun main(args: Array<String>) {
                launch(StackOverflow::class.java)
            }
        }
    
        override fun start(primaryStage: Stage) {
            val grabber: OpenCVFrameGrabber = OpenCVFrameGrabber(0)
            grabber.start()
    
            val imageView: ImageView = ImageView()
    
            Executors.newSingleThreadExecutor().execute {
                while (true) {
                    val frame = grabber.grabFrame()
                    imageView.image = frameToImage(frame)
                }
            }
    
            val scene: Scene = Scene(VBox(imageView), 800.0, 800.0)
            primaryStage.scene = scene
            primaryStage.show()
        }
    
        private fun frameToImage(frame: Frame): WritableImage {
            val bufferedImage: BufferedImage = java2DFrameConverter.getBufferedImage(frame)
            return SwingFXUtils.toFXImage(bufferedImage, null)
        }
    }