Using following Kotlin loop we are showing camera feed, detect human faces in feed and drawing face bounding boxes on video feed on screen.
Using workReducer counter we check face availability ever 0.5 second in image feed.
The problem is video feed freezes after in couple of seconds the first time we activated camera and after more or less 20 seconds if we restart camera using some buttons on screen.
It has been quite sometime I have been looking for solution without success.
Ps: If no face has been detected it never freezes. Also: when I remove
android.graphics.Rect( face.boundingBox.left,
face.boundingBox.top,
face.boundingBox.right,
face.boundingBox.bottom
) by commenting it freezes les often
imageReader.setOnImageAvailableListener({ reader ->
val image = reader.acquireLatestImage() ?: return@setOnImageAvailableListener
val bitImage = recordJpeg( image, cx)
bitImage?.let { bitmapCallback.onBitmapUpdated(it) }
if (workReducer > workReducerLimit) {
workReducer = 0
val inputImage = InputImage.fromMediaImage(image, 270)
faceDetector.process(inputImage).addOnSuccessListener { faces ->
val rects = faces.map { face ->
android.graphics.Rect(
face.boundingBox.left,
face.boundingBox.top,
face.boundingBox.right,
face.boundingBox.bottom
)
}
faceRectsCallback.onFaceRectsUpdated(rects)
}
.addOnFailureListener { e ->
image.close()
}
.addOnCompleteListener {
image.close()
}
} else {
image.close()
}
workReducer += 1
// Camera image read loop
}, Handler(context.mainLooper))
Well, freezing thing is happened because of a constant type not because of face detection functions or APIs.
After many hours I discovered by chance that the cause of freezing was inputImage constant.
I declare it as private var imageReader : ImageReader? = null outside of class my functions run inside.
Well it shows live feed from camera, draw face bounding box on it and shows bitmap images I am using to record and later convert video file smoothly.
While I was trying to find the cause of the problem I had to upgrade many things related to Kotlin, and Java. Now, Xcode looks very pretty after all these.
override fun onOpened(camera : CameraDevice) {
Log.d("my", "onOpened")
currentCameraDevice = camera
val surfaceTexture = textureView.surfaceTexture?.apply {
setDefaultBufferSize(previewSize.width, previewSize.height)
}
val previewSurface = Surface(surfaceTexture)
imageReader = ImageReader.newInstance(
previewSize.width,
previewSize.height,
ImageFormat.YUV_420_888,
2
)!!
val imageReaderSurface = imageReader!!.surface
val captureRequestBuilder =
camera.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW).apply {
addTarget(previewSurface)
addTarget(imageReader!!.surface)
}
val faceDetector = FaceDetection.getClient(
FaceDetectorOptions.Builder()
.setPerformanceMode(FaceDetectorOptions.PERFORMANCE_MODE_FAST)
.build()
)
val sessionConfig = SessionConfiguration(
SessionConfiguration.SESSION_REGULAR,
listOf(
OutputConfiguration(previewSurface),
OutputConfiguration(imageReaderSurface)
),
Executors.newSingleThreadExecutor(),
object : CameraCaptureSession.StateCallback() {
override fun onConfigured(session: CameraCaptureSession) {
currentCaptureSession = session
session.setRepeatingRequest(captureRequestBuilder.build(), null, null)
callback.onPreviewSizeAvailable(previewSize)
}
override fun onConfigureFailed(session: CameraCaptureSession) {
Log.e("my", "Failed to configure camera capture session")
}
}
)
camera.createCaptureSession(sessionConfig)
imageReader!!.setOnImageAvailableListener(
{ reader ->
val image = reader.acquireLatestImage()
?: return@setOnImageAvailableListener
try {
// To check if can produce bitmap.
val bitImage = recordJpeg(image, cx)
bitImage?.let { bitmapCallback.onBitmapUpdated(it) }
if(workReducer > workReducerLimit) {
workReducer = 0
val inputimage = InputImage.fromMediaImage(image, 270)
faceDetector.process(inputimage)
.addOnSuccessListener { faces ->
val rects = faces.map { face ->
if(!faceDetected && setup.recOn == 1) {
afterBitmapArray.clear()
preRecSize = bitmapArray.size
afterBitmapArray =
bitmapArray.toMutableList()
faceDetected = true
}
android.graphics.Rect(
face.boundingBox.left,
face.boundingBox.top,
face.boundingBox.right,
face.boundingBox.bottom
)
}
faceRectsCallback.onFaceRectsUpdated(rects)
}
.addOnFailureListener { e ->
Log.d("my", "Face detection failed: ${e.message}", e)
}
.addOnCompleteListener { image.close() }
} else {
image.close()
}
workReducer += 1
} catch (e : Exception) {
Log.d("my", "Error processing image: ${e.message}")
}
}, Handler(context.mainLooper)
)
}