I use YOLO based detector with NMS on output. If I use CoreML
directly, I get this error:
Image is not expected type 32BGRA or 32ARGB, instead is Unsupported (875704422)
Code to call looks like this:
do {
// I write my own wrappers around `CoreMLHelpers`
// If I pass image without crop, error is the same
let inSize = CVPixelBuffer.Size(width: 1280, height: 640)
let crop = CVPixelBuffer.Rect(x: 0, y: 0, width: inSize.width, height: inSize.height)
if var buffer = self.preparedBuffer, buffer.size == inSize {
imageBuffer.crop(rect: crop, outPixelBuffer: &buffer)
} else {
self.preparedBuffer = imageBuffer.crop(rect: crop, outSize: inSize)
}
let out = try model.prediction(image: self.preparedBuffer!, iouThreshold: 0.4, confidenceThreshold: 0.25)
//..
} catch {
NSLog("Detector error: \(error)")
}
I took example from Apple with Vision
framework. No error occurs with same input:
let imageRequestHandler = VNImageRequestHandler(cmSampleBuffer: buffer, orientation: vOrientation)
Code with CoreML
is much easier to understand and anyway I need custom crop of specific area. So it prefer to stay with it. Do anyone has same issue?
I spend 2 days and find the issue. I forgot that my camera out in YUV format. VNImageRequestHandler
knows about this format and convert inside by himself. So I changed the output format and everything works:
let videoOutput = AVCaptureVideoDataOutput()
videoOutput.videoSettings = [
String(kCVPixelBufferPixelFormatTypeKey): kCVPixelFormatType_32BGRA
]