iosyolocoreml

CoreML error: "Image is not expected type 32BGRA or 32ARGB"


I use YOLO based detector with NMS on output. If I use CoreML directly, I get this error:

Image is not expected type 32BGRA or 32ARGB, instead is Unsupported (875704422)

CoreML model input and output formats

Code to call looks like this:

do {
    // I write my own wrappers around `CoreMLHelpers`
    // If I pass image without crop, error is the same
    let inSize = CVPixelBuffer.Size(width: 1280, height: 640)
    let crop = CVPixelBuffer.Rect(x: 0, y: 0, width: inSize.width, height: inSize.height)
    if var buffer = self.preparedBuffer, buffer.size == inSize {
        imageBuffer.crop(rect: crop, outPixelBuffer: &buffer)
    } else {
        self.preparedBuffer = imageBuffer.crop(rect: crop, outSize: inSize)
    }
    
    let out = try model.prediction(image: self.preparedBuffer!, iouThreshold: 0.4, confidenceThreshold: 0.25)
    //..
} catch {
    NSLog("Detector error: \(error)")
}

I took example from Apple with Vision framework. No error occurs with same input:

let imageRequestHandler = VNImageRequestHandler(cmSampleBuffer: buffer, orientation: vOrientation)

Code with CoreML is much easier to understand and anyway I need custom crop of specific area. So it prefer to stay with it. Do anyone has same issue?


Solution

  • I spend 2 days and find the issue. I forgot that my camera out in YUV format. VNImageRequestHandler knows about this format and convert inside by himself. So I changed the output format and everything works:

    let videoOutput = AVCaptureVideoDataOutput()
    videoOutput.videoSettings = [
        String(kCVPixelBufferPixelFormatTypeKey): kCVPixelFormatType_32BGRA
    ]