Hello I have created an object detection model in create ML and imported it to my swift project but I can't figure out how to use it. Basically i'm just looking to give the model an input and then receive an output. I have opened the Ml model prediction tab and found the input and output variabels but i don't know how to implement it code wise. I have searched on the internet for an answer and found multiple code snippets for running ml models but I can't get them to work.
This is the ML Model: ML Model predictions
This is the code I have tried:
let model = TestObjectModel()
guard let modelOutput = try? model.prediction(imagePath: "images_(2)" as! CVPixelBuffer, iouThreshold: 0.5, confidenceThreshold: 0.5) else {
fatalError("Unexpected runtime error.")
}
print(modelOutput)
When running the code i get this error:
error: Execution was interrupted, reason: EXC_BREAKPOINT (code=1, subcode=0x106c345c0).
The process has been left at the point where it was interrupted, use "thread return -x" to return to the state before expression evaluation.
Ok first to all you have to decide which type of Input you have declared.You can see it, when you click on your model in the project navigator.
For example :
let mlArray = try? MLMultiArray(shape: [1024],dataType: MLMultiArrayDataType.float32)
mlArray![index] = x --> giving your array some data
let input = TestObjectModel(input: mlArray!)
do {
let options = MLPredictionOptions.init()
options.usesCPUOnly = true
let prediction = try? self. TestObjectModel.prediction(input: input, options: options)
--> now you can use prediction which is your output
} catch let err {
fatalError(err.localizedDescription) // Error computing NN outputs error
}
Another example for image as input for your model :
do {
if let resizedImage = resize(image: image, newSize: CGSize(width: 416, height: 416)), let pixelBuffer = resizedImage.toCVPixelBuffer() {
let prediction = try model.prediction(image: pixelBuffer)
let value = prediction.output[0].intValue
print(value)
}
} catch {
print("Error while doing predictions: \(error)")
}
func resize(image: UIImage, newSize: CGSize) -> UIImage? {
UIGraphicsBeginImageContextWithOptions(newSize, false, 0.0)
image.draw(in: CGRect(x: 0, y: 0, width: newSize.width, height: newSize.height))
let newImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return newImage
}
extension UIImage {
func toCVPixelBuffer() -> CVPixelBuffer? {
let attrs = [kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue, kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue] as CFDictionary
var pixelBuffer : CVPixelBuffer?
let status = CVPixelBufferCreate(kCFAllocatorDefault, Int(self.size.width), Int(self.size.height), kCVPixelFormatType_32ARGB, attrs, &pixelBuffer)
guard (status == kCVReturnSuccess) else {
return nil
}
CVPixelBufferLockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer!)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGContext(data: pixelData, width: Int(self.size.width), height: Int(self.size.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer!), space: rgbColorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue)
context?.translateBy(x: 0, y: self.size.height)
context?.scaleBy(x: 1.0, y: -1.0)
UIGraphicsPushContext(context!)
self.draw(in: CGRect(x: 0, y: 0, width: self.size.width, height: self.size.height))
UIGraphicsPopContext()
CVPixelBufferUnlockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
return pixelBuffer
}
}