Swift - switch between Core ML Model

I'm trying to compare predictions from different MLModels in SwiftUI. To do that I have to switch between them, but can't because every ML variable has its own class, so I get the error:

Cannot assign value of type 'ModelOne' to type 'ModelTwo'

Here's an example code:

import Foundation
import CoreML
import SwiftUI

let modelone = { //declaration model 1
do {
    let config = MLModelConfiguration()
    return try ModelOne(configuration: config)
} catch {
    /*...*/
}
}()

let modeltwo = { //declaration model 2
do {
    let config = MLModelConfiguration()
    return try ModelTwo(configuration: config)
} catch {
    /*...*/
}
}()

var imageused : UIImage! //image to classify
var modelstring = ""     //string of model user chosen
var modelchosen = modelone

Button(action: { //button user decide to use model two
   modelstring = "Model Two"

}) {/*...*/}

/*...*/
func classifyphoto() {

    guard let image = imageused as UIImage?,
          let imagebuffer = image.convertToBuffer() else {
        return
        
    }
    if  modelstring == "Model Two" { //if the user chosen model two, use ModelTwo
        modelchosen = modeltwo // Error: Cannot assign value of type 'ModelOne' to type 'ModelTwo'
    } else {
        modelchosen = modelone}
    
    let output = try? modelchosen.prediction(image: imagebuffer) //prediction with model chosen
 
    if let output = output {
        let results = output.classLabelProbs.sorted { $0.1 > $1.1 }
        _ = results.map { /*...*/
        }
    }
}

Thank you!

Swift is a statically typed language which means that in the general case you cannot assign a variable of one type to a variable of another type:

var int: Int = 42
int = "Hello, world!"  // Not allowed: cannot assign String to Int

The problem is that modelchosen is of type ModelOne since it is initialized with modelone, thus, you cannot later assign modeltwo to it as you are trying to do.

To make that working, you have first to identify the common capabilities of ModelOne and ModelTwo. Take a look at their definition. For instance, do their .predict(image:) method return the same type? It looks like you are trying to do image classification, so a common capability could be the capability to return a String describing the image (or a list of potential objects, etc.).

When you'll have identified the common capability, you'll be able to define the common interface of your different types. This common interface can be expressed in many ways:

• Using a base class
• Using a protocol
• Using an enum with payloads (union)

The following examples suppose that the common capabilities are:

• The two networks can both be initialized with a MLModelConfiuration
• They are used for image classification, i.e. they predict label (a String) describing a given image

Using a base class

The base class definition expresses those requirements like this:

class MLClassifier {
  init(from config: MLModelConfig) { 
    fatalError("not implemented")
  }

  func classify(image: ImageBuffer) -> String {
    fatalError("not implemented")
  }
}

You then derive this base class for the two models (example with the first one:

final class ModelOne: MLClassifier {
  init(from config: MLModelConfig) {
    // the specific implementation for `ModelOne`...
  }

  func classify(image: ImageBuffer) -> String {
    // the specific implementation for `ModelOne`..
  }
}

Finally, you can make the variable modelchosen to be of type MLClassifier to erase the underlying concrete type of the model:

var modelchosen: MLClassifier = ModelOne(from: config1)

As MLClassifier is a common base class for both ModelOne and ModelTwo you can dynamically change the type of modelchosen whenever you need:

// Later...
modelchosen = ModelTwo(from: config2)

The variable modelchosen being of type MLClassifier ensures that you can call the .classify(image:) method whatever the concrete model type is:

func classifyphoto() {
    guard let image = imageused as UIImage?,
          let imagebuffer = image.convertToBuffer() else {
        return
        
    }
    
    let output = modelchosen.classify(image: imageBuffer)
    // Update the UI...
}

Using protocols

Protocols are the modern and preferred way of expressing common interfaces in Swift, they should be used over classes when possible:

protocol MLClassifier {
  init(from config: MLModelConfig)
  func classify(image: ImageBuffer) -> String
}

// Implement the protocol for your models
struct ModelOne: MLClassifier {
  init(from config: MLModelConfig) { ... }
  func classify(image: ImageBuffer) -> String { ... }
}

// Store an instance of any `MLClassfier` using an existential
var classifier: any MLClassifier = ModelOne(from: config1)

// Later...
classifier = ModelTwo(from: config2)

---

To sum up, the key is to identify the common capabilities of the different types you are trying to unify. For instance, if the two models output at some point a classLabelProbs of the same type, then you could use this as the common abstraction.

---

As a last resort, you could wrap everything in a big if-else statement, event though it is not recommended since it is not very readable, is not a good way to encapsulate common behavior and leads to a lot of code repetition:

func classifyphoto() {
    guard let image = imageused as UIImage?,
          let imagebuffer = image.convertToBuffer() else {
        return
        
    }

    if modelstring == "Model Two" {
        // Use modeltwo
        let output = try? modeltwo.prediction(image: imagebuffer)
 
        if let output = output {
        let results = output.classLabelProbs.sorted { $0.1 > $1.1 }
        _ = results.map { /*...*/ }
    } else {
        // Use modelone
        let output = try? modelone.prediction(image: imagebuffer)
 
        if let output = output {
        let results = output.classLabelProbs.sorted { $0.1 > $1.1 }
        _ = results.map { /*...*/ }
    }
}