[SOLVED] Does CoreML inference time grow with iterations?

Does CoreML inference time grow with iterations?

I have to measure the inference time of my ML Model in Swift. To do so, I wanted to perform the inference several times and then average all inference times to have a more precise value. I noted that the inference time grew with the iterations. I must have done something wrong, I'm not familiar with Swift:

Duration :  4756 ms
Duration :  4879 ms
Duration :  5325 ms
Duration :  5712 ms
Duration :  5952 ms
Duration :  6059 ms
Duration :  6223 ms
Duration :  6244 ms
Duration :  6088 ms
Duration :  6286 ms

Here is a snippet of my code:

for _ in 1...nb_it {

    // Timer
    let tic = CFAbsoluteTimeGetCurrent()

    // Inference    
    _ = try model.prediction(input: input, options: options)

    // Timer
    let toc = CFAbsoluteTimeGetCurrent()
    let duration = Int32(1000 * (toc - tic))

    // Report
    print(String(format: "Duration : %5d ms", duration))
}

Is it normal? Any idea if it's not? Thanks in advance!

Solution

Yes, this is possible if the phone gets hot. Your model seems to be pretty large (4 seconds per prediction) so it is doing a lot of work, heating up the CPU and/or GPU. The phone will slow down to avoid overheating.