os system: macos Catalina 10.15.2
xcode: 11.3
coreml3.0
I give same model input to the same mlmodel. But the inference results are different using cpu device and gpu device.
The results are as follows, the left file is inference result (second column) using cpu and the right file is inference result (second column) using CpuAndGpu. I use beyond compare to compare the two files, and the data marked with red color are the difference.
Does anyone know about the problem and how to fix it?
This is not a problem per se. On the GPU, 16-bit floats are used while on the CPU 32-bit floats are used. 16-bit floats have less precision, which explains the different results you're getting.
Some numbers will be slightly larger, some will be slightly smaller, but generally these effects cancel out and you won't notice the difference.
(However, if your model generates images, you may notice pixel artifacts from the lower precision provided by the 16-bit floats.)
