[SOLVED] Error creating interpreter for tflite model on Android phone GPU but fine on CPU - do GPU and CPU support same set of tflite model operators?

Error creating interpreter for tflite model on Android phone GPU but fine on CPU - do GPU and CPU support same set of tflite model operators?

I am trying to run a tflite model in flutter (using the tflite_flutter package) on an Android phone and it successfully runs on the phone CPU. But when I use the phone GPU I get the error below that it cannot create the model interpreter.

I/tflite  ( 8113): Created TensorFlow Lite delegate for NNAPI.
I/flutter ( 8113): Unable to create interpreter, Caught Exception: Invalid argument(s): Unable to create interpreter.

followed by messages below:

E/flutter ( 8113): #0      Classifier.interpreter (package:ace_example/tflite/classifier.dart)
E/flutter ( 8113): #1      _MyCameraPageState.onLatestImageAvailable (package:ace_example/camera/view.dart:193:34)
E/flutter ( 8113): #2      CameraController.startImageStream.<anonymous closure> (package:camera/src/camera_controller.dart:442:20)
E/flutter ( 8113): #3      _rootRunUnary (dart:async/zone.dart:1434:47)
E/flutter ( 8113): #4      _CustomZone.runUnary (dart:async/zone.dart:1335:19)
E/flutter ( 8113): #5      _CustomZone.runUnaryGuarded (dart:async/zone.dart:1244:7)
E/flutter ( 8113): #6      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341:11)
E/flutter ( 8113): #7      _DelayedData.perform (dart:async/stream_impl.dart:591:14)
E/flutter ( 8113): #8      _StreamImplEvents.handleNext (dart:async/stream_impl.dart:706:11)
E/flutter ( 8113): #9      _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:663:7)
E/flutter ( 8113): #10     _rootRun (dart:async/zone.dart:1418:47)
E/flutter ( 8113): #11     _CustomZone.run (dart:async/zone.dart:1328:19)
E/flutter ( 8113): #12     _CustomZone.runGuarded (dart:async/zone.dart:1236:7)
E/flutter ( 8113): #13     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1276:23)
E/flutter ( 8113): #14     _rootRun (dart:async/zone.dart:1426:13)
E/flutter ( 8113): #15     _CustomZone.run (dart:async/zone.dart:1328:19)
E/flutter ( 8113): #16     _CustomZone.runGuarded (dart:async/zone.dart:1236:7)
E/flutter ( 8113): #17     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1276:23)
E/flutter ( 8113): #18     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
E/flutter ( 8113): #19     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
E/flutter ( 8113): 
I/ech.ace_exampl( 8113): Background concurrent copying GC freed 578(97KB) AllocSpace objects, 21(2520KB) LOS objects, 49% free, 2453KB/4907KB, paused 5.309ms total 31.741ms

I tried to use GPUDelegate class to initialize the interpreter as shown below but this also gave the same error.

if (Platform.isAndroid) {
      final gpuDelegateV2 = GpuDelegateV2(
          options: GpuDelegateOptionsV2(
            isPrecisionLossAllowed: false,
            // TfLiteGpuInferenceUsage.fastSingleAnswer,
            inferencePriority1: TfLiteGpuInferencePriority.minLatency,
            // TfLiteGpuInferencePriority.auto,
            // TfLiteGpuInferencePriority.auto,
          ));
      _interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegateV2);
      
      // InterpreterOptions()..useNnApiForAndroid = true;
      // _interpreterOptions = InterpreterOptions()..threads = 8;
    }

I'm thinking, could it be that there is some operator in my tflite model that is not supported by the GPU? Does anyone know if the GPU supports a smaller set of operators in tflite models than the CPU? Thanks.

Solution