androidtensorflow-litemediapipeimage-classification

Use Mobilenet tflite model with MediaPipe


I am doing MediaPipe Image Classification with the code below. It works when I use the standard model efficientnet_lite0.tflite:

            val baseOptions2 = BaseOptions.builder()
                .setDelegate(Delegate.CPU)
            //    .setModelAssetPath("assets\\efficientnet_lite0.tflite")
                .setModelAssetPath("assets\\mobilenet_v1_1.0_224.tflite")
                .build()
            val options2 = ImageClassifierOptions.builder()
                .setBaseOptions(baseOptions2)
                .setScoreThreshold(0.5f)
                .setRunningMode(RunningMode.IMAGE)
                .setMaxResults(5)
                .build()
            var imageClassifier = ImageClassifier.createFromOptions(mContext, options2)

                   // Run Classifier
                   val classifierResult: ImageClassifierResult? =
                   imageClassifier.classify(mpImage2)
                   if (classifierResult!!.classificationResult().classifications().size > 0) {

I want to use dogs.tflite, a model with dog breeds that I have made based on mobilenet_v1_1.0_224.tflite. But when I try to use it, I get these error messages:

E/native  (25421): E20240116 21:24:13.382606 25421 graph.cc:472] Input tensor has type float32: it requires specifying NormalizationOptions metadata to preprocess input images.
D/AndroidRuntime(25421): Shutting down VM
E/AndroidRuntime(25421): FATAL EXCEPTION: main

What, if anything, can I do? I have read the Image classification model customization guide but it means many many hours of work - work that I have already done when I made the dogs.tflite model. Can I add "NormalizationOptions metadata", whatever that is, to dogs.tflite? How?

Edit: It also works with mobilenet_v1_1.0_224-metadata.tflite from Kaggle. Can I somehow get the metadata from that model and put into my dogs.tflite? And/or use the dog_labels.txt file that goes with dogs.tflite?


Solution

  • After spending many hours on this and learning a lot, here is what I have found:

    1. I first used the Image classification model customization guide to build a new model, using the same images as for my dogs.tflite. It worked, but the model created was no good. Accuracy was very poor. The new model had NormalizationOptions metadata all right, but the data was incorrect! See this issue on GitHub. So many hours wasted!

    2. I finally found this Metadata Writer Tutorial. I ran it in Google Colab and could very easily add correct metadata to my dogs.tflite. It now works perfect!