tensorflowtensorflow.jstransfer-learningtensorflowjs-converter

My converted tensorflow Transfer Learning model always returns same results in Tensorflow JS


I have created a model which applied Mobilenet V2 for the convolutional base layers in Google colab. Then I converted it by using this command:

path_to_h5 = working_dir + '/Tensorflow_PY_Model/SavedModel.h5'
path_tfjs = working_dir + '/TensorflowJS'

!tensorflowjs_converter --input_format keras \
                        {path_to_h5} \
                       {path_tfjs}

I used an image to test classify it on both. In python, I use this code below to do the prediction:

from google.colab import files
from io import BytesIO
from PIL import Image
import matplotlib.pyplot as plt

uploaded = files.upload()
last_uploaded = list(uploaded.keys())[-1]
im = Image.open(BytesIO(uploaded[last_uploaded]))

im = im.resize(size=(224,224))
img = np.array(im)
img = img / 255.0
prediction1 = model.predict(img[None,:,:])
print(prediction1)

That code above returns this array:

[6.1504150e-05 4.8508531e-11 5.1813848e-15 2.1887154e-12 9.9993849e-01
  8.4171114e-13 1.4638757e-08 3.4268971e-14 7.5719299e-15 1.0649443e-16]]

After that I try to predict in Javascript with this code below:

async function predict(image) {
    var model = await tf.loadLayersModel('./TFJs/model.json');
    let predictions = model.predict(preprocessImage(image)).dataSync(); 
    console.log(predictions);

    return results;
}

function preprocessImage(image) {
    let tensor = tf.browser.fromPixels(image);
    const resizedImage = tensor.resizeNearestNeighbor([224,224]);
    const batchedImage = resizedImage.expandDims(0);
    return batchedImage.toFloat().div(tf.scalar(255)).sub(tf.scalar(1));
}

document.querySelector('input[type="file"]').addEventListener("change", async function () {
        if (this.files && this.files[0]) {
            img = document.getElementById("uploaded-img");
            img.onload = () => {
                URL.revokeObjectURL(img.src); // no longer needed, free memory
            };
            img.src = URL.createObjectURL(this.files[0]);
            predictionResult = await predict(model, img);
            displayResult(predictionResult);
        }
    });

However, with the same image as that I used when predicting on Python, it returns this result and it never change no matter I change the image.

Float32Array(10) [0.9489052295684814, 0.0036257198080420494, 0.000009185552698909305,
0.000029705168344662525, 0.04141413792967796, 1.4301890782775217e-9, 0.006003820803016424,
2.8357267645162665e-9, 0.000011812648153863847, 4.0659190858605143e-7]

So how to fix this problem? What more should I do? Thanks in advance for the answers and suggestions!


Solution

  • After I debug some possible causes, I realized that the problem is in this block code:

    document.querySelector('input[type="file"]').addEventListener("change", async function () {
            if (this.files && this.files[0]) {
                img = document.getElementById("uploaded-img");
                img.onload = () => {
                    URL.revokeObjectURL(img.src); // no longer needed, free memory
                };
                img.src = URL.createObjectURL(this.files[0]);
                predictionResult = await predict(model, img);
                displayResult(predictionResult);
            }
        });
    

    Firstly, I wanted to make it automated so it will just instantly display the picked image and predict in a pipeline. But it can't be done, because the src attribute of img would still be the same value as before the whole block executed.

    In my case, it executed the whole block until the prediction and result then the uploaded and wrong predicted ones appears altogether. So I finally made a change like adding another button only for predicting and take out the prediction lines from that block and putting them in another function. It works well at the end.

    document.querySelector('input[type="file"]').addEventListener("change", async function () {
            if (this.files && this.files[0]) {
                img = document.getElementById("uploaded-img");
                img.onload = () => {
                    URL.revokeObjectURL(img.src); // no longer needed, free memory
                };
                img.src = URL.createObjectURL(this.files[0]);
            }
        });
    
    document.querySelector('#predict-btn').addEventListener("click", async function ()  {
        img = document.getElementById("uploaded-img");
        predictionResult = await predict(model, img);
        displayResult(predictionResult);
    });
    

    Well, I am still curious if I can get these functions into a pipeline process so there would be only one upload button and the rest of works done by system.