pythonmachine-learningkerasgoogle-cloud-functionspredict

Deploying Keras model for prediction in Google Cloud Functions


I've been trying to deploy a very simple toy Keras model to Cloud Functions, which would predict the class of an image, but for reasons unknown, when the execution gets to the predict method, it gets stuck, does not throw any error, and eventually times out.

import functions_framework
import io
import numpy as np
import tensorflow as tf

from tensorflow.keras.models import load_model
from PIL import Image

model = load_model("gs://<my-bucket>/cifar10_model.keras")

class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']

def preprocess_image(image_file):
    img = Image.open(io.BytesIO(image_file.read()))
    img = img.resize((32, 32))
    img = np.array(img)
    img = img / 255.0
    img = img.reshape(1, 32, 32, 3)
    return img

@functions_framework.http
def predict(request):
    image = preprocess_image(request.files['image_file'])
    print(image.shape) # this prints OK
    prediction = model.predict(image)
    print(prediction) # this never prints
    predicted_class = class_names[np.argmax(prediction)]
    return f"Predicted class: {predicted_class}"

Debugging locally works fine, the prediction is fast as expected (the model weights file is 2MB). I also added several prints along the way (removed from the snippet above) and the execution works fine until the predict method.

Even though the minimal compute configuration should work, I tried reserving more memory and CPU, but nothing worked. The model is hosted at Storage, I tried downloading it first, but that didn't work either. I did also try making the prediction inside a tf.device('/cpu:0') context, passing a step=1 parameter and converting the image array to a Keras Dataset first, as suggested by ChatGPT, with the same results. Actually, nothing prints as a result of invoking predict at all. Calling call instead of predict got me nowhere.

What am I missing?


Solution

  • Even though the previous suggestion worked, I was invested into writing as little code as possible. Therefore, I came up with this solution:

    import functions_framework
    import io
    import numpy as np
    import tensorflow as tf
    
    from tensorflow.keras.models import load_model
    from PIL import Image
    
    class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
    
    def preprocess_image(image_file):
        img = Image.open(io.BytesIO(image_file.read()))
        img = img.resize((32, 32))
        img = np.array(img) / 255.0
        img = img.reshape(1, 32, 32, 3)
        return img
    
    @functions_framework.http
    def predict(request):
        model = load_model("gs://<my-bucket>/cifar10_model.keras")
        image = preprocess_image(request.files['image_file'])
        prediction = model.predict(image)
        predicted_class = class_names[np.argmax(prediction)]
        return f"Predicted class: {predicted_class}"
    

    It seems that the issue is that the model must be instantiated inside the function that processes the request. I would have never guessed!