I've been trying to deploy a very simple toy Keras model to Cloud Functions, which would predict the class of an image, but for reasons unknown, when the execution gets to the predict
method, it gets stuck, does not throw any error, and eventually times out.
import functions_framework
import io
import numpy as np
import tensorflow as tf
from tensorflow.keras.models import load_model
from PIL import Image
model = load_model("gs://<my-bucket>/cifar10_model.keras")
class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
def preprocess_image(image_file):
img = Image.open(io.BytesIO(image_file.read()))
img = img.resize((32, 32))
img = np.array(img)
img = img / 255.0
img = img.reshape(1, 32, 32, 3)
return img
@functions_framework.http
def predict(request):
image = preprocess_image(request.files['image_file'])
print(image.shape) # this prints OK
prediction = model.predict(image)
print(prediction) # this never prints
predicted_class = class_names[np.argmax(prediction)]
return f"Predicted class: {predicted_class}"
Debugging locally works fine, the prediction is fast as expected (the model weights file is 2MB). I also added several prints along the way (removed from the snippet above) and the execution works fine until the predict
method.
Even though the minimal compute configuration should work, I tried reserving more memory and CPU, but nothing worked. The model is hosted at Storage, I tried downloading it first, but that didn't work either. I did also try making the prediction inside a tf.device('/cpu:0')
context, passing a step=1
parameter and converting the image array to a Keras Dataset first, as suggested by ChatGPT, with the same results. Actually, nothing prints as a result of invoking predict
at all. Calling call
instead of predict
got me nowhere.
What am I missing?
Even though the previous suggestion worked, I was invested into writing as little code as possible. Therefore, I came up with this solution:
import functions_framework
import io
import numpy as np
import tensorflow as tf
from tensorflow.keras.models import load_model
from PIL import Image
class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
def preprocess_image(image_file):
img = Image.open(io.BytesIO(image_file.read()))
img = img.resize((32, 32))
img = np.array(img) / 255.0
img = img.reshape(1, 32, 32, 3)
return img
@functions_framework.http
def predict(request):
model = load_model("gs://<my-bucket>/cifar10_model.keras")
image = preprocess_image(request.files['image_file'])
prediction = model.predict(image)
predicted_class = class_names[np.argmax(prediction)]
return f"Predicted class: {predicted_class}"
It seems that the issue is that the model must be instantiated inside the function that processes the request. I would have never guessed!