I've deployed a tensorflow model in vertex AI platform using TFX Pipelines. The model have custom serving signatures but I'm strugling to specify the signature when I'm predicting.
I've the exact same model deployed in GCP AI Platform and I'm able to specify it.
According to the vertex documentation, we must pass a dictionary containing the Instances (List) and the Parameters (Dict) values.
I've submitted these arguments to this function:
instances: [{"argument_n": "value"}]
parameters: {"signature_name": "name_of_signature"}
Doesn't work, it still get the default signature of the model.
In GCP AI Platform, I've been able to predict directly specifying in the body of the request the signature name:
response = service.projects().predict(
name=name,
body={"instances": instances,
"signature_name": "name_of_signature"},
).execute()
@EDIT I've discovered that with the rawPredict method from gcloud it works.
Here is an example:
!gcloud ai endpoints raw-predict {endpoint} --region=us-central1 \
--request='{"signature_name":"name_of_the_signature", \
"instances": [{"instance_0": ["value_0"], "instance_1": ["value_1"]}]}'
Unfortunately, looking at google api models code it only have the predict method, not the raw_predict. So I don't know if it's available through python sdk right now.
Vertex AI is a newer platform with limitations that will be improved over time. “signature_name” can be added to HTTP JSON Payload in RawPredictRequest or from gcloud as you have done but right now this is not available in regular predict requests.
Using HTTP JSON payload :
Example:
input.json :
{
"instances": [
["male", 29.8811345124283, 26.0, 1, "S", "New York, NY", 0, 0],
["female", 48.0, 39.6, 1, "C", "London / Paris", 0, 1]],
"signature_name": <string>
}
curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/endpoints/${ENDPOINT_ID}:rawPredict \
-d "@input.json"