Azure API Create Serverless Endpoint Not Working

I am trying to create a Serverless Endpoint as outlined in the documentation here.

I tried both the API version 2024-04-01 and 2024-04-01-preview without success. I can create this resource without any issues from the console (and I tested it with multiple models), however, the API throws an error that leaves me guessing.

I've tried countless different variations of the request body, and there are errors which describe missing mandatory properties or incorrect payload formatting. This specific error comes up only when everything is seemingly "right" and expected to work.

Below is the error message:

{
  "error": {
    "code": "UserError",
    "message": "Error when parsing request; unable to deserialize request body",
    "details": [],
    "additionalInfo": [
      {
        "type": "ComponentName",
        "info": {
          "value": "managementfrontend"
        }
      },
      {
        "type": "Correlation",
        "info": {
          "value": {
            "operation": "128dfO*****IDc15c",
            "request": "3c447*****IDb757"
          }
        }
      },
      {
        "type": "Environment",
        "info": {
          "value": "swedencentral"
        }
      },
      {
        "type": "Location",
        "info": {
          "value": "swedencentral"
        }
      },
      {
        "type": "Time",
        "info": {
          "value": "2024-06-19T17:04:26.5506093+00:00"
        }
      }
    ]
  }
}

I suspect there is something wrong with my request body, but not sure what it might be, since it is based on the API docs, and I tried all the property-value combinations I could think of.

This is the last one I've tried:

{
    "properties": {
        "authMode": "Key",
        "contentSafety": {
            "contentSafetyStatus": "Disabled"
        },
        "modelSettings": {
            "modelId": "azureml://registries/azureml/models/mistralai-Mistral-7B-Instruct-v01/versions/10"
        }
    },
    "location": "swedencentral",
    "kind": "ServerlessEndpoint",
    "identity": {
        "type": "None"
    }
}

NOTE: The identity field provided in the API example doesn't work, it says the only supported value is None

Any insights into this would be appreciated!

I've tried sending the request from different platforms: Node runtime, Postman, Microsoft Learn UI, and I've tried many different variations of the payload.

As per this documentation the you need to first subscribe your workspace to the model offering and need to deploy.

You can use the following python code to subscribe and deploy.

from azure.ai.ml import MLClient
from azure.identity import InteractiveBrowserCredential
from azure.ai.ml.entities import MarketplaceSubscription, ServerlessEndpoint

client = MLClient(
    credential=InteractiveBrowserCredential(),
    subscription_id="<subscription_id>",
    resource_group_name="<resource_group_name>",
    workspace_name="<workspace_name>",
)



model_id="azureml://registries/azureml/models/mistralai-Mistral-7B-Instruct-v01"
subscription_name="mistralai"

marketplace_subscription = MarketplaceSubscription(
    model_id=model_id,
    name=subscription_name,
)

marketplace_subscription = client.marketplace_subscriptions.begin_create_or_update(
    marketplace_subscription
).result()

Deploy it

endpoint_name="tst-point"

serverless_endpoint = ServerlessEndpoint(
    name=endpoint_name,
    model_id=model_id
)

created_endpoint = client.serverless_endpoints.begin_create_or_update(
    serverless_endpoint
).result()

Even, after using this still you get error, because the model you are trying is not available for serverless endpoint.

ActivityCompleted: Activity=MarketplaceSubscription.BeginCreateOrUpdate, HowEnded=Failure, Duration=1892.28 [ms], Exception=HttpResponseError, ErrorCategory=UserError, ErrorMessage=(UserError) The requested model azureml://registries/azureml/models/mistralai-Mistral-7B-Instruct-v01 is not available.
Code: UserError
Message: The requested model azureml://registries/azureml/models/mistralai-Mistral-7B-Instruct-v01 is not available.
Additional Information:Type: ComponentName
Info: {
    "value": "managementfrontend"
}Type: Correlation
Info: {
    "value": {
        "operation": "2ad6056e4068088baaddef766bdf6848",
        "request": "a434bd0717b322eb"
    }
}Type: Environment
Info: {
    "value": "eastus2"
}Type: Location
Info: {
    "value": "eastus2"
}Type: Time
Info: {
    "value": "2024-06-21T09:37:02.3364133+00:00"
}Type: InnerError
Info: {
    "value": {

Check out the supported models here

Also, when you go to your workspace serverless endpoint and click on create.

Next, you will get list of available models.

Here, you can see only 2 models available under mistralai.