Following Meta Llama 2 7B product in Azure marketplace, I deployed it in azure ai service. AFter the deployment I got my endpoint as something like
https://mydemo-llama-serverless.eastus2.inference.ai.azure.com
I already enabled the private endpoint for my Azure AI but those endpoints can give the private endpoints for the domain privatelink.api.azureml.ms
and privatelink.notebooks.azure.net
however my Pay-as-you-go deployment having the domain as inference.ai.azure.com
which is still public.
Question: How can I make my inference endpoint for my Pay-as-you-go model deployment as private?
If the answer is not yet supported can someone guide me on any workaround or when we can expect this feature?
Finally, Microsoft has released this feature around July, read more here
I tried the solution and noticed one additional limitation will be
If an AI hub is disabled public network, It cannot able to reach the marketplace to subscribe the models. Please see my below error
To be exact the API which is failing is
https://ai.azure.com/api/<region>/modelEndpointDiscovery/v1.0/subscriptions/<>/resourceGroups/<>/providers/Microsoft.MachineLearningServices/workspaces/demoproject/marketplaceSubscriptions?api-version=2023-10-01-preview
with 403 Forbidden
To bypass this error, I have to make the Hub publicly accessible then subscribe to the model from the marketplace, and then deploy after all this is done. I have to revert my hub to disable public access after subscribing from the marketplace