Packer -ARM Builder Keyvault access denied to first party service

I have used the build_key_vault_name by brining my own keyvault which is having network restriction and allowed only to my private network within vnet however packer is somewhat using the Azure Compute to fetch the secret using a Public IP belongs to MS.

My Packer log

azure-arm: output will be in this color.
11:52:45  
11:52:45  ==> azure-arm: Running builder ...
11:52:45  ==> azure-arm: Getting tokens using client secret
11:52:45  ==> azure-arm: Getting tokens using client secret
11:52:45      azure-arm: Creating Azure Resource Manager (ARM) client ...
11:52:45  ==> azure-arm: Using existing resource group ...
11:52:45  ==> azure-arm:  -> ResourceGroupName : 'images-storage-rg-01'
11:52:45  ==> azure-arm:  -> Location          : 'eastus2'
11:52:45  ==> azure-arm: Setting the certificate in the KeyVault...
11:52:45  ==> azure-arm: Getting the certificate's URL ...
11:52:45  ==> azure-arm:  -> Key Vault Name        : 'packer-kv-01'
11:52:45  ==> azure-arm:  -> Key Vault Secret Name : 'packerKeyVaultSecret'
11:52:45  ==> azure-arm:  -> Certificate URL       : 'https://packer-kv-01.vault.azure.net/secrets/packerKeyVaultSecret/0899826f9b724a84af004756f9545236'
11:52:45  ==> azure-arm: Setting the certificate's URL ...
11:52:45  ==> azure-arm: Validating deployment template ...
11:52:45  ==> azure-arm:  -> ResourceGroupName : 'images-storage-rg-01'
11:52:45  ==> azure-arm:  -> DeploymentName    : 'pkrdpz90dt1tlc0'
11:52:45  ==> azure-arm: Deploying deployment template ...
11:52:45  ==> azure-arm:  -> ResourceGroupName : 'images-storage-rg-01'
11:52:45  ==> azure-arm:  -> DeploymentName    : 'pkrdpz90dt1tlc0'
11:52:45  ==> azure-arm: ERROR: -> DeploymentFailed : At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.
11:52:45  ==> azure-arm: ERROR:   -> Conflict
11:52:45  ==> azure-arm: ERROR:   -> ResourceDeploymentFailure : The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.
11:52:45  ==> azure-arm: ERROR:     -> KeyVaultAccessForbidden : Key Vault https://packer-kv-01.vault.azure.net/secrets/packerKeyVaultSecret/0899826f9b724a84af004756f9545236 either has not been enabled for deployment or the vault id provided, /subscriptions/****/resourceGroups/images-storage-rg-01/providers/Microsoft.KeyVault/vaults/packer-kv-01, does not match the Key Vault's true resource id.
11:52:45  ==> azure-arm:
11:52:45  ==> azure-arm: Code="DeploymentFailed" Message="At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details." Details=[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"KeyVaultAccessForbidden\",\r\n        \"message\": \"Key Vault https://packer-kv-01.vault.azure.net/secrets/packerKeyVaultSecret/0899826f9b724a84af004756f9545236 either has not been enabled for deployment or the vault id provided, /subscriptions/****/resourceGroups/images-storage-rg-01/providers/Microsoft.KeyVault/vaults/packer-kv-01, does not match the Key Vault's true resource id.\"\r\n      }\r\n    ]\r\n  }\r\n}"}]
11:52:45  ==> azure-arm: 
11:52:45  ==> azure-arm: The resource group was not created by Packer, deleting individual resources ...
11:52:45  ==> azure-arm: Removing the created Deployment object: 'pkrdpz90dt1tlc0'
11:52:45  ==> azure-arm: 
11:52:45  ==> azure-arm: The resource group was not created by Packer, not deleting ...
11:52:45  Build 'azure-arm' errored after 1 minute 17 seconds: unexpected EOF

My Keyvault audit logs

Access denied to first party service.
Caller: name=Compute;tid=f8cdef31-a31e-4b4a-93e4-5f571e91255a;appid=579d9c9d-4c83-4efc-8124-7eba65ed3356;oid=9486e527-3706-4e7b-8295-613aac964938;iss=https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/
Vault:packer-kv-01;location=eastus2

Public IP: 52.136.29.5 Operation: SecretGet

Question:

• What is this Object ID belonging to (ms tenant) 9486e527-3706-4e7b-8295-613aac964938 ?
• Why is Packer using "MS managed SPN" to fetch the secret and not the SPN that I gave?
• Why does the key vault get secret operation initiated from MS public IP and not the private IP?

Found the issue, The problem is we need to enable the below two option on the new keyvault

• Azure Virtual Machines for deployment

• Azure Resource Manager for template deployment

This will give the right access for packer(arm) to write the secret into the keyvault