azuredatabricksazure-keyvaultazure-cliscopes

Unable to add key to Azure key-vault backed Databricks scope

I am trying to create a keyvault backed scope in databricks. I am able to successfully create the scope but when I try to add a key to the scope I see the following error:

Error: b'{"error_code":"BAD_REQUEST","message":"Cannot write secrets to Azure KeyVault-backed scope abc"}'

These are steps I have followed and all commands were run on windows cmd:

  1. Create key vault in Azure
  2. Generate AAD token for databricks - az account get-access-token --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d
  3. Add AAD token to environment variables on windows
  4. Add AAD token to databricks cfg file on windows - databricks configure --aad-token
  5. Create scope - databricks secrets create-scope --scope abc --scope-backend-type AZURE_KEYVAULT --resource-id <keyvault-id> --dns-name <keyvault-dns> --initial-manage-principal users
  6. Add key to scope - databricks secrets put --scope abc --key abc-key << this where I see the error
Solution

  • According to the documentation this is not possible:

    To reference secrets stored in an Azure Key Vault, you can create a secret scope backed by Azure Key Vault. You can then leverage all of the secrets in the corresponding Key Vault instance from that secret scope. Because the Azure Key Vault-backed secret scope is a read-only interface to the Key Vault, the PutSecret and DeleteSecret Secrets API 2.0 operations are not allowed. To manage secrets in Azure Key Vault, you must use the Azure SetSecret REST API or Azure portal UI.

    Using Az CLI, you could use the az keyvault secret set command.