Airflow Connection: How to create an airflow connection to Azure Blob Storage using Service Principal?
I followed this doc to attempt to create an Azure Blob Storage connection. I believe for my case, I am looking to connect via token credentials.
However I get an error when trying to save with no error message. I'm a bit confused on whether I am providing client_id, tenant_id, client_secret, sub_id, etc. in the appropriate location as the UI doesn't specify those exact words except for tenant id. In the blob storage key (optional) input I have provided the client secret, and blob storage login (optional) holds the account name. For the extra section, do the values need to follow "extra__azure__..."?
Any suggestion are highly appreciated!
I am able to connect to the Blob Storage via azure cli. Airflow version is 2.4.3 running on Kubernetes as a Docker container.
To connect airflow with service principal to azure blob storage.
- First you need to create Service principal for appropriate azure application and get following details:
-- Application Client ID
-- Tenant ID
-- Client secret
- Assign Storage Blob Data Contributor role to your service principal in azure blob storage so that Airflow can access blob objects.
- Fill below feilds and test the connection
-- Connection Id : Enter a name for the connection.
-- Account Name : Enter Blob Service URL for your storage account. eg. https://mystorageaccount.blob.core.windows.net/
-- Blob Storage Login : Enter your Application (client) ID.
-- Blob Storage Key : Enter your client secret Value.
-- Tenant Id : Enter your Directory (tenant) ID.
REFERENCE : Create an Azure Blob Storage connection in Airflow
- How do I persist logs from a Dockerized .NET Core app in Azure DevOps and save them as artifacts?
- Azure won't show roleDefinition for directory roles
- How do we change the TLS version of Azure IoT Hub which is already created and deployed?
- Microsoft Graph API ADB2C signInActivity update lag
- How does Azure App Service manage port binding with docker container? Does it recognize Expose?
- Has there been a recent change to Invoke-Sqlcmd that break it, and if so what's the fix?
- Is using AddAzureClients and a clientFactory for Azure ServiceBus DI preferred over a singleton implementation?
- Redeploy .NET Aspire project after resource cleanup
- SocketException permission denied when trying to bind port 80 or 443 to Kestrel in Azure Container Instance with .NET 8
- Azure Functions with docker: How change port?
- Azure - How to update the password profile of a user in Azure AD B2C using the Microsoft Graph API?
- Unable to filter the service principals by appOwnerOrganizationId property
- How can I upload an image to Azure from an Azure Function with a BlobTrigger
- How can I see which mechanism granted the user the ability to activate roles via Azure Privileged Identity Management (PIM)
- How to resolve an "Invalid operands found for operator -eq" error in Microsoft Entra ID PowerShell code
- You don’t appear to have an active Azure subscription when creating new Kubernetes service connection in Azure DevOps
- Azure Application Insights JavaScript SDK loader script security
- How to fix 'Unable to Obtain Authentication Token' in Active Directory Authentication to Azure Analysis Server
- Python script to join 2 csv files with an existing excel file in Azure blob storage
- Can one use DPO (direct preference optimization) of GPT via CLI or Python on Azure?
- azure function via terraform: how to connect to service bus
- Azure WebJob is failing after some period of time on Linux plan
- Retry policy on blob leasing
- Azure auth for Outlook Graph API
- Copying storage data from one Azure account to another
- Windows Azure SDK for Node.js SQL Azure Provider
- Pipeline for stored procedure dedicated sql pool
- Webpack @azure/storage-blob node-fetch AbortSignal issue
- What is the Azure equivalent of GCP's Cloud Run?
- How stop execution of a stored procedure if the client closes the connection in SQL Azure database?