azureazure-data-factoryblobazure-data-lake

Azure Blob Failed to create Container container

I am trying to write a file using ADF, below are the details.

Source - ADLS Gen 2 Linked Service of Source is also pointing to ADLS Gen 2

Sink - ADLS Gen 2 Linked Service of Sink is Azure Blob storage

Now I am trying write the file and I have all the permission but I am getting 403 error forbidden.

Error details

ErrorCode=AzureBlobFailedToCreateContainer,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Unable to create Azure Blob container. Endpoint: 'https://andcgateway.blob.core.windows.net/', Container Name: 'file-drop-secured'.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (403) Forbidden.,Source=Microsoft.WindowsAzure.Storage,StorageExtendedMessage=RequestId:8bey03e0-rvbmxww-004bruvlycxss
 Time:Mon, 03 Jun 2024 16:11:57 GMT,,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=System,'

But when I am trying to write by providing data contributer access it is working fine and linked service test connection also working fine.

Is there any issue with sink linked service since I am using Azure Blob storage linked service to write the file into ADLS Gen 2.

I can create one more linkedin service with adls gen 2, but just want to understand what is causing the issue here

Please help me to understand the issue.

Solution

  • Is there any issue with sink linked service since I am using Azure Blob storage linked service to write the file into ADLS Gen 2

    No, you can use Blob linked service for this. I have tried Azure Blob linked service with ADLS gen2 target and you can see it got copied to the target successfully without any issue.

    Linked service:

    enter image description here

    Copy activity sink:

    enter image description here

    Result:

    enter image description here

    For me, it had public access in networking of storage account. It's not like it won't work Blob linked service. It will work for both ADLS and Blob linked services as well.

    As per this Doc

    The reason for your error might be using managed VNet IR for the source or sink linked services.

    You can go through the same doc to know more about it.