Is there an easy way, to synchronise an Azure file share to a blob store (ADLS gen2)?
I thought about using AzCopy running on a VM, but I'm wondering whether there is an easier solution.
Synchronise Azure files to blob store.
AzCopy is better tool to synchronize the data between Azure file share and Azure Blob storage.
You can use AzCopy to copy data between file shares and blob storage, and it supports both one-time and scheduled sync operations.
To use AzCopy for syncing data between Azure file shares and blob storage, you can execute the below command:
Command and Output:
azcopy sync "https://venkatxxxx.file.core.windows.net/share1/sample/?sv=2022-11-02&ss=bfqt&srt=sco&sp=rwdlacupyx&se=2024-06-27T14:41:41Z&st=2024-06-27T06:41:41Z&spr=https&sig=ozzzzzzzD" "https://venkatxxxx.blob.core.windows.net/test/data/?sv=2022-11-02&ss=bfqt&srt=sco&sp=rwdlacupyx&se=2024-06-27T14:41:41Z&st=2024-06-27T06:41:41Z&spr=https&sig=oVE3Szzzzzz3D" --recursive=true
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
Job 8c7e4zzzzf71d1c31ef has started
Log file is located at: C:\Users\zzz\.azcopy\8zzzzf-63f2-12f71d1c31ef.log
INFO: azcopy.exe 10.22.2: A newer version 10.25.1 is available to download
100.0 %, 5 Done, 0 Failed, 0 Pending, 5 Total, 2-sec Throughput (Mb/s): 0
Job 8c7zzzzzz1d1c31ef Summary
Files Scanned at Source: 5
Files Scanned at Destination: 0
Elapsed Time (Minutes): 0.1339
Number of Copy Transfers for Files: 5
Number of Copy Transfers for Folder Properties: 0
Total Number Of Copy Transfers: 5
Number of Copy Transfers Completed: 5
Number of Copy Transfers Failed: 0
Number of Deletions at Destination: 0
Total Number of Bytes Transferred: 272650
Total Number of Bytes Enumerated: 272650
Final Job Status: Completed
You can execute above command on a VM or any machine that has access to both the Azure file share and blob storage. To automate the sync operation, you can schedule the command to run at regular intervals using a tool like Azure Functions or Azure Logic Apps.
I thought about using AzCopy running on a VM, but I'm wondering whether there is an easier solution.
The other option is Azure data factory that creates a pipeline that transfers (Copy activity) data from an Azure file share to a blob container, and then schedule the pipeline to execute at regular intervals.
Reference:
Copy activity - Azure Data Factory & Azure Synapse | Microsoft Learn