I'm trying to download binary files from my Azure storage account. Initially, I was using CloudBlob.DownloadToFileAsync() which allowed me to supply a IProgress parameter and get progress updates of the transfer.
However, on bigger > 2gb files, DownloadToFileAsync was hanging. According to the documentation I needed to be using DownloadToFileParallelAsync to download larger files. I have implemented this, and confirm it now works, but now I'm unable to get the progress of the download as it does not offer a IProgress parameter.
Can anyone point me to how I can gather any useful progress data, or offer a workaround?
int parallelIOCount = SystemInfo.processorCount;
long rangeSizeInBytes = 16 * Constants.MB;
await cloudModuleBlob.DownloadToFileParallelAsync(targetTempModuleFile, FileMode.Create, parallelIOCount, rangeSizeInBytes, cancellationTokenSource.Token);
progressSlider.value = 1.0f;
//When the download is finished...
//Rename the temp file to the full version.
if (File.Exists(targetCiqModuleFile))
{
File.Delete(targetCiqModuleFile);
}
File.Move(targetTempModuleFile, targetCiqModuleFile);
Debug.Log("Download saved to: " + targetCiqModuleFile);
Solved it with a workarounnd. Rather than using DownloadToFileAsync I instead used DownloadRangeToStreamAsync to break the blob into smaller pieces and combined them on the client end. Works effectively with 16mb chunks.
//Create the file.
using (FileStream fileStream = File.Create(targetTempModuleFile))
{
long chunkSize = 16 * Constants.MB;
Int64 current = 0;
while (current < cloudModuleBlob.Properties.Length)
{
if ((current + chunkSize) > cloudModuleBlob.Properties.Length)
{
await cloudModuleBlob.DownloadRangeToStreamAsync(fileStream, current, (cloudModuleBlob.Properties.Length - current), default, default, default, progressHandler, cancellationToken);
}
else
{
await cloudModuleBlob.DownloadRangeToStreamAsync(fileStream, current, chunkSize, default, default, default, progressHandler, cancellationToken);
}
current = current + chunkSize;
}
}