azureazure-storageazure-blob-storageazure-worker-roles

How to upload a folder to Blob Storage?


I have a worker role that I use to pull data down from Blob Storage OnStart. Currently I'm testing this by uploading a test.txt file and then downloading it to a local directory. This is working fine.

I now would like to upload a folder to blob storage. This folder contains a batch script as well as several executables that the batch script calls.

What is the recommended way to accomplish this? I think zipping the folder and uploading the *.zip file would be easy... but then, once I download it locally for the worker role to handle, how would I unzip it without any third party libraries?

If there are better options, I'm open to any suggestions. Thanks for the help here - this community has been a huge help for me as I ramp up :)


Solution

  • You can take two approaches: Single-file (e.g. zip) or multi-file (with each file in its own blob). Here's my take on it, then a note about unzipping:

    Single zip file

    This is a very easy way to maintain a grouped set of files, like an apache install, or a set of static resources. Downloading to local storage from a blob is extremely simple. And, a zip file can handle any level of nested directories.

    Downside: To update a single file, you'd need to create a new zip; no way to simply upload one modified asset.

    individual blobs

    Separate blobs are great when you need to update individual files quickly without worrying about other files. Also, you can direct-link to these blobs whether public or (with Shared Access Signature) private and enbed links in web pages, etc. Look at my answer here, as well as @Sandrino's, for examples of this. Oh, and if you're planning on exposing blobs via CDN, they'll need to be in individual blobs.

    Downside: No absolute mapping to nested directories. Blob storage is arranged by account\container\blob. While you can simulate nested folders, you'd need to do some work to map individual files. To download individual blobs, you'd need to grab the container and call ListBlobs() to enumerate through individual blobs names.

    How to unzip

    The Eclipse project provides a vbs script which is trivial to use. From a Visual Studio project (or really any script), I'd consider downloading something like 7zip, which is free and trivial to install. Then just download the zip from blob storage to local storage (in the proper folder), and pass it to 7zip.

    I hope this provides you with enough guidance to make the correct decision. If it were me and I was storing a build (like tomcat), I'd keep the entire directory structure in a zip. That gives me assurance that I haven't broken something by modifying just a single file. And... I can keep a running history of tomcat versions easily, with multiple zips (in separate blobs).