[SOLVED] Storage for Azure VMs running Service Fabric cluster with Reliable Collections

Storage for Azure VMs running Service Fabric cluster with Reliable Collections

I am about to start delving into stateful services and Reliable Collections for Azure Service Fabric.

My node VMs are pretty much standard machines, and I notice that the D: drive is classed as temporary storage.

How does this work with stateful services and Reliable Collections?

I am wondering if I need to add managed storage, and if so, how would I get Service Fabric to use it?

Solution

The temporary storage you see in the nodes, as the name suggests, are meant to be used to store data temporarily, an example is when you have to process a a big file, you first download it, save to the disk, then start processing it. Once completed, you delete the file and go to the next step.

Temporary storage is not meant to be used as a permanent data or file store, if you do that, you can face many issues:

The node VM might get decommissioned and your data is lost(Hardware updates or failure)
These disks are unique per node, when your service more around, the data is not going together.

Your can loose the data on temporary disk on the following scenarios:

When you resize the VM,
When you shutdown or restart your VM,
When your VM is moved to a different host server(due to service healing, shutdown and restart),
When the host is updated,
When the host experiences a hardware failure etc.

Only use temporary to store files you are can loose. Please take a look to this post to know more.

Regarding reliable collections, they are stored in the VM disks, but you won't face the same issue because the data is replicated, so, in case of failure, a replica will be already stored in another node on another Fault Domain, that might not get affected in case of hardware failures. When the service more around, the reliable collections data is copied over from the other replicas(Primary or Secondary)

If you are planning to store files, I would recommend you using Azure Blob Storage or File Share, they have a built-in mechanism to replicate the data and keep your data safe from failures.