I operate a SAAS software system in which user data is stored in Azure blob storage with a separate container for each user. The storage account currently contains just over 1000 containers. I need to perform periodic full backups of the storage account and was planning to use Vaulted backup for this purpose. However, it turns out that Vaulted backup only supports 100 containers, which is nowhere near enough for my purposes. Object Replication is also not an option since that only supports 1000 containers.
I've considered restructuring the storage account, so that all data would be stored in a single container using folders for organization, but I've read that this can lead to worse performance, and it also feels less intuitive and maintainable to me.
Incremental / operational backup is not sufficient since I'm legally required to perform full weekly backups.
Currently, I've written code manually transferring the blobs using the API, but that's an awkward solution. I would much prefer to handle this with built-in Azure functionality.
Does anyone have any suggestions? I don't understand why Azure adds these arbitrary container limits.
Does anyone have any suggestions? I don't understand why Azure adds these arbitrary container limits.
The container limits in Azure's backup and replication solutions, such as the 100-container limit for Vaulted Backup and the 1000-container limit for Object Replication, are not arbitrary, It is based on a combination of technical, scalability, and performance considerations that Microsoft has designed into the platform.
I agree Gaurav Mantri's comment, you can use Multiple Azure Storage Accounts to Stay Within Azure Limits for Backup.
users or data logically and using multiple storage accounts is a solid strategy for managing Azure Blob Storage backups while adhering to the container limits.Azure Automation, Azure Functions, or Logic Apps.The above approach balances scalability and compliance without requiring a complete restructuring of your system.
Reference: