data-lakesqlakeupsolver

What lifecycle role can we apply on Upsolver buckets?


We are currently trying to determining approach reducing our S3 storage cost. Can we get some details round how it can be handled in Upsolver?

We have tried currently in process of identifying data freshness, usability and retention time


Solution

  • A lot depends on your data retention, recovery and replay needs. Upsolver allows to set data retention which will then delete the data (and save storage cost). Default is, data is retained for ever. But note that once data is deleted, replay won't be able to go back to that time frame.

    We can not do S3 Intelligent Tiering as well for the exact same purpose. Once the objects are moved to the Infrequent access, cost to retrieve would be quite high for replay scenarios. So while there can be substantial cost savings initially but if you end up doing replay, the IA tier cost could surpass the initial savings and end up costlier.