amazon-web-servicesamazon-s3snowflake-cloud-data-platformapache-iceberg

S3 Intelligent - Tiering with snowflake managed iceberg tables


Trying to save on huge rarely used data storage costs, with snowflake managed iceberg tables. Would you apply the economical s3 intelligent-Tiering storage, to the iceberg’s external volume on s3, instead of the pricy s3 standard policy?

Obviously s3 would chill rarely used data files, which is fine as long as:

Cold storage data is still accessible normally with just sql whenever needed (no additional requests needed from the s3 service to heat up cold\frozen files first…). After cold data was heated-defrosted by someone, it remains so for a reasonable time (a few good days even if not addressed again), before going deep freeze again. Very much appreciate your thoughts on this matter! TIA Rea

Not tried anything yet just designing.


Solution

  • Looking at the S3 Intelligent Tiers against your requirements:

    From How S3 Intelligent-Tiering works - Amazon Simple Storage Service:

    Activate the Archive Access and Deep Archive Access tiers only if your objects can be accessed asynchronously by your application. If the object that you are retrieving is stored in the Archive Access or Deep Archive Access tiers, you must first restore the object by using the RestoreObject operation.

    So, you should be fine as long as you don't use those last two levels.