amazon-web-servicesamazon-s3amazon-glacier

How to restore multiple files from a prefix in AWS


I have plenty of objects in AWS S3 Glacier, I'm trying to restore some of them which are on the same prefix (aka folder). However I can't find a way to restore them all at once, it might be worth mentioning that some of the elements in this prefix are prefixes themselves which I also want to restore.


Solution

  • I've managed to get it working. I had to write a simple bash script that iterates through all the objects in the bucket's prefix which are GLACIER or DEEP_ARCHIVE depending on the case. So there are two components to this:

    First, you need a file with all the objects:

    aws s3api list-objects-v2 --bucket someBucket --prefix 
    some/prefix/within/the/bucket/ --query "Contents[?StorageClass== 'GLACIER']" 
    -- output text | awk '{print $2}' > somefile.txt
    

    The list-objects-v2 will list all the objects in the prefix, with the awk '{print $2}' command we'll make sure the resulting file is iterable and contains just the names of the objects.

    Finally, iterate through the file restoring the objects:

    for i in $(cat somefile.txt); 
    do
        #echo "Sending request for object: $i"
        aws s3api restore-object --bucket $BUCKET --key $i --restore-request Days=$DAYS
        #echo "Request sent for object: $i"  
    done
    

    You can uncomment the echo commands to make the execution more verbose but it's unnecessary for the most part.