I have uploaded several thousand video files to an S3 bucket, then changed the bucket management settings to migrate the files to Glacier. I'm trying to retrieve a single file and copy it to my local machine. Typically, I follow the instructions here. I use the S3 management console, select the 'Action' to restore the selected file from Glacier and then download it using the following command:
aws s3 cp s3://my-bucket/my_video_file.mp4 .
This works as I want it to but I'm wondering if there is a way to restore the file from Glacier without needing to sign-in through the web browser and manually select it for retrieval. Looking through the documentation for aws s3 cp there is an option called --force-galcier-transfer but when I include it in my command I get the following:
Object is of storage class GLACIER. Unable to perform download operations on GLACIER objects. You must restore the object to be able to perform the operation. See aws s3 download help for additional parameter options to ignore or force these transfers.
Here's the passage in the command from the manual page:
--force-glacier-transfer (boolean) Forces a transfer request on all Glacier objects in a sync or recursive copy.
Is it possible to retrieve and download a single file from glacier in a single cli command or will I always need to use the management console to retrieve the file first? I'm also open to using a python script or something similar if it can be done that way.
You can restore a file from glacier using the CLI, but there is no way to both restore and download it in a single command, as you need to restore the file from glacier, then some time later, maybe hours later, after the restore is complete, download the file.
To restore the file, use a command like this:
aws s3api restore-object --bucket bucketname --key path/to/object.zip --restore-request '{"Days":25,"GlacierJobParameters":{"Tier":"Standard"}}'
You can check on the status of the restore request with a command like this:
aws s3api head-object --bucket bucketname --key path/to/object.zip
That will output a json object with details on S3 object, including the status:
.... While the restore is still in progress ...
"Restore": "ongoing-request=\"true\"",
.... Or, when the restore is done ...
"Restore": "ongoing-request=\"false\", expiry-date=\"...\"",
And from there, it's an object in S3, you can simply copy it to your local machine:
aws s3 cp s3://bucketnamepath/to/object.zip object.zip
Of course, scripting all of this is possible. boto3 in Python makes it fairly straight forward to follow this same pattern, but it's pretty much possible to do this in whatever language you prefer to use.