[SOLVED] Python upload a file to AWS S3 with existing presign URL from another account

Python upload a file to AWS S3 with existing presign URL from another account

I have a pre sign URL from a client, I want to store the file in my AWS S3 bucket

from requests import get
resp = get(url=[Presign URL], timeout=9000)
s3_client.upload_fileobj(resp.content, "my-bucket", "xyz/abc/hmm.avro")

with this code, I am getting the below error

{ValueError}Fileobj must implement read

I there a way to directly move the file

Solution

The error you are encountering is because the upload_fileobj method of the S3 client in boto3 is expecting a file-like object that has a read method. But the content attribute of the response object returned by requests.get is bytes.

you can use io.BytesIO to create a file-like object from the bytes:

import io
import boto3
from requests import get

presigned_url = "[Presign URL]"
response = get(url=presigned_url, timeout=9000)

if response.status_code == 200:
    file_like_object = io.BytesIO(response.content)
    s3_client = boto3.client('s3')
    s3_client.upload_fileobj(file_like_object, "my-bucket", "xyz/abc/hmm.avro")
else:
    print("Failed to fetch the file")

Read here for more info regarding files upload