How to PUT S3 objects from another AWS account into your own account S3 bucket using assume role?

I have a pretty typical use case where I have a role granted to me in AWS account (1234567890 - not under my control) to read data from their S3 bucket ('remote_bucket'). I can read the data from remote bucket just fine, but I no longer can dump it into my own bucket, because assuming role in another AWS account "hides" grants to resource on my own account. Last row fails with the error. How to solve this?

import boto3

# Create IAM client and local session
sts = boto3.client('sts')
local_sess = boto3.Session()
s3_local = local_sess.resource('s3')

role_to_assume_arn='arn:aws:iam::1234567890:role/s3_role'
role_session_name='test'

# Assume role in another account to access their S3 bucket
response = sts.assume_role(
    RoleArn = role_to_assume_arn,
    RoleSessionName = 'test',
    ExternalId = 'ABCDEFG12345'
)

creds=response['Credentials']

# open session in another account:
assumed_sess = boto3.Session(
    aws_access_key_id=creds['AccessKeyId'],
    aws_secret_access_key=creds['SecretAccessKey'],
    aws_session_token=creds['SessionToken'],
)

remote_bucket = 'remote_bucket'
s3_assumed = assumed_sess.resource('s3')
bk_assumed = s3_assumed.Bucket(remote_bucket)

for o in bk_assumed.objects.filter(Prefix="prefix/"):
    print(o.key)
    in_object = s3_assumed.Object(remote_bucket, o.key)
    content = in_object.get()['Body'].read()
    s3_local.Object('my_account_bucket', o.key).put(Body=content)

Error:

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

(TL;DR See the very end for a way to configure permissions that is easier than using Roles!)

The easiest way to move data between buckets is to use the copy_object() command. This command is sent to the destination bucket and "pulls" information from the source bucket.

This is made slightly more complicated when multiple AWS accounts are involved because the same set of credentials requires GetObject permission on the source bucket AND PutObject permission on the destination bucket.

Your situation appears to be:

• GetObject permission has been granted on the source bucket via a Role that you can assume
• However, that role does not have PutObject permission on the destination bucket

It is important that provided the role is also assigned permissions to write to the destination bucket.

To test this situation, I did the following:

• Created Bucket-A in Account-A (the source bucket) and uploaded some test files
• Created Bucket-B in Account-B (the destination bucket)
• Created Role-A in Account-A, specifying that it can be used by Account-B
• Assigned this IAM policy to Role-A:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:List*",
                "s3:Get*"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-a",
                "arn:aws:s3:::bucket-a/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:Put*"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-b/*"
            ]
        }
    ]
}

Note that the role has also been given permission to write to Bucket-B. This might not be present in your particular situation, but it is necessary otherwise Account-A will not permit the role to call Bucket-B!

To clarify: When using cross-account permissions, both accounts need to grant permission. In this case, Account-A is granting Role-A permission to write to Bucket-B, but Account-B also has to permit the write (see Bucket Policy below).

• Created User-B in Account-B and gave permissions to call AssumeRole on Role-A in Account-A:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AssumeRoleA",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::111111111111:role/role-a"
        }
    ]
}

• Created a Bucket Policy on Bucket-B allowing Role-A to put files into the bucket. This is important because Account-A does not have any access to resources in Account-B, but this bucket policy will allow Role-A to use the bucket.

  {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::bucket-b/*",
      "Principal": {
        "AWS": [
          "arn:aws:iam::111111111111:role/role-a"
        ]
      }
    }
  ]
}

• Updated my credentials file to use Role-A:

[role-a]
role_arn = arn:aws:iam::906972072995:role/role-a
source_profile = user-b

• Confirmed that I could access Bucket-A:

aws s3 ls s3://bucket-a --profile role-a

• Confirmed that I could copy objects from Bucket-A to Bucket-B:

aws s3 cp s3://bucket-a/foo.txt s3://stack-bucket-b/ --profile role-a

This worked successfully.

Summary

The above process might seem rather complex but it can be easily divided between the source and destination:

• Source: Provided Role-A that can read from Bucket-A, also with permissions to write to Bucket-B
• Destination: Bucket policy on Bucket-B allowing Role-A to write to it

If Account-A is not willing to provide role permissions that can both read from Bucket-A and write to Bucket-B, then there is another option:

• Ask Account-A to add a bucket policy to Bucket-A permitting User-B to GetObject
• User-B can then use normal (User-B) permissions to read the content from Bucket-A without any need for AssumeRole
• This is, obviously, a lot easier than assuming a role