python-3.xamazon-web-servicesamazon-s3boto3amazon-s3-access-points

Read and write from/to S3 bucket using access points with boto3

I have to access S3 bucket using access points with boto3.

I have created an access point with a policy to allow reading and writing (<access_point_arn> is my access point ARN):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": ["s3:GetObject", "s3:PutObject"],
            "Resource": "<access_point_arn>/object/*"
    ]
}

In the official documentation there is a mention about access points, where access point ARN has to come in place of bucket name (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html). There are no examples on the official documentation site for developers (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-access-points.html).

So based on the information I assume that the right way to use it is:

import boto3
s3 = boto3.resource('s3')
s3.Bucket('<access_point_arn>').download_file('hello.txt', '/tmp/hello.txt')

When I execute this code in Lambda with AmazonS3FullAccess managed policy attached I am getting an ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

Both Lambda and S3 access point are connected to the same VPC.

Solution

  • My first guess is that you are missing permissions that have to be defined (1) on the bucket (bucket policy) and (2) on the IAM user or role which you are using in the boto3 SDK.

    (1) From the documentation I can see that

    For an application or user to be able to access objects through an access point, both the access point and the underlying bucket must permit the request.

    You could, for instance, add a bucket policy that is delegating access control to access points so that you don't have to specify each principal that comes via the access points. An example is given in the linked docs.

    (2) As stated in your question, you are already using AmazonS3FullAccess policy in your LambdaExecutionRole. My only guess (i.e. what happened to me) is that there is, e.g., KMS encryption on the objects in your bucket and your role is missing permissions for kms actions. Try executing the function with Admin policy attached and see if it works. If it does, find out which specific permissions are missing.

    Some further notes: I assume you