AWS S3 Bucket Access Issue with IAM User Permissions

I'm currently facing an issue with accessing objects within my AWS S3 bucket using an IAM user that has been granted specific permissions. Here's a brief overview of my setup:

1. S3 Bucket Policy:

• I have an S3 bucket named eskolo-dev-bucket with the following policy attached:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowUserAccess",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::XXXXXXXXXX:user/eskoloAppUsers"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::eskolo-dev-bucket/*"
        }
    ]
}

2. IAM User Policy: The IAM user eskoloAppUsers has the following policies attached:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ListPolicy",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::eskolo-dev-bucket"
            ]
        },
        {
            "Sid": "ManipulatePolicy",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::eskolo-dev-bucket/*"
            ]
        }
    ]
}

3. My .Net Code with Appsettings.json

 public static async Task<bool> DoesS3ObjectExistAsync(AwsSettings awsSettings, string key)
 {
     try
     {
         using var s3Client = new AmazonS3Client(
             awsSettings.AccessKey,
             awsSettings.SecretKey,
             awsSettings.Region);
         var request = new GetObjectMetadataRequest
         {
             BucketName = awsSettings.Bucket,
             Key = key
         };

         await s3Client.GetObjectMetadataAsync(request);
         return true;
     }
     catch (AmazonS3Exception e)
     {
         if (e.StatusCode == HttpStatusCode.NotFound)
             return false;

         throw;
     }

Appsettings.json

"AwsSettings": {
  "AccessKey": "xxxxxxxxxx",
  "Bucket": "eskolo-dev-bucket",
  "LogoUri": "s3://eskolo-dev-bucket/development/assets/",
  "Region": "eu-north-1",
  "SecretKey": "xxxxxxx",
  "TemplateUri": "s3://eskolo-dev-bucket/development/email-templates/"
}

4. Issue Description:

• When attempting to access objects within the eskolo-dev-bucket via my.NET application using the GetObjectMetadataAsync method, I consistently receive a 404 error. The URIs I'm using are: s3://eskolo-dev-bucket/development/assets/eSkolo Logo.svg s3://eskolo-dev-bucket/development/email-templates/e-Skolo-OTP-template.hbs

5. Expected Behavior:

• I expect to be able to successfully retrieve metadata for these objects without receiving a 404 error.

6. Actual Behavior:

• Upon making the request, I receive a 404 error indicating that the requested resource does not exist.

7. Steps Taken to Resolve:

• Double-checked the bucket name and object paths for typos.
• Verified that the IAM user has the correct permissions.
• Ensured that the objects exist within the specified paths in the S3 bucket.

8. Additional Information:

• The IAM user was recently created and the permissions were updated accordingly.
• There are no network issues or firewalls blocking access to AWS services.

To test your policy, I did the following:

• Created a new S3 bucket programmer-in-the-making with default settings
• Created an IAM User and attached this policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ListPolicy",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::programmer-in-the-making"
            ]
        },
        {
            "Sid": "ManipulatePolicy",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::programmer-in-the-making/*"
            ]
        }
    ]
}

• Uploaded a file to the bucket
• Created an Access Key and Secret Key for the IAM User and stored it using aws configure --profile progammer-in-the-making
• Run AWS CLI commands:

% aws s3 ls s3://programmer-in-the-making --profile programmer-in-the-making
2024-07-04 12:37:30      54741 ghost.gif

% aws s3api head-object  --bucket programmer-in-the-making --key ghost.gif --profile programmer-in-the-making
{
    "AcceptRanges": "bytes",
    "LastModified": "2024-07-04T02:37:30+00:00",
    "ContentLength": 54741,
    "ETag": "\"131ca58bc2049ab827b0d6b21b746347\"",
    "ContentType": "image/gif",
    "ServerSideEncryption": "AES256",
    "Metadata": {}
}

(I used head-object as the closest AWS CLI operation to GetObjectMetadataAsync() that you are using.)

The AWS CLI requests were successful, proving that the policy is adequate to access objects in the bucket.

I then tried other commands to confirm that permissions are limited to those granted in the bucket:

% aws s3 ls --profile programmer-in-the-making

An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

Therefore, any problem either lies in your code or in your Access Key/Secret Key. The IAM permissions granted in the policy are correct.