Accessing python package from custom pypi s3 server in MWAA

Our team is looking to install a custom package we set up internally using a custom pypi server on s3 like this example. We then want to add it to a MWAA requirments.txt file. We configured the IAM policy for MWAA with full bucket access and bucket policy so that MWAA could access the static website to install the whl file. I also tested this locally with the MWAA docker container provided for testing. This worked correctly.

The issue comes when trying to try this on MWAA. I updated the requirments.txt with --extra-index-url www.s3-webite.com --trustedhost s3-url python-package-example==1.56

For some reason this breaks all the dags. When I check cloudwatch, the requirements log shows everything except for python-package-example is installed. ERROR: Could not find a version that satisfies the requirement python-package-example===1.56 (from versions: none)

I am fairly certain it is a networking issue. Has anyone ran into this when trying to add a custom package to your MWAA instance? ERROR: No matching distribution found for python-package-example===1.56

Example bucket policy

{
    "Version": "Date",
    "Id": "MWAApolicyforcustomS3",
    "Statement": [
        {
            "Sid": "AllowSpecificMWAAAccess",
            "Effect": "Allow",
            "Principal": {
                "AWS": "mwaa-role-example"
            },
            "Action": "s3:GetObject",
            "Resource": [
                "arn:aws:s3:::bucket",
                "arn:aws:s3:::bucket/*"
            ]
        }

So the issue I realized was that because it was a private subnet the way MWAA was installing pip packages so to fix the issue all I had to do was configure the bucket policy to allow access from the Nat gateway IP.