amazon-web-serviceselastic-map-reducemrjob

Minimum AWS policy requirements to run an EMR job


I'd like to run an Elastic Mapreduce on data from the S3 bucket com.test.mybucket, using the MRJob Python framework. However I have lots of other data in S3, and other EC2 instances that I don't want to touch. What is the minimum possible set of access credentials an AWS user would need to run a complete job?


Solution

  • Here's an example:

    {
        "Statement": [
            {
                "Action": [
                    "s3:GetObject",
                    "s3:ListBucket",
                    "s3:PutObject",
                    "s3:DeleteObject",
                    "s3:GetBucketLocation"
                ],
                "Resource": [
                    "arn:aws:s3:::com.test.mybucket*"
                ],
                "Effect": "Allow",
                "Sid": "Stmt1320976936189"
            },
            {
                "Action": [
                    "elasticmapreduce:*"
                ],
                "Resource": [
                    "*"
                ],
                "Effect": "Allow",
                "Sid": "Stmt1322766641851"
            },
            {
                "Action": [
                    "ec2:AuthorizeSecurityGroupIngress",
                    "ec2:CancelSpotInstanceRequests",
                    "ec2:CreateSecurityGroup",
                    "ec2:CreateTags",
                    "ec2:DescribeAvailabilityZones",
                    "ec2:DescribeInstances",
                    "ec2:DescribeSecurityGroups",
                    "ec2:DescribeSpotInstanceRequests",
                    "ec2:ModifyImageAttribute",
                    "ec2:ModifyInstanceAttribute",
                    "ec2:RequestSpotInstances",
                    "ec2:RunInstances",
                    "ec2:TerminateInstances"
                ],
                "Resource": [
                    "*"
                ],
                "Effect": "Allow",
                "Sid": "Stmt1323200725902"
            }
        ]
    }
    

    See also http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/index.html?environmentconfig_iam.html#ec2-iam-policies