amazon-web-servicesamazon-s3terraformelastic-load-balancer

Terraform ELB S3 Permissions Issue


I am having an issue using Terraform (v0.9.2) adding services to an ELB (I'm using: https://github.com/segmentio/stack/blob/master/s3-logs/main.tf).

When I run terraform apply I get this error:

* module.solr.module.elb.aws_elb.main: 1 error(s) occurred:

* aws_elb.main: Failure configuring ELB attributes: 
    InvalidConfigurationRequest: Access Denied for bucket: my-service-
    logs. Please check S3bucket permission
    status code: 409, request id: xxxxxxxxxx-xxxx-xxxx-xxxxxxxxx

My service looks like this:

module "solr" {
  source = "github.com/segmentio/stack/service"
  name = "${var.prefix}-${terraform.env}-solr"
  environment = "${terraform.env}"
  image = "123456789876.dkr.ecr.eu-west-2.amazonaws.com/my-docker-image"
  subnet_ids = "${element(split(",", module.vpc_subnets.private_subnets_id), 3)}"
  security_groups = "${module.security.apache_solr_group}"
  port = "8983"
  cluster = "${module.ecs-cluster.name}"
  log_bucket = "${module.s3_logs.id}"

  iam_role = "${aws_iam_instance_profile.ecs.id}"
  dns_name = ""
  zone_id = "${var.route53_zone_id}"
}

My s3-logs bucket looks like this:

module "s3_logs" {
  source = "github.com/segmentio/stack/s3-logs"
  name = "${var.prefix}"
  environment = "${terraform.env}"
  account_id = "123456789876"
}

I checked in S3 and the bucket policy looks like this:

{
  "Version": "2012-10-17",
  "Id": "log-bucket-policy",
  "Statement": [
  {
  "Sid": "log-bucket-policy",
  "Effect": "Allow",
  "Principal": {
  "AWS": "arn:aws:iam::123456789876:root"
  },
  "Action": "s3:PutObject",
  "Resource": "arn:aws:s3:::my-service-logs/*"
  }
  ]
}

As far as I can see ELB should have access to the S3 bucket to store the logs (it's running in the same AWS account).

The bucket and the ELB are all in eu-west-2.

Any ideas on what the problem could be would be much appreciated.


Solution

  • The docs for ELB access logs say that you want to allow a specific Amazon account to be able to write to S3, not your account.

    As such you want something like:

    {
      "Id": "Policy1429136655940",
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "Stmt1429136633762",
          "Action": [
            "s3:PutObject"
          ],
          "Effect": "Allow",
          "Resource": "arn:aws:s3:::my-loadbalancer-logs/my-app/AWSLogs/123456789012/*",
          "Principal": {
            "AWS": [
              "652711504416"
            ]
          }
        }
      ]
    }
    

    In Terraform you can use the aws_elb_service_account data source to automatically fetch the account ID used for writing logs as can be seen in the example in the docs:

    data "aws_elb_service_account" "main" {}
    
    resource "aws_s3_bucket" "elb_logs" {
      bucket = "my-elb-tf-test-bucket"
      acl    = "private"
    
      policy = <<POLICY
    {
      "Id": "Policy",
      "Version": "2012-10-17",
      "Statement": [
        {
          "Action": [
            "s3:PutObject"
          ],
          "Effect": "Allow",
          "Resource": "arn:aws:s3:::my-elb-tf-test-bucket/AWSLogs/*",
          "Principal": {
            "AWS": [
              "${data.aws_elb_service_account.main.arn}"
            ]
          }
        }
      ]
    }
    POLICY
    }
    
    resource "aws_elb" "bar" {
      name               = "my-foobar-terraform-elb"
      availability_zones = ["us-west-2a"]
    
      access_logs {
        bucket   = "${aws_s3_bucket.elb_logs.bucket}"
        interval = 5
      }
    
      listener {
        instance_port     = 8000
        instance_protocol = "http"
        lb_port           = 80
        lb_protocol       = "http"
      }
    }