amazon-s3s3cmd

S3 moving files between buckets on different accounts?

I'm doing some work for a client that has 2 separate AWS accounts. We need to move all the files in a bucket on one of their S3 accounts to a new bucket on the 2nd account.

We thought that s3cmd would allow this, using the format:

s3cmd cp s3://bucket1 s3://bucket2 --recursive

However this only allows me to use the keys of one account and I can't specify the accounts of the 2nd account.

Is there a way to do this without downloading the files and uploading them again to the 2nd account?

Solution

  • You don't have to open permissions to everyone. Use the below Bucket policies on source and destination for copying from a bucket in one account to another using an IAM user

    The below policy means – the IAM user - XXXX–XXXX-XXXX:src–iam-user has s3:ListBucket and s3:GetObject privileges on SourceBucket/* and s3:ListBucket and s3:PutObject privileges on DestinationBucket/*

    On the SourceBucket the policy should be like:

    {
  "Id": "Policy1357935677554",
  "Statement": [{
    "Sid": "Stmt1357935647218",
    "Action": ["s3:ListBucket"],
    "Effect": "Allow",
    "Resource": "arn:aws:s3:::SourceBucket",
    "Principal": {"AWS": "arn:aws:iam::XXXXXXXXXXXX:user/src–iam-user"}
  }, {
    "Sid": "Stmt1357935676138",
    "Action": ["s3:GetObject"],
    "Effect": "Allow",
    "Resource": "arn:aws:s3:::SourceBucket/*",
    "Principal": {"AWS": "arn:aws:iam::XXXXXXXXXXXX:user/src–iam-user"}
  }]
}

    On the DestinationBucket the policy should be:

    {
  "Id": "Policy1357935677555",
  "Statement": [{
    "Sid": "Stmt1357935647218",
    "Action": ["s3:ListBucket"],
    "Effect": "Allow",
    "Resource": "arn:aws:s3:::DestinationBucket",
    "Principal": {"AWS": "arn:aws:iam::XXXXXXXXXXXX:user/src–iam-user"}
  }, {
    "Sid": "Stmt1357935676138",
    "Action": ["s3:PutObject"],
    "Effect": "Allow",
    "Resource": "arn:aws:s3:::DestinationBucket/*",
    "Principal": {"AWS": "arn:aws:iam::XXXXXXXXXXXX:user/src–iam-user"}
  }]
}

    Command to be run is s3cmd cp s3://SourceBucket/File1 s3://DestinationBucket/File1