amazon-s3amazon-web-servicesboto

How do I translate an AWS S3 url into a bucket name for boto?

I'm trying to access the http://s3.amazonaws.com/commoncrawl/parse-output/segment/ bucket with boto. I can't figure out how to translate this into a name for boto.s3.bucket.Bucket().

This is the gist of what I'm going for:

s3 = boto.connect_s3()
cc = boto.s3.bucket.Bucket(connection=s3, name='commoncrawl/parse-output/segment')
requester = {'x-amz-request-payer':'requester'}
contents = cc.list(headers=requester)
for i,item in enumerate(contents):
    print item.__repr__()

I get "boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request ... The specified bucket is not valid..."

Solution

  • The bucket name would be commoncrawl. Everything that appears after that is really just part of the name of the keys that appear in the bucket.