pythonibm-cloudobject-storageibm-cloud-storage

"The specified bucket does not exist" when trying to list objects in IBM Cloud Object Storage using Python


Following IBM documentation, I'm trying to list objects in my COS bucket using Python and ibm-cos-sdk.

import ibm_boto3
from ibm_botocore.client import Config, ClientError

cos = ibm_boto3.resource("s3",
      ibm_api_key_id=params['API_KEY'],
      ibm_service_instance_id=params['COS_TGT_INSTANCE_CRN'],
      ibm_auth_endpoint=params['IAM_ENDPOINT'],
      config=Config(signature_version="oauth"),
      endpoint_url=params['COS_ENDPOINT']
)

for obj in cos.Bucket('sql-efbfb11b-fa01-4c49-8fe1-c70793be3f5f').objects.all():
  print(obj.key)

Which results in:

ibm_botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjects operation: The specified bucket does not exist.

I'm pretty sure the bucket does exist, because I can clearly see it in the output from

>>> for b in cos.buckets.all():
...  print(b.name)
... 
sql-efbfb11b-fa01-4c49-8fe1-c70793be3f5f

What am I doing wrong here?


Solution

  • The reason for the error is conceptual. You can see all buckets, but only can obtain details for buckets in the connected region. I ran into this a long time ago and solved it like this (tested then, not today):

    def buckets_json():
       # Get a list of all bucket names from the response
       buckets = [bucket['Name'] for bucket in cos.list_buckets()['Buckets']]
       locs2=[{"name":name,"loc":loc} for name,loc in locations(buckets=buckets).iteritems()]
       return jsonify(buckets=locs2)
    

    Another snippet which I found:

    def locations(buckets):
       locs={}
       for b in buckets:
          try: 
             locs[b]=cos.get_bucket_location(Bucket=b)['LocationConstraint']
          except: 
             locs[b]=None
             pass
       return locs