ibm-cloudobject-storage

IBM COS Python SDK | Create client/resource object authenticated via HMAC keys


Using IBM COS SDK, is it possible to create a COS client/resource without specifying the field ibm_api_key_id, instead specifying some combination of aws_access_key_id and aws_secret_access_key with other fields? Tried the following without success:

get_cos_resource(region: str = 'us-east'):
    return ibm_boto3.resource(
        's3',
        aws_access_key_id=KEY_ID,
        aws_secret_access_key=SECRET_KEY,
        config=ibm_botocore.client.Config( 
            signature_version='oauth'),
        endpoint_url=f'https://s3.{region}.cloud-object-storage.appdomain.cloud',
        ibm_service_instance_id=INSTANCE_ID)
s3 = get_cos_resource()
bucket = s3.Bucket("bucket-name")
print(list(bucket.objects.all()))

Returns:
ibm_botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied.

Note - the aws_access_key_id and aws_secret_access_key are service credentials of the specified ibm_service_instance_id.


Solution

  • For all concerned, the following fix should work:

    def get_cos_resource_hmac(region = "us-east"):
        return ibm_boto3.resource('s3',
                            aws_access_key_id=ACCESS_KEY_ID,
                            aws_secret_access_key=SECRET_ACCESS_KEY,
                            endpoint_url=f'https://s3.{region}.cloud-object-storage.appdomain.cloud'
                            )
    

    Apparently the fields: ibm_service_instance_id and config can't be incorporated into the API request. The redundancy of ibm_service_instance_id makes sense, since unlike ibm_api_key_id, which is an account wide authentication key, the aws_access_key_id and aws_secret_access_key are bound to a specific storage instance.

    Note - when authenticating using HMAC keys, remember to leave out the field ibm_service_instance_id when making write operations, such as create_bucket(), e.g.:

    client.create_bucket(
                    Bucket=bucket_name,
                    CreateBucketConfiguration={
                        'LocationConstraint': f'{region}-smart'
                    })