s3fs

s3fs -- bucket name contains illegal character

I am trying to mount an S3 bucket onto my ubuntu instance . But I want to only mount a "directory" or key in the s3 bucket.

bucketname : s3://my-bucket/ the directory or part of the s3 bucket I want to mount : s3://my-bucket/test/ mount point : /path/to/mount/dir/

I am using s3fs to achieve this like so:

 s3fs my-bucket/test/ /path/to/mount/dir

But I get the following error all the time : s3fs: BUCKET my-bucket/test/ -- bucket name contains an illegal character.

Any suggestions as to how I can mount a sub-directory of the s3 bucket with s3fs?

Thanks!

[CRT] s3fs_logger.cpp:LowSetLogLevel(203): change debug level from [CRT]to [INF]
[INF]     s3fs.cpp:set_mountpoint_attribute(3984): PROC(uid=1002, gid=1002) - MountPoint(uid=1002, gid=1002, mode=40777)
[INF] curl.cpp:InitMimeType(432): Loaded mime information from /etc/mime.types
[INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission.
[INF] s3fs.cpp:s3fs_init(3295): init v1.87(commit:38e1eaa) with OpenSSL
[INF] s3fs.cpp:s3fs_check_service(3411): check services.
[INF]       curl.cpp:CheckBucket(3235): check a bucket.
[WAN] curl.cpp:ResetHandle(1875): The S3FS_CURLOPT_KEEP_SENDING_ON_ERROR option could not be set. For maximize performance you need to enable this option and you should use libcurl 7.51.0 or later.
[INF]       curl_util.cpp:prepare_url(243): URL is https://s3.amazonaws.com/my-bucket/test/
[INF]       curl_util.cpp:prepare_url(276): URL changed is https://my-bucket.s3.amazonaws.com/test/
[INF]       curl.cpp:insertV4Headers(2572): computing signature [GET] [/test/] [] []
[INF]       curl_util.cpp:url_to_host(320): url is https://s3.amazonaws.com
[ERR] curl.cpp:RequestPerform(2256): HTTP response code 400, returning EIO. Body Text: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-south-1'</Message><Region>ap-south-1</Region><RequestId>5366A05FF406855D</RequestId><HostId>KmvHCBnlvnNZI/g3F2p+NfS0vP1Niq0+OPCZAP9u0pCjTOvpYzXfAB9QMlgQlcei2oi+lihn7F8=</HostId></Error>
[ERR] curl.cpp:CheckBucket(3261): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-south-1'</Message><Region>ap-south-1</Region><RequestId>5366A05FF406855D</RequestId><HostId>KmvHCBnlvnNZI/g3F2p+NfS0vP1Niq0+OPCZAP9u0pCjTOvpYzXfAB9QMlgQlcei2oi+lihn7F8=</HostId></Error>
[CRT] s3fs.cpp:s3fs_check_service(3444): Failed to connect region 'us-east-1'(default), so retry to connect region 'ap-south-1'.
[INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler
[INF]       curl.cpp:CheckBucket(3235): check a bucket.
[INF]       curl_util.cpp:prepare_url(243): URL is https://s3-ap-south-1.amazonaws.com/my-bucket/test/
[INF]       curl_util.cpp:prepare_url(276): URL changed is https://my-bucket.s3-ap-south-1.amazonaws.com/test/
[INF]       curl.cpp:insertV4Headers(2572): computing signature [GET] [/test/] [] []
[INF]       curl_util.cpp:url_to_host(320): url is https://s3-ap-south-1.amazonaws.com
[INF]       curl.cpp:RequestPerform(2266): HTTP response code 404 was returned, returning ENOENT
[ERR] curl.cpp:CheckBucket(3261): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>test/</Key><RequestId>FC8360185E0E5B79</RequestId><HostId>kwY8RPFw8d/6K9VL+cqc0KYDDgGVUVplbcvT+6PDBMvsdO0N53vQhfkfXRYzSQpy0sZ73f+er6o=</HostId></Error>
[CRT] s3fs.cpp:s3fs_check_service(3484): bucket not found(host=https://s3-ap-south-1.amazonaws.com) - result of checking service.
[ERR] s3fs.cpp:s3fs_exit_fuseloop(3285): Exiting FUSE event loop due to errors
[INF] s3fs.cpp:s3fs_destroy(3353): destroy
Solution

  • Try separating the bucket name from the bucket path with a colon:

    s3fs my-bucket:/test /path/to/mount/dir

    s3fs also requires that a directory object exists to mount a path. You can create this via the AWS CLI:

    aws s3api put-object --bucket my-bucket --key test/