amazon-s3riak-cs

Riak CS returns "s3.amazonaws.com" no matter what is in cs_root_host

cs_root_host is set up right:

grep root_host /var/lib/riak-cs/generated.configs/app.2015.07.09.13.59.07.config
    {cs_root_host,"s3.example.com"},

But when I upload file:

s3cmd put test.jpg s3://images --acl-public

I get in return:

Public URL of the object is: http://images.s3.amazonaws.com/test.jpg

Where is the issue?

Added: Here is output - everything looks fine, except the last line:

(example.com is just replacement for real domain which I don't want to public)


    s3cmd -d -c .s3cfg put newfile.jpg s3://images --acl-public
    DEBUG: ConfigParser: Reading file '.s3cfg'
    DEBUG: ConfigParser: access_key->YD...17_chars...U
    DEBUG: ConfigParser: bucket_location->RU
    DEBUG: ConfigParser: cloudfront_host->cloudfront.amazonaws.com
    DEBUG: ConfigParser: cloudfront_resource->/2010-07-15/distribution
    DEBUG: ConfigParser: default_mime_type->binary/octet-stream
    DEBUG: ConfigParser: delete_removed->False
    DEBUG: ConfigParser: dry_run->False
    DEBUG: ConfigParser: encoding->UTF-8
    DEBUG: ConfigParser: encrypt->False
    DEBUG: ConfigParser: follow_symlinks->False
    DEBUG: ConfigParser: force->False
    DEBUG: ConfigParser: get_continue->False
    DEBUG: ConfigParser: gpg_command->/usr/bin/gpg
    DEBUG: ConfigParser: gpg_decrypt->%(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
    DEBUG: ConfigParser: gpg_encrypt->%(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
    DEBUG: ConfigParser: gpg_passphrase->...-3_chars...
    DEBUG: ConfigParser: guess_mime_type->True
    DEBUG: ConfigParser: host_base->s3.example.com
    DEBUG: ConfigParser: host_bucket->%(bucket)s.s3.example.com
    DEBUG: ConfigParser: human_readable_sizes->False
    DEBUG: ConfigParser: list_md5->False
    DEBUG: ConfigParser: log_target_prefix->
    DEBUG: ConfigParser: preserve_attrs->True
    DEBUG: ConfigParser: progress_meter->True
    DEBUG: ConfigParser: proxy_host->127.0.0.1
    DEBUG: ConfigParser: proxy_port->80
    DEBUG: ConfigParser: recursive->False
    DEBUG: ConfigParser: recv_chunk->4096
    DEBUG: ConfigParser: reduced_redundancy->False
    DEBUG: ConfigParser: secret_key->kG...37_chars...=
    DEBUG: ConfigParser: send_chunk->4096
    DEBUG: ConfigParser: simpledb_host->sdb.amazonaws.com
    DEBUG: ConfigParser: skip_existing->False
    DEBUG: ConfigParser: socket_timeout->10
    DEBUG: ConfigParser: urlencoding_mode->normal
    DEBUG: ConfigParser: use_https->False
    DEBUG: ConfigParser: verbosity->WARNING
    DEBUG: Updating Config.Config encoding -> UTF-8
    DEBUG: Updating Config.Config follow_symlinks -> False
    DEBUG: Updating Config.Config verbosity -> 10
    DEBUG: Unicodising 'put' using UTF-8
    DEBUG: Unicodising 'newfile.jpg' using UTF-8
    DEBUG: Unicodising 's3://images' using UTF-8
    DEBUG: Command: put
    INFO: Compiling list of local files...
    DEBUG: DeUnicodising u'' using UTF-8
    DEBUG: DeUnicodising u'newfile.jpg' using UTF-8
    DEBUG: Unicodising 'newfile.jpg' using UTF-8
    DEBUG: Unicodising 'newfile.jpg' using UTF-8
    INFO: Applying --exclude/--include
    DEBUG: CHECK: newfile.jpg
    DEBUG: PASS: newfile.jpg
    INFO: Summary: 1 local files to upload
    DEBUG: Content-Type set to 'image/jpeg'
    DEBUG: String 'newfile.jpg' encoded to 'newfile.jpg'
    DEBUG: SignHeaders: 'PUT\n\nimage/jpeg\n\nx-amz-acl:public-read\nx-amz-date:Fri, 10 Jul 2015 09:55:37 +0000\n/images/newfile.jpg'
    DEBUG: CreateRequest: resource[uri]=/newfile.jpg
    DEBUG: Unicodising 'newfile.jpg' using UTF-8
    DEBUG: SignHeaders: 'PUT\n\nimage/jpeg\n\nx-amz-acl:public-read\nx-amz-date:Fri, 10 Jul 2015 09:55:37 +0000\n/images/newfile.jpg'
    newfile.jpg -> s3://images/newfile.jpg  [1 of 1]
    DEBUG: get_hostname(images): images.s3.example.com
    DEBUG: format_uri(): http://images.s3.example.com/newfile.jpg
     32600 of 32600   100% in    0s    14.49 MB/sDEBUG: Response: {'status': 200, 'headers': {'content-length': '0', 'server': 'nginx', 'connection': 'keep-alive', 'etag': '"89e39f454c69a1ce1fadec3a222fc292"', 'date': 'Fri, 10 Jul 2015 09:55:37 GMT', 'content-type': 'text/plain'}, 'reason': 'OK', 'data': '', 'size': 32600}
     32600 of 32600   100% in    0s   391.54 kB/s  done
    DEBUG: MD5 sums: computed=89e39f454c69a1ce1fadec3a222fc292, received="89e39f454c69a1ce1fadec3a222fc292"
    Public URL of the object is: http://images.s3.amazonaws.com/newfile.jpg
Solution

  • This is not a Riak CS issue. s3cmd itself produce public url string and print it.

    For my environment, with s3cmd of master branch of commit 7bdefc81823699069706ea3680bfa65ec8ad3db5 (just fetched today, 2015-07-14), it shows (seemingly) the corrent URL.

    % ~/g/s3cmd/build/scripts-2.7/s3cmd -c .s3cfg.15018.alice put rebar.config -P s3://test/a
rebar.config -> s3://test/a  [1 of 1]
2791 of 2791   100% in    0s   196.88 kB/s  done
Public URL of the object is: http://test.s3.example.com/a

    From the source code of s3cmd, it seems it uses host_bucket or host_base configuration depending on bucket name (or maybe other configurations.)

    Some other details on my environment