riakriak-cs

Riak CS how to get list of all buckets and all files


I was surprised to not find in documentation the answer on really simple questions: How can I get the list of all buckets and all files in my Riak CS installation? Also I need to have access to all those files in all buckets with some "root" account - how can I do that?

Anyway according to documentation: http://docs.basho.com/riakcs/latest/cookbooks/Account-Management/ A sample URL for a user listing request looks like this:

GET http://data.example.com/riak-cs/users -
or
s3curl --id admin -- http://data.mystorage.me/riak-cs/users

So, I've edited .s3crul:

%awsSecretAccessKeys = (
adminname => {
id => 'key here',
key => 'secret here',
},
);

and point s3curl directly to riak cs:

s3curl.pl --id adminname -- http://192.168.0.20:8080/riak-cs/users

Getting answer:

<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><Message>Access Denied</Message><Resource>/riak-cs/users</Resource><RequestId></RequestId></Error>

I'm sure that "adminname" is Admin account in Riak CS, I can see it in riac-cs-control panel.


Solution

  • I seems that you use custom host base, so what you have to do is to tell it to s3curl.pl. As you know, because S3 (or AWS v2) authentication can be path style or subdomain style, s3curl.pl can not use host part of URL or host header as is. Therefore, it has some hardcoded endpoints [1] for indentifing host base.

    Some forks of s3curl.pl can specify the custom endpoint by option argument [2]. If you setting cs_root_host as data.mystorage.me, then the example command line looks like:

    s3curl --id admin --endpoint data.mystorage.me \
        -- http://data.mystorage.me/riak-cs/users
    

    or by subdomain style,

    s3curl --id admin --endpoint data.mystorage.me \
        -- http://riak-cs.data.mystorage.me/users
    

    Once you get all user information, you can do anything, e.g listing buckets for certain user or listing all objects in certain bucket.

    [1] https://github.com/ksauzz/s3curl/blob/master/s3curl.pl#L30-L34 [2] https://github.com/ksauzz/s3curl/blob/master/s3curl.pl#L121