amazon-s3aws-sdk-cpp

With c++ S3 SDK, ListObjecsV2 with Minio returning no results


Below is some code which puts an object into an s3 bucket, fetches it back, and then tries to list the bucket. The put and get both work fine, but listing the bucket returns no keys (but doesn't error). I can list the bucket using the golang api and the same parameters (bucket name and empty prefix). Why won't it list the bucket?

Aws::SDKOptions opts;

opts.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Debug;
Aws::InitAPI(opts);

Aws::Auth::AWSCredentials credentials;
Aws::Client::ClientConfiguration config;
Aws::S3::S3Client client;

credentials.SetAWSAccessKeyId("accesskey");
credentials.SetAWSSecretKey("secretkey");
config.endpointOverride = "localhost:5553";
config.scheme = Aws::Http::Scheme::HTTP;
config.verifySSL = false;
client = Aws::S3::S3Client(credentials, config);

const char* bucket = "buck";
const char* key = "buck/key";

// Putting works!
//
Aws::S3::Model::PutObjectRequest pRequest;
pRequest.SetBucket(bucket);
pRequest.SetKey(key);

auto stream = Aws::MakeShared<Aws::StringStream>("tag", "some data");
pRequest.SetBody(stream);

Aws::S3::Model::PutObjectOutcome pOutcome = client.PutObject(pRequest);

if (!pOutcome.IsSuccess())
{
    printf("put failed S3 because %s: %s\n",
           pOutcome.GetError().GetExceptionName().c_str(),
           pOutcome.GetError().GetMessage().c_str());
    printf("\n\n\n\n");
    assert(false);
}

printf("object put\n");

// Getting works!
//
Aws::S3::Model::GetObjectRequest gRequest;
gRequest.SetBucket(bucket);
gRequest.SetKey(key);

Aws::S3::Model::GetObjectOutcome gOutcome = client.GetObject(gRequest);

if (!gOutcome.IsSuccess())
{
    printf("get failed S3 because %s: %s\n",
           gOutcome.GetError().GetExceptionName().c_str(),
           gOutcome.GetError().GetMessage().c_str());
    printf("\n\n\n\n");
    assert(false);
}

printf("object got\n");

char buf[1000];
memset(buf, 0, 1000);
gOutcome.GetResult().GetBody().read(buf, 1000);

printf("object say '%s'\n", buf);

// THIS IS THE PROBLEMATIC CODE
//
Aws::S3::Model::ListObjectsV2Request request;
request.SetBucket(bucket);                
request.SetPrefix("");
request.SetMaxKeys(1000);

Aws::S3::Model::ListObjectsV2Outcome outcome = client.ListObjectsV2(request);

if (!outcome.IsSuccess())
{
    printf("Failed to list files from S3 because %s: %s\n",
           outcome.GetError().GetExceptionName().c_str(),
           outcome.GetError().GetMessage().c_str());
    printf("\n\n\n\n");
    assert(false);
}

printf("num keys returned %d\n", outcome.GetResult().GetKeyCount());

for (const Aws::S3::Model::Object& obj : outcome.GetResult().GetContents())
{
    printf("Callback %s", obj.GetKey().c_str());
}

Aws::Utils::Logging::ShutdownAWSLogging();

exit(0);

I'm using minio as my object store, but with amazon s3 it works fine


Solution

  • looking at the difference in http requests and reading the code, it looks like you need to set m_useVirtualAddressing to true in the S3Client.

    Ok... who knew.

    My client says

       client = Aws::S3::S3Client(credentials, config, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, false);