I am trying to get a list of object names in the aws-publicdatasets bucket for commoncrawl. I have been looking at using Jets3t to createSignedGetUrl but I am not sure if I can get a list of the objects via the method below:
credentials = new AWSCredentials(accessId, secretId);
restS3Service = new RestS3Service(credentials);
restS3Service.setRequesterPaysEnabled(true);
Calendar cal = Calendar.getInstance();
cal.set(Calendar.HOUR, cal.get(Calendar.HOUR)+1);
Date expiryDate = new Date(cal.getTimeInMillis());
String getUrl = restS3Service.createSignedGetUrl(bucketName, null,
expiryDate, false);
S3Object obj = signedUrlHandler.getObjectWithSignedUrl(getUrl);
The Requester Pays feature can be used for a shared bucket, and allows to pay the usage of the shared bucket by an authenticated AWS user instead of the owner of the bucket.
So if you want to create a signed URL, then there is no any one can access that object via signed URL. How it can be identified by which AWS account it is accessed or what happens when it is accessed by no S3 user.
I think it is not possible.