amazon-web-servicesamazon-s3aws-sdk-js

ListObjectsV2 - Get only folders in an S3 bucket


I am using AWS S3 JS SDK. I have folders within folders in my S3 bucket and I would like to list only folders at a certain level.

This is the structure:

bucket/folder1/folder2/folder3a/file1
bucket/folder1/folder2/folder3a/file2
bucket/folder1/folder2/folder3a/file3
bucket/folder1/folder2/folder3a/...

bucket/folder1/folder2/folder3b/file1
bucket/folder1/folder2/folder3b/file2
bucket/folder1/folder2/folder3b/file3
bucket/folder1/folder2/folder3b/...

bucket/folder1/folder2/folder3c/file1
bucket/folder1/folder2/folder3c/file2
bucket/folder1/folder2/folder3c/file3
bucket/folder1/folder2/folder3c/...

As you can see, at the level of folder 3, I have multiple folders and each of those folders contain multiple items. I don't care about the items. I just want to list the folder names at level 3. Is there a good way to do this?

The only way I found is to use ListObjectsV2. But this gives me also the files which inflates the results set and I would need to do a manual filtering afterwards. Is there a way to get just the folder names at the API level?


Solution

  • This article answers all my questions. https://realguess.net/2014/05/24/amazon-s3-delimiter-and-prefix/

    The solution can be done using the combination of prefix and delimiter. In my examples the parameters should contain the following:

    const params = {
          Bucket: 'bucket',
          Prefix: 'folder1/folder2/',
          Delimiter: '/',
        };
    

    Be sure to not forget the slash at the end of the Prefix parameter.

    The list of folders will be in the CommonPrefixes attribute of the response object.

    To give you a real life example:

    ...
    const params = {
          Bucket: bucketName,
          Prefix: prefix + '/',
          MaxKeys: 25,
          Delimiter: '/',
        };
    const command = new ListObjectsV2Command(params);
    const results = await s3client.send(command);
    const foldersList = results.CommonPrefixes;
    ...