node.jsamazon-web-servicesamazon-s3aws-sdk

Check if a folder exists on S3 using node js aws-sdk


First of all, I know that there is no such thing as a folder, however, I am using the term just for simplicity. Now I have a lot of objects in my bucket. For some folders, I can use getObject to find if they exist or not, but for most of them, I get an error No such key when I clearly see that they are present. I would also like to say that I have tried headObject, listObjectV2 even but with no luck.

this is my params object
{
     Bucket: bucket //bucket is defined in the program
     Key: folder // defined in the program above
                 //example key - abc-1-1/00000N/30/2018.10.7.8/
                 //inside this folder are multiple files 
}

The goal is to find if the folder exists or not and based on that I do some processing. I saw a lot of answers to this question suggesting headObject, getObjects, etc. but none of them seem to work

This is my getObjects code snippet

params = {
            Bucket: bucket,
            Key: folder
        }
        s3.getObject(params, function (err, found) {

            if (err){ 
                console.log('bucket is'+bucket);
                ..........
                .....
            }
            else{
                ....
            }
            ..
            });

Solution

  • I finally found a workaround for this scenario. If your application requires you to check for the existence of a particular object("folder") and proceed further if the object("folder") actually exists, you can use this solution. I believe there is a problem with the way s3.getObject or s3.headObject work, anyway here is the solution.

    1) use the s3.listObjectsV2 method.

    2) check the Contents field of the response. If it is empty then the prefix you supplied does not exist. Here is the code snippet

    s3.listObjectsV2(params, function (err, found) {
    
                    if (err){ 
                        console.log(err);
                        errJson = {
                            'status': -1,
                            'message': 'Error while trying to list files'
                        };
                        callback(errJson);
    
                    } 
                    else {  
    
                        if (found.Contents.length === 0) {
    
                            errJson = {
                                'status': 0,
                                'message': 'Either the files are not present at s3 or the folder is incorrect '
                            }
                            callback(errJson);
                        }
                        else{
                                   .....
                        }
    ....
    });