Check if a folder exists on S3 using node js aws-sdk

Question

First of all, I know that there is no such thing as a folder, however, I am using the term just for simplicity. Now I have a lot of objects in my bucket. For some folders, I can use getObject to find if they exist or not, but for most of them, I get an error No such key when I clearly see that they are present. I would also like to say that I have tried headObject, listObjectV2 even but with no luck.

this is my params object
{
     Bucket: bucket //bucket is defined in the program
     Key: folder // defined in the program above
                 //example key - abc-1-1/00000N/30/2018.10.7.8/
                 //inside this folder are multiple files 
}

The goal is to find if the folder exists or not and based on that I do some processing. I saw a lot of answers to this question suggesting headObject, getObjects, etc. but none of them seem to work

This is my getObjects code snippet

params = {
            Bucket: bucket,
            Key: folder
        }
        s3.getObject(params, function (err, found) {

            if (err){ 
                console.log('bucket is'+bucket);
                ..........
                .....
            }
            else{
                ....
            }
            ..
            });

Answer 1

I finally found a workaround for this scenario. If your application requires you to check for the existence of a particular object("folder") and proceed further if the object("folder") actually exists, you can use this solution. I believe there is a problem with the way s3.getObject or s3.headObject work, anyway here is the solution.

1) use the s3.listObjectsV2 method.

2) check the Contents field of the response. If it is empty then the prefix you supplied does not exist. Here is the code snippet

s3.listObjectsV2(params, function (err, found) {

                if (err){ 
                    console.log(err);
                    errJson = {
                        'status': -1,
                        'message': 'Error while trying to list files'
                    };
                    callback(errJson);

                } 
                else {  

                    if (found.Contents.length === 0) {

                        errJson = {
                            'status': 0,
                            'message': 'Either the files are not present at s3 or the folder is incorrect '
                        }
                        callback(errJson);
                    }
                    else{
                               .....
                    }
....
});

Answer 2

In order to check whether a "folder" exists, check whether there's at least one object in the bucket whose key starts with the folder's name, plus a trailing slash.

This can be done using the listObjectV2 method, like so:

const s3 = ... // S3 client

const Bucket = 'mybucket';
const Prefix = 'myfolder/mysubfolder/';
const MaxKeys = 1; // If a single object is found, the folder exists.
const params = {
      Bucket,
      Prefix,
      MaxKeys
};

s3.listObjectsV2(params, (err, data) => {
  const folderExists = data.Contents.length > 0;

  // (folderExists == true) if the folder exists
});

Note: The above uses the definition of a folder to mean that there's at least one key with the folder as its prefix.

To detect "empty folders", maybe check for the folder + _$folder$ suffix, as mentioned here.