Why am I getting different errors when trying to read s3 key that does not exist

Question

I have two environments. One, I am running from desktop, and one I am running from an aws lambda. Both of these are python. The code I have on my machine is:

import boto3
s3 = boto3.resource('s3')
bucket = 'my-bucket'
obj = s3.Object(bucket,'test.txt')
try:
    text = obj.get()['Body'].read().decode()
    print("file exists"))
except s3.meta.client.exceptions.NoSuchKey:
    print("key does not exist")

When I run this, everything works perfectly fine, as intended.

When I run this on the lambda, I get

An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

My lambda permissions are correct, so it can't be that, as everything works when the file exists. I am deleting the file each run so it has nothing to do with the order I run my test code. To debug, I did this:

except Exception as e:
    print(e)

In both my lambda code and local code, to see what the actual error is. In my local machine, the error is:

An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

When I run it on my lambda, it is

An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

Why am I getting different error messages, and how do I account for this without a lazy "except all Exceptions" solution? Thank you

This is the CFT for my lambda

                {
                  "Effect": "Allow",
                  "Action": [
                    "s3:PutObject",
                    "s3:GetObject",
                    "s3:ListBucket",
                  ],
                  "Resource": {
                    "Fn::Sub": "arn:aws:s3:::my-bucket/*"
                  }

                }

Answer 1

From: https://forums.aws.amazon.com/thread.jspa?threadID=56531

As you discovered, Amazon S3 will return an AccessDenied error when a nonexistent key is requested and the requester is not allowed to list the contents of the bucket. By the Amazon S3 definition, not being allowed to list the contents of a bucket means not being allowed to discover whether a particular key exists. Returning NoSuchKey would leak information about the nonexistence of the requested key. Instead, Amazon S3 returns AccessDenied. AccessDenied does not say anything about the existence or nonexistence of the requested key, so no information is leaked.

You're probably missing the permission to list the contents of the bucket (s3:listBucket) in lambda.

Ref: https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-buckets