Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I tell how many objects I've stored in an S3 bucket?

People also ask

How do I count objects in S3 bucket?

Open the AWS S3 console and click on your bucket's name. In the Objects tab, click the top row checkbox to select all files and folders or select the folders you want to count the files for. Click on the Actions button and select Calculate total size.

How many objects are stored in S3?

Elastic scalability—S3 has no storage limit. Individual objects can be up to 5TB in size. Flexible data structure—each object is identified using a unique key, and you can use metadata to flexibly organize data.

How do I find the total size of my AWS S3 storage bucket or folder?

Get the Size of a Folder in S3 # Open the AWS S3 console and click on your bucket's name. Optionally use the search input to filter by folder name. Click on the checkbox next to your folder's name. Click on the Actions button and select Calculate total size.


Using AWS CLI

aws s3 ls s3://mybucket/ --recursive | wc -l 

or

aws cloudwatch get-metric-statistics \
  --namespace AWS/S3 --metric-name NumberOfObjects \
  --dimensions Name=BucketName,Value=BUCKETNAME \
              Name=StorageType,Value=AllStorageTypes \
  --start-time 2016-11-05T00:00 --end-time 2016-11-05T00:10 \
  --period 60 --statistic Average

Note: The above cloudwatch command seems to work for some while not for others. Discussed here: https://forums.aws.amazon.com/thread.jspa?threadID=217050

Using AWS Web Console

You can look at cloudwatch's metric section to get approx number of objects stored. enter image description here

I have approx 50 Million products and it took more than an hour to count using aws s3 ls


There is a --summarize switch which includes bucket summary information (i.e. number of objects, total size).

Here's the correct answer using AWS cli:

aws s3 ls s3://bucketName/path/ --recursive --summarize | grep "Total Objects:"

Total Objects: 194273

See the documentation


Although this is an old question, and feedback was provided in 2015, right now it's much simpler, as S3 Web Console has enabled a "Get Size" option:

enter image description here

Which provides the following:

enter image description here


There is an easy solution with the S3 API now (available in the AWS cli):

aws s3api list-objects --bucket BUCKETNAME --output json --query "[length(Contents[])]"

or for a specific folder:

aws s3api list-objects --bucket BUCKETNAME --prefix "folder/subfolder/" --output json --query "[length(Contents[])]"