AWS find max file size in S3 bucket

Question

I want to find the size and name of the biggest file in my S3 bucket.

Currently I have:

aws s3api list-objects --bucket bucket-name --output json --query "[max(Contents[].Size), length(Contents[])]"

which does not allow me to see the name of the file.

I also have the command to list the details of all files on the bucket:

aws s3api list-object-versions --bucket bucket-name --query 'Versions[*].Size'

What command will give me the name and size of the largest file(s) on the S3 bucket?

Answer 1

Using AWS CLI only, this will find the largest file:

aws s3api list-objects-v2 --bucket bucket-name --query "sort_by(Contents, &Size)[-1:]"

or to include non-current versions if applicable:

aws s3api list-object-versions --bucket bucket-name --query "sort_by(Versions[*], &Size)[-1:]"

Optional tweaks:

• Replace -1 with -N to find the largest N files.
• Include .[Key,Size] at the end of the --query to select only those fields.

Sadly I think the filtering is done client side because this downloaded 28 MB when run on a large bucket. However it is still a useful 1-liner despite not being quick.

Answer 2

The following should return the name and size of the largest file in the bucket "bucket-name".

aws s3api list-object-versions --bucket bucket-name | jq -r '.Versions[] | "\(.Key)\t \(.Size)"' | sort -k2 -r -n | head -1

The command above uses jq which you can install from https://stedolan.github.io/jq/download/