Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS CLI "s3 ls" command to list a date range of files in a virtual folder

I'm trying to list files from a virtual folder in S3 within a specific date range. For example: all the files that have been uploaded for the month of February.

I currently run a aws s3 ls command but that gives all the files:

aws s3 ls s3://Bucket/VirtualFolder/VirtualFolder --recursive --human-readable --summarize > c:File.txt

How can I get it to list only the files within a given date range?

like image 207
TheDude Avatar asked Mar 30 '16 18:03

TheDude


People also ask

How do I see how many files are in a S3 bucket?

Open the AWS S3 console and click on your bucket's name. In the Objects tab, click the top row checkbox to select all files and folders or select the folders you want to count the files for. Click on the Actions button and select Calculate total size.

How do I view contents of a S3 bucket?

Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . In the Buckets list, choose the name of the bucket that contains the object. In the Objects list, choose the name of the object for which you want an overview. The object overview opens.

Which command is used in CLI to upload a group of files on S3 bucket?

The s3 cp command uses the following syntax to upload a file stream from stdin to a specified bucket. The s3 cp command uses the following syntax to download an Amazon S3 file stream for stdout . For a few common options to use with this command, and examples, see Frequently used options for s3 commands.


Video Answer


1 Answers

You could filter the results with a tool like awk:

aws s3 ls s3://Bucket/VirtualFolder/VirtualFolder --recursive --human-readable --summarize \
| awk -F'[-: ]' '$1 >= 2016 && $2 >= 3 { print }'

Where awk splits each records using -, :, and space delimiters so you can address fields as:

  • $1 - year
  • $2 - month
  • $3 - day
  • $4 - hour
  • $5 - minute
  • $6 - second
like image 110
James Avatar answered Sep 25 '22 22:09

James