I have an S3 bucket that contains database backups. I am creating a script to download the latest backup (and eventually restore it somewhere else), but I'm not sure how to go about only grabbing the most recent file from a bucket.
Is it possible to copy only the most recent file from an S3 bucket to a local directory using AWS CLI tools?
You can use cp to copy the files from an s3 bucket to your local system. Use the following command: $ aws s3 cp s3://bucket/folder/file.txt .
Create bucket, enter the bucket name, choose region, copy settings from existing bucket. Create bucket. Once bucket created, go to the source bucket to which you want to copy the files from. Select all (if needed or else you can choose desired files and folders), Actions > Copy.
To download an entire bucket to your local file system, use the AWS CLI sync command, passing it the s3 bucket as a source and a directory on your file system as a destination, e.g. aws s3 sync s3://YOUR_BUCKET . . The sync command recursively copies the contents of the source to the destination.
And here is a bash script create based on @error2007s's answer. This script requires your aws profile and bucket name as variables, and downloads the latest object to your ~/Downloads folder:
#!/bin/sh PROFILE=your_profile BUCKET=your_bucket OBJECT="$(aws s3 ls --profile $PROFILE $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}')" aws s3 cp s3://$BUCKET/$OBJECT ~/Downloads/$OBJECT --profile $PROFILE
The above solutions are using Bash. If one wants to do the same thing in PowerShell for downloading on Windows here is the script:
# This assumes AWS CLI exe is in your path. $s3location = "s3://bucket-name" $files = $(aws s3 ls $s3location --recursive | sort | select -last 3) $dlPath = "C:\TEMP" foreach ($s3FileInfo in $files) { $filename = $s3FileInfo.Split()[-1] $path = "${s3location}/${filename}" aws s3 cp $path $dlPath echo("Done downloading ${path} to ${dlPath}") }
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With