Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to include and copy files that are in current directory to s3 (and not recursively)

I have some files that I want to copy to s3. Rather than doing one call per file, I want to include them all in one single call (to be as efficient as possible).

However, I only seem to get it to work if I add the --recursive flag, which makes it look in all children directories (all files I want are in the current directory only)

so this is the command I have now, that works

aws s3 cp --dryrun . mybucket --recursive --exclude * --include *.jpg

but ideally I would like to remove the --recursive to stop it traversing, e.g. something like this (which does not work)

aws s3 cp --dryrun . mybucket --exclude * --include *.jpg

(I have simplified the example, in my script I have several different include patterns)

like image 659
Susan D. Taylor Avatar asked Feb 11 '14 19:02

Susan D. Taylor


People also ask

How do I transfer local files to aws S3?

You have two options for uploading files: AWS Management Console: Use drag-and-drop to upload files and folders to a bucket. AWS CLI: With the version of the tool installed on your local machine, use the command line to upload files and folders to the bucket.

What is the command to copy file recursively in a folder to an S3 bucket?

“What is the command to copy files recursively in a folder to an S3 bucket? aws s3 cp filename S3://bucketname –-recursive aws s3 cp . S3://bucketname --recursive” Code Answer.

How can you download an S3 bucket including all folders and files?

aws s3 sync s3://mybucket . will download all the objects in mybucket to the current directory. This will download all of your files using a one-way sync. It will not delete any existing files in your current directory unless you specify --delete , and it won't change or delete any files on S3.


2 Answers

AWS CLI's S3 wildcard support is a bit primitive, but you could use multiple --exclude options to accomplish this. Note: the order of includes and excludes is important.

aws s3 cp --dryrun . s3://mybucket --recursive --exclude "*" --include "*.jpg" --exclude "*/*" 
like image 162
mfisherca Avatar answered Sep 21 '22 15:09

mfisherca


Try the command:

aws s3 cp --dryrun . s3://mybucket --recursive --exclude "*/" 

Hope it help.

like image 37
M. Dellwo Avatar answered Sep 17 '22 15:09

M. Dellwo