How to copy files from subdirectories directly to a folder using aws-cli

Question

I have a number of files that are contained within subfolders of /myfolder.

I would like to use aws cli to move them to /newfolder.

For example:

/myfolder/2017_01_01/file1.txt
/myfolder/2017_01_02/file2.txt
...

I want to copy the files out to a new folder:

/newfolder/file1.txt
/newfolder/file2.txt
...

When I use the cp command like below:

aws s3 cp s3://myfolder/ s3://newfolder/ --recursive --exclude '*' --include '*file*'

The correct files are copied, but the problem is that the files remain in their subfolders at the new location:

/newfolder/2017_01_01/file1.txt
/newfolder/2017_01_02/file2.txt
...

Is it possible to copy the files "out" of their subfolders so that they are directly put into /newfolder?

Answer 1

The easiest way to do this is a hybrid solution assuming you have linux shell: (make sure you create blank folder in your local computer and cd to it before doing the below steps)

1.aws s3 cp s3://yourBucket/myfolder . --recursive  --include "*.txt"
2.find . -name '*txt' -exec mv {} .  \;
3.aws s3 cp ./ s3://yourBucket/newfolder --exclude "*" --include "*.txt" --recursive