Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS CLI for searching a file in s3 bucket

Tags:

aws-cli

I want to search for a file name abc.zip in s3 buckets and there are nearly 60 buckets and each buckets have 2 to 3 levels subdirectories or folders .I tried to perform search using AWS CLI commands and below are the commands which i tried but even though the file is existing in the bucket.The results are not being displayed for the file.

aws s3api list-objects --bucket bucketname --region ca-central-1 \
    --recursive --query "Contents[?contains(Key, 'abc.zip')]"

aws s3 ls --summarize --human-readable --recursive bucketname \
    --region ca-central-1 | egrep 'abc.zip'

For all the above commands execution i dont see the filename in command line and when i manually check the bucket the file exists. Is there any way i can find the file.

like image 827
Aish Mahesh Avatar asked Aug 14 '18 20:08

Aish Mahesh


People also ask

How do I find files on AWS S3?

Perhaps you're looking for something more complex, but if you landed here trying to figure out how to simply find an object (file) by it's title, it's crazy simple: open the bucket, select "none" on the right hand side, and start typing in the file name.

How do I get files from AWS command line S3?

You can use cp to copy the files from an s3 bucket to your local system. Use the following command: $ aws s3 cp s3://bucket/folder/file.txt .

What CLI command will list all of the S3 Buckets you have access to?

To list your buckets, folders, or objects, use the s3 ls command. Using the command without a target or options lists all buckets.


2 Answers

I know this is ancient, but I found a way to do this without piping text to grep...

aws s3api list-objects-v2 --bucket myBucket --prefix 'myFolder' \
--query "Contents[*]|[?ends_with(Key,'jpg')].[Key]"

like image 52
Rowy Avatar answered Oct 26 '22 07:10

Rowy


Hmm. I used your command from #1 without "--recursive" because this throws Unknown options: --recursive. The file I was searching for is on the second level of the bucket and it was found. --region is also not used.

My guess is you are using some old version of AWS client or pointing to an incorrect bucket. My working command:

aws s3api list-objects --bucket XXXXX --query "Contents[?contains(Key, 'animate.css')]"

[
{
    "LastModified": "2015-06-14T23:29:03.000Z",
    "ETag": "\"e5612f9c5bc799b8b129e9200574dfd2\"",
    "StorageClass": "STANDARD",
    "Key": "css/animate.css",
    "Owner": {
        "DisplayName": "XXXX",
        "ID": "XXXX"
    },
    "Size": 78032
}
]

If you decide to upgrade your CLI client: https://github.com/aws/aws-cli/tree/master Current version is awscli-1.15.77 which you may check by aws --version.

like image 42
jacoor Avatar answered Oct 26 '22 05:10

jacoor