Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS S3: How to check if a file exists in a bucket using bash

I'd like to know if it's possible to check if there are certain files in a certain bucket.

This is what I've found:

Checking if a file is in a S3 bucket using the s3cmd

It should fix my problem, but for some reason it keeps returning that the file doesn't exist, while it does. This solution is also a little dated and doesn't use the doesObjectExist method.

Summary of all the methods that can be used in the Amazon S3 web service

This gives the syntax of how to use this method, but I can't seem to make it work.

Do they expect you to make a boolean variable to save the status of the method, or does the function directly give you an output / throw an error?

This is the code I'm currently using in my bash script:

existBool=doesObjectExist(${BucketName}, backup_${DomainName}_${CurrentDate}.zip)  if $existBool ; then         echo 'No worries, the file exists.' fi 

I tested it using only the name of the file, instead of giving the full path. But since the error I'm getting is a syntax error, I'm probably just using it wrong.

Hopefully someone can help me out and tell me what I'm doing wrong.

!Edit

I ended up looking for another way to do this since using doesObjectExist isn't the fastest or easiest.

like image 647
J. Swaelen Avatar asked Jan 26 '17 10:01

J. Swaelen


People also ask

How do I view contents of AWS S3 bucket?

To open the overview pane for an objectSign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . In the Buckets list, choose the name of the bucket that contains the object. In the Objects list, choose the name of the object for which you want an overview.

Can you query data from S3 bucket?

Amazon S3 Select and Amazon S3 Glacier Select enable customers to run structured query language SQL queries directly on data stored in S3 and Amazon S3 Glacier. With S3 Select, you simply store your data on S3 and query using SQL statements to filter the contents of S3 objects, retrieving only the data that you need.

How can I tell if S3 object is file or directory?

With the method "endsWith("/")" you can detect if the S3ObjectSummary is a folder or not. Hope this can helps someone. works like wonders!


1 Answers

Last time I saw performance comparisons getObjectMetadata was the fastest way to check if an object exists. Using the AWS cli that would be the head-object method, example:

aws s3api head-object --bucket www.codeengine.com --key index.html 

which returns:

{     "AcceptRanges": "bytes",     "ContentType": "text/html; charset=utf-8",     "LastModified": "Sun, 08 Jan 2017 22:49:19 GMT",     "ContentLength": 38106,     "ContentEncoding": "gzip",     "ETag": "\"bda80810592763dcaa8627d44c2bf8bb\"",     "StorageClass": "REDUCED_REDUNDANCY",     "CacheControl": "no-cache, no-store",     "Metadata": {} } 
like image 138
Dave Maple Avatar answered Sep 20 '22 07:09

Dave Maple