I have a GCS bucket containing some files in the path
gs://main-bucket/sub-directory-bucket/object1.gz
I would like to programmatically check if the sub-directory bucket contains one specific file. I would like to do this using gsutil.
How could this be done?
You can use the gsutil stat command.
Use the gsutil stat command. For accessing the sub-directories with more number of files use wildcards(*).
For example:
gsutil -q stat gs://some-bucket/some-subdir/*; echo $?
In your case:
gsutil -q stat gs://main-bucket/sub-directory-bucket/*; echo $?
Result 0 means exists; 1 means not exists
If your script allows for non-zero exit codes, then:
#!/bin/bash
file_path=gs://main-bucket/sub-directory-bucket/object1.gz
gsutil -q stat $file_path
status=$?
if [[ $status == 0 ]]; then
echo "File exists"
else
echo "File does not exist"
fi
But if your script is set to fail on error, then you can't use exit codes. Here is an alternative solution:
#!/bin/bash
trap 'exit' ERR
file_path=gs://main-bucket/sub-directory-bucket/object1.gz
result=$(gsutil -q stat $file_path || echo 1)
if [[ $result != 1 ]]; then
echo "File exists"
else
echo "File does not exist"
fi
There is also gsutil ls
(https://cloud.google.com/storage/docs/gsutil/commands/ls)
e.g.
gsutil ls gs://my-bucket/foo.txt
Output is either that same filepath or "CommandException: One or more URLs matched no objects.
"
Simply using the ls command and counting the number of rows of the output.
If 0 then file not there, if 1 the file exists.
file_exists=$(gsutil ls gs://my_bucket/object1.gz | wc -l)
The same could be used for many files of course.
files_number=$(gsutil ls gs://my_bucket/object* | wc -l)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With