BASH - FOR loop using LS and wildcard

Question

My directory contains the following files:

FILE1_h0_something_1950_other.gz
FILE2_h0_something_1950_other.gz
FILE3_h0_something_1950_other.gz

Here a portion of my bash script:

year=1950
for nc_gz in "$input_path/*h3*$year*.gz"; do
      gunzip $nc_gz
done

I noted that SOMETIMES (strange behavior... I except always or never) a file named h31950* is created inside my dir.

The code fails when directory contains no gz files.

Where is the issue?

Answer 1

for nc_gz in "$input_path/*h3*$year*.gz"; do

You're iterating over a list of one element. Since "$input_path/*h3*$year*.gz" is in double quotes, it's a single string. So if the value of input_path is /some/where and the value of year is 2017 then this runs the loop body once, with nc_gz set to /some/where/*h3*2017*.gz.

  gunzip $nc_gz

Here $nc_gz is unquoted, so this applies the split+glob operator to the result. In the example above, /some/where/*h3*2017*.gz is expanded to the list of matching files; except if there are no matching files, then this invokes gunzip with the argument /some/where/*h3*2017*.gz.

First, quote things properly:

• Use double quotes around variable substitutions.
• Don't use double quotes around wildcards.

for nc_gz in "$input_path/"*h3*"$year"*.gz; do
      gunzip "$nc_gz"
done

This still runs the loop once if the wildcard does not match any files, because of a shell misfeature that non-matching wildcard patterns are left unchanged instead of being expanded to an empty list. In bash (but not in plain sh), you can avoid this by setting the nullglob option.

#!/bin/bash
shopt -s nullglob
for nc_gz in "$input_path/"*h3*"$year"*.gz; do
  gunzip "$nc_gz"
done

In plain sh, you can avoid this by skipping non-existent files.

for nc_gz in "$input_path/"*h3*"$year"*.gz; do
  if [ ! -e "$nc_gz" ]; then continue; fi
  gunzip "$nc_gz"
done

Note that I assumed that this was simplified code and you do need the loop to do other things beyond calling gunzip. If all you need to do is unzip the files, then you can call gunzip a single time, since it supports multiple file arguments.

gunzip "$input_path/"*h3*"$year"*.gz

This works in the sense of unzipping the files, and it does nothing if the wildcard pattern does not match any file, however gunzip will complain about the non-existent file. Here bash's nullglob option won't help you since calling gunzip with no file name also doesn't work (it expects to uncompress data from its standard input). You need to test for the number of matching files.

unzip_matching_files () {
  if [ $# -eq 1 ] && [ ! -e "$1" ]; then return; fi
  gunzip "$@"
}
unzip_matching_files "$input_path/"*h3*"$year"*.gz