Bash arbitrary glob pattern (with spaces) in for loop

Question

Is there any way to reliably use an arbitrary globbing pattern that's stored in a variable? I'm having difficulty if the pattern contains both spaces and metacharacters. Here's what I mean. If I have a pattern stored in a variable without spaces, things seem to work just fine:

<prompt> touch aa.{1,2,3} "a b".{1,2,3}
<prompt> p="aa.?"
<prompt> for f in ${p} ; do echo "|$f|" ; done
|aa.1|
|aa.2|
|aa.3|
<prompt> declare -a A=($p) ; for f in "${A[@]}" ; do echo "|$f|" ; done
|aa.1|
|aa.2|
|aa.3|

However, as soon as I throw a space in the pattern, things become untenable:

<prompt> p="a b.?"
<prompt> for f in ${p} ; do echo "|$f|" ; done
|a|
|b.?|
<prompt> declare -a A=($p) ; for f in "${A[@]}" ; do echo "|$f|" ; done
|a|
|b.?|
<prompt> for f in "${p}" ; do echo "|$f|" ; done
|a b.?|
<prompt> for f in $(printf "%q" "$p") ; do echo "|$f|" ; done
|a\|
|b.\?|

Obviously, if I know the pattern in advance, I can manually escape it:

<prompt> for f in a\ b.* ; do echo "|$f|" ; done
|a b.1|
|a b.2|
|a b.3|

The problem is, I'm writing a script where I don't know the pattern in advance. Is there any way to reliably make bash treat the contents of a variable as a globbing pattern, without resorting to some sort of eval trickery?

Answer 1

You need to turn off word-splitting. To recap, this doesn't work:

$ p="a b.?"
$ for f in ${p} ; do echo "|$f|" ; done
|a|
|b.?|

This, however, does:

$ ( IFS=; for f in ${p} ; do echo "|$f|" ; done )
|a b.1|
|a b.2|
|a b.3|

IFS is the shell's "Internal Field Separator." It is normally set to a space, a tab, and a new line character. It is used for word splitting after variable expansion. Setting IFS to empty stops word splitting and, thereby, allows the glob to work.

Array example

The same applies to the array examples:

$ declare -a A=($p) ; for f in "${A[@]}" ; do echo "|$f|" ; done
|a|
|b.?|
$ ( IFS=; declare -a A=($p) ; for f in "${A[@]}" ; do echo "|$f|" ; done )
|a b.1|
|a b.2|
|a b.3|

Making sure that IFS gets returned to its normal value

In the examples above, I put the IFS assignment inside a subshell. Although not necessary, the advantage of that is that IFS returns automatically to its prior value as soon as the subshell terminates. If subshells are not appropriate for your application, here is another approach:

$ oldIFS=$IFS; IFS=; for f in ${p} ; do echo "|$f|" ; done; IFS=$oldIFS
|a b.1|
|a b.2|
|a b.3|

Matching patterns with shell-active characters

Suppose that we have files that have a literal * in their names:

$ touch ab.{1,2,3} 'a*b'.{1,2,3}
$ ls
a*b.1  ab.1  a*b.2  ab.2  a*b.3  ab.3

And, suppose that we want to match that star. Since we want the star to be treated literally, we must escape it:

$ p='a\*b.?'
$ ( IFS=; for f in ${p} ; do echo "|$f|" ; done )
|a*b.1|
|a*b.2|
|a*b.3|

Because the ? is not escaped, it is treated as a wildcard character. Because the * is escaped, it matches only a literal *.