Why does $@ work different from most other variables in bash?

Question

The $@ variable seems to maintain quoting around its arguments so that, for example:

$ function foo { for i in "$@"; do echo $i; done }
$ foo herp "hello world" derp
herp
hello world
derp

I am also aware that bash arrays, work the same way:

$ a=(herp "hello world" derp)
$ for i in "${a[@]}"; do echo $i; done
herp
hello world
derp

What is actually going on with variables like this? Particularly when I add something to the quote like "duck ${a[@]} goose". If its not space separated what is it?

Answer 1

Usually, double quotation marks in Bash mean "make everything between the quotation marks one word, even if it has separators in it." But as you've noticed, $@ behaves differently when it's within double quotes. This is actually a parsing hack that dates back to Bash's predecessor, the Bourne shell, and this special behavior applies only to this particular variable.

Without this hack (I use the term because it seems inconsistent from a language perspective, although it's very useful), it would be difficult for a shell script to pass along its array of arguments to some other command that wants the same arguments. Some of those arguments might have spaces in them, but how would it pass them to another command without the shell either lumping them together as one big word or reparsing the list and splitting the arguments that have whitespace?

Well, you could pass an array of arguments, and the Bourne shell really only has one array, represented by $* or $@, whose number of elements is $# and whose elements are $1, $2, etc, the so-called positional parameters.

An example. Suppose you have three files in the current directory, named aaa, bbb, and cc c (the third file has a space in the name). You can initialize the array (that is, you can set the positional parameters) to be the names of the files in the current directory like this:

set -- *

Now the array of positional parameters holds the names of the files. $#, the number of elements, is three:

$ echo $#
3

And we can iterate over the position parameters in a few different ways.

---

1) We can use $*:

$ for file in $*; do
>    echo "$file"
> done

but that re-separates the arguments on whitespace and calls echo four times:

aaa
bbb
cc
c

---

2) Or we could put quotation marks around $*:

$ for file in "$*"; do
>    echo "$file"
> done

but that groups the whole array into one argument and calls echo just once:

aaa bbb cc c

---

3) Or we could use $@ which represents the same array but behaves differently in double quotes:

$ for file in "$@"; do
>     echo "$file"
> done

will produce

aaa
bbb
cc c

because $1 = "aaa", $2 = "bbb", and $3 = "cc c" and "$@" leaves the elements intact. If you leave off the quotation marks around $@, the shell will flatten and re-parse the array, echo will be called four times, and you'll get the same thing you got with a bare $*.

---

This is especially useful in a shell script, where the positional parameters are the arguments that were passed to your script. To pass those same arguments to some other command -- without the shell resplitting them on whitespace -- use "$@".

# Truncate the files specified by the args
rm "$@"
touch "$@"

In Bourne, this behavior only applies to the positional parameters because it's really the only array supported by the language. But you can create other arrays in Bash, and you can even apply the old parsing hack to those arrays using the special "${ARRAYNAME[@]}" syntax, whose at-sign feels almost like a wink to Mr. Bourne:

$ declare -a myarray
$ myarray[0]=alpha
$ myarray[1]=bravo
$ myarray[2]="char lie"
$ for file in "${myarray[@]}"; do echo "$file"; done
alpha
bravo
char lie

Oh, and about your last example, what should the shell do with "pre $@ post" where you have $@ within double quotes but you have other stuff in there, too? Recent versions of Bash preserve the array, prepend the text before the $@ to the first array element, and append the text after the $@ to the last element:

pre aaa
bb
cc c post