Where does the recursive variable expansion in bash/shell numeric contexts come from?

Question

The POSIX spec states with regard to Arithmetic Expansion that

[i]f the shell variable x contains a value that forms a valid integer constant, optionally including a leading plus or minus sign, then the arithmetic expansions "$((x))" and "$(($x))" shall return the same value.

Which is a reasonable shortcut and cleans up complicated expressions rather nicely.

bash (versions 3.2.25(1)-release from CentOS 5 and 4.3.33(1)-release from debian unstable) as well as ksh (Version AJM 93t+ 2010-06-21 from CentOS 5) all seem to go one step farther then that however.

They all seem to recursively expand variables encountered in arithmetic expansion (and numeric contexts in [[ resulting from using the numeric operators).

Specifically:

$ set -x
$ bar=5
+ bar=5
$ foo=bar
+ foo=bar
$ [ foo -gt 4 ]; echo $?
+ '[' foo -gt 4 ']'
-bash: [: foo: integer expression expected
+ echo 2
2
$ [[ foo -gt 4 ]]; echo $?
+ [[ foo -gt 4 ]]
+ echo 0
0
$ [[ foo -eq 0 ]]; echo $?
+ [[ foo -eq 0 ]]
+ echo 1
1
$ [[ foo -eq 5 ]]; echo $?
+ [[ foo -eq 5 ]]
+ echo 0
0
$ (( foo == bar )); echo $?
+ ((  foo == bar  ))
+ echo 0
0
$ (( foo == 1 )); echo $?
+ ((  foo == 1  ))
+ echo 1
1

Where does this behavior come from and why would it ever be desirable?

It makes using [[ in place of [ explicitly less safe when used with numeric operators because invalid values and typographical errors go from being script errors to being silently valid (but likely erroneous) tests.

Edit: As a side question if anyone happens to know when this "feature" was added to bash/etc. I would be interested in knowing that as well.

Answer 1

It's worse than you think. The value of the variable is recursively treated as an arithmetic expression:

$ foo='bar+bar'
$ echo $((foo))
10

The bash manual section on Shell Arithmetic says:

The value of a variable is evaluated as an arithmetic expression when it is referenced, or when a variable which has been given the integer attribute using ‘declare -i’ is assigned a value.

The latter part means you can do:

$ declare -i int
$ int=bar+bar
$ echo $int
10

Note that none of this is a violation of the spec you quoted above. It only says what should be done if the value of the variable is an integer constant. It doesn't say what should be done if the value is something else, which leaves that open for implementations to add extensions like this. Bash Hackers Wiki explains it:

If the variable doesn't hold a value that looks like a valid expression (numbers or operations), the expression is re-used to reference, for example, the named parameters

If the eventual expansion is not a valid expression, you'll get an error:

$ foo='a b'
$ bar=foo
echo $((bar))
bash: a b: syntax error in expression (error token is "b")

So if your variable contains random stuff, it's likely to cause an error. But if it just contains a single word, which is valid variable syntax, that will evaluate as 0 if the variable isn't set.