Why does the bash "test -n" command give the wrong result for the $@ (dollar at) positional parameter while "! test -z" works?

Question

Using $@ inside a bash test expression gives odd results.

Here's a minimal test script to reproduce the issue (among the alternative formulations for the same test which do pass without error):

#! /bin/bash
# file path: ./bash_weirdness.sh

echo "args: '$@'"

echo "--- empty ---"

if test "" ; then echo      "1.A1.TEST      - not empty?"; fi
if test -z "" ; then echo   "1.A2.EMPTY     - empty?"; fi
if test -n "" ; then echo   "1.A3.NE        - not empty?"; fi
if ! test "" ; then echo    "1.B1.NOT.TEST  - empty?"; fi
if ! test -z "" ; then echo "1.B2.NOT.EMPTY - not empty?"; fi
if ! test -n "" ; then echo "1.B3.NOT.NE    - empty?"; fi

echo "--- space ---"

if test " " ; then echo      "2.A1.TEST      - not empty?"; fi
if test -z " " ; then echo   "2.A2.EMPTY     - empty?"; fi
if test -n " " ; then echo   "2.A3.NE        - not empty?"; fi
if ! test " " ; then echo    "2.B1.NOT.TEST  - empty?"; fi
if ! test -z " " ; then echo "2.B2.NOT.EMPTY - not empty?"; fi
if ! test -n " " ; then echo "2.B3.NOT.NE    - empty?"; fi

echo "--- \$@ ---"

if test "$@" ; then echo      "3.A1.TEST      - not empty?"; fi
if test -z "$@" ; then echo   "3.A2.EMPTY     - empty?"; fi
if test -n "$@" ; then echo   "3.A3.NE        - not empty?"; fi
if ! test "$@" ; then echo    "3.B1.NOT.TEST  - empty?"; fi
if ! test -z "$@" ; then echo "3.B2.NOT.EMPTY - not empty?"; fi
if ! test -n "$@" ; then echo "3.B3.NOT.NE    - empty?"; fi

echo "--- \$* ---"

if test "$*" ; then echo      "4.A1.TEST      - not empty?"; fi
if test -z "$*" ; then echo   "4.A2.EMPTY     - empty?"; fi
if test -n "$*" ; then echo   "4.A3.NE        - not empty?"; fi
if ! test "$*" ; then echo    "4.B1.NOT.TEST  - empty?"; fi
if ! test -z "$*" ; then echo "4.B2.NOT.EMPTY - not empty?"; fi
if ! test -n "$*" ; then echo "4.B3.NOT.NE    - empty?"; fi

The first two sections ('empty' and 'space') are in there to verify that test, test -n and test -z work fine; that's how bad the confusion is.

The problem occurs when $@ represents 'no arguments', i.e. when $# would be zero(0).

Three test runs show what's wrong: notice the '3.A3.NE - not empty?' report in the third run:

$ ./bash_weirdness.sh x
->
args: 'x'
--- empty ---
1.A2.EMPTY     - empty?
1.B1.NOT.TEST  - empty?
1.B3.NOT.NE    - empty?
--- space ---
2.A1.TEST      - not empty?
2.A3.NE        - not empty?
2.B2.NOT.EMPTY - not empty?
--- $@ ---
3.A1.TEST      - not empty?
3.A3.NE        - not empty?
3.B2.NOT.EMPTY - not empty?
--- $* ---
4.A1.TEST      - not empty?
4.A3.NE        - not empty?
4.B2.NOT.EMPTY - not empty?

Which is fine.

$ ./bash_weirdness.sh ""
->
args: ''
--- empty ---
1.A2.EMPTY     - empty?
1.B1.NOT.TEST  - empty?
1.B3.NOT.NE    - empty?
--- space ---
2.A1.TEST      - not empty?
2.A3.NE        - not empty?
2.B2.NOT.EMPTY - not empty?
--- $@ ---
3.A2.EMPTY     - empty?
3.B1.NOT.TEST  - empty?
3.B3.NOT.NE    - empty?
--- $* ---
4.A2.EMPTY     - empty?
4.B1.NOT.TEST  - empty?
4.B3.NOT.NE    - empty?

Which is fine too.

$ ./bash_weirdness.sh 
->
args: ''
--- empty ---
1.A2.EMPTY     - empty?
1.B1.NOT.TEST  - empty?
1.B3.NOT.NE    - empty?
--- space ---
2.A1.TEST      - not empty?
2.A3.NE        - not empty?
2.B2.NOT.EMPTY - not empty?
--- $@ ---
3.A2.EMPTY     - empty?
3.A3.NE        - not empty?
3.B1.NOT.TEST  - empty?
--- $* ---
4.A2.EMPTY     - empty?
4.B1.NOT.TEST  - empty?
4.B3.NOT.NE    - empty?

Which is not fine at all: notice the incorrect '3.A3.NE' result.

Answer 1

"$@" and test don't mix

test -n "$@"

This test isn't properly written. The -n test expects a single argument and tells you if that argument is an empty string or not. "$@" will expand into one word, multiple words, or even no words at all. It is therefore inappropriate to write test -n "$@".

If you want to check that arguments were passed, use one of these:

test $# -gt 0
[[ $# -gt 0 ]]
(($# > 0))
(($#))

If you want to check that arguments were passed and also are non-empty, then use "$*" instead. $* always expands to a single string, so it's compatible with test -n.

test -n "$*"
[[ -n $* ]]

The confusing case of test -n

$ test -n; echo $?
0

So why the heck does this test pass? Well, if you don't pass an argument to -n then bash doesn't see the -n as an operator. Instead, it understands the test as a form of

test STRING

If you write test STRING without an operator, it's implicitly equivalent to test -n STRING. test STRING is a shorthand way of testing if STRING is non-empty.

In other words, test -n is equivalent to

test -n '-n'

And this test passes since '-n' is a non-empty string.

"$@" is awesome

By the way, the way "$@" behaves is in fact a great thing. "$@" is the best way to access the full argument list while handling whitespace correctly and avoiding issues with word splitting and globbing. When in doubt, use "$@".

And if you want to store "$@" in a variable, use an array.

args=("$@")

for arg in "${args[@]}"; do
    echo "$arg"
done

That will preserve the multi-wordedness of "$@".

Answer 2

While transforming this confusing observed behaviour into a test script and question that might be suitable for StackOVerflow, several things happened which led to the answer to this 'why' question.

The TL;DR answer is in section 5 below; I describe the entire process of discovery because I (re)learned quite a few things along the way and I think others will follow the same, if not a very similar, path of discovery. Google is not always your friend!

1. total Google Fu fail

First, 'google' didn't deliver anything when you search for bash test $@ odd result or similar queries as it strips out the all-important $@. Rephrasing $@ as dollar at didn't spit out anything useful either, which got me rather worried. I even started to suspect the bash I was running (mSysGit bash on Windows).

Thanks to this debacle I went and looked up what the official name (jargon) for this $@ might be (It's maybe 2 decades ago that I last read a bash manual; I've been dabbling in it ever since and this is the moment where I receive punishment for my laziness in not studying up on a language that I write (small bits of) code in.)

2. The name is: 'positional parameters'. $@ (and $*)

Another round of searching led to http://www.tldp.org/LDP/abs/abs-guide.pdf which told me that $@ (and $* - oh, right, had forgotten all about you!) are 'positional parameters'.

3. what is the exact behaviour of test?

During this search frenzy I also found http://wiki.bash-hackers.org/commands/classictest#number_of_arguments_rules which explains how test behaves when the number of arguments is maybe not what you'ld expect.

But then I did have quotes around that $@, didn't I? So it must be a single argument for test then, even when the argument list is empty!? ($# = 0)

(Wrong! It isn't!)

The test expression evaluation rule set

Copying (with minimal edit to match it to this question) the very important section from the link above here -- this is what made my brain click a get a hunch, then a full understanding:

http://wiki.bash-hackers.org/commands/classictest#number_of_arguments_rules:

Number of Arguments Rules

The test builtin, especially hidden under its [ name, may seem simple but is in fact causing a lot of trouble sometimes. One of the difficulties is that the behaviour of test not only depends on its arguments but also on the number of its arguments.

Here are the rules taken from the manual (Note: This is for the command test, for [ the number of arguments is calculated without the final ], for example [ ] follows the "zero arguments" rule):

• 0 arguments

  The expression is false.

• 1 argument

  The expression is true if, and only if, the argument is not null.

• 2 arguments

  If the first argument is ! (exclamation mark), the expression is true if, and only if, the second argument is null.

  If the first argument is one of the unary conditional operators listed above under the syntax rules (e.g. -n and -z), the expression is true if the unary test is true.

  If the first argument is not a valid unary conditional operator, the expression is false.

• 3 arguments

  If the second argument is one of the binary conditional operators listed above under the syntax rules, the result of the expression is the result of the binary test using the first and third arguments as operands.

  If the first argument is !, the value is the negation of the two-argument test using the second and third arguments.

  If the first argument is exactly ( and the third argument is exactly ), the result is the one-argument test of the second argument. Otherwise, the expression is false. The -a and -o operators are considered binary operators in this case (Attention: This means the operator -a is not a file operator in this case!)

• 4 arguments

  If the first argument is !, the result is the negation of the three-argument expression composed of the remaining arguments. Otherwise, the expression is parsed and evaluated according to precedence using the rules listed above.

• 5 or more arguments

  The expression is parsed and evaluated according to precedence using the rules listed above.

These rules may seem complex, but it's not so bad in practice. Knowing them might help you to explain some of the "unexplicable" behaviours you might encounter:

(note: the next section is paraphrasing the original to match this question!)

 function test {
   if [ -n "$@" ] ; then echo "argument list is not empty"; fi
 }

 test

This code prints "argument list is not empty", even though -n something is supposed to be false if something is an empty string "" - why?

Here, "$@" expands to an empty argument list, i.e. "$@" results in actually nothing (Bash removes it from the command's argument list!). So the test is in fact [ -n ] and falls into the "one argument" rule, the only argument is "-n" which is not null and so the test returns true. Hence the usual solution, which is to quote the parameter expansion, e.g. [ -n "$var" ] so that the test has always 2 arguments, even if the second one is the null string, does not work for "$@".

These rules also explain why, for instance, -a and -o can have several meanings.

4. One more test run...

The description in http://www.tldp.org/LDP/abs/abs-guide.pdf and the non-obvious bits of behaviour of test as described in http://wiki.bash-hackers.org/commands/classictest#number_of_arguments_rules drove me to run a fourth test with the test script specified in the question:

$ ./bash_weirdness.sh x y z
->
args: 'x y z'
--- empty ---
1.A2.EMPTY     - empty?
1.B1.NOT.TEST  - empty?
1.B3.NOT.NE    - empty?
--- space ---
2.A1.TEST      - not empty?
2.A3.NE        - not empty?
2.B2.NOT.EMPTY - not empty?
--- $@ ---
util/bash_weirdness.sh: line 25: test: y: binary operator expected
util/bash_weirdness.sh: line 26: test: too many arguments
util/bash_weirdness.sh: line 27: test: too many arguments
util/bash_weirdness.sh: line 28: test: y: binary operator expected
3.B1.NOT.TEST  - empty?
util/bash_weirdness.sh: line 29: test: too many arguments
3.B2.NOT.EMPTY - not empty?
util/bash_weirdness.sh: line 30: test: too many arguments
3.B3.NOT.NE    - empty?
--- $* ---
4.A1.TEST      - not empty?
4.A3.NE        - not empty?
4.B2.NOT.EMPTY - not empty?

Now there's a hint!

5. Answering the 'why?' question

Now I know.

$@, even when quoted as "$@", 'somehow' is transformed to the exact number of arguments available in $@, which for an empty argument list ($# = 0) means that "$@" represents exactly nothing: test -n "$@" therefor 'expands' to test -n which, following the rules described in http://wiki.bash-hackers.org/commands/classictest#number_of_arguments_rules, is almost identical to the example described there and follows the same '1 argument rule': the -n in test -n is therefor not the test option: return true if string is non-empty but rather that 'one argument', not null and consequently test -n "$@" --> test -n --> test "-n" --> TRUE which produces the unexpected erroneous test result shown in the third test run in the question.

The 'somehow' is due to what bash must do internally to guarantee that the number of parameters in "$@" always equals the number of rounds in loop statements such as for f in "$@" ; do ... done and always matches the number of parameters $#.

If the processing of "$@" for an empty parameter list would produce an empty string "" rather than nothing at all, then such loop statements would execute one(1) round instead of the anticipated zero(0) rounds, which would be extremely counter-intuitive.

Once you've realized this, the description in http://www.tldp.org/LDP/abs/abs-guide.pdf for $@ versus $* is obvious.

6. Deja Vu. Now that I know, SO suddenly delivers comparable questions (and answers)

This felt exactly like the old days when I was reading UNIX man pages after having worked with VMS: those 'man pages' were very concise and were more times than not only useful to me once I had already obtained the knowledge through other channels.

[Edit] Retrospection Note:

I say "wouldn't have helped me" for many SO questions listed below only because at the time when I was writing this answer I reflected and wondered if my question really was a duplicate question after all. These other questions, and the answers provided there, are all very valuable in their own right.

The "wouldn't have helped me" phrase is there so you may understand that my brain needed something different to resolve my confusion and loss of trust in my machines. It is saying "I needed exactly this rather than one of those" and thus shows my internal argument whether this question of mine is a duplicate or not.

My own conclusion now: one question technically is almost the same (apart from the ! in ! test -z), while its answers came very close to what I needed, but, to me at least, this isn't a duplicate answer as it pays much more attention to the depth of the details of the why of test and $@ interaction. The precise question context required for this answer to be suitable makes this, at least to me, in retrospect, not a duplicate question. Though it comes close to being one.

Here they are:

• A case in which both test -n and test -z are true

  my Google Fu failed me; this one would very probably have answered my question straight away.

  It lacks one important bit of detail which is the bit that made my brain fire up and initiate understanding: http://wiki.bash-hackers.org/commands/classictest#number_of_arguments_rules

• Difference between $@ and $* in bash scripting

  Same subject; wouldn't have helped because it doesn't address the 'empty argument list' explicitly.

  The duplicate flag leads to:

• Parsing/passing command line arguments to a bash script - what is the difference between "$@" and "$*"?

  Also wouldn't have worked for me. Unless perhaps if I'ld run the scripts listed in the answer there. The crux is that "$@" eats the quotes and produces exactly nothing when $# = 0. Which is not obvious from "$1" ....

• What is the difference between "$@" and "$*" in Bash?

  Wouldn't have helped me; the more elaborate description of $@ given here doesn't mention the effect you get when you have zero(0) parameters ($# = 0); all consequences of the phase "starting with one" are clear only when you already know this peculiarity.

• Idiomatic way to test if no positional params are given?

  Which would have circumvented my problem due to too little active knowledge of the bash language: $# didn't belong to my 'active vocabulary` until about half an hour ago.