Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multi-line, double quoted string triggers history expansion on subsequent single-quoted commands it gets piped to

I am on GNU bash, version 4.3.11.

Say I want to print unique lines on a file. I am using this approach, which works well on a file:

$ cat a
9
10
9
11
$ awk '!seen[$0]++' a
9
10
11

However, if I get the input from stdin, using double quotes in a multi-line and piping to awk, it fails:

$ echo "9
> 10
> 9
> 11" | awk '!seen[$0]++'
bash: !seen[$0]++': event not found

That is, bash tries to expand the command seen, which of course does not know because it is a variable name. But it shouldn't happen, since the command is placed within single-quotes.

echoing in a single-quoted, multi-line input works well:

$ echo '9
> 10
> 9
> 11' | awk '!seen[$0]++'
9
10
11

The funny thing is that it also works well on a single-line input that is double-quoted:

$ printf "9\n10\n9\n11" | awk '!seen[$0]++'
9
10
11

I wonder why is Bash trying to expand history if it occurs after a mutiline input, even though the command itself uses single-quotes.

Other considerations:

Having a pipe in between does not fix it, either:

$ echo "9
> 10
> 9
> 11" | cat - | awk '!seen[$0]++'
bash: !seen[$0]++': event not found

And setting set +H turns history off, so it works well because it does not try to expand anything:

$ set +H
$ echo "9
> 10
> 9
> 11" | awk '!seen[$0]++'
9
10
11

I went through the canonical answer by rici on how to address error “bash: !d': event not found” in Bash command substitution and found many possible reasons, but none matches this behaviour.

like image 931
fedorqui 'SO stop harming' Avatar asked Jun 14 '16 09:06

fedorqui 'SO stop harming'


1 Answers

It is not a bug.

After asking this in the bash-bugs mailing list, I got the following answer:

History expansion is explicitly line-oriented.

It doesn't know about shell state, especially shell quoting state, that spans lines.

It does know about vaguely shell-like quoting that's common across a large set of Unix utilities -- since the history and readline libraries are used outside the shell -- and that a double quote introduces a quoted string in which single quotes are not significant and don't inhibit history expansion.

like image 113
fedorqui 'SO stop harming' Avatar answered Sep 30 '22 02:09

fedorqui 'SO stop harming'