Escaped Double Quote Side Effect in Windows Command Prompt

Question

I have encountered a very odd problem with the Windows command interpreter. It occurs on both XP and Windows 7. My description below applies to a Perl script, but this problem applies to running any kind of program on the command line which takes command line parameters.

For a test I am using the Perl script check-params.pl which simply outputs what it saw as parameters -

use strict;
use warnings;
while (my $param = shift @ARGV) {
    print "Param: [$param]\n";
}

Thus if I run -

perl check-params.pl "a b|<>" c^|^<^>cc

Then the output is

[Param: a b|<>]
[Param: c|<>cc]

as expected. The special chars | < > work fine inside the double quotes, but when outside quotes you escape them with ^.

However when a double quote is added to the quoted param eg in

perl check-params.pl "a\" b|<>"

Then I get the error -

> was unexpected at this time.

But if the double quote comes after the special char | < or >, then it works OK.

You can easily fix this by escaping the special | < > char with ^ inside the quoted parameter. eg

perl check-params.pl "a\" b^|"

However not only does the escaped double quote affect the special chars | < > in the current parameter but it affects these special chars inside any subsequent parameters.

eg if I do this -

perl check-params.pl "aa \"bb" ccc | perl check-pipe.pl

(where check-pipe.pl simply outputs what the pipe received -

use strict;
use warnings;
while (<STDIN>) {
    print "PIPE RECEIVED ---> $_";
}

)

then output is -

Param: [aa "bb]
Param: [ccc]
Param: [|]
Param: [perl]
Param: [check-pipe.pl]

ie it treats the pipe | as a literal character, and no piping occurs.

Has anybody any experience of this problem, and know of any workaround ?

I have written some Perl scripts to process web log files, and one uses regex's, which I pass inside a quoted param on the command line. The regex may contain characters such as | < >, and it may have a double quote also, which I enter as \". Thus the above problem is cropping up.

Any help much appreciated, thanks.

---

Thanks for reply, but on trying this on XP SP3, Perl script is seeing two command line params.

This problem is not related to Perl, eg

echo "a \"b|c"

produces error -

'c"' is not recognized as an internal or external command,
operable program or batch file.

but

echo "a b|\"c"

works, because the \" comes after the special char |.

It must be a bug in the command interpreter. I would be interested to know more about this, and find a workaround if possible. This problem messes up running some useful scripts. It is quite a glaring problem for a command interpreter. I tried on Windows 7 also, and same problem occurs there with the above echo test.

---

Additional Information:

I have updated the scripts check-params.pl and check-pipe.pl to give more useful information :-

check-params.pl :-

use strict;
use warnings;
while (my $param = shift @ARGV) {
    print "COMMAND LINE RECEIVED: [$param]\n";
}

check-pipe.pl :-

use strict;
use warnings;
while (<STDIN>) {
    print "PIPE RECEIVED: $_";
}

while (my $param = shift @ARGV) {
    print "COMMAND LINE RECEIVED: [$param]\n";
}

For example running :-

perl pl/utils/check-params.pl l1 l1-b l1-c | perl pl/utils/check-pipe.pl l2 l2-b | perl pl/utils/check-pipe.pl l3 l3-b | perl pl/utils/check-pipe.pl united arsenal rangers raith

produces output :-

PIPE RECEIVED: PIPE RECEIVED: PIPE RECEIVED: COMMAND LINE RECEIVED: [l1]
PIPE RECEIVED: PIPE RECEIVED: PIPE RECEIVED: COMMAND LINE RECEIVED: [l1-b]
PIPE RECEIVED: PIPE RECEIVED: PIPE RECEIVED: COMMAND LINE RECEIVED: [l1-c]
PIPE RECEIVED: PIPE RECEIVED: COMMAND LINE RECEIVED: [l2]
PIPE RECEIVED: PIPE RECEIVED: COMMAND LINE RECEIVED: [l2-b]
PIPE RECEIVED: COMMAND LINE RECEIVED: [l3]
PIPE RECEIVED: COMMAND LINE RECEIVED: [l3-b]
COMMAND LINE RECEIVED: [united]
COMMAND LINE RECEIVED: [arsenal]
COMMAND LINE RECEIVED: [rangers]
COMMAND LINE RECEIVED: [raith]

thus confirming that this command line will work as expected. The scripts will work with any number of pipes on the command line.

---

Additional Information:

I find I can get this to work using numerous workarounds - eg the ^" Harry mentioned is one example. Sometimes you can recraft the regex to avoid using the ". When there are two \" sequences before the special char | < > then the problem does not occur. Before running any commmand I run check-params.pl and check-pipe.pl to verify how the Perl scripts will see the command line parameters, eg the above edit to my OP describes these scripts, which I have modified since my OP.

Answer 1

The double quote in \" is considered to match the initial double quote, ending the quoted parameter. To include a double quote in a quoted parameter use two double quotes:

perl check-params.pl "a"" b|<>"

should produce the result you are expecting, depending on how perl.exe parses the command-line parameters (I've tested using command scripts as I don't have Perl installed). You'll need to change your script so that "" instead of \" is treated as a double quote.

Additional: this command line

echo "a \"b|c"

results in an error message starting c" is not recognized. This is not a bug.

A backslash is not an escape character, so the command line contains one quoted string ("a \") followed by a pipe (|) which makes the rest (c") an additional command, which the output from the echo command is piped into.

If you instead say

echo "a ""b|c"

then there is just one quoted string ("a ""b|c") and it will work as expected.

Unfortunately, I can now confirm that ActivePerl's command-line parsing causes

perl check-params.pl "a"" b|"

to be split into two parameters. You may need to fetch the entire command line (if there is a way to do that in Perl) and parse it yourself.

Additional:

perl check-params.pl ^"a\^" b^|c^"

produces the desired result. This works by escaping all the special characters, including the quote marks.