perl shasum vs bash shasum

Question

The following bash and Perl scripts mysteriously give different results. Why?

#!/bin/bash                                                                       
hash=`echo -n 'abcd' | /usr/bin/shasum -a 256`;
echo $hash;

#!/usr/bin/perl                                                                   
$hash = `echo -n 'abcd' | /usr/bin/shasum -a 256`;
print "$hash";

The bash script:

$ ./tst.sh
88d4266fd4e6338d13b845fcf289579d209c897823b9217da3e161936f031589 -

The Perl script:

$ ./tst.pl
61799467ee1ab1f607764ab36c061f09cfac2f9c554e13f4c7442e66cbab9403  -

the heck?

Answer 1

Summary: In your Perl script, -n is being treated as an argument to include in the output of echo, not a flag to suppress the newline. ( Try

$hash = `echo -n 'abcd'`;

to confirm). Use printf instead.

---

Perl uses /bin/sh to execute code in back tics. Even if /bin/sh is a link to bash, it will behave differently when invoked via that like. In POSIX mode,

echo -n 'abcd'

will output

-n abcd

that is, the -n option is not recognized as a flag to suppress a newline, but is treated as a regular argument to print. Replace echo -n with printf in each script, and you should get the same SHA hash from each script.

(UPDATE: bash 3.2, when invoked as sh, displays this behavior. Newer versions of bash seem to continue treating -n as a flag when invoked as sh.)

---

Even better, don't shell out to do things you can do in Perl.

use Digest::SHA;

$hash = Digest::SHA::sha256('abcd');

---

For the curious, here's what the POSIX spec has to say about echo. I'm not sure what to make of XSI conformance; bash echo requires the -e option to treat the escape characters specially, but nearly every shell—except old versions of bash, and then only under special circumstances—treats -n as a flag, not a string. Oh well.

The following operands shall be supported:

string
A string to be written to standard output. If the first operand is -n, or
if any of the operands contain a <backslash> character, the results are
implementation-defined.

On XSI-conformant systems, if the first operand is -n, it shall be treated
as a string, not an option. The following character sequences shall be
recognized on XSI-conformant systems within any of the arguments:

\a
Write an <alert>.
\b
Write a <backspace>.
\c
Suppress the <newline> that otherwise follows the final argument in the output. All characters following the '\c' in the arguments shall be ignored.
\f
Write a <form-feed>.
\n
Write a <newline>.
\r
Write a <carriage-return>.
\t
Write a <tab>.
\v
Write a <vertical-tab>.
\\
Write a <backslash> character.
\0num
Write an 8-bit value that is the zero, one, two, or three-digit octal number num.

Answer 2

If you do:

printf "%s" 'abcd' | /usr/bin/shasum -a 256

you get the 88d...589 hash. If you do:

printf "%s\n" '-n abcd' | /usr/bin/shasum -a 256

you get the 617...403 hash.

Therefore, I deduce that Perl is somehow running a different echo command, perhaps /bin/echo or /usr/bin/echo instead of the bash built-in echo, or maybe the built-in echo to /bin/sh (which might perhaps be dash rather than bash), and this other echo does not recognize the -n option as an option and outputs different data.

I'm not sure which other echo it is finding; on my machine which is running an Ubuntu 14.04 LTE derivative, bash, dash, sh (linked to bash), ksh, csh and tcsh all treat echo -n abcd the same way. But somewhere along the line, I think that there is something along these lines happening; the hash checksums being identical strongly point to it. (Maybe you have a 3.2 bash linked to sh; see the notes in the comments.)