Linux cat inline file with special characters

Question

I have the following script:

#!/bin/bash
cat << EOF
A^@B^AC^B
EOF

Where:

• '^@' = decimal value 0
• '^A' = decimal value 1
• '^B' = decimal value 2

So, the file looks like this:

xxd main.sh
00000000: 2321 2f62 696e 2f62 6173 680a 0a63 6174  #!/bin/bash..cat
00000010: 203c 3c20 454f 460a 4100 4201 4302 0a45   << EOF.A.B.C..E
00000020: 4f46 0a                                  OF.

When I run the script, decimal values 0 and 1 seem to disappear:

./main.sh | xxd
00000000: 4142 4302 0a                             ABC..

My questions is why? I was expecting to have an output like this:

Character 'A', followed by decimal value 0, followed by 'B', followed by decimal value 1, followed by 'C', followed by decimal value 2.

Answer 1

The reason seems two-fold.

First, note you must quote the word/delimiter in the here-document to avoid expansion(s):

If word is unquoted, all lines of the here-document are subjected to parameter expansion, command substitution, and arithmetic expansion, the character sequence \newline is ignored, and ‘\’ must be used to quote the characters ‘\’, ‘$’, and ‘`’.

So, if you write it like this:

#!/bin/bash
cat <<"EOF"
A^@B^AC^B
EOF

you'll preserve all characters except NULL (\0).

But you can not use NULL char in either variable, or an argument of the command line (see this answer).

You can use NULLs in pipes/files though, so you might want to encode the content of your here-document and decode it on fly (for example with xxd).