Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Identifying and removing null characters in UNIX

I have a text file containing unwanted null characters (ASCII NUL, \0). When I try to view it in vi I see ^@ symbols, interleaved in normal text. How can I:

  1. Identify which lines in the file contain null characters? I have tried grepping for \0 and \x0, but this did not work.

  2. Remove the null characters? Running strings on the file cleaned it up, but I'm just wondering if this is the best way?

like image 363
dogbane Avatar asked Mar 07 '10 23:03

dogbane


People also ask

How do I find a null character in Unix?

anywhere in the command line, actually. Actually, I believe it should be tr -d '\000' < file-with-nulls > file-without-nulls since < is part of the shell pipe functionality and not tr . Most shells will recognize & deal with < or > anywhere in the argument string, actually.

How do I remove a null character in Linux?

What does this command do? Using the -d switch we delete a character. A backslash followed by three 0's represents the null character. This just deletes these characters and writes the result to a new file.

How do you grep a null character?

On most modern Unixes, searching for null bytes is most easily done with GNU grep's ' grep -P ' option, which lets you supply a Perl-style regular expression that can include a direct byte value: grep -l -P '\x00' ....


2 Answers

I’d use tr:

tr < file-with-nulls -d '\000' > file-without-nulls 

If you are wondering if input redirection in the middle of the command arguments works, it does. Most shells will recognize and deal with I/O redirection (<, >, …) anywhere in the command line, actually.

like image 97
Pointy Avatar answered Nov 15 '22 19:11

Pointy


Use the following sed command for removing the null characters in a file.

sed -i 's/\x0//g' null.txt 

this solution edits the file in place, important if the file is still being used. passing -i'ext' creates a backup of the original file with 'ext' suffix added.

like image 27
rekha_sri Avatar answered Nov 15 '22 20:11

rekha_sri