Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parsing pipe delimited input in awk

Have seen many posts asking similar question. Can't get it working.

Input looks like:

<field one with spaces>|<field two with spaces>

Trying to parse with awk.

Have tried many variants from excellent posts:

FS = "^[\x00- ]*|[\x00- ]*[|][\x00- ]*|[\x00- ]*$";
FS = "^[\x00- ]*|[\x00- ]*\|[\x00- ]*|[\x00- ]*$";
FS = "^[\x00- ]*|[\x00- ]*\\|[\x00- ]*|[\x00- ]*$";

Still can't get the pipe delimiter to work.

Using CentOS.

Any help?

like image 379
scorpdaddy Avatar asked Aug 02 '11 19:08

scorpdaddy


People also ask

How do you use awk with a pipe?

The system function allows the user to execute operating system commands and then return to the awk program. The system function executes the command given by the string command. It returns, as its value, the status returned by the command that was executed. Show activity on this post.

What is the delimiter in awk?

The AWK Field Separator (FS) is used to specify and control how AWK splits a record into various fields. Also, it can accept a single character of a regular expression. Once you specify a regular expression as the value for the FS, AWK scans the input values for the sequence of characters set in the regular expression.

How do I split text in awk?

The awk function split(s,a,sep) splits a string s into an awk array a using the delimiter sep. Variable hms is an array so hms[2] is 34 . The last three statements are equivalent, but the last two more convenient for longer arrays. In the second you can specify the start index and number of elements to print.


1 Answers

 echo "field one has spaces | field two has spaces" \
 | awk '
   BEGIN {
      FS="|" 
 }
 {
   print $2
   print $1
   # or what ever you want
 }'

 #output

  field two has spaces
  field one has spaces

You can also reduce this to

awk -F'|' {
    print $2
    print $1
}'

Edit Also, not all awks can take a multi-character regex for the FS value.

Edit2 Somehow I missed this originally, but I see you are trying to include \x00 in the char classes pre and post of the | char. I assume you mean for \x00 == null char? I don't think you're going to be able to have awk parse a file with null chars embedded. You could prep-rocess your input like

 tr '\x00'   ' ' < file.txt > spacesForNulls.txt 

OR delete them altogether with

tr -d '\x00' < file.txt > deletedNulls.txt

and eliminate that part of your regex. But as above, some awk don't support regex for the FS value. And, I don't use the tr trick very much, you may find that it requires a slightly different notation for the null char, depending on your version of tr.

I hope this helps.

like image 156
shellter Avatar answered Sep 28 '22 01:09

shellter