Shell script to parse through a file ( csv ) and process line by line [duplicate]

Question

Hi Need a shell script to parse through the csv file - Line by line and then field by field ]

the file will look like this

X1,X2,X3,X4
Y1,Y2,Y3,Y4

I need to extract each of these X1,X2....

I wrote a script but it fails if the line exceeds one line..

Answer 1

Here's how I would do it.

First i set the IFS environment variable to tell read that "," is the field separator.

export IFS=","

Given the file "input" containing the data you provided, I can use the following code:

cat test | while read a b c d; do echo "$a:$b:$c:$d"; done

To quickly recap what is going on above. cat test | reads the file and pipes it to while. while runs the code between do and done while read returns true. read reads a line from standard input and separates it into variables ("a", "b", "c" and "d") according to the value of $IFS. Finally echo just displays the variables we read.

Which gives me the following output

X1:X2:X3:X4
Y1:Y2:Y3:Y4

BTW, the BASH manual is always good reading. You'll learn something new every time you read it.

Answer 2

Since eykanal mentioned AWk and and sed, I thought I'd show how you could use them.

awk -F, 'BEGIN{OFS="\n"}{$1=$1; print}' inputfile

or

sed 's/,/\n/' inputfile

Then a shell script could process their output:

awk_or_sed_cmd | while read -r field
do
    do_something "$field"
done

Of course, you could do the processing within the AWK script:

awk -F, '{for (i=1;i<=NF;i++) do_something($i)}' inputfile