Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parsing .csv file in bash, not reading final line

I'm trying to parse a csv file I made with Google Spreadsheet. It's very simple for testing purposes, and is basically:

1,2
3,4
5,6

The problem is that the csv doesn't end in a newline character so when I cat the file in BASH, I get

MacBook-Pro:Desktop kkSlider$ cat test.csv 
1,2 
3,4 
5,6MacBook-Pro:Desktop kkSlider$ 

I just want to read line by line in a BASH script using a while loop that every guide suggests, and my script looks like this:

while IFS=',' read -r last first
do
    echo "$last $first"
done < test.csv

The output is:

MacBook-Pro:Desktop kkSlider$ ./test.sh
1 2
3 4

Any ideas on how I could have it read that last line and echo it?

Thanks in advance.

like image 514
kkSlider Avatar asked Oct 26 '25 05:10

kkSlider


2 Answers

You can force the input to your loop to end with a newline thus:

#!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
    echo "$last $first"
done

Unfortunately, this may result in an empty line at the end of your output if the input already has a newline at the end. You can fix that with a little addition:

!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
    if [[ $last != "" ]] ; then
        echo "$last $first"
    fi
done

Another method relies on the fact that the values are being placed into the variables by the read but they're just not being output because of the while statement:

#!/bin/bash
while IFS=',' read -r last first
do
    echo "$last $first"
done <test.csv
if [[ $last != "" ]] ; then
    echo "$last $first"
fi

That one works without creating another subshell to modify the input to the while statement.


Of course, I'm assuming here that you want to do more inside the loop that just output the values with a space rather than a comma. If that's all you wanted to do, there are other tools better suited than a bash read loop, such as:

tr "," " " <test.csv
like image 200
paxdiablo Avatar answered Oct 28 '25 19:10

paxdiablo


cat file |sed -e '${/^$/!s/$/\n/;}'| while IFS=',' read -r last first; do echo "$last $first"; done
like image 24
Andrzej Chabierski Avatar answered Oct 28 '25 18:10

Andrzej Chabierski