Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to split a list by comma not space

I want to split a text with comma , not space in for foo in list. Suppose I have a CSV file CSV_File with following text inside it:

Hello,World,Questions,Answers,bash shell,script
...

I used following code to split it into several words:

for word in $(cat CSV_File | sed -n 1'p' | tr ',' '\n')
do echo $word
done

It prints:

Hello
World
Questions
Answers
bash
shell
script

But I want it to split the text by commas not spaces:

Hello
World
Questions
Answers
bash shell
script

How can I achieve this in bash?

like image 293
Eng.Fouad Avatar asked Oct 10 '11 20:10

Eng.Fouad


People also ask

How do you split a list with commas?

You can use the Python string split() function to split a string (by a delimiter) into a list of strings. To split a string by comma in Python, pass the comma character "," as a delimiter to the split() function. It returns a list of strings resulting from splitting the original string on the occurrences of "," .

How do you split a string by spaces and commas?

To split a string by space or comma, pass the following regular expression to the split() method - /[, ]+/ .

How do you read comma separated values in Linux?

You can use while shell loop to read comma-separated cvs file. IFS variable will set cvs separated to , (comma). The read command will read each line and store data into each field.


7 Answers

Using a subshell substitution to parse the words undoes all the work you are doing to put spaces together.

Try instead:

cat CSV_file | sed -n 1'p' | tr ',' '\n' | while read word; do
    echo $word
done

That also increases parallelism. Using a subshell as in your question forces the entire subshell process to finish before you can start iterating over the answers. Piping to a subshell (as in my answer) lets them work in parallel. This matters only if you have many lines in the file, of course.

like image 104
mkj Avatar answered Sep 30 '22 14:09

mkj


Set IFS to ,:

sorin@sorin:~$ IFS=',' ;for i in `echo "Hello,World,Questions,Answers,bash shell,script"`; do echo $i; done
Hello
World
Questions
Answers
bash shell
script
sorin@sorin:~$ 
like image 21
Sorin Avatar answered Sep 30 '22 16:09

Sorin


I think the canonical method is:

while IFS=, read field1 field2 field3 field4 field5 field6; do 
  do stuff
done < CSV.file

If you don't know or don't care about how many fields there are:

IFS=,
while read line; do
  # split into an array
  field=( $line )
  for word in "${field[@]}"; do echo "$word"; done

  # or use the positional parameters
  set -- $line
  for word in "$@"; do echo "$word"; done

done < CSV.file
like image 22
glenn jackman Avatar answered Sep 30 '22 16:09

glenn jackman


kent$  echo "Hello,World,Questions,Answers,bash shell,script"|awk -F, '{for (i=1;i<=NF;i++)print $i}'
Hello
World
Questions
Answers
bash shell
script
like image 33
Kent Avatar answered Sep 30 '22 14:09

Kent


Create a bash function

split_on_commas() {
  local IFS=,
  local WORD_LIST=($1)
  for word in "${WORD_LIST[@]}"; do
    echo "$word"
  done
}

split_on_commas "this,is a,list" | while read item; do
  # Custom logic goes here
  echo Item: ${item}
done

... this generates the following output:

Item: this
Item: is a
Item: list

(Note, this answer has been updated according to some feedback)

like image 26
Andrew Newdigate Avatar answered Sep 30 '22 16:09

Andrew Newdigate


Read: http://linuxmanpages.com/man1/sh.1.php & http://www.gnu.org/s/hello/manual/autoconf/Special-Shell-Variables.html

IFS The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is ``''.

IFS is a shell environment variable so it will remain unchanged within the context of your Shell script but not otherwise, unless you EXPORT it. ALSO BE AWARE, that IFS will not likely be inherited from your Environment at all: see this gnu post for the reasons and more info on IFS.

You're code written like this:

IFS=","
for word in $(cat tmptest | sed -n 1'p' | tr ',' '\n'); do echo $word; done;

should work, I tested it on command line.

sh-3.2#IFS=","
sh-3.2#for word in $(cat tmptest | sed -n 1'p' | tr ',' '\n'); do echo $word; done;
World
Questions
Answers
bash shell
script
like image 24
Ashley Raiteri Avatar answered Sep 30 '22 15:09

Ashley Raiteri


You can use:

cat f.csv | sed 's/,/ /g' |  awk '{print $1 " / " $4}'

or

echo "Hello,World,Questions,Answers,bash shell,script" | sed 's/,/ /g' |  awk '{print $1 " / " $4}'

This is the part that replace comma with space

sed 's/,/ /g'
like image 30
ozma Avatar answered Sep 30 '22 16:09

ozma