Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Bash: read a file line-by-line and process each segment as parameters to other prog

I have some dirty work to do, so a Bash script seems to be a good choice. I'm new to Bash, and the experience makes me kind of frustrated.

The file mapfiles.txt consists of lines as follow. Each line has four segments separated by a white space. Each segment represents a input parameter to an external program name 'prog'. For example, "cm19_1.png" is the filename, "0001" the index, "121422481" the longitude, and "31035995" the latitude.

File: mapfiles.txt

cm19_1.png 0001 121422481 31035995 cm19_2.png 0002 121423224 31035995 cm19_3.png 0003 121423967 31035995 … 

I want to execute similar commands to each line. As show below, the prog's input parameter order is slightly different. So it makes sense to write a bash script to handle the repeated work.

[Usage] prog <index> <longitude> <latitude> <filename> example: prog 0001 121422481 31035995 cm19_1.png 

Generally, the bash script will operate in this way:

  1. Read one line from mapfiles.txt
  2. Split the segments
  3. Call the prog with a correct parameter order

Here comes run.sh.

#!/bin/sh  input=mapfiles.txt cmd=prog  while read line do         file=$(echo $line | cut -d' ' -f1)         key=$(echo $line | cut -d' ' -f2)         log=$(echo $line | cut -d' ' -f3)         lat=$(echo $line | cut -d' ' -f4)         echo $cmd $key $log $lat $file done < "$input" 

What I expected is

prog 0001 121422481 31035995 cm19_1.png prog 0002 121423224 31035995 cm19_2.png prog 0003 121423967 31035995 cm19_3.png …  

The ACTUAL result I got is

 cm19_1.png21422481 31035995  cm19_2.png21423224 31035995  cm19_3.png21423967 31035995 

Problems that confused me

  1. Where is 'prog'?
  2. Where is the white space?
  3. What's wrong with the parameter order?

Hmm… I wrote this script on my Mac using vim and copy it to a Scientific Linux box and a gentoo box. These three guys get the same ridiculous outputs.

like image 470
Jianwen W. Avatar asked Oct 01 '11 09:10

Jianwen W.


People also ask

How do I process a file line by line in bash?

In Bash, you can use a while loop on the command line to read each line of text from a file and do something with it. Our text file is called “data. txt.” It holds a list of the months of the year. The while loop reads a line from the file, and the execution flow of the little program passes to the body of the loop.

How read file line by line in shell script and store each line in a variable?

We use the read command with -r argument to read the contents without escaping the backslash character. We read the content of each line and store that in the variable line and inside the while loop we echo with a formatted -e argument to use special characters like \n and print the contents of the line variable.


1 Answers

You can simplify this a lot:

while read file key log lat do   echo "$cmd" "$key" "$log" "$lat" "$file" done < "$input" 
like image 98
Mat Avatar answered Sep 21 '22 15:09

Mat