Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

extract some lines according to one field using bash scripting

Tags:

bash

Please I need your help to get the below result file from file1, the file1 is so huge, and I want according to the first column ( Field 1) to extract each part in a file. Thanks in advance.

File 1 :
1111 3 4 5 7
1111 2 4 6 8
1111 1 5 9 3
1112 4 6 8 9
1112 6 8 7 7
1113 6 6 6 6
1113 7 7 7 7
...

The result
we get all the files according to the first field, in the brief file the result should be 3 files (file_1111, file_1112 and file_1113) as below:
file_1111
1111 3 4 5 7
1111 2 4 6 8
1111 1 5 9 3

file_1112
1112 4 6 8 9
1112 6 8 7 7

file_1113
1113 6 6 6 6
1113 7 7 7 7

best regards,

H.R

like image 889
H.R Avatar asked Mar 25 '26 13:03

H.R


2 Answers

Give this tested version a try:

awk '{print>>"file_" $1;}' File_1

By default awk splits the line containing spaces in fields $1 $2 etc.

This one-liner command writes each line in the file associated to current line's first field: $1.

The test:

$ cat File_1
1111 3 4 5 7
1111 2 4 6 8
1111 1 5 9 3
1112 4 6 8 9
1112 6 8 7 7
1113 6 6 6 6
1113 7 7 7 7

$ awk '{print>>"file_" $1;}' File_1

$ ls
File_1  file_1111  file_1112  file_1113
$ cat file_1111
1111 3 4 5 7
1111 2 4 6 8
1111 1 5 9 3
$ cat file_1112
1112 4 6 8 9
1112 6 8 7 7
$ cat file_1113
1113 6 6 6 6
1113 7 7 7 7

Pure bash script would probably run many processes in a loop and I believe it should be slower than one awk command.

--edited-- It does not matter if File_1 is sorted or not. It will do the trick.

like image 130
Jay jargot Avatar answered Mar 28 '26 03:03

Jay jargot


If the data is sorted the way it is shown, shell can do it pretty well.

while read FileField DataFields || [ "$FileField" ] ;do
  echo "$FileField $DataFields" >> "file_$FileField"
done < "$yourInputFile"

If the data is not sorted as shown, the awk answer will likely be faster for large data sets.

... although The file field seems pretty pointless in the new files. You could probably get away with echo "$DataFields" >> "file_$FileField"

like image 44
technosaurus Avatar answered Mar 28 '26 02:03

technosaurus



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!