Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Fastest way to print a certain portion of a file using bash commands

Tags:

bash

shell

sed

currently I am using sed to print the required portion of the file. For example, I used the below command

sed -n 89001,89009p file.xyz

However, it is pretty slow as the file size is increasing (my file is currently 6.8 GB). I have tried to follow this link and used the command

sed -n '89001,89009{p;q}' file.xyz

But, this command is only printing the 89001th line. Kindly, help me.

like image 572
Sharma SRK Chaitanya Yamijala Avatar asked Aug 27 '16 03:08

Sharma SRK Chaitanya Yamijala


2 Answers

The syntax is a little bit different:

sed -n '89001,89009p;89009q' file.xyz

UPDATE:

Since there is also an answer with awk I made small comparison and as I thought - sed is a little bit faster:

$ wc -l large-file 
100000000 large-file
$ du -h large-file 
954M    large-file
$ time sed -n '890000,890010p;890010q' large-file > /dev/null

real    0m0.141s
user    0m0.068s
sys 0m0.000s
$ time awk 'NR>=890000{print} NR==890010{exit}' large-file > /dev/null

real    0m0.433s
user    0m0.208s
sys 0m0.008s`

UPDATE2:

There is a faster way with awk as posted by @EdMorton but still not as fast as sed:

$ time awk 'NR>=890000{print; if (NR==890010) exit}' large-file > /dev/null

real    0m0.252s
user    0m0.172s
sys     0m0.008s

UPDATE:

This is the fastest way I was able to find (head and tail):

$ time head -890010 large-file| tail -10 > /dev/null

real    0m0.085s
user    0m0.024s
sys     0m0.016s
like image 60
Dave Grabowski Avatar answered Sep 24 '22 09:09

Dave Grabowski


awk 'NR>=89001{print; if (NR==89009) exit}' file.xyz
like image 45
Ed Morton Avatar answered Sep 22 '22 09:09

Ed Morton