Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get some specific lines from huge text file in unix?

Tags:

unix

I use importing systems based on delimited text files. The files used can sometimes be almost 2 Gb big and I have to check some lines from that file. So I want to know how can I output (on another file, or just on screen) the lines of specific value? E.g. line number 1010123, 1002451, 994123, etc., exactly as they are in the source file?

like image 981
BogdanM Avatar asked Feb 05 '13 14:02

BogdanM


People also ask

How do you get a specific line from a file Linux?

Using the head and tail Commands The idea is: First, we get line 1 to X using the head command: head -n X input. Then, we pipe the result from the first step to the tail command to get the last line: head -n X input | tail -1.

How do I search for a specific line in a file in Unix?

To do this, press Esc , type the line number, and then press Shift-g . If you press Esc and then Shift-g without specifying a line number, it will take you to the last line in the file. To look for the next occurrence after the first, either press n or press / again and then press Enter .

How do I search for a specific word in a large text file in Linux?

Grep is a Linux / Unix command-line tool used to search for a string of characters in a specified file. The text search pattern is called a regular expression. When it finds a match, it prints the line with the result. The grep command is handy when searching through large log files.


2 Answers

To print line N, use:

sed 'Nq;d' file

To print multiple lines (assuming they are in ascending order) e.g. 994123, 1002451, 1010123:

sed '994123p;1002451p;1010123q;d' file

The q after the last line number tells sed to quit when it reaches the 1010123th line, instead of wasting time by looping over the remaining lines that we are not interested in. That is why it is efficient on large files.

like image 200
dogbane Avatar answered Oct 13 '22 13:10

dogbane


You can do this with many Unix tools, for instance with awk:

# print first 5 lines with awk
awk 'NR>=1&&NR<=5{print}NR>=6{exit}' file

# print selection of lines 
awk 'NR==994123||NR==1002451||NR==1010123{print}NR>1010123{exit}' file
like image 39
Chris Seymour Avatar answered Oct 13 '22 14:10

Chris Seymour