Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Powershell: search backwards from end of file

My script reads a log file once a minute and selects (and acts upon) the lines where the timestamp begins with the previous minute.

This is easy (the regex is simply "^$timestamp"), but when the log gets big it can take a while.

My thinking is the lines I want will always be near the bottom of the file, so I'd be searching far fewer lines if I started at the bottom and searched upwards, stopping when I get to the minute prior to the one I'm interested in.

My question is, how can I search from the bottom of the file instead of the top? Can I even say "read line $length", or even "read line n" (if so I could do a sort of binary search thing to find the length of the file and work backwards from there)?

Last question: would this even be faster (I'd still like to know how to do it even if it wouldn't be faster)?

Ideally, I'd like to do this all in my own code without installing anything extra.

Thanks

like image 277
mazz0 Avatar asked Apr 23 '12 16:04

mazz0


1 Answers

get-content bigfile.txt -tail 10

This words on huge files nearly instantly without any big memory usage.

I did it with a 22 GB text file in my testing.

Doing something like "get-context bigfile.txt | select -Last 10" works but it seems to have to load all of the lines (or objects in powershell) then does the select.

like image 198
ScottM Avatar answered Sep 27 '22 23:09

ScottM