Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

select -first 1 on a large file

Tags:

powershell

I am trying to run the following command on a very large text file. However, it's very slow

((cat largefile.txt | select -first 1).split(",")).count()

Is an alternative fast way in powershell? It seems the command will scan the whole file no matter what.

like image 379
ca9163d9 Avatar asked Jul 06 '12 21:07

ca9163d9


2 Answers

To only get the first x number of lines in a text file, use the –totalcount parameter:

((Get-Content largefile.txt -totalcount 1).split(",")).count
like image 86
jon Z Avatar answered Sep 29 '22 06:09

jon Z


It's worse than that - it will load the whole file and turn it into a string array.

Use the native .NET libraries to load just the first line:

$reader = [System.IO.File]::OpenText("largefile.txt")
$line = $reader.ReadLine()
$reader.Close()

(borrowed from How to process a file in Powershell line-by-line as a stream)

like image 25
Arithmomaniac Avatar answered Sep 29 '22 05:09

Arithmomaniac