Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I pick random unique lines from a text file in shell?

Tags:

shell

sed

awk

I have a text file with an unknown number of lines. I need to grab some of those lines at random, but I don't want there to be any risk of repeats.

I tried this:

jot -r 3 1 `wc -l<input.txt` | while read n; do
  awk -v n=$n 'NR==n' input.txt
done

But this is ugly, and doesn't protect against repeats.

I also tried this:

awk -vmax=3 'rand() > 0.5 {print;count++} count>max {exit}' input.txt

But that obviously isn't the right approach either, as I'm not guaranteed even to get max lines.

I'm stuck. How do I do this?

like image 726
Graham Avatar asked Apr 12 '12 06:04

Graham


1 Answers

This might work for you:

shuf -n3 file

shuf is one of GNU coreutils.

like image 173
potong Avatar answered Oct 10 '22 21:10

potong