Select random 3000 lines from a file with awk codes

Question

I want to select randomly 3000 lines from a sample.file which contains 8000 lines. I will do that with awk codes or do from command line. How can I do that?

Answer 1

If you have gnu sort, it's easy:

sort -R FILE | head -n3000

If you have gnu shuf, it's even easier:

shuf -n3000 FILE

Answer 2

awk 'BEGIN{srand();}
{a[NR]=$0}
END{for(i=1; i<=3000; i++){x=int(rand()*NR) + 1; print a[x];}}' yourFile

Answer 3

Fixed as per Glenn's comment:

awk 'BEGIN {
  a=8000; l=3000
  srand(); nr[x]
  while (length(nr) <= l) 
    nr[int(rand() * a) +  1]    
  }
NR in nr
  ' infile 

P.S. Passing an array to the length built-in function is not portable, you've been warned :)

Answer 4

You can use a combination of awk, sort, head/tail and sed to do this, such as with:

pax$ seq 1 100 | awk '
...$    BEGIN {srand()}
...$          {print rand() " " $0}
...$ ' | sort | head -5 | sed 's/[^ ]* //'
57
25
80
51
72

which, as you can see, selects five random lines from the one hundred generated in seq 1 100.

The awk trick prefixes each and every line in the file with a random number and space of the format "0.237788 ", then sort (obviously) sorts it based on that random number.

Then you use head (or tail if you don't have a head) to get the first (or last) N lines.

Finally, the sed will strip off the random number and space and the start of each line.

For your specific case, you could use something like (on one line):

awk 'BEGIN {srand()} {print rand() " " $0}' file8000.txt
    | sort
    | tail -3000
    | sed 's/[^ ]* //'
    >file3000.txt